sensu-client check-memory sample work working - sensu

I am trying to get sensu working.
The following is the sensu-client.log
ubuntu#ip:~$ sudo tail -f /var/log/sensu/sensu-client.log
{"timestamp":"2016-09-27T16:07:37.628182-0400","level":"info","message":"completing checks in progress","checks_in_progress":[]}
{"timestamp":"2016-09-27T16:07:38.128912-0400","level":"info","message":"closing client tcp and udp sockets"}
{"timestamp":"2016-09-27T16:07:38.129275-0400","level":"warn","message":"stopping reactor"}
{"timestamp":"2016-09-27T16:07:39.224377-0400","level":"warn","message":"loading config file","file":"/etc/sensu/config.json"}
{"timestamp":"2016-09-27T16:07:39.224487-0400","level":"warn","message":"loading config files from directory","directory":"/etc/sensu/conf.d"}
{"timestamp":"2016-09-27T16:07:39.224528-0400","level":"warn","message":"loading config file","file":"/etc/sensu/conf.d/check_mem.json"}
{"timestamp":"2016-09-27T16:07:39.224573-0400","level":"warn","message":"config file applied changes","file":"/etc/sensu/conf.d/check_mem.json","changes":{}}
{"timestamp":"2016-09-27T16:07:39.224618-0400","level":"warn","message":"applied sensu client overrides","client":{"name":"localhost","address":"127.0.0.1","subscriptions":["test","client:localhost"]}}
{"timestamp":"2016-09-27T16:07:39.230963-0400","level":"warn","message":"loading extension files from directory","directory":"/etc/sensu/extensions"}
{"timestamp":"2016-09-27T16:07:39.231048-0400","level":"info","message":"configuring sensu spawn","settings":{"limit":12}}
/etc/sensu/client.json contains
{
"rabbitmq": {
"host": "ipaddressofsensuserver",
"port": 5672,
"user": "username",
"password": "password",
"vhost": "/sensu"
},
"api": {
"host": "localhost",
"port": 4567
},
"checks": {
"test": {
"command": "echo -n OK",
"subscribers": [
"test"
],
"interval": 60
},
"memory-percentage": {
"command": "check-memory-percent.sh -w 50 -c 70",
"interval": 10,
"subscribers": [
"test"
]
}
},
"client": {
"name": "localhost",
"address": "127.0.0.1",
"subscriptions": [
"test"
]
}
}
I have copied check-memory-present.sh into /etc/sensu/conf.d folder
I was expecting the log file to run check-memory-percent every 10 seconds. What am I missing here ?

The Sensu client cannot operate entirely independent of the server, but it can schedule its own checks to run and have them be sent to the server through the transport (RabbitMQ in this case). You'll have to add "standalone": true to the check configuration in order to have this take effect, and then restart the sensu-client service.
So, the file /etc/sensu/conf.d/check_mem.json should look something like:
"checks": {
"memory-percentage": {
"command": "/etc/sensu/conf.d/check-memory-percent.sh -w 50 -c 70",
"interval": 10,
"standalone": true
}
}
Remember to remove the block from /etc/sensu/client.json as well, as you may get unexpected results if you have the same check name defined multiple times.

In Client.json, under "client", you need to add the subscriptions. Like in the example here. It should match the definition of "subscribers" for your check.

Related

debug with VScode a fastAPI project built with docker-compose with a postgresql database

I would like to be able to make my code stop on breakpoints with VScode. My project is built with docker-compose and works without debugging on port 8000.
Here are my configurations files:
docker-compose:
version: '3.4'
services:
murmurside:
image: murmurside
build: ./murmur_side
command: ["sh", "-c", "pip install debugpy -t /tmp && python /tmp/debugpy --wait-for-client --listen 0.0.0.0:5678 -m uvicorn app.main:app --host 0.0.0.0 --port 8000"]
volumes:
- ./murmur_side/:/murmur_side/
ports:
- 8000:8000
- 5678:5678
environment:
- DATABASE_URL=postgresql://USERNAME:PASSWORD#db/fastapi_db_2
db:
image: postgres:13-alpine
volumes:
- postgres_data2:/var/lib/postgresql/data/
expose:
- 5432
environment:
- POSTGRES_USER=USERNAME
- POSTGRES_PASSWORD=PASSWORD
- POSTGRES_DB=fastapi_db_2
volumes:
postgres_data2:
dockerfile :
# For more information, please refer to https://aka.ms/vscode-docker-python
FROM python:3.10-slim
EXPOSE 8000
WORKDIR /murmur_side
# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE=1
# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED=1
# Install pip requirements
COPY requirements.txt .
RUN python -m pip install -r requirements.txt
COPY . /murmur_side
# Creates a non-root user with an explicit UID and adds permission to access the /app folder
# For more info, please refer to https://aka.ms/vscode-docker-python-configure-containers
RUN adduser -u 5678 --disabled-password --gecos "" appuser && chown -R appuser /murmur_side
USER appuser
# During debugging, this entry point will be overridden. For more information, please refer to https://aka.ms/vscode-docker-python-debug
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "-k", "uvicorn.workers.UvicornWorker", "app.main:app"]
launch.json :
I tested a 'launch' configuration but the debugger then bumps on the database related code. It does not seem to properly link to the database : after DATABASE_URL = os.getenv("DATABASE_URL") DATABASE_URL stays empty.
{
"configurations": [
{
"name": "Docker: Python - Fastapi",
"type": "docker",
"request": "launch",
"preLaunchTask": "docker-run: debug",
"python": {
"pathMappings": [
{
"localRoot": "${workspaceFolder}",
"remoteRoot": "/app"
}
],
"projectType": "fastapi"
}
}
]
}
I also tested an 'attach' configuration. In that case, I get a debugger container launched at a random port but I get nothing when I browse to 127.0.0.1:randomPort
{
"configurations": [
{
"name": "Python: Remote Attach",
"type": "python",
"request": "attach",
"connect": {
"host": "0.0.0.0",
"port": 8000 # I also tried with 5678
},
"preLaunchTask": "docker-run: debug",
"pathMappings": [
{
"localRoot": "${workspaceFolder}",
"remoteRoot": "/murmur_side"
}
]
}
]
}
On this project I see that they added database credentials in the tasks.json. But I did not find any documentation stating that elsewhere. I tried that but I am unsure for the options other than username, password and dbname since I did not have to mention them in the docker-compose. Maybe I am also mistaking on the ports since there are several ones.
tasks.json:
{
"version": "2.0.0",
"tasks": [
{
"type": "docker-build",
"label": "docker-build",
"platform": "python",
"dockerBuild": {
"tag": "sd4timapi_pip_debug:latest",
"dockerfile": "${workspaceFolder}/murmur_side/Dockerfile",
"context": "${workspaceFolder}/murmur_side/",
"pull": true
}
},
{
"type": "docker-run",
"label": "docker-run: debug",
"dependsOn": [
"docker-build"
],
"dockerRun": { # I also tried without this section
"image": "sd4timapi_pip_debug:latest",
"volumes": [
{
"containerPath": "/murmur_side/",
"localPath": "${workspaceFolder}/murmur_side/"
}
],
"ports": [
{
"containerPort": 8000,
"hostPort": 8001, # because it doesn't allow me to put 8000 : "port is already allocated"
"protocol": "tcp"
}
],
"env": {
"APP_PORT": "8000", #UNSURE
"DEBUG": "TRUE",
"ENVIRONMENT": "local", #UNSURE
"POSTGRES_USER": "USERNAME",
"POSTGRES_PASS": "PASSWORD",
"POSTGRES_DBNAME": "fastapi_db_2",
"POSTGRES_HOST": "db_host", #UNSURE
"POSTGRES_PORT": "5432", #UNSURE
"POSTGRES_APPLICATION_NAME": "sd4timapi_pip_debug", #UNSURE
}
},
"python": {
"args": [
"app.main:app",
"--host",
"0.0.0.0",
"--port",
"8000"
],
"module": "uvicorn"
}
}
]
}
This works for me, for debugging my FastAPI app in VS Code.
Here's the ".vscode/launch.json" file:
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
// Debug with FastAPI
{
"name": "Python: FastAPI",
"type": "python",
"request": "launch",
"module": "uvicorn",
"args": [
"app.main:app",
"--host",
"0.0.0.0",
"--port",
"81",
"--reload"
],
"jinja": true,
"justMyCode": false
},
]
}
I get the following messages in the VS Code terminal logs, and then I can debug with breakpoints, etc.
Go to http://localhost:81/docs in your browser since I deployed to port 81.
INFO: Will watch for changes in these directories: ['/workspace']
INFO: Uvicorn running on http://0.0.0.0:81 (Press CTRL+C to quit)
INFO: Started reloader process [13707] using WatchFiles
INFO: Started server process [13801]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: 127.0.0.1:36070 - "GET /docs HTTP/1.1" 200 OK

Adding a custom tag based on topicName (wildcard) via using JmxTrans to send Kafka JMX to influxDb

Basically what i wanted to achieve was to get MessageInPerSec metric for all the topic in kafka and to add the custom tag as topicName in the influx db so as to query based on the topic not based on the 'ObjDomain' definition, below are my JmxTrans configuration, (Note using the wildcard for the topic as to fetch the data MessageInPerSec JMX attribute for all the topic)
{
"servers": [
{
"port": "9581",
"host": "192.168.43.78",
"alias": "kafka-metric",
"queries": [
{
"outputWriters": [
{
"#class": "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory",
"url": "http://192.168.43.78:8086/",
"database": "kafka",
"username": "admin",
"password": "root"
}
],
"obj": "kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec,topic=*",
"attr": [
"Count",
"MeanRate",
"OneMinuteRate",
"FiveMinuteRate",
"FifteenMinuteRate"
],
"resultAlias": "newTopic"
}
],
"numQueryThreads": 2
}
]
}
which yields a result in the Influx DB as follow
[name=newTopic, time=1589425526087, tags={attributeName=FifteenMinuteRate,
className=com.yammer.metrics.reporting.JmxReporter$Meter, objDomain=kafka.server,
typeName=type=BrokerTopicMetrics,name=MessagesInPerSec,topic=backblaze_smart},
precision=MILLISECONDS, fields={FifteenMinuteRate=1362.9446063537794, _jmx_port=9581
}]
and create tag with whole objDomain spefcified in the config, but i wanted to have topic as a seperate tag that is something as follow
[name=newTopic, time=1589425526087, tags={attributeName=FifteenMinuteRate,
className=com.yammer.metrics.reporting.JmxReporter$Meter, objDomain=kafka.server,
topic=backblaze_smart,
typeName=type=BrokerTopicMetrics,name=MessagesInPerSec,topic=backblaze_smart},
precision=MILLISECONDS, fields={FifteenMinuteRate=1362.9446063537794, _jmx_port=9581
}]
was not able to find any adequate documentation for the same on how to use the wildcard value of topic as a separate tag using jmxtrans and writing it to the InfluxDB.
You just need to add the following additional properties for Influx output writer. Just make sure you are using the latest version of jmxtrans release. The docs are here: https://github.com/jmxtrans/jmxtrans/wiki/InfluxDBWriter
"typeNames": ["topic"],
"typeNamesAsTags": "true"
I have listed your config with the above modifications.
{
"servers": [
{
"port": "9581",
"host": "192.168.43.78",
"alias": "kafka-metric",
"queries": [
{
"outputWriters": [
{
"#class": "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory",
"url": "http://192.168.43.78:8086/",
"database": "kafka",
"username": "admin",
"password": "root",
"typeNames": ["topic"],
"typeNamesAsTags": "true"
}
],
"obj": "kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec,topic=*",
"attr": [
"Count",
"MeanRate",
"OneMinuteRate",
"FiveMinuteRate",
"FifteenMinuteRate"
],
"resultAlias": "newTopic"
}
],
"numQueryThreads": 2
}
]
}

How could a spring-boot application determine if it is running on cloud foundry?

I'm writting a micro service with spring-boot. The db is mongodb. The service works perfectly in my local environment. But after I deployed it to the cloud foundry it doesn't work. The reason is connecting mongodb time out.
I think the root cause is the application doesn't know it is running on cloud. Because it still connecting 127.0.0.1:27017, but not the redirected port.
How could it know it is running on cloud? Thank you!
EDIT:
There is a mongodb instance bound to the service. And when I checked the environment information, I got following info:
{
"VCAP_SERVICES": {
"mongodb": [
{
"credentials": {
"hostname": "10.11.241.1",
"ports": {
"27017/tcp": "43417",
"28017/tcp": "43135"
},
"port": "43417",
"username": "xxxxxxxxxx",
"password": "xxxxxxxxxx",
"dbname": "gwkp7glhw9tq9cwp",
"uri": "xxxxxxxxxx"
},
"syslog_drain_url": null,
"volume_mounts": [],
"label": "mongodb",
"provider": null,
"plan": "v3.0-container",
"name": "mongodb-business-configuration",
"tags": [
"mongodb",
"document"
]
}
]
}
}
{
"VCAP_APPLICATION": {
"cf_api": "xxxxxxxxxx",
"limits": {
"fds": 16384,
"mem": 1024,
"disk": 1024
},
"application_name": "mock-service",
"application_uris": [
"xxxxxxxxxx"
],
"name": "mock-service",
"space_name": "xxxxxxxxxx",
"space_id": "xxxxxxxxxx",
"uris": [
"xxxxxxxxxx"
],
"users": null,
"application_id": "xxxxxxxxxx",
"version": "c7569d23-f3ee-49d0-9875-8e595ee76522",
"application_version": "c7569d23-f3ee-49d0-9875-8e595ee76522"
}
}
From my understanding, I think my spring-boot service should try to connect the port 43417 but not 27017, right? Thank you!
Finally I found the reason is I didn't specify the profile. After adding following code in my manifest.yml it works:
env:
SPRING_PROFILES_ACTIVE: cloud

Not able to set up my loopback model.Error:Persisted model has not been correctly attached to a DataSource

restraunt.json file
`{
"name": "restraunt",
"base": "PersistedModel",
"idInjection": true,
"options": {
"validateUpsert": true
},
"properties": {
"name": {
"type": "string",
"required": true
},
"location": {
"type": "string",
"required": true
}
},
"validations": [],
"relations": {},
"acls": [],
"methods": {}
}`
restraunt.js file
`module.exports = function(Restraunt) {
Restraunt.find({where:{id:1}}, function(data) {
console.log(data);
})
};`
model-config.json file
`"restraunt": {
"dataSource": "restrauntManagement"
}`
datasources.json file
`{
"db": {
"name": "db",
"connector": "memory"
},
"restrauntManagement": {
"host": "localhost",
"port": 0,
"url": "",
"database": "restraunt-management",
"password": "restraunt-management",
"name": "restrauntManagement",
"user": "rohit",
"connector": "mysql"
}
}`
I am able to get,put,post from the explorer which means the sql db has been set up properly but i am not able to 'find' from restraunt.js file.It throws an error.
"Error: Cannot call restraunt.find(). The find method has not been setup. The PersistedModel has not been correctly attached to a DataSource"
Besides that executing code in boot folder, there's a possibility to use event, emitted after attaching the model.
You can write your code right in model.js, not in boot folder.
Looks like:
Model.once("attached", function () {})
Model = Accounts (for example).
I know, this is an old topic, but maybe this helps someone else.
Try installing mysql connector again:
npm i -S loopback-connector-mysql
Take a look at your datasources.json, because mysql's port might be wrong, default port is 3306, also you could try changing localhost to 0.0.0.0.
"restrauntManagement": {
"host": "localhost", /* if you're using docker, you need to set it to 0.0.0.0 instead of localhost */
"port": 0, /* default port is 3306 */
"url": "",
"database": "restraunt-management",
"password": "restraunt-management",
"name": "restrauntManagement",
"user": "rohit",
"connector": "mysql"
}
model-config.json must be:
"restraunt": {
"dataSource": "restrauntManagement" /* this name must be the same name in datasources object key (in your case it is restrauntManagement not the connector name which is mysql) */
}
You also need to execute the migration for restaurant model:
create migration.js at /server/boot and add this:
'use strict';
module.exports = function(server) {
var mysql = server.dataSources.mysql;
mysql.autoupdate('restraunt');
};
you need to migrate every single model you'll use it. you also need to migrate the default models (ACL, AccessToken, etc...) if you're going to attach them to a datasource.
Also in the docs says you can't perform any operation inside the model.js file because the system (at that point) it is not fully loaded. Any operation you need to execute must be inside a .js file in the /boot directory because the system is completely loaded there. You can perform operations inside remote methods because the system is loaded as well.

chef-solo not updating postgres pg_hba.conf

I am using Chef Solo to provision a Vagrant Virtual Machine. Here is the relevant Vagrantfile snippet:
chef.run_list = [
"databox::default",
"mydbstuff"
]
chef.json = {
"postgresql": {
"config" : {
"listen_addresses": "*"
},
"pg_hba": [
{"type": "local", "db": "all", "user": "postgres", "addr": null, "method": "ident"},
{"type": "local", "db": "all", "user": "all", "addr": null, "method": "md5"},
{"type": "host", "db": "all", "user": "all", "addr": "127.0.0.1/32", "method": "md5"},
{"type": "host", "db": "all", "user": "all", "addr": "::1/128", "method": "md5"},
{"type": "local", "db": "all", "user": "vagrant", "addr": null, "method": "ident"},
{"type": "host", "db": "all", "user": "all", "addr": "192.168.248.1/24", "method": "md5"}
]
},
"databox": {
"db_root_password": "abc123",
"databases": {
"postgresql": [
{ "username": "db1", "password": "abc123", "database_name": "db1" },
{ "username": "db2", "password": "abc123", "database_name": "db2" }
]
}
}
}
The mydbstuff::default recipe looks like this:
postgresql_connection_info = {
:host => "localhost",
:port => node['postgresql']['config']['port'],
:username => 'postgres',
:password => node['postgresql']['password']['postgres']
}
postgresql_database_user 'vagrant' do
connection postgresql_connection_info
password 'vagrant'
action :create
end
node['databox']['databases']['postgresql'].each do |db|
postgresql_database_user 'vagrant' do
connection postgresql_connection_info
action :grant
database_name db.database_name
end
end
I am trying to allow connections by the local vagrant user without a password, and by any user from the VirtualBox private network. The pg_hba array in my chef.json has four lines that are copied from the default configuration and two lines to do the other stuff that I want to do. If I add these two lines to the pg_hba.conf file manually, they work just fine.
The problem is that my changes aren't actually written to the pg_hba.conf file. What's preventing them from being written?
It appears that the databox cookbook overwrites the Postgres permissions array using node.set instead of just modifying the part that it needs.
I have submitted a pull request to the project to change this behavior so that additional entries can be added to the file.
I faced same problem with chef-solo. My way out was to create a template for pg_hba.conf and replaced at the end of execution of recipe.