Orion version is 2.1.0
Orion is started in HTTPS using the -https option
We use the "HTTPS" protocol schema in URL in our subscriptions --> reference" : "https://cygnus.domain.com/notify"
When we insert an Entity matching the subscription, the Entity is created in Orion, but not in STH.
However Orion Logs return: Notification Successfully Sent to https://cygnus.domain.com:443/notify
If we use the "HTTP" protocol schema in URL in our subscriptions it works
If we use curl to notify dirctly Cygnus in HTTP or HTTPS it works
Orion Logs bellow:
time=Friday 22 Feb 11:24:28 2019.158Z | lvl=INFO | corr=N/A | trans=1550831768-689-00000000058 | from=pending | srv=pending | subsrv=pending | comp=Orion | op=logMsg.h[1832]:lmTransactionStart | msg=Starting transaction to https://cygnus.domain.com:443/notify
time=Friday 22 Feb 11:24:28 2019.159Z | lvl=INFO | corr=6a8319ac-3694-11e9-872e-0242c0a81006 | trans=1550831768-689-00000000056 | from=10.6.11.36 | srv=svctestnca | subsrv=/svcpath/testnca | comp=Orion | op=logMsg.h[1916]:lmTransactionEnd | msg=Transaction ended
time=Friday 22 Feb 11:24:28 2019.177Z | lvl=INFO | corr=N/A | trans=1550831768-689-00000000057 | from=pending | srv=pending | subsrv=pending | comp=Orion | op=httpRequestSend.cpp[615]:httpRequestSendWithCurl | msg=Notification Successfully Sent to https://cygnus.domain.com:443/notify
time=Friday 22 Feb 11:24:28 2019.159Z | lvl=INFO | corr=N/A | trans=1550831768-689-00000000058 | from=pending | srv=pending | subsrv=pending | comp=Orion | op=httpRequestSend.cpp[594]:httpRequestSendWithCurl | msg=Sending message 20 to HTTP server: sending message of 826 bytes to HTTP server
time=Friday 22 Feb 11:24:28 2019.176Z | lvl=INFO | corr=N/A | trans=1550831768-689-00000000058 | from=pending | srv=pending | subsrv=pending | comp=Orion | op=logMsg.h[1916]:lmTransactionEnd | msg=Transaction ended
Thanks for your help.
This problem has been solved
The subscription reference is https://cygnus.domain.com/notify
but Orion transform this in https://cygnus.domain.com:443/notify
We have a HAProxy to load balance requests. An ACL is present to accept the doamin cygnus.domain.com but no ACL is present to accept cygnus.domain.com:443
Modifying the ACL resolve the problem
Old ACL : acl IS_Cygnus hdr(host) -i cygnus.domain.com
New ACL : acl IS_Cygnus hdr_beg(host) -i cygnus.domain.com
Related
Hello I'm trying to launch the FIWARE orion context broker using an atlas mongodb
mongodb+srv://<user>:****************#<domainid>.mongodb.net/<aut_db>
I have tried multiple variations of the docker run command but I never get the connection to success
trace:
$ docker run --name orion -p 1027:1026 fiware/orion:latest -dbhost "<domainid>.mongodb.net/<aut_db>?ssl=true&retryWrites=true&w=majority" -dbuser <user> -dbpwd U9NjLafksdv9mavW -logLevel DEBUG
time=2021-12-01T14:38:42.188Z | lvl=INFO | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=contextBroker.cpp[1063]:main | msg=start command line </usr/bin/contextBroker -fg -multiservice -ngsiv1Autocast -disableFileLog -dbhost <domainid>.mongodb.net/<aut_db>?ssl=true&retryWrites=true&w=majority -dbuser <user> -dbpwd ****** -logLevel DEBUG>
time=2021-12-01T14:38:42.188Z | lvl=INFO | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=contextBroker.cpp[1137]:main | msg=Orion Context Broker is running
time=2021-12-01T14:40:23.011Z | lvl=ERROR | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=mongoConnectionPool.cpp[220]:mongoConnect | msg=Database Startup Error (cannot connect to mongo - doing 100 retries with a 1000 millisecond interval)
time=2021-12-01T14:40:23.011Z | lvl=FATAL | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=MongoGlobal.cpp[142]:mongoInit | msg=Fatal Error (MongoDB error)
time=2021-12-01T14:40:23.012Z | lvl=INFO | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=contextBroker.cpp[591]:exitFunc | msg=Orion shutdown completed
Has someone know how I can configure orion context broker to connect to an mongo atlas?
EDIT: I have checked and I have access from that server to the mongo-db with the mongo shell
Orion currently supports only mongodb:// connection strings, built from the -db, -dbhost, -rplSet, -dbTimeout, -dbuser, -dbpwd, -dbAuthMech, -dbAuthDb, -dbSSL and -dbDisableRetryWrites CLI parameters(or default values, if omitted) as can be see in the source code.
There is an issue in the Orion repository with the aim of making this more flexible, allowing to provide the connection string directly overriding the above parameters.
While that issue comes (contributions welcome ;) you could hack the Orion source code cited above to build an specific version for you with support for mongo+srv:// connection strings.
When I create a new mysql db, slashdb's test connection fails.
Here is how I log into mysql:
$ mysql -u 7stud -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 15
Server version: 5.5.5-10.4.13-MariaDB Homebrew
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| chat |
| ectoing_repo |
| ejabberd |
| information_schema |
| mydb |
| mysql |
| performance_schema |
| test |
+--------------------+
8 rows in set (0.00 sec)
mysql> use mydb;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+----------------+
| Tables_in_mydb |
+----------------+
| cheetos |
| greetings |
| mody |
| people |
+----------------+
4 rows in set (0.00 sec)
mysql> select * from people;
+----+--------+------+
| id | name | info |
+----+--------+------+
| 1 | 7stud | abc |
| 2 | Beth | xxx |
| 3 | Diane | xyz |
| 4 | Kathy | xyz |
| 5 | Kathy | xyz |
| 6 | Dave | efg |
| 7 | Tom | zzz |
| 8 | David | abc |
| 9 | Eloise | abc |
| 10 | Jess | xyz |
| 11 | Jeffsy | 2.0 |
| 12 | XXX | xxx |
| 13 | XXX | xxx |
+----+--------+------+
13 rows in set (0.00 sec)
In the slashdb form for creating a new database, here is the info I entered:
Hostname: 127.0.0.1
Port: 80
Database Login: 7stud
Database Password: **
Database Name: mydb
Then I hit the "Test Connection" button, whereupon I get a spinning wheel, which disappears after a few minutes, but no "Connection Successful" message. What am I doing wrong?
Now, I'm using port 3306:
mysql> SHOW GLOBAL VARIABLES LIKE 'PORT';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| port | 3306 |
+---------------+-------+
1 row in set (0.00 sec)
but when slashdb tries to connect, I get the error:
Host localhost:3306 is not accessible
Your port is wrong in the database connection. You said that your MySQL is configured on port 3306, but you also posted your SlashDB config for database with port 80. Please change that to 3306.
Also, not sure how if you don't need to enable remote access to MySQL. Even if your SlashDB is running on the same machine as the MySQL database, it uses TCP/IP to connect.
Here is the stdout
[lzhou#localhost script]$ perl dbauthtest_server.pl You are loading Catalyst::Engine::HTTP explicitly.
This is almost certainly a bad idea, as Catalyst::Engine::HTTP has been removed in this version of Catalyst.
Please update your application's scripts with:
catalyst.pl -force -scripts MyApp
to update your scripts to not do this. [debug] Debug messages enabled [debug] Statistics enabled [warn] You are running an old script!
Please update by running (this will overwrite existing files):
catalyst.pl -force -scripts DBAuthTest
or (this will not overwrite existing files):
catalyst.pl -scripts DBAuthTest
[debug] Loaded Config "/home/lzhou/ä¸è¼ /def-guide-to-catalyst-master/4439/catalyst-book-code/Chapter_6/DBAuthTest/dbauthtest.conf" [debug] Loaded plugins: .--------------------------------------------------------------------------------. | Catalyst::Plugin::ConfigLoader 0.34 | '--------------------------------------------------------------------------------'
[debug] Loaded PSGI Middleware: .--------------------------------------------------------------------------------. | Catalyst::Middleware::Stash | | Plack::Middleware::HTTPExceptions | | Plack::Middleware::RemoveRedundantBody 0.06 | | Plack::Middleware::FixMissingBodyInRedirect 0.12 | | Plack::Middleware::ContentLength | | Plack::Middleware::MethodOverride 0.20 | | Plack::Middleware::Head | '--------------------------------------------------------------------------------'
[debug] Loaded Request Data Handlers: .--------------------------------------------------------------------------------. | application/json | | application/x-www-form-urlencoded | '--------------------------------------------------------------------------------'
[debug] Loaded dispatcher "Catalyst::Dispatcher" [debug] Loaded engine "Catalyst::Engine" [debug] Found home "/home/lzhou/ä¸è¼ /def-guide-to-catalyst-master/4439/catalyst-book-code/Chapter_6/DBAuthTest" [debug] Loaded components: .---------------------------------------------------------------------+----------. | Class | Type |
+---------------------------------------------------------------------+----------+ | DBAuthTest::Controller::AuthUsers | instance | | DBAuthTest::Controller::Root | instance | | DBAuthTest::Model::AuthDB | instance | | DBAuthTest::Model::AuthDB::Roles | class | | DBAuthTest::Model::AuthDB::UserRoles | class | | DBAuthTest::Model::AuthDB::Users | class | | DBAuthTest::View::Web | instance | '---------------------------------------------------------------------+----------'
[debug] Loaded Private actions: .----------------------+--------------------------------------+------------------. | Private | Class | Method |
+----------------------+--------------------------------------+------------------+ | /end | DBAuthTest::Controller::Root | end | | /default | DBAuthTest::Controller::Root | default | | /authusers/profile | DBAuthTest::Controller::AuthUsers | profile | | /authusers/base | DBAuthTest::Controller::AuthUsers | base | | /authusers/add | DBAuthTest::Controller::AuthUsers | add | | /authusers/user | DBAuthTest::Controller::AuthUsers | user | '----------------------+--------------------------------------+------------------'
[debug] Loaded Path actions: .---------------------------------------+----------------------------------------. | Path | Private |
+---------------------------------------+----------------------------------------+ | /... | /default | '---------------------------------------+----------------------------------------'
[debug] Loaded Chained actions: .---------------------------------------+----------------------------------------. | Path Spec | Private |
+---------------------------------------+----------------------------------------+ | /authusers/add | /authusers/base (0) | | | => /authusers/add (0) | | /authusers/*/profile | /authusers/base (0) | | | -> /authusers/user (1) | | | => /authusers/profile (0) | '---------------------------------------+----------------------------------------'
[info] DBAuthTest powered by Catalyst 5.90124
But when i visit http://localhost:3000/ , connection is refused . it appears that there is no server running.
I'm using osquery to monitor servers on my network. The following osquery.conf captures snapshots, every minute, of the processes communicating over the network ports and publishes that data to Kafka:
{
"options": {
"logger_kafka_brokers": "cp01.woolford.io:9092,cp02.woolford.io:9092,cp03.woolford.io:9092",
"logger_kafka_topic": "base_topic",
"logger_kafka_acks": "1"
},
"packs": {
"system-snapshot": {
"queries": {
"processes_by_port": {
"query": "select u.username, p.pid, p.name, pos.local_address, pos.local_port, pos.remote_address, pos.remote_port from processes p join users u on u.uid = p.uid join process_open_sockets pos on pos.pid=p.pid where pos.remote_port != '0'",
"interval": 60,
"snapshot": true
}
}
}
},
"kafka_topics": {
"process-port": [
"pack_system-snapshot_processes_by_port"
]
}
}
Here's an example of the output from the query:
osquery> select u.username, p.pid, p.name, pos.local_address, pos.local_port, pos.remote_address, pos.remote_port from processes p join users u on u.uid = p.uid join process_open_sockets pos on pos.pid=p.pid where pos.remote_port != '0';
+--------------------+-------+---------------+------------------+------------+------------------+-------------+
| username | pid | name | local_address | local_port | remote_address | remote_port |
+--------------------+-------+---------------+------------------+------------+------------------+-------------+
| cp-kafka-connect | 13646 | java | 10.0.1.41 | 49018 | 10.0.1.41 | 9092 |
| cp-kafka-connect | 13646 | java | 10.0.1.41 | 49028 | 10.0.1.41 | 9092 |
| cp-kafka-connect | 13646 | java | 10.0.1.41 | 49026 | 10.0.1.41 | 9092 |
| cp-kafka-connect | 13646 | java | 10.0.1.41 | 50558 | 10.0.1.43 | 9092 |
| cp-kafka-connect | 13646 | java | 10.0.1.41 | 50554 | 10.0.1.43 | 9092 |
| cp-kafka-connect | 13646 | java | 10.0.1.41 | 49014 | 10.0.1.41 | 9092 |
| root | 1505 | sssd_be | 10.0.1.41 | 46436 | 10.0.1.89 | 389 |
...
| cp-ksql | 1757 | java | 10.0.1.41 | 56180 | 10.0.1.41 | 9092 |
| cp-ksql | 1757 | java | 10.0.1.41 | 53878 | 10.0.1.43 | 9092 |
| root | 19684 | sshd | 10.0.1.41 | 22 | 10.0.1.53 | 50238 |
| root | 24082 | sshd | 10.0.1.41 | 22 | 10.0.1.53 | 51233 |
| root | 24107 | java | 10.0.1.41 | 56052 | 10.0.1.41 | 9092 |
| root | 24107 | java | 10.0.1.41 | 56054 | 10.0.1.41 | 9092 |
| cp-schema-registry | 24694 | java | 10.0.1.41 | 50742 | 10.0.1.31 | 2181 |
| cp-schema-registry | 24694 | java | 10.0.1.41 | 47150 | 10.0.1.42 | 9093 |
| cp-schema-registry | 24694 | java | 10.0.1.41 | 58068 | 10.0.1.41 | 9093 |
| cp-schema-registry | 24694 | java | 10.0.1.41 | 47152 | 10.0.1.42 | 9093 |
| root | 25782 | osqueryd | 10.0.1.41 | 57700 | 10.0.1.43 | 9092 |
| root | 25782 | osqueryd | 10.0.1.41 | 56188 | 10.0.1.41 | 9092 |
+--------------------+-------+---------------+------------------+------------+------------------+-------------+
Instead of snapshots, I'd like osquery to capture differentials, i.e. to only publish the changes to Kafka.
I tried toggling the snapshot property from true to false. My expectation was that osquery would send the changes. For some reason, when I set "snapshot": false, no data is published to the process-port topic. Instead, all the data is routed to the catchall base_topic.
Can you see what I'm doing wrong?
Update:
I think I'm running into this bug: https://github.com/osquery/osquery/issues/5559
Here's a video walk-through: https://youtu.be/sPdlBBKgJmY
I filed a bug report, with steps to reproduce, in case it's not the same issue: https://github.com/osquery/osquery/issues/5890
Given the context, I can't immediately tell what is causing the issue you are experiencing.
In order to debug this, I would first try using the filesystem logger plugin instead of (or in addition to) the Kafka logger.
Do you get results to the Kafka topic when the query is configured as a snapshot? If so, are you able to verify that the results are actually changing such that a diff should be generate when the query runs in differential mode?
Can you see results logged locally when you use --logger_plugin=filesystem,kafka?
For some reason, I cannot get HandlerSocket to start listening when I start mariadb (version
10.0.14). I am using Cent OS 6.5.
my.cnf has the following settings:
handlersocket_port = 9998
handlersocket_port_wr = 9999
handlersocket_address = 127.0.0.1
Calling "SHOW GLOBAL VARIABLES LIKE 'handlersocket%'" from the mariaDb prompt shows:
+-------------------------------+-----------+
| Variable_name | Value |
+-------------------------------+-----------+
| handlersocket_accept_balance | 0 |
| handlersocket_address | 127.0.0.1 |
| handlersocket_backlog | 32768 |
| handlersocket_epoll | 1 |
| handlersocket_plain_secret | |
| handlersocket_plain_secret_wr | |
| handlersocket_port | 9998 |
| handlersocket_port_wr | 9999 |
| handlersocket_rcvbuf | 0 |
| handlersocket_readsize | 0 |
| handlersocket_sndbuf | 0 |
| handlersocket_threads | 16 |
| handlersocket_threads_wr | 1 |
| handlersocket_timeout | 300 |
| handlersocket_verbose | 10 |
| handlersocket_wrlock_timeout | 12 |
+-------------------------------+-----------+
I can start mariadb successfully, but when I check to see which ports are actively listening,
neither 9998 nor 9999 show up. I've checked the mysqld.log file, but no errors seem to be occurring.
Answering my own question here -
SELINUX needed to be set to permissive mode to get HandlerSocket started.