mosquitto auth plugin database issue - postgresql

I am using mosquitto auth plugin here is my mosquitto.config file
listener 1883
#listener 9001 127.0.0.1
#protocol websockets
auth_opt_backends postgres
auth_plugin /etc/mosquitto/auth-plug.so
auth_opt_dbname mqtt
auth_opt_host localhost
auth_opt_port 5432
auth_opt_user postgres
auth_opt_pass postgres
auth_opt_userquery SELECT password FROM account WHERE username = $1 limit 1
auth_opt_superquery SELECT COALESCE(COUNT(*),0) FROM account WHERE username = $1 AND super = 1
auth_opt_aclquery SELECT topic FROM acls WHERE (username = $1) AND (rw >= $2)
all postgres setting are default postgres settings.
when I run
sudo /usr/local/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
command. I am getting this error:
1456228482: mosquitto version 1.4.8 (build date 2016-02-22 18:23:09+0530) starting
1456228482: Config loaded from /etc/mosquitto/mosquitto.conf.
1456228482: |-- *** auth-plug: startup
1456228482: |-- ** Configured order: postgres
1456228482: |-- }}}} POSTGRES
1456228482: |-- HERE: SELECT COALESCE(COUNT(*),0) FROM account WHERE username = $1 AND super = 1
1456228482: |-- HERE: SELECT topic FROM acls WHERE (username = $1) AND (rw >= $2)
|-- We were unable to connect to the database
|-- *** ABORT.
If I uncomment these two line
#listener 9001 127.0.0.1
#protocol websockets
i get this error:
Error: Websockets support not available.
Error found at /etc/mosquitto/mosquitto.conf:4.
Error: Unable to open configuration file.
I was following these tutorial
Tutorial1
Tutorial2
after changing password of postgres and chaging the default port 1883
1456232627: mosquitto version 1.4.8 (build date 2016-02-22 18:23:09+0530) starting
1456232627: Config loaded from /etc/mosquitto/mosquitto.conf.
1456232627: |-- *** auth-plug: startup
1456232627: |-- ** Configured order: postgres
1456232627: |-- }}}} POSTGRES
1456232627: |-- HERE: SELECT COALESCE(COUNT(*),0) FROM account WHERE username = $1 AND super = 1
1456232627: |-- HERE: SELECT topic FROM acls WHERE (username = $1) AND (rw >= $2)
1456232627: Opening ipv4 listen socket on port 1884.
1456232627: Opening ipv6 listen socket on port 1884.
1456232627: Error: Invalid user 'mosquitto'.
from where "mosquitto" user is comming ?

Taking the what has been given in the comment to form an actual answer:
The websockets problem is down to not building with Websockets enabled in config.mk by modifying the following line:
WITH_WEBSOCKETS:=no
And then rebuilding. You will need to ensure you have the libwebsocket development files installed.
The We were unable to connect to the database error was a database configuration issue
Error: Invalid user 'mosquitto'. is because you are trying to run mosquitto as root and mosquitto is trying to drop the root privileges in order to safely run at a lower level. By default mosquitto will try to run as a user called mosquitto, in this case there is no mosquitto user defined on the system hence the error. You can change the user that it will try and become by adding a the following to the mosquitto.conf
user foo
This would cause mosquitto to run as user foo. You really should not leave mosquitto running as root.

Related

Postgres permissions.yml for fileLoad(permissions.yml) CONFIG_ERROR

I am following the https://deepstream.io/tutorials/plugins/database/postgres/
I have create a testDB with owner and permission to user james
In the conf/config.yml, I have the following configuration
plugins:
storage:
name: postgres
options:
user: james
database: testDB
password: james
host: localhost
port: 5432 #postgres default post
schema: ds #schema defaults to ds. Will be created if it doesn't exist
max: 10 #concurrent connections
idleTimeoutMillis: 30000 #timeout after which connection will be cut
writeInterval: 200 #amout of milliseconds during which writes will be buffered
notifications:
CREATE_TABLE: true #Get notified when tables are created
DESTROY_TABLE: true #Get notified when tables are dropped
INSERT: true # Get notified when records are created
UPDATE: false # Get notified when records are updated
However, when I run deepstream start, I got the following error
CONFIG_TRANSFORM | Loaded content from /Users/james/Workspace/deepstream.io/conf/permissions.yml for fileLoad(permissions.yml)
CONFIG_ERROR | Error loading module, exiting
Could someone please tell me what I have missed. Thank you for your help.
regards,
Johan
finally, I solved the problem, the configuration is correct, just to make sure that the postgres database and the role is done properly. The role needs to have a password and also LOGIN. Also, I clone the codes from github, so I can see what the error is when npm start. This will shows you verbose error message. If no error shown, npm test, this will show you all the missing packages and try to npm install again

Flyway not able to find role with postgres docker

I am trying to run my first flyway example using docker postgres image but getting the following error:
INFO: Flyway Community Edition 6.4.2 by Redgate
Exception in thread "main" org.flywaydb.core.internal.exception.FlywaySqlException:
Unable to obtain connection from database (jdbc:postgresql://localhost/flyway-service) for user 'flyway-service': FATAL: role "flyway-service" does not exist
-------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL State : 28000
Error Code : 0
Message : FATAL: role "flyway-service" does not exist
at org.flywaydb.core.internal.jdbc.JdbcUtils.openConnection(JdbcUtils.java:65)
at org.flywaydb.core.internal.jdbc.JdbcConnectionFactory.<init>(JdbcConnectionFactory.java:80)
I looked up into the docker container and can see that the user role flyway-service is created as part of the docker-compose execution:
$ docker exec -it flywayexample_postgres_1 bash
root#b2037e382112:/# psql -U flyway-service;
psql (12.2 (Debian 12.2-2.pgdg100+1))
Type "help" for help.
flyway-service=# \du;
List of roles
Role name | Attributes | Member of
----------------+------------------------------------------------------------+-----------
flyway-service | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
flyway-service=#
Main class is:
public static void main( String[] args ) {
var flyway = Flyway.configure().schemas("flyway_test_schema")
.dataSource("jdbc:postgresql://localhost/flyway-service", "flyway-service",
"password")
.load()
.migrate();
System.out.println( "Flyway example's hello world!" );
}
}
The migration called src/main/resources/db/migration/V1__Create_person_table.sql:
create table PERSON (
ID int not null,
NAME varchar(100) not null
);
Docker-compose yml file:
version: "3.8"
services:
postgres:
image: postgres:12.2
ports: ["5432:5432"]
environment:
- POSTGRES_PASSWORD=password
- POSTGRES_USER=flyway-service
I am running this code on MAC OSX. I assume, I am missing something obvious here, but not sure what! Any pointers would be appreciated.
Finally managed to figure out the issue with the help of a friend! The problem was not with the attached code but with a postgres daemon process running on the same port 5432 by an old Postgres installation.
I found the complete uninstallation procedure here. After removing the additional daemon process got only one port listening.
a$ lsof -n -i4TCP:5432
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
com.docke 654 root 50u IPv6 0x7ae1b5f8fbcf1cb 0t0 TCP *:postgresql (LISTEN)

PGBouncer : Cant connect on the right db

I'm actually facing an issue. I've installed pgbouncer on a production server, on which i've a Odoo instance and postgresql as well.
Perhaps :
In my logs, i'm having this :
2018-09-10 16:39:16.389 10123 WARNING C-0x1eb5478:
(nodb)/(nouser)#unix(18272):6432 pooler error: no such database: postgres
2018-09-10 16:39:16.389 10123 LOG C-0x1eb5478: (nodb)/(nouser)#unix(18272):6432 login failed: db=postgres user=oerppreprod
Here is the actual conf of pgbouncer :
pgbouncer_archive = host=127.0.0.1 port=5432 dbname=archive
admin_users = postgres
ignore_startup_parameters = extra_float_digits
With aswell, the default config (i've only added/edited this).
Why is he trying to connect on the postgres database ?
When i go back on the previous conf (without PGBouncer, just swapping from port 6432 to 5432), everything is working ....
Any idea ?
Thanks in advance !
I had the same issue, and in my situation. Maybe it will be usefull to somebody:
I have solved this by a few steps:
At the beginning of every request - your Framework or PDO (or else) running the initial query to check if database you asking is exists in the postgres data to process you request.
I have replaced the part of line "user=project_user password=mytestpassword" from the database section of pgbouncer.ini file. As I tested, if you replace this part - then the pgbouncer will use your userlist.txt file (or your selected auth), in my case, it was the userlist.txt.
Added the line "postgres = host=127.0.0.1 port=5432 dbname=postgres"
[databases]
postgres = host=127.0.0.1 port=5432 dbname=postgres
my_database = host=127.0.0.1 port=5432 dbname=my_database
My userlist.txt file looks like this (I am using auth_type = md5, so my password was in md5):
"my_user" "md5passwordandsoelse"
I have added my admin users to my pgbouncer.ini file:
admin_users = postgres, my_user
After all manipulations I advise you to check from which user u are running queries, by usin this simple query:
select current_user;
At the end, with this query you must to receive you selected username (in my case it was - my_user)
p.s. also I must to mention, that I was using 127.0.0.1 - because my pgbouncer is installed on the same server with postgres.

Hook up Grails to Postgresql

I am trying to hook up my Grails app to Postgresql. My versions:
Grails Version: 3.2.4
| Groovy Version: 2.4.7
| JVM Version: 1.8.0_60
Postgres installed through Homebrew version 9.6.1
Pouring postgresql-9.6.1.yosemite.bottle.tar.gz
added newest postgres jdbc driver in build.gradle
dependencies {
runtime 'org.postgresql:postgresql:9.4-1206-jdbc4'
}
In application.yml under development datasource I've added
driverClassName: org.postgresql.Driver
dialect: org.hibernate.dialect.PostgreSQLDialect
dbCreate: none
username: myUsername
url: jdbc:postgresql://localhost:5432/business
I have a domain class named Firm with properties that match the firm table in business database.
package bus.proto
class Firm {
int id
String firm_name
static constraints = {
}
}
Postgres query
business=# select * from firm;
id | firm_name
----+--------------
1 | example_firm
(1 row)
business=#
I am now confused as to how to figure out my configuration. I had rebooted my Mac and not had any instances of Postgresql running. I the ran
brew services start postgres
and then ran a
sudo lsof -i -n -P | grep TCP
to get
postgres 807 username 5u IPv6 0x3ba11f77231b4ebb 0t0 TCP [::1]:5432 (LISTEN)
postgres 807 username 6u IPv4 0x3ba11f772808980b 0t0 TCP 127.0.0.1:5432 (LISTEN)
When I hit the firm/index page I get the exception
ERROR: column this_.version does not exist Position: 30
thrown by the line
def firms = Firm.list()
which is located in the index() method in the FirmController.
I'm not sure how to pinpoint the problem after searching for the error message. Any help would be greatly appreciated!

psql can connect to a unix domain socket, but py-postgresql with the same parameters gets 'Permission denied'

Problem description:
My system user is milosz, which is mapped to the PostgreSQL user project_great in pg_ident.conf. I am using peer authentication to connect to a PostgreSQL database over a unix domain socket. This connection method works when using psql, but fails to work when using py-postgresql using the same parameters from within a Python script.
Here I am successfully connecting to the database using psql:
$ psql -U project_great \
> -d project_great \
> -h /var/run/postgresql
psql (9.3.4)
Type "help" for help.
project_great=>
Here is database_test.py:
#!/usr/bin/env python3
import postgresql
params = {
'user': 'project_great',
'database': 'project_great',
'unix': '/var/run/postgresql',
}
connection = postgresql.open(**params)
Here I am attempting to connect to the database by running ./database_test.py:
$ ./database_test.py
Traceback (most recent call last):
File "./database_test.py", line 11, in <module>
sys.exit(main(sys.argv))
File "./database_test.py", line 13, in main
connection = postgresql.open(**params)
File "/home/milosz/devel/project_great/.virtualenv/lib/python3.3/site-packages/postgresql/__init__.py", line 94, in open
c.connect()
File "/home/milosz/devel/project_great/.virtualenv/lib/python3.3/site-packages/postgresql/driver/pq3.py", line 2422, in connect
self._establish()
File "/home/milosz/devel/project_great/.virtualenv/lib/python3.3/site-packages/postgresql/driver/pq3.py", line 2548, in _establish
self.typio.raise_client_error(could_not_connect, creator = self, cause = exc)
File "/home/milosz/devel/project_great/.virtualenv/lib/python3.3/site-packages/postgresql/driver/pq3.py", line 514, in raise_client_error
raise client_error
postgresql.exceptions.ClientCannotConnectError: could not establish connection to server
CODE: 08001
LOCATION: CLIENT
CONNECTION: [failed]
failures[0]:
socket'/var/run/postgresql'
Traceback (most recent call last):
File "/home/milosz/devel/project_great/.virtualenv/lib/python3.3/site-packages/postgresql/protocol/client3.py", line 136, in connect
self.socket = self.socket_factory(timeout = timeout)
File "/home/milosz/devel/project_great/.virtualenv/lib/python3.3/site-packages/postgresql/python/socket.py", line 64, in __call__
s.connect(self.socket_connect)
PermissionError: [Errno 13] Permission denied
The above exception was the direct cause of the following exception:
postgresql.exceptions.ConnectionRejectionError: Permission denied
CODE: 08004
LOCATION: CLIENT
CONNECTOR: [Unix] pq://project_great#[unix::var:run:postgresql]/project_great
category: None
DRIVER: postgresql.driver.pq3.Driver
Since the parameters to the two connections are ostensibly the same and my permissions on the socket and its containing directory are fairly open, I do not know what the issue is. Using TCP is not a solution; I want to use unix domain sockets. The py-postgresql documentation indicates that connecting using unix domain sockets should work.
Configuration:
pg_hba.conf:
# TYPE DATABASE USER ADDRESS METHOD OPTION
local all all peer map=default
pg_ident.conf:
# MAPNAME SYSTEM-USERNAME PG-USERNAME
default postgres postgres
default milosz project_great
postgresql.conf:
...
port = 5432
unix_socket_directories = '/var/run/postgresql'
...
Here are the permissions on my socket directory:
$ ll /var/run/postgresql/
total 8
drwxrwsr-x 2 postgres postgres 100 May 17 00:20 ./
drwxr-xr-x 31 root root 900 May 17 00:41 ../
-rw-r--r-- 1 postgres postgres 5 May 17 00:20 9.3-main.pid
srwxrwxrwx 1 postgres postgres 0 May 17 00:20 .s.PGSQL.5432=
-rw------- 1 postgres postgres 70 May 17 00:20 .s.PGSQL.5432.lock
The PostgreSQL user project_great has been granted all privileges on the database project_great and both the user and the database exist.
I do not have a ~/.pgpass.
Environment:
Ubuntu 13.10
Python 3.3
PostgreSQL 9.3
py-postgresql 1.1.0
Craig Ringer suggested to run both programs under strace to find out if there is a relevant difference in the system calls. I grepped for /var/run/postgresql and found that while psql ran connect like so:
connect(4, {sa_family=AF_LOCAL, sun_path="/var/run/postgresql/.s.PGSQL.5432"}, 110) = 0
./database_test.py ran connect like so:
connect(4, {sa_family=AF_LOCAL, sun_path="/var/run/postgresql"}, 21) = -1 EACCES (Permission denied)
wherein lay the issue.
While psql expects the path to the directory in which the unix domain socket is located, py-postgresql expects the full path to the socket. Therefore, the fix was to modify database_test.py to read like so:
#!/usr/bin/env python3
import postgresql
params = {
'user': 'project_great',
'database': 'project_great',
'unix': '/var/run/postgresql/.s.PGSQL.5432',
}
connection = postgresql.open(**params)
Honestly, I feel silly for not having tried the full path as an earlier debugging step!