no pg_hba.conf entry for host / Connect call failed / Invalid data directory for cluster 12 main - Postgresql Ubuntu - postgresql

I'm trying to move my bot to an Ubuntu virtual server from Vultr but it's having a problem connecting to the postgres database. I've tried editing the config from md5 to true, and host to local, etc. But those only give me different errors and also make it stop working on my original machine too. It's working perfectly fine on my Windows machine. Here is the error I'm facing:
asyncpg.exceptions.InvalidAuthorizationSpecificationError: no pg_hba.conf entry for host "[local]", user "postgres", database "xxx", SSL off
So I've tried to change this line:
async def create_db_pool():
bot.pg_con = await asyncpg.create_pool(database='xxx', user='postgres', password='???')
to this:
async def create_db_pool():
bot.pg_con = await asyncpg.create_pool(database='xxx', user='postgres', password='???', ssl=True)
and that gives me this error:
asyncpg.exceptions._base.InterfaceError: `ssl` parameter can only be enabled for TCP addresses, got a UNIX socket path: '/run/postgresql/.s.PGSQL.5432'
So I don't know what else to try. I've been stuck on this for a while. If it's relevant, it connects at the bottom of the bot.py file like this:
bot.loop.run_until_complete(create_db_pool())
Whether ssl is True or not, the database seems to still function on my Windows machine. But I can't get it to work on my Ubuntu virtual server.
If I edit my config to this:
# TYPE DATABASE USER ADDRESS METHOD
# IPv4 local connections:
host all all 0.0.0.0/0 md5
# IPv6 local connections:
host all all ::/0 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
host replication all 0.0.0.0/0 md5
host replication all ::/0 md5
Then I get a call error like this:
OSError: Multiple exceptions: [Errno 111] Connect call failed ('::1', 5432, 0, 0), [Errno 111] Connect call failed ('127.0.0.1', 5432)
This is really driving me crazy. I have no idea what to do. I bought this virtual server to host my bot on but I can't even get it to connect to the database.
When I simply type psql in the terminal, I get this error:
Error: Invalid data directory for cluster 12 main
Postgres is not working as intended in basically any way. I'm using Vultr.com to host the Ubuntu server, if that matters. And connecting with PuTTy.

Your pg_hba.conf has multiple syntax errors. The "localhost" connection type is not allowed at all, and the "local" connection type does not accept an IP address field. The server would refuse to start/restart with the file you show, and if you try to reload a running server it will just keep using the previous settings.
LOG: invalid connection type "localhost"
CONTEXT: line 4 of configuration file "/home/jjanes/pgsql/data/pg_hba.conf"
LOG: invalid authentication method "127.0.0.1/32"
CONTEXT: line 5 of configuration file "/home/jjanes/pgsql/data/pg_hba.conf"
LOG: invalid authentication method "::1/128"
CONTEXT: line 9 of configuration file "/home/jjanes/pgsql/data/pg_hba.conf"
LOG: invalid connection type "localhost"
CONTEXT: line 10 of configuration file "/home/jjanes/pgsql/data/pg_hba.conf"
LOG: invalid authentication method "127.0.0.1/32"
CONTEXT: line 102 of configuration file "/home/jjanes/pgsql/data/pg_hba.conf"
FATAL: could not load pg_hba.conf
LOG: database system is shut down

Related

Expose mongodb using cloudflare zero trust tunnels and connect via pymongo

Hi I am currently trying to set up a mongo db on my home server and expose it to the internet using cloudflare tunnels.
I have a service up and running and have the following for the connection.
client = MongoClient('<DATABASE_URL>')
I get this error...
pymongo.errors.InvalidURI: Invalid URI scheme: URI must begin with 'mongodb://' or 'mongodb+srv://'
I am tunneling the default ip that mongo gives you.
UPDATE
I tested connecting to the db and just printing the database to the console. I got this result
Database(MongoClient(host=['<my_domain>:27107'], document_class=dict, tz_aware=False, connect=True), 'test_db')
I assume that because it says "connect=true" that means it is connecting to the database now.
I tried to add a collection to the database using an example I got online and this is the error I received...
Traceback (most recent call last):
File "/home/michael/mongo.py", line 18, in <module>
x = mycol.insert_one(mydict)
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/collection.py", line 628, in insert_one
self._insert_one(
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/collection.py", line 569, in _insert_one
self.__database.client._retryable_write(acknowledged, _insert_command, session)
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1475, in _retryable_write
with self._tmp_session(session) as s:
File "/home/michael/anaconda3/lib/python3.9/contextlib.py", line 119, in __enter__
return next(self.gen)
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1757, in _tmp_session
s = self._ensure_session(session)
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1740, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1685, in __start_session
self._topology._check_implicit_session_support()
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/topology.py", line 538, in _check_implicit_session_support
self._check_session_support()
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/topology.py", line 554, in _check_session_support
self._select_servers_loop(
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/topology.py", line 238, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: No servers found yet, Timeout: 30s, Topology Description: <TopologyDescription id: 63d172246419f5effc5e32d3, topology_type: Unknown, servers: [<ServerDescription ('<my_domain>', 27107) server_type: Unknown, rtt: None>]>
For reference this is what my pymongo test file looks like.
mongo.py
import pymongo
con = pymongo.MongoClient("mongodb://<my_domain>:27107")
db = con["test_db"]
mycol = db["customers"]
print(mycol)
print(db)
mydict = { "name": "John", "address": "Highway 37" }
x = mycol.insert_one(mydict)
If it's a standard installation, you need to make sure cloudflare tunnel is exposing port 27017. The ingress rule must be:
tcp://localhost:27017
To connect, just use:
pymongo.MongoClient("mongodb://user:psw#host.YourTLD/table")
It's a good idea to activate authentication if you're exposing the whole server to the internet. You can do it by setting authentication on the mongodb server, or at the cloudflare zero trust edge following this guide:
https://developers.cloudflare.com/cloudflare-one/tutorials/mongodb-tunnel/
I guess this is the case here:
you have a locally deployed mongodb (not some external VM)
you've set a Cloudflare tunnel in order to expose mongodb over dns
and you are having problems to connect to mongodb using that dns
So I've recently been trying to do the same, and I got over it with these steps:
First off, make sure that your service type, in Cloudflare Zero Trust, is TCP
URL is probably localhost, make sure you specified port
download cloudflared: Apple Silicon & everything else probably
run this on your local machine that you want to connect from: cloudflared access tcp --hostname <hostname you've set on Cloudflare ZT> --url <url you want to be forwarded to>. For example: cloudflared access tcp --hostname mongo.example.com --url localhost:3000
Then try to connect with your app to the localhost:3000.
How does this work?
Well, first you install cloudflared service, which forwards encrypted connection from an app on your machine to the outer internet.
You can protect access to that forwarded service/app using access rules. I also recommend protecting your app/service, you can do it from MongoDB or Cloudflare ZT, or both.
Then, you run cloudflared app on your target machine
connect to Cloudflare servers which forwards your MongoDB instance connection to the specified port on your local machine
you can access it as its local deployment

Authentication Issues (KRB5\GSS)

We are looking to migrate some systems away from MSSQL. We have our first few environments built and currently using LDAP, which is OK but has a good number of flaws.
I followed this link to setup Kreberos\GSS for the most part: https://info.crunchydata.com/blog/windows-active-directory-postgresql-gssapi-kerberos-authentication
Off the bat I got the below error when trying to connect:
psql: error: SSPI continuation error: The specified target is unknown or unreachable
I believe the SPN is setup properly:
setspn -S POSTGRES/server.domain.local domain\service_account
I suspect something is wrong in the keytab file, as there is an extra "" between the server FQDN and domain:
Keytab name: FILE:/opt/pgsql/server.keytab
KVNO Principal
---- --------------------------------------------------------------------------
5 postgres#server.domain.local\#DOMAIN.LOCAL
Server side error:
2020-12-28 18:37:43.820 EST [64534] user#DOMAIN.LOCAL#postgres FATAL: GSSAPI authentication failed for user "user#DOMAIN.LOCAL"
2020-12-28 18:37:43.820 EST [64534] user#DOMAIN.LOCAL#postgres DETAIL: Connection matched pg_hba.conf line 95: "host all all 0.0.0.0/0 gss"
I'd appreciate any feedback and thank you!

Need help setting up MongoDB for SSL

I am trying to configure mongodb for ssl. I have the two certs within a directory on Ubuntu, but when I try to restart the service with the mongodb.conf set up correctly, the service will not start. If I comment out the lines in the mongodb.conf file that I added, I can then start mongodb. I think the syntax is wrong, and not the certs them self.
#SSL options
sslMode = requireSSL
#Enable SSL on normal ports
#sslOnNormalPorts = true
# SSL Key file and password
sslPEMKeyFile = /path/to/cert
sslPEMKeyPassword = password
sslCAFile = /path/to/cert
I get this error when I try to start the server with these lines not commented out
stop: Unknown instance:
mongodb start/running, process 7725
If i try to get into mongo shell i get this(assuming this is because I could not restart the service properly)
Thu Jul 21 14:32:07.660 Error: couldn't connect to server 127.0.0.1:27017 at src/mongo/shell/mongo.js:145
exception: connect failed
The mongodb.conf file is a YAML file so you need to format it as such. Meaning you can't use tabs. Also it does look like the syntax you're using isn't correct.
Try this:
net:
#SSL options
ssl:
mode: requireSSL
# SSL Key file and password
PEMKeyFile: /path/to/cert
PEMKeyPassword: password
CAFile: /path/to/cert
Also, I know it's commented out but just wanted to mention, the sslOnNormal ports option is deprecated. See here: https://docs.mongodb.com/manual/reference/configuration-options/#net.ssl.sslOnNormalPorts

Authentication problems using phpPgAdmin / PGSQL

I experiencing some problems with PostgreSQL that i need to install to run a SIlverpeas project.
Normally, the postgres part is correctly configured, because i've sent a command to make the user silverpeas owner of the database silverpeas, and i can connect by ssh using "pgsql -U silverpeas silverpeas"
Now, i'm trying to make my phpPgAdmin work. He is on the same server Centos 6.7 than Postgresql (in version 9.5).
When i'm trying to connect, i have a connection failed message, and sometimes, when i'm trying to troubleshoot the problem, i have the "trying to hack your system" message.
When i'm looking for the pgsql log file, after a connection trial, i have this message :
< 2016-03-16 11:32:39.378 CET >LOG: could not connect to Ident server at address "::1", port 113: Connection refused
< 2016-03-16 11:32:39.378 CET >FATAL: Ident authentication failed for user "silverpeas"
< 2016-03-16 11:32:39.378 CET >DETAIL: Connection matched pg_hba.conf line 84: "host all all ::1/128 ident"
The problem is, my /var/lib/pgsql/9.5/data/pg_hba.conf is like this :
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all md5
# IPv4 local connections:
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 md5
host all all 10.1.1.0/8 md5
And if i looked for the /var/lib/pgsql/9.5/data/postgresql.conf file :
listen_addresses = 'localhost'
And my phpPgAdmin configuration is :
// use 'localhost' for TCP/IP connection on this computer
$conf['servers'][0]['host'] = 'localhost';
// Database port on server (5432 is the PostgreSQL default)
$conf['servers'][0]['port'] = 5432;
// Database SSL mode
// Possible options: disable, allow, prefer, require
// To require SSL on older servers use option: legacy
// To ignore the SSL mode, use option: unspecified
$conf['servers'][0]['sslmode'] = 'allow';
// Change the default database only if you cannot connect to template1.
// For a PostgreSQL 8.1+ server, you can set this to 'postgres'.
$conf['servers'][0]['defaultdb'] = 'silverpeas';
Suggestions ?
The pg_hba.conf file doesn't match the log file.
You use md5 method (good) but the log file talks about ident method.
Have you change and reload your postgresql conf before ?

How to specify port for PostgreSQL?

I am lost. I have localhost database (PostgreSQL) and I have to add port for connection (in app.config -- connection string). I alredy tried:
localhost:port
localhost,port
(localhost),port
(local),port
None of this work, everytime I got error "The requested name is valid, but no data of the requested type was found" thrown by System.Net.Dns.InternalGetHostByName with message "cannot open connection".
So how do you specify the port? I checked this on computer with just single instance of DB server, so port could be ommitted and then it works. But I need adding port.
Update
<add key="ConnectionString" value="Server=localhost;
Port=5434;
Database=XXXXXXX;Initial Catalog=XXXXXXXXX;
UserID=XXXXX;Password=XXXXX;Encoding=UNICODE;" />
Now it works with both "localhost" and "127.0.0.1" (direct IP).
Use a separate keyword for the port:
Server=127.0.0.1;Port=...;User Id=...;Password=...;Database=...;