Logstash(docker) unable to connect database.
Error => Unable to connect to database. Tried 1 times {:message=>Java::OrgPostgresqlUtil::PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections., :exception=>Sequel::DatabaseConnectionError
logstash.conf file:
input{
jdbc {
jdbc_driver_library => '/usr/share/logstash/logstash-core/lib/jars/postgresql-42.5.0.jar'
jdbc_driver_class => "org.postgresql.Driver"
jdbc_connection_string => "jdbc:postgresql://localhost:2022/testdb"
jdbc_user => "postgres"
jdbc_password => "12345"
statement => "SELECT * FROM users"
schedule => "* * * * *"
}
}
output {
elasticsearch {
hosts => "elasticsearch:9200"
index => "%{indexName}"
}
}
logstash.yml file:
http.host: "0.0.0.0"
path.config: /usr/share/logstash/pipeline
xpack.monitoring.elasticsearch.hosts: ["localhost:9200"]
xpack.monitoring.enabled: true
How can i connect to database?
I fixed this problem as follows =>
1-)My postgreSQL port is "2022" so i changed IPV4 port=127.0.0.1/22 in pg_hba.conf file.
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/22 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/22 trust
host replication all ::1/128 trust
host all all all md5
2-) Use computer ip address(ipconfig) instead of localhost=
old= jdbc:postgresql://localhost:2022/testdb
new= jdbc:postgresql://192.168.1.83:2022/testdb
Hi I am currently trying to set up a mongo db on my home server and expose it to the internet using cloudflare tunnels.
I have a service up and running and have the following for the connection.
client = MongoClient('<DATABASE_URL>')
I get this error...
pymongo.errors.InvalidURI: Invalid URI scheme: URI must begin with 'mongodb://' or 'mongodb+srv://'
I am tunneling the default ip that mongo gives you.
UPDATE
I tested connecting to the db and just printing the database to the console. I got this result
Database(MongoClient(host=['<my_domain>:27107'], document_class=dict, tz_aware=False, connect=True), 'test_db')
I assume that because it says "connect=true" that means it is connecting to the database now.
I tried to add a collection to the database using an example I got online and this is the error I received...
Traceback (most recent call last):
File "/home/michael/mongo.py", line 18, in <module>
x = mycol.insert_one(mydict)
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/collection.py", line 628, in insert_one
self._insert_one(
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/collection.py", line 569, in _insert_one
self.__database.client._retryable_write(acknowledged, _insert_command, session)
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1475, in _retryable_write
with self._tmp_session(session) as s:
File "/home/michael/anaconda3/lib/python3.9/contextlib.py", line 119, in __enter__
return next(self.gen)
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1757, in _tmp_session
s = self._ensure_session(session)
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1740, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1685, in __start_session
self._topology._check_implicit_session_support()
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/topology.py", line 538, in _check_implicit_session_support
self._check_session_support()
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/topology.py", line 554, in _check_session_support
self._select_servers_loop(
File "/home/michael/anaconda3/lib/python3.9/site-packages/pymongo/topology.py", line 238, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: No servers found yet, Timeout: 30s, Topology Description: <TopologyDescription id: 63d172246419f5effc5e32d3, topology_type: Unknown, servers: [<ServerDescription ('<my_domain>', 27107) server_type: Unknown, rtt: None>]>
For reference this is what my pymongo test file looks like.
mongo.py
import pymongo
con = pymongo.MongoClient("mongodb://<my_domain>:27107")
db = con["test_db"]
mycol = db["customers"]
print(mycol)
print(db)
mydict = { "name": "John", "address": "Highway 37" }
x = mycol.insert_one(mydict)
If it's a standard installation, you need to make sure cloudflare tunnel is exposing port 27017. The ingress rule must be:
tcp://localhost:27017
To connect, just use:
pymongo.MongoClient("mongodb://user:psw#host.YourTLD/table")
It's a good idea to activate authentication if you're exposing the whole server to the internet. You can do it by setting authentication on the mongodb server, or at the cloudflare zero trust edge following this guide:
https://developers.cloudflare.com/cloudflare-one/tutorials/mongodb-tunnel/
I guess this is the case here:
you have a locally deployed mongodb (not some external VM)
you've set a Cloudflare tunnel in order to expose mongodb over dns
and you are having problems to connect to mongodb using that dns
So I've recently been trying to do the same, and I got over it with these steps:
First off, make sure that your service type, in Cloudflare Zero Trust, is TCP
URL is probably localhost, make sure you specified port
download cloudflared: Apple Silicon & everything else probably
run this on your local machine that you want to connect from: cloudflared access tcp --hostname <hostname you've set on Cloudflare ZT> --url <url you want to be forwarded to>. For example: cloudflared access tcp --hostname mongo.example.com --url localhost:3000
Then try to connect with your app to the localhost:3000.
How does this work?
Well, first you install cloudflared service, which forwards encrypted connection from an app on your machine to the outer internet.
You can protect access to that forwarded service/app using access rules. I also recommend protecting your app/service, you can do it from MongoDB or Cloudflare ZT, or both.
Then, you run cloudflared app on your target machine
connect to Cloudflare servers which forwards your MongoDB instance connection to the specified port on your local machine
you can access it as its local deployment
We are looking to migrate some systems away from MSSQL. We have our first few environments built and currently using LDAP, which is OK but has a good number of flaws.
I followed this link to setup Kreberos\GSS for the most part: https://info.crunchydata.com/blog/windows-active-directory-postgresql-gssapi-kerberos-authentication
Off the bat I got the below error when trying to connect:
psql: error: SSPI continuation error: The specified target is unknown or unreachable
I believe the SPN is setup properly:
setspn -S POSTGRES/server.domain.local domain\service_account
I suspect something is wrong in the keytab file, as there is an extra "" between the server FQDN and domain:
Keytab name: FILE:/opt/pgsql/server.keytab
KVNO Principal
---- --------------------------------------------------------------------------
5 postgres#server.domain.local\#DOMAIN.LOCAL
Server side error:
2020-12-28 18:37:43.820 EST [64534] user#DOMAIN.LOCAL#postgres FATAL: GSSAPI authentication failed for user "user#DOMAIN.LOCAL"
2020-12-28 18:37:43.820 EST [64534] user#DOMAIN.LOCAL#postgres DETAIL: Connection matched pg_hba.conf line 95: "host all all 0.0.0.0/0 gss"
I'd appreciate any feedback and thank you!
I just began to learn about scala and want to find a way to connect to PostgreSQL with scalalikejdbc. I set up PostgreSQL on machine1 and want to connect to one from machine2.
This is my PostgreSQL configurations:
postgresql.conf
listen_addresses = '*' # what IP address(es) to listen on;
# comma-separated list of addresses;
# defaults to 'localhost'; use '*' for all
pg_hba.conf
# "local" is for Unix domain socket connections only
local all all md5
# IPv4 local connections:
host all all 127.0.0.1/32 ident
# IPv6 local connections:
host all all ::1/128 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all peer
host replication all 127.0.0.1/32 ident
host replication all ::1/128 ident
host all postgres 192.168.1.1/24 md5
This is my piece of code to connect to PostgreSQL that written by scala
def main(args: Array[String]): Unit = {
val jdbcDriver = "org.postgresql.Driver"
val jdbcUser = "postgres"
val jdbcPassword = "******"
val jdbcUrl = "jdbc:postgresql://192.168.50.1:5432/jdbc"
Class.forName(jdbcDriver)
ConnectionPool.singleton(jdbcUrl, jdbcUser, jdbcPassword)
val data = DB.readOnly { implicit session =>
sql"""SELECT * FROM Users""".
map { resultSet =>
resultSet.int(1) -> resultSet.int(2)
}
.list.apply().toMap
}
I get an error that "Connection to 192.168.50.1:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections."
Anyone can explain to me. Thanks a lot
I'm trying to move my bot to an Ubuntu virtual server from Vultr but it's having a problem connecting to the postgres database. I've tried editing the config from md5 to true, and host to local, etc. But those only give me different errors and also make it stop working on my original machine too. It's working perfectly fine on my Windows machine. Here is the error I'm facing:
asyncpg.exceptions.InvalidAuthorizationSpecificationError: no pg_hba.conf entry for host "[local]", user "postgres", database "xxx", SSL off
So I've tried to change this line:
async def create_db_pool():
bot.pg_con = await asyncpg.create_pool(database='xxx', user='postgres', password='???')
to this:
async def create_db_pool():
bot.pg_con = await asyncpg.create_pool(database='xxx', user='postgres', password='???', ssl=True)
and that gives me this error:
asyncpg.exceptions._base.InterfaceError: `ssl` parameter can only be enabled for TCP addresses, got a UNIX socket path: '/run/postgresql/.s.PGSQL.5432'
So I don't know what else to try. I've been stuck on this for a while. If it's relevant, it connects at the bottom of the bot.py file like this:
bot.loop.run_until_complete(create_db_pool())
Whether ssl is True or not, the database seems to still function on my Windows machine. But I can't get it to work on my Ubuntu virtual server.
If I edit my config to this:
# TYPE DATABASE USER ADDRESS METHOD
# IPv4 local connections:
host all all 0.0.0.0/0 md5
# IPv6 local connections:
host all all ::/0 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
host replication all 0.0.0.0/0 md5
host replication all ::/0 md5
Then I get a call error like this:
OSError: Multiple exceptions: [Errno 111] Connect call failed ('::1', 5432, 0, 0), [Errno 111] Connect call failed ('127.0.0.1', 5432)
This is really driving me crazy. I have no idea what to do. I bought this virtual server to host my bot on but I can't even get it to connect to the database.
When I simply type psql in the terminal, I get this error:
Error: Invalid data directory for cluster 12 main
Postgres is not working as intended in basically any way. I'm using Vultr.com to host the Ubuntu server, if that matters. And connecting with PuTTy.
Your pg_hba.conf has multiple syntax errors. The "localhost" connection type is not allowed at all, and the "local" connection type does not accept an IP address field. The server would refuse to start/restart with the file you show, and if you try to reload a running server it will just keep using the previous settings.
LOG: invalid connection type "localhost"
CONTEXT: line 4 of configuration file "/home/jjanes/pgsql/data/pg_hba.conf"
LOG: invalid authentication method "127.0.0.1/32"
CONTEXT: line 5 of configuration file "/home/jjanes/pgsql/data/pg_hba.conf"
LOG: invalid authentication method "::1/128"
CONTEXT: line 9 of configuration file "/home/jjanes/pgsql/data/pg_hba.conf"
LOG: invalid connection type "localhost"
CONTEXT: line 10 of configuration file "/home/jjanes/pgsql/data/pg_hba.conf"
LOG: invalid authentication method "127.0.0.1/32"
CONTEXT: line 102 of configuration file "/home/jjanes/pgsql/data/pg_hba.conf"
FATAL: could not load pg_hba.conf
LOG: database system is shut down