Keycloak - Connection Null issue when spun up on AWS ECS - keycloak

I have checked that all relevant parameters are being parsed correctly and i am able to see them in my AWS ECS service. When the service is spun up i get the below error WARN  [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (ServerService Thread Pool -- 60) IJ000604: Throwable while attempting to get a new connection: null: javax.resource.ResourceException: IJ031084: Unable to create connection which then follows with this error Caused by: org.postgresql.util.PSQLException: FATAL: database "keycloak" does not exist
I am unsure if the two are related. I validated relevant parameters and have tried to imitate the same issue locally.
Any help or advice would be aprpeciated
TYIA

Related

Host name resolve error for AWS Fargate task instances

I'm trying to run a dockerized WildFly 26 EE 9.1 preview on a AWS Fargate cluster but it fails to start with
[31m09:15:52,203 ERROR [org.jboss.as.controller.management-operation] (ServerService Thread Pool -- 76) WFLYCTL0013: Operation ("add") failed - address: ([("subsystem" => "webservices")]): java.lang.RuntimeException: java.net.UnknownHostException: 7a0765d6977e48a98e3b16f7533674f7-3378193245: 7a0765d6977e48a98e3b16f7533674f7-3378193245: Name does not resolve
at org.jboss.as.webservices//org.jboss.as.webservices.dmr.WSSubsystemAdd.createServerConfig(WSSubsystemAdd.java:106)
at org.jboss.as.webservices//org.jboss.as.webservices.dmr.WSSubsystemAdd.performBoottime(WSSubsystemAdd.java:93)
at org.jboss.a...#18.0.4.Final//org.jboss.as.controller.AbstractBoottimeAddStepHandler.performBoottime(AbstractBoottimeAddStepHandler.java:144)
at org.jboss.a...#18.0.4.Final//org.jboss.as.controller.AbstractBoottimeAddStepHandler.performRuntime(AbstractBoottimeAddStepHandler.java:119)
at org.jboss.a...#18.0.4.Final//org.jboss.as.controller.AbstractAddStepHandler$1.execute(AbstractAddStepHandler.java:164)
at org.jboss.a...#18.0.4.Final//org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:1045)
at org.jboss.a...#18.0.4.Final//org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:777)
at org.jboss.a...#18.0.4.Final//org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:466)
at org.jboss.a...#18.0.4.Final//org.jboss.as.controller.ParallelBootOperationStepHandler$ParallelBootTask.run(ParallelBootOperationStepHandler.java:384)
at org.jbos...#2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jbos...#2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
at org.jbos...#2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jbos...#2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1348)
at java.base/java.lang.Thread.run(Thread.java:831)
at org.jbos...#2.4.0.Final//org.jboss.threads.JBossThread.run(JBossThread.java:513)
Caused by: java.net.UnknownHostException: 7a0765d6977e48a98e3b16f7533674f7-3378193245: 7a0765d6977e48a98e3b16f7533674f7-3378193245: Name does not resolve
at java.base/java.net.InetAddress.getLocalHost(InetAddress.java:1644)
at org.jboss...#3.3.3.Final//org.jboss.ws.common.management.AbstractServerConfig.setWebServiceHost(AbstractServerConfig.java:133)
at org.jboss.as.webservices//org.jboss.as.webservices.config.ServerConfigImpl.setWebServiceHost(ServerConfigImpl.java:97)
at org.jboss.as.webservices//org.jboss.as.webservices.dmr.WSSubsystemAdd.createServerConfig(WSSubsystemAdd.java:104)
... 14 more
Caused by: java.net.UnknownHostException: 7a0765d6977e48a98e3b16f7533674f7-3378193245: Name does not resolve
at java.base/java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:932)
at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1517)
at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:851)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1507)
at java.base/java.net.InetAddress.getLocalHost(InetAddress.java:1639)
The cluster is deployed in a private subnet but I have a security group that allows all outgoing traffic and all traffic is routed through a NAT Gateway. I recall using the same CloudFormation scripts for WildFly 24:ish and it worked fine so it shouldn't(tm) be a networking issue.
Is the host name generally supposed to be resolvable?
I’m not sure if this issue still persists on your side, but for me it was resolved as soon as I enabled "DNS Hostnames" in VPC ("DNS resolution" was enabled already). I have no idea why this option matters, but even with disabling public IP in Service defintion of ECS the wildfly threw the error. Since the option is enabled (which only seems to enable hostnames for public ips) it is working fine.

ZuulException: Forwarding error in Spring cloud running locally

I get this very common zuul error, when running spring cloud microservices locally.
If I specify microservice address in application.yml it works fine, so I am wondering why it doesn't discovery the service address properly without explicitly specifying it, especially that this is a new behaviour.
zuul:
routes:
my-service:
path: /myservice/**
#url: http://localhost:8081
Any hints what could have caused this?
Here is some part of stacktrace.
Caused by: java.lang.RuntimeException: java.net.SocketException: Permission denied: connect
at rx.exceptions.Exceptions.propagate(Exceptions.java:57)
at rx.observables.BlockingObservable.blockForSingle(BlockingObservable.java:463)
at rx.observables.BlockingObservable.single(BlockingObservable.java:340)
at com.netflix.client.AbstractLoadBalancerAwareClient.executeWithLoadBalancer(AbstractLoadBalancerAwareClien t.java:112)
... 137 common frames omitted
Caused by: java.net.SocketException: Permission denied: connect
Thanks!

Dataproc trying to connect to Postgres through JDBC, missing permissions

Thanks in advance...
I want to connect/write using JDBC APIs to a Postgres SQL instance running using Cloud SQL programmatically. I have used following jars:
postgresql
postgres-socket-factory
postgres-socket-factory-1.0.11-jar-with-dependencies.jar
I am running Dataproc which will try to connect using step #1, but I get following exception:
2019-04-01 11:05:03.998 IST
Something unusual has occurred to cause the driver to fail. Please report this exception.
at org.postgresql.Driver.connect(Driver.java:277)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at rdsConnector$.getConnection(rdsConnector.scala:33)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:721)
Caused by: java.lang.RuntimeException: Unable to retrieve information about Cloud SQL instance [projectID:us-east1:dB]
at com.google.cloud.sql.core.SslSocketFactory.obtainInstanceMetadata(SslSocketFactory.java:459)
at com.google.cloud.sql.core.SslSocketFactory.fetchInstanceSslInfo(SslSocketFactory.java:333)
at com.google.cloud.sql.core.SslSocketFactory.getInstanceSslInfo(SslSocketFactory.java:313)
at com.google.cloud.sql.core.SslSocketFactory.createAndConfigureSocket(SslSocketFactory.java:194)
at com.google.cloud.sql.core.SslSocketFactory.create(SslSocketFactory.java:160)
at com.google.cloud.sql.postgres.SocketFactory.createSocket(SocketFactory.java:96)
at org.postgresql.core.PGStream.<init>(PGStream.java:62)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:91)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:192)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
at org.postgresql.Driver.makeConnection(Driver.java:454)
at org.postgresql.Driver.connect(Driver.java:256)
... 11 more
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
2019-04-01 11:05:03.000 IST
User class threw exception: org.postgresql.util.PSQLException: Something unusual has occurred to cause the driver to fail. Please report this exception.
I understand this as a permission issue, but since I am using Dataproc to connect to Postgres what permission is missing? if I was running from a local laptop machine then I have to set GOOGLE_APPLICATION_CREDENTIALS to a json file. But what is the process in case of Dataproc?
The JDBC SocketFactory uses the Application Default Credentials strategy for accessing account credentials.
For Cloud Dataproc, a default service account is provided for you at [project-number]-compute#developer.gserviceaccount.com. You grant this account the "Cloud SQL Client" IAM role, the JDBC SocketFactory will use it to authenticate, and thus you will be able to connect to your application.
Just leave further notes for future visitors:
If the error you're getting is like:
{
"code" : 403,
"errors" : [ {
"domain" : "global",
"message" : "Insufficient Permission",
"reason" : "insufficientPermissions"
} ],
"message" : "Request had insufficient authentication scopes.",
"status" : "PERMISSION_DENIED"
}
then it could be caused by issues with the scopes (Cloud API access scopes) of the VM instances. One way to fix this is adding the scope of sql-admin when creating the Dataproc cluster. For example:
gcloud dataproc clusters create <your-cluster-name> \
--region=<your-region> \
--zone=<your-zone> \
--scopes=https://www.googleapis.com/auth/sqlservice.admin \
...
Another way is edited that in the VM instances.
See the gcloud documentation for details

How to configure PostgreSQL database for deploying alfresco on tomcat 8?

I have built alfresco(version 5.2) from source on ubuntu 16.04. I want to deploy alfresco on tomcat 8. The deployment is successful however the PostgreSQL database is not getting configured as required. I have followed the steps as given in http://docs.alfresco.com/5.1/tasks/postgresql-config.html
I observe the home page as given in image alfresco_page
Am I missing onto something here that the PostgreSQL database is not getting configured. Is there any other configuration that needs to be done that I have missed ?
UPDATE
The alfresco.log gave me this
2017-08-01 05:53:54,406 WARN [org.alfresco.web.scripts.servlet.X509ServletFilterBase] [localhost-startStop-1] clientAuth does not appear to be set for Tomcat. clientAuth must be set to 'want' for X509 Authentication
2017-08-01 05:53:54,416 WARN [org.alfresco.web.scripts.servlet.X509ServletFilterBase] [localhost-startStop-1] Attempting to set clientAuth=want through JMX...
2017-08-01 05:53:54,427 WARN [org.alfresco.web.scripts.servlet.X509ServletFilterBase] [localhost-startStop-1] Unable to set clientAuth=want through JMX.
2017-08-01 05:53:55,139 ERROR [org.apache.solr.core.CoreContainer] [coreLoadExecutor-5-thread-1] Error creating core [collection1]: Could not load conf for core collection1: Error loading solr config from solr/collection1/conf/solrconfig.xml
org.apache.solr.common.SolrException: Could not load conf for core collection1: Error loading solr config from solr/collection1/conf/solrconfig.xml
at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:66)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:489)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: Error loading solr config from solr/collection1/conf/solrconfig.xml
at org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:154)
at org.apache.solr.core.ConfigSetService.createSolrConfig(ConfigSetService.java:80)
at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:61)
... 7 more
Caused by: java.io.IOException: Can't find resource 'solrconfig.xml' in classpath or '/root/tomcat85/output/build/webapps/solr/collection1/conf'
at org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:362)
at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:308)
at org.apache.solr.core.Config.<init>(Config.java:117)
at org.apache.solr.core.Config.<init>(Config.java:87)
at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:167)
at org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:145)
... 9 more
2017-08-01 05:54:09,634 WARN [org.hibernate.cfg.SettingsFactory] [localhost-startStop-1] Could not obtain connection metadata
org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1549)
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
at org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConnection(LocalDataSourceConnectionProvider.java:83)
at org.hibernate.cfg.SettingsFactory.buildSettings(SettingsFactory.java:84)
at org.hibernate.cfg.Configuration.buildSettings(Configuration.java:2079)
at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1304)
at org.springframework.orm.hibernate3.LocalSessionFactoryBean.newSessionFactory(LocalSessionFactoryBean.java:863)
at org.springframework.orm.hibernate3.LocalSessionFactoryBean.buildSessionFactory(LocalSessionFactoryBean.java:782)
at org.springframework.orm.hibernate3.AbstractSessionFactoryBean.afterPropertiesSet(AbstractSessionFactoryBean.java:188)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1573)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1511)
Things to check:
Is postgres running (ps -ef|grep postgres)?
Can you use psql to connect to postgres using the db.name, db.username, and db.password that are configured in alfresco-global.properties?
Did you follow the step in the docs about editing pg_hba.conf to make sure that postgres is configured to allow password based authentication?
Also, it is exceedingly rare to need to build Alfresco from source unless you are making changes to the low-level classes themselves, which is not recommended.

Springboot HikariCP

I using springboot with HikariCP, but after a while my app crash and I got the error:
org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection
...
Caused by: org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection
....
Caused by: java.sql.SQLTransientConnectionException: HikariPool-6 - Connection is not available, request timed out after 30000ms.
This is my aplication.properties
spring.datasource.type=com.zaxxer.hikari.HikariDataSource
spring.datasource.url=jdbc:postgresql://localhost:5432/db_dnaso
#spring.datasource.url=jdbc:postgresql://172.16.1.10:5432/db_dnaso
spring.datasource.username=postgres
spring.datasource.password=dna44100
spring.datasource.driver-class-name=org.postgresql.Driver
So I have a lot of save, find and anothers access to DB, how can I visualize how method are blocking my connection?
tks
It looks like your your database server is running out of connection. Default value for property maximumPoolSize in Hikari is 10. That means it will try to create 10 connection on server startup and it will not start if not able to acquire 10 connection or may fail if your db server pool size having less connection in the pool as you are creating using Hikari configuration.
If you are able to start Spring Boot server and then facing this issue then try enabling leakDetectionThreshold and check which connection is taking more time and not returning to Hikari pool .
spring:
datasource:
hikari:
leak-detection-threshold: 2000
Enable the leakDetectionThreshold, set to something like 1 minute (60000ms). It is likely that you have a connection leak somewhere ... a connection is borrowed but never closed (returned).
In springboot, you can set spring.datasource.hikari.leak-detection-threshold=10000 (in milliseconds) in application.properties.
And the default value for maximumPoolSize is 10. You can change it with spring.datasource.hikari.maximum-pool-size=xx.