I am receiving the following warning stack trace when starting up WSO2:
TID: [-1234] [] [2018-06-20 14:53:46,020] WARN {org.wso2.andes.store.rdbms.RDBMSAndesContextStoreImpl} - Rollback failed on removing node information node id: NODEwso2-manager.dev.reach.ie.local/10.3.30.202 {org.wso2.andes.store.rdbms.RDBMSAndesContextStoreImpl}
org.postgresql.util.PSQLException: Cannot rollback when autoCommit is enabled.
Followed by:
TID: [-1234] [] [2018-06-20 14:53:46,025] ERROR {org.wso2.andes.server.Main} - Exception during startup. Triggering shutdown {org.wso2.andes.server.Main}
org.wso2.andes.kernel.AndesException: Unable to initialise application registry
at org.wso2.andes.server.Broker.startupImpl(Broker.java:308)
at org.wso2.andes.server.Broker.startup(Broker.java:110)
at org.wso2.andes.server.Main.startBroker(Main.java:217)
at org.wso2.andes.server.Main.execute(Main.java:206)
and
Caused by: org.wso2.andes.kernel.AndesException: Error occurred while removing node information node id: NODEwso2-manager.dev.reach.ie.local/10.3.30.202
I'm just asking for some guidance as to what the issue might be.
Thanks,
Simply set your connection string to autocommit a false.
Take a look to Can I change AutoCommit property of a JDBC connection multiple times.
and the configuration docs
https://docs.wso2.com/m/mobile.action#page/87687527
Related
I have been running PAM 7.9/JBPM 7.48 for about a year under JBOSS EAP 7.3. My JBPM's KieServer persists using SQL Server. I repeatedly deployed the KieServer yesterday but deploying today fails.
2021-12-16 15:25:53,645 ERROR [org.jboss.as.controller.management-operation] (DeploymentScanner-threads - 1) WFLYCTL0348: Timeout after [300] seconds waiting for service container stability. Operation will roll back. Step that first updated the service container was 'full-replace-deployment' at address '[]'
2021-12-16 15:26:03,649 ERROR [org.jboss.as.controller.management-operation] (DeploymentScanner-threads - 1) WFLYCTL0190: Step handler org.jboss.as.server.deployment.DeploymentHandlerUtil$4#74e289e9 for operation full-replace-deployment at address [] failed handling operation rollback -- java.util.concurrent.TimeoutException: java.util.concurrent.TimeoutException
at org.jboss.as.controller.OperationContextImpl.waitForRemovals(OperationContextImpl.java:523)
at org.jboss.as.controller.AbstractOperationContext$Step.handleResult(AbstractOperationContext.java:1518)
I have already set the property to increase the timeout for the deployment but it still complains about a 5 second timeout that must be controlled by another property
2021-12-16 13:40:47,039 ERROR [org.jboss.as.controller.management-operation] (DeploymentScanner-threads - 1) WFLYCTL0349: Timeout after [5] seconds waiting for service container stability while finalizing an operation. Process must be restarted. Step that first updated the service container was 'deploy' at address '[("deployment" => "kie-server.war")]'
I have changed the logging level to trace in order to gain all the information I can. How else can I debug / solve this issue?
There are two factors that may be contributing to this, but I don't have a good approach for addressing them.
There was a Windows Update yesterday (likely due to the recent Log4j exploit)
Some people at my company are having problems connecting to the SQL Server database. I am not seeing log messages about KieServer being unable to connect to the DB, but when it cannot reaching the DB the KieServer fails to start.
I am trying to upgrade jboss/keycloak 6.0.1 to 13.0.1 which is running as StatefulSet in k8s. I have converted my standalone-ha.xml and I am getting the following error:
[0m[32m13:05:34,632 DEBUG [org.infinispan.persistence.manager.PersistenceManagerImpl] (ServerService Thread Pool -- 68) PersistenceManagerImpl encountered an exception during startup of stores: java.util.concurrent.CompletionException: org.infinispan.persistence.spi.PersistenceException: ISPN000580: Failed to migrate persisted data.
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1739)
at org.jboss.threads#2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads#2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
at org.jboss.threads#2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads#2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1348)
at org.jboss.as.clustering.common#23.0.2.Final//org.jboss.as.clustering.context.ContextReferenceExecutor.execute(ContextReferenceExecutor.java:49)
at org.jboss.as.clustering.common#23.0.2.Final//org.jboss.as.clustering.context.ContextualExecutor$1.run(ContextualExecutor.java:70)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.infinispan.persistence.spi.PersistenceException: ISPN000580: Failed to migrate persisted data.
at org.infinispan#11.0.9.Final//org.infinispan.persistence.file.SingleFileStore.migrateFromV1(SingleFileStore.java:373)
at org.infinispan#11.0.9.Final//org.infinispan.persistence.file.SingleFileStore.start(SingleFileStore.java:160)
at org.infinispan#11.0.9.Final//org.infinispan.persistence.support.NonBlockingStoreAdapter.lambda$start$0(NonBlockingStoreAdapter.java:108)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
... 7 more
Caused by: protostream.com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
at org.infinispan.protostream#4.3.5.Final//protostream.com.google.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:102)
at org.infinispan.protostream#4.3.5.Final//protostream.com.google.protobuf.CodedInputStream$ArrayDecoder.readTag(CodedInputStream.java:627)
at org.infinispan.protostream#4.3.5.Final//org.infinispan.protostream.impl.RawProtoStreamReaderImpl.readTag(RawProtoStreamReaderImpl.java:45)
at org.infinispan.protostream#4.3.5.Final//org.infinispan.protostream.WrappedMessage.readMessage(WrappedMessage.java:275)
at org.infinispan.protostream#4.3.5.Final//org.infinispan.protostream.ProtobufUtil.fromWrappedByteArray(ProtobufUtil.java:162)
at org.infinispan#11.0.9.Final//org.infinispan.marshall.persistence.impl.PersistenceMarshallerImpl.objectFromByteBuffer(PersistenceMarshallerImpl.java:155)
at org.infinispan#11.0.9.Final//org.infinispan.persistence.file.SingleFileStore.migrateFromV1(SingleFileStore.java:333)
... 10 more
Any idea how to tackle this error?
Do you try to update your Keycloak system while still running any instances? This is not supported by Keycloak. It's recommended to shut down all instances and start migration with one instance, when upgrading major versions!
Between Keycloak 6 and 13 the underlying Infinispan versions have changed and thus at some point also the de-/serialization mechanism. Most possibly that's the cause for your errors.
The error was due the cache. In order to make it work the cache will need to be migrated from the syntax that keycloak 6 uses to 13. In our case, we preferred to start with a new cache.
Official solution from Red Hat:
Delete saved web session data by removing all of the the contents of the directory
$JBOSS_HOME/standalone/data/infinispan/web
After we setup PostgreSQL server with SSL certificates we get this error very often.
This happens on the flows with a lot of interaction with the database(update a lot of entries/insert)
HikariPool-1 - Connection marked as broken because of SQLSTATE(08006), ErrorCode(0)
j.n.SocketException: Connection reset by peer (Write failed)
at j.n.SocketOutputStream.socketWrite0(SocketOutputStream.java)
at j.n.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
at j.n.SocketOutputStream.write(SocketOutpu`enter code here`tStream.java:150)
at s.s.s.SSLSocketOutputRecord.deliver(SSLSocketOutputRecord.java:320)
at s.s.s.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:983)
... 63 common frames omitted
Wrapped by: j.n.s.SSLProtocolException: Connection reset by peer (Write failed)
at s.security.ssl.Alert.createSSLException(Alert.java:126)
at s.s.s.TransportContext.fatal(TransportContext.java:321)
at s.s.s.TransportContext.fatal(TransportContext.java:264)
at s.s.s.TransportContext.fatal(TransportContext.java:259)
at s.s.s.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:988)
at j.i.BufferedOutputStream.write(BufferedOutputStream.java:123)
at j.i.FilterOutputStream.write(FilterOutputStream.java:108)
at o.p.core.PGStream.send(PGStream.java:252)
at o.p.c.v.QueryExecutorImpl.sendParse(QueryExecutorImpl.java:1440)
at o.p.c.v.QueryExecutorImpl.sendOneQuery(QueryExecutorImpl.java:1767)
at o.p.c.v.QueryExecutorImpl.sendQuery(QueryExecutorImpl.java:1328)
at o.p.c.v.QueryExecutorImpl.execute(QueryExecutorImpl.java:300)
... 56 common frames omitted
Wrapped by: o.p.u.PSQLException: An I/O error occurred while sending to the backend
I was able to solve this issue by adding below properties:
spring.datasource.test-while-idle=true
spring.datasource.validation-query=SELECT 1;
spring.datasource.time-between-eviction-runs-millis=60000
spring.datasource.hikari.idle-timeout=600000
spring.datasource.hikari.validationTimeout=300000
spring.datasource.remove-abandoned=true
spring.datasource.remove-abandoned-timeout=60
spring.datasource.hikari.connectionTimeout=30000
spring.datasource.hikari.idleTimeout=600000
spring.datasource.hikari.maxLifetime=1800000
Fixed the problem by updating jdk version from 11.0.2 to 11.0.5. Related to
JDK-8221253 TLSv1.3 may generate TLSInnerPlainText longer than 2^14+1 bytes
I am working through the SSL example which means starting the locator using the properties files instead of default. So it's
start locator --name=IsLocator --properties-file=..\config\gemfire.properties --security-properties-file=..\config\gfsecurity.properties
Then whether the SSL is set in the properties files or not, this causes the locator process to terminate with an error
The Locator process terminated unexpectedly with exit status 1. Please refer to the log file in D:\Geode\bin\IsLocator for full details.
Sep 21, 2017 11:23:22 AM org.apache.geode.distributed.LocatorLauncher failOnStart
INFO: locator is exiting due to an exception
java.lang.RuntimeException: java.io.IOException: Unable to write to deploy directory: D:\tmp
at org.apache.geode.internal.JarDeployer.loadPreviouslyDeployedJarsFromDisk(JarDeployer.java:410)
at org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1191)
at org.apache.geode.internal.cache.GemFireCacheImpl.basicCreate(GemFireCacheImpl.java:758)
at org.apache.geode.internal.cache.GemFireCacheImpl.create(GemFireCacheImpl.java:745)
at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:173)
at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:212)
at org.apache.geode.distributed.internal.InternalLocator.startCache(InternalLocator.java:723)
at org.apache.geode.distributed.internal.InternalLocator.startDistributedSystem(InternalLocator.java:710)
at org.apache.geode.distributed.internal.InternalLocator.startLocator(InternalLocator.java:324)
at org.apache.geode.distributed.LocatorLauncher.start(LocatorLauncher.java:651)
at org.apache.geode.distributed.LocatorLauncher.run(LocatorLauncher.java:553)
at org.apache.geode.distributed.LocatorLauncher.main(LocatorLauncher.java:195)
Caused by: java.io.IOException: Unable to write to deploy directory: D:\tmp
at org.apache.geode.internal.JarDeployer.verifyWritableDeployDirectory(JarDeployer.java:333)
at org.apache.geode.internal.JarDeployer.loadPreviouslyDeployedJarsFromDisk(JarDeployer.java:389)
... 11 more`
The same error happens if GFSH is started as an administrator.
This problem resolves if you remove the deploy-working-dir property.
I configured the rhq-agent.bat. When I try to run it. It has the followings errors:
2015-06-25 14:51:24,891 ERROR [RHQ Server Polling Thread] (enterprise.communications.command.client.JBossRemotingRemoteCommunicator)- {JBossRemotingRemoteCommunicator.init-callback-failed}The initialize callback has failed. It will be tried again. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker after 1 attempt(s) -> java.net.ConnectException:Connection timed out: connect. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker after 1 attempt(s)
2015-06-25 14:51:42,987 ERROR [main] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.plugin-update-failure}Failed to update the plugins.. Cause: java.lang.IllegalStateException: The sender object is currently not sending commands now. Command not sent: [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getLatestPlugins], targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService}]]
I could solve it myself the problem was the security token. I modified those fields in the registry vierwer (Windows). When I restarted the agent it worked.