We are using OrientDB 3.0.10 version and we are planning to migrate our application from Java11 to Java17. But application is stopped working due to OrientDB exception. On the latest version of OrientDB also same issue exists.
22-Mar-2022 12:35:15.140 ERROR [Catalina-utility-1] o.a.catalina.core.StandardContext.listenerStart - Exception sending context initialized event to listener instance of class [com.progress.fathom.servlet.FathomContextInitializer]
java.lang.ExceptionInInitializerError: null
at javassist.util.proxy.DefineClassHelper.<clinit>(DefineClassHelper.java:216)
at javassist.util.proxy.FactoryHelper.toClass(FactoryHelper.java:128)
at javassist.util.proxy.ProxyFactory.createClass3(ProxyFactory.java:552)
at javassist.util.proxy.ProxyFactory.createClass2(ProxyFactory.java:537)
at javassist.util.proxy.ProxyFactory.createClass1(ProxyFactory.java:473)
at javassist.util.proxy.ProxyFactory.createClass(ProxyFactory.java:444)
at com.orientechnologies.orient.object.enhancement.OObjectEntityEnhancer.getProxiedInstance(OObjectEntityEnhancer.java:97)
at com.orientechnologies.orient.object.db.OObjectDatabaseTx.newInstance(OObjectDatabaseTx.java:211)
at com.orientechnologies.orient.object.db.OObjectDatabaseTx.newInstance(OObjectDatabaseTx.java:115)
at com.orientechnologies.orient.object.enhancement.OObjectEntitySerializer.serializeObject(OObjectEntitySerializer.java:146)
at com.orientechnologies.orient.object.db.OObjectDatabaseTx.save(OObjectDatabaseTx.java:499)
at com.orientechnologies.orient.object.db.OObjectDatabaseTx.save(OObjectDatabaseTx.java:444)
at com.progress.isq.ipqos.AbstractDatabase.updateDatabaseSchemaVersion(AbstractDatabase.java:455)
at com.progress.isq.ipqos.AbstractDatabase.checkAndUpdateSchema(AbstractDatabase.java:481)
at com.progress.isq.ipqos.AbstractDatabase.applySchema(AbstractDatabase.java:399)
at com.progress.isq.ipqos.AbstractDatabase.openDatabase(AbstractDatabase.java:374)
at com.progress.isq.ipqos.AbstractDatabase.open(AbstractDatabase.java:296)
at com.progress.isq.ipqos.Probe.loadProject(Probe.java:1817)
at com.progress.isq.ipqos.Probe.loadAndInitializeProjectResources(Probe.java:1195)
at com.progress.isq.ipqos.Probe.createAndInitializeBaselineResources(Probe.java:678)
at com.progress.isq.ipqos.Probe.start(Probe.java:391)
at com.progress.fathom.servlet.FathomContextInitializer.contextInitialized(FathomContextInitializer.java:298)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4768)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5230)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:726)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:698)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:696)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1024)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1911)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:123)
at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:825)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:475)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1618)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:319)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:423)
at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:366)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:946)
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:835)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1396)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1386)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module #46762e3e
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:199)
at java.base/java.lang.reflect.Method.setAccessible(Method.java:193)
at javassist.util.proxy.SecurityActions$3.run(SecurityActions.java:94)
at javassist.util.proxy.SecurityActions$3.run(SecurityActions.java:90)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)
at javassist.util.proxy.SecurityActions.getMethodHandle(SecurityActions.java:89)
at javassist.util.proxy.DefineClassHelper$SecuredPrivileged$2.getDefineClassMethodHandle(DefineClassHelper.java:143)
at javassist.util.proxy.DefineClassHelper$SecuredPrivileged$2.<init>(DefineClassHelper.java:136)
at javassist.util.proxy.DefineClassHelper$SecuredPrivileged.<clinit>(DefineClassHelper.java:134)
... 52 common frames omitted
To fix this we have added --add-opens=java.base/java.lang=ALL-UNNAMED to our application as mitigation but we know that this is not the valid fix for the issue. OrientDB needs to come up with a fix i think.
Another thing is OrientDB already added other java packages to Add-Opens in orient jar manifest file
Manifest-Version: 1.0
Implementation-Title: OrientDB Object
Implementation-Version: 3.2.5
Built-By: tglman
Specification-Vendor: Orient Technologies
Specification-Title: OrientDB Object
Implementation-Vendor-Id: com.orientechnologies
Implementation-Vendor: Orient Technologies
Implementation-Build-Date: 2022-02-14 18:19:43+0000
Add-Opens: jdk.unsupported/sun.misc=ALL-UNNAMED java.base/sun.security
.x509=ALL-UNNAMED
X-Compile-Target-JDK: 8
Implementation-Build: c4298657c01683192ba0b7bfffdf82226c164506
X-Compile-Source-JDK: 8
Created-By: Apache Maven 3.6.3
Build-Jdk: 1.8.0_312
Specification-Version: 3.2
Implementation-URL: http://www.orientechnologies.com
Is OrientDB officially supporting Java17 and is there any plan to fix this issue?
I'm on Ubuntu 18.04 with JAVA openjdk 10.0.2 and Eclipse Photon (PHP edition).
DLTK Indexing seems to not work properly:
If I put my pointer on a PHP function , I have all documentation from PHP doc.
But if I put the pointer on any other functions (from symfony for example or on one of mine) nothing appears.
I have already check the build path and try to make a clean reinstall.
I also tried to delete files in : eclipse-workspace/.metadata/.plugins/org.eclipse.dltk.core.index.lucene/index
In eclipse logs, the only message I can't understand is :
java.lang.IllegalArgumentException: Maximum limit of resources to transfer is: 1000000
at org.eclipse.ui.part.ResourceTransfer.nativeToJava(ResourceTransfer.java:178)
at org.eclipse.swt.dnd.Clipboard.getContents(Clipboard.java:323)
at org.eclipse.swt.dnd.Clipboard.getContents(Clipboard.java:243)
at org.eclipse.ui.internal.navigator.resources.actions.PasteAction.lambda$0(PasteAction.java:187)
at org.eclipse.swt.widgets.Synchronizer.syncExec(Synchronizer.java:236)
at org.eclipse.ui.internal.UISynchronizer.syncExec(UISynchronizer.java:147)
at org.eclipse.swt.widgets.Display.syncExec(Display.java:5941)
at org.eclipse.ui.internal.navigator.resources.actions.PasteAction.updateSelection(PasteAction.java:184)
at org.eclipse.ui.actions.BaseSelectionListenerAction.selectionChanged(BaseSelectionListenerAction.java:127)
at org.eclipse.ui.internal.navigator.resources.actions.EditActionGroup.updateActionBars(EditActionGroup.java:156)
at org.eclipse.ui.internal.navigator.resources.actions.EditActionProvider.updateActionBars(EditActionProvider.java:62)
at org.eclipse.ui.navigator.NavigatorActionService$3.run(NavigatorActionService.java:269)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
at org.eclipse.ui.navigator.NavigatorActionService.fillActionBars(NavigatorActionService.java:261)
at org.eclipse.ui.navigator.CommonNavigatorManager.selectionChanged(CommonNavigatorManager.java:220)
at org.eclipse.jface.viewers.Viewer$1.run(Viewer.java:158)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
at org.eclipse.ui.internal.JFaceUtil.lambda$0(JFaceUtil.java:47)
at org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:176)
at org.eclipse.jface.viewers.Viewer.fireSelectionChanged(Viewer.java:155)
at org.eclipse.jface.viewers.StructuredViewer.updateSelection(StructuredViewer.java:2182)
at org.eclipse.jface.viewers.StructuredViewer.setSelection(StructuredViewer.java:1719)
at org.eclipse.jface.viewers.TreeViewer.setSelection(TreeViewer.java:1084)
at org.eclipse.ui.navigator.CommonViewer.setSelection(CommonViewer.java:373)
at org.eclipse.jface.viewers.Viewer.setSelection(Viewer.java:383)
at org.eclipse.wst.jsdt.internal.ui.packageview.PackageExplorerContentProvider$7.run(PackageExplorerContentProvider.java:1514)
at org.eclipse.wst.jsdt.internal.ui.packageview.PackageExplorerContentProvider.runUpdates(PackageExplorerContentProvider.java:198)
at org.eclipse.wst.jsdt.internal.ui.packageview.PackageExplorerContentProvider.runPendingUpdates(PackageExplorerContentProvider.java:190)
at org.eclipse.wst.jsdt.internal.ui.packageview.PackageExplorerContentProvider$1.runInUIThread(PackageExplorerContentProvider.java:168)
at org.eclipse.ui.progress.UIJob.lambda$0(UIJob.java:98)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:185)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:5026)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4586)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1173)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:339)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1062)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:156)
at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:628)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:339)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:563)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:151)
at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:155)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:199)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:137)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:107)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:391)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:246)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:659)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:595)
at org.eclipse.equinox.launcher.Main.run(Main.java:1501)
at org.eclipse.equinox.launcher.Main.main(Main.java:1474)
Thanks for your help
enuts
I am using Spring boot application to develop Postgresql based application using Eclipse IDE(openjdk version 1.8.0_151)
I have configured gradle as follows:
compile group: 'org.postgresql', name: 'postgresql', version: '42.1.4'
compile group: 'org.springframework.boot', name: 'spring-boot-starter-jdbc'
I am using Spring JdbcTemplate to access database. Everything works perfect in Eclipse IDE. I want to create a war file and deploy into Tomcat 9 server. But unfortunately, I got the following error message:
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: No suitable driver found for jdbc:postgresql://server:5432/db_pm
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:619)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:684)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:716)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:726)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:776)
at org.springframework.jdbc.core.JdbcTemplate.queryForList(JdbcTemplate.java:853)
at com.kpaudel.pm.PasswordManagerDAO.getSimilarServices(PasswordManagerDAO.java:108)
at com.kpaudel.pm.PasswordManagerDAO$$FastClassBySpringCGLIB$$482fbdc8.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
at com.kpaudel.pm.PasswordManagerDAO$$EnhancerBySpringCGLIB$$8f3e1618.getSimilarServices(<generated>)
at com.kpaudel.telegram.bots.PasswordManagerBot.onUpdateReceived(PasswordManagerBot.java:87)
at java.util.ArrayList.forEach(ArrayList.java:1255)
at org.telegram.telegrambots.generics.LongPollingBot.onUpdatesReceived(LongPollingBot.java:27)
at org.telegram.telegrambots.updatesreceivers.DefaultBotSession$HandlerThread.run(DefaultBotSession.java:301)
Caused by: java.sql.SQLException: No suitable driver found for jdbc:postgresql://server:5432/db_pm
at java.sql.DriverManager.getConnection(DriverManager.java:689)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:153)
at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:144)
at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnectionFromDriver(AbstractDriverBasedDataSource.java:196)
at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnection(AbstractDriverBasedDataSource.java:159)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
... 19 common frames omitted
I have copied Postgresql driver (postgresql-42.1.4.jar) into $CATALINA_HOME/lib directory also, that also did not help.
I have installed Jboss AMQ 7.0.1.
Everything working fine. But getting following exception if I try to browse Wiki in Web Management Console.
javax.management.InstanceNotFoundException : io.fabric8:type=Fabric
Stack trace:
javax.management.InstanceNotFoundException: io.fabric8:type=Fabric
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1375)
at com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:920)
at org.jolokia.handler.ExecHandler.extractMBeanParameterInfos(ExecHandler.java:167)
at org.jolokia.handler.ExecHandler.extractOperationTypes(ExecHandler.java:143)
at org.jolokia.handler.ExecHandler.doHandleRequest(ExecHandler.java:84)
at org.jolokia.handler.ExecHandler.doHandleRequest(ExecHandler.java:40)
at org.jolokia.handler.JsonRequestHandler.handleRequest(JsonRequestHandler.java:89)
at org.jolokia.backend.MBeanServerExecutorLocal.handleRequest(MBeanServerExecutorLocal.java:109)
at org.jolokia.backend.MBeanServerHandler.dispatchRequest(MBeanServerHandler.java:159)
at org.jolokia.backend.LocalRequestDispatcher.dispatchRequest(LocalRequestDispatcher.java:99)
at org.jolokia.backend.BackendManager.callRequestDispatcher(BackendManager.java:413)
at org.jolokia.backend.BackendManager.handleRequest(BackendManager.java:158)
at org.jolokia.http.HttpRequestHandler.executeRequest(HttpRequestHandler.java:197)
at org.jolokia.http.HttpRequestHandler.handleGetRequest(HttpRequestHandler.java:86)
at org.jolokia.http.AgentServlet$4.handleRequest(AgentServlet.java:430)
at org.jolokia.http.AgentServlet.handleSecurely(AgentServlet.java:301)
at org.jolokia.http.AgentServlet.handle(AgentServlet.java:277)
at org.jolokia.http.AgentServlet.doGet(AgentServlet.java:237)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at io.hawt.web.AuthenticationFilter$3.run(AuthenticationFilter.java:201)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at io.hawt.web.AuthenticationFilter.executeAs(AuthenticationFilter.java:198)
at io.hawt.web.AuthenticationFilter.doFilter(AuthenticationFilter.java:136)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1621)
at io.hawt.web.CORSFilter.doFilter(CORSFilter.java:42)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1621)
at io.hawt.web.XXSSProtectionFilter.doFilter(XXSSProtectionFilter.java:28)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1621)
at io.hawt.web.XFrameOptionsFilter.doFilter(XFrameOptionsFilter.java:28)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1621)
at io.hawt.web.SessionExpiryFilter.process(SessionExpiryFilter.java:148)
at io.hawt.web.SessionExpiryFilter.doFilter(SessionExpiryFilter.java:46)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1613)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:541)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1593)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1239)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:481)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1141)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:564)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:122)
at org.eclipse.jetty.util.thread.strategy.ExecutingExecutionStrategy.invoke(ExecutingExecutionStrategy.java:58)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:201)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:133)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590)
at java.lang.Thread.run(Thread.java:745)
After some search I found it may be a reason of Fabric up-gradation, but not sure what exactly, I have to do to do that.
To my knowledge all the wiki functionality in the Hawtio-based web console has been disabled in Red Hat JBoss AMQ so I think this can safely be ignored.
I'm trying to access data on IBM COS from Data Science Experience based on this blog post.
First, I select 1.0.8 version of stocator ...
!pip install --user --upgrade pixiedust
import pixiedust
pixiedust.installPackage("com.ibm.stocator:stocator:1.0.8")
Restarted kernel, then ...
access_key = 'xxxx'
secret_key = 'xxxx'
bucket = 'xxxx'
host = 'lon.ibmselect.objstor.com'
hconf = sc._jsc.hadoopConfiguration()
hconf.set("fs.s3d.service.endpoint", "http://" + host)
hconf.set("fs.s3d.service.access.key", access_key)
hconf.set("fs.s3d.service.secret.key", secret_key)
file = 'mydata_file.tsv.gz'
inputDataset = "s3d://{}.service/{}".format(bucket, file)
lines = sc.textFile(inputDataset, 1)
lines.count()
However, that results in the following error:
Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: java.lang.AbstractMethodError: com/ibm/stocator/fs/common/IStoreClient.setStocatorPath(Lcom/ibm/stocator/fs/common/StocatorPath;)V
at com.ibm.stocator.fs.ObjectStoreFileSystem.initialize(ObjectStoreFileSystem.java:104)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:258)
at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:229)
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:199)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:251)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:249)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:249)
at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:251)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:249)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:249)
at org.apache.spark.api.python.PythonRDD.getPartitions(PythonRDD.scala:53)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:251)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:249)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:249)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1927)
at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:932)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:378)
at org.apache.spark.rdd.RDD.collect(RDD.scala:931)
at org.apache.spark.api.python.PythonRDD$.collectAndServe(PythonRDD.scala:453)
at org.apache.spark.api.python.PythonRDD.collectAndServe(PythonRDD.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:507)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:237)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:280)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:785)
Note: My first attempt at connecting to IBM COS resulted in a different error. That attempt is captured here: No FileSystem for scheme: cos
No need to install stocator it is already there. As Roland mentioned, new installation most likely would collide with the pre-installed one and cause conflicts.
Try ibmos2spark:
https://stackoverflow.com/a/46035893/8558372
Let me know if you still facing problems.
Don't force-install a new Stocator unless you have a really good reason.
I highly recommend the Spark aaS documentation at:
https://console.bluemix.net/docs/services/AnalyticsforApacheSpark/index-gentopic1.html#genTopProcId2
Please choose the correct COS endpoints from:
https://ibm-public-cos.github.io/crs-docs/endpoints
and PLEASE use the private endpoints if you're working from within the IBM Cloud. It will be much faster and cheaper.
It's got examples of how to access COS data using all the nice helpers. It'll boil down to
import ibmos2spark
credentials = {
'endpoint': 's3-api.us-geo.objectstorage.service.networklayer.com', #just an example. Your url might be different
'access_key': 'my access key',
'secret_key': 'my secret key'
}
bucket_name = 'my bucket name'
object_name = 'mydata_file.tsv.gz'
cos = ibmos2spark.CloudObjectStorage(sc, credentials)
lines = sc.textFile(cos.url(object_name, bucket_name),1)
lines.count()
Chris, I usually don't use the 'http://' in the endpoint and that works for me. Not sure if that is the problem here.
Here is how I access the COS objects from DSX notebooks
endpoint = "s3-api.dal-us-geo.objectstorage.softlayer.net"
hconf = sc._jsc.hadoopConfiguration()
hconf.set("fs.s3d.service.endpoint",endpoint)
hconf.set("fs.s3d.service.access.key",Access_Key_ID)
hconf.set("fs.s3d.service.secret.key",Secret_Access_Key)
inputObject = "s3d://<bucket>.service/<file>"
myRDD = sc.textFile(inputObject,1)
DSX has a version of stocator on the classpath for Spark 2.0 and Spark 2.1 kernels. The one you installed in your instance is likely to get into conflict with the pre-installed version.