Does auto update functionality in install4j work if the the app is a web app as a service which has no launcher? I have tried many ways mentioned in web resources but no luck. Please help. I am using install4j Windows Edition 6.1.6.
Edit 1:
#Ingo Kegel,
I added -Dinstall4j.keepLog=true -Dinstall4j.alternativeLogfile=E:/install4j-log/log.txt as per your suggestion and got the below log. Please help.
[INFO] com.install4j.runtime.beans.actions.control.SetVariableAction [ID 630]: Execute action
Property script: com.install4j.script.I4jScript_Internal_38
Property variableName: updateDescriptorEntry
Property failIfNull: true
Property onlyIfUndefined: false
Property responseFileVariable: false
Property rollbackSupported: false
[ERROR] com.install4j.runtime.beans.actions.control.SetVariableAction [ID 630]: Execute action not successful after 4 ms
EDIT 2:
Below are the logs after executing code (After the exe file has been downloaded)
UpdateChecker.executeScheduledUpdate(Arrays.asList("-q"), false, null);
Logs:
[INFO] com.install4j.runtime.beans.actions.update.CheckForUpdateAction [ID 629]: Execute action
Property connectTimeout: 10000
Property connectionFailureScript: null
Property readTimeout: 20000
Property requestHeaders: []
Property url: http://localhost:8181/astra-downloads/updates.xml
Property variable: updateDescriptor
Property acceptAllCertificates: false
Property askForProxy: true
Property rollbackSupported: false
Property showError: true
Download: http://localhost:8181/astra-downloads/updates.xml to C:\WINDOWS\TEMP\i4jupd4650856858406689282.xml; size: 366 bytes
Variable changed: updateDescriptor=com.install4j.runtime.installer.config.update.UpdateDescriptorImpl#82c94f[class com.install4j.runtime.installer.config.update.UpdateDescriptorImpl]
Execute action successful after 44 ms
[INFO] com.install4j.runtime.beans.actions.control.SetVariableAction [ID 630]: Execute action
Property script: com.install4j.script.I4jScript_Internal_63
Property variableName: updateDescriptorEntry
Property failIfNull: true
Property onlyIfUndefined: false
Property responseFileVariable: false
Property rollbackSupported: false
Branch 1
3.0.1
Variable changed: updateDescriptorEntry=UpdateDescriptorEntry [target: 22, file: GreytipAstra_windows_3_0_1.exe][class com.install4j.runtime.installer.config.update.UpdateDescriptorEntryImpl]
Execute action successful after 4 ms
[INFO] com.install4j.runtime.beans.actions.control.SetVariableAction [ID 632]: Execute action
Property script: com.install4j.script.I4jScript_Internal_65
Property variableName: updaterNewVersion
Property failIfNull: false
Property onlyIfUndefined: false
Property responseFileVariable: false
Property rollbackSupported: false
Variable changed: updaterNewVersion=3.0.1[class java.lang.String]
Execute action successful after 0 ms
[INFO] com.install4j.runtime.beans.actions.control.SetVariableAction [ID 633]: Execute action
Property script: com.install4j.script.I4jScript_Internal_66
Property variableName: updaterDownloadUrl
Property failIfNull: false
Property onlyIfUndefined: false
Property responseFileVariable: false
Property rollbackSupported: false
Variable changed: updaterDownloadUrl=http://localhost:8181/astra-downloads/GreytipAstra_windows_3_0_1.exe[class java.lang.String]
Execute action successful after 0 ms
[INFO] com.install4j.runtime.beans.actions.control.SetVariableAction [ID 634]: Execute action
Property script: com.install4j.script.I4jScript_Internal_67
Property variableName: updaterDownloadFile
Property failIfNull: false
Property onlyIfUndefined: false
Property responseFileVariable: false
Property rollbackSupported: false
Variable changed: updaterDownloadFile=C:/Users/Abhishek/Downloads\GreytipAstra_windows_3_0_1.exe[class java.lang.String]
Execute action successful after 0 ms
[INFO] com.install4j.runtime.beans.actions.net.DownloadFileAction [ID 635]: Execute action
Property connectTimeout: 10000
Property connectionFailureScript: null
Property md5Url:
Property readTimeout: 20000
Property requestHeaders: []
Property targetFile: C:\Users\Abhishek\Downloads\GreytipAstra_windows_3_0_1.exe
Property url: http://localhost:8181/astra-downloads/GreytipAstra_windows_3_0_1.exe
Property acceptAllCertificates: false
Property askForProxy: true
Property checkForMd5Sums: false
Property deleteOnExit: false
Property dontAskForRetry: false
Property retryIfInterrupted: true
Property rollbackSupported: false
Property showError: true
Property showFileName: true
Property showProgress: true
Download: http://localhost:8181/astra-downloads/GreytipAstra_windows_3_0_1.exe to C:\Users\Abhishek\Downloads\GreytipAstra_windows_3_0_1.exe; size: 43569152 bytes
Execute action successful after 2640 ms
[INFO] com.install4j.runtime.beans.actions.files.SetModeAction [ID 637]: Execute action
Property directoryFilter: null
Property fileFilter: null
Property fileTarget: everything
Property files: [C:\Users\Abhishek\Downloads\GreytipAstra_windows_3_0_1.exe]
Property filesRoot: null
Property mode: 755
Property recursive: false
Property rollbackSupported: true
Property showFileNames: true
Property showProgress: false
Execute action successful after 0 ms
[INFO] com.install4j.runtime.beans.actions.update.ScheduleUpdateAction [ID 638]: Execute action
Property arguments: []
Property installerFile: C:\Users\Abhishek\Downloads\GreytipAstra_windows_3_0_1.exe
Property maxCancelCount: 10
Property maxTries: 3
Property version: 3.0.1
Property rollbackSupported: false
Execute action successful after 3 ms
[INFO] com.install4j.runtime.beans.screens.StartupScreen [ID 627]: command: move 1 screens, executing actions, checking condition
[INFO] cleaning up
[INFO] Finished
EDIT 3:
Below is the logs of UpdateChecker.executeScheduledUpdate(Arrays.asList("-q","-Dinstall4j.keepLog=true","-Dinstall4j.alternativeLogfile=E:/install4j-log/log2.txt"), false, null);
[INFO] logger started at Fri Sep 15 15:27:01 IST 2017
[INFO] executable name: C:\Users\Abhishek\Downloads\GreytipAstra_windows_3_0_1.exe
[INFO] install4j version: 6.1.6 (build 6459)
[INFO] Properties:
-- listing properties --
java.runtime.name=Java(TM) SE Runtime Environment
exe4j.moduleName=C:\Users\Abhishek\Downloads\GreytipAs...
sun.boot.library.path=c:\windows\temp\e4j101b.tmp_dir150546...
exe4j.semaphoreName=Local\c:_users_abhishek_downloads_gre...
java.vm.version=24.80-b11
java.vm.vendor=Oracle Corporation
java.vendor.url=http://java.oracle.com/
path.separator=;
exe4j.consoleCodepage=cp0
java.vm.name=Java HotSpot(TM) Client VM
file.encoding.pkg=sun.io
user.script=
user.country=US
sun.os.patch.level=
install4j.exeDir=C:\Users\Abhishek\Downloads\
java.vm.specification.name=Java Virtual Machine Specification
user.dir=C:\WINDOWS\TEMP\e4j101B.tmp_dir150546...
java.runtime.version=1.7.0_80-b15
java.awt.graphicsenv=sun.awt.Win32GraphicsEnvironment
java.endorsed.dirs=c:\windows\temp\e4j101b.tmp_dir150546...
os.arch=x86
java.io.tmpdir=C:\WINDOWS\TEMP\
line.separator=
java.vm.specification.vendor=Oracle Corporation
user.variant=
exe4j.tempDir=C:\WINDOWS\TEMP\e4j101B.tmp_dir150546...
os.name=Windows NT (unknown)
sun.java2d.noddraw=true
sun.jnu.encoding=Cp1252
java.library.path=C:\WINDOWS\TEMP\e4j101B.tmp_dir150546...
sun.awt.enableExtraMouseButtons=true
java.specification.name=Java Platform API Specification
java.class.version=51.0
sun.management.compiler=HotSpot Client Compiler
install4j.language=en
exe4j.isInstall4j=true
os.version=10.0
install4j.updateStorageDir=C:\WINDOWS\system32\config\systemprof...
user.home=C:\Windows\System32\config\systemprofile
user.timezone=Asia/Calcutta
java.awt.printerjob=sun.awt.windows.WPrinterJob
java.specification.version=1.7
file.encoding=Cp1252
user.name=ABHISHEK-PC$
java.class.path=C:\WINDOWS\TEMP\e4j101B.tmp_dir150546...
java.vm.specification.version=1.7
sun.arch.data.model=32
java.home=c:\windows\temp\e4j101b.tmp_dir150546...
sun.java.command=C:\Users\Abhishek\Downloads\GreytipAs...
exe4j.launchName=C:\Users\Abhishek\DOWNLO~1\GREYTI~1.EXE
java.specification.vendor=Oracle Corporation
user.language=en
awt.toolkit=sun.awt.windows.WToolkit
java.vm.info=mixed mode
exe4j.unextractedPosition=26515602
java.version=1.7.0_80
java.ext.dirs=c:\windows\temp\e4j101b.tmp_dir150546...
sun.boot.class.path=c:\windows\temp\e4j101b.tmp_dir150546...
java.vendor=Oracle Corporation
install4j.appDir=C:\Windows\Temp\
file.separator=\
java.vendor.url.bug=http://bugreport.sun.com/bugreport/
sun.cpu.endian=little
sun.io.unicode.encoding=UnicodeLittle
install4j.alternativeLogfile=E:/install4j-log/log2.txt
install4j.lockFile=C:\WINDOWS\system32\config\systemprof...
install4j.keepLog=true
sun.desktop=windows
sun.cpu.isalist=pentium_pro+mmx pentium_pro pentium+m...
[INFO] com.install4j.runtime.beans.actions.misc.RequestPrivilegesAction [ID 304]: Execute action
Property allRequested: false
Property failIfNotRootUnix: true
Property obtainIfAdminMac: false
Property obtainIfNormalWin: false
Property obtainIfNormalMac: false
Property obtainIfAdminWin: true
Property failIfNotObtainedWin: true
Property failIfNotObtainedMac: true
Property rollbackSupported: false
Execute action successful after 1 ms
[INFO] com.install4j.runtime.beans.screens.StartupScreen [ID 1]: command: move 1 screens, executing actions, checking condition
[INFO] com.install4j.runtime.beans.screens.WelcomeScreen [ID 2]: Show screen
[INFO] checking writable with maximum
[INFO] com.install4j.runtime.beans.screens.WelcomeScreen [ID 2]: thread name AWT-EventQueue-0
EDIT 4:
My code in Application:
try {
ApplicationLauncher.launchApplication("626", null, false, new ApplicationLauncher.Callback() {
public void exited(int exitValue) {
//TODO add your code here (not invoked on event dispatch thread)
log.debug("******Exited "+exitValue+"********");
log.debug("****Inside block UpdateChecker.isUpdateScheduled() "+String.valueOf(UpdateChecker.isUpdateScheduled()));
boolean updateScheduled=UpdateChecker.isUpdateScheduled();
if(updateScheduled){
log.debug("***************Running UpdateChecker.executeScheduledUpdate*******************");
UpdateChecker.executeScheduledUpdate(Arrays.asList("-q","-Dinstall4j.keepLog=true","-Dinstall4j.alternativeLogfile=E:/install4j-log/log2.txt"), false, null);
log.debug("***************Completed UpdateChecker.executeScheduledUpdate*******************");
}
}
public void prepareShutdown() {
//TODO add your code here (not invoked on event dispatch thread)
log.debug("******prepare shut down********");
}
}
);
} catch (IOException e) {
e.printStackTrace();
//TODO handle invocation failure
}
And the jstack error is as below:
EDIT 5:
Below is the jstack log:
2017-09-18 17:31:12
Full thread dump Java HotSpot(TM) Client VM (24.80-b11 mixed mode):
"TimerQueue" daemon prio=6 tid=0x17ecec00 nid=0x3708 waiting on condition [0x1861f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x03850980> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(Unknown Source)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
at java.util.concurrent.DelayQueue.take(Unknown Source)
at javax.swing.TimerQueue.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Locked ownable synchronizers:
- <0x03884b18> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
"AWT-EventQueue-0" prio=6 tid=0x17d3f000 nid=0x2670 waiting on condition [0x1841e000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x08ca4218> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(Unknown Source)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
at java.awt.EventQueue.getNextEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)
Locked ownable synchronizers:
- None
"AWT-Windows" daemon prio=6 tid=0x17d2fc00 nid=0x40c runnable [0x1838f000]
java.lang.Thread.State: RUNNABLE
at sun.awt.windows.WToolkit.eventLoop(Native Method)
at sun.awt.windows.WToolkit.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Locked ownable synchronizers:
- None
"AWT-Shutdown" prio=6 tid=0x17d2f400 nid=0xe68 in Object.wait() [0x182ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x08c79758> (a java.lang.Object)
at java.lang.Object.wait(Object.java:503)
at sun.awt.AWTAutoShutdown.run(Unknown Source)
- locked <0x08c79758> (a java.lang.Object)
at java.lang.Thread.run(Unknown Source)
Locked ownable synchronizers:
- None
"Java2D Disposer" daemon prio=10 tid=0x17d2ec00 nid=0x3d64 in Object.wait() [0x1826f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x08c797e8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(Unknown Source)
- locked <0x08c797e8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(Unknown Source)
at sun.java2d.Disposer.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Locked ownable synchronizers:
- None
"Service Thread" daemon prio=6 tid=0x1783bc00 nid=0x468 runnable [0x00000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"C1 CompilerThread0" daemon prio=10 tid=0x1783a000 nid=0x1768 waiting on condition [0x00000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Attach Listener" daemon prio=10 tid=0x17837800 nid=0x389c waiting on condition [0x00000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Signal Dispatcher" daemon prio=10 tid=0x17835400 nid=0x3258 runnable [0x00000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Finalizer" daemon prio=8 tid=0x0101a800 nid=0x4394 in Object.wait() [0x033bf000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x08ab74f8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(Unknown Source)
- locked <0x08ab74f8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(Unknown Source)
at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
Locked ownable synchronizers:
- None
"Reference Handler" daemon prio=10 tid=0x01015800 nid=0x3d50 in Object.wait() [0x0127f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x08ab7588> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:503)
at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)
- locked <0x08ab7588> (a java.lang.ref.Reference$Lock)
Locked ownable synchronizers:
- None
"main" prio=6 tid=0x010ac400 nid=0x3138 in Object.wait() [0x00798000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x038517d0> (a com.install4j.runtime.installer.controller.CommandSink)
at java.lang.Object.wait(Object.java:503)
at com.install4j.runtime.installer.controller.Controller.start(Controller.java:87)
- locked <0x038517d0> (a com.install4j.runtime.installer.controller.CommandSink)
at com.install4j.runtime.installer.Installer.runInProcess(Installer.java:59)
at com.install4j.runtime.installer.Installer.main(Installer.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:65)
at com.exe4j.runtime.WinLauncher.main(WinLauncher.java:101)
at com.install4j.runtime.launcher.WinLauncher.main(WinLauncher.java:26)
Locked ownable synchronizers:
- None
"VM Thread" prio=10 tid=0x01012400 nid=0x1bf0 runnable
"VM Periodic Task Thread" prio=10 tid=0x17856000 nid=0x2fdc waiting on condition
JNI global references: 449
EDIT 6:
Below is the log of UpdateChecker.executeScheduledUpdate(Arrays.asList("-q","-Dinstall4j.keepLog=true","-Dinstall4j.alternativeLogfile=E:/install4j-log/log2.txt"), false, null);:
[INFO] logger started at Tue Sep 19 11:20:00 IST 2017
[INFO] executable name: C:\Users\Abhishek\Downloads\GreytipAstra_windows_3_0_1.exe
[INFO] install4j version: 7.0.1 (build 7043)
[INFO] Properties:
-- listing properties --
java.runtime.name=Java(TM) SE Runtime Environment
exe4j.moduleName=C:\Users\Abhishek\Downloads\GreytipAs...
sun.boot.library.path=c:\windows\temp\e4j554d.tmp_dir150580...
exe4j.semaphoreName=Local\c:_users_abhishek_downloads_gre...
java.vm.version=24.80-b11
java.vm.vendor=Oracle Corporation
java.vendor.url=http://java.oracle.com/
path.separator=;
exe4j.consoleCodepage=cp0
java.vm.name=Java HotSpot(TM) Client VM
file.encoding.pkg=sun.io
user.script=
user.country=US
sun.os.patch.level=
install4j.exeDir=C:\Users\Abhishek\Downloads\
java.vm.specification.name=Java Virtual Machine Specification
user.dir=C:\WINDOWS\TEMP\e4j554D.tmp_dir150580...
java.runtime.version=1.7.0_80-b15
java.awt.graphicsenv=sun.awt.Win32GraphicsEnvironment
java.endorsed.dirs=c:\windows\temp\e4j554d.tmp_dir150580...
os.arch=x86
java.io.tmpdir=C:\WINDOWS\TEMP\
line.separator=
java.vm.specification.vendor=Oracle Corporation
user.variant=
exe4j.tempDir=C:\WINDOWS\TEMP\e4j554D.tmp_dir150580...
os.name=Windows NT (unknown)
sun.java2d.noddraw=true
sun.jnu.encoding=Cp1252
java.library.path=C:\WINDOWS\TEMP\e4j554D.tmp_dir150580...
sun.awt.enableExtraMouseButtons=true
java.specification.name=Java Platform API Specification
java.class.version=51.0
sun.management.compiler=HotSpot Client Compiler
install4j.language=en
exe4j.isInstall4j=true
os.version=10.0
install4j.updateStorageDir=C:\WINDOWS\system32\config\systemprof...
user.home=C:\Windows\System32\config\systemprofile
user.timezone=Asia/Calcutta
java.awt.printerjob=sun.awt.windows.WPrinterJob
java.specification.version=1.7
file.encoding=Cp1252
user.name=ABHISHEK-PC$
java.class.path=C:\WINDOWS\TEMP\e4j554D.tmp_dir150580...
java.vm.specification.version=1.7
sun.arch.data.model=32
java.home=c:\windows\temp\e4j554d.tmp_dir150580...
sun.java.command=C:\Users\Abhishek\Downloads\GreytipAs...
exe4j.launchName=C:\Users\Abhishek\DOWNLO~1\GREYTI~1.EXE
java.specification.vendor=Oracle Corporation
user.language=en
awt.toolkit=sun.awt.windows.WToolkit
java.vm.info=mixed mode
exe4j.unextractedPosition=25300990
java.version=1.7.0_80
java.ext.dirs=c:\windows\temp\e4j554d.tmp_dir150580...
sun.boot.class.path=c:\windows\temp\e4j554d.tmp_dir150580...
java.vendor=Oracle Corporation
install4j.appDir=C:\Windows\Temp\
file.separator=\
java.vendor.url.bug=http://bugreport.sun.com/bugreport/
sun.cpu.endian=little
sun.io.unicode.encoding=UnicodeLittle
install4j.alternativeLogfile=E:/install4j-log/log2.txt
install4j.lockFile=C:\WINDOWS\system32\config\systemprof...
install4j.keepLog=true
sun.desktop=windows
sun.cpu.isalist=pentium_pro+mmx pentium_pro pentium+m...
[INFO] com.install4j.runtime.beans.actions.misc.RequestPrivilegesAction [ID 304]: Execute action
Property allRequested: false
Property failIfNotObtainedMac: true
Property failIfNotObtainedWin: true
Property failIfNotRootUnix: true
Property obtainIfAdminMac: false
Property obtainIfAdminWin: true
Property obtainIfNormalMac: false
Property obtainIfNormalWin: false
Property rollbackSupported: false
Property updateInstallationDirectory: true
Execute action successful after 1 ms
[INFO] com.install4j.runtime.beans.screens.StartupScreen [ID 1]: command: move 1 screens, executing actions, checking condition
[INFO] com.install4j.runtime.beans.screens.WelcomeScreen [ID 2]: Show screen
[INFO] checking writable with maximum
I would recommend updating to install4j 7 which has a lot of new auto-update functionality which works well for services.
With install4j 7, you add a "Background update downloader" application on the Installer->Screen & Actions step. The key action in that application template is the "Schedule update installation" action.
In your service, you launch the update downloader like this:
ApplicationLauncher.launchApplication("<ID>", null, true, null);
in its own thread where <ID> is replaced with the ID of the application. This call blocks and when it returns an update installer may have been downloaded. You can check this condition with
UpdateChecker.isUpdateScheduled()
Now you only need to trigger the execution of the downloaded installer. You can either do that immediately, or at startup if you do not want to terminate the service. The code snippet you need is this:
UpdateChecker.executeScheduledUpdate(Arrays.asList("-q"), false, null);
As part of the update installation, services are shut down. Services cannot auto-restart (which GUI launchers could do if you passed true as the second parameter to the above method call) but have to be started with a "Start a service" action which you probably already have in your installer.
For more information and screen shots, see
https://www.ej-technologies.com/products/install4j/whatsnew7.html
and search for "Background auto-update".
Related
I am trying to install and execute JSS7 Stack over eclipse-EE from local Linux
machine, Currently using JBOSS version- 7.1.
Linux machine details ->
Distributor ID: RedHatEnterpriseServer
Description: Red Hat Enterprise Linux Server release 5.10 (Tikanga)
Release: 5.10
JDK installed -> java version "1.7.0_80"
SCTP installed -> lksctp-tools-1.0.6-3.el5.i386.rpm
I have imported the JSS7 project from GIT into my workspace.. JBOSS and JMX tool is connected and call is been executed ..
Result is the SCTP is not getting connected , listed below are the error logs:
*On GUI :-
L1 State -> SCTP: Disconnected M3UA: pFsm:DOWN lFsmP:DOWN
L2 state -> SCCP: Rspc: Enabled Rss: Enabled
L3 state -> TCAP+CAP: Started
*In Terminal logs :-
6520 [AWT-EventQueue-0] INFO org.mobicents.protocols.sctp.netty.NettySctpManagementImpl - Removing allocated resources: Servers=0, Associations=1
6521 [AWT-EventQueue-0] INFO org.mobicents.protocols.sctp.netty.NettyAssociationImpl - Management requested to stop Association [name=Ass_main, associationType=CLIENT, ipChannelType=SCTP, hostAddress=200.20.20.20, hostPort=9999, peerAddress=100.10.10.10, peerPort=9999, serverName=, extraHostAddress=[]]
7078 [AWT-EventQueue-0] INFO org.restcomm.protocols.ss7.m3ua.impl.M3UAManagementImpl - M3UA configuration file path /xyz/SimM3uaServer_main_m3ua1.xml
7116 [AWT-EventQueue-0] ERROR org.restcomm.protocols.ss7.m3ua.impl.M3UAManagementImpl - Error setting Assciation=Ass_main for the AspFactory=testasp while loading from XML
java.lang.Exception: No Association found for name=Ass_main
at org.mobicents.protocols.sctp.netty.NettySctpManagementImpl.getAssociation(NettySctpManagementImpl.java:991)
at org.restcomm.protocols.ss7.m3ua.impl.M3UAManagementImpl.loadActualData(M3UAManagementImpl.java:1126)
at org.restcomm.protocols.ss7.m3ua.impl.M3UAManagementImpl.loadVer2(M3UAManagementImpl.java:1166)
at org.restcomm.protocols.ss7.m3ua.impl.M3UAManagementImpl.load(M3UAManagementImpl.java:1040)
at org.restcomm.protocols.ss7.m3ua.impl.M3UAManagementImpl.start(M3UAManagementImpl.java:292)
at org.restcomm.protocols.ss7.tools.simulator.level1.M3uaMan.initM3ua(M3uaMan.java:652)
at org.restcomm.protocols.ss7.tools.simulator.level1.M3uaMan.start(M3uaMan.java:549)
at org.restcomm.protocols.ss7.tools.simulator.management.TesterHostImpl.start(TesterHostImpl.java:503)
at org.restcomm.protocols.ss7.tools.simulatorgui.TestingForm$5.actionPerformed(TestingForm.java:256)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2018)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2341)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
at java.awt.Component.processMouseEvent(Component.java:6516)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3321)
at java.awt.Component.processEvent(Component.java:6281)
at java.awt.Container.processEvent(Container.java:2229)
at java.awt.Component.dispatchEventImpl(Component.java:4872)
at java.awt.Container.dispatchEventImpl(Container.java:2287)
at java.awt.Component.dispatchEvent(Component.java:4698)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4832)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4492)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4422)
at java.awt.Container.dispatchEventImpl(Container.java:2273)
at java.awt.Window.dispatchEventImpl(Window.java:2719)
at java.awt.Component.dispatchEvent(Component.java:4698)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:747)
at java.awt.EventQueue.access$300(EventQueue.java:103)
at java.awt.EventQueue$3.run(EventQueue.java:706)
at java.awt.EventQueue$3.run(EventQueue.java:704)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:87)
at java.awt.EventQueue$4.run(EventQueue.java:720)
at java.awt.EventQueue$4.run(EventQueue.java:718)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:717)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:91)
7118 [AWT-EventQueue-0] ERROR org.restcomm.protocols.ss7.m3ua.impl.M3UAManagementImpl - Error starting the AspFactory=testasp while loading from XML
java.lang.NullPointerException
at org.restcomm.protocols.ss7.m3ua.impl.AspFactoryImpl.start(AspFactoryImpl.java:196)
at org.restcomm.protocols.ss7.m3ua.impl.M3UAManagementImpl.loadActualData(M3UAManagementImpl.java:1134)
at org.restcomm.protocols.ss7.m3ua.impl.M3UAManagementImpl.loadVer2(M3UAManagementImpl.java:1166)
at org.restcomm.protocols.ss7.m3ua.impl.M3UAManagementImpl.load(M3UAManagementImpl.java:1040)
at org.restcomm.protocols.ss7.m3ua.impl.M3UAManagementImpl.start(M3UAManagementImpl.java:292)
at org.restcomm.protocols.ss7.tools.simulator.level1.M3uaMan.initM3ua(M3uaMan.java:652)
at org.restcomm.protocols.ss7.tools.simulator.level1.M3uaMan.start(M3uaMan.java:549)
at org.restcomm.protocols.ss7.tools.simulator.management.TesterHostImpl.start(TesterHostImpl.java:503)
at org.restcomm.protocols.ss7.tools.simulatorgui.TestingForm$5.actionPerformed(TestingForm.java:256)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2018)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2341)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
at java.awt.Component.processMouseEvent(Component.java:6516)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3321)
at java.awt.Component.processEvent(Component.java:6281)
at java.awt.Container.processEvent(Container.java:2229)
at java.awt.Component.dispatchEventImpl(Component.java:4872)
at java.awt.Container.dispatchEventImpl(Container.java:2287)
at java.awt.Component.dispatchEvent(Component.java:4698)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4832)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4492)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4422)
at java.awt.Container.dispatchEventImpl(Container.java:2273)
at java.awt.Window.dispatchEventImpl(Window.java:2719)
at java.awt.Component.dispatchEvent(Component.java:4698)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:747)
at java.awt.EventQueue.access$300(EventQueue.java:103)
at java.awt.EventQueue$3.run(EventQueue.java:706)
at java.awt.EventQueue$3.run(EventQueue.java:704)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:87)
at java.awt.EventQueue$4.run(EventQueue.java:720)
at java.awt.EventQueue$4.run(EventQueue.java:718)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:717)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:91)
Please let me know , where am going wrong?.. i want the SCTP to get connected..
Getting these errors:
2018-01-22 18:00:59,797 [ServerService Thread Pool -- 79] ERROR org.quartz.ee.servlet.QuartzInitializerListener - Quartz Scheduler failed to initialize: org.quartz.SchedulerException: SchedulerPlugin class 'org.quartz.plugins.xml.XMLSchedulingDataProcessorPlugin' could not be instantiated. [See nested exception: java.lang.ClassNotFoundException: Unable to load class org.quartz.plugins.xml.XMLSchedulingDataProcessorPlugin by any known loaders.]
2018-01-22 18:00:59,797 [ServerService Thread Pool -- 79] ERROR stderr - org.quartz.SchedulerException: SchedulerPlugin class 'org.quartz.plugins.xml.XMLSchedulingDataProcessorPlugin' could not be instantiated. [See nested exception: java.lang.ClassNotFoundException: Unable to load class org.quartz.plugins.xml.XMLSchedulingDataProcessorPlugin by any known loaders.]
2018-01-22 18:00:59,805 [ServerService Thread Pool -- 79] ERROR stderr - Caused by: java.lang.ClassNotFoundException: Unable to load class org.quartz.plugins.xml.XMLSchedulingDataProcessorPlugin by any known loaders.
2018-01-22 18:00:59,805 [ServerService Thread Pool -- 79] ERROR stderr - Caused by: java.lang.LinkageError: Failed to link org/quartz/plugins/xml/XMLSchedulingDataProcessorPlugin (Module "deployment.Reports.ear:main" from Service Module Loader)
2018-01-22 18:00:59,806 [ServerService Thread Pool -- 79] ERROR stderr - Caused by: java.lang.NoClassDefFoundError: org/quartz/jobs/FileScanListener
2018-01-22 18:00:59,807 [ServerService Thread Pool -- 79] ERROR stderr - Caused by: java.lang.ClassNotFoundException: org.quartz.jobs.FileScanListener from [Module "deployment.Reports.ear:main" from Service Module Loader]
quartz.properties file:
org.quartz.scheduler.instanceName = Scheduler
org.quartz.scheduler.instanceId = AUTO
org.quartz.scheduler.skipUpdateCheck = true
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 10
org.quartz.threadPool.threadPriority = 5
org.quartz.jobStore.misfireThreshold = 60000
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.oracle.OracleDelegate
org.quartz.jobStore.useProperties = false
org.quartz.jobStore.dataSource = quartzDataSource
org.quartz.jobStore.tablePrefix = QUARTZ.QRTZ_
#org.quartz.jobStore.tablePrefix = QRTZ_
org.quartz.jobStore.isClustered = true
org.quartz.jobStore.clusterCheckinInterval = 20000
org.quartz.dataSource.quartzDataSource.jndiURL=java:QuartzDataSource
org.quartz.dataSource.quartzDataSource.java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
org.quartz.plugin.jobInitializer.class = org.quartz.plugins.xml.XMLSchedulingDataProcessorPlugin
org.quartz.plugin.jobInitializer.fileNames = quartz-jobs-xml
org.quartz.plugin.jobInitializer.failOnFileNotFound = true
org.quartz.plugin.jobInitializer.scanInterval = 0
org.quartz.plugin.jobInitializer.wrapInUserTransaction =false
My quartz-jobs.xml file is in the same directory as my quartz.properties.xml file. I tried playing around with that path a little. But I think the errors are just saying it can't instantiate my jobs.
I needed to add a new quartz library to my build to include the XMLSchedulingDataProcessorPlugin that I reference in my quartz.properties.
In my root build.gradle I add this to my ext.libraries section:
quartz_jobs: 'org.quartz-scheduler:quartz-jobs:2.3.0'
Then in my web services build.gradle where I use quartz, I add the library to my providedCompile section:
libraries.quartz-jobs
I am using EMR services from Amazon Web Services and am attempting to run a count query on an external table that I've built. The data for the table is stored in mongodb and the table is an external table in Hive. The query I'm trying to run is
select user_id, count (*) from myTable group by user_id;
I can query select * from myTable but I cannot do any other queries. When I try to I get this error:
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 container FAILED -1 0 0 -1 0 0
Reducer 2 container KILLED 1 0 0 1 0 0
----------------------------------------------------------------------------------------------
VERTICES: 00/02 [>>--------------------------] 0% ELAPSED TIME: 0.03 s
----------------------------------------------------------------------------------------------
Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1476467351971_0008_2_00, diagnostics=[Vertex vertex_1476467351971_0008_2_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator
at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:70)
at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:89)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$1.run(RootInputInitializerManager.java:151)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$1.run(RootInputInitializerManager.java:148)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.dag.app.dag.RootInputInitializerManager.createInitializer(RootInputInitializerManager.java:148)
at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInputInitializers(RootInputInitializerManager.java:121)
at org.apache.tez.dag.app.dag.impl.VertexImpl.setupInputInitializerManager(VertexImpl.java:3986)
at org.apache.tez.dag.app.dag.impl.VertexImpl.access$3100(VertexImpl.java:204)
at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.handleInitEvent(VertexImpl.java:2818)
at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:2765)
at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:2747)
at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:59)
at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1888)
at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:203)
at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2242)
at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2228)
at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
... 25 more
Caused by: java.lang.RuntimeException: Failed to load plan: hdfs://ip-172-31-33-88.ec2.internal:8020/tmp/hive/hadoop/8c1eca9a-84ba-4d79-b39c-e633f1c6a646/hive_2016-10-14_18-57-38_728_4862537458701624850-1/hadoop/_tez_scratch_dir/157c0e27-0bfa-4acf-b0e9-b2fed595a8de/map.xml: org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find class: com.mongodb.hadoop.hive.input.HiveMongoInputFormat
Serialization trace:
inputFileFormatClass (org.apache.hadoop.hive.ql.plan.PartitionDesc)
aliasToPartnInfo (org.apache.hadoop.hive.ql.plan.MapWork)
at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:451)
at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:298)
at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.<init>(HiveSplitGenerator.java:131)
... 30 more
Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find class: com.mongodb.hadoop.hive.input.HiveMongoInputFormat
Serialization trace:
inputFileFormatClass (org.apache.hadoop.hive.ql.plan.PartitionDesc)
aliasToPartnInfo (org.apache.hadoop.hive.ql.plan.MapWork)
at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:156)
at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClass(SerializationUtilities.java:180)
at org.apache.hive.com.esotericsoftware.kryo.serializers.DefaultSerializers$ClassSerializer.read(DefaultSerializers.java:326)
at org.apache.hive.com.esotericsoftware.kryo.serializers.DefaultSerializers$ClassSerializer.read(DefaultSerializers.java:314)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:759)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObjectOrNull(SerializationUtilities.java:198)
at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClassAndObject(SerializationUtilities.java:175)
at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:161)
at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:213)
at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:686)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:205)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializeObjectByKryo(SerializationUtilities.java:583)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializePlan(SerializationUtilities.java:492)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializePlan(SerializationUtilities.java:469)
at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:411)
... 32 more
Caused by: java.lang.ClassNotFoundException: com.mongodb.hadoop.hive.input.HiveMongoInputFormat
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154)
... 55 more
]
Vertex killed, vertexName=Reducer 2, vertexId=vertex_1476467351971_0008_2_01, diagnostics=[Vertex received Kill in NEW state., Vertex vertex_1476467351971_0008_2_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1476467351971_0008_2_00, diagnostics=[Vertex vertex_1476467351971_0008_2_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator
at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:70)
at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:89)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$1.run(RootInputInitializerManager.java:151)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$1.run(RootInputInitializerManager.java:148)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.dag.app.dag.RootInputInitializerManager.createInitializer(RootInputInitializerManager.java:148)
at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInputInitializers(RootInputInitializerManager.java:121)
at org.apache.tez.dag.app.dag.impl.VertexImpl.setupInputInitializerManager(VertexImpl.java:3986)
at org.apache.tez.dag.app.dag.impl.VertexImpl.access$3100(VertexImpl.java:204)
at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.handleInitEvent(VertexImpl.java:2818)
at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:2765)
at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:2747)
at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:59)
at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1888)
at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:203)
at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2242)
at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2228)
at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
... 25 more
Caused by: java.lang.RuntimeException: Failed to load plan: hdfs://ip-172-31-33-88.ec2.internal:8020/tmp/hive/hadoop/8c1eca9a-84ba-4d79-b39c-e633f1c6a646/hive_2016-10-14_18-57-38_728_4862537458701624850-1/hadoop/_tez_scratch_dir/157c0e27-0bfa-4acf-b0e9-b2fed595a8de/map.xml: org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find class: com.mongodb.hadoop.hive.input.HiveMongoInputFormat
Serialization trace:
inputFileFormatClass (org.apache.hadoop.hive.ql.plan.PartitionDesc)
aliasToPartnInfo (org.apache.hadoop.hive.ql.plan.MapWork)
at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:451)
at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:298)
at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.<init>(HiveSplitGenerator.java:131)
... 30 more
Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find class: com.mongodb.hadoop.hive.input.HiveMongoInputFormat
Serialization trace:
inputFileFormatClass (org.apache.hadoop.hive.ql.plan.PartitionDesc)
aliasToPartnInfo (org.apache.hadoop.hive.ql.plan.MapWork)
at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:156)
at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClass(SerializationUtilities.java:180)
at org.apache.hive.com.esotericsoftware.kryo.serializers.DefaultSerializers$ClassSerializer.read(DefaultSerializers.java:326)
at org.apache.hive.com.esotericsoftware.kryo.serializers.DefaultSerializers$ClassSerializer.read(DefaultSerializers.java:314)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:759)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObjectOrNull(SerializationUtilities.java:198)
at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClassAndObject(SerializationUtilities.java:175)
at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:161)
at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:213)
at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:686)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:205)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializeObjectByKryo(SerializationUtilities.java:583)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializePlan(SerializationUtilities.java:492)
at org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializePlan(SerializationUtilities.java:469)
at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:411)
... 32 more
Caused by: java.lang.ClassNotFoundException: com.mongodb.hadoop.hive.input.HiveMongoInputFormat
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154)
... 55 more
]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1476467351971_0008_2_01, diagnostics=[Vertex received Kill in NEW state., Vertex vertex_1476467351971_0008_2_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1
hive>
I am using a single node EMR cluster from AWS with the "Core Hadoop" configuration, but I had the same errors using a three node cluster. I have already added the Mongo Hadoop Core, Mongo Hadoop Hive, Mongo Java driver to the master node.. These are the jars I'm using:
mongo-hadoop-hive-2.0.1.jar
mongo-java-driver-3.3.0.jar
remotecontent?filepath=org%2Fmongodb%2Fmongo-hadoop%2Fmongo-hadoop-core%2F2.0.1%2Fmongo-hadoop-core-2.0.1.jar
These jars were downloaded from Maven.
A similar question had previously been asked on StackOverflow and was resolved by adding the src for Apache Tez 0.8.4 and building it. Apache Tez 0.8.4 is already on the cluster though, at least according to the EMR configuration we chose.
We are using Hadoop 2.7.2 and Hive 2.1.0.
I am using Flume and trying to publish the output "/usr/bin/vmstat 1" to a kafka topic.
But I get exceptions as mentioned in the stack trace. Please help me to find solution to the problem.
==============================================================================
Flume Config
tier1.sources = source1
tier1.channels = channel1
tier1.sinks = sink1
tier1.sources.source1.type = exec
tier1.sources.source1.command = /usr/bin/vmstat 1
tier1.sources.source1.channels = channel1
tier1.channels.channel1.type = memory
tier1.channels.channel1.capacity = 10000
tier1.channels.channel1.transactionCapacity = 1000
tier1.sinks.sink1.type = org.apache.flume.sink.kafka.KafkaSink
tier1.sinks.sink1.topic = sink1
tier1.sinks.sink1.brokerList = localhost:9092
tier1.sinks.sink1.channel = channel1
tier1.sinks.sink1.batchSize = 20
==============================================================================
Exception
2015-09-23 10:39:36,936 (lifecycleSupervisor-1-3) [INFO - org.apache.flume.source.ExecSource.start(ExecSource.java:169)] Exec source starting with command:/usr/bin/vmstat 1
2015-09-23 10:39:36,944 (lifecycleSupervisor-1-1) [ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:253)] Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor#12c50438 counterGroup:{ name:null counters:{} } } - Exception follows.
java.lang.NoClassDefFoundError: scala/collection/IndexedSeqOptimized
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:643)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
at org.apache.flume.sink.kafka.KafkaSink.start(KafkaSink.java:163)
at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)
Caused by: java.lang.ClassNotFoundException: scala.collection.IndexedSeqOptimized
at java.net.URLClassLoader$1.run(URLClassLoader.java:214)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
... 23 more
Caused by: java.util.zip.ZipException: invalid LOC header (bad signature)
at java.util.zip.ZipFile.read(Native Method)
at java.util.zip.ZipFile.access$1300(ZipFile.java:46)
at java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:485)
at java.util.zip.ZipFile$2.fill(ZipFile.java:268)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at sun.misc.Resource.getBytes(Resource.java:124)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:273)
at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
... 28 more
2015-09-23 10:39:36,954 (lifecycleSupervisor-1-3) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.register(MonitoredCounterGroup.java:120)] Monitored counter group for type: SOURCE, name: source1: Successfully registered new MBean.
2015-09-23 10:39:36,977 (lifecycleSupervisor-1-3) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:96)] Component type: SOURCE, name: source1 started
2015-09-23 10:39:36,977 (lifecycleSupervisor-1-3) [DEBUG - org.apache.flume.source.ExecSource.start(ExecSource.java:187)] Exec source started
2015-09-23 10:39:36,977 (lifecycleSupervisor-1-1) [ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:264)] Unsuccessful attempt to shutdown component: {} due to missing dependencies. Please shutdown the agentor disable this component, or the agent will bein an undefined state.
java.lang.NullPointerException
at org.apache.flume.sink.kafka.KafkaSink.stop(KafkaSink.java:171)
at org.apache.flume.sink.DefaultSinkProcessor.stop(DefaultSinkProcessor.java:53)
at org.apache.flume.SinkRunner.stop(SinkRunner.java:115)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:259)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)
2015-09-23 10:39:39,978 (lifecycleSupervisor-1-3) [INFO - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:233)] Component SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor#12c50438 counterGroup:{ name:null counters:{} } } is in error state, and Flume will notattempt to change its state
^C2015-09-23 10:39:42,250 (pool-3-thread-1) [INFO - org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:376)] Command [/usr/bin/vmstat 1] exited with 130
We have an enterprise application running on WL 12c using eclipselink as the deafult JPA provider. MS SQL server 2008 is the backend database
There is an entity on which i have used the following definition . the entity doesnt have any relationship with other entities
#Entity
#Table(name="TCALCNX")
#Cache(isolation=CacheIsolationType.ISOLATED, expiry=0, alwaysRefresh=true)
#OptimisticLocking(type=OptimisticLockingType.SELECTED_COLUMNS, selectedColumns= {#Column(name="REC_UDT_TS")})
As you can see there is an optimistic lock on a selected field which means all users can read the entity but while
updating the entity, a check is made to ensure the object hasnt changed since it was read. If it was we get a optimisticlock exception.
Whats happening in my case is when i run the application with optimisticlock on,i rarely get optmisticlock exception but high number of deadlock exceptions.
If i run the app with optimistic lock turned off am not getting any deadlock exceptions.
Internal Exception: java.sql.SQLTransactionRollbackException: [FMWGEN][SQLServer
JDBC Driver][SQLServer]Transaction (Process ID 338) was deadlocked on lock
| communication buffer resources with another process and has been chosen as the
deadlock victim. Rerun the transaction.
Error Code: 1205
what has the optimistic lock has to do with deadlocks
Ok assume that there are two transactions that read the object and modififed it . Both the transactions are updating at the same time .
if this is the case then i shud be getting the deadlock exception with the optimisticlock turned off. But am not getting
any deadlock exception when optimisticlock is turned off.
when i turn on the optimistic lock, looks like there is a lot of concurrency causing deadlock
Is it like with optimsticlock turned on , the updates are not ordered thus causing lot of concurrency
Is there a way to find out if the transactions are ordered or not in ecliselink
the backend database isolation level is read_committed.
Please dont hesitate to post any comment, there could be a clue from anyanswer from which i can get the answer
Entity
#Entity
#Table(name="TCALCNX")
#Cache(isolation=CacheIsolationType.ISOLATED, expiry=0, alwaysRefresh=true)
#OptimisticLocking(type=OptimisticLockingType.SELECTED_COLUMNS,selectedColumns= {#Column(name = "REC_UDT_TS")} )
public class CallContentEntity implements Serializable {
public static final String RECORDSTATUS_ADDED = "01";
public static final String RECORDSTATUS_UPDATED = "02";
public static final String RECORDSTATUS_MARKED_FOR_DELETION = "15";
#Id
#Column(name="CAL_ID_NR")
private String CallID;
#Column(name="CAL_CNX_B")
private byte[] CallContent;
#Column(name="REC_UDT_TS")
private Timestamp UpdateTimestamp;
#Column(name="REC_STS_CD", length=2)
private String RecordStatusCode;
#Column(name="SRC_DAT_CTR_NR", length=2)
private String SourceDataCnterNum;
#Column(name="REC_TYP_VER_NR", length=4)
private String VersionNumber;
#Column(name="REC_SEQ_NR")
private int SequenceNumber;
private static final long serialVersionUID = 1L;
public CallContentEntity() {
super();
}
public CallContentEntity(String callID) {
super();
LookUpValues lkup= LookUpValues.getInstance();
setCallID(callID);
setCallContent(null);
setRecordStatusCode(RECORDSTATUS_ADDED);
setUpdateTimestamp(new java.sql.Timestamp(Calendar.getInstance().getTimeInMillis()));
String sdatacntr= Integer.toString(lkup.getEnvironmentValue());
setSourceDataCnterNum(sdatacntr);
setVersionNumber(lkup.getLookUpValue("callcontentversion"));
setSequenceNumber(1);
}
public String getCallID() {
return this.CallID;
}
public void setCallID(String CallID) {
this.CallID = CallID;
}
public byte[] getCallContent() {
return this.CallContent;
}
public void setCallContent(byte[] CallContent) {
this.CallContent = CallContent;
}
public Timestamp getUpdateTimestamp() {
return this.UpdateTimestamp;
}
public void setUpdateTimestamp(Timestamp UpdateTimestamp) {
this.UpdateTimestamp = UpdateTimestamp;
}
public String getRecordStatusCode() {
return this.RecordStatusCode;
}
public void setRecordStatusCode(String RecordStatusCode) {
this.RecordStatusCode = RecordStatusCode;
}
public String getSourceDataCnterNum() {
return SourceDataCnterNum;
}
public void setSourceDataCnterNum(String sourceDataCnterNum) {
SourceDataCnterNum = sourceDataCnterNum;
}
public String getVersionNumber() {
return VersionNumber;
}
public void setVersionNumber(String versionNumber) {
VersionNumber = versionNumber;
}
public int getSequenceNumber() {
return SequenceNumber;
}
public void setSequenceNumber(int sequenceNumber) {
SequenceNumber = sequenceNumber;
}
Exception
Exception stack:
Local Exception Stack:
Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.3.2.v20111125-r10461): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLTransactionRollbackException: [FMWGEN][SQLServer JDBC Driver][SQLServer]Transaction (Process ID 338) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
Error Code: 1205
Call: SELECT CAL_ID_NR, CAL_CNX_B, REC_STS_CD, REC_SEQ_NR, SRC_DAT_CTR_NR, REC_UDT_TS, REC_TYP_VER_NR FROM TCALCNX WHERE (CAL_ID_NR = ?)
bind => [P4F22420140806182408001244]
Query: ReadObjectQuery(name="readObject" referenceClass=CallContentEntity sql="SELECT CAL_ID_NR, CAL_CNX_B, REC_STS_CD, REC_SEQ_NR, SRC_DAT_CTR_NR, REC_UDT_TS, REC_TYP_VER_NR FROM TCALCNX WHERE (CAL_ID_NR = ?)")
at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:333)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:644)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:535)
at org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:1717)
at org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:566)
at org.eclipse.persistence.sessions.server.ClientSession.executeCall(ClientSession.java:207)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:207)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:193)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.selectOneRow(DatasourceCallQueryMechanism.java:666)
at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRowFromTable(ExpressionQueryMechanism.java:2656)
at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRow(ExpressionQueryMechanism.java:2627)
at org.eclipse.persistence.queries.ReadObjectQuery.executeObjectLevelReadQuery(ReadObjectQuery.java:450)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1081)
at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:844)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1040)
at org.eclipse.persistence.queries.ReadObjectQuery.execute(ReadObjectQuery.java:418)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:1128)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2871)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1516)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1498)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1449)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.executeQuery(EntityManagerImpl.java:820)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.findInternal(EntityManagerImpl.java:760)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityManagerImpl.java:653)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityManagerImpl.java:532)
at sun.reflect.GeneratedMethodAccessor131.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at weblogic.persistence.BasePersistenceContextProxyImpl.invoke(BasePersistenceContextProxyImpl.java:111)
at weblogic.persistence.TransactionalEntityManagerProxyImpl.invoke(TransactionalEntityManagerProxyImpl.java:82)
at weblogic.persistence.BasePersistenceContextProxyImpl.invoke(BasePersistenceContextProxyImpl.java:92)
at $Proxy99.find(Unknown Source)
at com.ups.ivr.ins.ejb.CallContentSession.Lookup(CallContentSession.java:207)
at com.ups.ivr.ins.ejb.CallContentSession_95sf62_CallContentSessionLocalImpl.__WL_invoke(Unknown Source)
at weblogic.ejb.container.internal.SessionLocalMethodInvoker.invoke(SessionLocalMethodInvoker.java:31)
at com.ups.ivr.ins.ejb.CallContentSession_95sf62_CallContentSessionLocalImpl.Lookup(Unknown Source)
at com.ups.ivr.ins.ejb.CallManager.getCurrentCallContent(CallManager.java:402)
at com.ups.ivr.ins.ejb.CallManager_11k7mo_CallManagerLocalImpl.__WL_invoke(Unknown Source)
at weblogic.ejb.container.internal.SessionLocalMethodInvoker.invoke(SessionLocalMethodInvoker.java:31)
at com.ups.ivr.ins.ejb.CallManager_11k7mo_CallManagerLocalImpl.getCurrentCallContent(Unknown Source)
at com.ups.ivr.ins.ejb.LookupSessionBean.Processdata(LookupSessionBean.java:184)
at com.ups.ivr.ins.ejb.LookupSessionBean_1i73wg_LookupSessionBeanLocalImpl.__WL_invoke(Unknown Source)
at weblogic.ejb.container.internal.SessionLocalMethodInvoker.invoke(SessionLocalMethodInvoker.java:31)
at com.ups.ivr.ins.ejb.LookupSessionBean_1i73wg_LookupSessionBeanLocalImpl.Processdata(Unknown Source)
at com.ups.ivr.ins.mwproxy.ClientRequestProcessor.doPost(ClientRequestProcessor.java:140)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:751)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:844)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:242)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:216)
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:132)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:338)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:221)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3292)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3262)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.provider.WlsSubjectHandle.run(WlsSubjectHandle.java:57)
at weblogic.servlet.internal.WebAppServletContext.doSecuredExecute(WebAppServletContext.java:2171)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2097)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2075)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1514)
at weblogic.servlet.provider.ContainerSupportProviderImpl$WlsRequestExecutor.run(ContainerSupportProviderImpl.java:254)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Caused by: java.sql.SQLTransactionRollbackException: [FMWGEN][SQLServer JDBC Driver][SQLServer]Transaction (Process ID 338) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at weblogic.jdbc.sqlserverbase.ddb_.b(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddb_.a(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddb9.b(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddb9.a(Unknown Source)
at weblogic.jdbc.sqlserver.tds.ddr.v(Unknown Source)
at weblogic.jdbc.sqlserver.tds.ddr.a(Unknown Source)
at weblogic.jdbc.sqlserver.tds.ddq.a(Unknown Source)
at weblogic.jdbc.sqlserver.tds.ddr.a(Unknown Source)
at weblogic.jdbc.sqlserver.tds.ddr.a(Unknown Source)
at weblogic.jdbc.sqlserver.ddh.a(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddcq.k(Unknown Source)
at weblogic.jdbc.sqlserverbase.dddm.next(Unknown Source)
at weblogic.jdbc.wrapper.ResultSet_weblogic_jdbc_sqlserverbase_dddn.next(Unknown Source)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.processResultSet(DatabaseAccessor.java:699)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:621)
... 62 more
UPDATE EXCEPTION
<2014-08-06> [P4F21420140806182553001248] <12:27:24,293> {ERROR} CallContentSession - CallContent Update Exception.
Failed to write Lookup content entry:(P4F21420140806182553001248)
Exception stack:
weblogic.transaction.RollbackException: Unexpected exception in beforeCompletion: sync=org.eclipse.persistence.transaction.JTASynchronizationListener#1c1f9da7
Internal Exception: java.sql.SQLTransactionRollbackException: [FMWGEN][SQLServer JDBC Driver][SQLServer]Transaction (Process ID 338) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
Error Code: 1205
Call: UPDATE TCALCNX SET CAL_CNX_B = ?, REC_STS_CD = ?, REC_UDT_TS = ? WHERE ((CAL_ID_NR = ?) AND (REC_UDT_TS = ?))
bind => [[B#1c1f9699, 02, 2014-08-06 12:27:22.645, P4F21420140806182553001248, 2014-08-06 12:27:21.287]
Query: UpdateObjectQuery(CALL ID :P4F21420140806182553001248
REC_UDT_TS :2014-08-06 12:27:22.645
REC_STS_CD:02
SRC_DT_CNTR:10
VERSION_NUM :001
REC_SEQ_NR:1)
at weblogic.transaction.internal.TransactionImpl.throwRollbackException(TransactionImpl.java:1884)
at weblogic.transaction.internal.ServerTransactionImpl.internalCommit(ServerTransactionImpl.java:376)
at weblogic.transaction.internal.ServerTransactionImpl.commit(ServerTransactionImpl.java:268)
at weblogic.transaction.internal.TransactionManagerImpl.commit(TransactionManagerImpl.java:308)
at weblogic.transaction.internal.TransactionManagerImpl.commit(TransactionManagerImpl.java:302)
at com.ups.ivr.ins.ejb.CallContentSession.Update(CallContentSession.java:276)
at com.ups.ivr.ins.ejb.CallContentSession_95sf62_CallContentSessionLocalImpl.__WL_invoke(Unknown Source)
at weblogic.ejb.container.internal.SessionLocalMethodInvoker.invoke(SessionLocalMethodInvoker.java:31)
at com.ups.ivr.ins.ejb.CallContentSession_95sf62_CallContentSessionLocalImpl.Update(Unknown Source)
at com.ups.ivr.ins.ejb.CallManager.saveCall(CallManager.java:130)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.ups.ivr.ins.pojo.concurrent.Task$1.run(Task.java:60)
at com.ups.ivr.ins.pojo.concurrent.Task.run(Task.java:127)
at weblogic.work.j2ee.J2EEWorkManager$WorkWithListener.run(J2EEWorkManager.java:184)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Caused by: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.3.2.v20111125-r10461): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLTransactionRollbackException: [FMWGEN][SQLServer JDBC Driver][SQLServer]Transaction (Process ID 338) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
Error Code: 1205
Call: UPDATE TCALCNX SET CAL_CNX_B = ?, REC_STS_CD = ?, REC_UDT_TS = ? WHERE ((CAL_ID_NR = ?) AND (REC_UDT_TS = ?))
bind => [[B#1c1f9699, 02, 2014-08-06 12:27:22.645, P4F21420140806182553001248, 2014-08-06 12:27:21.287]
Query: UpdateObjectQuery(CALL ID :P4F21420140806182553001248
REC_UDT_TS :2014-08-06 12:27:22.645
REC_STS_CD:02
SRC_DT_CNTR:10
VERSION_NUM :001
REC_SEQ_NR:1)
at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:324)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:840)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeNoSelect(DatabaseAccessor.java:906)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:592)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:535)
at org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:1717)
at org.eclipse.persistence.sessions.server.ClientSession.executeCall(ClientSession.java:253)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:207)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:193)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.updateObject(DatasourceCallQueryMechanism.java:749)
at org.eclipse.persistence.internal.queries.StatementQueryMechanism.updateObject(StatementQueryMechanism.java:432)
at org.eclipse.persistence.internal.queries.DatabaseQueryMechanism.updateObjectForWriteWithChangeSet(DatabaseQueryMechanism.java:1042)
at org.eclipse.persistence.queries.UpdateObjectQuery.executeCommitWithChangeSet(UpdateObjectQuery.java:84)
at org.eclipse.persistence.internal.queries.DatabaseQueryMechanism.executeWriteWithChangeSet(DatabaseQueryMechanism.java:287)
at org.eclipse.persistence.queries.WriteObjectQuery.executeDatabaseQuery(WriteObjectQuery.java:58)
at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:844)
at org.eclipse.persistence.queries.DatabaseQuery.executeInUnitOfWork(DatabaseQuery.java:743)
at org.eclipse.persistence.queries.ObjectLevelModifyQuery.executeInUnitOfWorkObjectLevelModifyQuery(ObjectLevelModifyQuery.java:108)
at org.eclipse.persistence.queries.ObjectLevelModifyQuery.executeInUnitOfWork(ObjectLevelModifyQuery.java:85)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2871)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1516)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1498)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1449)
at org.eclipse.persistence.internal.sessions.CommitManager.commitChangedObjectsForClassWithChangeSet(CommitManager.java:265)
at org.eclipse.persistence.internal.sessions.CommitManager.commitAllObjectsWithChangeSet(CommitManager.java:128)
at org.eclipse.persistence.internal.sessions.AbstractSession.writeAllObjectsWithChangeSet(AbstractSession.java:3799)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabase(UnitOfWorkImpl.java:1415)
at org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.commitToDatabase(RepeatableWriteUnitOfWork.java:636)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabaseWithChangeSet(UnitOfWorkImpl.java:1505)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.issueSQLbeforeCompletion(UnitOfWorkImpl.java:3143)
at org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.issueSQLbeforeCompletion(RepeatableWriteUnitOfWork.java:346)
at org.eclipse.persistence.transaction.AbstractSynchronizationListener.beforeCompletion(AbstractSynchronizationListener.java:157)
at org.eclipse.persistence.transaction.JTASynchronizationListener.beforeCompletion(JTASynchronizationListener.java:68)
at weblogic.transaction.internal.ServerSCInfo.doBeforeCompletion(ServerSCInfo.java:1259)
at weblogic.transaction.internal.ServerSCInfo.callBeforeCompletions(ServerSCInfo.java:1234)
at weblogic.transaction.internal.ServerSCInfo.startPrePrepareAndChain(ServerSCInfo.java:123)
at weblogic.transaction.internal.ServerTransactionImpl.localPrePrepareAndChain(ServerTransactionImpl.java:1355)
at weblogic.transaction.internal.ServerTransactionImpl.globalPrePrepare(ServerTransactionImpl.java:2172)
at weblogic.transaction.internal.ServerTransactionImpl.internalCommit(ServerTransactionImpl.java:300)
at weblogic.transaction.internal.ServerTransactionImpl.commit(ServerTransactionImpl.java:267)
at weblogic.transaction.internal.TransactionManagerImpl.commit(TransactionManagerImpl.java:307)
at weblogic.transaction.internal.TransactionManagerImpl.commit(TransactionManagerImpl.java:301)
at com.ups.ivr.ins.ejb.CallContentSession.Update(CallContentSession.java:276)
at com.ups.ivr.ins.ejb.CallContentSession_95sf62_CallContentSessionLocalImpl.__WL_invoke(Unknown Source)
at weblogic.ejb.container.internal.SessionLocalMethodInvoker.invoke(SessionLocalMethodInvoker.java:31)
at com.ups.ivr.ins.ejb.CallContentSession_95sf62_CallContentSessionLocalImpl.Update(Unknown Source)
at com.ups.ivr.ins.ejb.CallManager.saveCall(CallManager.java:130)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.ups.ivr.ins.pojo.concurrent.Task$1.run(Task.java:60)
at com.ups.ivr.ins.pojo.concurrent.Task.run(Task.java:126)
... 3 more
Caused by: java.sql.SQLTransactionRollbackException: [FMWGEN][SQLServer JDBC Driver][SQLServer]Transaction (Process ID 338) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at weblogic.jdbc.sqlserverbase.ddb_.b(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddb_.a(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddb9.b(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddb9.a(Unknown Source)
at weblogic.jdbc.sqlserver.tds.ddr.v(Unknown Source)
at weblogic.jdbc.sqlserver.tds.ddr.a(Unknown Source)
at weblogic.jdbc.sqlserver.tds.ddq.a(Unknown Source)
at weblogic.jdbc.sqlserver.tds.ddr.a(Unknown Source)
at weblogic.jdbc.sqlserver.ddj.m(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddel.e(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddel.a(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddde.a(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddel.v(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddel.x(Unknown Source)
at weblogic.jdbc.sqlserverbase.ddde.executeUpdate(Unknown Source)
at weblogic.jdbc.wrapper.PreparedStatement.executeUpdate(PreparedStatement.java:167)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:831)
... 54 more
I'm not sure what this has to do with Optimistic locking (it likely does not). You should reduce what you are doing within your transactions so that any locks they obtain are held for shorter periods, such as only calling flush when it is absolutely required. Flush causes statements to be issued to the DB which will obtain locks earlier than might be needed, and will be held until the transaction fully commits, making it easier for transactions to conflict.
Optimistic locking in EclipseLink does nothing special other than add the changed field value to the where clause when modifying the corresponding entity. This does not affect the ordering of statements. It could be that your application's handling of optimistic lock exceptions is causing a delay or extra concurrent load if it retries the same operations over and over on a contentious row.
Optimistic Locking is a strategy where you read a record, take note of a version number (other methods to do this involve dates, timestamps or checksums/hashes) and check that the version hasn't changed before you write the record back. When you write the record back you filter the update on the version to make sure it's atomic.
When using optimistic locking we still have a deadlock situation which is not covered by the lock reordering:
Tx1 and Tx2 transactions executing in parallel on two nodes N1 and N2 and writing the keys {a,b}
consistentHash(a) = {N3} and consistentHash(b) = {N4}
with some right timing, during prepare time it is possible for these two transactions to deadlock:
Tx1 lock acquired on "a" # N3
Tx2 lock acquired on "b" # N4
Tx1 cannot progress acquiring lock on "b" # N4, that lock is acquired by Tx2
Tx2 cannot acquire lock on "a" # N3 as that lock is held by Tx1
Tx1 and Tx2 are waiting for each other ⇒ deadlock
A fast solution is to change Optimistic Lock to Pessimistic Lock in order to handle the situations, else i suggest you this link.