Unity Error: Unable to convert classes into dex format - unity3d

Question

I have created an Android Unity plugin (.aar file) which provides some custom positioning data to my Unity game. In my Unity script, I use,
var x = customClass.CallStatic<float>("getHeadX");
In order to get some location data. This method is called per frame to get the updated data (polling method) which makes it inefficient. Instead, I decided to call a C# method in my Unity script from my java code (plugin side) when the updated data is ready. To do this, in my java plugin, I wrote,
import com.unity3d.player.UnityPlayer;
...
UnityPlayer.UnitySendMessage("Manager", // gameObject name
"PluginCallback", // this is a callback in C#
"Hello from android plugin"); // msg which is not needed actually
However, the compiler complained that the package com.unity3d.player.UnityPlayer does not exist. So I copied classes.jar file from
C:\Program Files
(x86)\Unity\Editor\Data\PlaybackEngines\AndroidPlayer\Variations\mono\Release\Classes
\classes.jar
into the 'libs' folder of my android plugin's project. I built it successfully and copied the generated .aar file (mylibrary-release.aar) into Assets\Plugins\Android folder of my Unity project.
When I build the Unity project (Using 'Internal' build system), it gives me this error:
IOException: Failed to Move File / Directory from
'Temp/StagingArea\android-libraries\mylibrary-release\classes.jar' to
'Temp/StagingArea\android-libraries\mylibrary-release\libs\classes.jar'.
UnityEditor.Android.PostProcessor.Tasks.ProcessAAR.Execute
...
This error happens because the classes.jar dependency has name conflict with classes.jar (made by unity out of my plugin). So I changed the dependency name to unity_classes.jar and this resolved the issue but now I'm getting a new error when building my unity application:
CommandInvokationFailure: Unable to convert classes into dex format.
C:/Program Files/Java/jdk1.8.0_102\bin\java.exe -Xmx2048M
Dcom.android.sdkmanager.toolsdir="C:/Users/kamran.shamloo/AppData/Local/Android/Sdk\tools"
-Dfile.encoding=UTF8 -jar "C:\Program Files (x86)\Unity\Editor\Data\PlaybackEngines\AndroidPlayer/Tools\sdktools.jar"
stderr[ Uncaught translation error:
java.lang.IllegalArgumentException: already added:
Lbitter/jnibridge/JNIBridge; Uncaught translation error:
java.lang.IllegalArgumentException: already added:
Lbitter/jnibridge/JNIBridge$a; Uncaught translation error:
java.lang.IllegalArgumentException: already added:
Lcom/unity3d/player/NativeLoader; Uncaught translation error:
java.lang.IllegalArgumentException: already added:
Lcom/unity3d/player/ReflectionHelper; Uncaught translation error:
java.lang.IllegalArgumentException: already added:
Lcom/unity3d/player/ReflectionHelper$1; Uncaught translation error:
java.lang.IllegalArgumentException: already added:
Lcom/unity3d/player/ReflectionHelper$a; Uncaught translation error:
java.lang.IllegalArgumentException: already added:
Lcom/unity3d/player/UnityPlayer; Uncaught translation error:
java.lang.IllegalArgumentException: already added:
Lcom/unity3d/player/UnityPlayer$1; Uncaught translation error:
java.lang.IllegalArgumentException: already added:
Lcom/unity3d/player/UnityPlayer$10;
...

Answer 1

The classes.jar (which you later renamed it to 'unity_classes.jar') gets automatically included into the built .apk by Unity, so you shouldn't include it yourself again (in your Android plugin), even under a different name.
In other words, first your android plugin "embeds" this .jar file inside itself. Then, when Unity adds your plugin into the game project, it also adds its own copy of that .jar file (it does that for all android projects). Consequently, Unity essentially ends up having 2 copies of the same .jar file in the project and complains by saying 'unable to convert classes into dex format'.
So although you have to use this library (.jar file) in your project but you should not bundle this library with your project. How to do that? If you're using Android studio, you can achieve this by marking the .jar file as a dependency with the provided scope.
To do this, you can:
1) Go to 'build.gradle' of your module and change this:
compile files('libs/jars/unity_classes.jar')
to this:
provided files('libs/jars/unity_classes.jar')
Or
2) you can right-click on the module (in Android pane) and choose Open Module Settings. Then go to Dependencies tab. Find the Unity module (which I renamed it to unity_classes.jar) and change its scope to Provided.