Error in twitter4j classes while feeding data from twitter using flume - twitter4j

Question

I am using the sources provided by cloudera flume-sources-1.0-SNAPSHOT uploaded on github.
I am getting error below. the method does not exist in the configuration.class the jar provided by cloudera in twitter4j.
How to solve this issue?
{
14/06/12 03:04:56 INFO node.PollingPropertiesFileConfigurationProvider: Configuration provider starting
14/06/12 03:04:56 INFO node.PollingPropertiesFileConfigurationProvider: Reloading configuration file:conf/tagent
14/06/12 03:04:56 INFO conf.FlumeConfiguration: Processing:HDFS
14/06/12 03:04:56 INFO conf.FlumeConfiguration: Processing:HDFS
14/06/12 03:04:56 INFO conf.FlumeConfiguration: Processing:HDFS
14/06/12 03:04:56 INFO conf.FlumeConfiguration: Added sinks: HDFS Agent: TwitterAgent
14/06/12 03:04:56 INFO conf.FlumeConfiguration: Processing:HDFS
14/06/12 03:04:56 INFO conf.FlumeConfiguration: Processing:HDFS
14/06/12 03:04:56 INFO conf.FlumeConfiguration: Processing:HDFS
14/06/12 03:04:56 INFO conf.FlumeConfiguration: Processing:HDFS
14/06/12 03:04:56 INFO conf.FlumeConfiguration: Processing:HDFS
14/06/12 03:04:56 INFO conf.FlumeConfiguration: Post-validation flume configuration contains configuration for agents: [TwitterAgent]
14/06/12 03:04:56 INFO node.AbstractConfigurationProvider: Creating channels
14/06/12 03:04:56 INFO channel.DefaultChannelFactory: Creating instance of channel MemChannel type memory
14/06/12 03:04:57 INFO node.AbstractConfigurationProvider: Created channel MemChannel
14/06/12 03:04:57 INFO source.DefaultSourceFactory: Creating instance of source Twitter, type com.cloudera.flume.source.TwitterSource
14/06/12 03:04:57 ERROR node.PollingPropertiesFileConfigurationProvider: Unhandled error
java.lang.NoSuchMethodError: twitter4j.conf.Configuration.getRequestHeaders()Ljava/util/Map;
at twitter4j.StreamingReadTimeoutConfiguration.getRequestHeaders(TwitterStreamImpl.java:664)
at twitter4j.internal.http.HttpClientWrapper.<init>(HttpClientWrapper.java:47)
at twitter4j.TwitterStreamImpl.<init>(TwitterStreamImpl.java:54)
at twitter4j.TwitterStreamFactory.<clinit>(TwitterStreamFactory.java:40)
at com.cloudera.flume.source.TwitterSource.<init>(TwitterSource.java:64)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:355)
at java.lang.Class.newInstance(Class.java:308)
at org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:42)
at org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:327)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
}

Answer 1

Check the twitter4j jars version you are using.
twitter4j.conf.Configuration.getRequestHeaders() method is not in the latest versions.
But it is in twitter4j 2.x

Answer 2

You have to replace all the twitter4j-3.0.3 libraries placed in flume/lib folder by the twitter4j-2.x.x version. The reason is that same class name is conflicting in twitter4j-3.0.3 and flume-sources-1.0-SNAPSHOT.jar
To avoid this you have to make changes in the twitter4j properties by putting the following property in twitter4j.properties as class path:
streamBaseURL=https://stream.twitter.com/1.1/ `
Or you can download the new flume-source-1.0-snapshot.jar from here.
This issue has been resolved by the Twitter community, for a full explanation you can check this discussion.

Answer 3

When you receive
twitter4j.conf.Configuration.getRequestHeaders() method error.
Try below steps, it resolved my issue.
Remove the below 3 jar files from /usr/lib/flume-ng/lib/ directory.
twitter4j-core-3.0.3.jar
twitter4j-media-support-3.0.3.jar
twitter4j-stream-3.0.3.jar
Add the below jar files to /usr/lib/flume-ng/lib/ directory.
(twitter4j jars should be latest version.)
flume-sources-1.0-SNAPSHOT.jar
twitter4j-core-4.0.7.jar
twitter4j-async-4.0.7.jar
twitter4j-stream-4.0.7.jar