The below program is to connect to Oracle 11g and fetch the records. How ever it is giving me NullPointerException for the coder at pipeline.apply().
I have added the ojdbc14.jar to the project dependencies.
public static void main(String[] args) {
Pipeline p = Pipeline.create(PipelineOptionsFactory.create());
p.apply(JdbcIO.<KV<Integer, String>>read()
.withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
"oracle.jdbc.driver.OracleDriver", "jdbc:oracle:thin:#hostdnsname:port/servicename")
.withUsername("uname")
.withPassword("pwd"))
.withQuery("select EMPID,NAME from EMPLOYEE1")
.withRowMapper(new JdbcIO.RowMapper<KV<Integer, String>>() {
public KV<Integer, String> mapRow(ResultSet resultSet) throws Exception {
return KV.of(resultSet.getInt(1), resultSet.getString(2));
}
}));
p.run();
}
is giving the below error.Any clue?
Exception in thread "main" java.lang.NullPointerException: coder
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:228)
at org.apache.beam.sdk.io.jdbc.JdbcIO$Read.validate(JdbcIO.java:283)
at org.apache.beam.sdk.io.jdbc.JdbcIO$Read.validate(JdbcIO.java:216)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:399)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:307)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:47)
at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:158)
at org.apache.beam.examples.v030.JdbcUtil.main(JdbcUtil.java:21)
Hi there!
Sorry the error message is not very helpful, but in fact it is a validation step. I have filed BEAM-959 to improve this.
You are required to provide a coder such as via
.withCoder(KvCoder.of(VarIntCoder.of(), StringUtf8Coder.of())`
I have filed BEAM-960 to improve automation of this coder, like we have in most other places in Beam.
Try this.
pipeline.apply(( JdbcIO.<KV<Integer, String>>read().withCoder(KvCoder.of(VarIntCoder.of(),StringUtf8Coder.of()))
.withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
"com.mysql.jdbc.Driver", "jdbc:mysql://localhost:3306/deepakgoyal")
.withUsername("root")
.withPassword("root"))
.withQuery("select empid, name from employee")
.withRowMapper(new JdbcIO.RowMapper<KV<Integer, String>>() {
public KV<Integer, String> mapRow(ResultSet resultSet) throws Exception {
return KV.of(resultSet.getInt(1), resultSet.getString(2));
}
})
))
And don't forget to add MySQL connector jar in your project.
Related
I am trying to write Integration test for spring batch aplication , in my project there are approx 10+jobs
I want to run only a single job but unable to achieve any Suggestion.
#SpringBatchTest
#RunWith(SpringRunner.class)
#ContextConfiguration(classes= MyApp.class)
#SpringBootTest
#Slf4j
public class JobATest {
JobLauncherTestUtils jobLauncherTestUtils = new JobLauncherTestUtils();
#Autowired
#Qualifier(JOB_A)
Job joba;
#Before
public void setUp() throws Exception {
log.debug("CAME HERE setUp {} ",joba.getName());
jobLauncherTestUtils.setJob(joba);
}
#After
public void tearDown() throws Exception {
}
#Test
public void processAJob() throws Exception {
jobLauncherTestUtils.launchJob();
}
}
ERROR
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException:
Error creating bean with name 'jobLauncherTestUtils': Unsatisfied dependency expressed
through method 'setJob' parameter 0; nested exception is
org.springframework.beans.factory.NoUniqueBeanDefinitionException:
No qualifying bean of type 'org.springframework.batch.core.Job' available:
expected single matching bean but found 2: **joba,jobb**
When using #SpringBatchTest, it is expected that the test context contains a single job bean. This is mentioned in the javadoc of the annotation.
There is an open issue for that which we might consider for the next major release. Please upvote or add a comment if you have a suggestion for an improvement. I also invite you to check the thread on Multiple Job unit testing with #SpringBatchTest which could help you as well.
I'm trying to execute the following SQL statement every time the Database Session gets refreshed. I have a Spring Boot 2.0.1.RELEASE with JPA application and a PostgreSQL Database.
select set_config('SOME KEY', 'SOME VALUE', false);
As the PostgreSQL documentation states the is_local parameter is used to indicate that this configuration value will apply just for the current transaction -if true- or will be attached to the session (as I require) -if false-
The problem is that I'm not aware when Hibernate/Hikari are refreshing the db session, so, in practice, the application start failing when it has a couple of minutes running, as you can imagine...
My approach -that is not working yet- is to implement a EmptyInterceptor, for that I have added a DatabaseCustomizer class to inject my hibernate.session_factory.interceptor properly in a way that Spring can fill out all my #Autowires
DatabaseInterceptor.class
#Component
public class DatabaseInterceptor extends EmptyInterceptor {
#Autowired
private ApplicationContext context;
#Override
public void afterTransactionBegin(Transaction tx) {
PersistenceService pc = context.getBean(PersistenceService.class);
try {
pc.addPostgresConfig("SOME KEY", "SOME VALUE");
System.out.println("Config added...");
} catch (Exception e) {
e.printStackTrace();
}
}
}
DatabaseCustomizer.class
#Component
public class DatabaseCustomizer implements HibernatePropertiesCustomizer {
#Autowired
private DatabaseInterceptor databaseInterceptor;
#Override
public void customize(Map<String, Object> hibernateProperties) {
hibernateProperties.put("hibernate.session_factory.interceptor", databaseInterceptor);
}
}
Obviously, there is a problem with this approach because when I #Override the afterTransactionBegin method to start another transaction I get an Infinite loop.
I tried to look something inside that Transaction tx that could help to be sure that this transaction is not being generated by my own addPostgresConfig but there is not much on it.
Is there something else I could try to achieve this?
Thanks in advance,
i dunno how make one sample test using kafka, i tried to follow the spring guide but dont work.
Can someone help me?
zzzzz zz z z z z z z z z z z z
#RunWith(SpringRunner.class)
#SpringBootTest
#DirtiesContext
public class EnrollSenderTest {
#Autowired
public EnrollSender producer;
#Autowired
private BinderFactory<MessageChannel> binderFactory;
#Autowired
private MessageCollector messageCollector;
#SuppressWarnings("unchecked")
#Test
public void test() {
Message<String> message = new GenericMessage<>("hello");
producer.sendEnroll(message);
Message<String> received = (Message<String>) messageCollector.forChannel(producer.getOutput()).poll();
assertThat(received.getPayload(), equalTo("hello"));
}
}
And my class Producer is:
#Service
#EnableBinding(Source.class)
public class EnrollSender {
private final MessageChannel output;
public EnrollSender(Source output) {
this.output = output.output();
}
public void sendEnroll(Object enroll) {
output.send(MessageBuilder.withPayload(enroll).build());
}
public MessageChannel getOutput() {
return output;
}
}
But gives the following error:
java.lang.IllegalStateException: Failed to load ApplicationContext
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'messageCollector' defined in class path resource [org/springframework/cloud/stream/test/binder/TestSupportBinderAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cloud.stream.test.binder.MessageCollector]: Factory method 'messageCollector' threw exception; nested exception is java.lang.NoSuchMethodError: org.springframework.cloud.stream.binder.BinderFactory.getBinder(Ljava/lang/String;Ljava/lang/Class;)Lorg/springframework/cloud/stream/binder/Binder;
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cloud.stream.test.binder.MessageCollector]: Factory method 'messageCollector' threw exception; nested exception is java.lang.NoSuchMethodError: org.springframework.cloud.stream.binder.BinderFactory.getBinder(Ljava/lang/String;Ljava/lang/Class;)Lorg/springframework/cloud/stream/binder/Binder;
Caused by: java.lang.NoSuchMethodError: org.springframework.cloud.stream.binder.BinderFactory.getBinder(Ljava/lang/String;Ljava/lang/Class;)Lorg/springframework/cloud/stream/binder/Binder;
Marius Bogoevici, my dependencys
dependencyManagement {
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:Camden.SR4"
}
}
compile 'org.springframework.cloud:spring-cloud-starter-stream-kafka'
compile group: 'org.springframework.cloud', name: 'spring-cloud-stream-test-support', version: '1.1.1.RELEASE'
Looks like you have a mismatched dependency set on the classpath (i.e. an older version of Spring Cloud Stream core).
You can solve this by removing the version for spring-cloud-stream-test-support because the Camden.SR4 BOM will provide the correct one.
Moreover, if you want to test with an embedded Kafka instance, you can find an example here: https://github.com/spring-cloud/spring-cloud-stream-samples/blob/master/multibinder/src/test/java/multibinder/RabbitAndKafkaBinderApplicationTests.java#L57
(The example shows you how to configure the Kafka binder with an embedded broker for testing - it also shows how to use two different binders within the same app, but probably you don't care about that).
This is because of the incompatible versions as pointed out by Marius above.
You would either need Camden.SR5 that has compatible versions of Spring Cloud Stream and Spring Cloud Stream test support or Camden.SR4 with Spring Cloud Stream test support version 1.1.0.RELEASE.
This is change that went in between 1.1.0.RELEASE and 1.1.1.RELEASE of Spring Cloud Steram:
I am trying to access data from different databases based on a runtime variable. For this purpose, I have a custom implementation of MongoOperations. My implementation is same as MongoTemplate except my getDb() method looks like below:
public DB getDb() {
return mongoDbFactory.getDb(PropertyManager.getCurrentTenant().getCode());
}
While reading data in a transaction, I am getting below error:
[TransactionSynchronizationUtils] - TransactionSynchronization.beforeCompletion threw exception
java.lang.IllegalStateException: No value for key [Mongo: localhost/127.0.0.1:27017] bound to thread
It appears harmless as this exception is only logged:
public static void triggerBeforeCompletion() {
for (TransactionSynchronization synchronization : TransactionSynchronizationManager.getSynchronizations()) {
try {
synchronization.beforeCompletion();
}
catch (Throwable tsex) {
logger.error("TransactionSynchronization.beforeCompletion threw exception", tsex);
}
}
}
But I am having a hard time figuring out why this is happening. Any clues?
Versions:
spring-data-mongodb: 1.2.3.RELEASE
spring-tx: 4.0.5.RELEASE
There was a bug raised for similar issue which was fixed in 1.1 GA release.
It worked by extending
SimpleMongoDbFactory: returning custom DB in DB getDb(String dbName).
MongoTemplate: Supplying above factory.
I'm trying to do a transformation between two models,
every things seems to be ok except the use of the eclipse in standalone mode,
I got errors when trying to execute the tansformation programatically from java,
How to solve that? its urg. Thanks for your help and sorry for my bad english.
Errors :
ResourcesPlugin.getWorkspace().getRoot();
Exception in thread "main" java.lang.IllegalStateException: Workspace is closed.
at org.eclipse.core.resources.ResourcesPlugin.getWorkspace(ResourcesPlugin.java:399)
File f = new File(Path);
URI transformationURI = URI.createFileURI(f.getAbsolutePath());
resource = resourceSet.getResource(transformationURI, true);`
Exception in thread "main" org.eclipse.emf.ecore.resource.impl.ResourceSetImpl$1DiagnosticWrappedException: org.xml.sax.SAXParseException: Content is not allowed in prolog.
at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.handleDemandLoadException(ResourceSetImpl.java:315)
at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.demandLoadHelper(ResourceSetImpl.java:274)
at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.getResource(ResourceSetImpl.java:397)
When i omit the line :
//resource = resourceSet.getResource(transformationURI, true);
I get an other exception :
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.eclipse.m2m.qvt.oml.TransformationExecutor.doLoad(TransformationExecutor.java:205)
at org.eclipse.m2m.qvt.oml.TransformationExecutor.loadTransformation(TransformationExecutor.java:108)
at org.eclipse.m2m.qvt.oml.TransformationExecutor.execute(TransformationExecutor.java:137)
Caused by: java.lang.NullPointerException
at org.eclipse.m2m.internal.qvt.oml.compiler.UnitResolverFactory$Registry$1.readFactories(UnitResolverFactory.java:66)
at org.eclipse.m2m.internal.qvt.oml.compiler.UnitResolverFactory$Registry$1.(UnitResolverFactory.java:44)
at org.eclipse.m2m.internal.qvt.oml.compiler.UnitResolverFactory$Registry.(UnitResolverFactory.java:43)
... 6 more
As covered in this question you must run an Eclipse application to ensure all the necessary Eclipse setup is completed.
For a 'headless' application which does not have a UI the IApplication class can be very simple:
public class Application implements IApplication
{
#Override
public Object start(final IApplicationContext context)
throws Exception
{
.. your code here ...
return IApplication.EXIT_OK;
}
#Override
public void stop()
{
// No action
}
private static void ensurePluginStarted(String id)
{
Bundle bundle = Platform.getBundle(id);
if (bundle != null)
{
if ((bundle.getState() & Bundle.ACTIVE) == 0)
{
try
{
bundle.start(Bundle.START_TRANSIENT);
}
catch (BundleException ex)
{
ex.printStackTrace();
}
}
}
}
}
This application can use the Eclipse workspace but not the UI. In some cases it may be necessary to call the ensurePluginStarted method to ensure that all the plug-ins you are going to use are running. Any plug-in which tries to use the UI will fail.
If you want a UI then a full Eclipse RCP is required.