I have a simple spring project which try to retrieve a document from couchbase using spring-data-couchbase. I have configured the config by extending AbstractCouchbaseConfiguration. Everything works perfectly fine.
Since I use couchbase as a cache, now I need to set the operation timeout to a lower value. Anybody can shed some light on how to do it?
According to the docs, the correct answer is wrong. That's not the way it should be done...
When you extend from AbstractCouchbaseConfiguration
Default settings can be customized through the DefaultCouchbaseEnvironment.Builder or through the setting of system properties. Latter ones take always precedence and can be used to override builder settings at runtime too. http://docs.couchbase.com/sdk-api/couchbase-java-client-2.0.0/com/couchbase/client/java/env/DefaultCouchbaseEnvironment.html
For instance, to customize the timeout connection:
#Override
protected CouchbaseEnvironment getEnvironment() {
DefaultCouchbaseEnvironment.builder().connectTimeout(15000);
return super.getEnvironment();
}
There are other options that can be assigned this way.
According the docs (https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html ),
Here is the application.properties :
spring.couchbase.env.timeouts.connect=5000ms # Bucket connections timeouts.
spring.couchbase.env.timeouts.key-value=2500ms # Blocking operations performed on a specific key timeout.
spring.couchbase.env.timeouts.query=7500ms # N1QL query operations timeout.
spring.couchbase.env.timeouts.socket-connect=1000ms # Socket connect connections timeout.
spring.couchbase.env.timeouts.view=7500ms # Regular and geospatial view operations timeout.
To define a timeout for the CouchbaseClient you have to provide it using the ConnectionFactory. Sadly, the current version of spring-data-couchbase doesn't provide a simple way to do that.
The class responsible to create connection factories is ConnectionFactoryBean, and it has a setter for the operations timeout, but I couldn't find anything for #Configuration classes.
Since you are extending AbstractCouchbaseConfiguration, you might want to override couchbaseClient():
public class MyCouchbaseConfiguration extends AbstractCouchbaseConfiguration {
...
private final CouchbaseConnectionFactoryBuilder builder = new CouchbaseConnectionFactoryBuilder();
private CouchbaseConnectionFactory connectionFactory;
...
#Override
#Bean(destroyMethod = "shutdown")
public CouchbaseClient couchbaseClient() throws Exception {
setLoggerProperty(couchbaseLogger());
if(connectionFactory == null){
builder.setOpTimeout(myTimeout);
// Set another parameters.
...
connectionFactory = builder.buildCouchbaseConnection(
bootstrapUris(bootstrapHosts()),
getBucketName(),
getBucketPassword()
);
}
return new CouchbaseClient(connectionFactory);
}
}
Also, you can call directly CouchbaseFactoryBean but it's not a good practice if you are not configuring your application using XML bean definitions.
Here is the XML configuration just in case:
<bean id="couchbase" class="org.springframework.data.couchbase.core.CouchbaseFactoryBean">
<property name="opTimeout" value="1000"/> <!-- 1 sec -->
<property name="bucket" value="myBucket"/>
<property name="password" value="myPassword"/>
<property name="host" value="myHost"/>
</bean>
<couchbase:template id="couchbaseTemplate"/>
For Spring Data Couchbase 2, adding the following property in application.properties did it
spring.couchbase.env.timeouts.connect=20000
I used queryTimeout to set the value of operation timeout as shown below:
CouchbaseEnvironment env = DefaultCouchbaseEnvironment.builder()
.connectTimeout(50000).socketConnectTimeout(50000)
.queryTimeout(1000)
.build();
this will ensure you get the response within 1s.
use this link to get different available options:
https://docs.couchbase.com/java-sdk/2.7/client-settings.html#timeout-options
Related
I have a seda queue where i set queueSize option according to the camel documentation
The route i have looks like:
from("seda:someQueue?concurrentConsumers=10&queueSize=10")
.process(someProcessor);
I'm getting the following error due to the queueSize option:
org.apache.camel.FailedToCreateRouteException: Failed to create route....bla bla bla..
There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{queueSize=10}].....
[stacktrace continues here]
Can anyone point out what's wrong?
I'm using Java 8, Camel 2.9.13
Notice that the documentation says that the option queueSize is component only, which mean you need to configure it on the SedaComponent instead. In other words you cannot configure it on the endpoint as you do in your route above.
For up to date documentation and better docs on Camel components, then browse github pages at: https://github.com/apache/camel/blob/master/components/camel-seda/src/main/docs/seda-component.adoc
Those docs are up to date and show both component vs endpoint options in different tables, so its easier to know the difference.
For those who have the same question, this is how i use the queueSize now
Initialize a new seda component,
SedaComponent sedaComponent = new SedaComponent();
sedaComponent.setQueueSize(3);
context.addComponent("sedaComponent", sedaComponent);
then use this component at the route like,
from("seda:someEndPoint?concurrentConsumers=5")
.to("sedaComponent:someOtherSedaEndPoint?blockWhenFull=true");
Create a specific queue . it is quarkus example replace Named as bean and ApplicationScoped to Configuration for Spring boot
#ApplicationScoped
public class ConnectionConf {
#Named("NonLimitQueue")
#Produces
public BlockingQueue arrayDeque(){
return new ArrayBlockingQueue(30000);
}
}
camel side
from("seda:queue=#NonLimitQueue")
.convertBodyTo(String.class).log("${body}")
Replace queueSize
with
size(query param in apache document)
from("seda:someQueue?concurrentConsumers=10&queueSize=10")
.process(someProcessor);
I am writing a CLI app with Mybatis. In my app, when i go to different menus, it prompts for the user and password for the particular database that menu goes against.
I want to use Guice and Mybatis to handle all this but i have a slight problem. I want to use the XML config file to handle the mybatis config per database, but the user and pass from each connection has to come from the UI. So basically, i want to load mybatis xml file for a particular connection, then insert the credentials for the particular connection the user typed in, then bind those to the guice injector for that set of menus.
I can do it in java with a property object pretty easy, but i can't figure out how to do it with loading the XML first, then augmenting it with certain settings before loading.
Has anyone tried this?
If you are using mybatis guice this can be done by providing your dataSourceProvider for MyBatisModule like this:
Class<? extends Provider<DataSource>> dataSourceProviderType = YourDataSourceProvider.class;
Injector injector = Guice.createInjector(
new MyBatisModule() {
#Override
protected void initialize() {
bindDataSourceProviderType(dataSourceProviderType);
// other initialization ...
}
},
// other modules
);
YourDataSourceProvider should be able to create DataSource using credentials gotten from UI.
In this case you still can use xml mappers for mybatis.
I have a JdbcCursorItemReader which reads from a database and uses a row mapper to turn the resultset into an object. The requirement is now to marshall the object into XML, which must then be placed onto a message queue.
Three questions:
1. Should a marshaller in Spring Batch be implemented as an ItemProcessor or as an ItemWriter?
2. There is no need to write the XML to a file, but rather simply send it as a string object in a message. What is the best way in Spring Batch to do this?
3. If the marshaller and message dispatcher should both be implemented as separate writers, how is it possible to chain them.
Any example code or links would be very appreciated.
Thanks
I've thought Spring Batch could use a ChannelItemWriter for a while now. If you approached this from the perspective that you write the XML snips to a Spring Integration Channel (that can be then backed by a message queue), you could write the custom ItemWriter that way. That would allow you to approach the item conversion a couple different ways (either in the ItemProcessor or in via a transformer in Spring Integration). If you pursue this approach, I'd recommend contributing the writer back to Spring Batch.
Create a custom writer which will be used when data is written via ItemWriter.
A sample code HERE with some modifications as per your requirement:
import java.io.StringWriter;
import java.io.Writer;
import javax.xml.stream.XMLEventWriter;
import javax.xml.stream.XMLOutputFactory;
import javax.xml.stream.XMLStreamException;
public class CustomXMLWriter extends org.springframework.batch.item.xml.StaxEventItemWriter<User> {
private Writer myWriter;
/**
* Subclasses can override to customize the writer.
*/
#Override
protected XMLEventWriter createXmlEventWriter(XMLOutputFactory outputFactory, Writer writer)
throws XMLStreamException {
myWriter = new StringWriter();
return outputFactory.createXMLEventWriter(myWriter);
}
#Override
public void close() {
String outputXML = myWriter.toString() + "</" + this.getRootTagName() + ">";
System.out.println(outputXML);
// code here to add the resultant XML string in message queue
super.close();
}
}
In this example you have to mention output resource but the resultant XML file always contains just root tag name.
XML:
<bean id="customXMLItemWriter" class="com.mkyong.CustomXMLWriter">
<property name="resource" value="file:xml/outputs/users.xml" />
<property name="marshaller" ref="userUnmarshaller" />
<property name="rootTagName" value="users" />
</bean>
What is the best way to take output from a Spring Batch item writer - obviously run as part of a job - and to place it onto a Spring Integration message channel? I understand that one option is to code a Spring Batch listener (extending ItemWriteListener), which would be injected with a message channel, and then to attach the listener to the batch job. Is there another option?
One option to inject to the ItemWriter a Spring Integration <gateway>:
<int:gateway id="itemWriterGateway" service-interface="com.my.proj.MyGateway"/>
class MyItemWriter implements ItemWriter<Foo> {
#Autowired
private MyGateway gateway;
public void write(List<Foo> items) {
for(Foo foo : items) {
this.gateway.send(foo);
}
}
}
Another option to use ItemWriterAdapter:
<bean id="myItemWriter" class="org.springframework.batch.item.adapter.ItemWriterAdapter">
<property name="targetObject" ref="itemWriterGateway"/>
<property name="targetMethod" value="send"/>
</bean>
I won't argue that StepListener may have value for you too. But here there is need to understand what you want to achieve using Spring Integration from Batch Job...
UPDATE
Actually, Listener won't help you, because the ItemWriter doesn't return anything and it encapsulates its logic around underlying resource.
As you say that you are using StaxEventItemWriter and its Resource is some output file, maybe there is a reason to decouple your logic and read that result File using <int-file:inbound-channel-adapter>...
UPDATE 2
) why won't the listener help
Because you need a result of ItemReader, otherwise you have to build your XML one more time in the SI flow from items (ItemWriteListener#afterWrite).
How could the the StaxEventItemWriter output, and the inbound adapter connect to, a resource which is held in memory?
How does your resource is in memory, if StaxEventItemWriter requires that it should exists?
From other side you can share that resource with ItemWriter and some SI endpoint. And right: you can do it from ChunkListener#afterChunk and get your resource from, e.g. jobParameters and send it to the SI Gateway.
I'm absolutely newbie in Apache Camel. I'd like to create a very simple app in which it would accept WS calls and save the payload into a database using JPA.
The payload's structure is quite simple. The root is a Marriage object. It contain some String and int and Date fields, a wife, a husband and a list of children (Person objects).
My goal is to save these data into two tables of a database: MARRIAGE, PERSON.
I've successfully created a jaxws:endpoint in which I listen and respond a dummy response.
I've created the tables and JPA entities.
I don't know how to "connect" the WS implementation with the spring configured JpaTemplate. Should I solve this problem with Camel routing using somehow a #Converter class or #Injet it into the WS implementing class by Spring. I'm confused.
Should I use cxf endpoint instead of jaxws endpoint?
You need to use camle-cxf endpoint if you want to use camel. What I would do is expose the endpoint as a camle-cxf endpoint. Something like this:
<camel-cxf:cxfEndpoint id="listenerEndpoint"
address="http://0.0.0.0:8022/Dummy/services/Dummy"
wsdlURL="wsdl/DummyService.wsdl"
xmlns:tns="http://dummy.com/ws/Dummy"
serviceName="tns:Dummy"
endpointName="tns:DummyService">
<camel-cxf:properties>
<entry key="schema-validation-enabled" value="true"/>
<entry key="dataFormat" value="PAYLOAD"/>
</camel-cxf:properties>
</camel-cxf:cxfEndpoint>
Then I would have a simple Spring bean like this:
<bean id="processor" class="com.dummy.DummyProcessor">
<property name="..." value="..."/> //there goes your data source of jdbc template or whatever...
</bean>
If you want to use JPA just configure all the configuration and inject your entity manager into this bean.
The actual class would look something like this:
public class DummyProcessor {
#Trancational //If you need transaction to be at this level...
public void processRequest(Exchange exchange) {
YourPayloadObject object = exchange.getIn().getBody(YourPayloadObject.class);
//object - is your object from SOAP request, now you can get all the data and store it in the database.
}
}
The camel route would be like this:
<camel:camelContext trace="true" id="camelContext" >
<camel:route id="listenerEndpointRoute">
<camel:from uri="cxf:bean:listenerEndpoint?dataFormat=POJO&synchronous=true" />
<camel:log message="Got message. The expected operation is :: ${headers.operationName}"/>
<camel:choice>
<camel:when>
<camel:simple>${headers.operationName} == 'YourPayloadObject'</camel:simple>
<camel:bean ref="processor" method="processRequest"/>
</camel:when>
</camel:choice>
<camel:log message="Got message before sending to target: ${headers.operationName}"/>
<camel:to uri="cxf:bean:someTargetEndpointOrSomethingElse"/>
<camel:log message="Got message received from target ${headers.operationName}"/>
</camel:route>
</camel:camelContext>
Hope this helps.