I am new to camel-jpa and would need your help with the following problem:
I need to read data from a database table, transform it and save it into another database:
<route id="FromEmployee1ToEmployee2">
<from uri="jpa1://Employee1?consumeDelete=false&consumer.namedQuery=getAll" />
<bean ref="transformerBean"/>
<to uri="jpa2://Employee2"/>
</route>
This is already working great!
But the problem now is that I need to look up some data for Employee1 from a different table (I need to read the "last_modified" date for that entry). In SQL I would simply do like this: select last_modified from table2 where table2.id = <employee.ID>. but how can I realise this with camel-jpa?
The camel-jpa component offers options to use a query, such as a named query. This allows you to write in SQL like using the JPL (I think thats the JPA name for SQL).
There is a little example at
http://camel.apache.org/jpa
here is my solution (just in case someone is struggling with the same problem):
since its not possible to dynamically pass attributes to a named query in a camel route (at least, I couldn't find any way to do it...), i used a bean to deal with it:
<route...
<from uri="jpa1://Entity1" />
<bean ref="MyBean" />
<to uri="jpa2://Entity2" />
</route>
and within the bean, i use a (autowired) DAO to call my queries. This allows me to do all the kinds of content enrichment i need ...
well, it works great so far, but i think there could me a more elegant way of doing this with camel...
BR,
M
Related
How can I write spring data JPA exist finBy methods for
WHERE Id="1" AND (STATUS ="N" OR STATUS ="K")
I had tried findByIdAndStatusOrStatus(1,"N","K") which is not working
If you have the id then you have a unique entity no need to search for it, but i suppose you just gave an unfortunate example.
Try using the following it will work:
findBySomePropertyAndStatusIn(SomeProperty property,Collection<Status> statuses);
After learning SpringBoot, I wanted go further to handle integration tests using (DBUnit and SpringTestDBUnit). Throughout the process, everything was going well until I came across setting values for boolean datatyped columns on the dataset. (Contents of the dataset is given below)
<?xml version="1.0" encoding="UTF-8"?>
<dataset>
<Client code="0001" name="client_one" />
<Client code="0002" name="client_two" />
<Client code="0003" name="client_three" active="false" />
<Client code="0004" name="client_four" />
</dataset>
Adding active="false" attribute to Client record [code=0003], my integration tests fails and showing me this message Exception processing table name='Client' which was resulted the Client record [code=0001] violates the active not null column constraint.
After fixing the error (on branch DBUnit_For_Boolean_Columns_Attempt_One) by supplying values for active column on all records (which is a bit off the specification), it worked. But my target was able to run the integration tests successfully with the dataset written above.
The question is how can the integration tests be successful using the dataset above? As of now, I'm having a hard time implementing solutions so I've created a Bitbucket repository for you to see and help on-hand.
Changelogs
2015/02/04 changes
Improve question contents
Added Bitbucket repository
(I had a different answer here, which came from a complete misdiagnosis of the issue, as I missed the fact OP was using FlatXmlDataSet).
From FlatXmlDataSet's documentation:
Table metadata is deduced from the first row of each table by default. Beware that DbUnit may think a table misses some columns if the first row of that table has one or more null values.
This seems to be the issue here, as the first entry does not specify a value for active. To avoid this problem, you can use DBUnit's column sensing property, which essentially makes DBUnit read the entire XML before deducing the table's structure:
FlatXmlDataSetBuilder builder = new FlatXmlDataSetBuilder();
builder.setInputSource(new File("path/to/dataSet.xml"));
builder.setColumnSensing(true); // HERE!
IDataSet dataSet = builder.build();
Alternatively, if you aren't creating the builder yourself and do not want to manipulate it, just make sure to have the active column in every element, especially the first one:
<dataset>
<client code="0001" name="client_one" active="false" />
<client code="0002" name="client_two" active="true" />
</dataset>
Here is my query which I'm using to fetch the data from my table.
SELECT XMLQUERY('$INFO/root/database_systems/system/#name = ("SYS1","SYS2","SYS3")')
FROM MyTable WHERE ACCT_ID = 'ID-1234';
Ok actually it is returning me true. Just because of the first value SYS1. It exists in the hierarchy but not the others. I just want to compare multiple values.
Please suggest a way to achieve this functionality. Thanks
<root>
<database_systems>
<system name="SYS1">1</system>
</database_systems>
</root>
If I understand correctly, you want to check for a database_systems element that contains at least all of the child elements:
<system name="SYS1">...</system>
<system name="SYS2">...</system>
<system name="SYS3">...</system>
If that is correct then you need to AND your conditions together, what you had previously was an OR:
SELECT XMLQUERY('not(empty($INFO/root/database_systems[system/#name eq "SYS1"][system/#name eq "SYS2"][system/#name eq "SYS3"]))')
FROM MyTable WHERE ACCT_ID = 'ID-1234';
I have used three predicates to achieve the AND, and then I check that a match was found using not(empty(...)). There are plenty of other ways to achieve this too.
With Hibernate I'm used to do something like the following:
select n from NetworkElement n join fetch n.site s where s.active is true
However, EclipseLink complains a lot about this:
Caused by: Exception [EclipseLink-8024] (Eclipse Persistence Services - 2.0.0.v20091127-r5931): org.eclipse.persistence.exceptions.JPQLException
Exception Description: Syntax error parsing the query [select n from NetworkElement n join fetch n.site s], line 1, column 49: syntax error at [s].
(The query on the stack is different from the one above, but the result is the same)
I've tried different combinations, none of which worked:
select n from NetworkElement n join fetch n.site where n.site.active is true
select n from NetworkElement n join fetch n.site as site where site.active is true
I also tried switching to a different entity in my domain model, suspecting that maybe my mapping is not correct. Still, the same problem.
Could it be that I can only achieve this using a query hint? I don't want to do that.
By the way, I'm using EcliseLink as shipped with Netbeans 6.8 and Glassfish v3.
I'd appreciate any help!
Rodrigo
The main issue is that the JPQL syntax does not allow for aliasing fetch joins and that is why EclipseLink uses query hints for this functionality. There is an Enhancement Request to add aliasing join fetches directly in the JPQL and if you would like to see it completed please vote for it. ( https://bugs.eclipse.org/bugs/show_bug.cgi?id=293775 ).
--Gordon Yorke
Well, it seems that one is not allowed to alias a fetch join in JPQL, indeed.. It works with Hibernate because it supports aliasing through HQL, and you are allowed to issue HQL to a JPA Query object.
Therefore, I had no choice but to switch to named queries with query hints. I don't really like declaring queries with annotations because of the high verbosity on the entity classes, so I added a orm.xml file to the persistence unit's jar, and did the following:
<!-- Fetch network elements -->
<named-query name="fetchNetworkElements">
<query>select n from NetworkElement n</query>
<lock-mode>NONE</lock-mode>
<hint name="eclipselink.join-fetch" value="n.site" />
<hint name="eclipselink.join-fetch" value="n.site.area" />
</named-query>
Hope this gives some clue to anybody struggling with the same shortcomings of the raw JPQL.
I guess, it is quite common task. There is a table A let's say in Visual Foxpro and needs to be transformed to table B in SQL Server 2008.
Operation is going to be repetitive. Column names may be changed , some input columns dropped , column types may be changed ( string <=> int ), some output columns generated ( GUID , identity ) . Also normalization may happen where input table results in 2 joined with pivot table - let's drop this case for simplicity
I know about SSIS existence , probably there are scripts which can do what I described ?
I've already written C# program which reads XML ( snippet below ) and does mapping and out key generation but just thought whether all that - another wheel reinvention.
<Tables>
<Table inTbl="A" outTbl="B">
<Keys>
<Key name="it_pk" type="GUID">
</Keys>
<Mappings>
<Column inCol="hxkey" outCol="it_tariff" />
<Column inCol="txt" outCol="it_description" />
<Column inCol="uq1" outCol="it_uq1" />
<Column inCol="date1" outCol="it_startdate" />
<Column inCol="date2" outCol="it_stopdate" />
</Mappings>
</Table>
</Tables>
This is pretty much exactly what SSIS is designed for, however it's not as "generic" as your current system, but it's very easy to set up what you've described.
You have a control flow and a data flow. Your control flow can contain many data flows (e.g. sync table A, then B, then C) and other tasks (e.g. move files, iterate, custom scripts).
In your data flow you define your source, destination and transformations. Your source can be pretty much anything. Then, you can do derived columns and change data type tranformations to do what you've described. Then your destination takes the transformed data (e.g. SQL table, SQL insert statement, SQL update statement)
This is done using Visual Studio, and I reckon you could knock out a simple package to sync two tables in 5 minutes. Then, you can schedule it using SQL agent.
This is called ETL btw (extract, transform, load) in case you wanted to evaluate other options, but I think SSIS would fit nicely.
In addition to the excellent answer above, SSIS is a good platform for implementing programmatic solutions in this space even if the Visual Studio package designer tools aren't appropriate.
Actually the platform is more flexible and better finished than the Visual Studio tools built on top of it.