I am new to Talend.
Can you please explain me which is the best method to increase the performance. Is it using query or Talend component and also please explain me the possible reasons as well
Related
I'm working with Firebird 2.5. I have used MySQL table partitioning in the past to help optimize very large tables by creating partitions based on year. I would like to do the same thing, if possible, in Firebird but I'm having trouble finding any documentation.
Does anyone know if this is possible and if so, can you please point me toward some documentation?
Firebird does not support table partitioning, which is also why you can't find anything about it in the documentation.
Depending on the exact performance problem you're trying to solve and the queries you use, choosing your indexes well may already solve part of the problem.
I want to read data from db using spring jdbctemplate in batches of specific size. I found batchupdate to update table data, but couldn't find any select in batches. Can someone please help me here?
If I understand right what you are looking for then
Use setFetchSize method to set the fetch size. This would select only specific size.
for writing , we call it batch. For reading, we call it paging. in fact, you can implement by yourself. Here is an example.
I would like to know if there is a way to output to a MongoDB in Luigi. I see in the documentation they support files (local FS, HDFS), S3, PostgreSQL but not MongoDB. If not, could someone explain me why not? Maybe it is a bad idea to have it? I would like to store the data in a database because then I can explore it by querying it. However I am using mongodb and I would not like to install another database. I do not need a relational database as I am using the database only to store and query ( NoSql ) without relationships, so the best option is mongodb.
Basically I need a task to read the data and save it in the database. Then the next task take this output and process the data.
Any recommendation, suggestion or clarification is more than welcome. Thanks!
You can try using mortar-luigi.
Check out this link for MongoDB tasks and this example.
I couldn't use
SQL=TRACE
since, there so many queries executed each second, but wish to see the same result for only one Query.
Is there anyway to get this.
Thanks.
No, there is no way to do this via configuration. Perhaps you could write a custom JDBCListener and try to log only the SQL you care about? Good luck!
I am using Weblogic 10.3 and EJB3 with TopLinks(EclipseLinks) JPA implementation. I am facing an issue with my named queries, which are performing slow. However If I use EclipseLinks JPAEntityManager's Expression query to achieve same, its faster. I am not able to understand why my standard JPA named queries are slow. Has anyone faced similiar issue or Can someone explain the root cause of this issue.
Am I missing any basic tunning or parameter???
That is very odd. Named queries should be in general faster as they are parsed once.
What is the specific query and its SQL? Are you comparing it to the same Expression query, or something different, which is its SQL?
Try enabling the EclipseLink PerformanceProfiler on the queries, or try using a Java profiler such as JProfiler.