I am using JdbcCursorItemReader in Spring batch. I have a .yaml file which contains externaldatasourcedetails and query. I need to pass dynamic date value from javaBatchReaderclass. So my question is usingJdbcCursorItemReader` how to send the query parameter.
My query looks like this:
select * from employee where joining date > to_date(?, 'MM/DD/YYYY')
I am doing a date conversion here.
Thanks in advance
You can pass the date as a job parameter and configure your JdbcCursorItemReader with it. You can find an example here: https://github.com/spring-projects/spring-batch/blob/master/spring-batch-core/src/test/resources/org/springframework/batch/core/resource/ListPreparedStatementSetterTests-context.xml#L24-L40
Related
I am not a coder, and very much flying blind, so please excuse the simplicity of this query.
I am streaming Firebase Firestore updates to a BigQuery table using the Firebase extension "Stream collection to BigQuery" which I am then linking as a DataSource in Google Data Studio. This is currently working as intended.
I have 2 questions:
Is there a more efficient way to convert a Firebase timestamp into a BigQuery Date/Time value? The Firebase Timestamp shows in JSON format in the BigQuery table as follows:
{"created_time":{"_seconds":1647554254,"_nanoseconds":234000000}}
My BigQuery SQL code to convert it (which works) is:
DATETIME(TIMESTAMP_SECONDS(CAST(JSON_VALUE(DATA,'$.created_time._seconds') AS int64)),"Africa/Johannesburg") AS createDate
Is there a more efficient way to do this, or is this reasonable?
How do I reference the createDate computed field (above) in another computed field ageDays within this same query? I haven't found it in Google or StackOverflow, either because of poor phrasing or its just too basic a query. I tried using a table alias referencing the createDate computed field (e.g. T.createDate) but no dice. My very ugly workaround was therefore just to reperform the createDate calculation in it's entirety (which feels wrong) in my new computed column ageDays as follows:
DATE_DIFF(current_date("Africa/Johannesburg"),DATETIME(TIMESTAMP_SECONDS(CAST(JSON_VALUE(DATA,'$.created_time._seconds') AS int64)),"Africa/Johannesburg"), DAY) AS ageDays
Would be sincerely grateful for any insights - many thanks.
For your requirement, JSON_EXTRACT can also be used instead of JSON_VALUE. You can use below query to get the expected output.
select
date(timestamp_seconds(cast(json_extract( data , '$.created_time._seconds') as int64))) AS Date_Created
from `project.dataset.timetable`
Output
Table alias cannot be used to reference a field in another column with a SELECT statement as it has limited visibility. Alias can be used with Order By, Group By or Having clauses in a SELECT statement. The best way to get the ageDays is by again computing the whole createDate field.
I want to use a query in a copy job for my source in an Azure Data Factory pipeline together with a date function - here is the dummy query:
SELECT * FROM public.report_campaign_leaflet WHERE day="{today - 1d}"
I´ve found some documentation about dynamic content and some other stuff but no information on how to use date functions directly in a sql query.
Maybe someone has a hint for me?
Thanks & best,
Michael
Here is the possible solution for your problem.
In your copy activity, at the source side, you choose query in Use Query option, and then in the query box you write an expression
Here is the expression #concat('SELECT * FROM public.report_campaign_leaflet WHERE day=','"',formatDateTime(adddays(utcnow(),-1), 'yyyy-MM-dd'),'"')
formatDateTime function will just format the output of addDays(utcnow(),-1) into yyyy-MM-dd format
Again, you can have a parameter in your pipeline processDate for example, and to set this value from expression in trigger definition, and then just to call that parameter in the query. (suggestion)
You need to replace the double quote (") with two single quotes (''):
#concat('SELECT * FROM public.report_campaign_leaflet WHERE day=','''',formatDateTime(adddays(utcnow(),-1), 'yyyy-MM-dd'),'''')
Currently I have a PostgreSQL query which calculates expression:
SELECT sum(timestamp2 - timestamp1 [some other math]) from <...> WHERE <...>.
Is there a way to do it with Slick? I tried to fetch raw data (login and logout) from database and process it, but this method is too expensive, so I need to process data on the database side.
This is how you do calculated columns
https://github.com/slick/slick/issues/1314
So in your case it will be
def ts(row: YourTableClass[_]) = row.timestamp2 -- row.timestamp1
Next you can do aggregation like shown in nmat's link
I'm a beginner to Talend. I have stopped at one point in my requirement.
My requirement is I have to pass the filecount value to MySQL query on LIMIT Clause. Ex: if I can get file count from tfilerowcount item in talend. Once received this value, I have to pass it to tMysqlInput query as
select * from table_name limit "+tFileRowCount_1.filecount+";
You can get the number of rows in a file using below
(Integer)globalMap.get("tFileRowCount_1_COUNT")
Hope this would help you out.
My requirement:
fetch ONE object (e.g RetainInfo ) from table RETAIN_INFO if VERSION column has max value
Does CRUD repository support for an interface method like
findOneByMaxRetVersionAndCountry("DEFAULT")
Equivalent db2 sql:
select RET_ID, max(ri.RET_VERSION) from RETAIN_INFO ri where ri. COUNTRY='DEFAULT' group by RET_ID fetch first 1 rows only;
This query selects an ID, but I would actually want the RetainInfo object corresponding the SINGLE row returned by the query.
I prefer to get that without using custom query, i.e using findBy or some other method/interface supported by Spring CRUD.
You could use limiting in combination with sorting (spring data reference:limit query results). Declare a method similar to the following in your CrudRepository interface :
RetainInfo findTopByCountryOrderByRetVersionDesc(String country);
You can also use findFirst to get the first result. Before getting the result, make sure to use Orderby and then the ascending(Asc) or descending(Desc). As an example if you want to order by version and retrieve based on productName
RetainInfo findFirstByProductNameOrderByVersionDesc(String productName);
Spring Data doesn't provide an expression to select a max value. All supported query parts could be found in the Spring 1.2.0.RELEASE docs: Appendix A. Namespace reference or line 182 of org.springframework.data.repository.query.parser.Part.
Also feel free to create a feature request at Spring's Jira page.