I want to put in oozie some sqoop commands in order to be executed everyday and fetch data for previous date:
The table has a column date_prof and it has values like:
2020-09-02 05:03:02
2021-02-19 06:04:15
2021-02-10 19:05:20
etc...
Because its timestamp I am trying to have only the yyyy-MM-dd to get only the date, so my query inside sqoop is like:
select * from table date_prof like 'from_uixtime(date_sub(current_date,1),'yyyy-MM-dd')%'
But because of the '' around the function it reads it as string.
convert date_prof to date:
select * from table where date(date_prof) = date_sub(current_date,1)
Related
I have a postgres table "log", which has a column called "timestamp" which has the date and time of files in a folder.
I need to retrieve the latest timestamp from table and pass this in a "for condition" but initially the table will be empty, from second iteration i need to fetch from the table using pyspark
Please let me know how to go about it
so far i tried
log_qry = """select timestamp from log order by timestamp desc limit 1"""
cursor.execute = log_qry
conn.commit
this seems to be not working
Your query should be like below:
select timestamp from log order by timestamp desc limit 1
it will return 0 records if there is no record in your table log.
better you try it with max like below:
select max(timestamp) from log
it will return 1 record always, if table is empty then it will return null else it will return the max timestamp from the column timestamp.
Don't use reserve keywords and column name
timestamp is a reserved word and should be double-quoted if used as a name in a query.
If null is not acceptable for your "for condition" then coalesce it to a date/time very very long ago.
select coalesce(max("timestamp"), '0001-01-01T00:00:00'::timestamp) from "log";
So my table consists of a column of the type: timestamp with timezone.
For example,
This is an entry from the column:
2016-07-01 07:01:03+00
I would like to be able to group using just "2016-07-01" in my grouping-aggregation query.
How can I extract the date from the timestamp column and group using just that?
Thanks.
try ... group by date_column_name::date
I need storage 2 colums with date_time in my postgresql DB table.
date_time date_time_human
676484556463346 09.06.2017 9:38:00
date_time - like oracle timestamp( this date in seconds)
date_time_human - date in normal form
What type of field should be?
INSERT INTO tabl (date_time, date_time_human) VALUES(now(), now())
It seems to me that if you have the same value in these fields, then there is no sense in starting it twice.
Create one field of type timestamp. And in seconds you can output and record using functions.
Data Type Formatting Functions
I have the following Command table in a postgresql 9.3 database:
Command
-------------------------------------------------------------
id | purchased_date | ...
integer NOT NULL | timestamp without time zone NOT NULL | ...
Now I want to fill my table with data from a CSV file. My CSV file contains for example the following values:
1,"2016-04-18 09:37:30"
2,"2016-04-17 09:37:30"
...
When I do \i 'my_csv_file.csv' it works great. What I want to do now is to have the CSV file with dynamic dates in order to not regenerate the CSV file when I want to reload my database (this is for test purposes). I would like to have something like:
1,"CURRENT_DATETIME - INTERVAL '1 DAY'"
2,"CURRENT_DATETIME - INTERVAL '2 DAY'"
But when I execute the same command \i 'my_csv_file.csv' I have the error ERROR: date/time value "current" is no longer supported. Is that possible to do what I want ?
You can use temporary table with offsets, then insert from it.
CSV:
1,"INTERVAL '1 DAY'"
2,"INTERVAL '2 DAY'"
Queries:
create temporary table tmp_table (
id int,
offset_ interval
);
Import csv into this table, then
insert into main_table (id, purchased_date )
(select id, now()-offset_ from tmp_table);
I want to insert date and time in oracle database, I have created the table with columns
create table myadmin
( employe_id number(5),
supervisor Varchar2(20),
department Varchar2(20),
action Varchar2(20),
sdate date,
stime date)
While inserting the values below it gives an error. Please tell me how to insert the time ?
insert into myadmin
( employe_id,supervisor,department,action,sdate,stime) values
(83,'gaurav','helpdesk','pick','23-jan-2013','09:43:00');
You have to use keyword to_date for date insert in oracle like this.
to_date('23-01-2013','dd-mm-yyyy')
Basically you have to use keyword to_date('your date','your date format').
You can also add date and time together if you want and it would be something like this
to_date('23-01-2013 09:43:00','dd-mm-yyyy hh24:mi:ss')
A date in Oracle always has a date part and a time part. Having date and time in two separate columns only makes sense, if it can occur that date is null and time is not. (And still, you could set date to an improbable value like 1.1.0001 then.)
However, if you want to stick to those two separate fields, then make your string a datetime with the to_date function specifying the format used:
insert into myadmin
( employe_id,supervisor,department,action,sdate,stime) values
(83,'gaurav','helpdesk','pick',to_date('23-01-2013','dd-mm-yyyy'), to_date('09:43:00', 'hh24:mi:ss'));