I have a large data and in that one field be like Wed Sep 15 19:17:44 +0100 2010 and I need to insert that field in Hive.
I am getting troubled for choosing data type. I tried both timestamp and date but getting null values when loading from CSV file.
The data type is a String as it is text. If you want to convert it, I would suggest a TIMESTAMP. However you will need to do this conversion yourself while loading the data or (even better) afterwards.
To convert to a timestamp, you can use the following syntax:
CAST(FROM_UNIXTIME(UNIX_TIMESTAMP(<date_column>,'FORMAT')) as TIMESTAMP)
Your format seems complex though. My suggestion is to load it as a string and then just do a simple query on the first record until you get it working.
SELECT your_column as string_representation,
CAST(FROM_UNIXTIME(UNIX_TIMESTAMP(<date_column>,'FORMAT')) as TIMESTAMP) as timestamp_representation
FROM your_table
LIMIT 1
You can find more information on the format here: http://docs.oracle.com/javase/6/docs/api/java/text/SimpleDateFormat.html
My advice would be to concat some substrings first and try to convert only the day, month, year part before you look at time and timezone et cetera.
Related
I am extracting three values (server, region, max(date)) from my postgresql> But I want to extract an additional 4th field which should be the numerical addition of 1 to 3rd field. I am unable to use date add function as in the database date field is defined as an integer.
date type in DB
date|integer|not null
tried using cast and date add function
MAX(s.date)::date + cast('1 day' as interval)
Error Received
ERROR: cannot cast type integer to date
Required output
select server, region, max(alarm_date), next date from table .....
testserver, europe, 20190901, 20190902
testserver2, europe, 20191001, 20191002
next date value should be the addition to alarm_date
To convert an integer like 20190901 to a date, use something like
to_date(CAST(s.date AS text), 'YYYYMMDD')
It is a bad idea to store dates as integers like that. Using the date data type will prevent corrupted data from entering the database, and it will make all operations natural.
First solution that came to my mind:
select (20190901::varchar)::date + 1
Which output 2019-09-02 as type date.
Other solutions can be found here.
I'm new to PostgreSQL and I have the following question:
I have a table with just an id-column and a data-column, which uses the jsonb-type. Inside the jsonb-object I have a datetime field. I read in various posts, that I should use the ISO-8601 dateformat to store in the DB.
I want to filter my table by date like this:
SELECT * FROM table WHERE data->'date' > '2016-01-01T00:00'
Is this really the best date-format for this purpose?
Thanks in advance :)
IMHO Your query should produce
ERROR: operator does not exist: jsonb > timestamp with time zone
If I get it right. In case you change -> to ->> it should get a text value instead of jsonb field (which is also not comparable to timestamp).
It should be smth like
SELECT * FROM table WHERE (data->>'date')::timestamptz > '2016-01-01T00:00' to work
The big advantage of that format is that string order corresponds to date order, so a comparison like the one you quote in your question would actually work as intended.
A second advantage is that a timestamp in that format can easily be converted to a PostgreSQL timestamp with time zone value, because the type input function understands this format.
I hope you are not dealing with dates “before Christ”, because it wouldn't work so easily with those.
I am building a map in CartoDB which uses Postgres. I'm simply trying to display my dates as: 10-16-2014 but, haven't been able to because Postgres includes an unneeded timestamp in every date column.
Should I alter the column to remove the timestamp or, is it simply a matter of a (correct) SELECT query? I can SELECT records from a date range no problem with:
SELECT * FROM mytable
WHERE myTableDate >= '2014-01-01' AND myTableDate < '2014-12-31'
However, my dates appear in my CartoDB maps as: 2014-10-16T00:00:00Z and I'm just trying to get the popups on my maps to read: 10-16-2014.
Any help would be appreciated - Thank you!
You are confusing storage with display.
Store a timestamp or date, depending on whethether you need time or not.
If you want formatted output, ask the database for formatted output with to_char, e.g.
SELECT col1, col2, to_char(col3, 'DD-MM-YY'), ... FROM ...;
See the PostgreSQL manual.
There is no way to set a user-specified date output format. Dates are always output in ISO format. If PostgreSQL let you specify other formats without changing the SQL query text it'd really confuse client drivers and applications that expect the date format the protocol specifies and get something entirely different.
You have two basic options.
1 Change the column from a timestamp to a date column.
2 Cast to date in your SQL query (i.e. mytimestamp::date works).
In general if this is a presentation issue, I don't usually think that is a good reason to muck around with the database structure. That's better handled by client-side processing or casting in an SQL query. On the other hand if the issue is a semantic one, then you may want to revisit your database structure.
Please pardon my ignorance if I have missed any documentation/solution for the same. But I searched the web and could not find an answer.
I have a simple question. In the DB2 table,I have a column of type date and the with data of format 04/25/2013 12:00:00AM . When I query the DB2 database, I want to obtain just the date and not the timestamp i.e to obtain "04/25/2013" and not "04/25/2013 12:00:00AM". I tried DATE(column name) and just gave back the complete value including the time stamp.
This looks like a TIMESTAMP and not a DATE column. If it is indeed a TIMESTAMP column try this:
select varchar_format(current timestamp, 'MM/DD/YYYY') from sysibm.sysdummy1 ;
Just replace the current timestamp in the above example with your column and sysibm.sysdummy1 with your table.
The good thing about varchar_format is that it lets you easily format the timestamp. Just change the 'MM/DD/YYYY' part to 'YYYY.MM.DD' to get a format like '2017.08.18'.
I have create one field in sql server database as nvarchar datatype and store some date like 'd/MM/yyyy' and 'dd/MM/yyyy' format previously. Now i want to get all data in 'dd/MM/yyyy' format using query it is possible?
You can cast the field to datetime in the query:
select cast(YourField as datetime)
from YourTable
where isdate(YourField) = 1
The where isdate(YourField) = 1 part is necessary to filter out rows where the value is no valid date (it's a nvarchar field, so there could be things like abc in some rows!)
But you should really change the field to datetime in the long term, as already suggested by Christopher in his comment.
Casting like described above is always error-prone because of the many different data formats in different countries.
For example, I live in Germany where the official date format is dd.mm.yyyy.
So today (December 9th) is 9.12.2011, and running select cast('9.12.2011' as datetime) on my machine returns the correct datetime value.
Another common format is mm/dd/yyyy, so December 9th would be 12/9/2011.
Now imagine I have a nvarchar field with a date in this format on my German machine:
select cast('12/9/2011' as datetime) will return September 12th (instead of December 9th)!
Issues like this can easily be avoided by using the proper type for the column, in this case datetime.