I need to calculate DATEDIFF in minutes between 2 columns of timestamp type.
There are so many simple examples on the web, but none of them work really much properly using psycopg2 + sqlalchemy.
I've tried:
from sqlalchemy import as sa
from datetime import datetime
# con is a standard pool of connections :class:Connection
con.execute(
sa.func.datediff(
sa.literal_column('minute'),
datetime.utcnow(),
datetime.utcnow(),
)
)
it throws:
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) column "minute" does not exist
LINE 1: SELECT datediff(minute, '2017-02-27T15:04:33.217559'::timest...
^
[SQL: 'SELECT datediff(minute, %(datediff_2)s, %(datediff_3)s) AS datediff_1'] [parameters: {'datediff_3': datetime.datetime(2017, 2, 27, 15, 4, 33, 217596), 'datediff_2': datetime.datetime(2017, 2, 27, 15, 4, 33, 217559)}]
if I try:
con.execute(
sa.func.datediff(
'minute',
datetime.utcnow(),
datetime.utcnow(),
)
)
I receive:
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) function datediff(unknown, timestamp without time zone, timestamp without time zone) does not exist
LINE 1: SELECT datediff('minute', '2017-02-27T12:27:49.369724'::time...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
[SQL: 'SELECT datediff(%(datediff_2)s, %(datediff_3)s, %(datediff_4)s) AS datediff_1'] [parameters: {'datediff_4': datetime.datetime(2017, 2, 27, 12, 27, 49, 369740), 'datediff_2': 'minute', 'datediff_3': datetime.datetime(2017, 2, 27, 12, 27, 49, 369724)}]
Any ideas how to make it right?
PostgreSQL does not have a datediff function. To get the number of minutes, use the SQL expression:
trunc((extract(epoch FROM newer_date) - extract(epoch FROM older_date)) / 60)
extract(epoch FROM …) converts the timestamp to number of seconds.
/ 60 converts to seconds to minutes
trunc(…) removes the fractional part.
So probably try
sa.func.trunc((
sa.extract('epoch', datetime.utcnow()) -
sa.extract('epoch', datetime.utcnow())
) / 60)
Related
I have a string field that is being pulled from a table and I'm trying to cast that as a date in a view I created. I keep getting an error when trying to Cast as a date though. The format of the field looks like this:
July 19, 2020 or
August 8, 2020 etc..
I get an error that states
"Failed to parse input string "July 19, 2020"
or one of the other dates in the data when trying to use DATE_Parse. Or I get
"Invalid Date:August 8, 2020"
if I try to use the CAST function.
Below is my query when trying to CAST the date:
select
noteattributes.value.name as name_type, noteattributes.value.value as name_value, CAST(noteattributes.value.value as DATE) as DATE_TEST, order_number
from test.orders,
unnest(note_attributes) as noteattributes
where noteattributes.value.name = 'Pickup-Date'
Convert String to Date
Below is for BigQuery Standard SQL
You should use PARSE_DATE instead of CAST as in below example
PARSE_DATE('%B %d, %Y', date_as_string)
You can test, play with this using example below
#standardSQL
WITH `project.dataset.table` AS (
SELECT 'July 19, 2020' date_as_string UNION ALL
SELECT 'August 8, 2020'
)
SELECT PARSE_DATE('%B %d, %Y', date_as_string) AS date_as_date
FROM `project.dataset.table`
with output
Row date_as_date
1 2020-07-19
2 2020-08-08
I am having a MongoDb collection of 70M items (200GB) and I am trying to get those in the range [from_date, to_date] using the command:
from_date = datetime.datetime(2015, 1, 14, 9, 46, 23)
to_date = datetime.datetime(2015, 1, 14, 9, 46, 24)
db.collection.find({"datetime": {"$gte": from_date, "$lte": to_date}})
However it takes a lot of time to retrieve those items even for a single query. Is there any more efficient way to do this?
How can I insert 2016-08-29-23.17.147253 in db2 timestamp column?
INSERT INTO TEST VALUES (2015,2016-08-29-23.17.147253, 5000 , 0, 0, 0, 0, 0, 0, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP )
CURRENT_TIMESTAMP is working fine, 2016-08-29-23.17.147253 not
Your format of timestamp seems incorrect. It must be "A timestamp is a seven-part value representing a date and time by year, month, day, hour, minute, second, and microsecond". Also, enclose the timestamp value in single quotes in INSERT statement. For more infor check :
http://www.ibm.com/support/knowledgecenter/SSEPEK_10.0.0/intro/src/tpc/db2z_datetimetimestamp.html
Try with quote like this:
INSERT INTO TEST VALUES (2015, '2016-08-29-23.17.147253', 5000 , 0, 0, 0, 0, 0, 0, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP )
On the mongo shell this returns a document just fine:
> db.orderbook_log.findOne({'time': { '$gte': new Date(2014, 9, 24, 17, 38, 20, 546000), '$lt': new Date(2014, 10, 24, 17, 39, 20, 546000)}})
//... returns document with this time stamp:
"time" : ISODate("2014-10-25T00:47:30.819Z")
Notice I used "9" for October because JavaScript's months are 0-11.
And I also tested with "23" as the day because it looks like JS days are also 0-indexed, and that also returned a document: "time" : ISODate("2014-10-24T17:32:13.595Z")
atime = datetime.datetime(2014, 10, 24, 17, 38, 20, 546000)
btime = datetime.datetime(2014, 10, 24, 17, 39, 20, 546000)
future_book = log.find_one({"time": {"$gte": atime, "$lt": btime}})
But when I execute find_one in pymongofuture_book is None
All I'm really trying to do is loop though the first 100 records or so and get a record that occurred a relative minute later.
Javascript days are not zero-indexed, alas. Only months are.
I see in your Javascript you're adding 546,000 ms to the first date, so that results in 2014-10-24 at 17:48:26. Javascript then converts to your local timezone, so in my case it adds 5 hours:
> new Date(2014, 10, 24, 17, 39, 20, 546000)
ISODate("2014-11-24T22:48:26Z")
This is then compared (ignoring timezones) with the "time" field in your documents.
Better to remove the final milliseconds argument, and use the MongoDB shell's ISODate function, which is designed to be more rational than Javascript Dates:
> ISODate('2014-10-24T17:38:20')
ISODate("2014-10-24T17:38:20Z")
That will then compare to your documents in the way you expect, and it should match the PyMongo behavior. I suggest you drop the milliseconds argument from your Python datetime constructors, too, to clarify things.
Given a specific DateTime, and using just SQL, how can I determine if Daylight Savings Time will be in effect on that day? This is purely for "local" use, so I don't need to worry about it working on different environments or locales, just my own SQL Server 2008R2 server.
So, for example,
declare #myDate datetime='2 Jun 2014 14:00'
and then somehow evaluating #myDate would return a result that indicates "yes, this is in DST", where as
declare #myDate datetime='2 Dec 2014 14:00'
... would be "no, this isn't in DST".
The datetime specified will only be during working hours (8am-8pm), so I'm not too concerned about values that fall "in between" DST and non-DST. I'm in the UK, which goes between UTC and UTC+1.
Seems to be answered here:
Get Time zones in SQL Server 2008 R2
Essentially you need to load the Olson timezone database, or some subset of it, into your SQL server database and then you can access it using SQL.
I've cobbled together the following code, which makes the (British) assumptions that DST starts on the last Sunday in March, and ends in the last Sunday in October. It's fairly specific to my requirements, but seems to do the job - if anyone knows of a more elegant solution...
declare #myDate datetime='2 Jun 2014'
-- Declare October in this year
declare #october datetime=DATEADD(month,9, DATEADD(yy, DATEDIFF(yy,0,#myDate), 0))
-- Declare March in this year
declare #march datetime=DATEADD(month,3, DATEADD(yy, DATEDIFF(yy,0,#myDate), 0))
-- Find last Sunday in March
declare #DSTStart datetime=DATEADD(dd,
-DATEPART(WEEKDAY,
DATEADD(dd, -DAY(DATEADD(mm, 1, #march)),
DATEADD(mm, 1, #march))) + 1,
DATEADD(dd, -DAY(DATEADD(mm, 1, #march)), DATEADD(mm, 1, #march)))
-- Find last Sunday in October
declare #DSTEnd datetime= DATEADD(dd,
-DATEPART(WEEKDAY,
DATEADD(dd, -DAY(DATEADD(mm, 1, #october)),
DATEADD(mm, 1, #october))) + 1,
DATEADD(dd, -DAY(DATEADD(mm, 1, #october)), DATEADD(mm, 1, #october)))
if (#mydate BETWEEN #DSTStart and #DSTEnd) print 'DST'