i need to select and get rows which are related to previous 1 and 24 hours. to handle 24 hour i used this code. what about 1 hour how can i get that?
yesterday = datetime.now() - timedelta(1)
yesterday = datetime.strftime(yesterday, '%Y-%m-%d')
q.sendSync('{[x;z]select from trades where DateTime > x,symbol=z}', np.datetime64(yesterday,'D'), np.string_('ETH-USDT'))
kdb saved dataframe
DateTime symbol
0 2022-04-13 12:59:00.171 b'ETH-USDT'
1 2022-04-13 12:30:00.171 b'ETH-USDT'
2 2022-04-13 10:55:00.171 b'ETH-USDT'
3 2022-04-12 10:59:00.171 b'ETH-USDT'
4 2022-04-10 10:53:00.185 b'ETH-USDT'
5 2022-04-09 10:50:01.114 b'ETH-USDT'
instead of yesterday you can use this as the time for the query - here we are just subtracting an hour from the current time and formatting it as a date and hour + minute rather than just a day
hourago = datetime.strftime(datetime.now() - timedelta(hours = 1), '%Y-%m-%d %H:%M')
q.sendSync('{[x;z]select from trades where DateTime > x,symbol=z}', np.datetime64(hourago,'ms'), np.string_('ETH-USDT'))
Its also worth noting you can do this purely in kdb without the need for the python date/time manipulation
q.sendSync('{select from trades where DateTime > .z.p-0D01,symbol=x}', np.string_('ETH-USDT'))
.z.p gives the current timestamp and -0D01 subtracts an hour from this time.
Related
How can I get the number of days passed in the current quarter?
For example, if today is 1/2/2021, it will return 2.
If today is 2/5, it will return 36.
If today is 4/2, it will return 2.
Use date_trunc() to get the start of the quarter and subtract dates:
WITH cte(day) AS (
VALUES
(date '2021-01-02')
, (date '2021-02-05')
, (date '2021-04-02')
)
SELECT day
, day - date_trunc('quarter', day)::date + 1 AS days_passed_in_quarter
FROM cte;
day | days_passed_in_quarter
------------+------------------------
2021-01-02 | 2
2021-02-05 | 36
2021-04-02 | 2
+ 1 to fix off-by-one error as you clearly want to include the current day as "passed".
Always use unambiguous ISO 8601 date format (YYYY-MM-DD - 2021-02-05), which is the default in Postgres and always unambiguous, or you depend on the current datestyle setting (and may be in for surprises). Also avoids misunderstandings general communication.
Related:
PostgreSQL: between with datetime
I'm trying to get the date a record/document was inserted into a MongoDB collection. I'm using the ObjectId.getTimestamp() function to get the timestamp from the date.
The function returns an ISODate object, and going through this question, it looks like ISODate is just a wrapper around the native JavaScript Date. But when I use the getDay() and getMonth() methods on the ISODate object, it returns incorrect results, though hours, minutes and year seems fine.
Usage (Mongo shell) -
> db.user.count()
1
> db.user.findOne()._id.getTimestamp()
ISODate("2018-10-26T12:52:31Z")
> db.user.findOne()._id.getTimestamp().getDay()
5
> db.user.findOne()._id.getTimestamp().getMonth()
9
> db.user.findOne()._id.getTimestamp().getFullYear()
2018
> db.user.findOne()._id.getTimestamp().getHours()
12
> db.user.findOne()._id.getTimestamp().getMinutes()
52
>
According to this -
> db.user.findOne()._id.getTimestamp().getDay()
5
> db.user.findOne()._id.getTimestamp().getMonth()
9
day is 5 and month is 9, but it should be 26 and 10 respectively.
Screenshot -
What am I doing wrong?
Date.getDay() returns day of the week (0-6, being 0 - sunday, 1 - monday, etc...), but Date.getDate() returns day of the month.
db.user.findOne()._id.getTimestamp().getDate()
26
For Date.getMonth(), JavaScript Date return values 0 - 11, being 0 - january, 1 - february, etc.. 9 - october.
db.user.findOne()._id.getTimestamp().getMonth() + 1
10
LINK https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date#Getter
Excel file(104976x10) includes large data.
A column: Time (unit year)
B column: Year
C column: Day of the year
D column: Hour
E column: Minute
and others including values
I would like to convert column which begins with B column until E column to date format like 'dd/mm/yyyy HH:MM'.
Example for the data:
1998,41655251 1998 152 1 0 12,5 12,0 11,8 11,9 12,0
I would like to do date instead of 2-th, 3-th, 4-th and 5-th columns.
1998,41655251 01/06/1998 01:00 12,5 12,0 11,8 11,9 12,0
or
1998,41655251 01/06/1998 01:00 1998 152 1 0 12,5 12,0 11,8 11,9 12,0
Welcome to SO.
Matlab has two types of date-format:
datetime, introduced in 2014b.
datenum, introcuced in long ago (before 2006b), it is basically a double precision value giving the number of days from January 0, 0000.
I think the best way is to use datetime, and give it the year, month, day, hour and minute values like this:
t=datetime(1998,0,152,1,0,0)
t= '01-May-1998 01:00:00'
As you can see the days automatically overflow into the months. But I end up 1st of may, not 1st of june like in your example.
to change the format:
t.Format='dd/MM/yyyy hh:mm'
t= '01/05/1998 01:00'
to convert it to a string, you can simply use string(t)
This is an example that combines the above functions to read an xlsx file and writes a new one with the updated column.
data=xlsread('test.xlsx');
S = size(data);
t = datetime(data(:,2),0,data(:,3),data(:,4),data(:,5),0);
t.Format='dd/MM/yyyy HH:mm';
data2=num2cell(data(:,1));
data2(:,2)=cellstr(string(t));
data2(:,3:S(2)-3)=num2cell(data(:,6:end));
xlswrite('test2.xlsx',data2);
I try to understand why
print(pd.Timestamp("2015-01-01") - pd.DateOffset(day=1))
does not result in
pd.Timestamp("2014-12-31")
I am using Pandas 0.18. I run within the CET timezone.
You can check pandas.tseries.offsets.DateOffset:
*kwds
Temporal parameter that add to or replace the offset value.
Parameters that add to the offset (like Timedelta):
years
months
weeks
days
hours
minutes
seconds
microseconds
nanoseconds
Parameters that replace the offset value:
year
month
day
weekday
hour
minute
second
microsecond
nanosecond
print(pd.Timestamp("2015-01-01") - pd.DateOffset(days=1))
2014-12-31 00:00:00
Another solution:
print(pd.Timestamp("2015-01-01") - pd.offsets.Day(1))
2014-12-31 00:00:00
Also it is possible to subtract Timedelta:
print(pd.Timestamp("2015-01-01") - pd.Timedelta(1, unit='d'))
pd.DateOffset(day=1) works (ie no error is raised) because "day" is a valid parameter, as is "days".
Look at the below one: "day" resets the actual day, "days" adds to the original day.
pd.Timestamp("2019-12-25") + pd.DateOffset(day=1)
Timestamp('2019-12-01 00:00:00')
pd.Timestamp("2019-12-25") + pd.DateOffset(days=1)
Timestamp('2019-12-26 00:00:00')
Day(d) and DateOffset(days=d) do not behave exactly the same when used on timestamps with timezone information (at least on pandas 0.18.0). It looks like DateOffset add 1 day while keeping the hour information while Day adds just 24 hours of elapsed time.
>>> # 30/10/2016 02:00+02:00 is the hour before the DST change
>>> print(pd.Timestamp("2016-10-30 02:00+02:00", tz="Europe/Brussels") + pd.offsets.Day(1))
2016-10-31 01:00:00+01:00
>>> print(pd.Timestamp("2016-10-30 02:00+02:00", tz="Europe/Brussels") + pd.DateOffset(days=1))
2016-10-31 02:00:00+01:00
I've seen threads where the document has Start Date and End Date "widgets" where users type in their dates, however, I'm looking for a dynamic solution, for example on the table below, when I select a date, say "1/1/2004", I only want to see active players (this would exclude Michael Jordan only).
Jersey# Name RookieYr RetirementYr Average PPG
23 Michael Jordan 1/1/1984 1/1/2003 24
33 Scotty Pippen 1/1/1987 1/1/2008 15
1 Derrick Rose 1/1/2008 1/1/9999 16
25 Vince Carter 1/1/1998 1/1/9999 18
The most flexible way is to IntervalMatch the RookieYr * RetireYr dates into a table of all dates. See http://qlikviewcookbook.com/recipes/download-info/count-days-in-a-transaction-using-intervalmatch/ for a complete example.
Here's the interval match for your data. You'll can obviously create your calendar however you want.
STATS:
load * inline [
Jersey#, Name, RookieYr, RetirementYr, Average, PPG
23, Michael Jordan, 1/1/1984, 1/1/2003, 24
33, Scotty Pippen, 1/1/1987, 1/1/2008, 15
1, Derrick Rose, 1/1/2008, 1/1/9999, 16
25, Vince Carter, 1/1/1998, 1/1/9999, 18
];
let zDateMin=37000;
let zDateMax=40000;
DATES:
LOAD
Date($(zDateMin) + IterNo() - 1) as [DATE],
year( Date($(zDateMin) + IterNo() - 1)) as YEAR,
month( Date($(zDateMin) + IterNo() - 1)) as MONTH
AUTOGENERATE 1
WHILE $(zDateMin)+IterNo()-1<= $(zDateMax);
INTERVAL:
IntervalMatch (DATE) load RookieYr, RetirementYr resident STATS;
left join (DATES) load * resident INTERVAL; drop table INTERVAL;
There's not much to it you need to load 2 tables one with the start and end dates and one with the calendar dates then you interval match the date field to the start and end field and from there it will work the last join is just to tidy up a bit.
The result of all of that is this ctrl-t. Don't worry about the Syn key it is required to maintain the interval matching.
Then you can have something like this.
Derrick Rose is also excluded since he had not started by 1/1/2004