I have a collection with $vehicleId and $Scraped Date. I am trying to get the avg days a car is in inventory. And I want to calculate it for all the historical days.
Sample Doc
{"_id":{"$oid":"5e1b46d853848fae2832e01a"},"Scraped Date":{"$date":{"$numberLong":"1578845911324"}},"vehicleId":{"$numberInt":"1376788"}}
{"_id":{"$oid":"5e1b46d853848fae2832e01b"},"Scraped Date":{"$date":{"$numberLong":"1578845911324"}},"vehicleId":{"$numberInt":"1376771"}}
{"_id":{"$oid":"5e1b46d853848fae2832e01c"},"Scraped Date":{"$date":{"$numberLong":"1578845911324"}},"vehicleId":{"$numberInt":"1376734"}}
{"_id":{"$oid":"5e1b46d853848fae2832e01d"},"Scraped Date":{"$date":{"$numberLong":"1578845911324"}},"vehicleId":{"$numberInt":"1376706"}}
{"_id":{"$oid":"5e1b46d853848fae2832e01e"},"Scraped Date":{"$date":{"$numberLong":"1578845911324"}},"vehicleId":{"$numberInt":"1376505"}}
collection.aggregate([
{'$group': {
'_id' : {'vehicleId': '$vehicleId'},
'date' : {'$addToSet': "$Scraped Date"}
} }
]
)
This code is giving me a list of dates the vehicleId was found in the inventory. How can I convert this to list of dates with avg length the cars were in inventory for that day? I could think of finding the avg length of the dates column but that wont give the me the data day wise.
The current output looks like this in a dataframe:
dataframe view
I figured out a solution. Created a simple for loop for every date and then used the $match query to first filter the results and then calculate the avg length. The question is closed for now. I will update the code in the original question in a while
Related
This question already has an answer here:
How to compare dates or date against today with query on google sheets?
(1 answer)
Closed 5 months ago.
So I have a table that looks like this:
date
goal
10/1/2022
10000
10/2/2022
10000
10/3/2022
10000
10/4/2022
10000
10/5/2022
10000
10/6/2022
10000
I would like to create a formula that Sums the goal column for the dates less than or equal to the current day. I currently have this:
=query(A4:B1000, "select SUM(B) WHERE A <= today()")
But this is throwing the following error: Unable to parse query string for Function QUERY parameter 2: PARSE_ERROR: Encountered " "(" "( "" at line 1, column 31. Was expecting one of: <EOF> "group" ... "pivot" ... "order" ... "skipping" ... "limit" ... "offset" ... "label" ... "format" ... "options" ... "and"
Any thoughts on how to proceed would be helpful. Thanks!
This is a typical use case for SUMIF, which will sum values in a range that meet a condition (a condition on that same range, or on a corresponding range):
=sumif(A4:A1000,"<="&TODAY(),B4:B1000)
will sum the values in B4:B1000 for which the corresponding value in A4:A1000 is less than or equal to today.
Or you can still use a query if you prefer as long as you get the right syntax - see this for example.
=query(A2:B1000, "select SUM(B) WHERE A <= date '"& text(today(),"yyyy-mm-dd")&"'")
Dates in my locale are in dd/mm/yyyy format so 6/10/2022 is tomorrow at time of writing.
Let there be a MongoDB collection Data that holds the history of temperatures of some items. So the items are like this:
{
"itemId" : "ABCD",
"timePoint" : NumberLong("1618922410288"),
"temperature" : 15.15
}
meaning, that the item "ABCD" had a temperature 15.15 at 1618922410288ms since the epoch.
What is the query that will result in the history of some span-average temperatures of the given item starting from the given time point?
So e.g. for itemId="ABCD", span=60*60*1000, startingTimePoint=0 it has to be a query that will result to the hourly-avarage temperatures of "ABCD" starting throm 1970-01-01 00:00:00.000.
I would also like the query not to be "welded" with the average, but rather accept an arbitrary aggregation function, so it will be easy to use the same technique for avg, min, max, or something else. But this is the second question.
How can i filter database entries that have a datetime less than 60min in the past?
I tried some date operations as follows in mongodb with two fields timestamp and marketstartime that are of type date in all my documents:
{"$subtract": ["$timestamp", "$marketstartime"]}
but it returns always null for that operation. Why?
My timestamp and marketstartime entries in the db are in date type and look as follows, this should be correct:
2017-12-23 12:00:00.000Z
The actual question I’m trying to solve: How can I get all entries that have a timestamp less than 60 min in the past from now?
A query can composed for documents with timestamp value set less than 60 minutes ago.
from datetime import datetime, timedelta
query = {
'$timestamp': {
'$lt': datetime.now() + timedelta(minutes=-60)
}
}
cursor = db.collection.find(query)
assuming I have the following nested document structure, where my document contains nested routes with an array of date time values.
{
property_1: ...,
routes: [
{
start_id: 1,
end_id: 2,
execution_times: ['2016-08-28T11:11:47+02:00', ...]
}
]
}
Now I could filter my documents that match certain execution_times with something like this.
query: {
filtered: {
query: {
match_all: { }
},
filter: {
nested: {
path: 'routes',
filter: {
bool: {
must: [
{
terms: {
'routes.execution_times': ['2016-08-28T11:11:47+02:00', ...]
}
},
...
]
}
}
}
}
}
}
But what if I would like to filter my documents based on execution dates. What's the best way achieving this?
Should I use a range filter to map my dates to time ranges?
Or is it better to use a script query and do a conversion of the execution_times to dates there?
Or is the best way to change the document structure to contain both, the execution_date and execution_time?
Update
"The dates are not a range but individual dates like [today, day after tomorrow, 4 days from now, 10 days from now]"
Well, this is still a range as a day means 24 hours. So if you store your field as date time, you can use leverage range query : from 20-Nov-2010 00:00:00 TO 20-Nov-2010 23:59:59 with appropriate time zone for a specific day.
If you store it as a String then you will lose all the flexibility of date maths and you would be able to do only exact String matches. You will then have to do all the date manipulations at the client side to find exact matches and ranges.
I suggest play with range queries using Sense plugin and I am sure it will satisfy almost all your requirements.
-----------------------
You should make sure that you use appropriate date-time mapping for your field and use range filter over that field. You don't need to split into 2 separate fields. Date maths will allow you to query just based on date.
This will make your life much easier if you want to do aggregations over date time field.
Reference:
Date Maths:
https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#date-math
Date Mapping : https://www.elastic.co/guide/en/elasticsearch/reference/current/date.html
Date Range Queries:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html
if i want to find a document created on a specific Day, until now i used a range
from the first minute of the day, to the last minute of the day in seconds , sth like :
query":{"dtCreated":{"$gte":{"sec":1381356782,"usec":0},"$lt":{"sec":1389356782,"usec":0}}}
is is possible to to somehow find all documents where only the Day, Month and year equals "dtCreated" ?
in pseudocode like :
query:{"dtCreated":ISODate("2014-01-23")} <- i know that may not be a valid iso date
but what i want is to find all documents for one day without using lt and gt ?
Sry for bad english and for any hints thanks in advance!
You can do it with the aggregation framework using the date aggregation operators.
Assuming dtCreated is an ISODate field, you could try something like this:
query = [
{
'$project': {
'year': {'$year':'$dtCreated'},
'month': {'$month':'$dtCreated'},
'day':{'$dayOfMonth':'$dtCreated'}
}
},
{
'$match' : {'year':'2014', 'month':'1', day:'1'}
}
]
db.mycollection.aggregate(query)
Edit: as orid rightly remarks, though this is an answer to your question (query for date without using date range), it's not a good way to solve your problem. I would probably do it this way: a range greater than or equal to today, but less than tomorrow
db.foo.find({'dtCreated':{'$gte':ISODate("2014-01-23"), '$lt':ISODate("2014-01-24")}})