I'm making a Power BI report where data that I need to show is stored in a PostgreSQL database.
In the table where I query there is data from 4 years ago until today, but for my report I only need the last week of records (I know that I can filter using Power BI but my goal is make the lighter query as possible).
The fields of the database related to time, are in UNIX Timestamp, so I'm filtering it now by this way:
SELECT
DATABASE.INCIDENT_NUMBER
,DATABASE.SUBMITTER
,DATABASE.CREATE_DATE
,DATABASE.MODIFIED_DATE
,DATABASE.CLOSED_DATE
,DATABASE.SUBJECT
FROM
DATABASE
WHERE
1643670000 < DATABASE.CREATE_DATE
ORDER BY DATABASE.INCIDENT_NUMBER, DATABASE.CREATE_DATE ASC
That is fine, but I want to improve it making a dynamic query which returns the records from last week till today, without putting a constant of UNIX timestamp.
How can I make that?
That's an excellent example why it is a bad idea to store timestamps as numbers. The correct data type is timestamp or timestamp with time zone.
If you had used the correct data type, your condition would be as simple as
WHERE current_date - 7 < database.create_date
But with numbers, you have to convert back and forth:
WHERE to_timestamp(EXTRACT('epoch' FROM current_date) - 7) < database.create_date
I want to get the maximum of a time series per day, so one data point each day at time 00:00:00. The maximum should be calculated over the range 00:00:00 until 23:59:59 for each day.
What i got so far:
SELECT max("temperature") FROM "Temperature" WHERE $timeFilter GROUP BY time(1d)
($timeFilter is used by Grafana for displaying only the selected time range)
With that query i get the output data points at the wrong time.
EDIT:
When i run
> precision rfc3339
> SELECT max("temperature") FROM "Temperature" WHERE time > now() - 7d GROUP BY time(1d) fill(null)
name: Temperature
time max
---- ---
2020-03-22T00:00:00Z 4.5
2020-03-23T00:00:00Z 9.687
2020-03-24T00:00:00Z 10.75
2020-03-25T00:00:00Z 8.5
2020-03-26T00:00:00Z 11.062
2020-03-27T00:00:00Z 10.25
...
in the CLI, the timestamps seem right.
But in Grafana the data points are placed at 02:00 each day.
Thanks!
Result from the InfluxDB is in the UTC. But Grafana interpolates timestamp to your browser timezone by default (so your browser/local environment reports your local timezone UTC+2). You can change this behavior in the dashboard configuration, for example you can keep timestamps in the UTC:
I think i found a solution myself:
Click '+' next to GROUP BY and select tz(), then enter the desired time zone.
I have dashboard with a single stat panel showing the number of bookings for the current day. I need to show the same information (in order to compare) for the same of the week for last year on the same dashboard.
I know there is the "Time shift" feature in the panel but I don't want to set 364 days because I will not be sure to compare the same day.
Is there an easy way to do that?
Reagards,
If you're using InfluxDB/Flux you could timeShift your data independently from the Grafana UI.
I have a problem with visualization of my data on Kibana 4. In discover menu I have the real date that I have insert, but in visualize menu it's not the same date.
This is my data :
{"index":{"_id":"1"}}
{"data1":912914,"date":"2015-01-01","title":"my-title","data2":2939186,"data3":226447}
{"index":{"_id":"2"}}
{"data1":910145,"date":"2015-01-02","title":"my-title","data2":2936572,"data3":225800}
{"index":{"_id":"3"}}
{"data1":917742,"date":"2015-01-03","title":"my-title","data2":2942237,"data3":226889}
{"index":{"_id":"4"}}
{"data1":911823,"date":"2015-01-04","title":"my-title","data2":2925900,"data3":230025}
And the result in discover menu :
This result is good with the good date, but now in visualize menu :
So, as you can see, on the second image the date is the date from the day before and note the real date as I want.
Someone can help me to resolve this problem?
Thanks in advance
Elasticsearch stores all dates in GMT.
When kibana displays dates, it displays them in your browsers local time (which appears to be offset from GMT by 1 hour).
If you put the times into ES with the correct time zone it may display them properly, but you have not specified how you are getting the data into ES. If it is with logstash, you could try adding a timezone parameter to your date filter.
I have data from a text file I'm reading into a postgres 9.1 table, and the data looks like this:
451,22:30:00,22:30:00,San Jose,1
451,22:35:00,22:35:00,Santa Clara,2
451,22:40:00,22:40:00,Lawrence,3
451,22:44:00,22:44:00,Sunnyvale,4
451,22:49:00,22:49:00,Mountain View,5
451,22:53:00,22:53:00,San Antonio,6
451,22:57:00,22:57:00,California Ave,7
451,23:01:00,23:01:00,Palo Alto,8
451,23:04:00,23:04:00,Menlo Park,9
451,23:07:00,23:07:00,Atherton,10
451,23:11:00,23:11:00,Redwood City,11
451,23:15:00,23:15:00,San Carlos,12
451,23:18:00,23:18:00,Belmont,13
451,23:21:00,23:21:00,Hillsdale,14
451,23:24:00,23:24:00,Hayward Park,15
451,23:27:00,23:27:00,San Mateo,16
451,23:30:00,23:30:00,Burlingame,17
451,23:33:00,23:33:00,Broadway,18
451,23:38:00,23:38:00,Millbrae,19
451,23:42:00,23:42:00,San Bruno,20
451,23:47:00,23:47:00,So. San Francisco,21
451,23:53:00,23:53:00,Bayshore,22
451,23:58:00,23:58:00,22nd Street,23
451,24:06:00,24:06:00,San Francisco,24
It is from a timetable for a commuter rail line, Caltrain. I'm trying to query stations, to get train arrival and departure times. I did this several months ago in MySql, and I got
select * from trains as a, trains as b where a.trip_id=b.trip_id and a.st
op_id='San Antonio' and b.stop_id='San Carlos' and a.arrival_time < b.arrival_ti
me;
So far so good, pretty straightforward. However, when I tried copying the data into a postgres database, I got an error for the various columns that had times after midnight, either 24 or 25:00:00 something. However, if I change them to be 00:00:00 and 01:00:00 something, won't that mess with the query? A time after midnight will appear to be before the starting time? MySql apparently didn't have a problem with those times, and I'm not sure what to do. I'm thinking I should use the last column, or maybe convert the times to something that doesn't take into account PM/AM?
You should try using the interval type for the time columns. Those will keep track of the number of hours, minutes, and seconds instead of trying to record a time of day.
See the PostgreSQL documentation on dates and times.
An interval can have a time component greater than 24 hours, unlike the time datatype that is confined to 00:00 <= x <= 23:59.