I've stumbled on this page in PostgreSQL wiki, where it's advised to not use BETWEEN with timestamps:
Why not?
BETWEEN uses a closed-interval comparison: the values of both ends of
the specified range are included in the result.
This is a particular problem with queries of the form
SELECT * FROM blah WHERE timestampcol BETWEEN '2018-06-01' AND
'2018-06-08'
This will include results where the timestamp is exactly 2018-06-08
00:00:00.000000, but not timestamps later in that same day. So the
query might seem to work, but as soon as you get an entry exactly on
midnight, you'll end up double-counting it.
Can anyone explain how this "double-counting" can occur?
Often when using BETWEEN, you want to use multiple ranges (in separate queries) which cumulatively cover all the data.
If the next invocation of your query uses BETWEEN '2018-06-08' AND '2018-06-015', then exact midnight of 2018-06-08 will be included in both invocations. On the other hand, if the next one uses BETWEEN '2018-06-09' AND '2018-06-015', then all of 2018-06-08 except for exact midnight has been overlooked. There is no easy way (using BETWEEN) to construct ranges which cover every data point once and only once.
Related
I am trying to use parameter variation in AnyLogic. My inputs are 3 parameters, each varying 5 times. My output is water demand. What I need from parameter variation is the way in which demand changes according to the different combinations of the three parameters. I imagine something like: there are 10,950 rows (one for each day), the first column is time (in days), the second column are the values for the first combination, the second column is the second combination, and so on and so forth. What would be the best way to track this metadata to then be able to export it to excel? I have added a "dataset" to my main to track demand through each simulation, but I am not sure what to add to the parameter variation experiment interface to track the output across the different iterations. It would also be helpful to have a way to know which combination of inputs produced a given output (for example, have the combination be the name for each column). I see that there are Java Actions, but I haven't been able to figure out the code to do what I need. I appreciate any help with this matter.
The easiest approach is just to track this in output database tables which are then exported to Excel at the end of your run. As long as these tables include outputs from multiple runs (and are, for example, only cleared at the start of the experiment not the run), your Parameter Variation experiment will end up with an Excel file having outcomes from all the runs. (You will probably need to turn off parallel execution in the PV experiment so you don't run into issues trying to write to the same Excel file in parallel.)
So, for example, you might have tables:
run_details with columns id, parm1, parm2 and parm3 (with proper column names given your actual parameters and some unique ID generated for each run)
output_demand with columns run_id, sim_time_hrs and demand_value (if, say, you're storing some demand value each hour of simulated time) where run_id cross-references the run's ID in run_details
(There is extra complexity in how you could allocate a unique run ID and how and when you write to/clear those tables, but I'm just presenting the core design. You can also get round the need-serial-execution point by programmatically controlling when you export to Excel, rather than using the built-in "Export tables at the end of model execution" capability, but that's also more complicated.)
I would like to exclude a time range from a RRULE.
For example the RRULE:
DTSTART;VALUE=DATE-TIME:20210104T142711Z
RRULE:FREQ=HOURLY;INTERVAL=2
will create a recurrent event every two hours, but I would like to exclude a period of time during the night, for example from 22:00 to 6:00.
I have read how to exclude dates using the EXDATE property, but I have not found information in the RFC about how to exclude a period of time. Is this documented somewhere or is there a recommended way of doing this?
I suggest try
RRULE:FREQ=DAILY;BYHOUR=6,8,10,12,14,16,18,20
and test in all the applications you expect to deal with this. Some calendar applications don't do FREQ=HOURLY. EG:Last I looked, google didn't do FREQ=HOURLY. I don't know about BYHOUR. Test whether the applications will do BYHOUR?
This https://icalendar.org/iCalendar-RFC-5545/3-3-10-recurrence-rule.html explains the limiting or expanding of the number of recurrences.
If those are not accepted by your intended receiving applications, then another approach may be to generate multiple VEVENTS for each hour, all using FREQ=DAILY.
I'm building a Grafana dashboard with some Stat panels that show average, minimum, and maximum time values (see below) for specific fields in my database. I'm storing the data in seconds and setting the value's units to seconds after which the panel displays the time in weeks, days, hours, etc. For the sake of consistency, I would like everything to be shown in days but I haven't been able to find a way to force units for the output value. If it's possible to do this, could someone please point me to some docs, or something that could show me which configurations to make in my panels?
So far, I've tried (without success):
Configure each panel to use units of days
The result of this was that everything showed up in years, etc.
Configure each panel to create a new field by performing a binary calculation where I converted from seconds to days and then I updated the units to be days
The result was that the values were not changed at all -> instead of showing X days, or whatever, it just showed the value in seconds without the units. I'm not sure what I messed up there, but it didn’t change anything.
I found this link that discusses setting a time range for queries
This didn’t end up being useful for what I was trying to do because it was actually geared towards changing the query to a specific date range rather than the output.
I looked through the transformations documentation, stat panel documentation, and a few other panel documentation pages in an effort to see if there was any information on how to do it but I was unable to find anything on forcing the output value to use a specific unit.
Edit:
So I kept messing around with the dashboard and got a solution that works - i.e. it's a "good-enough" solution (see below) - but, now, I'm curious if it's possible to show the units along with the value without it converting it to some other unit. Does anyone have any ideas about this?
One thing to note is that the data for this image is different than the data for the previous one so I'm expecting an inexact conversion to days.
You can use a custom unit. It is a bit tricky to enter the unit in the UI because of the automatic selection but if you enter i.e. "days" (without the quotes) in the Unit field ans instead of leving the field with tab or mouse click use the scrollbar of the combobox and select the last entry "Custom Unit: days".
Hope it helps. And for the record: I am using Garafana 7.1.4
Central to a project I'm working on is a highlighting-mechanic that can be applied to certain items on the website. The idea is, that this highlighted-status is only active for a certain amount of time.
I'm trying to find the most performant way to achieve this (in querying, setting status, checking status and revoking it)
A first approach would be to set simply set a value 'highlighted:true' to the item. This seems to be the most performant way to query for highlighted items. The Drawback I see here, is that there also needs to be stored a date for the highlighting-action, but furthermore there needs to run an interval to check on the highlighted items and potentially revoke their highlighted status. Also the exact moment when the item stops beeing highlighted can't be determined exactly, since its depending on the interval of the check-function.
A second approach would be to mainly store the date of the highlighting-action and run the query against it. It seems that the query of highlighted objects is way less performant, since every item ever is beeing checked, and on top its not just a boolean, but a proper function that throws those differnt date-values around to check if it is still valid. On the upside there is no external cleanup-function neccessary and every highlighting period ends perfectly on time.
Would love to have your input on this. Is there maybe a clever pattern on this?
I have created a complex sql server 2008/coldfusion search page, that searches thru a variety of tables.
On the left is a list of the categories, plus an everything category, by each category or type of result is a total number of results of that type found in the current search result.
I have everything fine, but I am hoping there is a more optimal approach.
Because everytime i filter the search to a specific category, i still have to get all the results, so as to make sure the everything category has the correct totals.
And because of this, I have realized this is a problem I've had in lots of other programs in coldfusion/sql.
Where you want to reduce the number of results by some field in the select, but you need to keep the original recordcount total.
But you really don't want to re-run the whole massive query everytime, when you just need to get the trimmed results.
This program is 1 cfc, 1 cfm, 1 stored procedure, and jquery/ajax inside the cfm to call the cfc.
The cfm calls the cfc when it originally get's a form submitted search request, and then any filtering does the same thing.
However if there are more than 20 results then it show's a button at the bottom to do via ajax get 20 more records.
My main goal is to improve performance, make sure i keep an accurate record of what the record count is before any filtering is done, without having to rerun the unfiltered query every time.
This is a kind of complex problem, so there might not be any answers...
Thank you all for trying..
I would run the "big" query once, then pop it into a SESSION variable. Then I'd use Query-of-Query to return subsets based on filters.
The main query always exists, so you can query against that or use metadata like bigQuery.recordCount. Your QofQ is a smaller set of data you can use for display. And you can re-apply filters without having to return to the database.
Well you need to run the query (or a count(*)) at least once to get the total number. You could:
Cache this query and refer to the
cached query's recordcount again
and again
Store the record count in the session scope until the next time it is run for this user