Implementation: 22.200.001
Interface: Rest API
Request: GET
Params: $filter
Value: LastModifiedDateTime gt "yyyy-MM-ddThh:mm:ss.000"
EndPoints: All
Problem - When requesting any records greater than LastModifiedDateTime the system returns records that are within the span of the date (e.g. 0:00 hrs to 23:59.999") requested instead of any records that are ACTUALLY greater than the datetime requested.
Question - Is this by design or is this a bug? If by design, why store the time? Why report time down to milliseconds?
Programming Issue - Causes undo update SQL calls for records that do not need to be updated. Yes, one could query prior to updating but that defeats the purpose of the filter.
Any insight to this issue would be greatly appreciated.
No programming issue but rather an implementation question.
Related
the question I'd like to ask was raised some time ago (FIWARE Orion: How to retrieve the servicePath of an entity?) but as far as I've seen, there is no final answer.
In short, I'd like to retrieve the service path of entities when I exec a GET query to /v2/entitites which returns multiple results.
In our FIWARE instance, we strongly rely on the servicePath element to differentiate between entities with the same id. It is not a good design choice but, unfortunately, we cannot change it as many applications use that id convention at the moment.
There was an attempt three years ago to add a virtual field 'servicePath' to the query result (https://github.com/telefonicaid/fiware-orion/pull/2880) but the pull request was discarded because it didn't include test coverage for that feature and the final NGSIv2 spec didn't include that field.
Is there any plan to implement such feature in the future? I guess the answer is no, what brings me to the next question: is there any other way to do it which does not involve creating subscriptions (we found that the initial notification of a subscription does give you that info, but the notification is limited to 1000 results, what is too low for the number of entities we want to retrieve, and it does not allow pagination either)?
Thanks in advance for your responses.
A possible workaround is to use an attribute (provided by the context producer application) to keep the service path. Somehow, this is the same idea of the builtin attribute proposed in PR #2880.
I'm pretty confused concerning this hip thing called NoSQL, especially CloudantDB by Bluemix. As you know, this DB doesn't store the values chronologically. It's the programmer's task to sort the entries in case he wants the data to.. well.. be sorted.
What I try to achive is to simply get the last let's say 100 values a sensor has sent to Watson IoT (which saves everything in the connected CloudantDB) in an ORDERED way. In the end it would be nice to show them in a D3.css style kind of graph but that's another task. I first need the values in an ordered array.
What I tried so far: I used curl to get the data via PHP from https://averylongID-bluemix.cloudant.com/iotp_orgID_iotdb_2018-01-25/_all_docs?limit=20&include_docs=true';
What I get is an unsorted array of 20 row entries with random timestamps. The last 20 entries in the DB. But not in terms of timestamps.
My question is now: Do you know of a way to get the "last" 20 entries? Sorted by timestamp? I did a POST request with a JSON string where I wanted the data to be sorted by the timestamp, but that doesn't work, maybe because of the ISO timestamp string.
Do I really have to write a javascript or PHP script to get ALL the database entries and then look for the 20 or 100 last entries by parsing the timestamp, sorting the array again and then get the (now really) last entries? I can't believe that.
Many thanks in advance!
I finally found out how to get the data in a nice ordered way. The key is to use the _design api together with the _view api.
So a curl request with the following URL / attributes and a query string did the job:
https://alphanumerical_something-bluemix.cloudant.com/iotp_orgID_iotdb_2018-01-25/_design/iotp/_view/by-date?limit=120&q=name:%27timestamp%27
The curl result gets me the first (in terms of time) 120 entries. I just have to find out how to get the last entries, but that's already a pretty good result. I can now pass the data on to a nice JS chart and display it.
One option may be to include the timestamp as part of the ID. The _all_docs query returns documents in order by id.
If that approach does not work for you, you could look at creating a secondary index based on the timestamp field. One type of index is Cloudant Query:
https://console.bluemix.net/docs/services/Cloudant/api/cloudant_query.html#query
Cloudant query allows you to specify a sort argument:
https://console.bluemix.net/docs/services/Cloudant/api/cloudant_query.html#sort-syntax
Another approach that may be useful for you is the _changes api:
https://console.bluemix.net/docs/services/Cloudant/api/database.html#get-changes
The changes API allows you to receive a continuous feed of changes in your database. You could feed these changes into a D3 chart for example.
I have a publication where I send a "record set" of items. Among these items, some have a field with a modification date (Type: Date).
I need to compare the date field with the current date in order to allow/forbid a user interface action. If my date is more than 24hours ago, the action is forbidden.
Initially, I wanted to create a dedicated publication in order to expose only the _id of the items with a Date field inferior to 24h from now.
When reading the excellent answer from #Dan Dascalescu here, I understood that I can't have different minimongo collections if the original Mongodb collection is the same: even if I use different subscriptions everything end up in the same minimongo collection/"record set".
I could just read and compare the Date field on client side and allow/forbid the action but is that secure? Can the client change the date manually? What would be the right way to achieve this?
Any checks that you do to forbid an action have a security implication. There are approaches that you can use here:
use Methods server side along with Meteor.call client side.
use deny rules if it's collection related. That way you get isomorphic behavior for free and instant feedback on client without sacrificing security.
Explanation:
I am able to query the Google Core reporting APIv3 using the client library to get data on pageviews for specific URLs of a website I am working on. I want to get data(pageviews) for each day within a specified range. So far I am simply looping through the range, sending individual request to the API. in each request I am setting the same value for the start date and the end date.
Problem:
Obviously this gets the job done, BUT it is certainly not the best way to go about it. Because, assumming I want to get data for the past 3 months for each of about 2000 URIs. Then I will need 360000 number of requests and that value is well over the limit quota defined by Google.
Potential solution: So one way I thought of solving this issue is probably to send a request setting start-date and end-date to be a week apart but the API will return a sum of the values rather than the individual values.
main question: So is there a way to insist that these values should not be added up and returned as a sum but rather returned (as associative array or something like that) separately for each.
I hope the question is clear and that there is a solution! Thank you!
Very straightforward:
Metric: ga:pageview, Dimension: ga:date, Set a filter for your pagepath, and set a start-date and end-date.
Example:
https://www.googleapis.com/analytics/v3/data/ga?ids=ga%3Axxyyzz&dimensions=ga%3Adate&metrics=ga%3Apageviews&filters=ga%3Apagepath%3D%3D%2Ffaq.html&start-date=2013-06-27&end-date=2013-07-11&max-results=50
This will return the pageviews for that the faq.html& page for each day in the time-frame.
You should check out the QueryExplorer. Great tool to find out how to structure queries.
I have already gone through this
How best to design a REST API with multiple filters?
This does help when you have say 3 or 4 filtering criteria and you can accomodate that in the query String.
However let's take this example
You want to get call details about 20 telephone numbers, between a certain startdate and enddate.
Now I do agree ideally one should be advised to make individual queries for each number and then on the client side collate all data.
However for certain Live systems that would mean 20 rounds of queries on the switches or cdr databases. That is 20 request-response cycles plus the client having to collate and order them again based on time. While in the database level it would have been a simple single query that can return an ordered data and transformed back into a REST xml response that the client can embed on their system.
If we are to use GET the query string will get really confusing and has a limit as well.
Any suggestions to get around this issue.
Of course we can send a POST request with an xml having all numbers in it but that is against REST Get principles.
In case of GET use OData queries. For example when your start and end dates represented as numbers (unix time) URI could look like:
GET http://operatorcalls.com/Calls/Details?$filter=Date le 1342699200 and Date gt 1342526400
What you seem to be missing is an important concept of REST, caching. This can be done, as an example, in the browser, for a single client. Or it can be done as a shared cache between all the clients and the live production system (whatever it may be). Thus reducing queries against a live production system, or in your example, actual switches.
You should really take some time to read Fieldings thesis, and understand that REST is an architectural style.
I found a solution here Handling multiple parameters in a URI (RESTfully) in Java
but not quite happy with it.
So in effect we will end up using /cdr?numbers=number1,number2,number3 ...
However not too pleased with it as there is a limit to Query String in the url and also doesn't really seem to be an elegant solution. Anyone found any solution to this in their own implementation?
Basically not using POST for this kind of Fetch requests and also not using cumbresome and lengthy Query Strings.
We are using Jersey but also open to using CXF or Spring REST