I would like to know if it is posible to order results by field in Orion NGSI-LD, using orderBy param.
This functionality is already implemented in Orion NGSIv2 (https://fiware-orion.readthedocs.io/en/master/orion-api.html#ordering-results).
I have tried to use orderBy param in GET http://localhost:1026/ngsi-ld/v1/entities?type=Incident&options=keyValues&orderBy=!code but it does not seem to work.
However, if the search is performed using entity id as orderBy value, it works, GET http://localhost:1026/ngsi-ld/v1/entities?type=Incident&options=keyValues&orderBy=id.
Yeah, sorry. That "orderBy" URL param is strictly NGSIv2. NGSI-LD doesn't (currently) support any ordering. It's not standardized and up to the implementations. Orion-LD orders the response by "createdAt", as this makes pagination easier. NOTE that it's not ordered by Entity ID.
That said, we could include this new feature in the NGSI-LD standard. I'm part of the ETSI ISG CIM group that defines the NGSI-LD API and I'll take this up in our next meeting (Tuesday, Jan 17).
Related
The standard REST verb for returning a value GET can take different parameters to select what to "get". Often there is one that takes an id to get a single value, and often some sort of search criteria to get a list.
Is there a standard way to specify the filtering and sorting of the data that is being searched for? For example, if I have an invoice record I'd like to write a GET query that says "give me all invoices for customer 123, with total > $345 and return in descending order of date".
If I were writing this myself I'd have something like:
GET http://example.com/mydata?query="customer=123&&total>345.00"&order="date"
(Note I didn't urlencode the url for clarity, though obviously that is required in practice, but I hope you get what I mean.)
I can certainly write something for this, but I am wondering if there is a standardized way to do this?
Is there a standard way to specify the filtering and sorting of the data that is being searched for?
Not that I'm aware of.
Note that HTTP doesn't really have queries (yet); HTTP has resource identifiers.
We've got a standard for resource identifiers (RFC 3986) and a standard for URI templates (RFC 6570) that describes how to produce a range of identifiers via variable expansion.
But as far as I can tell there is no published "standard" that automatically transforms a URI into a SQL query.
It's possible that one of the "convention over configuration" frameworks (ex: Rails) might have something useful here, but I haven't found it.
I'm designing a REST api and interested if anyone can help with best practice in the following scenario.
I have...
GET Customers/{customerId}/Orders - to get all customer orders
GET Customers/{customerId}/Orders/{orderId} - to get a particular order
I need to provide the ability to get a customers most recent order. What is best practice in this scenario? Simply get all and sort by date or provide a specific method?
I need to provide the ability to get a customers most recent order.
Of course you could provide query parameters to filter, sort and slice the orders collection, but why not making it simpler and give the latest order if the client needs it?
You could use something like (returning a representation of a single order):
GET /customers/{customerId}/orders/latest
The above URL will map an order that will change over the time and it's perfectly fine.
Say there is also a case where you need last 5 orders. How would your route(s) look like?
The above approach focus on the ability to get a customers most recent order requirement. If returning the last 5 orders requirement eventually comes up after some time, I would probably introduce another mapping such as /recent that returns a representation of a collection with the recent orders and accepts a query parameter that indicates the amount of orders to be returned (5 would be the default value if the parameter is omitted).
The /latest mapping would still be valid and would return a representation of the very latest order only.
Providing query parameters to filter, sort and slice the orders collection is still a valid approach.
The key is: If you know the client who will consume the API, target it to their needs. Otherwise, make it more generic. And when modifying the API, be careful with breaking changes and versioning the API is also welcome.
I think there is no need for another route.
Pass something like &order=-created_at&limit=1 in your get request
Or &order=created_at&orderby=DESC&limit=1 (note I'm not sure about naming your params so maybe you could use &count=1 instead of &limit=1, ditto order params)
I think it also depends whether you are using pagination or not on that route, so perhaps additional params are required
Customers/{customerId}/Orders?order=-created_at&limit=1
The Github API for the similar use case is using latest, to fetch the single resource which is latest.
https://docs.github.com/en/rest/reference/repos#get-the-latest-release
So to fetch a single resource which is latest you can use.
GET /customers/{customerId}/orders/latest
However would like to know what community think about this.
IMO the resource/latest gives an impression that the response will be a list of resource sorted by latest to oldest.
This is sort of a follow-up to someone else's question about filtering/querying a list of cars. There the recommendation for a RESTful filtering request was to put filter expressions in the query of the URI, like this:
/cars?color=blue&type=sedan&doors=4
That's fine. But what if my filtering query becomes more complicated and I need to use Boolean operators, such as:
((color=blue OR type=sedan) AND doors=4) OR color=red
That is, I want to find a four-door blue car or a four-door sedan, but if the car is red I'll take it without caring about any of the other properties.
Is there any sort of convention for providing Boolean expressions in a RESTful URI's query parameters? I suppose I could by create some new querying expression language and put it in a POST, but that seems like a heavy and proprietary approach. How are others solving this?
It is perfectly okay to use
/cars/color:blue/type:sedan/doors:4
instead of
/cars?color=blue&type=sedan&doors=4
The URL standard says only that the path should contain the hierarchical part, and the query should contain the non-hierarchical. Since this is a map-reduce, using / is perfectly valid.
In your case you need a query language to describe your filters. If I were you I would copy an already existing solution, for example the query language of a noSQL database which has a REST API.
I think resource query language is what you need. I think you could use it like this:
/sthg?q="(foo=3|foo=bar)&price=lt=10"
or forget the default queryString parser, and like this:
/sthg?(foo=3|foo=bar)&price=lt=10
I suggest you to read the manual for further details.
Since I found no other URL compatible query language (yet), I think the only other option to serialize another query language and send it in a param, like SparSQL
http://localhost:8003/v1/graphs/sparql?query=your-urlencoded-query
by marklogic7. Hydra defines a freeTextQuery in its vocab, so they follow the same approach. But I'll ask Markus about this. It's a complicated topic, since according to the self-descriptive messages constraint you should describe somewhere what type of query language you use in the URL. I am not sure about this. :S
conclusion:
In order to support ad-hoc search queries we need a standard way to describe them in the link meta-data. Currently there are only a few standards about this. The most widely used standard is URI templates which does not support nested statements, operators, etc... for what I know. There is a draft called link descriptions which tries to fill the gap, but it is incomplete.
One possible workaround to define an URI template with a single q parameter which has rdf:type of x:SearchQuery and rdfs:range of xsd:string, and create another vocab about how to describe such a x:SearchQuery. After that the description could be used to build search forms, and validate queries sent to the server. Already existing queries could be supported too with this approach, so we don't need a new one.
So this problem can be solved with vocabs or new URI template standards.
I have seen many use a query string as you have provided - much like a SQL query string.
Here are just two examples:
Socrata (Open Data Portal company)'s SoQL (SQL variant): http://dev.socrata.com/consumers/cookbooks/querying-block-ranges.html
openFDA (API from fda.gov for open data) uses a similar string-based query parameter which maps to ElasticSearch queries, I believe: https://open.fda.gov/api/reference/#query-syntax
Try using 1 for true, 0 for false.
/_api/web/lists/getbytitle('XYZ')/items?$filter=Active eq 1
Some days ago when I made a query to Orion context broker with regular expressions, the response included all the entities that match that regular expression, now only retrieve the data of the older entities and if i want to access to the most recent entities, I need to make another request with a more specific regular expression.
The problem is that now I need to make two different querys with differents regular expression to retrieve the old and new entities when a week ago I only had to make one with the most generic regular expression.
Imagine that the structure of the id of my entities are:
dev:name_proyect:name_object:name_service
Some days ago I only need this generic expression to retrieve all the data:
dev:name_proyect:.*
Now I need two queryes to obtain the same result:
dev:name_proyect:.* -> the response only include old entities
dev:name_proyect:name_object:.* ->the response retrieve only the new data
So, the use of regular expression have changed in Orion CB? I need to do something else now to retrieve all the data with only one query with the most generic regular expression?
The instance of Orion that I am using is:
http://orion.lab.fi-ware.eu:1026
Note that since 0.14.0, Orion implements pagination, which details are described in the corresponding section in the user manual.
There is a default limit of 20 in the number of entities retrieved by queryContext (and related convenience operations). If you want to increase that limit, use the limit URI parameter, e.g.
POST http://orion.lab.fi-ware.eu:1026/ngsi10/queryContext?limit=200
The absolute maximum limit is 1,000. If the query covers more than 1,000 result, the client have to retrieve the data in several chainned requests, using the offset parameters (see details in the above reference).
You can get a hint on how many entities are covered by the query using the details parameter, e.g:
POST http://orion.lab.fi-ware.eu:1026/ngsi10/queryContext?details=on
...
"errorCode": {
"code": "200",
"details": "Count: 2342",
"reasonPhrase": "OK"
}
Trying to nail down what the sort order is for Shopify's REST collections. Specifically I'm working with orders and customers at the moment.
I found this closed thread discussing the ability to sort collections, and the API docs don't mention it at all for either orders or customers.
However, customers have a documented search API, which does have an order parameter as an option. I'm not sure whether I can use it as a sort of substitute for the regular list API call. This doesn't seem to work properly for example - it returns more than one result.
/admin/customers/search.json?query=&limit=1
Orders don't have a documented search endpoint, but I do get a response when hitting
/admin/orders/search.json?query=&limit=1
Although it has the same issues as the customer search endpoint. I found this thread saying that orders are always returned most recent to newest by date - and inspecting the response I'm getting now that seems to be true although I could have sworn I've seen them come back in a different orders, it almost seems indeterminate.
Would like to know if that's the case for sure, and the same for customers. I seem to be getting them returned back in created_at order ascending for customers. Is that always the case?
Also, ID's for both customers and orders don't seem to be in created_at order, which is bizzare given that the have the since_id parameter in pretty much all their collections (which I found and promptly built my incremental pulling strategy on top of). I guess I'll have to use created_at instead.
The thread you linked to (apart from being really old) is referring to Collections (with a capital C), which are how groups of products are defined in Shopify (e.g. Shoes, Coats, Hats, etc.).
The 'collections' you're referring to (Customers, Products, Orders, etc.) are returned in descending date order (i.e. newest first) if the since_id parameter is not supplied, and in ascending order if it is supplied. Note that this should correspond with a descending id ordering, that's the idea at any rate as it allows you to use since_id to paginate properly (as you're doing).
Double check that you're getting wonky id ordering and if you are, please post the store domain you're seeing it on as well as a sample of order ids so that we can look into it.