Designing a REST API with redundant data: client-side or server-side data processing - rest

I'm developing a JSON-based REST API allowing to get information about documents where a 'document' resource has the following form:
{
"id": 1,
"name": "document 1",
...
"fields":
[
{"name": "field1", "category": "category1", ...},
{"name": "field2", "category": "category2", ...},
...
]
}
The GET /documents/:id route is quite straight-forward, and I want the GET /documents route to provide an array of basic information related to documents instead of just links or IDs only, something like:
[
{"id": 1, "name": "document 1"},
...
]
Now the UI needs to display the list of documents with the list of categories the fields of a document belong to:
document 1 (category1, category2)
document 2 (category2, category3, category4)
...
The first solution would be to add the fields field to each document in the response of GET /documents and let the client compute the list of categories (maybe this could lead to a poor UX when there are a lot of documents to display), eg:
[
{"id": 1, "name": "document1", "fields":
[
{"name": "field1", "category": "category1", ...},
...
]
]
The second one would be to add that piece of (redundant) information server-side to avoid to add too much data in the response and avoid the client to loop through each document's fields (but now the API is more dependent on how the UI presents the data), eg:
[
{"id", "name": "document 1", "categories": ["category1", "category2"]},
...
]
According to your experiences, which solution should I use?

Related

In which case meta's whatsapp payload examples will receive with multiple element in array

Meta's whatsapp API integration and response on webhook,
https://developers.facebook.com/docs/whatsapp/cloud-api/webhooks/payload-examples
I am new to the whatsapp cloud integration and I am confused why inbound message response of webhook is too weird with nested array, in which cases facebook(meta) will give an multiple elements in nested of nested array.
Is it good way to get entry[0].changes[0].value.messages[0].text.body or I require to add loop on every case?
What are the changes we will received multiple elements?
{
"object": "whatsapp_business_account",
"entry": [{
"id": "WHATSAPP_BUSINESS_ACCOUNT_ID",
"changes": [{
"value": {
"messaging_product": "whatsapp",
"metadata": {
"display_phone_number": PHONE_NUMBER,
"phone_number_id": PHONE_NUMBER_ID
},
"contacts": [{
"profile": {
"name": "NAME"
},
"wa_id": PHONE_NUMBER
}],
"messages": [{
"from": PHONE_NUMBER,
"id": "wamid.ID",
"timestamp": TIMESTAMP,
"text": {
"body": "MESSAGE_BODY"
},
"type": "text"
}]
},
"field": "messages"
}]
}]
}
You can read the documentation of graph-api webhook,
https://developers.facebook.com/docs/graph-api/webhooks/getting-started#validate-payloads
Event Notifications are aggregated and sent in a batch with a maximum of 1000 updates. However batching cannot be guaranteed so be sure to adjust your servers to handle each Webhook individually.
You can also check the property-wise batch possibility in the provided link.

MongoDB - how to properly model relations

Let's assume we have the following collections:
Users
{
"id": MongoId,
"username": "jsloth",
"first_name": "John",
"last_name": "Sloth",
"display_name": "John Sloth"
}
Places
{
"id": MongoId,
"name": "Conference Room",
"description": "Some longer description of this place"
}
Meetings
{
"id": MongoId,
"name": "Very important meeting",
"place": <?>,
"timestamp": "1506493396",
"created_by": <?>
}
Later on, we want to return (e.g. from REST webservice) list of upcoming events like this:
[
{
"id": MongoId(Meetings),
"name": "Very important meeting",
"created_by": {
"id": MongoId(Users),
"display_name": "John Sloth",
},
"place": {
"id": MongoId(Places),
"name": "Conference Room",
}
},
...
]
It's important to return basic information that need to be displayed on the main page in web ui (so no additional calls are needed to render the table). That's why, each entry contains display_name of the user who created it and name of the place. I think that's a pretty common scenario.
Now my question is: how should I store this information in db (question mark values in Metting document)? I see 2 options:
1) Store references to other collections:
place: MongoId(Places)
(+) data is always consistent
(-) additional calls to db have to be made in order to construct the response
2) Denormalize data:
"place": {
"id": MongoId(Places),
"name": "Conference room",
}
(+) no need for additional calls (response can be constructed based on one document)
(-) data must be updated each time related documents are modified
What is the proper way of dealing with such scenario?
If I use option 1), how should I query other documents? Asking about each related document separately seems like an overkill. How about getting last 20 meetings, aggregate the list of related documents and then perform a query like db.users.find({_id: { $in: <id list> }})?
If I go for option 2), how should I keep the data in sync?
Thanks in advance for any advice!
You can keep the DB model you already have and still only do a single query as MongoDB introduced the $lookup aggregation in version 3.2. It is similar to join in RDBMS.
$lookup
Performs a left outer join to an unsharded collection in the same database to filter in documents from the “joined” collection for processing. The $lookup stage does an equality match between a field from the input documents with a field from the documents of the “joined” collection.
So instead of storing a reference to other collections, just store the document ID.

MongoDb query - aggregation, group, filter, max

I am trying to figure out specific mongoDb query, so far unsuccessfully.
Documents in my collections looks someting like this (contain more attributes, which are irrelevant for this query):
[{
"_id": ObjectId("596e01b6f4f7cf137cb3d096"),
"code": "A",
"name": "name1",
"sys": {
"cts": ISODate("2017-07-18T12:40:22.772Z"),
}
},
{
"_id": ObjectId("596e01b6f4f7cf137cb3d097"),
"code": "A",
"name": "name2",
"sys": {
"cts": ISODate("2017-07-19T12:40:22.772Z"),
}
},
{
"_id": ObjectId("596e01b6f4f7cf137cb3d098"),
"code": "B",
"name": "name3",
"sys": {
"cts": ISODate("2017-07-16T12:40:22.772Z"),
}
},
{
"_id": ObjectId("596e01b6f4f7cf137cb3d099"),
"code": "B",
"name": "name3",
"sys": {
"cts": ISODate("2017-07-10T12:40:22.772Z"),
}
}]
What I need is to get current versions of documents, filtered by code or name, or both. Current version means that from two(or more) documents with same code, I want pick the one which has latest sys.cts date value.
So, result of this query executed with filter name="name3" would be the 3rd document from previous list. Result of query without any filter would be 2nd and 3rd document.
I have an idea how to construct this query with changed data model but I was hoping someone could lead me right way without doing so.
Thank you

Restful GET URLs for Queries, When there are multiple queries need to serve

I have a similar structure as follows in Mongo DB for Users.
{
"id": 112121
"name": "Strange",
"age": 27,
"type": "Manager"
"email" : "strage#gamil.com",
"items" : [
{
"id": "1001",
"name": "laptop",
"details": [
{
"attributeKey": "ManifaturedYear",
"attributeValue" : "2010"
},
{
"attributeKey": "EligibaleToDsipose",
"attributeValue" : "2018"
}
]
},
{
"id": "1002",
"name": "phone",
"details": [
{
"attributeKey": "ManifaturedYear",
"attributeValue" : "2010"
},
{
"attributeKey": "EligibaleToDsipose",
"attributeValue" : "2018"
}
]
}
]
}
I am providing a restful URIs to manipulate this resource. And has URLS as follows
CREATE
POST api/users
POST api/users/{id}/items e.g. api/user/12121/items
POST api/users/{id}/items/{id}/details e.g. api/user/23223/items/222/details
UPDATE | DELETE
[ PUT | DELETE ] api/users/{id}
[ PUT | DELETE ] api/users/{id}/items/{id}
[ PUT | DELETE ] api/users/{id}/items/{id}/details/{attributeKey}
My Question is How I support Get Requests
Get Items owned by Managers? A, B or C
A. GET api/users/type/{type}/items e.g. GET api/users/type/Manager/items
B. GET api/users/{type}/items e.g. GET api/users/Manager/items # conflicts with id
C. GET api/users/items?type=Manager
How to get Items of user with email address abc#gmail.com
A. GET api/users/email/abc#gamil.com/items
B. GET api/users/items?userEmail=abc#gmail.com
How to get Items of Users who are older than 30?
Any good reference I can take to designing my rest URIs
My suggestion, if possible, would be to break out items as a top-level resource. /users/{id} would still have a collection of items owned by that user. Depending on your needs, you might only embed them as links, rather than full representations, in the user response. /items would be the canonical collection of all items. That would give you these resources:
/users
/users/{id}
/users/{id}/items
/items
If you do this, then:
1: GET /items?ownerType=Manager
2: GET /items?ownerEmail=abc#example.com (a collection of items) -or- GET /users?email=abc#example.com (a collection with one user)
3: GET /items?ownerOlderThan=30

Does the OData protocol provide a way to transform an array of objects to an array of raw values?

Is there a way specify in an OData query that instead of certain name/value pairs being returned, a raw array should be returned instead? For example, if I have an OData query that results in the following:
{
"#odata.context": "http://blah.org/MyService/$metadata#People",
"value": [
{
"Name": "Joe Smith",
"Age": 55,
"Employers": [
{
"Name": "Acme",
"StartDate": "1/1/1990"
},
{
"Name": "Enron",
"StartDate": "1/1/1995"
},
{
"Name": "Amazon",
"StartDate": "1/1/1999"
}
]
},
{
"Name": "Jane Doe",
"Age": 30,
"Employers": [
{
"Name": "Joe's Crab Shack",
"StartDate": "1/1/2007"
},
{
"Name": "TGI Fridays",
"StartDate": "1/1/2010"
}
]
}
]
}
Is there anything I can add to the query to instead get back:
{
"#odata.context": "http://blah.org/MyService/$metadata#People",
"value": [
{
"Name": "Joe Smith",
"Age": 55,
"Employers": [
[ "Acme", "1/1/1990" ],
[ "Enron", "1/1/1995" ],
[ "Amazon", "1/1/1999" ]
]
},
{
"Name": "Jane Doe",
"Age": 30,
"Employers": [
[ "Joe's Crab Shack", "1/1/2007" ],
[ "TGI Fridays", "1/1/2010" ]
]
}
]
}
While I could obviously do the transformation client side, in my use case the field names are very large compared to the data, and I would rather not transmit all those names over the wire nor spend the CPU cycles on the client doing the transformation. Before I come up with my own custom parameters to indicate that the format should be as I desire, I wanted to check if there wasn't already a standardized way to do so.
OData provides several options to control the amount of data and metadata to be included in the response.
In OData v4, you can add odata.metadata=minimal to the Accept header parameters (check the documentation here). This is the default behaviour but even with this, it will still include the field names in the response and for a good reason.
I can see why you want to send only the values without the fields name but keep in mind that this will change the semantic meaning of the response structure. It will make it less intuitive to deal with as a json record on the client side.
So to answer your question, The answer is 'NO',
Other options to minimize the response size:
You can use the $value OData option to gets the raw value of a single property.
Check this example:
services.odata.org/OData/OData.svc/Categories(1)/Products(1)/Supplier/Address/City/$value
You can also use the $select option to cherry pick only the fields you need by selecting a subset of properties to include in the response