Does the OData protocol provide a way to transform an array of objects to an array of raw values? - rest

Is there a way specify in an OData query that instead of certain name/value pairs being returned, a raw array should be returned instead? For example, if I have an OData query that results in the following:
{
"#odata.context": "http://blah.org/MyService/$metadata#People",
"value": [
{
"Name": "Joe Smith",
"Age": 55,
"Employers": [
{
"Name": "Acme",
"StartDate": "1/1/1990"
},
{
"Name": "Enron",
"StartDate": "1/1/1995"
},
{
"Name": "Amazon",
"StartDate": "1/1/1999"
}
]
},
{
"Name": "Jane Doe",
"Age": 30,
"Employers": [
{
"Name": "Joe's Crab Shack",
"StartDate": "1/1/2007"
},
{
"Name": "TGI Fridays",
"StartDate": "1/1/2010"
}
]
}
]
}
Is there anything I can add to the query to instead get back:
{
"#odata.context": "http://blah.org/MyService/$metadata#People",
"value": [
{
"Name": "Joe Smith",
"Age": 55,
"Employers": [
[ "Acme", "1/1/1990" ],
[ "Enron", "1/1/1995" ],
[ "Amazon", "1/1/1999" ]
]
},
{
"Name": "Jane Doe",
"Age": 30,
"Employers": [
[ "Joe's Crab Shack", "1/1/2007" ],
[ "TGI Fridays", "1/1/2010" ]
]
}
]
}
While I could obviously do the transformation client side, in my use case the field names are very large compared to the data, and I would rather not transmit all those names over the wire nor spend the CPU cycles on the client doing the transformation. Before I come up with my own custom parameters to indicate that the format should be as I desire, I wanted to check if there wasn't already a standardized way to do so.

OData provides several options to control the amount of data and metadata to be included in the response.
In OData v4, you can add odata.metadata=minimal to the Accept header parameters (check the documentation here). This is the default behaviour but even with this, it will still include the field names in the response and for a good reason.
I can see why you want to send only the values without the fields name but keep in mind that this will change the semantic meaning of the response structure. It will make it less intuitive to deal with as a json record on the client side.
So to answer your question, The answer is 'NO',
Other options to minimize the response size:
You can use the $value OData option to gets the raw value of a single property.
Check this example:
services.odata.org/OData/OData.svc/Categories(1)/Products(1)/Supplier/Address/City/$value
You can also use the $select option to cherry pick only the fields you need by selecting a subset of properties to include in the response

Related

How to filter OData collection where attribute does not exist?

I have an OData collection where the data looks like this:
{
"#odata.context": "http://localhost:5488/odata/$metadata#folders",
"value": [
{
"name": "samples",
"_id": "79a91bc9-9083-4442-ac8d-ad30777ac8c8",
"creationDate": "2019-08-05T04:39:00.670Z",
"modificationDate": "2019-08-05T04:39:00.670Z",
"shortid": "18xQnNv"
},
{
"name": "Population",
"folder": {
"shortid": "18xQnNv"
},
"_id": "7406269b-669c-41ce-92f3-f540792df07e",
"creationDate": "2019-08-05T04:39:00.750Z",
"modificationDate": "2019-08-05T04:39:00.750Z",
"shortid": "0ppeLV"
},
{
"name": "Invoice",
"folder": {
"shortid": "18xQnNv"
},
"_id": "525aff6a-6b10-4ad6-93ce-e9c753e8ade0",
"creationDate": "2019-08-05T04:39:00.790Z",
"modificationDate": "2019-08-05T04:39:00.790Z",
"shortid": "G3i2B3"
},
{
"name": "Default",
"_id": "58daf5aa-1f13-4ff9-be1f-8cb11a812485",
"creationDate": "2019-08-07T22:56:45.160Z",
"modificationDate": "2019-08-07T22:56:45.160Z",
"shortid": "Sm8LpmP"
}
]
}
I want to exclude the objects which have the attribute "folder". I've tried using a GET request: http://localhost:5488/odata/folders?$filter=folder eq null with no luck. Is this even possible and is there a way to filter my request like this?
You might be able to use the all lambda operator to accomplish this. The operator all will always return true on empty collections. So if you make a condition that no folder attribute that actually exists will ever evaluate to true on, then the result should be a filter of only those objects that have an empty attribute.
This is just a theory. You'll need to test, but it would maybe look something like this on your sample.
http://localhost:5488/odata/folders?$filter=folder/all(f:f/shortid eq 'xxxxxx')
You didn't mention the version of OData your working with but lambda expressions are at least V4 and later. Possibly earlier, not sure.

Return Connected Node's Properties Instead of RID using OUT() in OrientDB

I am attempting to store a connected Node's data in a property of a selected Node in OrientDB via the OUT() projection. e.g.:
SELECT *, OUT("Has_Friend") AS Friends FROM Person
Given that a "Person" Node is connected to several "Friend" Nodes via the "Has_Friend" Edge, I would like the actual Friend Node properties to be stored in the "Friends" property on each Person Node returned by this query. e.g.:
{
"result": [
{
"Name": "Joe",
"Friends": [
{
"Name": "Ben",
"Title": "Mr."
},
{
"Name": "Stan",
"Title": "Dr."
}
]
},
{
"Name": "Tim",
"Friends": [
{
"Name": "Terrance",
"Title": "Esq."
},
{
"Name": "Sarah",
"Title": "Dr."
}
]
}
]
}
However, the query only stores the RID of each "Friend" Node in the "Friends" property rather than the actual data of that "Friend" Node. e.g.:
{
"result": [
{
"Name": "Joe",
"Friends": [
"#228:1",
"#227:1"
]
},
{
"Name": "Tim",
"Friends": [
"#225:1",
"#226:1"
]
}
]
}
I've searched the OrientDB documentation but am unsure as to how I might accomplish this. I suspect there's a way to nest queries for those Friend nodes inside of the primary query, but I'm not entirely sure how to do that. Any insight is greatly appreciated!
try to use expand() function. It would expands the document pointed by that link and give all properties of this document. So your query should looks like this one:
SELECT expand(in("Has_Friend")) AS Friend FROM Person

Delete sub-document from array in array of sub documents

Let's imagine a mongo collection of - let's say magazines. For some reason, we've ended up storing each issue of the magazine as a separate document. Each article is a subdocument inside an Articles-array, and the authors of each article is represented as a subdocument inside the Writers-array on the Article-subdocument. Only the name and email of the author is stored inside the article, but there is an Writers-array on the magazine level containing more information about each author.
{
"Title": "The Magazine",
"Articles": [
{
"Title": "Mongo Queries 101",
"Summary": ".....",
"Writers": [
{
"Name": "tom",
"Email": "tom#example.com"
},
{
"Name": "anna",
"Email": "anna#example.com"
}
]
},
{
"Title": "Why not SQL instead?",
"Summary": ".....",
"Writers": [
{
"Name": "mike",
"Email": "mike#example.com"
},
{
"Name": "anna",
"Email": "anna#example.com"
}
]
}
],
"Writers": [
{
"Name": "tom",
"Email": "tom#example.com",
"Web": "tom.example.com"
},
{
"Name": "mike",
"Email": "mike#example.com",
"Web": "mike.example.com"
},
{
"Name": "anna",
"Email": "anna#example.com",
"Web": "anna.example.com"
}
]
}
How can one author be completely removed from a magazines?
Finding magazines where the unwanted author exist is quite easy. The problem is pulling the author out of all the sub documents.
MongoDB 3.6 introduces some new placeholder operators, $[] and $[<identity>], and I suspect these could be used with either $pull or $pullAll, but so far, I haven't had any success.
Is it possible to do this in one go? Or at least no more than two? One query for removing the author from all the articles, and one for removing the biography from the magazine?
You can try below query.
db.col.update(
{},
{"$pull":{
"Articles.$[].Writers":{"Name": "tom","Email": "tom#example.com"},
"Writers":{"Name": "tom","Email": "tom#example.com"}
}},
{"multi":true}
);

how to create Jsonpath file to load data in redshift

one of my sample record for Json:
{
"viewerId": "Ext-04835139",
"sid5": "269410578:2995631181:2211755370:3307088398:33879957",
"firstHbTimems": 1.506283958371E12,
"ipAddress": "74.58.57.31",
"streamUrl": "https://dc3-ll-livedazn-dznlivejp.hs.llnwd.net/live/channel/1007/all/stream.m3u8?event_id=61824040049&h=c912885e2a69ffa7ea84f45dc18c004d",
"asset": "[nlq9biy7trxl1cjceg70rogvd] Saints # Panthers",
"os": "IOS",
"osVersion": "10.3.3",
"deviceModel": "iPhone",
"geoInfo": {
"city": 63666,
"state": 3851,
"isp": 120,
"longitudeTimes1K": -73562,
"country": 37,
"dma": 0,
"asn": 5769,
"latitudeTimes1K": 45502,
"publicIP": 1245329695
},
"totalPlayingTime": 4.097,
"totalBufferingTime": 0.0,
"VST": 1.411,
"avgBitrate": 202.0,
"playStateSwitch": [
"{'seqNum': 0, 'eventNum': 0, 'sessionTimeMs': 7, 'startPlayState': 'eUnknown', 'endPlayState': 'eBuffering'}",
"{'seqNum': 1, 'eventNum': 5, 'sessionTimeMs': 1411, 'startPlayState': 'eBuffering', 'endPlayState': 'ePlaying'}"
],
"bitrateSwitch": [
],
"errorEvent": [
],
"tags": {
"LSsportName": "Football",
"c3.device.model": "iPhone+6+Plus",
"LSvideoType": "LIVE",
"c3.device.ua": "DAZN%2F5560+CFNetwork%2F811.5.4+Darwin%2F16.7.0",
"LSfixtureId": "5trxst8tv7slixckvawmtf949",
"genre": "Sport",
"LScompetitionName": "NFL+Game+Pass",
"show": "NFL+Game+Pass",
"c3.cmp.0._type": "DEVATLAS",
"c3.protocol.type": "cws",
"LSsportId": "9ita1e50vxttzd1xll3iyaulu",
"stageId": "8hm0ew6b8m7907ty8vy8tu4tl",
"LSvenueId": "na",
"syndicator": "None",
"applicationVersion": "2.0.8",
"deviceConnectionType": "wifi",
"c3.client.marketingName": "iPhone+6+Plus",
"playerVersion": "1.2.6.0",
"c3.cmp.0._id": "da",
"drmType": "AES128",
"c3.sh": "dc3-ll-livedazn-dznlivejp.hs.llnwd.net",
"c3.pt.ver": "10.3.3",
"applicationType": "ios",
"c3.viewer.id": "Ext-04835139",
"LSinterfaceLanguage": "en",
"c3.pt.os": "IOS",
"playerVendor": "Open+Source",
"c3.client.brand": "Apple",
"c3.cws.sf": "7",
"c3.cmp.0._ver": "1",
"c3.client.hwType": "Mobile+Phone",
"c3.pt.os.ver": "10.3.3",
"isAd": "false",
"c3.device.cver.bld": "2.124.0.33357",
"stageName": "Regular+Season",
"c3.client.osName": "iOS",
"contentType": "Live",
"c3.device.cver": "2.124.0",
"LScompetitionId": "wy3kluvb4efae1of0d8146c1",
"expireDate": "na",
"c3.client.model": "iPhone+6+Plus",
"c3.client.manufacturer": "Apple",
"LSproductionValue": "na",
"pubDate": "2017-09-23",
"c3.cluster.name": "production",
"accountType": "FreeTrial",
"c3.adaptor.type": "eCws1_7",
"c3.device.brand": "iPhone",
"c3.pt.br": "Non-Browser+Apps",
"contentId": "nlq9biy7trxl1cjceg70rogvd",
"streamingProtocol": "FairPlay",
"LSvenueName": "na",
"c3.device.type": "Mobile",
"c3.protocol.level": "2.4",
"c3.player.name": "AVPlayer",
"contentName": "Saints+%40+Panthers",
"c3.device.manufacturer": "Apple",
"c3.framework": "AVFoundation",
"c3.pt": "iOS",
"c3.device.ver": "6+Plus",
"c3.video.isLive": "T",
"c3.cmp.0._cfg_ver": "1504808821",
"c3.cws.clv": "2.124.0.33357",
"LScountryCode": "America%2FEl_Salvador"
},
"playername": "AVPlayer",
"isLive": "T",
"playerVersion": "1.2.6.0"
}
How to create jsonpath file to load it in redshift ?
Thanks
You have a nested array within your json - so a jsonpath will not expand that out for you.
You have a couple of choices on how to proceed:
You can load your data at the higher level (e.g. playStateSwitch
rather than seqNum within that) - and then try to use redshift to
process that data. This can be tricky as you cannot explode json
data from an array in redshift.
You can preprocess the data using e.g. aws glue / python / pyspark
or some other etl tool that can handle these nested arrays.
It all depends on the end goal, which is not clear form the above description.
I will approach the solution in the following order
Define which fields and array values that are required to be loaded into the Redshift. If the need is to copy all the records then the next check is how to handle the multiple array records.
If array or key/value are missing as part of JSON source then JSONPath will not work as is - So, better to update the JSON to add the missing array prior to COPY the data set over to RS.
The JSON update can be done using Linux commands or external tools like JP or refer additional reference
If all the values in the nested arrays are required then an alternative work around will be using external table an example
Otherwise, the JSONPATH file can be developed in this format
{
"jsonpaths": [
"$.viewerId", ///root level fields
...
"$geoInfo.city", /// object hierarchy
...
"$playStateSwitch[0].seqNum" ///define the required array number
...
]
}
Hope, this helps.

Facebook realtime.. How are different objects batched together

Facebook realtime docs specify that the callback data is of the following format
{
"object": "user",
"entry": [
{
"uid": 1335845740,
"changed_fields": [
"name",
"picture"
],
"time": 232323
}, {
"uid": 1234,
"changed_fields": [
"friends"
],
"time": 232325
}]}
If two different objects, say 'user' and 'page' are to be sent back, does facebook
Batch them together?
Send them separately?
Different objects are sent in different requests. But if there are many user requests then it will batch those user requests and send it. And also you can have different end points for user and page. The format of page request is also different.
It looks something like this
{
"object": "page",
"entry": [
{
"id": "408518775908252",
"time": 1360643280,
"changes": [
{
"field": "feed",
"value": {
"item": "like",
"verb": "add",
"user_id": 5900878
}
}
]
}
]
}