How can I manipulate one of the values in normalizePayload (I need it anyway to convert result to session)
I need start_time and end_time to be multipled with 1000 in order to get smoothly into an attr('date')
"result": [
{
"end_time": 1412687629.42063,
"start_time": 1412687629.26851,
},
{
"end_time": 1412688377.15329,
"start_time": 1412688377.11507,
},
...
The current code I have is:
App.SessionSerializer = DS.ActiveModelSerializer.extend({
normalizePayload: function(payload) {
return {
sessions: payload.result
};
}
});
Sorry I gave you bad info in the answer in your other question. The normalizePayload method should manipulate the payload hash directly:
App.SessionSerializer = DS.ActiveModelSerializer.extend({
normalizePayload: function(payload) {
payload.sessions= payload.result;
delete payload.result;
delete payload.metadata;
return payload;
}
});
Related
I have a Mongoose schema for the document Company, that has several fields. One of these (documents_banks) is a "free" field, of dictionary type, because I don't know the names of the keys in advance.
The problem is that, when I save the document (company.save()) even if the resulting saved document has the new sub_docs, in the DB no new sub_docs are actually saved.
var Company = new Schema({
banks: [{ type: String }], // array of Strings
documents_banks: {} // free field
});
Even if documents_banks is not restricted by the Schema, it will have this structure (in my mind):
{
"bank_id1": {
"doc_type1": {
"url": { "type": "String" },
"custom_name": { "type": "String" }
},
"doc_type2": {
"url": { "type": "String" },
"custom_name": { "type": "String" }
}
},
"bank_id2": {
"doc_type1": {
"url": { "type": "String" },
"custom_name": { "type": "String" }
}
}
}
But I don't know in advance names of keys bank_id neither doc_type, so I used the Dictionary type (documents_banks:{}).
Now, this below is the function I use to save new sub_docs in documents_banks. The same logic I always use to save new sub_docs.. Anyway this time, it seems saved, but it's not.
function addBankDocument(company_id, bank_id, doc_type, url, custom_name) {
// retrieve the company document
Company.findById(company_id)
.then(function(company) {
// create empty sub_docs if needed
if (!company.documents_banks) {
company.documents_banks = {};
}
if (!company.documents_banks[bank_id]) {
company.documents_banks[bank_id] = {};
}
// add the new sub_doc
company.documents_bank[bank_id][doc_type] = {
"url": url,
"custom_name": custom_name
};
return company.save();
})
.then(function(saved_company) {
// I try to check if the new obj has been saved
console.log(saved_company.documents_bank[bank_id][doc_type]);
// and it actually prints the new obj!!
});
}
The saved_company returned by the .save() actually has the new sub_docs, but if I check the DB there is not the new sub_doc! I can save just the first one, all the others are not stored.
So, the console.log() always print the new sub_docs, but actually in the DataBase, just the first sub_doc is saved, not the others. So at the end, saved_company always has 1 sub_doc, the first one.
It seems very strange to me, since saved_company has the new sub_docs. What can be happened?
This below is a real extract from by DB, and it will contains forever just the sub_doc "doc_bank#1573807781414", others will be not present in the DB.
{
"_id": "5c6eaf8efdc21500146e289c", // company_id
"banks": [ "MPS" ],
"documents_banks": {
"5c5ac3e025acd98596021a9a": // bank_id
{
"doc_bank#1573807781414": // doc_type
{
"url": "http://...",
"custom_name": "file1"
}
}
}
}
Versions:
$ npm -v
6.4.1
$ npm show mongoose version
5.7.11
$ node -v
v8.16.0
It seems that, since mongoose doesn't know the exact model of the subdoc, it can't know when it changes. So I have to use markModified to notify changes of the "free field" (also known as dictionary or MixedType) with this:
company_doc.documents_banks["bank_id2"]["doc_type3"] = obj; // modify
company_doc.markModified('documents_banks'); // <--- notify changes
company_doc.save(); // save changes
As I understood, markModified force the model to 'update' that field during the save().
I'm finding an approach to resolve cross-venue vistor report for my client, he wants an HTTP API that return the total unique count of his customer who has visited more than one shop in day range (that API must return in 1-2 seconds).
The raw data sample (...millions records in reality):
--------------------------
DAY | CUSTOMER | VENUE
--------------------------
1 | cust_1 | A
2 | cust_2 | A
3 | cust_1 | B
3 | cust_2 | A
4 | cust_1 | C
5 | cust_3 | C
6 | cust_3 | A
Now, I want to calculate the cross-visitor report. IMO the steps would be as following:
Step 1: aggregate raw data from day 1 to 6
--------------------------
CUSTOMER | VENUE VISIT
--------------------------
cus_1 | [A, B, C]
cus_2 | [A]
cus_3 | [A, C]
Step 2: produce the final result
Total unique cross-customer: 2 (cus_1 and cus_3)
I've tried somes solutions:
I firstly used MongoDB to store data, then using Flask to write an API that uses MongoDB's utilities: aggregation, addToSet, group, count... But the API's response time is unacceptable.
Then, I switched to ElasticSearch with hope on its Aggregation command sets, but they do not support pipeline group command on the output result from the first "terms" aggregation.
After that, I read about Redis Sets, Sorted Sets,... But they couldn't help.
Could you please show me a clue to solve my problem.
Thank in advanced!
You can easily do this with Elasticsearch by leveraging one date_histogram aggregation to bucket by day, two terms aggregations (first bucket by customer and then by venue) and then only select the customers which visited more than one venue any given day using the bucket_selector pipeline aggregation. It looks like this:
POST /sales/_search
{
"size": 0,
"aggs": {
"by_day": {
"date_histogram": {
"field": "date",
"interval": "day"
},
"aggs": {
"customers": {
"terms": {
"field": "customer.keyword"
},
"aggs": {
"venues": {
"terms": {
"field": "venue.keyword"
}
},
"cross_selector": {
"bucket_selector": {
"buckets_path": {
"venues_count": "venues._bucket_count"
},
"script": {
"source": "params.venues_count > 1"
}
}
}
}
}
}
}
}
}
In the result set, you'll get customers 1 and 3 as expected.
UPDATE:
Another approach involves using a scripted_metric aggregation in order to implement the logic yourself. It's a bit more complicated and might not perform well depending on the number of documents and hardware you have, but the following algorithm would yield the response 2 exactly as you expect:
POST sales/_search
{
"size":0,
"aggs": {
"unique": {
"scripted_metric": {
"init_script": "params._agg.visits = new HashMap()",
"map_script": "def cust = doc['customer.keyword'].value; def venue = doc['venue.keyword'].value; def venues = params._agg.visits.get(cust); if (venues == null) { venues = new HashSet(); } venues.add(venue); params._agg.visits.put(cust, venues)",
"combine_script": "def merged = new HashMap(); for (v in params._agg.visits.entrySet()) { def cust = merged.get(v.key); if (cust == null) { merged.put(v.key, v.value) } else { cust.addAll(v.value); } } return merged",
"reduce_script": "def merged = new HashMap(); for (agg in params._aggs) { for (v in agg.entrySet()) {def cust = merged.get(v.key); if (cust == null) {merged.put(v.key, v.value)} else {cust.addAll(v.value); }}} def unique = 0; for (m in merged.entrySet()) { if (m.value.size() > 1) unique++;} return unique"
}
}
}
}
Response:
{
"took": 1413,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 7,
"max_score": 0,
"hits": []
},
"aggregations": {
"unique": {
"value": 2
}
}
}
I am getting an Error when running a script in the Postman Tests tab, When trying to check that a property is not null.
My JSON response:
{
"statusMessage": "Success",
"timeStamp": "2018-01-23 05:13:16.7",
"numberOfRecords": 7,
"parties": [
{
"shippingAddress": null,
"shippingDetails": null,
"paExpirationDate": "",
"historyDate": "01/22/2018",
"renewal": {
"renewalRx": "N",
"priorRxNumber": "",
"priorSB": "",
"priorFillNumber": ""
},
"noOfRefillingRemaining": "1",
"ndc": "00074-3799-06",
"rxId": "7004942",
"fillingNumber": "0"
},
{
"shippingAddress": {
"addressLine1": "2150 St",
"addressLine2": "Home Line 2",
"city": "Bronx",
"state": "NY",
"zipCode": "10453",
"addressSeqNumber": "1",
"medFacilityIndicator": "N"
}
}
]
}
My postman script is:
var jsonData = JSON.parse(responseBody);
var parties = jsonData.parties;
parties.forEach(function(data){
if(data.shippingAddress!==null && data.shippingAddress.addressLine1 !== null ){
postman.setEnvironmentVariable("addressLine1",data.shippingAddress.addressLine1);
}
I am getting the following error:
"Error running tests for results: TypeError: Cannot read property 'addressLine1' of null"
You could try this, I changed your code slightly but this would work:
var parties = pm.response.json().parties
for(i = 0; i < parties.length; i++) {
if(parties[i].shippingAddress !== null) {
pm.environment.set("addressLine1", parties[i].shippingAddress.addressLine1)
}
}
I tested this locally with the Schema that you provided and that it wrote 2150 St to my environment file.
The schema you posted doesn't seem to be a complete one, I think that the parties array has a shippingAddress property which is either null or it is an object containing the shippingAddress details - I might be wrong but I can't get my head around the data that you posted.
I don't think what you're searching on the in the if statement is correct and it wouldn't work the way you have it because if the first condition is null (like in your response data) it wouldn't ever meet the second condition because the object wouldn't be there and that shippingAddress.addressLine1 reference would always show that error.
Or you could use your code like this:
var jsonData = JSON.parse(responseBody)
var parties = jsonData.parties
parties.forEach(function(data) {
if(data.shippingAddress !== null) {
postman.setEnvironmentVariable("addressLine1",data.shippingAddress.addressLine1)
}
})
I'm requesting bucket=audio_summary for songs that rank highly in hotttness.
The top 100 hotttessst songs all return track.status: 'complete' but the audio summary is always an empty object.
How do I get audio summary data like time_signature and tempo? Doesn't track.status: complete imply that this information should be included in the response?
Can't Feel My Face
https://developer.echonest.com/api/v4/track/profile?api_key=*****************&format=json&id=SOMVZDS14DDE5909E7&bucket=audio_summary
{
"response": {
"status": {
"version": "4.2",
"code": 0,
"message": "Success"
},
"track": {
"status": "complete",
"id": "SOZOIDR14C02B654D4",
"audio_summary": {}
}
}
}
It looks like you have the SongId rather than the TrackId for this song. If you replace 'track' with 'song' in your query you get the expected results.
This:
https://developer.echonest.com/api/v4/song/profile?api_key=*****************&format=json&id=SOMVZDS14DDE5909E7&bucket=audio_summary
Returns this:
{
"response":{
"status":{
"version":"4.2",
"code":0,
"message":"Success"
},
"songs":[
{
"artist_id":"ARYUDWF12F2B89BB33",
"artist_name":"The Weeknd",
"id":"SOMVZDS14DDE5909E7",
"audio_summary":{
"key":9,
"analysis_url":"http://echonest-analysis.s3.amazonaws.com/TR/A7NwGAWSmhnc53M8w1rLk-eA_tsN8OUHaqV5C5FTbb9BnCPZVnxWOGld1wLxJJ-xdNV8xKJbbZlCRSKJc%3D/3/full.json?AWSAccessKeyId=AKIAJRDFEY23UEVW42BQ&Expires=1454028841&Signature=HeUxbEJt4f0ncipjD1Gamtuj08E%3D",
"energy":0.781735,
"liveness":0.12022,
"tempo":107.954,
"speechiness":0.042317,
"acousticness":0.124391,
"instrumentalness":0.0,
"mode":0,
"time_signature":4,
"duration":216.46667,
"loudness":-5.528,
"audio_md5":"",
"valence":0.586261,
"danceability":0.713659
},
"title":"Can't Feel My Face"
}
]
}
}
The only reason I was able to track this down was that throwing any ID into the track API give back the stub response you were seeing and I was able to get the audio_summary for this song using the spotify id so I knew it was available.
I'm in the early phases of the development of a client app to an existing REST service and I'm trying to decide what to use for server communication. So far I'm loving Restangular documentation, it seems really solid, but I'm worried it's not going to work with the service because the responses look something like this:
{
"0": {
"name": "John",
"total": 230,
"score": 13
},
"1": {
"name": "Sally",
"total": 190,
"score": 12
},
"2": {
"name": "Harry",
"total": 3,
"score": 0
},
"...": "..."
}
I can't find in the docs if something like this is supported or how am I supposed to handle this type of response. Has anyone tried? Any ideas?
I'm the creator of Restangular :).
You can use that response with Restangular. You need to use the responseInterceptor. I guess that your response looks like that when you're getting an array. So you need to do:
RestangularProvider.setListTypeIsArray(false)
RestangularProvider.setResponseExtractor(function(response, operation) {
// Only for lists
if (operation === 'getList') {
// Gets all the values of the property and created an array from them
return _.values(response)
}
return response;
});
With this, it's going to work :)
Please let me know if this worked out fr you
You could do this if if you attached a "then" statement to the promise that you return.
That is, instead of this:
return <RectangularPromise>;
you have this:
return <RectangularPromise>.then(function(response){return _.values(response);};
Essentially, it's the same as #mgonto's answer, but doing it this way means that you don't have to modify your whole Rectangular Service if you only need to do this for one endpoint. (Although, another way to avoid this is to create multiple instances of the service as outlined in the readme).
unfortunately listTypeIsArray is not described in readme, so I haven't tried this option
but I resolved similar problem as described here
https://github.com/mgonto/restangular/issues/100#issuecomment-24649851
RestangularProvider.setListTypeIsArray() has been deprecated.
If you are expecting an object then handle it within addResponseInterceptor
app.config(function(RestangularProvider) {
// add a response interceptor
RestangularProvider.addResponseInterceptor(function(data, operation, what, url, response, deferred) {
var extractedData;
// .. to look for getList operations
if (operation === "getList") {
// .. and handle the data and meta data
extractedData = data.data.data;
extractedData.meta = data.data.meta;
} else {
extractedData = data.data;
}
return extractedData;
}); });
If you are expecting string response then convert it to array.
app.config(function(RestangularProvider) {
RestangularProvider.addResponseInterceptor(function(data, operation, what, url, response, deferred) {
var newData = data;
if (angular.isString(data)) {
newData = [data]; //covert string to array
}
return newData;
}); });