I am trying to call the TwitchAPI and insert some of the returned data into MongoDB. However, every time I get this error: Error: Meteor requires document _id fields to be non-empty strings or ObjectIDs.
The Twitch API response for a single stream/channel looks like this:
{
"streams": [
{
"_id": 11220687552,
"game": "League of Legends",
"viewers": 11661,
"created_at": "2014-09-30T01:10:36Z",
"_links": {
"self": "http://api.twitch.tv/kraken/streams/mushisgosu"
},
"preview": {
"small": "http://static-cdn.jtvnw.net/previews-ttv/live_user_mushisgosu-80x50.jpg",
"medium": "http://static-cdn.jtvnw.net/previews-ttv/live_user_mushisgosu-320x200.jpg",
"large": "http://static-cdn.jtvnw.net/previews-ttv/live_user_mushisgosu-640x400.jpg",
"template": "http://static-cdn.jtvnw.net/previews-ttv/live_user_mushisgosu-{width}x{height}.jpg"
},
"channel": {
"_links": {
"self": "https://api.twitch.tv/kraken/channels/mushisgosu",
"follows": "https://api.twitch.tv/kraken/channels/mushisgosu/follows",
"commercial": "https://api.twitch.tv/kraken/channels/mushisgosu/commercial",
"stream_key": "https://api.twitch.tv/kraken/channels/mushisgosu/stream_key",
"chat": "https://api.twitch.tv/kraken/chat/mushisgosu",
"features": "https://api.twitch.tv/kraken/channels/mushisgosu/features",
"subscriptions": "https://api.twitch.tv/kraken/channels/mushisgosu/subscriptions",
"editors": "https://api.twitch.tv/kraken/channels/mushisgosu/editors",
"videos": "https://api.twitch.tv/kraken/channels/mushisgosu/videos",
"teams": "https://api.twitch.tv/kraken/channels/mushisgosu/teams"
},
"background": null,
"banner": "http://static-cdn.jtvnw.net/jtv_user_pictures/mushisgosu-channel_header_image-c5c08cce281b7be3-640x125.jpeg",
"display_name": "MushIsGosu",
"game": "League of Legends",
"logo": "http://static-cdn.jtvnw.net/jtv_user_pictures/mushisgosu-profile_image-b1c8bb5fd700025e-300x300.png",
"mature": false,
"status": "CLG hi im Gosu - Challenger AD - Smurfing Master!",
"partner": true,
"url": "http://www.twitch.tv/mushisgosu",
"video_banner": "http://static-cdn.jtvnw.net/jtv_user_pictures/mushisgosu-channel_offline_image-7e3401b20cb5d739-640x360.png",
"_id": 41939266,
"name": "mushisgosu",
"created_at": "2013-03-31T21:12:14Z",
"updated_at": "2014-09-30T03:08:55Z",
"abuse_reported": null,
"delay": 60,
"followers": 318914,
"profile_banner": null,
"profile_banner_background_color": null,
"views": 25963780,
"language": "en-us"
}
}
],
"_total": 8477,
"_links": {
"self": "https://api.twitch.tv/kraken/streams?limit=1&offset=0",
"next": "https://api.twitch.tv/kraken/streams?limit=1&offset=1",
"featured": "https://api.twitch.tv/kraken/streams/featured",
"summary": "https://api.twitch.tv/kraken/streams/summary",
"followed": "https://api.twitch.tv/kraken/streams/followed"
}
}
The part of my server method that tries to insert the data
Meteor.call('getStreams', function(err, res) {
var data = res.data.streams;
console.log(data);
data.forEach(function(item) {
console.log(item._id);
Streams.insert({
_id: item._id,
title: item.channel.status,
author: item.channel.display_name,
url: item.url
});
});
});
getStreams simple defines the url to call and sets some variable. As you can see I am console logging the expected _id so I know it is returning a valid string but I am still getting the error. Currently, when I make the call I return 100 streams at a time and iterate through them to save the 4 fields above. Ideally, I would like to save each stream object as its own entry in the DB but all my attempts to do that have resulted in the same error and I also read somewhere that the version on "miniMongo" bundled with Meteor does not support inserting an array of objects in bulk...I also have read that miniMong does not support Collection.save() so sadly I think it will be more later to update the contents of each _id with the latest API call info since I cant just use .save to update and insert in the same statement.
I am not sure if it has any impact but I did try setting autoIndexId to false when creating the collection and it doesn't seem to matter:
Streams = new Meteor.Collection('streams', {autoIndexId: false});
Any insight is appreciated.
The problem is that the twitch _id is NOT a String, it appears to be a Number (I can tell by the output of your JSON : the number is not surrounded by quotes).
What I'd do is let Meteor generate its own internal Mongo IDs and store the twitch _id as a separate property instead.
Streams.insert({
twitchId: item._id,
title: item.channel.status,
author: item.channel.display_name,
url: item.url
});
You will have to retrieve the streams by twitchId instead of _id, but it's hardly a problem, right ?
Related
Meta's whatsapp API integration and response on webhook,
https://developers.facebook.com/docs/whatsapp/cloud-api/webhooks/payload-examples
I am new to the whatsapp cloud integration and I am confused why inbound message response of webhook is too weird with nested array, in which cases facebook(meta) will give an multiple elements in nested of nested array.
Is it good way to get entry[0].changes[0].value.messages[0].text.body or I require to add loop on every case?
What are the changes we will received multiple elements?
{
"object": "whatsapp_business_account",
"entry": [{
"id": "WHATSAPP_BUSINESS_ACCOUNT_ID",
"changes": [{
"value": {
"messaging_product": "whatsapp",
"metadata": {
"display_phone_number": PHONE_NUMBER,
"phone_number_id": PHONE_NUMBER_ID
},
"contacts": [{
"profile": {
"name": "NAME"
},
"wa_id": PHONE_NUMBER
}],
"messages": [{
"from": PHONE_NUMBER,
"id": "wamid.ID",
"timestamp": TIMESTAMP,
"text": {
"body": "MESSAGE_BODY"
},
"type": "text"
}]
},
"field": "messages"
}]
}]
}
You can read the documentation of graph-api webhook,
https://developers.facebook.com/docs/graph-api/webhooks/getting-started#validate-payloads
Event Notifications are aggregated and sent in a batch with a maximum of 1000 updates. However batching cannot be guaranteed so be sure to adjust your servers to handle each Webhook individually.
You can also check the property-wise batch possibility in the provided link.
Apologies for the bad title, struggling to think of another.
Currently, I have 2 tables, publication and publicationStatus.
A publication will looking something like:
{
"id": "ckyil950d00027v2str5ljo7h",
"url_slug": "ckyil950e00037v2s4mqxvaho",
"type": "PEER_REVIEW",
"title": "Intersting review",
"content": "Content is optional at this stage",
"doi": "1093/ajae/aaq063",
"createdBy": "test-user-1",
"createdAt": "2022-01-17T11:12:50.845Z",
"updatedAt": "2022-01-17T11:12:50.847Z",
"publicationStatus": [
{
"status": "LIVE",
"createdAt": "2022-01-19T11:12:50.846Z",
"id": "ckyil950e00047v2sx4urbfte"
},
{
"status": "DRAFT",
"createdAt": "2022-01-17T11:12:50.846Z",
"id": "ckyil950e00047v2sx4urbfth"
}
],
"user": {
"id": "test-user-1",
"firstName": "Test",
"lastName": "User 1"
}
}
Where publication has a 1 to many relationship with publicationStatus.
What I need to do is a find query where it only returns a publication if the latest publicationStatus for that publication, has a status of LIVE.
Any ideas?
Edit:
The closest I could come to is this psuedo code:
await prisma.publication.findFirst({
where: {
id,
publicationStatus: {
where: {
status: 'LIVE'
},
take: 1,
orderBy: {
createdAt: 'desc'
}
},
}
});
This code does not work, but demonstrates a picture of what I'm trying to achieve.
Unfortunately, I don't think it's possible to do what you're hoping directly with a Prisma Query at the moment. I can suggest two possible workarounds though:
Approach 1: Fetch the most recent publicationStatus along with the publication and check the status inside your node application.
This is what the query would look like:
let publication = await prisma.publication.findFirst({
where: {
id
},
include: {
publicationStatus: {
orderBy: {
createdAt: 'desc'
},
take: 1
}
}
});
// check publication.publicationStatus[0].status and handle appropriately
Approach 2: Write a raw SQL query using the queryRaw method.
For a single publication, I think it would be easier to use approach 1. However, if you want to return all LIVE publications (you mentioned this in a comment), the performance characteristics of approach 1 might be undesirable.
Every time I query something from database I get objects like this:
{
"title": "faq",
"description": "",
"type": "application/pdf",
"size": 122974,
"filename": "faq.pdf",
"order": 0,
"createdAt": "2017-08-17T08:10:33.101Z",
"updatedAt": "2017-08-17T08:10:33.101Z",
"ACL": {
"role:cxcccccc_public": {
"read": true
},
"role:cxaaaaaaa_admin": {
"read": true,
"write": true
}
},
"objectId": "l6L5J1mRpH",
"__type": "Object",
"className": "Document"
}
A lot of these fields are Parse-specific: createdAt, updatedAt, ACL, className...
I didn't insert them when I added the row in the database but still they are returned when I do a query.
I want to expose this data through a clean REST API so I want to get rid of them.
Is there a way to get only the data I specified at insert time?
For now I am using lowdash, filtering with
data = _.omit(data, ['ACL', '__type', 'className', ...])
I am trying to figure out specific mongoDb query, so far unsuccessfully.
Documents in my collections looks someting like this (contain more attributes, which are irrelevant for this query):
[{
"_id": ObjectId("596e01b6f4f7cf137cb3d096"),
"code": "A",
"name": "name1",
"sys": {
"cts": ISODate("2017-07-18T12:40:22.772Z"),
}
},
{
"_id": ObjectId("596e01b6f4f7cf137cb3d097"),
"code": "A",
"name": "name2",
"sys": {
"cts": ISODate("2017-07-19T12:40:22.772Z"),
}
},
{
"_id": ObjectId("596e01b6f4f7cf137cb3d098"),
"code": "B",
"name": "name3",
"sys": {
"cts": ISODate("2017-07-16T12:40:22.772Z"),
}
},
{
"_id": ObjectId("596e01b6f4f7cf137cb3d099"),
"code": "B",
"name": "name3",
"sys": {
"cts": ISODate("2017-07-10T12:40:22.772Z"),
}
}]
What I need is to get current versions of documents, filtered by code or name, or both. Current version means that from two(or more) documents with same code, I want pick the one which has latest sys.cts date value.
So, result of this query executed with filter name="name3" would be the 3rd document from previous list. Result of query without any filter would be 2nd and 3rd document.
I have an idea how to construct this query with changed data model but I was hoping someone could lead me right way without doing so.
Thank you
How can I do something like:
sort('object.property')
(object is defined as type 'json' in the model)
with Waterline?
Note that I am using the latest stable build 0.9.~ which does not have associations yet.
A quick test showed the sails-mongo adapter allows sorting of JSON attributes out of the box (it is transparently reached to the mongo database).
First I created a blank controller and model using:
sails generate foo
Then I defined a data property with type json on my model:
module.exports = {
attributes: {
data: 'json'
}
};
I created several objects by doing a HTTP POST with data like this (I increased the sort value):
{
"data": {
"sort": 1
}
}
Afterwards I fetched my collection using this GET request:
http://localhost:1337/foo?sort=data.sort+desc
which will internally result in a call like
Foo.find().sort('data.sort desc').exec(callback);
The server response is now sorted by the given property:
[
{
"data": {
"sort": 3
},
"createdAt": "2014-04-13T09:35:49.734Z",
"updatedAt": "2014-04-13T09:35:49.734Z",
"id": "534a5a7553f1e98e09d1d86b"
},
{
"data": {
"sort": 2
},
"createdAt": "2014-04-13T09:35:45.814Z",
"updatedAt": "2014-04-13T09:35:45.814Z",
"id": "534a5a7153f1e98e09d1d86a"
},
{
"data": {
"sort": 1
},
"createdAt": "2014-04-13T09:35:41.958Z",
"updatedAt": "2014-04-13T09:35:41.958Z",
"id": "534a5a6d53f1e98e09d1d869"
}
]