i have a lookup activity in ADF which retrieves data from table like below
{
"count": 1,
"value": [
{
"Notebook_path": "/Users/Name#work.com/Project/ingestion/Cust1/ADT"
}
],
"effectiveIntegrationRuntime": "AutoResolveIntegrationRuntime (um)",
"billingReference": {
"activityType": "PipelineActivity",
"billableDuration": [
{
"meterType": "AzureIR",
"duration": 0.016666666666666666,
"unit": "DIUHours"
}
]
},
"durationInQueue": {
"integrationRuntimeQueue": 0
}
}
now i want to use the Notebook_path from above results and pass it to a Notebook activity to run that particular notebook as in the output, from lookup activity.
since its in array its creating problems . would appreciate some help here .
tried a lot of things but dint help.
Thanks in Advance
I have a similar lookup output. The following is a reference image of the same.
You can use the following dynamic content to get the desired result.
#activity('Lookup1').output.value[0]['notebook_path']
My pipeline failed because I don't have a notebook, but you can see that the activity is trying to access the required path.
Related
Overview - I'm trying to sync some data to google firestore using Integromat. However I cannot seem to find the correct way to save an array as the output for a particular field. It seems like this should be easy but everything I've tried has failed so far.
Example, using the simplest form of this I have an input string like so
Input JSON:
{ "Brand": "Ford", "widgets": [0,1,2] }
And I basically just want to save that in the same structure into firebase, but I can't seem to configure the 'Update Firestore Document' module correctly. The closest I can do is to save it as a string, so it looks like this in firebase:
Output Firestore: { "Brand": "Ford", "widgets": "0,1,2" }
Below I'm attaching images of the integromat setup, showing how I'm trying to hookup the output values into the firestore module. When I try and pass the array directly I get the error message. "Array of objects expected in the parameter 'Value'"
I have reviewed your integration and one thing that needs to be fixed is the way you are passing the Array of Values in Google Cloud FireStore through Integromat. When reviewing the API the expected format for FireStore looks something like this,
{
"fields": {
"widgets": {
"arrayValue": {
"values": [
{
"integerValue": 0
},
{
"integerValue": 1
},
{
"integerValue": 2
}
]
}
},
"brand": {
"stringValue": "Ford"
}
}
}
But, Since the app is developed in Integromat there are few changes while you are using Map instead of setting each item in Firestore Array. After reviewing it, Integromat expects objects with the following fields as arrays that you need to create and pass instead of using widgets[] as you have done in the current implementation.
[
value : 0,
valueType : "integerValue"
]
To achieve this, I have created the following Scenario(Not Sure how operation effective is it, but I get it working),
Data Structure Used is following which you can use through the generator and is used for Aggregator and Parse JSON module,
[{
"value": 0,
"valueType": "integerValue"
}, {
"value": 1,
"valueType": "integerValue"
}, {
"value": 2,
"valueType": "integerValue"
}]
Array Aggregator in second last step will then aggregate value and valueType and will be used in Google FireStore as,
I have code that uses existing collection that I want to update name field value which is present in taskMap key how i run with run liqubase changeset for that .
sample code :
"changes": [
{
"collectionName": "bulk_import_job_spec"
},
{
"findOneAndUpdate": {
{"_id": 101},
[{"$set": {"task_map": {"import": {"_id": 1, "name": "Category Association"}}}}]
},
{
"returnNewDocument": true
}
]
your questions is quite vague. Please provide some more details so that we can understand your issue.
I am new to NoSQL and morphia. I am using Morphia to query MongoDB.
I have a sample collection as below:
[
{
"serviceId": "id1",
"serviceName": "ding",
"serviceVersion": "1.0",
"files": [
{
"fileName": "b.html",
"fileContents": "contentsA"
},
{
"fileName": "b.html",
"fileContents": "contentsB"
}
]
},
{
"serviceId": "id2",
"serviceName": "ding",
"serviceVersion": "2.0",
"files": [
{
"fileName": "b.html",
"fileContents": "contentsA"
},
{
"fileName": "b.html",
"fileContents": "contentsB"
}
]
}
]
I would like to fetch an element in "files" List , given service name, service version and filename., using Morphia.
I was able to get what I want using the query below:
db.ApiDoc.find({ serviceName: "ding", serviceVersion: "2.0"}, { files: { $elemMatch: { fileName: "b.html" } } }).sort({ "_id": 1}).skip(0).limit(30);
What I tried so far :
I tried using "elemmatch" api that morphia has, but no luck.
query = ...createQuery(
Result.class);
query.and(query.criteria("serviceName").equal("ding"),
query.criteria("serviceVersion").equal(
"2.0"));
query.filter("files elem",BasicDBObjectBuilder.start("fileName", "a.html").get());
I seem to get the entire Result collection with all the files. I would like to get only the matched files(by filename).
can some one help me how I can get this to work?
Thanks
rajesh
I don't believe it's possible to get just the matching sub element. You can request just to have the 'files' array returned but all elements will be included in the result set and you will have to refilter in your code.
The other option is to make Files a collection of its own with a serviceId field and then you'll have more power to load only certain files.
It's possible to do that.
the filter doesn't really work like projection.
try this :
datastore.createQuery(Result.class)
.field("serviceName").equal("dong")
.field("serviceVersion").equal("2.0")
.field("files.filename").equal("a.html")
.project("files.$.filename", true);
my gamefamilies collection looks like this
{
"_id": ObjectId('54cc3ee7894ae60c1c9d6c74'),
"game_ref_id": "REF123",
..
"yearwise_details": [
{
"year": 1,
...
"other_details": [
{
"type": "cash",
"openingstock": 988
..
},
{
"type": "FLU",
"openingstock": 555
..
},
..other items
]
},
{
"year": 2,
...
"other_details": [
{
"type": "cash",
"openingstock": 3000,
....
},
...
{
"type": "ghee",
"openingstock": 3000,
...
},
..
]
}
]
}
My update query
db.gamefamilies.update({"game_ref_id": "REF123", "teamname": "manisha","yearwise_details.year": 2, "yearwise_details.other_details.type": "ghee"}, {"$set": {"yearwise_details.0.other_details.$.openingstock": 555} });
Document is getting picked up correctly. I expect to update year 2's item type="ghee" but instead year 1's 2nd item (type FLU) gets updated. What am I doing wrong ?
Any help would be greatly appreciated.
regards
Manisha
Unfortunately, there is not yet support for nested $ positional operator updates.
So you can hardcode the update with
db.gamefamilies.update({"game_ref_id": "REF123",
"teamname": "manisha",
"yearwise_details.year": 2,
"yearwise_details.other_details.type": "ghee"},
{"$set":
{"yearwise_details.1.other_details.$.openingstock": 555}});
But notice that the yearwise_details.1.other_details is hardcoding that you want the second value of the array (it is 0-indexed, so the 1 is referencing the second element). I am assuming you found the command you have in your question because it worked for the first element of the array. But it will only ever work on the first element and the command above will only ever work on the second element.
i want to query mongodb in sailsjs.
this is structure of my db
{
"users": [
"52ed09e1d015533c124015d5",
"52ed4bc75ece1fb013fed7f5"
],
"user_msgs": [
{
"sender": "52ed09e1d015533c124015d5",
"sendTo": "52ed4bc75ece1fb013fed7f5",
"msg": "ss"
}
],
"createdAt": ISODate("2014-02-06T16:12:17.751Z"),
"updatedAt": ISODate("2014-02-06T16:12:17.751Z"),
"_id": ObjectID("52f3b461f46da23c111582f6")
}
I want to search those "users" who who match array [
"52ed09e1d015533c124015d5",
"52ed4bc75ece1fb013fed7f5"
]
Message.find({user: ["52ed09e1d015533c124015d5","52ed4bc75ece1fb013fed7f5"]})
this query returns all objects which contains 1 OR 2 ..but i need only those which exacly match 1 AND 2,
i have also tried $all, and etc.. but did not worked
please tell me how to write query with sailsjs supported syntex to get those user
You'll need to use the native Mongo adapter for this:
Message.native(function(err, collection) {
collection.find({users:{'$all':["52ed09e1d015533c124015d5","52ed4bc75ece1fb013fed7f5"]}}).toArray(function(err, results) {
// Do something with results
});
});
Message.find()
.where({users: "52ed09e1d015533c124015d5", "52ed4bc75ece1fb013fed7f5"})
.exec(function(){
// do something
});
While the above code may work to pull in just those users. I think a better solution would be to define your user ID's in your message model.
I would add the following attributes to your messages model:
senderID: {
type: 'string'
},
receiverID: {
type: 'string'
}
Now you can make that query more efficient by using the following query:
Message.find()
.where({senderID: "52ed09e1d015533c124015d5"})
.where({receiverID: "52ed4bc75ece1fb013fed7f5"})
.exec(function(){
// do something
});
This is the route I would take.