Only keep items that match string in array in Azure Data Factory - azure-data-factory

I have a very large array, similar to this one:
{
"name":"latest_test",
"value":[
{
"name":"2016-06-27-12Z",
"type":"Folder"
},
{
"name":"2016-06-28-00Z",
"type":"Folder"
},
{
"name":"2016-06-28-12Z",
"type":"Folder"
},
{
"name":"2016-06-29-00Z",
"type":"Folder"
},
{
"name":"2016-06-29-12Z",
"type":"Folder"
}
]
}
I only want to keep the items that have 2016-06-29 in their name. Such that I have a new array that only consists of 2016-06-29-00Z and 2016-06-29-12Z.
I tried to use a filter with #contains(item(), '2016-06-29') but this returns 0.

item() is the entire object for an element in the array. In order to filter on a property have to specify the property also.
Change the filter condition to
#contains(item().name, '2016-06-29')

Related

How can I return the element I'm looking for inside a nested array?

I have a database like this:
[
{
"universe":"comics",
"saga":[
{
"name":"x-men",
"characters":[
{
"character":"wolverine",
"picture":"618035022351.png"
},
{
"character":"cyclops",
"picture":"618035022352.png"
}
]
}
]
},
{
"universe":"dc",
"saga":[
{
"name":"spiderman",
"characters":[
{
"character":"venom",
"picture":"618035022353.png"
}
]
}
]
}
]
and with this code I manage to update one of the objects in my array. specifically the object where character: wolverine
db.mydb.findOneAndUpdate({
"universe": "comics",
"saga.name": "x-men",
"saga.characters.character": "wolverine"
}, {
$set: {
"saga.$[].characters.$[].character": "lobezno",
"saga.$[].characters.$[].picture": "618035022354.png",
}
}, {
new: false
}
)
it returns all my document, I need ONLY the document matched
I would like to return the object that I have updated without having to make more queries to the database.
Note
I have been told that my code does not work well as it should, apparently my query to update this bad, I would like to know how to fix it and get the object that matches these search criteria.
In other words how can I get this output:
{
"character":"wolverine",
"picture":"618035022351.png"
}
in a single query using filters
{
"universe": "comics",
"saga.name": "x-men",
"saga.characters.character": "wolverine"
}
My MongoDB knowledge prevents me from correcting this.
Use the shell method findAndModify to suit your needs.
But you cannot use the positional character $ more than once while projecting in MongoDb, so you may have to keep track of it yourself at client-side.
Use arrayFilters to update deeply nested sub-document, instead of positional all operator $[].
Below is a working query -
var query = {
universe: 'comics'
};
var update = {
$set: {
'saga.$[outer].characters.$[inner].character': 'lobezno',
'saga.$[outer].characters.$[inner].picture': '618035022354.png',
}
};
var fields = {
'saga.characters': 1
};
var updateFilter = {
arrayFilters: [
{
'outer.name': 'x-men'
},
{
'inner.character': 'wolverine'
}
]
};
db.collection.findAndModify({
query,
update,
fields,
arrayFilters: updateFilter.arrayFilters
new: true
});
If I understand your question correctly, your updating is working as expected and your issue is that it returns the whole document and you don't want to query the database to just to return these two fields.
Why don't you just extract the fields from the document returned from your update? You are not going to the database when doing that.
var extractElementFromResult = null;
if(result != null) {
extractElementFromResult = result.saga
.filter(item => item.name == "x-men")[0]
.characters
.filter(item => item.character == "wolverine")[0];
}

mongodb need to populate a new field with an old fields value, without destroying other data

I have a situation where a model changed at some point in time and I am faced with (for argument sake) half my data liks like this
{
_id: OID,
things: [{
_id:OID,
arm: string,
body: string
}],
other: string
}
and the other half of my data look like this
{
_id: OID,
things: [{
_id:OID,
upper_appendage: string,
body: string
}],
other: string
}
I would like to 'correct' half of the data - so that I DON'T have to accommodate both names for 'arm' in my application code.
I have tried a couple different things:
The first errors
db.getCollection('x')
.find({things:{$exists:true}})
.forEach(function (record) {
record.things.arm = record.things.upper_appendage;
db.users.save(record);
});
and this - which destroys all the other data in
db.getCollection('x')
.find({things:{$exists:true}})
.forEach(function (record) {
record.things = {
upper_appendage.arm = record.things.upper_appendage
};
db.users.save(record);
});
Keeping in mind that there is other data I want to maintain...
How can I do this???
the $rename operator should have worked for this job but unfortunately it doesn't seem to support nested array fields (as of mongodb server 4.2). instead you'd need a forEach like the following:
db.items.find({
things: {
$elemMatch: {
arm: {
$exists: true
}
}
}
}).forEach(function(item) {
for (i = 0; i != item.things.length; ++i)
{
item.things[i].upper_appendage = item.things[i].arm;
delete item.things[i].arm; ;
}
db.items.update({
_id: item._id
}, item);
})
note: i've assumed you want to make all records have upper_appendageand get rid of 'arm' field. if it's the other way you want, just switch things around.

How to remove property from each item in object with MongoDB

I have an object in my document like this:
revisions: {
<revisionId>: {
...
someProperty: { ... },
},
...
}
I would like to remove someProperty from every nested object in revisions object (i.e. iterate through all the object revisionId keys). I'm asking because if it's not possible I'd rather convert this to an array than do it on the server and possibly overwrite any updates in the mean time.
I tried $unset but I'm only aware of the $[] operator which only works for arrays e.g.
$unset: {
'revisions.$[].someProperty': 1
}
Thanks
You're almost there:
{ $unset: { someProperty: "" } }
The value doesn't matter from memory
You could then use a cursor to iterate through each doc and remove the unwanted property.
db.collection.find().forEach(<function>)
EDIT: Sorry, realising it's a nested model with arbitrary key for the top level property, makes it more tricky.
t.forEach(function( row) {
var newRevisions = [];
row.revisions.fields.forEach( function( revision ){
delete revision.someProperty;
newRevisions.push(revision);
} )
t.update(
{ _id: row._id },
{ "$set": { "revisions": newRevisions} }
);
});
Huge caveat, totally untested, but should give you starting point.
For each row
For each revision in revisions object
Remove the someProperty property
Set the revisions property back the collection based on _id

Update nested array object (put request)

I have an array inside a document of a collection called pown.
{
_id: 123..,
name: pupies,
pups:[ {name: pup1, location: somewhere}, {name: pup2, ...}]
}
Now a user using my rest-service sends the entire first entry as put request:
{name: pup1, location: inTown}
After that I want to update this element in my database.
Therefore I tried this:
var updatedPup = req.body;
var searchQuery = {
_id : 123...,
pups : { name : req.body.name }
}
var updateQuery = {
$set: {'pups': updatedPup }
}
db.pown.update(searchQuery, updateQuery, function(err, data){ ... }
Unfortunately it is not updating anythig.
Does anyone know how to update an entire array-element?
As Neil pointed, you need to be acquainted with the dot notation(used to select the fields) and the positional operator $ (used to select a particular element in an array i.e the element matched in the original search query). If you want to replace the whole element in the array
var updateQuery= {
"$set":{"pups.$": updatedPup}
}
If you only need to change the location,
var updateQuery= {
"$set":{"pups.$.location": updatedPup.location}
}
The problem here is that the selection in your query actually wants to update an embedded array element in your document. The first thing is that you want to use "dot notation" instead, and then you also want the positional $ modifier to select the correct element:
db.pown.update(
{ "pups.name": req.body.name },
{ "$set": { "pups.$.locatation": req.body.location }
)
That would be the nice way to do things. Mostly because you really only want to modify the "location" property of the sub-document. So that is how you express that.

Mongodb: assert that all elements in an array have a field not null

Given a collection with documents like this:
Task Collection document
[
{
"_id"=>BSON::ObjectId('54d674b64d42504b6a000000'),
"submissions"=>
[{"_id"=>BSON::ObjectId('54d674b64d42504b6a010000'),
"grade"=>nil,
"user_id"=>BSON::ObjectId('54d1e2454d42503069060000')},
{"_id"=>BSON::ObjectId('54d674b64d42504b6a020000'),
"grade"=>nil,
"user_id"=>BSON::ObjectId('54d1e2454d42503069070000')},
{"_id"=>BSON::ObjectId('54d674b64d42504b6a030000'),
"grade"=>nil,
"user_id"=>BSON::ObjectId('54d1e2454d42503069080000')}
],
},
{
"_id"=>BSON::ObjectId('54d674b64d42504b6a100000'),
"submissions"=>
[{"_id"=>BSON::ObjectId('54d674b64d42504b6a010000'),
"grade"=>5,
"user_id"=>BSON::ObjectId('54d1e2454d42503069060000')},
{"_id"=>BSON::ObjectId('54d674b64d42504b6a020000'),
"grade"=>7,
"user_id"=>BSON::ObjectId('54d1e2454d42503069070000')},
{"_id"=>BSON::ObjectId('54d674b64d42504b6a030000'),
"grade"=>nil,
"user_id"=>BSON::ObjectId('54d1e2454d42503069080000')}
],
},
{
"_id"=>BSON::ObjectId('54d674b64d42509b6a000000'),
"submissions"=>
[{"_id"=>BSON::ObjectId('54d674b64d42504b6a010000'),
"grade"=>8,
"user_id"=>BSON::ObjectId('54d1e2454d42503069060000')},
{"_id"=>BSON::ObjectId('54d674b64d42504b6a020000'),
"grade"=>7,
"user_id"=>BSON::ObjectId('54d1e2454d42503069070000')},
{"_id"=>BSON::ObjectId('54d674b64d42504b6a030000'),
"grade"=>6,
"user_id"=>BSON::ObjectId('54d1e2454d42503069080000')}
],
}
]
How can I get all the tasks documents whose submissions array contains no nil grade?
The result in the example would contain just the last one.
I am using Mongoid, but I welcome a pure Mongodb query too.
Using elem_match and a negative comparison:
Task.where(:submissions.elem_match => { :grade.ne => nil })