I have some documents with the following structure...
{
user: "Joe",
lists: [
{ listId: "1234", listName: "dogs" },
{ listId: "5678", listName: "cats" }
]
}
I am trying to prepend a string to each listId field but I am stuck. Amongst other things I have tried...
db.users.updateMany(
{"lists.listId": /^[0-9a-f]{20,}$/},
[{$set:
{"lists.listId.$[]": {"$concat": ["0000", "$lists.listId"]}}
}]
)
But I got the error message: "FieldPath field names may not start with '$'"
Variations on this write results into the appropriate field, but not the results I'm after.
I've bashed my head against the docs for a few hours now but all the references I can find to using the positional operator to reference the value of the field that is being updated use the field name directly, not referenced as a property like I am doing. I've not really messed with pipelines a lot before and I'm finding it all a bit confusing! Someone kindly helped me with a closely related problem yesterday, using $map, and that worked great for a plain array of strings but I haven't had any luck adapting that to an array of objects with string properties. Sorry if this is Mongo 101, the docs are good, but there's a lot of them and I'm not sure which bits are relevant to this.
You can do it like this:
db.collection.users({},
[
{
"$set": {
lists: {
$map: {
input: "$lists",
in: {
$mergeObjects: [
{
"listName": "$$this.listName",
"listId": {
$concat: [
"0000",
"$$this.listId"
]
}
}
]
}
}
}
}
}
],
{
"multi": true
})
Here is the working example: https://mongoplayground.net/p/Q8kUTB6X5JY
Related
I'm trying to achieve a join on three String fields between two collections on MongoDB v. 4.4.3: one containing the original documents, the other the translations.
Both document types look like this:
{
"_id" : ObjectId("60644367b521563be8044f07"),
"dsId" : "2051918",
"lcId" : "data_euscreenXL_EUS_15541BBE705033639D4E06691D7A5D2E",
"pgId" : "1",
(...)
This MongoDB query does what I need, embedding the Translations in the result:
db.Original.aggregate([
{ $match: { query parameters } },
{ $lookup:
{
from: "Translation",
let: { "origDsId": "$dsId", origLcId: "$lcId", "origPgId": "$pgId" },
pipeline: [
{ $match:
{ $expr:
{ $and:
[
{ $eq: [ "$dsId", "$$origDsId" ] },
{ $eq: [ "$lcId", "$$origLcId" ] },
{ $eq: [ "$pgId", "$$origPgId" ] }
]
}
}
},
{ $project: { dsId: 0, _id: 0 } }
],
as: "translations"
}
}])
However, I can't figure out how to write the equivalent Morphia query. I updated to Morphia v.2.2, which adds the required features, but it's all very new and hasn't yet been documented on morphia.dev; I couldn't find much more on Javadoc either. This Morphia unit test on Github looked interesting and I tried copying that approach:
Aggregation<Original> query = datastore.aggregate(Original.class)
.match(eq("dsId", datasetId), eq("lcId", localId))
.lookup(Lookup.lookup(Translation.class)
.let("origDsId", value("$dsId"))
.let("origLcId", value("$lcId"))
.let("origPgId", value("$pgId"))
.pipeline(Match.match(expr(Expressions.of()
.field("$and",
array(Expressions
.of().field("$eq",
array(field("dsId"),
field("$origDsId"))),
Expressions
.of().field("$eq",
array(field("lcId"),
field("$origLcId"))),
Expressions
.of().field("$eq",
array(field("pgId"),
field("$origPgId"))))))))
.as("translations"));
...
This returns the Original documents, but fails to join the Translations.
The problem is that the syntax of the pipeline stage is rather puzzling. I wonder if anyone can shed some light on this?
the unit test example does not use (or need?) the double-$ form seen in "$$origDsId"? From the MongoDB documentation I understand that this form is used to refer to externally defined variables (eg in the "let" assignment before the "pipeline") but they don't work in the quoted example either;
what is the role of the static ArrayExpression "array"? It looks as if it's a kind of assignment container, where Expressions.of().field("$eq", array(field("dsId"), field("$origDsId"))) might mean something like "dsId" = "$origDsId" - which would be what I need (if it would work ;) )
I tried all sorts of combinations, using field("$origDsId"), value("$origDsId"), field("$$origDsId"), value("$$origDsId"), etcetera, but having no luck so far.
Thanks in advance!
I have a collection filled with docs that contain an "ip_addresses" field, which is an array of strings (IPs). I want to append '/32' to all of these values in all of my docs that don't already have a CIDR range suffix.
Here's my issue:
I don't know how to use the current value of the IP which is being iterated on.
Even if I did, $concat doesn't seem to work and throws an error even with placeholder values (as in the query below) - The dollar ($) prefixed field '$concat' in 'ip_addresses.0.$concat' is not valid for storage.
Here is my current query which throws the error:
db.getCollection('docs').update(
{},
{ $set: { "ip_addresses.$[ip]": { "$concat": [ "1.1.1.1", "/32" ] } } },
{
arrayFilters: [ { "ip": { $not: /.+\/\d{1,2}/ } } ],
multi: true
}
)
I'd appreciate help using the current values in the array in the $concat command and resolving the error.
Let's tackle each of the problems.
2 - $concat is an aggregation operator, hence it cannot be used for an update. you can view the list of available operators for an update here. You might notice that none of these "work" on actual document values which brings me back to the first question you had.
1 - In order you use a current document values within an update you have to use pipelined updates which is only available for Mongo v4.2+, if you're using a lesser Mongo version you have to fetch the documents into memory and do the update one by one in code. If you are on Mongo v4.2+ then the pipeline update syntax goes like this:
db.collection.updateMany(
{ip_addresses: {$exists: true}},
[
{
$set: {
ip_addresses: {
$map: {
input: '$ip_addresses',
as: 'ipAddress',
in: {
$cond: [
{
$regexMatch: {input: "$$ipAddress", regex: /.+\/\d{1,2}/}
},
'$$ipAddress',
{
$concat: [
'$$ipAddress',
'/32'
]
}
]
}
}
}
}
}
]
);
I am fairly new to MongoDB and cant seem to find a solution to this problem.
I have a database of documents that has this structure:
{
id: 1
elements: [ {elementId: 1, nr1: 1, nr2: 3}, {elementId:2, nr1:5, nr2: 10} ]
}
I am looking for a query that can add a value nr3 which is for example nr2/nr1 to all the objects in the elements array, so that the resulting document would look like this:
{
id: 1
elements: [ {elementId: 1, nr1: 1, nr2: 3, nr3:3}, {elementId:2, nr1:5, nr2: 10, nr3: 2} ]
}
So I imagine a query along the lines of this:
db.collection.updateOne({id:1}, {$set:{"elements.$[].nr3": nr2/nr1}})
But I cant find how to get the value of nr2 and nr1 of the same object in the array.
I found some similar questions on stackoverflow stating this is not possible, but they were 5+ years old, so I thought maybe they have added support for something like this.
I realize I can achieve this with first querying the document and iterate over the elements-array doing updates along the way, but for the purpose of learning I would love to see if its possible to do this in one query.
You can use update with aggregation pipeline starting from MongoDB v4.2,
$map to iterate loop of elements
divide nr2 with nr1 using $divide
merge current object and new field nr3 using $mergeObjects
db.collection.updateOne(
{ id: 1 },
[{
$set: {
elements: {
$map: {
input: "$elements",
in: {
$mergeObjects: [
"$$this",
{ nr3: { $divide: ["$$this.nr2", "$$this.nr1"] } }
]
}
}
}
}
}]
)
Playground
db.collection.update(
{ id:1},
{ "$set": { "elements.$[elem].nr3":elements.$[elem].nr2/elements.$[elem].nr1} },
{ "multi": true }
);
I guess this should work
I tried to use length on an array that is within another array and got an error:
db.quotes.find( { $where: "this.booksWithQuote.authors.length>0" } )
// fails with this.booksWithQuote.authors is undefined
I'm basing this syntax on this example MongoDB – find all documents where an array / list size is greater than N that works:
db.domain.find( {booksWithQuote: {$exists:true}, $where:'this.booksWithQuote.length>0'} )
// Above works
So I'm wondering if this is possible, how can I find all documents that have array1.array2 where array2 length is greater than zero.
I've tried the following but it fails with a syntax error:
db.quotes.find( {
"booksWithQuote": {$exists: true} },
"booksWithQuote.authors": {$exists: true} },
$where: "booksWithQuote.authors.length>0" } )
// fails with this.booksWithQuote.authors is undefined
It's worth pointing out that if I knew the author of a book, the nested array with array searching worked. Pretty cool!
db.quotes.find( { "booksWithQuote.authors" : "Sam Fisher" } )
// Returns all quotes that have a book that has the author Sam Fisher
But in my case I'm just trying to find all of the quotes that have more than one author on any given book.
To follow along, consider this example.
I have a collection of quotes, and each quote has a list of books where the quote was used, and each book has an array of authors.
Below is some sample data so you can understand the structure of the data. It shows one quote with no books, another quote with a book but no authors, and a third quote with books and several authors.
[
{
quoteText: "Love to code",
booksWithQuote: [ ]
},
{
quoteText: "Another Quotesake",
booksWithQuote: [
{
title: "Where is elmo",
authors: [ ]
}
]
},
{
quoteText: "For goodness sake",
booksWithQuote: [
{
title: "The search for Elmo",
authors: [
"John Smith",
"Sam Fisher",
"Jim Wiggins"
]
},
{
title: "Finding Elmo",
authors: [ "Sam Fisher" ]
},
{
title: "Mercy me",
authors: [ ]
}
]
}
]
So to reiterate, How can I find all documents that have an array within another array where the 2nd array has one or more elements?
You can use $exists operator and refer to first element of an array using dot notation:
db.col.find({ "booksWithQuote.authors.0": { $exists: true } })
MongoDB documentation: https://www.mongodb.com/docs/manual/reference/operator/query/expr/
you can use $expr to achieve this
$expr:{$gte: [{$size: "$selectedArray"}, size]}
I've seen similar questions but far to complex for a newbie like myself. Plus none of the answers offered a syntax break down.
My database:
name :"Kim"
points: [
{ category:"Purchase",
points: -50},
{ category: "Wage",
points : 275},
{ category: "Purchase",
points: -40}
]
name :"Meghan"
points: [
{ category:"Contest",
points: 130},
{ category: "Purchase",
points : -25},
{ category: "Games",
points : 50}
]
]
So in the mongo shell I'm trying to get it find every points.category that equals "Purchase" but only return to me the last one. I attempted to achieve this by using aggregate, $project and $slice. My issue is that I don't understand the syntax well enough to know whether $slice goes inside $project or outside separated by a comma. I understand the syntax when I use $project by itself or $slice by itself but I have no clue how to use all these things together to something magical. If someone could help me solve my problem and explain how to combine all these things properly together, I'd be forever indebted. I've read over the docs but nothing showed me how to use everything all at once.
I want to search the student who's name is Kim points to find the last Purchase she made. The result I want is
[{points:-40}]
If you can use more projection with other aggregation operator, you may try this.
db.students.aggregate([
{ $match : { name : "Kim" } },
{ $project:
{
points: {
$filter : { input: "$points", as:"pts", cond: {$eq: ["$$pts.category", "Purchase"]} }
}
}
},
{ $project:
{
_id : 0,
points: { $arrayElemAt: [{$slice: [ "$points.points", -1]},0] }
}
}
])