Using MongoDB 3.6
I have document structure like below
`
{
"_id" : ObjectId("5d88"),
"Equipments" : [
{
"InnerEquipments" : {
"AssetId" : 678
},
"AssetID" : 456
}
],
"AssetID" : 123
}
I want rename the field from AssetID/AssetId to Asset_ID at all levels.
How can I do this with mongo shell.
The following code can do the trick:
// Getting all documents from the collection
var data = db.collection.find({},{"_id":0}).toArray();
// Converting the data into JSON string
var string = JSON.stringify(data);
// Replacing all variations of assetid with Asset_ID
string = string.replace(/assetid/ig,"Asset_ID");
// Removing existing documents from collection
db.collection.remove({});
// Converting the string back to JSON array and inserting it into the DB
db.collection.insertMany(JSON.parse(string));
Before:
{
"_id" : ObjectId("5d89e9ab0558a18dd9cfc03a"),
"Equipments" : [
{
"InnerEquipments" : {
"AssetId" : 678
},
"AssetID" : 456
}
],
"AssetID" : 123
}
After:
{
"_id" : ObjectId("5d89eea80558a18dd9cfc03b"),
"Equipments" : [
{
"InnerEquipments" : {
"Asset_ID" : 678
},
"Asset_ID" : 456
}
],
"Asset_ID" : 123
}
You need to make a script for this.
I have not tested it. Make changes accordingly.
db.collection.find({}).forEach(function(doc) {
if(doc['AssetID']) {
doc['Asset_ID'] = doc['AssetID'];
delete doc['AssetID'];
} else if (doc['AssetId']) {
doc['Asset_ID'] = doc['AssetId'];
delete doc['AssetId']
}
if(doc.Equipments && doc.Equipments.length) {
doc.Equipments.forEach(function(rec) {
if(rec['AssetID']) {
rec['Asset_ID'] = rec['AssetID'];
delete rec['AssetID'];
} else if (rec['AssetId']) {
rec['Asset_ID'] = rec['AssetId'];
delete rec['AssetId']
}
if(rec['InnerEquipments']['AssetID']) {
rec['InnerEquipments']['Asset_ID'] = rec['InnerEquipments']['AssetID'];
delete rec['InnerEquipments']['AssetID'];
} else if (rec['InnerEquipments']['AssetId']) {
rec['InnerEquipments']['Asset_ID'] = rec['InnerEquipments']['AssetId'];
delete rec['InnerEquipments']['AssetId']
}
})
}
db.collection.update({'_id':doc._id},doc);
});
Related
Basically I want to update all documents inside one collection. The update is just adding 2 hours to date fields present in each document.
The documents all follow a basic structure like this :
{
code : 1,
file : {
dates : {
start : 2018-05-27 22:00:00.000Z,
end : 2018-05-27 22:00:00.000Z,
},
otherInfos : {
...
...
}
}
}
Here is my query :
var cursor = db.getCollection('files').find({});
while(cursor.hasNext()){
e = cursor.next();
let delta = 120*60*1000; //2 hours
if(e.file.dates) {
let fileStartDate = e.file.dates.start ? new Date(e.file.dates.start.getTime() + delta) : null;
let fileEndDate = e.file.dates.end ? new Date(e.file.dates.end.getTime() + delta) : null;
if(fileStartDate) {
e.file.dates.start = fileStartDate;
}
if(fileEndDate) {
e.file.dates.end = fileEndDate;
}
}
print(e);
db.getMongo().getDB('myDB').files.updateOne(
{"code":e.code},
{
$set: {"file.dates.start": fileStartDate, "file.dates.end": fileEndDate}
})
}
I am testing the query with around 20 documents and the first 10 are perfectly printed and updated with +2hours as expected but then for the second half the dates remain the exact same than before (both with the print and update).
All the documents have the same structure and same Date type so I don't understand why the query doesn't go all the way.
EDIT :
Here is a document that was succesfully updated :
{
"_id" : ObjectId("5b36c7fdd515e80009e7cc84"),
"code" : "1",
"file" : {
"dates" : {
"start" : ISODate("2018-06-11T22:00:00.000Z"),
"end" : ISODate("2018-06-11T22:00:00.000Z")
}
}
}
became as expected
{
"_id" : ObjectId("5b36c7fdd515e80009e7cc84"),
"code" : "1",
"file" : {
"dates" : {
"start" : ISODate("2018-06-12T00:00:00.000Z"),
"end" : ISODate("2018-06-12T00:00:00.000Z")
}
}
}
but for example this document :
{
"_id" : ObjectId("5b36c7ffd515e80009e7cf03"),
"code" : "15",
"file" : {
"dates" : {
"start" : ISODate("2018-09-02T22:00:00.000Z"),
"end" : ISODate("2019-09-26T22:00:00.000Z")
}
}
}
stayed the exact same
With MongoDBv4.2+, you can do an update with aggregation pipeline. Use $add to increment 2 hour * 60 minute * 60 seconds * 1000 milliseconds.
db.collection.update({},
[
{
"$set": {
"file.dates.start": {
$add: [
"$file.dates.start",
7200000
]
},
"file.dates.end": {
$add: [
"$file.dates.end",
7200000
]
}
}
}
],
{
multi: true
})
Here is the Mongo playground for your reference.
db.getMongo().getDB('myDB').files.updateOne(
{"code":e.code},
{
$set: {"file.dates.start": fileStartDate, "file.dates.end": fileEndDate}
})
updateOne only allows update on one document
You should use updateMany() to update more than 1 document
https://www.mongodb.com/docs/manual/reference/method/db.collection.updateMany/
Got a weird bug that I can't quite figure out.
I have some pymongo code that looks like this:
from pymongo import UpdateOne
client = pymongo.MongoClient()
...
def update_image_locations(user_key, dataset_key, preset_name,
keys_and_coords):
db = docdb_client.db
col = db.col
operations = []
query = {'ownerKey': user_key, 'imageInfo.datasetKey': dataset_key}
for key_and_coords in keys_and_coords:
query['key'] = key_and_coords['key']
operations.append(
pymongo.UpdateOne(
query, {
'$set': {
'imageInfo.presets.%s.coords' % preset_name:
key_and_coords['coords']
}
}))
print(operations)
if len(operations) > 0:
print(col.bulk_write(operations, ordered=False).bulk_api_result)
# This section fails with a KeyError.
cursor = col.find({
'ownerKey': user_key,
'imageInfo.datasetKey': dataset_key
}, {'imageInfo': 1}
)
for doc in cursor:
print(doc['imageInfo']['presets'])
If I print out the bulk_write output, I get the following.
{'writeErrors': [], 'writeConcernErrors': [], 'nInserted': 0, 'nUpserted': 0, 'nMatched': 65, 'nModified': 65, 'nRemoved': 0, 'upserted': []}
which as far as I can tell is exactly what I expect.
However, I get KeyError failures for all but the last document in the collection when I try to iterate through the documents that should ostensibly have the new field. If I then go into the actual mongodb shell, I can confirm that only the last operation from the bulk_write seems to have actually gone off.
Based on the bulk_api_result I would expect that all of the documents would be updated, instead of only the last one. What's going on?
EDIT:
As requested, before and after queries. I'm not showing the full doc because there's a lot of vector embedding info that's going to muddle things.
Query:
> db.user_uploads.find({}, {'imageInfo.presets': 1})
Before:
{ "_id" : ObjectId("6074792104cc23375a8f979a"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f979b"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f979c"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f979d"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f979e"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f979f"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f97a0"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f97a1"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f97a2"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f97a3"), "imageInfo" : { } }
After:
{ "_id" : ObjectId("6074792104cc23375a8f979a"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f979b"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f979c"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f979d"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f979e"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f979f"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f97a0"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f97a1"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f97a2"), "imageInfo" : { } }
{ "_id" : ObjectId("6074792104cc23375a8f97a3"), "imageInfo" : { "presets" : { "preset_one" : { "coords" : [ 2.229365348815918, 1.4654869735240936 ] } } } }
Turns out the answer has to do with how the query is constructed. Specifically, this works:
for key_and_coords in keys_and_coords:
query = {'key': key_and_coords['key']}
operations.append(
pymongo.UpdateOne(
query, {
'$set': {
'imageInfo.presets.%s.coords' % preset_name:
key_and_coords['coords']
}
}))
and this fails:
query = {}
for key_and_coords in keys_and_coords:
query['key'] = key_and_coords['key']
operations.append(
pymongo.UpdateOne(
query, {
'$set': {
'imageInfo.presets.%s.coords' % preset_name:
key_and_coords['coords']
}
}))
I think what's happening here is some async javascript-esque magic, where the query object is passed by reference to the bulk operation which then executes them once all of the bulk operations are in place. Since the query is passed by reference, the actual key value gets overwritten each time until the last one (which is also why only the last object is updated). Unfortunately this was tough to catch because printing out the queries and the operations both looked fine, but the async kicked in at execution. Still, not really an issue with pymongo after all.
Thanks to everyone who responded!
this is my mongoDb footballers collection :
[
{
"_id" : ObjectId("5d83b4a7e5511f28847f1884"),
"prenom" : "djalil",
"pseudo" : "dja1000",
"email" : "djalil#gmail.com",
"selectionned" : [
{
"_id" : "5d83af3be5511f28847f187f",
"role" : "footballeur",
"prenom" : "Gilbert",
"pseudo" : "Gilbert",
},
{
"_id" : "5d83b3d5e5511f28847f1883",
"role" : "footballeur",
"prenom" : "Xavier",
"pseudo" : "xav4544",
}
]
},
{
"_id" : ObjectId("5d83afa8e5511f28847f1880"),
"prenom" : "Rolande",
"pseudo" : "Rolande4000",
"email" : "rolande#gmail.com",
"selectionned" : [
{
"_id" : "5d83b3d5e5511f28847f1883",
"role" : "footballeur",
"prenom" : "Xavier",
"pseudo" : "xav4544",
}
]
}
}
How could I delete each selectionned people who has the 5d83b3d5e5511f28847f1883 _id through all of the collection?
I do need xavier to deseappear from any 'selectionned' array , just like doing a 'delete cascade' in SQL language
This is what I've tried with no luck :
function delete_fb_from_all(fb){
var ObjectId = require('mongodb').ObjectID; //working
var idObj = ObjectId(fb._id); //working
try {
db.collection('footballers').remove( { "selectionned._id" : idObj } );
console.log('All have been erased');
} catch (e) {
console.log(e);
}
}
And this too is not working :
db.collection('footballers.selectionned').remove( { "_id" : idObj } );
i really dont know how to do this.
i'm trying out this right now :
db.collection.update({'footballers.selectionned': idObj }, {$pull: {footballers:{ selectionned: idObj}}})
This is the error :
TypeError: db.collection.update is not a function
I think that the solution is maybe there :
https://docs.mongodb.com/manual/reference/operator/update/pull/#pull-array-of-documents
EDIT 1
i'm currently trying ou this :
var ObjectId = require('mongodb').ObjectID; //working
var idObj = ObjectId(fb._id); //working
try {
db.collection('footballers').update(
{ },
{ $pull: { selectionned: { _id: idObj } } },
{ multi: true }
)
} catch (e) {
console.log(e);
}
SOLVED :
Specifiying the email, it is now working, I guess the problem was comin from the _id field :
try {
db.collection('footballers').update(
{ },
{ $pull: { selectionned: { email: fb.email } } },
{ multi: true }
)
} catch (e) {
console.log(e);
}
Object ID :
The issue is may be on your object id creation. No need to make string-id with mongoDB object id.
// No need
var ObjectId = require('mongodb').ObjectID;
var idObj = ObjectId(fb._id);
// do as normal string
db.collection('footballers').remove( { "selectionned._id" : fb._id } );
I need to update my mongo Document following is my schema
{ "_id" : 7742, "current_count" : { "abc" : 6 } }
And need to update current_count object where i don't know the key and if the same key occurred need to replace the value as below.
{ "_id" : 7742, "current_count" : { "abc" : 6 ,"xyz":10} }
if "abc" encounters just need to replace it's value,
{ "_id" : 7742, "current_count" : { "abc" : 456 ,"xyz":10} }
You just need to loop over the keys which you getting as payload and need to create and set dynamic key value pair to $set operator
const data = {
abc: 456,
xyz: 10
}
const object = {}
for (var key in data) {
object[`current_count.${key}`] = data[key]
}
Which would output as
current_count.abc: 456
current_count.xyz: 10
And then just put it in your query
const query = { "_id" : 7742 };
const update = { "$set": object };
await Model.findOneAndUpdate(query, update)
Need help on MongoDB nested query. Below is my mongo collection.
Preference collection
{
"_id" : "user123",
"preferences" : {
"product-1" : {
"frequency" : "Weekly",
"details" : {
"email" : {
"value" : "On"
}
}
},
"product-2" : {
"preferencesFor" : "mpc-other",
"preferencesForType" : "Product",
"details" : {
"email" : {
"value" : "Off"
}
}
},
"product-3" : {
"preferencesFor" : "mpc-other",
"preferencesForType" : "Product",
"details" : {
"email" : {
"value" : "On"
}
}
}
}
}
Product Collection
{
"_id" : "product-1",
"name" : "Geo-Magazine"
}
{
"_id" : "product-2",
"name" : "History-Magazine"
}
{
"_id" : "product-3",
"name" : "Science-Magazine"
}
product-1, product-2... are keys from a Map.
The keys are stored in another collection Product Collection.
Can I create a nested query to cross-reference the product keys from another table?
I need the output in the below table format. Please suggest.
user123 product-1 email On
user123 product-2 email Off
user123 product-3 email On
I tried the below but can't get result. Please suggest.
var cursor = db.productSummary.find();
while(cursor.hasNext()){
var sku = cursor.next()._id;
var skuCol = "preferences."+sku+".details.email";
var skuVal = "preferences."+sku+".details.email.value";
db.marketingPreferences.find( {}, {_id:1, skuCol:1, skuVal:1});
}
> var myCursor = db.productSummary.find();
> while(myCursor.hasNext()){
var sku = myCursor.next()._id;
var skuCol = "preferences."+sku+".details.email";
var skuVal = "$preferences."+sku+".details.email.value";
var result = db.marketingPreferences.aggregate([{"$project":{"_id":1,value:skuVal,preferences:{$literal: sku}}}],{allowDiskUse: true});
while(result.hasNext()){
printjson(result.next());
}
}
Result
{ "_id" : "user123", "preferences" : "product-1", "value" : "On" }
{ "_id" : "user123", "preferences" : "product-2", "value" : "Off" }
{ "_id" : "user123", "preferences" : "product-3", "value" : "On" }
There's a difference between MongoDB and normal SQL DB. Firstly, when you query a MongoDB collection, it doesn't return a row as it will in a SQL db. What you get here is a document similar to JSON.
Also when you use preferences.product-1.details.email : 1 it wont return you the word 'email', rather it will return you the value ie. {"value" : "On" }.
Using this: db.preference.find({},{"_id":1,"preferences.product1.details.email.value":1})
you will be able to get two details which are user123 and On and you can get product-1 from your previous query. You can store these values in a variable and keep printing them to obtain the table necessary. Also you would need another cursor to store the result of the second second query that you would do.
Here's what your query will produce if it was single standalone query:
> db.preference.find({},{"_id":1,"preferences.product1.details.email.value":1})
.pretty()
{
"_id": "user123",
"preferences": {
"product-1": {
"details": {
"email": {
"value": "On"
}
}
}
}
}
public static void test(){
MongoCollection<Document> collection = getDatadase().getCollection("product");
MongoCollection<Document> pref = getDatadase().getCollection("pref");
List<Document> allDocList = collection.find().into(new ArrayList<Document>());
for(Document doc:allDocList){
System.out.println(doc.get("_id"));
String preferences = doc.get("_id")+"";
String sku = "$preferences."+preferences+".details.email.value";
Document aggregation = new Document().append("$project", new Document().append("_id", 1).append("value", sku));
List<Document> pipeline = new ArrayList<Document>();
pipeline.add(aggregation);
List<Document> aggList = pref.aggregate(pipeline).into(new ArrayList<Document>());
for(Document doc1:aggList){
System.out.println(doc1.append("preferences", preferences));
}
}
}
This Will return
product-1
Document{{_id=user123, value=On, preferences=product-1}}
product-2
Document{{_id=user123, value=Off, preferences=product-2}}
product-3
Document{{_id=user123, value=On, preferences=product-3}}