Reading up on MongoDB performance troubleshooting I see that MongoDB supports appending comments on queries. We thought we might add our correlationid to queries so we can correlate which features result in slow queries and/or check whether when we discover slowness in a feature if we can spot some slowness in mongodb.
How can we add comments to queries and/or commands through the .net driver?
How can we add comments to queries and/or commands through the .net driver?
You can utilise FindOptions.Comment to set a comment on a query or a command. For example:
var collection = database.GetCollection<MyObject>("collectionName");
var filter = Builders<MyObject>.Filter.Eq("Name", "Foo");
FindOptions myFindOptions = new FindOptions();
myFindOptions.Comment = "THIS IS FEATURE XYZ";
var cursor = collection.Find<MyObject>(filter, myFindOptions).ToList();
Once you set the database profiler i.e. via db.setProfilingLevel(), you can then query the (database).system.profile collection to find it. For example:
db.system.profile.find({
ns:"dbName.collectionName",
"command.comment":"THIS IS FEATURE XYZ"
});
// Example result:
{
"op": "query",
"ns": "dbName.collectionName",
"command": {
"find": "collectionName",
"filter": {
"Name": "Foo"
},
"comment": "THIS IS FROM FEATURE XYZ",
"$db": "dbName",
"lsid": {
"id": UUID("6b722166-f50b-409c-85f0-2711633baff2"))
}
},
....
}
The above .NET/C# code snippet is written using MongoDB .NET/C# driver v2.9.3.
Related
I have exported one of my collections from MongoDB and it looks like the following:
[
{
"_class": "test.Tag",
"title": "My Title",
"_id": {
"$oid": "637a2bc4a9c297b306dd0be9"
}
}
]
Now, I want to add another item to this collection.
I have created a json file exactly in the same format.
However, when I use the following command:
db.tag.insertMany([{"_class": "test.Tag", "title": "New Title","_id": {"$oid": "637a4171833047f9e046f3c9"}}])
I get the error:
MongoBulkWriteError: _id fields may not contain '$'-prefixed fields: $oid is not valid for storage.
I found a couple of similar questions on stackoverflow, e.g. this one, but they didn't help me.
How should I insert something in my database?
When I use MongoDB Compass, there is no problem. It works. But when I try it on my real database on the server, it doesn't work.
If it helps, I am using Mongo version 6.0.1.
Your help would be appreciated.
In the legacy mongo shell there is no helper for extended json format , but in mongosh you can use EJSON to deserialize the extended json and insert it as follow:
> db.test4.insertMany([EJSON.deserialize( {"_id": { "$oid": "637aa8243d58f36141e59c8d"} } )])
{
acknowledged: true,
insertedIds: { '0': new ObjectId("637aa8243d58f36141e59c8d") }
}
>
( The legacy mongo shell was deprecated in MongoDB 5.0 and removed in MongoDB 6.0 )
Can anyone tell me why when I use DataExplorer for CosmoDb DB I get the following:
{
"id": "d502b51a-e70a-40f1-9285-3861880b8d90",
"Version": 1,
...
}
But when I use Robomongo I get:
{
"Version" : 1,
...
}
minus the id?
Thanks
I tried to repro your scenario but it all worked correctly.
The Mongo document in Portal Data Explorer:
The Mongo document in Robo 3T:
They both have the id property.
Are you applying Projections on Robomongo / Robo 3T?
At this moment cosmodb works separately SQL API and Mongo API, each one has different implementation, SQL API use JSON and Mongo use BSON, you need to be clear this while you are creating the document.
If you create the document with a BSON-based tool like Robo3t for example, you are going to get something like this:
{
"_id": {
"$oid": "5be0d98b9cdcce3c6ce0f6b8"
},
"name": "Name",
"id": "5be0d98b9cdcce3c6ce0f6b8",
...
}
Instead, if you create your document with JSON-based like Data Explorer, you are going to get this:
{
"name": "Name",
"id": "6c5c05b4-dfce-32a5-0779-e30821e6c510",
...
}
As you can see, BSON-based needs that _id and inside $oid be implemented to works right, while JSON-based only id is required. So, you need to add the properties while you save the document (see below) or open it with the right tool, as Matias Quaranta recommend, use Azure Storage Explorer or even Data Explorer to get both protocols properly.
Also, if you use a system to create the document and you want to use BSON format, You need to add the $oid, for example in core net is something like this:
public bool TryGetMemberSerializationInfo(string memberName, out BsonSerializationInfo serializationInfo)
{
switch (memberName)
{
case "Id":
serializationInfo = new BsonSerializationInfo("_id", new ObjectIdSerializer(), typeof(ObjectId));
return true;
case "Name":
serializationInfo = new BsonSerializationInfo("name", new StringSerializer(), typeof(string));
return true;
default:
serializationInfo = null;
return false;
}
}
I am new to Cloudant and I need to do a simple query on a specific document field. My documents have the following structure and I need to get only the documents with status=SIGNED
{
"_id": "3ddb4058f3b24a7a9c585f997e30ff78",
"_rev": "3-757c82c48f4e7c333911be6859aff74e",
"fileName": "Generali Architects",
"status": "SIGNED",
"user": "italy",
"_attachments": {
"Generali Architects": {
"content_type": "application/pdf",
"revpos": 3,
"digest": "md5-9hqSif7CzQ2yvKxSSbj+dw==",
"length": 323653,
"stub": true
}
}
}
Reading Cloudant documentation I created the following Design Document with a related view which returns exactly what I expected
Then from my Java application I use the following code
String cloudantView = "_design/signedDocs/status-signed-iew";
List<SignDocDocument> docs =
db.view(cloudantView).includeDocs(true).query(SignDocDocument.class);
which always returns me "org.lightcouch.NoDocumentException: Object Not Found"
Any idea which kind of mistake I am making here?
Thank you very much
Is it the typo in "_design/signedDocs/status-signed-iew"; e.g. that should be "_design/signedDocs/status-signed-view"; (depending on how your java library works...).
Always worth checking the view by direct access in your browser, too, just to make sure it's returning the expected data.
I understand Manual referencing being, one document containing the reference to another.
As in the MongoDB's documentation:
original_id = ObjectId()
db.places.insert({
"_id": original_id,
"name": "Broadway Center",
"url": "bc.example.net"
})
db.people.insert({
"name": "Erin",
"places_id": original_id,
"url": "bc.example.net/Erin"
})
I am able to use a find on the places to get a certain name.
db.places.find({name : "Broadway Center"});
This will give the id. And then my application can use this ID to query the people to see who live here?
But, If i dont have a application. How would I go about doing this solely in a mongo shell? I was thinking about using find, and then iterating over the cursor using a forEach? But this seems a bit hacky.
Any suggestions?
You don't need to loop through using the cursor from find(), just use findOne() because a single document is returned with this method and you have access to the document directly (no need to apply cursor methods to the result). In mongo shell you can do this:
var result = db.places.findOne({name : "Broadway Center"});
if (result) {
var place_id = result._id;
var peopleArray = db.people.find({ "places_id": place_id }).toArray();
if (peopleArray.length > 0) { printjson (peopleArray[0]); }
}
I have different types of data that would be difficult to model and scale with a relational database (e.g., a product type)
I'm interested in using Mongodb to solve this problem.
I am referencing the documentation at mongodb's website:
http://docs.mongodb.org/manual/tutorial/model-referenced-one-to-many-relationships-between-documents/
For the data type that I am storing, I need to also maintain a relational list of id's where this particular product is available (e.g., store location id's).
In their example regarding "one-to-many relationships with embedded documents", they have the following:
{
name: "O'Reilly Media",
founded: 1980,
location: "CA",
books: [12346789, 234567890, ...]
}
I am currently importing the data with a spreadsheet, and want to use a batchInsert.
To avoid duplicates, I assume that:
1) I need to do an ensure index on the ID, and ignore errors on the insert?
2) Do I then need to loop through all the ID's to insert a new related ID to the books?
Your question could possibly be defined a little better, but let's consider the case that you have rows in a spreadsheet or other source that are all de-normalized in some way. So in a JSON representation the rows would be something like this:
{
"publisher": "O'Reilly Media",
"founded": 1980,
"location": "CA",
"book": 12346789
},
{
"publisher": "O'Reilly Media",
"founded": 1980,
"location": "CA",
"book": 234567890
}
So in order to get those sort of row results into the structure you wanted, one way to do this would be using the "upsert" functionality of the .update() method:
So assuming you have some way of looping the input values and they are identified with some structure then an analog to this would be something like:
books.forEach(function(book) {
db.publishers.update(
{
"name": book.publisher
},
{
"$setOnInsert": {
"founded": book.founded,
"location": book.location,
},
"$addToSet": { "books": book.book }
},
{ "upsert": true }
);
})
This essentially simplified the code so that MongoDB is doing all of the data collection work for you. So where the "name" of the publisher is considered to be unique, what the statement does is first search for a document in the collection that matches the query condition given, as the "name".
In the case where that document is not found, then a new document is inserted. So either the database or driver will take care of creating the new _id value for this document and your "condition" is also automatically inserted to the new document since it was an implied value that should exist.
The usage of the $setOnInsert operator is to say that those fields will only be set when a new document is created. The final part uses $addToSet in order to "push" the book values that have not already been found into the "books" array (or set).
The reason for the separation is for when a document is actually found to exist with the specified "publisher" name. In this case, all of the fields under the $setOnInsert will be ignored as they should already be in the document. So only the $addToSet operation is processed and sent to the server in order to add the new entry to the "books" array (set) and where it does not already exist.
So that would be simplified logic compared to aggregating the new records in code before sending a new insert operation. However it is not very "batch" like as you are still performing some operation to the server for each row.
This is fixed in MongoDB version 2.6 and above as there is now the ability to do "batch" updates. So with a similar analog:
var batch = [];
books.forEach(function(book) {
batch.push({
"q": { "name": book.publisher },
"u": {
"$setOnInsert": {
"founded": book.founded,
"location": book.location,
},
"$addToSet": { "books": book.book }
},
"upsert": true
});
if ( ( batch.length % 500 ) == 0 ) {
db.runCommand( "update", "updates": batch );
batch = [];
}
});
db.runCommand( "update", "updates": batch );
So what is doing in setting up all of the constructed update statements into a single call to the server with a sensible size of operations sent in the batch, in this case once every 500 items processed. The actual limit is the BSON document maximum of 16MB so this can be altered appropriate to your data.
If your MongoDB version is lower than 2.6 then you either use the first form or do something similar to the second form using the existing batch insert functionality. But if you choose to insert then you need to do all the pre-aggregation work within your code.
All of the methods are of course supported with the PHP driver, so it is just a matter of adapting this to your actual code and which course you want to take.