I am currently using MongoChef (4.3.0) to access an Azure Document DB (using DocumentDB protocol support for MongoDB).
The data in question is from an Application Insights continuous export has the following type of data within (there is other data there but this is the key part i am interested in...)
{
... other fields ...
"request" : [
{
"name" : "GET /images/loading_man.gif",
"count" : NumberInt(1),
"responseCode" : NumberInt(200),
"success" : NumberInt(1),
"url" : "http://<removed>.cloudapp.azure.com/<something>/images/loading_man.gif"
... other fields ...
}
]
... other fields ...
}
Using MongoChef I can perform some basic query like the following without issue;
{ "request": { $exists: true } }
but anything more complicated seems to return nothing or not run at all
{ "request.0.url": { $exists: true } }
{ "request.0.url": /.*man.*/i }
If I Export this data and Import it to my local MongoDb I am indeed able to perform such searches on the data in question without issue.
Any ideas how I could perform this type of search on the data in question without needing to export it?.
(this is a programming issue, because I want to do the above in a python program!)
Well it looks like all I needed to do was use $elemMatch!
{ "request": { $elemMatch: {url:/.*man.*/i } }}
this is as I understand the recommended way but also 'faster'?
Related
I'm using AnypointStudio to create an API and I'm trying to search for a document using the ObjectId on MongoDB but I can't make it work. I have tried searching with the use of other fields in the collection and it works just fine. Here is the value of the Find query I am currently using.
output application/json
---
{
"_id" : "5c088f4264c73358f4f7e3c0"
}
Here's an example result I have upon searching with the use of the uid field.
{
"_id": {
"$oid": "5c088f4264c73357f4f7e3c0"
},
"uid": "test",
"name": "test",
"validUser": true,
"oauth_client_id": "55628a5730ad44719e63b34c36604401",
"customer": "test customer"
}
Can anyone please help me to search document using the ObjectId? Thanks
I finally got the answer with some experimentations. I noticed the structure of the _id field and tried using this in the query "_id" : {"$oid" : "5c088f4264c73358f4f7e3c0"} but won't work since $ symbol is a reference symbol(?). So I tried using \ before the $ and it works.
output application/json
---
{
"_id" : {"\$oid" : "5c088f4264c73358f4f7e3c0"}
}
The example you show should work, you may need to wrap it like this
{
_id : ObjectId("5c088f4264c73358f4f7e3c0")
}
Hope this helps
I am using azure cosmosdb mongo api and unable to run bellow command on it. It runs fine when executed on locally hosted mongo service. Is there any way around for this?
db.getCollection('requests').update(
{claims: { $elemMatch: { id:1000 }}},
{$set:{"claims.$.billForMonth":"1"}}
)
My data is
{
"_id" : NumberLong(1000),
"location" : "pune",
"claims" : [
{
"id" : NumberLong(1000),
"type" : "broadband",
"billForMonth" : 4
}]
}
I could reproduce the issue on my side, when I do the following update operatation, it indicates I have updated the data successfully.
But if I query the data, I find billForMonth is not updated.
If I update a non-array field (such as location field), it works fine.
db.testc1.update(
{claims: { $elemMatch: { id:1000 }}},
{$set:{"location":"pune1"}}
)
It seems that cosmos db:mongo api does not support Array Update now, this feature will be supported in future. You will find the following information in this article.
I'm quite new to MongoDb and I'm trying to get my head around the grouping/counting functions. I want to retrieve a list of votes per track ID, ordered by their votes. Can you help me make sense of this?
List schema:
{
"title" : "Bit of everything",
"tracks" : [
ObjectId("5310af6518668c271d52aa8d"),
ObjectId("53122ffdc974dd3c48c4b74e"),
ObjectId("53123045c974dd3c48c4b74f"),
ObjectId("5312309cc974dd3c48c4b750")
],
"votes" : [
{
"track_id" : "5310af6518668c271d52aa8d",
"user_id" : "5312551c92d49d6119481c88"
},
{
"track_id" : "53122ffdc974dd3c48c4b74e",
"user_id" : "5310f488000e4aa02abcec8e"
},
{
"track_id" : "53123045c974dd3c48c4b74f",
"user_id" : "5312551c92d49d6119481c88"
}
]
}
I'm looking to generate the following result (ideally ordered by the number of votes, also ideally including those with no entries in the votes array, using tracks as a reference.):
[
{
track_id: 5310af6518668c271d52aa8d,
track_votes: 1
},
{
track_id: 5312309cc974dd3c48c4b750,
track_votes: 0
}
...
]
In MySQL, I would execute the following
SELECT COUNT(*) AS track_votes, track_id FROM list_votes GROUP BY track_id ORDER BY track_votes DESC
I've been looking into the documentation for the various aggregation/reduce functions, but I just can't seem to get anything working for my particular case.
Can anyone help me get started with this query? Once I know how these are structured I'll be able to apply that knowledge elsewhere!
I'm using mongodb version 2.0.4 on Ubuntu 12, so I don't think I have access to the aggregation functions provided in later releases. Would be willing to upgrade, if that's the general consensus.
Many thanks in advance!
I recommend you to upgrade your MongoDB version to 2.2 and use the Aggregation Framework as follows:
db.collection.aggregate(
{ $unwind:"$votes"},
{ $group : { _id : "$votes.track_id", number : { $sum : 1 } } }
)
MongoDB 2.2 introduced a new aggregation framework, modeled on the concept of data processing pipelines. Documents enter a multi-stage pipeline that transforms the documents into an aggregated result.
The output will look like this:
{
"result":[
{
"_id" : "5310af6518668c271d52aa8d", <- ObjectId
"number" : 2
},
...
],
"ok" : 1
}
If this is not possible to upgrade, I recommend doing it in a programmatically way.
I persist my Python classes to JSON and write them to MongoDB. I'm using a JSON Encoder which has been presented here. This results in a slight complex JSON structure which does not follow the simple "string" key/values in all the MongoDB tutorials. Here is an excerpt of a class written to JSON:
{
"_id" : ObjectId("51f2397c86a49a5e6dec4633"),
"__module__" : "experiment.experimentmanager",
"experiments" : {
"FOO" : [
{
"__module__" : "experiment.experiment",
"enable_routing_table_trace" : false,
"__class__" : "Experiment",
"scenario_name" : "foobar",
"enable_network_visualize" : false
},
}
I'm wondering how I could check for example if there is a key "FOO" in experiments using MongoDBs find? Thanks in advance!
To check if field exists use$exists operator.
To access complex structures use dot notation. For example:
db.inventory.find( { "experiments.FOO": { $exists: true} } )
This query should work:
db.collection.find({
"experiments.FOO": {
$exists:true
}
});
If we have
Blog{
Name 'Blog1'
Tags ['testing','visual-studio','2010','c#']
}
Blog{
Name 'Blog2'
Tags ['parallel','microsoft','c#']
}
Via the console we can execute and find all blog posts that contains some of the provided tags:
db.BlogPost.find({ 'Tags' : { '$regex' : ['/^Test/', '/^microsoft/', '/^visual/', '/^studio/', '/^c#/'] } });
How can we write the same query in c# 10gens driver ?
Is there any alternative if it can not be written via the 10gens c# driver ?
Query.Match only support one regex. Can we provide him multiple regexes, or we should combine
Query.Or(Query.Match("Test"), Query.Match("Micro"), Query.Match("Visual"))
I've managed to solve it with
I've managed to do it with
{ "$or" : [{ "Tags" : /^programm/i }, { "Tags" : /^microsoft/i }, { "Tags" : /^visual/i }, { "Tags" : /^studio/i }, { "Tags" : /^assert/i }, { "Tags" : /^2010/i }, { "Tags" : /^c#/i }] }
But something tells me that this is an ugly hack that may result in performance issues. What do you think guys ?
The final answer to the problem can be found on:
Official mongodb forum
Yes, the MongoDB c# driver should automatically do the right thing with instances of System.Text.RegularExpressions.Regex.
So you should be able to build the exact same query, except that you would use instances of Regex for ^Test, ^microsoft, ^visual, etc instead of strings.