I have Cosmos db as our database and my java application runs on Spring Boot. I'm using Spring Data's MongoRepository to query the database.
I have a requirement where I need to find results based on a field and it needs to be case insensitive.
What I tried ...
appending mongorepository method with "IgnoreCase"
adding regex in query ... #Query("{fieldName: {$regex: '^?0$', $options: 'i'}}")
The above solutions work with my mongodb(on my local) but doesn't work on cosmosdb .
I thought that the issue might be because of version of the software I'm using or some driver I'm using so I ran the same query on mongocompass (connecting to mongodb) and robo3t (connecting to cosmosdb), but the query doesn't with cosmos.
So my assumption is cosmos doesn't support the above queries
Now i'm not going to switch to mongodb since i'm not the one making the decision, i'm stuck with cosmos and I have to provide solution to the latter.
I've searched about collation, but haven't found any results or documentation on how to do it with cosmosdb and most likely this is not an option for me.
So is there any other way to query the database with no case sensitivity or is there anything wrong which I'm doing here?
This code is working fine on my local machine.
Bulk.find({"xyz":23}).upsert().update({$set : 5465});
Bulk.execute(function (err, data) {});
When I have moved this code to Azure, it wasnt working. I recognize that cosmosDB doesn't support upsert. Is that ryt ?
Reference :
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-feature-support#database-commands
Should i replace with find and insert or update as normal ? or is there any other solution available ? Please help.
Yes, based on the doc MongoDB query language support , upsert() command is not supported by cosmos db mongo api. As I known, no shortcuts here so far. You need to encapsulate methods to determine whether a document exists, and then decide to insert or update.
Or, just to declare that the Cosmos Document DB SDK supports Upsert method. Please refer to the case: How can I perform an UPSERT using Azure DocumentDB?.
Cosmos Document DB is a good choice if you could do data migrations.
Hope it helps you.
I have two mongo collections. One we can call a template and second is instance. Every time new instance is created, rather large data field is copied from template to instance. Currently the field is retrieved from mongo db template collection in application and then sent back to db as a part of instance collection insert.
Would it be possible to somehow perform this copy on insert directly in mongo db, to avoid sending several megabytes over the network back and forth?
Kadira is reporting 3 seconds lag due to this. And documents are only going to get bigger.
I am using Meteor, but I gather that that should not influence the answer much.
I have done some searching and I can't really find an elegant solution for you. The two ways I can think of doing it are:
1.) Fork a process to run a mongo command to copy your template as your new instance via db.collection.copyTo().
http://eureka.ykyuen.info/2015/02/26/meteor-run-shell-command-at-server-side/
https://docs.mongodb.org/manual/reference/method/db.collection.copyTo/
Or
2.) Attempt to access the raw mongo collection rather than the minimongo collection meteor provides you with so you can use the db.collection.copyTo() functionality supplied by Mongo.
var rawCollection = Collection.rawCollection();
rawCollection.copyTo(newCollection);
Can meteor mongo driver handle $each and $position operators?
I haven't tried accessing the rawCollection to see if copyTo is available, and I also don't know if it will bring it into meteor before writing out the new collection. I'm just throwing this out here as an idea for you; hopefully someone else has a better one.
I've been test driving Alteryx for the last week or so and was wondering if anyone has successfully connected to a Mongo Labs data base using the Alteryx Mongo Input and Output tools. I've tried numerous times and can's seem to get it to work.
Yes, I am using that MongoDB extractor on a daily basis to generate TDE files to feed Tableau, since Tableau does not offer dedicated extractors, but I admit it was difficult to get started.
Extraction is now swift from a MongoDB running on AWS, make sure you indicate the proper collection, and be aware of one flaw with the current version of extractor, as of Alteryx 9.5: it is missing the flag to send to the DB that would let you read from a Read Only configured MongoDB. It is on the priority list for V10. In the meantime, you should connect to the DB that can be written to, and to ease your DBA, show that Alteryx workflows can't write to it unless you drag into the workflow the MongoDB Output Tool.
Also you can use the Properties / Criteria window to set filters on the indexed field of your MongoDB. After much research, I found out the precise syntax for filtering the extraction by date:
{_id: {$gt: ObjectId("54a4fe800000000000000000")}}
Since each MongoDB ObjectId contains an embedded timestamp of its creation time.
To get the proper time, you can use this excellent website:
http://steveridout.github.io/mongo-object-time/
And if you need fancier filters, here is some help with the syntax:
http://www.querymongo.com/
I hope that helps...
I have a Mongo database that I did not create or architect, is there a good way to introspect the db or print out what the structure is to start to get a handle on what types of data are being stored, how the data types are nested, etc?
Just query the database by running the following commands in the mongo shell:
use mydb //this switches to the database you want to query
show collections //this command will list all collections in the database
db.collectionName.find().pretty() //this will show all documents in the database in a readable format; do the same for each collection in the database
You should then be able to examine the document structure.
There is actually a tool to help you out here called Variety:
http://blog.mongodb.org/post/21923016898/meet-variety-a-schema-analyzer-for-mongodb
You can view the Github repo for it here: https://github.com/variety/variety
I should probably warn you that:
It uses MR to accomplish its tasks
It uses certain other queries that could bring a production set-up to a near halt in terms of performance.
As such I recommend you run this on a development server or a hidden node of a replica or something.
Depending on the size and depth of your documents it may take a very long time to understand the rough structure of your database through this but it will eventually give one.
This will print name and its type
var schematodo = db.collection_name.findOne()
for (var key in schematodo) { print (key, typeof key) ; }
I would recommend limiting the result set rather than issuing an unrestricted find command.
use mydb
db.collectionName.find().limit(10)
var z = db.collectionName.find().limit(10)
Object.keys(z[0])
Object.keys(z[1])
This will help you being to understand your database structure or lack thereof.
This is an open-source tool that I, along with my friend, have created - https://pypi.python.org/pypi/mongoschema/
It is a Python library with a pretty simple usage. You can try it out (even contribute).
One option is to use the Mongoeye. It is open-source tool similar to the Variety.
The difference is that Mongoeye is a stand-alone program (Mongo Shell is not required) and has more features (histograms, most frequent values, etc.).
https://github.com/mongoeye/mongoeye
Few days ago I found GUI client MongoDB Compass with some nice visualizations. See the product overview. It comes directly from the mongodb people and according to their doc:
MongoDB Compass is designed to allow users to easily analyze and understand the contents of their data collections within MongoDB...
You may've asked about validation schema. Here's the answer how to get it:
How to retrieve MongoDb collection validator rules?
Use Mongo Compass
which does a sample as explained here
Which does a random sample of 1000 documents to get you the schema - it could miss something but it's the only rational option if you database is several GBs.
Visualisation
The schema then can be exported as JSON
Documentation
You can use MongoDB's tool mongodump. On running it, a dump folder is created in the directory from which you executed mongodump. In that folder, there are multiple folders that correspond to the databases in MongDB, and there are subfolders that correspond to the collections, and files that correspond to the documents.
This method is the best I know of, as you can also make out the schema of empty collections.