Data migration is not happening properly to CosmosDB using data Migration tool - mongodb

I am exporting document using "Sudio 3T" and it generates below JSON.
{ "_id" : { "$oid" : "5aa8bf0077bbfe296c1727de" }, "_t" : "DiscreteProperty", "Name" : "AHRI_bCertified"}
Next I using migration tool import JSON to Azure CosmosDB. Result is, it uploads but not viewing the document because it doesn't _id field.
Note: I am not providing anything in "Id Field" of target screen(CosmosDB).

I followed your steps but did't reproduce your issue.Please refer to my details.
sample .json file:
[
{
"_id" : { "$oid" : "5aa8bf0077bbfe296c1727de" },
"_t" : "DiscreteProperty",
"Name" : "AHRI_bCertified"
}
]
target info:
import result:

Related

Query MongoDB collection

I have a MongoDB collection like this:
{
"_id" : {
"processUuid" : "653d0937-2afc-4915-ad42-d2b69f344402",
"partnerId" : "p377"
},
"isComplete" : true,
"tasks" : {
"dbb361a7-4b73-4691-bde5-2b160346464f" : {
"sku" : "4060079000048",
"status" : "FAILED",
"errorList" : [....]
},
"790dbc6f-563d-4eb7-931c-3cc604563dc1" : {
"sku" : "4060079000130",
"status" : "SUCCESSFUL",
"errorList" : [....]
},
... more tasks
}
... more processes
}
I want to query a certain sku and couldnt find a way. I know i could write an aggregation pipeline to project the inner part of a task, but then i would lose the task identifier which i need in order to add some stuff to a task.
I know the document structure is weird, i would have done it different, but its given.
I tried what i found about querying nested documents and so on, but unfortunately didnt get it. Any hints are appreciated.

cosmosdb mongo api not working for some commands

I am using cosmosdb on azure
In that I am using mongodb api
I have a "request" collection inside that there is a "claims" array
If I use this command:
db.getCollection('requests').find({"claims.id": 1002})
It is not working in cosmosdb mongo api but working for local mongo service instance I have hosted.
my request object is as below
{
"_id" : NumberLong(1001),
"claims" : [ {
"type" : "broadband",
"id" : NumberLong(1002),
"createdOn" : NumberLong(1462799667905)
} ]
}
Not all of MongoDB's query syntax / capabilities are implemented. This appears to be such a case.
However, this slight workaround should work for you - I just tested it on my own CosmosDB (MongoDB API) collection:
db.getCollection('request').find({claims: { $elemMatch: { id:1002 }}}).pretty()
{
"_id" : 1001,
"claims" : [
{
"type" : "broadband",
"id" : 1002,
"createdOn" : NumberLong("1462799667905")
}
]
}
Note that you can also call db.request.find() without the need for a call to getCollection().

How to load the array of sub documents data from mongodb to hive

We are trying to use the mongodb data in hive, document has array of subdocuments.. How can I load the complex data into hive?
Here is the sample json:
{
"_id" : ObjectId("582c8cb9913e2f21e062aaa6"),
"acct" : NumberLong(12345),
"history" : [
{
"startDate" : ISODate("2016-09-01T16:00:00.000Z"),
"endDate" : ISODate("2016-09-30T16:00:00.000Z"),
"averageDailyBal" : "2653.85"
},
{
"startDate" : ISODate("2016-10-01T16:00:00.000Z"),
"endDate" : ISODate("2016-10-31T16:00:00.000Z"),
"averageDailyBal" : "1840.15"
},
{
"startDate" : ISODate("2016-11-01T16:00:00.000Z"),
"endDate" : ISODate("2016-11-30T17:00:00.000Z"),
"averageDailyBal" : "2796.14"
}
]
}
Thanks...
As you know The data in MongoDB is stored in JSON format, So you can use any Json serde to parse the data in it.
Refer this

MongoDB DBReference how to?

i'm learning MongoDB and i have the next questions.
There are my MongoDB documents
This is coordenada document
> db.coordenada.find().pretty()
{
"_id" : ObjectId("5579b81342a31549b67ad00c"),
"longitud" : "21.878382",
"latitud" : "-102.277364"
}
{
"_id" : ObjectId("5579b85542a31549b67ad00d"),
"longitud" : "21.878626",
"latitud" : "-102.280379"
}
{
"_id" : ObjectId("5579b89442a31549b67ad00e"),
"longitud" : "21.878845",
"latitud" : "-102.283512"
}
{
"_id" : ObjectId("5579b8bf42a31549b67ad00f"),
"longitud" : "21.879253",
"latitud" : "-102.286698"
}
{
"_id" : ObjectId("5579b8dd42a31549b67ad010"),
"longitud" : "21.879203",
"latitud" : "-102.291558"
}
{
"_id" : ObjectId("5579b8fd42a31549b67ad011"),
"longitud" : "21.878427",
"latitud" : "-102.296375"
}
{
"_id" : ObjectId("5579b91d42a31549b67ad012"),
"longitud" : "21.877571",
"latitud" : "-102.299659"
}
And this is rutas document
> db.rutas.find().pretty()
{
"_id" : "1",
"nombre" : "Ruta Penal",
"numero" : "20",
"coordenadas" : [
DBRef("coordenada", "5579b91d42a31549b67ad012")
]
}
{
"_id" : "2",
"nombre" : "Ruta Penal",
"numero" : "20",
"coordenadas" : [
DBRef("coordenada", "5579b91d42a31549b67ad012")
]
}
{
"_id" : "3",
"nombre" : "Ruta Penal",
"numero" : "20",
"coordenadas" : [
DBRef("coordenada", "5579b85542a31549b67ad00d")
]
}
{
"_id" : 6,
"nombre" : "Ruta Penal",
"numero" : "20",
"coordenadas" : [
DBRef("coordenada", "5579b85542a31549b67ad00d")
]
}
>
What i'm tryin to do, it's obtain the "longitud" and "latitud" from "coordenada" but only for the "numero" 20 of "rutas" document for instance
How can i do this?
PS sorry for the spanish terms.
According to the mongodb site for DBRef, you need to use drivers to unpack reference. I don't think mongo shell can unpack it for you.
http://docs.mongodb.org/manual/reference/database-references/
To resolve DBRefs, your application must perform additional queries to return the referenced documents. Many drivers have helper methods that form the query for the DBRef automatically. The drivers [1] do not automatically resolve DBRefs into documents.
DBRefs provide a common format and type to represent relationships among documents. The DBRef format also provides common semantics for representing links between documents if your database must interact with multiple frameworks and tools.
Unless you have a compelling reason to use DBRefs, use manual references instead.
Based on that, I would suggest to change it using manual reference (just the document id) instead.
To answer your question however, you can use any language drivers but below is an example in Python using pymongo:
from pymongo import MongoClient
from bson.objectid import ObjectId
from bson.dbref import DBRef
client = MongoClient()
db = client.testDB
rutas_20 = list(db.rutas.find({"numero": "20"}))
for ruta in rutas_20:
for coordenada in ruta.get('coordenada'):
coord_doc = db.coordenada.find_one({"_id": ObjectId(coordenada.id) })
print coord_doc.get('longitud'), coord_doc.get('latitud')
You can also use db.dereference(DBRef()) as well.
Hope it helps.
Cheers.
Yes, you can definitely obtain the latitude and longitude of the particular item by referencing the object id of the other class.
To use momgo dbRef you have to use the specific drivers depending on the particular language you are using. The driver documentation will tell you about the functions you can use.
I use PHP and hence refer to,
http://www.php.net/manual/en/class.mongodbref.php/

Retrieving only the relevant part of a stored document

I'm a newbie with MongoDB, and am trying to store user activity performed on a site. My data is currently structured as:
{ "_id" : ObjectId("4decfb0fc7c6ff7ff77d615e"),
"activity" : [
{
"action" : "added",
"item_name" : "iPhone",
"item_id" : 6140,
},
{
"action" : "added",
"item_name" : "iPad",
"item_id" : 7220,
}
],
"name" : "Smith,
"user_id" : 2
}
If I want to retrieve, for example, all the activity concerning item_id 7220, I would use a query like:
db.find( { "activity.item_id" : 7220 } );
However, this seems to return the entire document, including the record for item 6140.
Can anyone suggest how this might be done correctly? I'm not sure if it's a problem with my query, or with the structure of the data itself.
Many thanks.
You have to wait the following dev: https://jira.mongodb.org/browse/SERVER-828
You can use $slice only if you know insertion order and position of your element.
Standard queries on MongoDb always return all document.
(question also available here: MongoDB query to return only embedded document)