I am having problems exporting subdocuments that are stored in MongoDB to a .CSV.
My data: a mongo collection that contains a unique user ID and scores from personality quizzes.
I would like a CSV that has three columns: user_id, name, raw_score. To add a further layer of complexity, within the 'scales' subdocument some users will have more than two entries (some quizzes produced more than 2 personality scores).
An example of my data minus documents that I am not interested in:
"assessment":{
"user_id" : "5839b1a654842f35617ad100",
"submissions" : {
"results" : {
"scales" : [
{
"scale" : {
"name" : "Security",
"code" : "SEC",
"multiplier" : 1
},
"raw_score" : 2
},
{
"scale" : {
"name" : "Power",
"code" : "POW",
"multiplier" : -1
},
"raw_score" : 3
}
],
}
}
}
}
I have tried using mongoexport but this produces a CSV that only has a user_id column.
rekuss$ mongoexport -d production_hoganx_app -c assessments --type=csv -o app_personality.csv -f user_id,results.scales.scale.name,results.scales.raw_score
Any ideas where I am going wrong?
Please let me know if you need anymore information.
Many thanks
You should try removing '=' sign from type. You could try --type csv
Related
I'm used to working with firebase where I can access a document directly by fetching data from the db like so.
db.collection('collectionName/documentID').get();
I can't seem to find any documentation regarding doing something similar in mongodb. Do I have to use a find query to grab data from a mongodb or have I missed something? Thanks
I'm thinking
const collection = db.collection('collectionName');
collection.findOne({_id: ObjectId('documentID'); });
Since mongo consolse is an interactive javascript shell, One way would be to create a method similar to this:
function collectionNameGet(idToFind) {
return db.collection.find({_id: idToFind });
}
In the mongo shell you can directly get it as below:
db.st4.find({"_id" : "1234"})
Result set:
{ "_id" : "1234", "raw" : { "meas" : { "meas1" : { "data" : "blabla" }, "mesa2" : { "data" : "foo" } } } }
Or by default mongo id as:
db.st1.find({"_id" : ObjectId("5c578d57ce9ba4a066ca2fa4")})
{ "_id" : ObjectId("5c578d57ce9ba4a066ca2fa4"), "name" : "Just a name", "users" : [ "user1", "user2" ] }
For display the result in pretty format
db.st1.find({"_id" : ObjectId("5c578d57ce9ba4a066ca2fa4")}).pretty()
Result set:
{
"_id" : ObjectId("5c578d57ce9ba4a066ca2fa4"),
"name" : "Just a name",
"users" : [
"user1",
"user2"
]
}
Here st4 is my collection name in the database test, so once you are on mongo shell do the below steps before above query:
use test
db.st1.insert({"name" : "Just a name", "users" : [ "user1", "user2" ] })
and then you can query by default _id generated mongo, you can simply make a query to get the recently added documents in the collection st1 as below:
db.st1.find().sort({_id:-1}).limit(1)
Hope this will help you out to do some basic query on mongo shell
I am new to the mongodb. I have one mongo database which contains the around 150 collections. Currently I want to create the master document for each collection. Master document will contain all the fields in that collection. e.g. Lets say I have a users collection and it contains the two records as
{
"_id" : ObjectId("59e5e384cbead107344e6420"),
"user_name" : "test",
"pass" : "system",
"email": "em#gm.com"
}
{
"_id" : ObjectId("59e5e384cbead107344e6420"),
"user_name" : "test1",
"org":{
"name":"My Org"
},
"pass" : "test"
}
After merging the two documents I will get the master record as
{
"_id" : ObjectId("59e5e384cbead107344e6420"),
"user_name" : "test",
"email" : "em#gm.com",
"org":{
"name":"My Org"
},
"pass" : "test"
}
Here I am not worried about the data. I need the structure of the document and data types of the fields. I have thousands of the records which are not managed properly. I need a master template which will contain the all the possible fields in all the documents of one collection. I have to create such template for each collection.
Appreciating your help.
I am trying to export the MongoDB output to CSV format. But have trouble.
See the following document in my collection:
db.save.find().pretty();
{
"_id" : ObjectId("58884b11e1370511b89d8267"),
"domain" : "google.com",
"emails" : [
{
"email" : "f#google.com",
"first" : "James",
"Last" : "fer"
},
{
"email" : "d#gmail.com",
"first" : "dear",
"last" : "near"
}
]
}
Exporting the document to csv
C:\MongoDB\Server\bin>mongoexport.exe -d Trial -c save -o file.csv --type csv --fields domain,emails
2017-01-25T12:50:54.927+0530 connected to: localhost
2017-01-25T12:50:54.929+0530 exported 1 record
The output file is:
domain,emails
google.com,"[{""email"":""f#google.com"",""first"":""James"",""Last"":""fer""},{""email"":""d#gmail.com"",""first"":""dear"",""last"":""near""}]"
But if I import the same file, the output is different then it was in the actual collection. See the example:
> db.sir.find().pretty()
{
"_id" : ObjectId("5888529fa26b65ae310d026f"),
"domain" : "google.com",
"emails" : "[{\"email\":\"f#google.com\",\"first\":\"James\",\"Last\":\"fer\"},{\"email\":\"d#gmail.com\",\"first\":\"dear\",\"last\":\"near\"}]"
}
I do not want that extra \ in my import document. That's it. Please tell me if it is avoidable and if yes, then what should be the format of CSV to be given for import.
This is not expected format. So let me know how I can make the proper format. Kindly help me with this query.
I have two collections, user_logs and users, user_logs documents have user_id field so I need some data from user_logs but in the same query I want to check if some other field from user related to the current user_log is empty. How should I do this?
A query can only access one collection at a time. Mongodb doesn't support joins.
They that's why they recommend that you embed the referenced data inside the document.
If the logs documents for each user isn't too big, then you can change the embed that info inside the user collection.
Giving you something like this.
Embedded User Collection:
{
user_id : "uid1",
logs : [
{ message : "Error: System shutdown", date : "2014-11-11" },
{ message : "Error: System shutdown", date : "2014-11-13" }
]
}
However, if you want to keep your current structure then you're going to have to perform two queries to find related info between the users and user_logs collections.
Example
db.user_logs.insert([
{ _id : "ul1", log : "code 1", user_id : "u1" },
{ _id : "ul2", log : "code 2", user_id : "u1" }
]);
db.users.insert([
{ _id : "u1", name : "bob", user_logs_id : "ul1" },
{ _id : "u2", name : "smith", user_logs_id : "ul2" }
]);
var userId = db.user_logs.findOne({}).user_id;
db.users.findOne({ _id : userId })
//outputs
{ "_id" : "u1", "name" : "bob", "user_logs_id" : "ul1" }
The question is:
Consider the following location: [-72, 42] and the range (circle) of radius 2 around this point. Write a query to find all the states that intersect this range (circle). Then, you should return the total population and the number of cities for each of these states. Rank the states based on number of cities.
I have written this so far:
db.zips.find({loc: {$near: [-72, 42], $maxDistance: 2}})
and a sample output of that is:
{ "city" : "WOODSTOCK", "loc" : [ -72.004027, 41.960218 ], "pop" : 5698, "state" : "CT", "_id" : "06281" }
In SQL i would simply do a group by "state", how would i be able to do that here while also counting all the cities and total population?
assuming you follow the mongoimport routine for its zipcode data (i brought mine into a collection called zips7):
mongoimport --db mydb --collection zips7 --type json --file c:\users\drew\downloads\zips.json
or
mongoimport --db mydb --collection zips7 --type json --file /data/playdata/zips.json
(depending on your OS and paths)
then
db.zips7.ensureIndex({loc:"2d"})
db.zips7.find({loc: {$near: [-72, 42], $maxDistance: 2}}).forEach(function(doc){
db.zips8.insert(doc);
});
note that db.zips7.stats() shows like 30k rows and zips8 has 100 rows
db.zips8.aggregate( { $group :
{ _id : "$state",
totalPop : { $sum : "$pop" },
town_count:{$sum:1}
}}
)
{
"result" : [
{
"_id" : "RI",
"totalPop" : 39102,
"town_count" : 10
},
{
"_id" : "MA",
"totalPop" : 469583,
"town_count" : 56
},
{
"_id" : "CT",
"totalPop" : 182617,
"town_count" : 34
}
],
"ok" : 1
}
Syntax in mongoid
Zips.where(:loc => {"$within" => {"$centerSphere"=> [[lng.to_f,lat.to_f],miles.to_f/3959]}})
Example:
Zips.where(:loc => {"$within" => {"$centerSphere"=> [[-122.4198185,37.7750454],2.0/3959]}})