I am having a collection named "Books" and my sub documents like this
{
Country : 'XYZ'
Books : [
{"name" : "book1", "url" : "book1url", "auth_email" : "emailid1"},
{"name" : "book2", "url" : "book2url", "auth_email" : "emailid2"},
{"name" : "book3", "url" : "book3url", "auth_email" : "emailid3"},
{"name" : "book4", "url" : "book4url", "auth_email" : "emailid4"}
..........................................
]
}
I want to export it to a csv file with the following format
Country | name | url | auth_email| name | url | auth_email | ................
XYZ book1 book1url emailid1 book2 book2url emailid2 ...................
I did something like this
mongoexport -host localhost -db test -collection Books -csv > TopBooksOnline.csv -f Country,Books.name,Books.url,Books.auth_email
But I got empty content in the fields "Books.name", "Books.url" and " Books.auth_email"
If I do this
mongoexport -host localhost -db test -collection Books -csv > TopBooksOnline.csv -f Country,Books
Then it will club all my sub-docs into one.
How should I proceed ?
I don't think that the CSV format allows you to split up a single document as multiple documents, as this is something that you would need if you have nested documents. It would most definitely not create multiple fields for each sub-document as you are expecting. If you want to do that, then you can quite easily write your own CSV exporter which allows you a lot more flexibility anyway.
Related
I am having problems exporting subdocuments that are stored in MongoDB to a .CSV.
My data: a mongo collection that contains a unique user ID and scores from personality quizzes.
I would like a CSV that has three columns: user_id, name, raw_score. To add a further layer of complexity, within the 'scales' subdocument some users will have more than two entries (some quizzes produced more than 2 personality scores).
An example of my data minus documents that I am not interested in:
"assessment":{
"user_id" : "5839b1a654842f35617ad100",
"submissions" : {
"results" : {
"scales" : [
{
"scale" : {
"name" : "Security",
"code" : "SEC",
"multiplier" : 1
},
"raw_score" : 2
},
{
"scale" : {
"name" : "Power",
"code" : "POW",
"multiplier" : -1
},
"raw_score" : 3
}
],
}
}
}
}
I have tried using mongoexport but this produces a CSV that only has a user_id column.
rekuss$ mongoexport -d production_hoganx_app -c assessments --type=csv -o app_personality.csv -f user_id,results.scales.scale.name,results.scales.raw_score
Any ideas where I am going wrong?
Please let me know if you need anymore information.
Many thanks
You should try removing '=' sign from type. You could try --type csv
I am trying to export the MongoDB output to CSV format. But have trouble.
See the following document in my collection:
db.save.find().pretty();
{
"_id" : ObjectId("58884b11e1370511b89d8267"),
"domain" : "google.com",
"emails" : [
{
"email" : "f#google.com",
"first" : "James",
"Last" : "fer"
},
{
"email" : "d#gmail.com",
"first" : "dear",
"last" : "near"
}
]
}
Exporting the document to csv
C:\MongoDB\Server\bin>mongoexport.exe -d Trial -c save -o file.csv --type csv --fields domain,emails
2017-01-25T12:50:54.927+0530 connected to: localhost
2017-01-25T12:50:54.929+0530 exported 1 record
The output file is:
domain,emails
google.com,"[{""email"":""f#google.com"",""first"":""James"",""Last"":""fer""},{""email"":""d#gmail.com"",""first"":""dear"",""last"":""near""}]"
But if I import the same file, the output is different then it was in the actual collection. See the example:
> db.sir.find().pretty()
{
"_id" : ObjectId("5888529fa26b65ae310d026f"),
"domain" : "google.com",
"emails" : "[{\"email\":\"f#google.com\",\"first\":\"James\",\"Last\":\"fer\"},{\"email\":\"d#gmail.com\",\"first\":\"dear\",\"last\":\"near\"}]"
}
I do not want that extra \ in my import document. That's it. Please tell me if it is avoidable and if yes, then what should be the format of CSV to be given for import.
This is not expected format. So let me know how I can make the proper format. Kindly help me with this query.
I wanna connect mongodb and elasticsearch. I used mongo connector to connect them. I followed instruction from below link to setup==>
http://vi3k6i5.blogspot.in/2014/12/using-elastic-search-with-mongodb.html
I am able to connect mongodb and elasticsearch. But by default mongo connector created indices in elasticsearch for all databases of mongodb.
I want to create only one index for my one database and I want to insert only selected field of documents. for example: in mongo shell==>
use hotels
db.restaurants.insert(
{
"address" : {
"street" : "2 Avenue",
"zipcode" : "10075",
"building" : "1480",
"coord" : [ -73.9557413, 40.7720266 ],
},
"borough" : "Manhattan",
"cuisine" : "Italian",
"grades" : [
{
"date" : ISODate("2014-10-01T00:00:00Z"),
"grade" : "A",
"score" : 11
},
{
"date" : ISODate("2014-01-16T00:00:00Z"),
"grade" : "B",
"score" : 17
}
],
"name" : "Vella",
"restaurant_id" : "41704620"
}
)
This will create database hotels and collection restaurants. Now I want to create index and I want to put only address field in elasticsearch for that index.
Below are the steps what I tried but thats not working :
First I start mongo connector like below :
Imomadmins-MacBook-Pro:~ jayant$ mongo-connector -m localhost:27017 -t localhost:9200 -d elastic_doc_manager --oplog-ts oplogstatus.txt
Logging to mongo-connector.log.
Then from new shell tab, I made command like :
curl -XPUT 'http://localhost:9200/hotels.restaurants/'
curl -XPUT "http://localhost:9200/hotels.restaurants/string/_mapping" - d'{
"string": {
"properties" : {
"address" : {"type" : "string"}
}
}
}'
But only index is created in elasticsearch named as hotels.restaurants. I can't see any document for index hotels.restaurants.
Please suggest me how to add document for hotels.restaurants
Well I got an answer to my question, while starting mongo connector we can specify collection name and the list of fields we are interested in. Please check below command ==>
$ mongo-connector -m localhost:27017 -t localhost:9200 -d elastic_doc_manager --oplog-ts oplogstatus.txt --namespace-set hotels.restaurants --fields address,grades,name
Scope:
I have a collection on MongoDB that I want to export to a .csv file. I have done this already, with a similar database, using the MongoExport.exe, executing it right on the server (windows machine, hosting the MongoDB database).
Problem:
Once I run the following script
mongoexport.exe --fieldFile fields.txt --db AppleStore --collection AppleStoreApps --out applestore.csv --csv --port 21766
I start getting the following error messages
Invalid BSON object type for CSV output:10
It works for some cases, but seems like the majority of records gets this error.
More Information:
This is an example of JSON object on mongoDB, that should be exported:
{
"_id" : ObjectId("545c05ea74671a1d1c572da9"),
"url" : "https://itunes.apple.com/us/app/dc-eventos/id782560424?mt=8",
"name" : "DC Eventos",
"developerName" : "FERNANDO COSTA",
"developerUrl" : "https://itunes.apple.com/us/artist/fernando-costa/id729986271",
"price" : 0,
"isFree" : true,
"thumbnailUrl" : "http://a4.mzstatic.com/us/r30/Purple6/v4/ee/a2/5e/eea25e3f-8f12-9dce-c86f-37e5e3d9a8dc/icon350x350.jpeg",
"compatibility" : "Requires iOS 5.0 or later. Compatible with iPhone, iPad, and iPod touch. This app is optimized for iPhone 5.",
"category" : "Business",
"updateDate" : ISODate("2014-03-22T03:00:00.000Z"),
"version" : "1.82.82.542",
"size" : "16.3 MB",
"languages" : [
"English"
],
"minimumAge" : 4,
"ageRatingReasons" : [],
"rating" : {
"starsRatingCurrentVersion" : 0,
"starsVersionAllVersions" : 0,
"ratingsCurrentVersion" : 0,
"ratingsAllVersions" : 0
},
"topInAppPurchases" : null
}
mongoexport is likely choking on empty array -- "ageRatingReasons" : [] -- and null objects. examine the records one by one and check for a pattern.
csv cannot 'do' arrays and objects hence the need for json and xml. try exporting json and then convert with a variety of json to csv converters that will handle complex or custom flattening of objects such as [] to 0 or skipped commas val,,val whatever is needed. the jsontocsv convertor must also permit turning off validating, simply because ObjectId("545c05ea74671a1d1c572da9") is invalid json.
I have a MondoDB Collection 'quotes' with lineItems 'list'.. as shown below...
db.quotes.find();
> { "_id" : "51d31c4a0364a1b7f7cf45f7",
"accountName" : "NewAccountName",
"className" : "com.db.model.Quote",
"cost" : "0",
"lineItems" : [ { "lineNo" : 1, "product" : { "sku" : "MW216", "description" : "JBoss EAP", "cost" : "1043.5" }, "quotePrice" : "1230", "quantity" : 4 },
{ "lineNo" : 2, "product" : { "sku" : "MW217", "description" : "JBoss EDS, "cost" : "15178.18"}, "quotePrice" : "0", "quantity" : 3} ],
"quoteNumber" : "22005",
"shipping" : "GROUND"
}
I am using the MongoDB Query as shown..
{
'collectionName' : 'quotes',
'findQuery' : {
$where : "this.quoteNumber == $P!{QuoteNo}"
}}
I would like to Display each lineItem as one row.
lineNo | sku | Description
1 | MW216 | JBoss EAP
2 | MW217 | JBoss EDS
How to design this in the with JasperReports using iReport Designer?
Currently when using the 'lineItems' field from the Report Inspector and placing that in the Report shows every thing that is there in the List as one object. I am trying to read each field in the list and display it in the report grouping by lineItems as shown above.
Any help or clues will be appreciated and Thanks for your time in helping me out.
First you should absolutely not be using $where - instead of $where : "this.quoteNumber == $P!{QuoteNo}" you should simply use db.quotes.find({quoteNumber:YOURQUOTENO}) this will execute a normal MongoDB query on the server, rather than spawning a Javascript shell to execute $where statement.
If you want to see each line item separate you want to use aggregation framework $unwind like this:
db.quotes.aggregate({$unwind:"$lineItems"})
This will return one document per each lineItem so if you have five documents with each of them having three lineItems in the array you would get back 15 documents.