I'm having trouble figuring out how to upload csv data to my MongoLab database. From my terminal I have used
sudo mongoimport --db heroku_hkr86p3z -u <dbusername> -p <dbpassword> --collection contributors --type csv --headerline --file /Users/tonywinglau/Desktop/independent-expenditure.csv
and
sudo mongoimport --host mongodb://<username>:<password>#ds035310.mlab.com:35310/heroku_hkr86p3z --db heroku_hkr86p3z -u <username> -p <password> --collection contributors --type csv --headerline --file /Users/tonywinglau/Desktop/independent-expenditure.csv
both of which respond with
Failed: error connecting to db server: no reachable servers
imported 0 documents
From what I have read it might be something to do with my 'mongo config' file (I can't find it if it does exist) being set to only connect with localhost? How do I import data directly into my mongolab hosted database?
Your command line should look like this:
mongoimport -d <databasename> -c <collectionname> --type csv --file <filelocation/file.csv> --host <hostdir example:ds011291.mlab.com> --port <portnumber example:11111> -u <username> -p <password> --headerline
The host direction and the port number it gived by mlab when you create the database.
Example:
ds000000.mlab.com:000000/databaseName
Related
I am using a dockerfile and bash script
It should be noted that the mongo db is already running and I can interactively connect to it so this is not an issue.
Dockerfile
FROM mongo
WORKDIR /root
ENV PATH /root/bin:$PATH
WORKDIR /root/bin
COPY import_files/latest.csv .
COPY import_files/field_file_types.txt .
COPY scripts/import.sh .
RUN chmod +x import.sh
CMD sh /root/bin/import.sh
import.sh
#!/bin/bash
ENV_HOST="mongo1"
ENV_USER="user"
ENV_PASS="pass"
ENV_DB="mydb"
ENV_TABLE="mycollection"
# mongoimport -h "${ENV_HOST}:27017" -u "${ENV_USER}" -p "${ENV_PASS}" -c $ENV_TABLE --authenticationDatabase ${ENV_DB} --type csv --file ./latest.csv --columnsHaveTypes --fieldFile=./field_file_types.txt
mongoimport --uri "mongodb://${ENV_USER}:${ENV_PASS}#${ENV_HOST}/${ENV_DB}" -c $ENV_TABLE --type csv --file ./latest.csv --columnsHaveTypes --fieldFile=./field_file_types.txt
exit 0
This is my mongo import statement
mongoimport
-h "mongo1:27017"
-u "user"
-p "pass"
-c "mycollection"
--authenticationDatabase "mydb"
--type csv
--file ./latest.csv
--columnsHaveTypes
--fieldFile=./field_file_types.txt
Which in my console outputs like this:
mongodb://user:pass#mongo1:27017/admin mydb mycollection
And I get error:
error parsing command line options: Invalid Options: Cannot specify different username in connection URI and command-line option ("user" was specified in the URI and "mycollection" was specified in the --username option)
I have also tried it as a uri:
mongoimport --uri "mongodb://user:pass#mongo1/mydb" -c mycollection --type csv --file ./latest.csv --columnsHaveTypes --fieldFile=./field_file_types.txt
Which in my console looks like this:
Mongo import csv = mongodb://user:pass#mongo1:27017/admin mydb mycollection
Enter password:
Why is it asking for password? I provided it as above? Also its exactly the same output as the previous example.
It turns out when I was running docker-compose, I had a script which granted me permission to all databases: (but only if I login as an admin)
mongo <<EOF
use admin
db.createUser(
{
user: "${MONGO_USER}",
pwd: "${MONGO_PASS}",
roles: [ { role: "userAdminAnyDatabase", db: "admin" }, "readWriteAnyDatabase" ]
}
)
use ${MONGO_DB}
db.createCollection("${MONGO_COLLECTION}")
EOF
This means when I connect to mongo I must connect via 'admin' in order to access the other databases. To access the database in question I can include it in the uri string but I must also add -
--authenticationDatabase admin
import script example:
mongoimport --uri "mongodb://user:passt#mongo1/db" -c todos --type csv --file /data/db/latest.csv --columnsHaveTypes --fields "id.auto(),todoText.string(),created_timestamp.date(2006-01-02 15:04:05),completed.date(2006-01-02 15:04:05)" --authenticationDatabase admin
I want to build a Mongo container, and insert some data stored in the folder /mongo_mock_data in this container. Here are my lines of code:
MONGO_ID=$(docker run --publish-all -d --mount type=bind,source=${PWD}/mongo_mock_data,target=/tmp/mongo_mock_data mongo)
MONGO_PORT=$( docker inspect $MONGO_ID | jq -r '.[0].NetworkSettings.Ports."27017/tcp"[0].HostPort' )
MONGODB_URL=mongodb://localhost:${MONGO_PORT}
docker exec $MONGO_ID mongoimport --host $MONGODB_URL --db algo --collection train --type json --file /tmp/mongo_mock_data/train_data.json --jsonArray
docker exec $MONGO_ID mongoimport --host $MONGODB_URL --db algo --collection calculation --type json --file /tmp/mongo_mock_data/predict_data.json --jsonArray
docker exec $MONGO_ID mongoimport --host $MONGODB_URL --db algo --collection train --type json --file /tmp/mongo_mock_data/cluster_data.json --jsonArray
I have the following error:
error parsing command line options: error parsing URI from mongodb:///localhost:32773/?replicaSet=mongodb:: error parsing uri: must have at least 1 host
any ideas what Im doing wrong?
EDIT:
If I try:
MONGODB_URL=localhost:${MONGO_PORT}
docker exec $MONGO_ID mongoimport --host $MONGODB_URL --db pulse_algo --collection train --type json --file /tmp/mongo_mock_data/train_data.json --jsonArray
instead of
MONGODB_URL=mongodb://localhost:${MONGO_PORT}
docker exec $MONGO_ID mongoimport --host $MONGODB_URL --db algo --collection train --type json --file /tmp/mongo_mock_data/train_data.json --jsonArray
I have the following error:
error connecting to host: could not connect to server: server selection error: server selection timeout
Is the third slash in mongodb:/// a transcription error? If not, then what is happening is that MongoDB is trying to connect to a server at '' (ie, nothing) to a database named localhost:32773, rather than a server at localhost:32773 & the default database.
So, finally, I solved my issue by dropping the argument --host:
MONGO_ID=$(docker run --publish-all -d --mount type=bind,source=${PWD}/mongo_mock_data,target=/tmp/mongo_mock_data mongo)
MONGO_PORT=$( docker inspect $MONGO_ID | jq -r '.[0].NetworkSettings.Ports."27017/tcp"[0].HostPort' )
docker exec $MONGO_ID mongoimport --db algo --collection train --type json --file /tmp/mongo_mock_data/train_data.json --jsonArray
docker exec $MONGO_ID mongoimport --db algo --collection calculation --type json --file /tmp/mongo_mock_data/predict_data.json --jsonArray
docker exec $MONGO_ID mongoimport --db algo --collection train --type json --file /tmp/mongo_mock_data/cluster_data.json --jsonArray
In my case, I was getting this error because of passing option --csv instead of --type=csv to mongoexport. Correction of that fixed this issue.
mongoexport type options
Error message could've been better.
I'm facing a problem in importing a CSV file to my db. I tried this command:
mongoimport --db meteor -h localhost:3001 -c TollPlaza -d meteor --headerline --type csv --file TollPlaza.csv
I referred to this question but am still having a problem.
Try this query :
mongoimport --db meteor --host localhost --port 3001 -c TollPlaza -d
meteor --headerline --type csv --file TollPlaza.csv
Maybe seperate host and port
The query is perfectly fine.
Follow the instructions
run the meteor project
Open a one more command prompt in same location
make sure that csv file present in the same directory.
run the query.
check the DB (show collections).
I have a json file and am trying to insert in bulk to mongodb
I do
mongoimport --host localhost --db testdb --collection testdbjson --username user -- password pass --type json --file /home/pet/mng/json_test.json
It gives the following error
SyntaxError: Unexpected identifier
Please help!
the document mongoimport the correct syntax
mongoimport -host 127.0.0.1 -port 27017 -u user -p pass -collection testdbjson --db testdb --file /home/pet/mng/json_test.json
or
mongoimport -host 127.0.0.1:27017 -u user -p pass -collection testdbjson --db testdb --file /home/pet/mng/json_test.json
I had the exact same error now - I figured it out. I think you have connected to your mongo instance on mongolab first before you wanted to import.
You must not connect to mongolab first - disconnect by typing in "exit" in the shell to get back to command prompt. Then issue the mongoimport command again, the import should work this time.
UPDATE: this post applied to meteor.com free hosting, which has been shutdown and replaced with Galaxy, a paid Meteor hosting service
I'm using this command
C:\kanjifinder>meteor mongo --url kanjifinder.meteor.com
to get access credentials to my deployed mongo app, but I can't get mongoimport to work with the credentials. I think I just don't exactly understand which part is the username, password and client. Could you break it down for me?
result from server (I modified it to obfuscate the real values):
mongodb://client:e63aaade-xxxx-yyyy-93e4-de0c1b80416f#meteor.m0.mongolayer.com:27017/kanjifinder_meteor_com
my mongoimport attempt (fails authentication):
C:\mongodb\bin>mongoimport -h meteor.m0.mongolayer.com:27017 -u client -p e63aaade-xxxx-yyyy-93e4-de0c1b80416f --db meteor --collection kanji --type csv --file c:\kanjifinder\kanjifinder.csv --headerline
OK got it. This helped:
http://docs.mongodb.org/manual/reference/connection-string/
mongoimport --host meteor.m0.mongolayer.com --port 27017 --username client --password e63aaade-xxxx-yyyy-93e4-de0c1b80416f --db kanjifinder_meteor_com --collection kanji --type csv --file c:\kanjifinder\kanjifinder.csv --headerline
Using mongodump and mongorestore also works:
Dump data from existing mongodb (mongodb url: mongodb://USER:PASSWORD#DBHOST/DBNAME)
mongodump -h DBHOST -d DBNAME -u USER -p PASSWORD
This will create a "dump" directory, with all the data going to dump/DBNAME.
Get the mongodb url for the deployed meteor app (i.e. www.mymeteorapp.com)
meteor mongo --url METEOR_APP_URL
Note: the PASSWORD expires every min.
Upload the db dump data to the meteor app (using an example meteor db url)
mongorestore -u client -p dcc56e04-a563-4147-eff4-5ae7c1253c9b -h production-db-b2.meteor.io:27017 -db www_mymeteorapp_com dump/DBNAME/
All the data should get transferred!
If you get auth_failed error message your mongoimport version is too different from what's being used in meteor.com. So you need to upgrade. For ubuntu see https://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/#install-the-latest-stable-version-of-mongodb
#!/bin/sh
# Script to import csvfile to meteor application deployed to free meteor.com hosting.
# Make sure your versions of mongo match with the metor.com mongo versions.
# As Jan 2016 it seems to be 3.x something. Tested with mongoimport 3.12.
if [ $# -eq 0 ]
then
echo "usage: $0 xxx.meteor.com collection filename.csv"
exit 1
fi
URL=$1
COLLECTION=$2
FILE=$3
echo Connecting to $URL, please stand by.... collection=$COLLECTION file=$FILE
PUPMS=`meteor mongo --url $URL | sed 's/mongodb:\/\// -u /' | sed 's/:/ -p /' | sed 's/#/ -h /' | sed 's/\// -d /'`
mongoimport -v $PUPMS --type csv --headerline --collection $COLLECTION --file $FILE