According to Astronomer docs here:
Despite this, I'm still not quite sure how to structure the JSON in Extras for this. I've tried:
{ uri: mongodb+srv://myuser:mypass#my-cluster.dwxnd.gcp.mongodb.net/mydb?retryWrites=true&w=majority } in the Extras but that doesn't work:
It seems like this should be obvious, yet I am struggling. What's the correct way, using our MongoDB URI from MongoDB Atlas, to create this connection in Airflow?
This setup worked for me in MongoDB Atlas. Extra part is important as it adds mongodb+srv to the final connection URI. Make sure you have provider package installed (http://airflow.apache.org/docs/apache-airflow-providers-mongo/stable/index.html).
Conn Id: mongo_connection
Conn Type: MongoDB
Host: cluster0.mtfak.mongodb.net
Schema: MyDatabaseName
Login: myuser
Password: mypass
Port: empty
Extra: {"srv": true}
This is what I would try:
Conn Type: mongodb+srv (or mongodb)
Host:my-cluster.blahlah.mongodb.net,
Login: <username>, Password: <password>.
Schema: admin (or your authDB)
The JSON object is as simple as this
{ retryWrite:true,
<field>:value,
w:majority
}
Related
I'm trying to connect my deno application to mongodb but I get error.
import {MongoClient} from "https://deno.land/x/mongo#v0.21.2/mod.ts";
const client = await new MongoClient();
await client.connect("mongodb+srv://deno:i4vY8AtCEhr6ReqB#sample.7jp1l.mongodb.net/deno?retryWrites=true&w=majority");
const db = client.database("notes");
export default db;
everything seem to be fine but when I run the app, I get this error.
error: Uncaught (in promise) Error: MongoError: "Connection failed: failed to lookup address information: nodename nor servname provided, or not known"
throw new MongoError(`Connection failed: ${e.message || e}`);
^
at MongoClient.connect (client.ts:93:15)
at async mongodb.ts:4:1
2 problems that I see:
The code snippet above only works with Mongo installed in local machine.
The connection string use DNS Seed List, but the current library couldn't resolve to a list of hosts
To make it works with Mongo Atlas, you need to call a connect method with difference parameters and find a correct (static) host instead a (dynamic) DNS Seed List:
const client = new MongoClient();
const db = await client.connect({
db: '<your db or collection with work with>',
tls: true,
servers: [
{
host: '<correct host - the way to get the host - see bellow>',
port: 27017,
},
],
credential: {
username: '<your username>',
password: '<your password>',
mechanism: 'SCRAM-SHA-1',
},
});
How to get the correct host:
Open your Cluster in Mongo Atlas
Select Connect button
Select Connect to application option
Select Driver: Node.js and Version: 2.2.12 or later
Here you will see a list of host follow # character
Thank you #nthung.vlvn for the clue. Indeed the host needs to be a primary shard. It fixed the lookup address information, but I had another error, that my credentials are incorrect. I had to add db "admin" to credential:
credential: {
username: '<your username>',
password: '<your password>',
db: "admin",
mechanism: 'SCRAM-SHA-1',
}
That is weird, because I do not have admin db in my Atlas. Anyway it started to work.
In knex documentation of configuration of knexfile.js for PostgreSQL, they have a property called client, which looks this way:
...
client: 'pg'
...
However, going through some other projects that utilize PostgreSQL I noticed that they have a different value there, which looks this way:
...
client: 'postgresql'
...
Does this string correspond to the name of some sort of command line tool that is being used with the project or I misunderstand something?
Postgresql is based on a server-client model as described in 'Architectural Fundamentals'
psql is the standard cli client of postgres as mentioned here in the docs.
A client may as well be a GUI such as pg-admin, or a node-package such as 'pg' - here's a list.
The client parameter is required and determines which client adapter will be used with the library.
You should also read the docs of 'Server Setup and Operation'
To initialize the library you can do the following (in this case on localhost):
var knex = require('knex')({
client: 'mysql',
connection: {
host : '127.0.0.1',
user : 'your_database_user',
password : 'your_database_password',
database : 'myapp_test'
}
})
The standard user of the client deamon ist 'postgres' - which you can use of course, but its highly advisable to create a new user as stated in the docs and/or apply a password to the standard user 'postgres'.
On Debian stretch i.E.:
# su - postgres
$ psql -d template1 -c "ALTER USER postgres WITH PASSWORD 'SecretPasswordHere';"
Make sure you delete the command line history so nobody can read out your pwd:
rm ~/.psql_history
Now you can add a new user (i.E. foobar) on the system and for postgres
# adduser foobar
and
# su - postgres
$ createuser --pwprompt --interactive foobar
Lets look at the following setup:
module.exports = {
development: {
client: 'xyz',
connection: { user: 'foobar', database: 'my_app' }
},
production: { client: 'abc', connection: process.env.DATABASE_URL }
};
This basically tells us the following:
In dev - use the client xyz to connect to postgresqls database my_app with the user foobar (in this case without pwd)
In prod - retrieve the globalenv the url of the db-server is set to and connect via the client abc
Here's an example how node's pg-client package opens a connection pool:
const pool = new Pool({
user: 'foobar',
host: 'someUrl',
database: 'someDataBaseName',
password: 'somePWD',
port: 5432,
})
If you could clarify or elaborate your setup or what you like to achieve a little more i could give you some more detailed info - but i hope that helped anyways..
I am using both elasticsearch plugins and mongodb plugins in my grails project.
The application.yml file is like:
mongodb:
host: "localhost"
port: 27017
databaseName: "search"
elasticSearch:
client:
mode: local
disableAutoIndex: false
datastoreImpl: mongoDatastore
bulkIndexOnStartup: true
index:
store.type: mmapfs
I commented out the hibernate in build.grdle because I need to use mongodb as your primary datastore.
//compile "org.grails.plugins:hibernate4"
Then the mongodb worked well since I could use
MyDomain.findByXXXXX()
to query the data in my mongodb.
But when I used
def result = MyDomain.search("${query}")
or
def result = elasticSearchService.search("${query}")
and print the result, it showed
[total:0, searchResults:[]]
That seemed the elasticSearch plugins worked but it did not connect to the mongodb.I wonder whether datastoreImpl: mongoDatastore worked or not.
Is there something I am missing? Appreciate your help in advance.
Thank You
I've been working around some ways to use .native() to do a simple aggregation function in sails with mongo.
Already following the steps to install dependencies. (http://sailsjs.org/documentation/reference/waterline-orm/models/native)
But still it returns me this error : .native is not a function
Did I missed something ?
You may be using the wrong adapter. You can check this in your models.js in the connection key. It might be commented out, if it is, it's going to connect to to local disk. Check that in connections.js, the name of your object that has mongodb config is named the same as models.js. eg.
connections.js
mongoServer: {
adapter: 'sails-mongo',
host: 'localhost',
port: 27017,
database: 'dbname'
}
models.js
connection: 'mongoServer'
I am right now attempting my first Heroku deployment of a SailsJS API. My app uses SailsJS v0.11 andsails-mongo 0.11.2.
I have updated config/connections.js to include the connection information to MongoDB database I have hosted for free at Mongolab.
mongodb: {
adapter: 'sails-mongo',
url: "mongodb://db-user:password123#ds047812.mongolab.com:47812/testing-db"
}
Also updated config/models.js to point to that adapter.
module.exports.models = {
connection: 'mongodb',
migrate: 'safe'
};
This is basically all I have changed from running the code locally, when I deploy to Heroku the app crashes and I get this error...
/home/zacharyhustles/smallChangeAPI/node_modules/connect-mongo/lib/connect-mongo.js:186
throw err;
^
at Socket.emit (events.js:107:17)
2015-07-08T19:37:00.778316+00:00 app[web.1]:
at Socket.<anonymous> (/app/node_modules/connect-mongo/node_modules/mongodb/lib/mongodb/connection/connection.js:534:10)
Error: Error connecting to database: failed to connect to [localhost:27017]
How do I get rid of this, and make sure Sails does not try connecting to localhost db?
Ok, the problem was with storing sessions.
My solution was to setup a Redis database to store sessions.
In config/sessions.js make sure everything is commented out except for the method you want for session store.
Mine looked like this:
adapter: 'redis',
host: 'example.redistogo.com',
port: 1111,
db: '/redistogo',
pass: 'XXXXXYYYYYYXYXYXYYX',
This solved my posted problem, hope this helps another person out.