We have a spring boot application (2.2.2) which uses mongodb. As we want to deploy it in AWS we need to migrate to DocumentDB. But when we migrate, things which were working in MongoDB are no longer working. See the following code:
public int getCount(String collection, List<Bson> pipeline, MongoTemplate mongoTemplate) {
List<Bson> newpipeline = new ArrayList<Bson>();
newpipeline.addAll(pipeline);
String cntStr = "{$group:{_id:null, val:{'$sum':1}}}";
newpipeline.add(BasicDBObject.parse(cntStr));
MongoDatabase mongo = mongoTemplate.getDb();
int count = 0;
try {
count = mongo.getCollection(collection).aggregate(newpipeline).first().getInteger("val");
} catch (NullPointerException e) {
} catch (Exception e) {
e.printStackTrace();
}
return count;
}
This works in mongodb, but not in documentdb.Gives error like:
{"timestamp":"2021-03-29T08:29:58.101+0000","status":500,"error":"Internal Server Error","message":"Command failed with error 18537: 'Could not convert date to string: date component was outside the supported range of 0-9999: 10000' on server xxxx-manager-document-db.cn30hpxqos6b.eu-central-1.docdb.amazonaws.com:27017. The full response is {\"ok\": 0.0, \"operationTime\": {\"$timestamp\": {\"t\": 1617006598, \"i\": 1}}, \"code\": 18537, \"errmsg\": \"Could not convert date to string: date component was outside the supported range of 0-9999: 10000\"}","path":"/pmt/api/v1/pm/analytics/getKpiByDimension"}
Is it possible that some of the cursor apis do not work in documentDB?
Another option that you have is to install MongoDB on an EC2 instance then you can use the MongoDB API within a Spring BOOT Application. To learn how to install MongoDB on EC2, see:
Install and configure MongoDB community edition
There is an example of creating a Spring BOOT app that uses MongoDB to store application data here.
Creating the MongoDB web application item tracker
Related
I am using MongoDB URI (mongodb://user:pass#IP:27017/myDB?retryWrites=false&connectTimeoutMS=10000) configuration in Spring-boot. I observed approx every 5 min. Mongodb not responding for first hit, Second hit working fine.
Some time getting this message "Opened connection [connectionId{localValue:8}]" in log.
Java Configuration.
#Bean
public MongoDbFactory mongoDbFactory() {
return new SimpleMongoDbFactory(new MongoClientURI(prop.getDbConnectionUri()));
}
#Bean
public MongoTemplate mongoTemplate() {
log.info("Loging MongoDB Config Loging...");
MongoTemplate mongoTemplate = new MongoTemplate(mongoDbFactory());
return mongoTemplate;
}
Please suggest any other optional configuration required in Mongo URI.
After so many Changes,I fixed this problem by changing below system configuration. This is not related to mongo configuration.
Open sysctl.conf by using "sudo vim /etc/sysctl.conf" command.
Change "net.ipv4.tcp_keepalive_time = 220".
Save Changes.
Run "sysctl --system" command.
And Restart your micro service.
Now Mongo Connection working fine.
I am trying to setup mongoDB in my Project (using Quasar Framework)
What I did for preparation:
Setup MongoDB Atlas on Windows and create a user, a database and a collection
Create a Connection String
installed mongodb in node_modules (npm install —-save mongodb)
What I tested
Using axios to get test data in JSON format (worked)
Using axios to access the mongoDB (Didn't find a way to do it - Maybe this is not supported)
creating a separate js file to connect to mongoDB (see below)
Access mongoDB via js
After I failed to access mongoDB with axios I tried to connect using this code:
const mongo = require('mongodb')
const MongoClient = mongo.MongoClient
const uri = YOUR_CONNECTION_STRING
var client;var mongoClient = new MongoClient(uri, { reconnectTries :
Number.MAX_VALUE, autoReconnect : true, useNewUrlParser : true })
mongoClient.connect((err, db) => { // returns db connection
if (err != null) {
console.log(err)
return
}
client = db
})
Problem here is, I don't know how to implement this into my vue/quasar app. Do I have to save this as a component or can I access it directly using <script src="app.js">?
Is using the above code the correct way to connect to my mongoDB database or is there an easier way?
I have an application based using .Net Core 2.2 that is connecting to MondoDb cluster V3.6 in Atlas. The application is hosted in Azure as a Linux Docker container. The app is using MongoDB .Net driver 2.7.3. The app periodically (once in a couple minutes) receives the following timeout exceptions:
System.TimeoutException at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException
and
System.TimeoutException at MongoDB.Driver.Core.Connections.TcpStreamFactory.ConnectAsync
The mongo client instance is configured according to the MongoDb docs, i.e.
var url = MongoUrl.Create("mongodb+srv://user:password#cluster.gcp.mongodb.net/?authSource=admin&retryWrites=true&ssl=true");
var clientSettings = MongoClientSettings.FromUrl(url);
clientSettings.SslSettings = new SslSettings() { EnabledSslProtocols = SslProtocols.Tls12 };
void SocketConfigurator(Socket s) => s.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.KeepAlive, true);
clientSettings.ClusterConfigurator = builder =>
builder.ConfigureTcp(tcp => tcp.With(socketConfigurator: (Action<Socket>)SocketConfigurator));
return new MongoClient(clientSettings);
I checked number of SO questions including MongoDB C# 2.0 TimeoutException and SocketTimeout with opened connection in MongoDB but the suggestions seem to be either outdated (reported as fixed in the current version of driver) or don't have permanent positive effect (setting timeouts in the connection string i.e. connectTimeoutMS=90000&socketTimeoutMS=90000&maxIdleTimeMS=90000). The second one (setting tcp_keepalive_time) seems to be not applicable to a docker container in Azure. Please help.
Have you tried setting like this:
var client = new MongoClient(new MongoClientSettings
{
Server = new MongoServerAddress("xxxx"),
ClusterConfigurator = builder =>
{
builder.ConfigureCluster(settings => settings.With(serverSelectionTimeout: TimeSpan.FromSeconds(10)));
}
});
I have a MongoDB setup on azure, and I am tring to connect to it via azure function.
These are the steps I took:
Creating a Simple Azure Function
Installed the MongoDB Driver on Azure, To install the MongoDB Node.js driver, I went go to .scm.azurewebsites.net, and clicked on 'Debug Console' -> 'PowerShell'.
I Navigated to the D:\home\site\wwwroot directory and clicked on the plus icon to create a new file called package.json.
I Created and saved the below package.json file.
{
"name": "nameofunction",
"dependencies": {
"mongodb": "3.x"
}
}
Next, I ran npm install from the shell.
From the Azure Function I should be able to connect to MongoDB and execute a query using the below code.
const mongodb = require('mongodb');
const url = "mongodb://cosmod: <PASSWORD>==#cosmodb.documents.azure.com:10255/?ssl=true&replicaSet=globaldb";
module.exports = async function (context, req) {
mongodb.connect(url, function(error, client) {
if (error) throw error;
var dbo = client.db("mydb");
dbo.createCollection("customers", function(err, res) {
if (err) throw err;
context.log("Collection created!");
db.close();
});
});
};
My code is throwing up a Status: 500 Internal Server Error
The more I look at the code, the more i can not understand why this should not work.
The package-lock.jsonhas all the dependencies loaded after I ran npm install in the shell.
I appreciate any help in resolving this.
This seems weired, I also followed same and was able to connect to my db.
Can you please check your cosmos connectiondb, mongo compatible connection string? Are you able to connect that from other mongo clients
Status: 500 Internal Server Error
I assume that it dues to the code mongodb.connect(url, function(error, client)
please change the code to
mongodb.MongoClient.connect(uri, function(error, client)
A review of the Azure Cosmos DB account- Quick start documentation accessible via the Azure Cosmos DB account menu side blade; connecting the MongoDB app is among others via:
the Node.js 2.2 driver and
the Node.js 3.0 driver
I was using the Node.js 2.2 driver connection string in the azure function which is not compatible with the Node.js 3+ driver dependency in my app. Using the Node.js 3.0 driver connection string, I was able to connect the MongoDB app, without the error. The double equality sign in the password string is url encoded in the 3+ driver.
Node.js 3+ driver connection string
var mongoClient = require("mongodb").MongoClient;
mongoClient.connect("mongodb://cosmodb:<PWD>%3D%3D#cosmodb.documents.azure.com:10255/?ssl=true", function (err, client) {
client.close();
});
Node.js 2.2 driver connection string
var mongoClient = require("mongodb").MongoClient;
mongoClient.connect("mongodb://cosmodb:<PWD>==#cosmodb.documents.azure.com:10255/?ssl=true", function (err, db) {
db.close();
});
I have a Jhipster Spring boot project. Recently I shifted from mlabs standalone sandboxes to Atlas cluster sandbox M0 Free tier replica set. It even worked and I had made some database operations on it. But now for some reason then there is a read permission error
Error creating bean with name 'mongobee' defined in class path resource [DatabaseConfiguration.class]: Invocation of init method failed; nested exception is com.mongodb.MongoQueryException: Query failed with error code 8000 and error message 'user is not allowed to do action [find] on [test.system.indexes]' on server ********-shard-00-01-mfwhq.mongodb.net:27017
You can see the full stack here https://pastebin.com/kaxcr7VS
I have searched high and low and all I could find is that M0 tier user doesn't have permissions to overwrite admin database which I am not doing.
Even now connection to Mlabs DB works fine but have this issue on Atlas DB M0 tier.
Mongo DB version : 3.4
Jars and It's version
name: 'mongobee', version: '0.10'
name: 'mongo-java-driver', version: '3.4.2'
#Neil Lunn
The userId I am using to connect is that of admin's and the connection read and write works through shell or Robo3T(mongo client)
After discussion with MongoDB support team, MongoDB 3.0 deprecates direct access to the system.indexes collection, which had previously been used to list all indexes in a database. Applications should use db.<COLLECTION>.getIndexes() instead.
From MongoDB Atlas docs it can be seen that they may forbid calls to system. collections:
Optionally, for the read and readWrite role, you can also specify a collection. If you do not specify a collection for read and readWrite, the role applies to all collections (excluding some system. collections) in the database.
From the stacktrace it's visible that MongoBee is trying to make this call, so it's now the library issue and it should be updated.
UPDATE:
In order to fix an issue until MongoBee has released new version:
Get the latest sources of MongoBee git clone git#github.com:mongobee/mongobee.git, cd mongobee
Fetch pull request git fetch origin pull/87/head:mongobee-atlas
Checkout git checkout mongobee-atlas
Install MongoBee jar mvn clean install
Get compiled jar from /target folder or local /.m2
Use the jar as a dependency on your project
Came across this issue this morning. Heres a quick and dirty monkey-patch for the problem:
package com.github.mongobee.dao;
import com.github.mongobee.changeset.ChangeEntry;
import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.DBObject;
import java.util.List;
import static com.github.mongobee.changeset.ChangeEntry.CHANGELOG_COLLECTION;
public class ChangeEntryIndexDao {
public void createRequiredUniqueIndex(DBCollection collection) {
collection.createIndex(new BasicDBObject()
.append(ChangeEntry.KEY_CHANGEID, 1)
.append(ChangeEntry.KEY_AUTHOR, 1),
new BasicDBObject().append("unique", true));
}
public DBObject findRequiredChangeAndAuthorIndex(DB db) {
DBCollection changelogCollection = db.getCollection(CHANGELOG_COLLECTION);
List<DBObject> indexes = changelogCollection.getIndexInfo();
if (indexes == null) return null;
for (DBObject index : indexes) {
BasicDBObject indexKeys = ((BasicDBObject) index.get("key"));
if (indexKeys != null && (indexKeys.get(ChangeEntry.KEY_CHANGEID) != null && indexKeys.get(ChangeEntry.KEY_AUTHOR) != null)) {
return index;
}
}
return null;
}
public boolean isUnique(DBObject index) {
Object unique = index.get("unique");
if (unique != null && unique instanceof Boolean) {
return (Boolean) unique;
} else {
return false;
}
}
public void dropIndex(DBCollection collection, DBObject index) {
collection.dropIndex(index.get("name").toString());
}
}
Caused by: java.lang.NoSuchMethodError: com.github.mongobee.dao.ChangeEntryIndexDao.<init>(Ljava/lang/String;)V
at com.github.mongobee.dao.ChangeEntryDao.<init>(ChangeEntryDao.java:34)
at com.github.mongobee.Mongobee.<init>(Mongobee.java:87)
at com.xxx.proj.config.DatabaseConfiguration.mongobee(DatabaseConfiguration.java:62)
at com.xxx.proj.config.DatabaseConfiguration$$EnhancerBySpringCGLIB$$4ae465a5.CGLIB$mongobee$1(<generated>)
at com.xxx.proj.config.DatabaseConfiguration$$EnhancerBySpringCGLIB$$4ae465a5$$FastClassBySpringCGLIB$$f202afb.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358)
at com.xxx.proj.config.DatabaseConfiguration$$EnhancerBySpringCGLIB$$4ae465a5.mongobee(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
... 22 common frames omitted
jhipster 5 must be using a different version, because i get that when implementing the above code. looks like its expecting a different version.
This access to system.indexes is an open issue in mongobee. The issue has been fixed in the project, but the project was abandoned before the fix was ever released.
Due to this project abandonment, two successor libraries have since been forked from mongobee which have fixed this issue: Mongock and mongobeeJ.
Switching your application's dependency from the mongobee library to one of these successor libraries will allow you to run mongobee database migrations on Atlas.
To summarize these libraries:
Mongock - Forked from mongobee in 2018. Actively maintained. Has evolved significantly from the original, including built-in support for Spring, Spring Boot, and both versions 3 & 4 of the Mongo Java driver.
mongobeeJ - Forked from mongobee in 2018. Five updated versions have been released. Minimal evolution from the original mongobee. Mongo Java Driver 4 support was implemented in August, 2020. This project was deprecated in August, 2020, with a recommendation from its creators to use a library such as Mongock instead.