MonoDb creates connection every single operation on collection - mongodb

I'm using mongodb-java-driver, and nothing else except that. I created singleten EJB with connection to Mongo.
#Singleton
public class MongoConnection {
private DB db = null;
private MongoClient mongoClient = null;
#PostConstruct
public void init() {
try {
mongoClient = new MongoClient("localhost", 27017);
db = mongoClient.getDB("mydb");
} catch (UnknownHostException e) {
e.printStackTrace();
}
}
public MongoConnection() {
}
public DB getDb() {
return db;
}
public DBCollection getCollectionInDatabase(String collection) {
DBCollection coll;
coll = db.getCollection(collection);
return coll;
}
}
I get this ejb in ApplicationScoped bean (JSF) (just to be sure, that I will have only ONE instance of DB connection).
#Named("appMongo")
#ApplicationScoped
public class MongoApplicationScope implements Serializable{
private static final long serialVersionUID = 1L;
#EJB MongoConnection mu;
public MongoConnection getMu() {
return mu;
}
public void setMu(MongoConnection mu) {
this.mu = mu;
}
}
Then in request scoped bean I get data from db
#Named("mongoBean")
#SessionScoped
public class MongoBean implements Serializable {
private static final long serialVersionUID = 1L;
#Inject MongoApplicationScope mongoAccess;
public void mongoDzialanie() {
DBCollection coll = mongoAccess.getMu().getDb().getCollection("oko"); //at this step everything is correct
System.out.println(coll.getCount()); //new connection is created text from mongoDB console -> connection accepted from 127.0.0.1:57700 #2 (2 connections now open)
}
Why even if I have the same "db" object instance I can't get data without creating new connection, why I can't share this connection as it should be due to pooling?
}

MongoDB drivers open new connections on the background each time you work with a collection. Drivers decide when to open a new one. I believe it depends on the driver's implementation.
You can control the max number of connections opened by setting poolSize value (default is 5 for Node.JS MongoDb driver http://mongodb.github.io/node-mongodb-native/2.2/api/MongoClient.html#connect) It could be different for Java or other languages. Check your documentation.
If you are going to have more than one db object, each object will have its own connection pool. In my case I have mainDb and logsDb. Each have pool of 10. Therefore, up to 20 concurrent connections will be opened.
Finally, if you are using node.js driver, make sure to pass Number as value and not string (i.e poolSize:10). This will save you hours/days of troubleshooting :)

Related

Why MongoDB Atlas does not work with reactive Spring Data and Spring Boot?

I'm developing an easy sample application. One component is a gateway service using Spring Boot and Reactive Spring Data for Mongo, because that's where I want to store user and login informations.
For testing out different solutions, I wanted to use MongoDB Atlas. So, I set up an application. But when I want to save just a sample user, nothing happens, the data is not saved to the database. However it looks like the application is connected to the MongoDb Atlas. No error logs about failed connections.
This is the main class, where I have the #EnableReactiveMongoRepositories annotation:
#SpringBootApplication
#EnableReactiveMongoRepositories("com.bkk.sm.authentication.repository")
public class GatewayApplication {
public static void main(String[] args) {
SpringApplication.run(GatewayApplication.class, args);
}
}
Here is how I set up Mongo in application.yml and the repository:
spring:
data:
mongodb:
database: users
uri: mongodb+srv://${MONGO_USER}:${MONGO_PASSWORD}#taocluster.qa3sd.mongodb.net/users?retryWrites=true&w=majority
#Repository
public interface ReactiveUserRepository extends ReactiveMongoRepository<User, String> {
Mono<User> findByUsername(String username);
}
I don't use any specific reactive MongoDB config, I don't extend the AbstractReactiveMongoConfiguration (this is really just a bout to experiment how does this work) and I use the defaults.
In my UserDetailsServiceImpl, I try to save a sample record, just right after the bean is constructed:
#Slf4j
#Service
public class UserDetailsServiceImpl implements UserDetailsService {
private ReactiveUserRepository repository;
public UserDetailsServiceImpl(ReactiveUserRepository repository) {
this.repository = repository;
}
#PostConstruct
public void setup() {
BCryptPasswordEncoder encoder = new BCryptPasswordEncoder();
String pwd = encoder.encode("user");
User user = User.builder()
.username("user")
.password(pwd)
.accountExpired(false)
.accountLocked(false)
.activationCode(null)
.activatedTime(Date.from(Instant.now()))
.email("user#user.com")
.enabled(true)
.firstName("User")
.failedLoginAttempts(0)
.lastModificationTime(Date.from(Instant.now()))
.lastName("User")
.middleName("User")
.passwordExpiryTime(Date.from(Instant.now()))
.registrationTime(Date.from(Instant.now()))
.roles(List.of(CompanyRole.builder().companyCode("bkk")
.companyName("Beszterce KK")
.role(Role.ROLE_USER)
.build())
)
.passwordExpiryTime(null)
.version(0)
.build();
this.repository.save(user).map(user1 -> {
log.info("User saved. {}", user1);
return user1;
}).onErrorResume(Objects::nonNull, throwable -> {
log.error("Something is not right here.", throwable);
return Mono.error(throwable);
}).switchIfEmpty(Mono.defer(() -> {
log.info("Cannot save ure={}", user.toString());
return Mono.error(new Exception("WTF?"));
}));
}
... SOME MORE METHODS COME HERE
}
When it executes the this.repository.save(user) line, nothing happens. Well, I tried to debug and went deeper into the framework but ultimately, nothing happens. That's why I added some log messages. But nothing. If I put a breakpoint to the map or onErrorResume or switchIfEmpty branches, the execution doesn't stop there. No log is written to console other that this line:
2022-04-09 00:02:46.061 INFO 72528 --- [ntLoopGroup-3-7] org.mongodb.driver.connection : Opened connection [connectionId{localValue:7, serverValue:78530}] to taocluster-shard-00-02.qa3sd.mongodb.net:27017
And here is my data object where I declare the collection name:
#Getter
#Builder
#NoArgsConstructor
#AllArgsConstructor
#Document(collection = "users")
public class User implements UserDetails {
#Id
private String id;
#Indexed
#NonNull
private String username;
... SOME MORE FIELDS COME HERE ...
}
So, my question is, what am I doing wrong? Why I don't see anything added to my MongoDB Atlas sample database? Where I just set the 0.0.0.0/0 for accepting connections from everywhere for the time being of testing this stuff out.
Any help would be appreciated.

Use multiple mongo DBs in same application for same model & same Repository

I need to implement Spring boot - MongoDb application where There are 2 mongo DBs which have exact same database name & collections. Based on User making a request, i need to choose whether to fetch data from DB1 or DB2 (only difference in mongo URI host - IP).
E.g. I need some way to create 2 mongoTemplates like mTempA & mTempB in my Repository & based on some condition, use either of the template to execute query as below:
#Repository
public class MyCustomRepository {
private Logger logger = LoggerFactory.getLogger(MyCustomRepository.class);
#Autowired
private MongoTemplateA mongoTemplateA;// Need to know if this is possible & how
#Autowired
private MongoTemplateB mongoTemplateB;// Need to know if this is possible & how
public List<MyModel> findByCriteria(MyRequest request) {
List<MyModel> result;
//Query query = <build query based on request>
if (request.getUserType().equals("A")) {
result = mongoTemplateA.find(query, MyModel.class);
} else {
result = mongoTemplateB.find(query, MyModel.class);
}
logger.debug("Result fetched with {} records", result.size());
return result;
}
}
I don't want to have 2 separate Repo (Class or Interfaces) or different models to be used. Just want to have 2 different mongoTemplates to be injected in single repo.
Is this possible? If yes, please give some example code.
I have followed below tutorial:
https://dzone.com/articles/multiple-mongodb-connectors-with-spring-boot
As rightly pointed out by #Lucia, below is how it can be done:
Have 2 different configuration placeholders
#Configuration
#EnableMongoRepositories(basePackages = "com.snk.repository", mongoTemplateRef = "mongoTemplateA")
public class MongoConfigA {
// Configuration class for DB 1 access
}
#Configuration
#EnableMongoRepositories(basePackages = "com.snk.repository", mongoTemplateRef = "mongoTemplateB")
public class MongoConfigB {
// Configuration class for DB 2 access
}
Get one class which will help in reading custom properties for mongo db properties in application.properties:
#ConfigurationProperties(prefix = "mongodb")
public class MultipleMongoProperties {
private MongoProperties adb = new MongoProperties();
private MongoProperties bdb = new MongoProperties();
public MongoProperties getAdb() {
return adb;
}
public MongoProperties getBdb() {
return bdb;
}
}
Add a configuration class to create mongoTemplates:
#Configuration
#EnableConfigurationProperties(MultipleMongoProperties.class)
public class MultipleMongoConfig {
#Autowired
private MultipleMongoProperties mongoProperties = new MultipleMongoProperties();
#Bean(name = "mongoTemplateA")
#Primary
public MongoTemplate mongoTemplateA() {
return new MongoTemplate(aDbFactory(this.mongoProperties.getAdb()));
}
#Bean(name = "mongoTemplateB")
public MongoTemplate mongoTemplateB() {
return new MongoTemplate(bDbFactory(this.mongoProperties.getBdb()));
}
#Bean
#Primary
public MongoDbFactory aDbFactory(final MongoProperties mongo) {
return new SimpleMongoDbFactory(new MongoClientURI(mongo.getUri()));
}
#Bean
public MongoDbFactory bDbFactory(final MongoProperties mongo) {
return new SimpleMongoDbFactory(new MongoClientURI(mongo.getUri()));
}
}
Add below decelerations to your service/repository:
#Autowired
#Qualifier("mongoTemplateA")
private MongoTemplate mongoTemplateA;
#Autowired
#Qualifier("MongoTemplateB")
private MongoTemplate MongoTemplateB;
Add below properties in your application.properties:
mongodb.adb.uri=mongodb://user:pass#myhost1:27017/adb
mongodb.bdb.uri=mongodb://user:pass#myhost2:27017/bdb
If you have mongo rplica set, URL can be set as:
mongodb.adb.uri=mongodb://user:pass#myhost1,myhost2,myhost13/adb?replicaSet=rsName
mongodb.bdb.uri=mongodb://user:pass#myhost1,myhost2,myhost13/bdb?replicaSet=rsName
Based on your logic, use either of the template.
Thought, there are few catches:
Notice the #Primary annotation, one bean needs to be marked as primary. I haven't find any solution if no template is marked primary.
If any of the mongo DB is down & application is started/restarted, application will not start/deploy. to avoid this, #Autowired needs to be changed to #Autowired(required = false).
If any of the mongo DB is down & application is already running, it automatically uses 2nd mongo BD (which is not down). So, even if you want to use A DB, if it's down, requests are processed with B DB & vice-versa.

Spring Boot + MongoDB: How to reuse connections

I'm using SpringBoot + MongoDB. I created my object as follows.
I am able to #Autowrite the DocumentStoreConfig object in my Service/Controller and make calls to Mongo.
Sample call:
#Autowired
private DocumentStoreConfig docStoreConfig;
this.docStoreConfig.mongoClient().getDatabase("db_name").getCollection(collection).insertOne(doc);
Problem I see is that each call does a 'new' MongoClient and opens up a new connections.
What is the guidance on setting up a pool.. or reusing the same connection object rather than making the painful cost of opening a brand new connection.
#Configuration
public class DocumentStoreConfig extends AbstractMongoConfiguration {
#Value("${spring.data.mongodb.uri}")
private String connectionString;
#Value("${documentstore.database}")
private String databaseName;
#Override
public String getDatabaseName() {
return this.databaseName;
}
#Override
public MongoClient mongoClient() {
System.out.println("**** \n\n\n NEW MONGO \n\n\n");
return new MongoClient(new MongoClientURI(this.connectionString));
}
public MongoCollection<Document> getFailureCollection() {
return this.mongoClient().getDatabase(this.databaseName).getCollection("failure");
}
}

Morphia, Embed Mongo and Spring. Address already in use

I am trying use MongoDB, Morphia and Spring and test it, so I started use Embedded Mongo.
When I had only one DAO to persist I did not had any problem with my tests, however, in some cases I needed use more than one DAO, and in that cases my injected Datasore give me an problem: addr already in use.
My Spring Test Database Configuration is this:
#Configuration
public class DatabaseMockConfig {
private static final int PORT = 12345;
private MongodConfigBuilder configBuilder;
private MongodExecutable mongodExecutable;
private MongodProcess mongodProcess;
#Bean
#Scope("prototype")
public MongodExecutable getMongodExecutable() {
return this.mongodExecutable;
}
#Bean
#Scope("prototype")
public MongodProcess mongodProcess() {
return this.mongodProcess;
}
#Bean
public IMongodConfig getMongodConfig() throws UnknownHostException, IOException {
if (this.configBuilder == null) {
configBuilder = new MongodConfigBuilder().version(Version.Main.PRODUCTION).net(new Net(PORT, Network.localhostIsIPv6()));
}
return this.configBuilder.build();
}
#Autowired
#Bean
#Scope("prototype")
public Datastore datastore(IMongodConfig mongodConfig) throws IOException {
MongodStarter starter = MongodStarter.getDefaultInstance();
this.mongodExecutable = starter.prepare(mongodConfig);
this.mongodProcess = mongodExecutable.start();
MongoClient mongoClient = new MongoClient("localhost", PORT);
return new Morphia().createDatastore(mongoClient, "morphia");
}
#Autowired
#Bean
#Scope("prototype")
public EventDAO eventDAO(final Datastore datastore) {
return new EventDAO(datastore);
}
#Autowired
#Bean
#Scope("prototype")
public EditionDAO editionDAO(final Datastore datastore) {
return new EditionDAO(datastore);
}
}
And my DAO classes are similar to that
#Repository
public class EventDAO {
private final BasicDAO<Event, ObjectId> basicDAO;
#Autowired
public EventDAO(final Datastore datastore) {
this.basicDAO = new BasicDAO<>(Event.class, datastore);
}
...
}
My test class is similar to that:
#ContextConfiguration(classes = AppMockConfig.class)
#RunWith(SpringJUnit4ClassRunner.class)
public class EventDAOTest {
#Autowired
private EventDAO eventDAO;
#Autowired
private MongodExecutable mongodExecutable;
#Autowired
private MongodProcess mongodProcess;
#Rule
public ExpectedException expectedEx = ExpectedException.none();
#After
public void tearDown() {
this.mongodProcess.stop();
this.mongodExecutable.stop();
}
...
}
I use prototype scope to solve problem with singleton and make sure that my mock database is clean when I start my test, after that I stop mongod process and mongod executable.
However since I need use more than one DAO I receive that error:
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'editionDAO' defined in class br.com.mymusicapp.spring.DatabaseMockConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [org.mongodb.morphia.Datastore]: :
Error creating bean with name 'datastore' defined in class br.com.mymusicapp.spring.DatabaseMockConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.mongodb.morphia.Datastore]:
Factory method 'datastore' threw exception; nested exception is java.io.IOException: Could not start process: ERROR: listen(): bind() failed errno:98 Address already in use for socket: 0.0.0.0:12345
2015-01-04T01:05:04.128-0200 [initandlisten] ERROR: addr already in use
I know what the error means, I just do not know how can I design my Configuration to solve that. As last option I am considering install a localhost MongoDB just for tests, however I think could be a better solution
That is based on the embedded mongod by flapdoodle, right?
If you want to run multiple tests in parallel (could be changed via JUnit annotations, but it's probably faster in parallel), you cannot use a single, hardcoded port. Instead, let the embedded process select an available port automatically.

MongoDB IRepository db Connections

This is what I have so far with regards to my IRepository for MongoDB and was wondering whether or not I'm on the right lines?
public abstract class Repository<TEntity> : IRepository<TEntity> {
private const string _connection = "mongodb://localhost:27017/?safe=true";
private MongoDatabase _db;
protected abstract string _collection{get;}
public Repository() {
this._db = MongoServer.Create(_connection).GetDatabase("Photos");
}
public IQueryable<TEntity> FindAll() {
return this._db.GetCollection<TEntity>(_collection).FindAll().AsQueryable();
}
}
This way I can create my PhotoRepository class that inherits from here and supplies the required _collection name.
I just want to make sure that I'm opening the connection to the db in the correct place and in the correct way.
Yes, this is fine. MongoServer.Create will return the same instance of MongoServer when passed the same connection string, so it is safe to call MongoServer.Create as many times as you want.