Looking for some proper documentation on async commit for mongo db . We have a spring boot app where we are trying to generate audits for our domain objects , we would like to commit the audits generated by javers into mongo db asynchronously while our main SQL based transaction is fr of this mongodb call. Any pointers on this would be really helpful.
If you are using the Javers Spring Boot Mongo starter, you can simply put the #JaversAuditableAsync ann on a repository method.
There are limitations:
It works only with Mongo and there is no integration for magical-autogenerated ReactiveMongoRepository yet. So you have to put the #JaversAuditableAsync on an actual method which does the save.
#Repository
interface DummyObjectReactiveRepository
extends ReactiveMongoRepository<DummyObject, String> { }
...
#Repository
class MyRepository {
#Autowired DummyObjectReactiveRepository dummyObjectReactiveRepository;
#JaversAuditableAsync
void save(DummyObject d){
dummyObjectReactiveRepository.save(d)
}
}
Related
I am trying to configure in memory embedded Mongo database with Spring Boot similar to H2 Database. Is there any option available in Spring Mongo?
MongoDB does not have an in-memory storage engine.
You can use tmpfs or zram to create a partition backed by memory and store the data in that partition.
You can create In-memory mongodb-server by using following dependency
<dependency>
<groupId>de.bwaldvogel</groupId>
<artifactId>mongo-java-server</artifactId>
<version>1.38.0</version>
</dependency>
Then declare mongo client in config class
#Override
public MongoClient mongoClient() {
MongoServer server = new MongoServer(new MemoryBackend());
// bind on a random local port
InetSocketAddress serverAddress = server.bind();
return mongoClient=MongoClients.create(
MongoClientSettings.builder()
.applyToClusterSettings(builder ->
builder.hosts(Arrays.asList(new ServerAddress(serverAddress))))
.build());
}
I am using MongoDB URI (mongodb://user:pass#IP:27017/myDB?retryWrites=false&connectTimeoutMS=10000) configuration in Spring-boot. I observed approx every 5 min. Mongodb not responding for first hit, Second hit working fine.
Some time getting this message "Opened connection [connectionId{localValue:8}]" in log.
Java Configuration.
#Bean
public MongoDbFactory mongoDbFactory() {
return new SimpleMongoDbFactory(new MongoClientURI(prop.getDbConnectionUri()));
}
#Bean
public MongoTemplate mongoTemplate() {
log.info("Loging MongoDB Config Loging...");
MongoTemplate mongoTemplate = new MongoTemplate(mongoDbFactory());
return mongoTemplate;
}
Please suggest any other optional configuration required in Mongo URI.
After so many Changes,I fixed this problem by changing below system configuration. This is not related to mongo configuration.
Open sysctl.conf by using "sudo vim /etc/sysctl.conf" command.
Change "net.ipv4.tcp_keepalive_time = 220".
Save Changes.
Run "sysctl --system" command.
And Restart your micro service.
Now Mongo Connection working fine.
I'm trying to use Liquibase version 3.6 to maintain database version changes. I'm able to execute database version changes when i need to execute single file changes.
I have using below code to execute version changes successfully, but my intention is to dynamically execute the change logs from a folder and not single file. I'm using only Java for all configuration of Liquibase
#Startup
#Singleton
#TransactionManagement(TransactionManagementType.BEAN)
public class InitializerBean {
#Resource(mappedName = "java:/M2M_RESOURCES")
private DataSource ds;
#PostConstruct
protected void bootstrap() {
ResourceAccessor resourceAccessor = new ClassLoaderResourceAccessor(getClass().getClassLoader());
try (Connection connection = ds.getConnection()) {
JdbcConnection jdbcConnection = new JdbcConnection(connection);
Database db = DatabaseFactory.getInstance().findCorrectDatabaseImplementation(jdbcConnection);
Liquibase liquiBase = new Liquibase("data/liquibase/", resourceAccessor, db);
liquiBase.update("Development");
} catch (SQLException | LiquibaseException e) {
}
}
}
When the first parameter of Liquibase class is single file , liquibase is able to execute changes but when I intent to execute all file of single folder is not able to track and execute changes.
I'm using JDK 1.8 and file in data/liquibase/ is dbChangelog.sql and dbChangelog_2.sq. This all code is deployed on Wildfly 10 as part of ear archive
In my spring boot application i have configured two databases in my yml file. So now I want to specify which database is to use in each MongoRepository. Is this possible? If so, how to do it? Thanks in advance.
yml file :
spring:
data:
mongodb:
first:
host: 127.0.0.1
port: 27017
database: db_admin
rest:
base-path: /admin
second:
host: 127.0.0.1
port: 27018
database: `user_forms`
rest:
base-path: /users
So in User MongoRepository i want to use the user_forms database.
The user MongoRepository :
#RepositoryRestResource(collectionResourceRel = "users",path = "users")
public interface UserRepository extends MongoRepository<User, String> {
List<User> findByUserId(String id);
}
Spring Boot auto configuration provides a default MongoTemplate to facilitate generating MongoDB Repositories.
Nevertheless if you want to use multi MongoDB databases, you would need to
register MongoTemplates for every MongoDB databases
designate each of the MongoTemplates to base packages of MongoRepositories accordingly, so that Spring Data MongoDB is informed to use the right template when generating repositories.
For example, say you registered 2 MongoTemplates, namely templateAdmin and templateUser, configured explicitly to their MongoDB. Then you could use code like this to inform Spring Data MongoDB which repositories use which template:
#EnableMongoRepositories(
basePackages = "com.the.base.package.name.of.admin.repositories",
mongoTemplateRef = "adminTemplateBeanName")
#Configuration
public class AdminMongoConfig {
#Bean
public MongoTemplate adminTemplateBeanName() {
//...
}
}
Configure Multiple MongoDB repositories with Spring Data Mongo Basically explained almost all points what I have said. So my wild guess is that you may have a misconception that MongoTemplate is irrelevant to MongoRepository, which is not the case.
Adding an answer despite having marked the question as a duplicate, as there is some confusion whether MongoRepository uses MongoTemplate.
When you inject your MongoRepository, it is proxied by an instance of SimpleMongoRepository class. it has a field
private final MongoOperations mongoOperations;
MongoOperations is an interface, and MongoTemplate is its implementation.
Now, the question is where does this default mongo template come from in a Spring Boot application.
Look at MongoDataAutoConfiguration
#Bean
#ConditionalOnMissingBean
public MongoTemplate mongoTemplate(MongoDbFactory mongoDbFactory,
MongoConverter converter) {
return new MongoTemplate(mongoDbFactory, converter);
}
Therefore, I am still claiming it is a duplicate of: Configure Multiple MongoDB repositories with Spring Data Mongo
I have a Jhipster Spring boot project. Recently I shifted from mlabs standalone sandboxes to Atlas cluster sandbox M0 Free tier replica set. It even worked and I had made some database operations on it. But now for some reason then there is a read permission error
Error creating bean with name 'mongobee' defined in class path resource [DatabaseConfiguration.class]: Invocation of init method failed; nested exception is com.mongodb.MongoQueryException: Query failed with error code 8000 and error message 'user is not allowed to do action [find] on [test.system.indexes]' on server ********-shard-00-01-mfwhq.mongodb.net:27017
You can see the full stack here https://pastebin.com/kaxcr7VS
I have searched high and low and all I could find is that M0 tier user doesn't have permissions to overwrite admin database which I am not doing.
Even now connection to Mlabs DB works fine but have this issue on Atlas DB M0 tier.
Mongo DB version : 3.4
Jars and It's version
name: 'mongobee', version: '0.10'
name: 'mongo-java-driver', version: '3.4.2'
#Neil Lunn
The userId I am using to connect is that of admin's and the connection read and write works through shell or Robo3T(mongo client)
After discussion with MongoDB support team, MongoDB 3.0 deprecates direct access to the system.indexes collection, which had previously been used to list all indexes in a database. Applications should use db.<COLLECTION>.getIndexes() instead.
From MongoDB Atlas docs it can be seen that they may forbid calls to system. collections:
Optionally, for the read and readWrite role, you can also specify a collection. If you do not specify a collection for read and readWrite, the role applies to all collections (excluding some system. collections) in the database.
From the stacktrace it's visible that MongoBee is trying to make this call, so it's now the library issue and it should be updated.
UPDATE:
In order to fix an issue until MongoBee has released new version:
Get the latest sources of MongoBee git clone git#github.com:mongobee/mongobee.git, cd mongobee
Fetch pull request git fetch origin pull/87/head:mongobee-atlas
Checkout git checkout mongobee-atlas
Install MongoBee jar mvn clean install
Get compiled jar from /target folder or local /.m2
Use the jar as a dependency on your project
Came across this issue this morning. Heres a quick and dirty monkey-patch for the problem:
package com.github.mongobee.dao;
import com.github.mongobee.changeset.ChangeEntry;
import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.DBObject;
import java.util.List;
import static com.github.mongobee.changeset.ChangeEntry.CHANGELOG_COLLECTION;
public class ChangeEntryIndexDao {
public void createRequiredUniqueIndex(DBCollection collection) {
collection.createIndex(new BasicDBObject()
.append(ChangeEntry.KEY_CHANGEID, 1)
.append(ChangeEntry.KEY_AUTHOR, 1),
new BasicDBObject().append("unique", true));
}
public DBObject findRequiredChangeAndAuthorIndex(DB db) {
DBCollection changelogCollection = db.getCollection(CHANGELOG_COLLECTION);
List<DBObject> indexes = changelogCollection.getIndexInfo();
if (indexes == null) return null;
for (DBObject index : indexes) {
BasicDBObject indexKeys = ((BasicDBObject) index.get("key"));
if (indexKeys != null && (indexKeys.get(ChangeEntry.KEY_CHANGEID) != null && indexKeys.get(ChangeEntry.KEY_AUTHOR) != null)) {
return index;
}
}
return null;
}
public boolean isUnique(DBObject index) {
Object unique = index.get("unique");
if (unique != null && unique instanceof Boolean) {
return (Boolean) unique;
} else {
return false;
}
}
public void dropIndex(DBCollection collection, DBObject index) {
collection.dropIndex(index.get("name").toString());
}
}
Caused by: java.lang.NoSuchMethodError: com.github.mongobee.dao.ChangeEntryIndexDao.<init>(Ljava/lang/String;)V
at com.github.mongobee.dao.ChangeEntryDao.<init>(ChangeEntryDao.java:34)
at com.github.mongobee.Mongobee.<init>(Mongobee.java:87)
at com.xxx.proj.config.DatabaseConfiguration.mongobee(DatabaseConfiguration.java:62)
at com.xxx.proj.config.DatabaseConfiguration$$EnhancerBySpringCGLIB$$4ae465a5.CGLIB$mongobee$1(<generated>)
at com.xxx.proj.config.DatabaseConfiguration$$EnhancerBySpringCGLIB$$4ae465a5$$FastClassBySpringCGLIB$$f202afb.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358)
at com.xxx.proj.config.DatabaseConfiguration$$EnhancerBySpringCGLIB$$4ae465a5.mongobee(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
... 22 common frames omitted
jhipster 5 must be using a different version, because i get that when implementing the above code. looks like its expecting a different version.
This access to system.indexes is an open issue in mongobee. The issue has been fixed in the project, but the project was abandoned before the fix was ever released.
Due to this project abandonment, two successor libraries have since been forked from mongobee which have fixed this issue: Mongock and mongobeeJ.
Switching your application's dependency from the mongobee library to one of these successor libraries will allow you to run mongobee database migrations on Atlas.
To summarize these libraries:
Mongock - Forked from mongobee in 2018. Actively maintained. Has evolved significantly from the original, including built-in support for Spring, Spring Boot, and both versions 3 & 4 of the Mongo Java driver.
mongobeeJ - Forked from mongobee in 2018. Five updated versions have been released. Minimal evolution from the original mongobee. Mongo Java Driver 4 support was implemented in August, 2020. This project was deprecated in August, 2020, with a recommendation from its creators to use a library such as Mongock instead.