I need to initialize a Mongo DB with some script file like Spring do with JPA and import.sql file.. but how?
Can someone help me?
You could use something similar that's done by mongeez. This is basically a starter for spring-boot that runs scripts before spring-data-mongodb beans are initialized.
You could tag along and leverage spring-boot's initialization lifecycle, where after wiring the beans, it executes all CommandLineRunner beans.
#SpringBootApplication
public class YourApplication {
final Logger logger = LoggerFactory.getLogger(getClass());
#Autowired
private MongoRepository repo;
#Bean
CommandLineRunner preLoadMongo() throws Exception {
return args -> {
//repo.doSOmethingInMongoDB
}
public static void main(String[] args) {
SpringApplication.run(YourApplication.class, args);
}
Related
I have a small service on SpringBoot and Mongodb as a DB.
I need to be able create a small collection with one document ( very basic: id, name, status) on startup. An analog of sql create table if not exists, but for mongo. How do I do that?
I tried to initialize values in the document attributes, but it didn't help.
Currently, collection and the document appear only if I use API to add it.
You may want to use something like ApplicationRunner or CommandLineRunner which can be defined as a bean.
Example:
#SpringBootApplication
public class MyApplication {
public static void main(String[] args) {
SpringApplication.run(MyApplication .class, args);
}
#Bean
public CommandLineRunner initialize(MyRepository myRepository) {
return args -> {
// Insert elements into myRepository
};
}
}
Both CommandLineRunner and ApplicationRunner are functional interfaces, so we can use a lambda for them. Spring Boot will execute them at the startup of the application.
You can leverage the spring internal event mechanism.
When your application is ready, spring triggers the event ApplicationReadyEvent
You can listen to this event and init your collection:
#Component
public class DataInit implements ApplicationListener<ApplicationReadyEvent> {
private final MyRepository myRepository;
public DataInit(MyRepository myRepository) {
this.myRepository = myRepository;
}
#Override
public void onApplicationEvent(ApplicationReadyEvent event) {
// init data
}
}
We have a spring boot rest api (spring boot 2.3.0.RELEASE) that uses spring cloud sleuth (version 2.2.3.RELEASE).
At some point, we use the trace id from spring sleuth as data. The trace id is fetched by autowiring the Tracing bean and then accessing the current span. Lets say we defined a bean SimpleCorrelationBean with:
#Autowired
private Tracer tracer;
public String getCorrelationId() {
return tracer.currentSpan().context().traceIdString();
}
This seem to work perfectly when running the spring boot application, but when we try to access the tracer.currentSpan() in the unit tests, this is null. It looks like spring cloud sleuth is not creating any span while running tests..
I think it has something to do with the application context that is set up during the unit test, but I don't know how to enable spring cloud sleuth for the test application context.
Below is a simple test class where the error occurs in simpleTest1. In simpleTest2, no error occurs.
simpleTest1 errors because tracer.currentSpan() is null
#ExtendWith({ RestDocumentationExtension.class, SpringExtension.class })
#SpringBootTest(classes = MusicService.class)
#WebAppConfiguration
#ActiveProfiles("unit-test")
#ComponentScan(basePackageClasses = datacast2.data.JpaConfig.class)
public class SimpleTest {
private static final Logger logger = LoggerFactory.getLogger(SimpleTest.class);
#Autowired
private WebApplicationContext context;
private MockMvc mockMvc;
#Autowired
private FilterChainProxy springSecurityFilterChain;
#Autowired
private SimpleCorrelationBean simpleCorrelationBean;
#Autowired
private Tracer tracer;
#BeforeEach
public void setup(RestDocumentationContextProvider restDocumentation) throws Exception {
this.mockMvc = MockMvcBuilders.webAppContextSetup(this.context)
.apply(documentationConfiguration(restDocumentation))
.addFilter(springSecurityFilterChain).build();
}
#Test
public void simpleTest1() throws Exception {
try {
String correlationId = simpleCorrelationBean.getCorrelationId();
}catch(Exception e) {
logger.error("This seem to fail.", e);
}
}
#Test
public void simpleTest2() throws Exception {
//It looks like spring cloud sleuth is not creating a span, so we create one ourselfs
Span newSpan = this.tracer.nextSpan().name("simpleTest2");
try (Tracer.SpanInScope ws = this.tracer.withSpanInScope(newSpan.start())) {
String correlationId = simpleCorrelationBean.getCorrelationId();
}
finally {
newSpan.finish();
}
}
}
The question is: how to enable spring cloud sleuth for a mockMvc during unit tests?
The issue here is that MockMvc is created manually instead of relying on autoconfiguration. In this particular case custom configuration of MockMvc could be necessary. However, at least for my version of Spring Boot (2.7.6), there is no need to manually configure MockMvc, even though I use Spring Security and Spring Security Test. I couldn't figure out how to enable tracing when manually configuring MockMvc though.
I am new to Spring Batch. I have configured my job with inmemoryrepository. But still, it seems it is using DB to persist job Metadata.
My spring batch Configuration is :
#Configuration
public class BatchConfiguration {
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
private JobBuilderFactory jobBuilder;
#Bean
public JobLauncher jobLauncher() throws Exception {
SimpleJobLauncher job =new SimpleJobLauncher();
job.setJobRepository(getJobRepo());
job.afterPropertiesSet();
return job;
}
#Bean
public PlatformTransactionManager getTransactionManager() {
return new ResourcelessTransactionManager();
}
#Bean
public JobRepository getJobRepo() throws Exception {
return new MapJobRepositoryFactoryBean(getTransactionManager()).getObject();
}
#Bean
public Step step1(JdbcBatchItemWriter<Person> writer) throws Exception {
return stepBuilderFactory.get("step1")
.<Person, Person> chunk(10)
.reader(reader())
.processor(processor())
.writer(writer).repository(getJobRepo())
.build();
}
#Bean
public Job job( #Qualifier("step1") Step step1) throws Exception {
return jobBuilder.get("myJob").start(step1).repository(getJobRepo()).build();
}
}
How to resolve above issue?
If you are using Sprint boot
a simple property in your application.properties will solve the issue
spring.batch.initialize-schema=ALWAYS
For a non-Spring Boot setup:This error shows up when a datasource bean is declared in the batch configuration. To workaround the problem I added an embedded datasource, since I didn't want to create those tables in the application database:
#Bean
public DataSource mysqlDataSource() {
// create your application datasource here
}
#Bean
#Primary
public DataSource batchEmbeddedDatasource() {
// in memory datasource required by spring batch
EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
return builder.setType(EmbeddedDatabaseType.H2)
.addScript("classpath:schema-drop-h2.sql")
.addScript("classpath:schema-h2.sql")
.build();
}
The initialization scripts can be found inside the spring-batch-core-xxx.jar under org.springframework.batch.core package.Note I used an in-memory database but the solution is valid also for other database systems.
Those who face the same problem with MySql database in CentOS(Most Unix based systems).
Table names are case-sensitive in Linux. Setting lower_case_table_names=1 has solved the problem.
Find official document here
For those using versions greater then spring-boot 2.5 this worked inside of application.properties
spring.batch.jdbc.initialize-schema = ALWAYS
This solved my case:
spring.batch.jdbc.initialize-schema=ALWAYS
I am trying to use my JPA repositories in order to save test data into h2 to be then used by a spring batch integration test.
Here is my integration test:
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.NONE, classes = Batch.class)
public class MessageDigestMailingStepIT extends AbstractBatchIntegrationTest {
#Autowired
#Qualifier("messagesDigestMailingJob")
private Job messagesDigestMailingJob;
#Autowired
private JobLauncher jobLauncher;
#Autowired
private JobRepository jobRepository;
#Autowired
private UserAccountRepository userAccountRepository;
#Autowired
private MessageRepository messageRepository;
private JobLauncherTestUtils jobLauncherTestUtils;
#Before
public void setUp() {
this.jobLauncherTestUtils = new JobLauncherTestUtils();
this.jobLauncherTestUtils.setJobLauncher(jobLauncher);
this.jobLauncherTestUtils.setJobRepository(jobRepository);
this.jobLauncherTestUtils.setJob(messagesDigestMailingJob);
}
#Test
#Transactional
public void shouldSendMessageDigestAndUpdateNotificationSent() {
UserAccount userAccount = DomainFactory.createUserAccount("me#example.com");
userAccountRepository.save(userAccount);
JobParameters jobParameters = new JobParametersBuilder().addDate("execution_date", new Date()).toJobParameters();
jobLauncherTestUtils.launchStep("messagesDigestMailingStep", jobParameters);
//Assertions
}
}
Notice the #Transactional on the test method. Unfortunately Spring batch uses its own transactions and my use of #Transactional clashes with spring batch transactions.
Here is the error message I get:
java.lang.IllegalStateException: Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove #Transactional annotations from client).
Can someone please advise how to insert test data to be available for a spring batch integration test?
edit: For good measure, here is the definition of the AbstractBatchIntegrationTest class:
#AutoConfigureTestEntityManager
#AutoConfigureJson
#AutoConfigureJsonTesters
#RunWith(SpringJUnit4ClassRunner.class)
#ActiveProfiles(Profiles.TEST)
#ComponentScan(basePackages = {"com.bignibou.it.configuration", "com.bignibou.configuration"})
public abstract class AbstractBatchIntegrationTest {
}
edit: I have decided to rely only on the #Sql annotation as follows:
#Sql(scripts = "insert_message.sql", executionPhase = Sql.ExecutionPhase.BEFORE_TEST_METHOD)
#Sql(scripts = "clean_database.sql", executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD)
#Test
public void shouldSendMessageDigestAndUpdateNotificationSent() {
...
Remove #Transactional from the test so that the UserAccount gets immediately persisted to the database. Then use #Sql with ExecutionPhase.AFTER_TEST_METHOD to execute a clean-up script (or inlined statement) to manually undo the changes performed during the test.
I am trying use MongoDB, Morphia and Spring and test it, so I started use Embedded Mongo.
When I had only one DAO to persist I did not had any problem with my tests, however, in some cases I needed use more than one DAO, and in that cases my injected Datasore give me an problem: addr already in use.
My Spring Test Database Configuration is this:
#Configuration
public class DatabaseMockConfig {
private static final int PORT = 12345;
private MongodConfigBuilder configBuilder;
private MongodExecutable mongodExecutable;
private MongodProcess mongodProcess;
#Bean
#Scope("prototype")
public MongodExecutable getMongodExecutable() {
return this.mongodExecutable;
}
#Bean
#Scope("prototype")
public MongodProcess mongodProcess() {
return this.mongodProcess;
}
#Bean
public IMongodConfig getMongodConfig() throws UnknownHostException, IOException {
if (this.configBuilder == null) {
configBuilder = new MongodConfigBuilder().version(Version.Main.PRODUCTION).net(new Net(PORT, Network.localhostIsIPv6()));
}
return this.configBuilder.build();
}
#Autowired
#Bean
#Scope("prototype")
public Datastore datastore(IMongodConfig mongodConfig) throws IOException {
MongodStarter starter = MongodStarter.getDefaultInstance();
this.mongodExecutable = starter.prepare(mongodConfig);
this.mongodProcess = mongodExecutable.start();
MongoClient mongoClient = new MongoClient("localhost", PORT);
return new Morphia().createDatastore(mongoClient, "morphia");
}
#Autowired
#Bean
#Scope("prototype")
public EventDAO eventDAO(final Datastore datastore) {
return new EventDAO(datastore);
}
#Autowired
#Bean
#Scope("prototype")
public EditionDAO editionDAO(final Datastore datastore) {
return new EditionDAO(datastore);
}
}
And my DAO classes are similar to that
#Repository
public class EventDAO {
private final BasicDAO<Event, ObjectId> basicDAO;
#Autowired
public EventDAO(final Datastore datastore) {
this.basicDAO = new BasicDAO<>(Event.class, datastore);
}
...
}
My test class is similar to that:
#ContextConfiguration(classes = AppMockConfig.class)
#RunWith(SpringJUnit4ClassRunner.class)
public class EventDAOTest {
#Autowired
private EventDAO eventDAO;
#Autowired
private MongodExecutable mongodExecutable;
#Autowired
private MongodProcess mongodProcess;
#Rule
public ExpectedException expectedEx = ExpectedException.none();
#After
public void tearDown() {
this.mongodProcess.stop();
this.mongodExecutable.stop();
}
...
}
I use prototype scope to solve problem with singleton and make sure that my mock database is clean when I start my test, after that I stop mongod process and mongod executable.
However since I need use more than one DAO I receive that error:
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'editionDAO' defined in class br.com.mymusicapp.spring.DatabaseMockConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [org.mongodb.morphia.Datastore]: :
Error creating bean with name 'datastore' defined in class br.com.mymusicapp.spring.DatabaseMockConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.mongodb.morphia.Datastore]:
Factory method 'datastore' threw exception; nested exception is java.io.IOException: Could not start process: ERROR: listen(): bind() failed errno:98 Address already in use for socket: 0.0.0.0:12345
2015-01-04T01:05:04.128-0200 [initandlisten] ERROR: addr already in use
I know what the error means, I just do not know how can I design my Configuration to solve that. As last option I am considering install a localhost MongoDB just for tests, however I think could be a better solution
That is based on the embedded mongod by flapdoodle, right?
If you want to run multiple tests in parallel (could be changed via JUnit annotations, but it's probably faster in parallel), you cannot use a single, hardcoded port. Instead, let the embedded process select an available port automatically.