Morphia, Embed Mongo and Spring. Address already in use - mongodb

I am trying use MongoDB, Morphia and Spring and test it, so I started use Embedded Mongo.
When I had only one DAO to persist I did not had any problem with my tests, however, in some cases I needed use more than one DAO, and in that cases my injected Datasore give me an problem: addr already in use.
My Spring Test Database Configuration is this:
#Configuration
public class DatabaseMockConfig {
private static final int PORT = 12345;
private MongodConfigBuilder configBuilder;
private MongodExecutable mongodExecutable;
private MongodProcess mongodProcess;
#Bean
#Scope("prototype")
public MongodExecutable getMongodExecutable() {
return this.mongodExecutable;
}
#Bean
#Scope("prototype")
public MongodProcess mongodProcess() {
return this.mongodProcess;
}
#Bean
public IMongodConfig getMongodConfig() throws UnknownHostException, IOException {
if (this.configBuilder == null) {
configBuilder = new MongodConfigBuilder().version(Version.Main.PRODUCTION).net(new Net(PORT, Network.localhostIsIPv6()));
}
return this.configBuilder.build();
}
#Autowired
#Bean
#Scope("prototype")
public Datastore datastore(IMongodConfig mongodConfig) throws IOException {
MongodStarter starter = MongodStarter.getDefaultInstance();
this.mongodExecutable = starter.prepare(mongodConfig);
this.mongodProcess = mongodExecutable.start();
MongoClient mongoClient = new MongoClient("localhost", PORT);
return new Morphia().createDatastore(mongoClient, "morphia");
}
#Autowired
#Bean
#Scope("prototype")
public EventDAO eventDAO(final Datastore datastore) {
return new EventDAO(datastore);
}
#Autowired
#Bean
#Scope("prototype")
public EditionDAO editionDAO(final Datastore datastore) {
return new EditionDAO(datastore);
}
}
And my DAO classes are similar to that
#Repository
public class EventDAO {
private final BasicDAO<Event, ObjectId> basicDAO;
#Autowired
public EventDAO(final Datastore datastore) {
this.basicDAO = new BasicDAO<>(Event.class, datastore);
}
...
}
My test class is similar to that:
#ContextConfiguration(classes = AppMockConfig.class)
#RunWith(SpringJUnit4ClassRunner.class)
public class EventDAOTest {
#Autowired
private EventDAO eventDAO;
#Autowired
private MongodExecutable mongodExecutable;
#Autowired
private MongodProcess mongodProcess;
#Rule
public ExpectedException expectedEx = ExpectedException.none();
#After
public void tearDown() {
this.mongodProcess.stop();
this.mongodExecutable.stop();
}
...
}
I use prototype scope to solve problem with singleton and make sure that my mock database is clean when I start my test, after that I stop mongod process and mongod executable.
However since I need use more than one DAO I receive that error:
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'editionDAO' defined in class br.com.mymusicapp.spring.DatabaseMockConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [org.mongodb.morphia.Datastore]: :
Error creating bean with name 'datastore' defined in class br.com.mymusicapp.spring.DatabaseMockConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.mongodb.morphia.Datastore]:
Factory method 'datastore' threw exception; nested exception is java.io.IOException: Could not start process: ERROR: listen(): bind() failed errno:98 Address already in use for socket: 0.0.0.0:12345
2015-01-04T01:05:04.128-0200 [initandlisten] ERROR: addr already in use
I know what the error means, I just do not know how can I design my Configuration to solve that. As last option I am considering install a localhost MongoDB just for tests, however I think could be a better solution

That is based on the embedded mongod by flapdoodle, right?
If you want to run multiple tests in parallel (could be changed via JUnit annotations, but it's probably faster in parallel), you cannot use a single, hardcoded port. Instead, let the embedded process select an available port automatically.

Related

Spring Cloud Contract for AMQP - autowiring issues with RabbitTemplate

I am trying to run a contract test for Spring AMQP using Spring Cloud Contract. However I am running into an issue with autowiring of the RabbitTemplate. In my Base Test class below, the autowired RabbitTemplate expects a ConnectionFactory with valid connection details (host and port of RabbitMQ brokers). Since contract tests are not expected to actually connect to the message broker, there is no host and port supplied to the connection factory in the test env.
I get the error
org.springframework.amqp.AmqpIOException: java.net.UnknownHostException: ${queue.hosts}: nodename nor servname provided, or not known
at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:71)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:476)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:614)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.createConnection(ConnectionFactoryUtils.java:240)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:1810)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1784)
at org.springframework.amqp.rabbit.core.RabbitTemplate.send(RabbitTemplate.java:864)
Is the RabbitTemplate supposed to be mocked? I tried it but even that isn't working.
Also tried passing mock ConnectionFactory to the actual RabbitTemplate but it tries to get a real connection from the mock factory.
How do you get around the problem that it tries to make an actual connection?
Base Test Class
#RunWith(SpringRunner.class)
#ContextConfiguration(classes = QueueConfiguration.class)
#AutoConfigureMessageVerifier
public class CreateNotificationBase {
private String message;
#Autowired
private RabbitTemplate rabbitTemplate;
#Before
public void setUp() {
client = new ChangeNotificationClient();
message = client.buildChangeNotificationMessage();
}
protected void onUserCreation() {
rabbitTemplate.send("change_notification_exchange",
RoutingKey.CHANGE_NOTIFICATION_KEY.getName(),
org.springframework.amqp.core.MessageBuilder.withBody(message.getBytes()).build());
}
}
Contract Definition
label: user_create
input:
triggeredBy: onUserCreation()
outputMessage:
sentTo: change_notification_exchange
body: ''' {"data":["....."jsonapi":{"version":"1.0"}} '''
Auto-generated Test
public class CreateTest extends CreateNotificationBase {
#Inject ContractVerifierMessaging contractVerifierMessaging;
#Inject ContractVerifierObjectMapper contractVerifierObjectMapper;
#Test
public void validate_create() throws Exception {
// when:
onUserCreation();
// then:
ContractVerifierMessage response = contractVerifierMessaging.receive("change_notification_exchange");
assertThat(response).isNotNull();
// and:
Object responseBody = (contractVerifierObjectMapper.writeValueAsString(response.getPayload()));
// assertions
;
}
}

Use multiple mongo DBs in same application for same model & same Repository

I need to implement Spring boot - MongoDb application where There are 2 mongo DBs which have exact same database name & collections. Based on User making a request, i need to choose whether to fetch data from DB1 or DB2 (only difference in mongo URI host - IP).
E.g. I need some way to create 2 mongoTemplates like mTempA & mTempB in my Repository & based on some condition, use either of the template to execute query as below:
#Repository
public class MyCustomRepository {
private Logger logger = LoggerFactory.getLogger(MyCustomRepository.class);
#Autowired
private MongoTemplateA mongoTemplateA;// Need to know if this is possible & how
#Autowired
private MongoTemplateB mongoTemplateB;// Need to know if this is possible & how
public List<MyModel> findByCriteria(MyRequest request) {
List<MyModel> result;
//Query query = <build query based on request>
if (request.getUserType().equals("A")) {
result = mongoTemplateA.find(query, MyModel.class);
} else {
result = mongoTemplateB.find(query, MyModel.class);
}
logger.debug("Result fetched with {} records", result.size());
return result;
}
}
I don't want to have 2 separate Repo (Class or Interfaces) or different models to be used. Just want to have 2 different mongoTemplates to be injected in single repo.
Is this possible? If yes, please give some example code.
I have followed below tutorial:
https://dzone.com/articles/multiple-mongodb-connectors-with-spring-boot
As rightly pointed out by #Lucia, below is how it can be done:
Have 2 different configuration placeholders
#Configuration
#EnableMongoRepositories(basePackages = "com.snk.repository", mongoTemplateRef = "mongoTemplateA")
public class MongoConfigA {
// Configuration class for DB 1 access
}
#Configuration
#EnableMongoRepositories(basePackages = "com.snk.repository", mongoTemplateRef = "mongoTemplateB")
public class MongoConfigB {
// Configuration class for DB 2 access
}
Get one class which will help in reading custom properties for mongo db properties in application.properties:
#ConfigurationProperties(prefix = "mongodb")
public class MultipleMongoProperties {
private MongoProperties adb = new MongoProperties();
private MongoProperties bdb = new MongoProperties();
public MongoProperties getAdb() {
return adb;
}
public MongoProperties getBdb() {
return bdb;
}
}
Add a configuration class to create mongoTemplates:
#Configuration
#EnableConfigurationProperties(MultipleMongoProperties.class)
public class MultipleMongoConfig {
#Autowired
private MultipleMongoProperties mongoProperties = new MultipleMongoProperties();
#Bean(name = "mongoTemplateA")
#Primary
public MongoTemplate mongoTemplateA() {
return new MongoTemplate(aDbFactory(this.mongoProperties.getAdb()));
}
#Bean(name = "mongoTemplateB")
public MongoTemplate mongoTemplateB() {
return new MongoTemplate(bDbFactory(this.mongoProperties.getBdb()));
}
#Bean
#Primary
public MongoDbFactory aDbFactory(final MongoProperties mongo) {
return new SimpleMongoDbFactory(new MongoClientURI(mongo.getUri()));
}
#Bean
public MongoDbFactory bDbFactory(final MongoProperties mongo) {
return new SimpleMongoDbFactory(new MongoClientURI(mongo.getUri()));
}
}
Add below decelerations to your service/repository:
#Autowired
#Qualifier("mongoTemplateA")
private MongoTemplate mongoTemplateA;
#Autowired
#Qualifier("MongoTemplateB")
private MongoTemplate MongoTemplateB;
Add below properties in your application.properties:
mongodb.adb.uri=mongodb://user:pass#myhost1:27017/adb
mongodb.bdb.uri=mongodb://user:pass#myhost2:27017/bdb
If you have mongo rplica set, URL can be set as:
mongodb.adb.uri=mongodb://user:pass#myhost1,myhost2,myhost13/adb?replicaSet=rsName
mongodb.bdb.uri=mongodb://user:pass#myhost1,myhost2,myhost13/bdb?replicaSet=rsName
Based on your logic, use either of the template.
Thought, there are few catches:
Notice the #Primary annotation, one bean needs to be marked as primary. I haven't find any solution if no template is marked primary.
If any of the mongo DB is down & application is started/restarted, application will not start/deploy. to avoid this, #Autowired needs to be changed to #Autowired(required = false).
If any of the mongo DB is down & application is already running, it automatically uses 2nd mongo BD (which is not down). So, even if you want to use A DB, if it's down, requests are processed with B DB & vice-versa.

How to use together spring-data-ldap and spring-security-ldap?

In a project using spring-security-ldap I need to perform some LDAP queries and I added spring-data-ldap. Suddenly I can't connect anymore to the embedded LDAP registry and I get:
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'run': Invocation of init method failed; nested exception is org.springframework.ldap.CommunicationException: localhost:8389; nested exception is javax.naming.CommunicationException: localhost:8389 [Root exception is java.net.ConnectException: Connexion refusée (Connection refused)]
Here is the security config which work as expected:
public class SpringSecurityConfig extends WebSecurityConfigurerAdapter
{
#Override
protected void configure(HttpSecurity http) throws Exception
{
http.authorizeRequests().antMatchers("/admins").hasRole("ADMINS")
.antMatchers("/users").hasRole("USERS")
.anyRequest().fullyAuthenticated()
.and()
.httpBasic();
}
#Override
public void configure(AuthenticationManagerBuilder auth) throws Exception
{
auth
.ldapAuthentication()
.userDnPatterns("uid={0},ou=people")
.userSearchBase("ou=people")
.userSearchFilter("uid={0}")
.groupSearchBase("ou=groups")
.groupSearchFilter("uniqueMember={0}")
.contextSource(contextSource())
.passwordCompare()
.passwordAttribute("userPassword");
}
#Bean
public DefaultSpringSecurityContextSource contextSource()
{
log.info("*** SpringSecurityConfig.contextSource(): Inside contextSource");
DefaultSpringSecurityContextSource contextSource = new DefaultSpringSecurityContextSource(
Arrays.asList("ldap://localhost:8389/"), "dc=toto,dc=com");
contextSource.afterPropertiesSet();
return contextSource;
}
}
Now, if I want to use spring-data-ldap, I add this:
#Repository
public interface MyLdapRepository extends LdapRepository<LdapUser>
{
}
#Entry(base="ou=users", objectClasses = {"person", "inetOrgPerson", "top"})
#NoArgsConstructor
#AllArgsConstructor
#Getter
#Setter
public class LdapUser
{
#Id
private Name id;
#Attribute(name = "uid")
private String uid;
#Attribute(name = "cn")
private String cn;
}
And I try to make some queries, for example:
#SpringBootApplication
#Slf4j
public class Run extends SpringBootServletInitializer
{
#Autowired
private RdfLdapRepository rdfLdapRespository;
public static void main(String[] args)
{
SpringApplication.run(Run.class, args);
}
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder builder)
{
return builder.sources(Run.class);
}
#PostConstruct
public void setup()
{
log.info("### setup(): the LDIF file has been loaded");
Iterable<LdapUser> users = rdfLdapRespository.findAll();
users.forEach(user -> log.info("\"### setup(): names {}", user.getUid()));
}
}
I get Connection Refused. Commenting out the setup() method, everything works as expected again. I suspect some missmatch between the Ldaptemplate used by spring-data-ldap and the DefaultSpringSecurityContextSource in the security config.
Does anyone know what might be the problem here ?
Many thanks in advance;
Kind regards,
Nicolas
Problem solved. Everything was due to the fact that another instance of the Spring embedded LDAP directory server was running in the same Tomcat container. I'm not sure how this interacted with the spring-data-ldap and why it appeared only in this context but using the following command was capital as it helped me understand the issue:
> lsof -i:8389
This way I noticed that another LDAP embedded server was active and I understood why (consequence of repetitive deployment on the same Tomcat container).

Using embedded MongoDB in Spring JUnit #WebMvcTest

I'm currently using MongoDB in my Spring application. Since I added Mongo my endpoint tests no longer work due to the following error:
No qualifying bean of type 'xxx' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {}
The repository which I Autowire in a controller is as follows:
private final RuleRepository ruleRepository;
#Autowired
public TestController(RuleRepository ruleRepository) {
this.ruleRepository = ruleRepository;
}
I assume that this has to do with Mongo and the fact that I currently use AutoConfiguration for it. For the test I added the Flapdoodle Embed Mongo dependency, which seems to be used for testing purposes in a lot of examples, to my pom.xml with the scope set to test:
<dependency>
<groupId>de.flapdoodle.embed</groupId>
<artifactId>de.flapdoodle.embed.mongo</artifactId>
<version>2.0.3</version>
<scope>test</scope>
</dependency>
My test class looks like this:
RunWith(SpringRunner.class)
#WebMvcTest(value = RouteController.class, secure = false)
#ActiveProfiles("test")
public class TestControllerEndpointTests {
#Autowired
private MockMvc mockMvc;
#Autowired
private RuleRepository ruleRepository;
#Before
public void setupTests() {
//Setup for the tests
}
//Actual tests
}
I also created a Configuration class for the Mongo Test Database, but I don't know how to correctly register it:
#Configuration
#Profile("test")
public class TestMongoConfig {
#Autowired
private MongoProperties properties;
#Autowired(required = false)
private MongoClientOptions options;
#Bean(destroyMethod = "close")
public Mongo mongo(MongodProcess mongodProcess) throws IOException {
Net net = mongodProcess.getConfig().net();
return new MongoClient(net.getServerAddress().getHostName(), net.getPort());
}
#Bean(destroyMethod = "stop")
public MongodProcess mongodProcess(MongodExecutable mongodExecutable) throws IOException {
return mongodExecutable.start();
}
#Bean(destroyMethod = "stop")
public MongodExecutable mongodExecutable(MongodStarter mongodStarter, IMongodConfig iMongodConfig) throws IOException {
return mongodStarter.prepare(iMongodConfig);
}
#Bean
public IMongodConfig mongodConfig() throws IOException {
return new MongodConfigBuilder().version(Version.Main.PRODUCTION).build();
}
#Bean
public MongodStarter mongodStarter() {
return MongodStarter.getDefaultInstance();
}
}
How do I get an endpoint test that is annotated with #WebMvcTest to use the embedded Mongo database?
After banging our heads for a while we found the #AutoConfigureDataMongo annotation.
import org.springframework.boot.test.autoconfigure.data.mongo.AutoConfigureDataMongo;
#RunWith(SpringRunner.class)
#WebMvcTest(value = SampleController.class, secure = false)
#AutoConfigureDataMongo
public class SampleControllerTest {
Just annotate your controller with it and you should see org.springframework.boot.autoconfigure.mongo.embedded.EmbeddedMongo logs when running this controller tests.

Spring Data MongoTemplate not throwing DataAccessException

I am trying to learn MongoDB and in the same time write a simple REST application using Spring framework.
I have a simple model:
#Document
public class Permission extends documentBase{
#Indexed(unique = true)
private String name;
public Permission(String name) {
this.name = name;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
Then I have a simple DAO:
#Repository
#Transactional
#Profile({"production","repositoryTest","mongoIntegrationTest"})
public class DaoImpl implements DAO {
#Autowired
protected MongoTemplate mongoTemplate;
public <T> T addObject(T object) {
mongoTemplate.insert(object);
return object;
}
The I have my integration tests:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = { "classpath:mvc-dispatcher-servlet.xml", classpath:IntegrationContext.xml"},loader = TestXmlContextLoader.class)
#ActiveProfiles("mongoIntegrationTest")
public class RepositoryIntegrationTest extends AccountTestBase{
#Autowired DAO repository;
#Autowired WebApplicationContext wac;
#Test
public void AddPermission() {
Permission permission_1 = new Permission("test");
Permission permission_2 = new Permission("test");
repository.addObject(permission_1);
repository.addObject(permission_2);
}
}
My configuration:
<!-- MongoDB host -->
<mongo:mongo host="${mongo.host.name}" port="${mongo.host.port}"/>
<!-- Template for performing MongoDB operations -->
<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate"
c:mongo-ref="mongo" c:databaseName="${mongo.db.name}"/>
I am expecting that, on adding "permission_2" their would be a exception thrown from MongoDB, which would be translated by Spring,, and catched as a DataAccessException in the DAO.
Looking at the log files from MongoDb I can see that a duplicated exception is thrown but it never reaches my DAO.
So,, I guess I am doing something wrong,,, but at the moment,, I am blind to my own misstakes.
//lg
Make sure you configure the WriteConcern of the MongoTemplate to something non-default (e.g. WriteConcern.SAFE). By default MongoDB is in fire-and-forget mode and does not throw exceptions on index violations or server errors in general.
Still struggling with this.
Finnally I succeded to get the exeption translation working. MongoDb throws a exception which is translated to Spring Data exception.
Now I am stuck with another problem.
My DAO shown above has also the following code:
#ExceptionHandler(DataAccessException.class)
public void handleDataAccessException(DataAccessException ex) {
// For debug only
DataAccessException test = ex;
test.printStackTrace();
}
I was expecting this code to catch the exception thrown,, but this is not the case.
Why not?
//lasse