I'm using Fongo not only for unit tests but also for integration tests so I would like to initialize Fongo with some collections, is that possible?
This is my java config (based on Oliver G. answer):
#EnableAutoConfiguration(exclude = {
EmbeddedMongoAutoConfiguration.class,
MongoAutoConfiguration.class,
MongoDataAutoConfiguration.class
})
#Configuration
#ComponentScan(basePackages = { "com.foo" },
excludeFilters = { #ComponentScan.Filter(classes = { SpringBootApplication.class })
})
public class ConfigServerWithFongoConfiguration extends AbstractFongoBaseConfiguration {
private static final Logger log = LoggerFactory.getLogger(ConfigServerWithFongoConfiguration.class);
#Autowired
ResourcePatternResolver resourceResolver;
#Bean
public Jackson2RepositoryPopulatorFactoryBean repositoryPopulator() {
Jackson2RepositoryPopulatorFactoryBean factory = new Jackson2RepositoryPopulatorFactoryBean();
try {
factory.setResources(resourceResolver.getResources("classpath:static/collections/*.json"));
} catch (IOException e) {
log.error("Could not load data", e);
}
return factory;
}
}
When I run my IT tests, on the log it appears Reading resource: file *.json but the tests fails because they retrieve nothing (null) from Fongo database.
Tests are annotated with:
#RunWith(SpringRunner.class)
#SpringBootTest(classes={ConfigServerWithFongoConfiguration.class})
#AutoConfigureMockMvc
#TestPropertySource(properties = {"spring.data.mongodb.database=fake"})
#DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS)
Lol, I feel so stupid right now. Was format issue. JSON collections must be formated like this:
[
{/*doc1*/},
{/*doc2*/},
{/*doc3*/}
]
I was missing the [] and comma separated documents.
Related
I was updating spring boot from 2.5.1 to 2.7 together with the r2dbc and postgres dependencies. I did no change the application.yml or test setup. Before the update my repository tests run fine with testcontainers, but now I see this exception which is thrown by a #AfterEach that tries to clean the DB:
2022-05-29 10:04:52.447 INFO 16673 --- [tainers-r2dbc-0] 🐳 [postgres:13.2] : Container postgres:13.2 started in PT1.244757S
Failed to execute SQL script statement #1 of InputStream resource [resource loaded through InputStream]: DROP SCHEMA public CASCADE; nested exception is io.r2dbc.postgresql.client.ReactorNettyClient$PostgresConnectionClosedException: Cannot exchange messages because the connection is closed
org.springframework.r2dbc.connection.init.ScriptStatementFailedException: Failed to execute SQL script statement #1 of InputStream resource [resource loaded through InputStream]: DROP SCHEMA public CASCADE; nested exception is io.r2dbc.postgresql.client.ReactorNettyClient$PostgresConnectionClosedException: Cannot exchange messages because the connection is closed
at org.springframework.r2dbc.connection.init.ScriptUtils.lambda$runStatement$9(ScriptUtils.java:571)
This is my abstract RepositoryTest:
#DataR2dbcTest
#ActiveProfiles("test")
internal abstract class RepositoryTest {
#Autowired
protected lateinit var connectionFactory: ConnectionFactory
#AfterEach
fun clean() {
runSql(
"""
DROP SCHEMA public CASCADE;
CREATE SCHEMA public;
"""
)
}
protected fun runSql(sql: String) {
runScript(InputStreamResource(sql.byteInputStream()))
}
protected fun runScript(sqlScript: Resource) {
runBlocking {
val connection = connectionFactory.create().awaitFirst()
ScriptUtils.executeSqlScript(connection, sqlScript)
.block() // <---- throws the said exception, but it worked before the update.
}
}
}
My actual test looks like this:
internal class MyRepoTest : RepositoryTest() {
#Autowired
private lateinit var myRepo: MyRepository
#Test
fun someTest() {
val userId = 3429L
val myEntities = ...
runBlocking { myRepo.saveAll(myEntities).collect() }
val result = myRepo.findAllByUserId(userId).asFlux()
StepVerifier.create(result)
.expectNextMatches { it.userId == userId}
.expectNextMatches { it.userId == userId}
.verifyComplete()
}
}
I guess the way I try to execute the SQL commands is not fine, how should I do it?
val connection = connectionFactory.create().awaitFirst()
ScriptUtils.executeSqlScript(connection, sqlScript)
.block() // <---- throws the said exception, but it worked before the update.
EDIT
I figured out that using ResourceDatabasePopulator works fine:
7protected fun runScript(sqlScript: Resource) {
runBlocking {
ResourceDatabasePopulator(sqlScript).populate(connectionFactory).block()
}
}
But I still would like to understand why the original implementation now fails.
Using Spring Data MongoDB with MongoRepository. I have this bean
#Bean
public Jackson2RepositoryPopulatorFactoryBean repositoryPopulator() {
Jackson2RepositoryPopulatorFactoryBean factory = new Jackson2RepositoryPopulatorFactoryBean();
try {
factory.setResources(resourceResolver.getResources("classpath:static/collections/*.json"));
} catch (IOException e) {
log.error("Could not load data", e);
}
return factory;
}
which just works fine with fongo (db is dropped at the end of a test run) but not with real mongo. If I leave the bean as it is and I switch to real mongo instance, then I get my data base populated but only the first run, if I re-run the project (+tests) then it fails because it's already populated (getting DuplicateKeyException).
How do I populate only on the case the repositories are empty?
Consider using data migration tools like Mongobee. This is basically Liquibase/Flyway for MongoDB.
#Bean
public Jackson2RepositoryPopulatorFactoryBean repositoryPopulator() throws Exception {
Jackson2RepositoryPopulatorFactoryBean factory = new Jackson2RepositoryPopulatorFactoryBean();
try {
Resource[] resources = resourceResolver.getResources("classpath:static/collections/*.json");
//resources to list so I can add only the necessary resources
List<Resource> resourcesToFill = new ArrayList<>();
for (Resource r : resources) {
String collection = r.getFilename().substring(0, r.getFilename().length() - 5);
if (!mongoTemplate().collectionExists(collection))
resourcesToFill.add(r);
}
//back to Array...
resources = new Resource[resourcesToFill.size()];
for(int i=0; i<resources.length; i++)
resources[i] = resourcesToFill.get(i);
factory.setResources(resources); // <-- the reason of this shitty code, why the hell use Array?
} catch (IOException e) {
log.error("Could not load data", e);
}
return factory;
}
I try to integrate CXF WS to jhipster stack, so avoid xml configuration.
The first class to config service
#EnableWs
#Configuration
#AutoConfigureAfter(WebConfigurer.class)
public class WebServiceConfig extends WsConfigurerAdapter {
#Bean
public ServletRegistrationBean dispatcherServlet() {
CXFServlet cxfServlet = new CXFServlet();
return new ServletRegistrationBean(cxfServlet, "/soap/*");
}
#Bean(name = "cxf")
public SpringBus springBus() {
return new SpringBus();
}
#Bean
public Hello hello() {
return new HelloPortImpl();
}
#Bean
public Endpoint endpoint() {
EndpointImpl endpoint = new EndpointImpl(springBus(), hello());
endpoint.publish("/hello");
return endpoint;
}
}
The second file :
#WebService(targetNamespace = "http://service.ws.sample/", name = "Hello")
public interface Hello {
#WebResult(name = "return", targetNamespace = "")
#RequestWrapper(localName = "sayHello", targetNamespace = "http://service.ws.sample/", className = "com.orange.api.rfid.tacites.proxyauth.web.restWS.SayHello")
#WebMethod(action = "urn:SayHello")
#ResponseWrapper(localName = "sayHelloResponse", targetNamespace = "http://service.ws.sample/", className = "com.orange.api.rfid.tacites.proxyauth.web.restWS.SayHelloResponse")
public java.lang.String sayHello(
#WebParam(name = "myname", targetNamespace = "")
java.lang.String myname
);
}
The third file
#javax.jws.WebService(
serviceName = "HelloService",
portName = "HelloPort",
targetNamespace = "http://service.ws.sample/",
endpointInterface = "com.orange.api.rfid.tacites.proxyauth.web.restWS.Hello")
public class HelloPortImpl implements Hello {
private static final Logger LOG = Logger.getLogger(HelloPortImpl.class.getName());
public java.lang.String sayHello(java.lang.String myname) {
LOG.info("Executing operation sayHello" + myname);
try {
return "Welcome to CXF Spring boot " + myname + "!!!";
} catch (java.lang.Exception ex) {
ex.printStackTrace();
throw new RuntimeException(ex);
}
}
}
In my logs when start spring boot, i have this line:
[DEBUG] com.sun.xml.bind.v2.schemagen.XmlSchemaGenerator - Wrigin XML Schema for com.sun.xml.bind.v2.schemagen.XmlSchemaGenerator#6a08fd54[http://service.ws.sample/=com.sun.xml.bind.v2.schemagen.XmlSchemaGenerator$Namespace#76617add]
com.sun.xml.bind.v2.util.StackRecorder: null
at com.sun.xml.bind.v2.schemagen.XmlSchemaGenerator.write(XmlSchemaGenerator.java:441) [jaxb-impl-2.2.jar:2.2]
The problem is Jhipster index.html is not found and in http://localhost:8080/soap/hello i have No binding operation info while invoking unknown method with params unknown
I think the cxf servlet kill first one servlet, how to configure to coexist both?
Regards
Try renaming your WebServiceConfig.dispatcherServlet method to something else as there is probably a bean with this name defined by Spring Boot that you're overriding.
To solve the issue, i add to WebConfigurer.java:
/**
* Initialize cxf - ws
*/
private void initCxf(ServletContext servletContext) {
log.debug("Initialize cxf - ws");
ServletRegistration.Dynamic cxfServlet = servletContext.addServlet("CxfWS", new CXFServlet());
cxfServlet.addMapping("/soap/*");
cxfServlet.setLoadOnStartup(3);
}
I've got this error when not specifying the proper wsdl name. Verify that the Soap Service instance uses a proper path to wsdl.
I have the following two bean definitions for Spring Data Redis. I cant seem to find the relevant documentation to determine the scopes(singleton,request or session) of these beans for a web app.
#Bean
public StringRedisTemplate redisTemplate() throws Exception {
StringRedisTemplate redisTemplate = new StringRedisTemplate();
redisTemplate.setConnectionFactory(jedisConnectionFactory());
return redisTemplate;
}
#Bean
public StringRedisConnection stringRedisConnection() throws Exception {
return new DefaultStringRedisConnection(redisTemplate().getConnectionFactory().getConnection());
}
Thanks to #Christoph Strobl recommendation here is the implementation Iam currently using
public List<String> testAutoComplete(String key,String query, int limitCount){
StringRedisSerializer serializer = new StringRedisSerializer();
RedisZSetCommands.Range range = Range.range();
range.gt(query);
RedisZSetCommands.Limit limit = new RedisZSetCommands.Limit();
limit.count(limitCount);
return template.execute(new RedisCallback< List<String>>() {
public List<String> doInRedis(RedisConnection connection) {
Set<byte[]> results = connection.zRangeByLex(serializer.serialize(key), range,limit);
List<String> resultAsString = new ArrayList<String>();
for(byte[] result : results){
resultAsString.add(serializer.deserialize(result));
}
return resultAsString;
}
},false);
}
I am looking for a way to (re)deploy an exploded bundle (meaning not jarred but in a folder) to a running Apache Felix OSGi container from within Eclipse, preferably using a launch task.
I found this question, which has an answer that comes close but it depends on typing commands into a Gogo shell, which is not convenient for long-term development use. I'd like to use Eclipse's launch task mechanism for this, but if there are alternatives that are equally fast and convenient I am open to that as well.
Now I think that if I can fire Gogo shell commands from an Eclipse launch tasks, that would be a solution, but I also can't get my head around how to do that. I presume I need the Remote Shell bundle for that right?
I am starting to think about writing a telnet client in Java that can connect to the Remote Shell bundle and execute Gogo commands in an automated fashion. I have seen some example of that already which I can modify to suit my needs... However I am getting a 'reinventing the wheel' kind of feeling from that. Surely there is a better way?
Some background to help you understand what I am doing:
I have set up an Eclipse 'OSGiContainer' project which basically contains the Apache Felix jar and the third party bundles I want to deploy (like Gogo shell), similar to the project setup described here. Then I created a second 'MyBundle' project that contains my bundle. I want to start the OSGi container by launching the OSGiContainer project, and then just develop on my bundle and test my changes by launching the MyBundle project into the OSGiContainer that I just want to keep running the whole time during development.
Project layout:
OSGiContainer
bin (contains felix jar)
bundles (third party bundles)
conf (Felix' config.properties file)
MyBundle
src
target
classes
I am then able to deploy my bundle to the OSGi container by invoking these commands on the Gogo shell:
install reference:file:../MyBundle/target/classes
start <bundleId>
To re-deploy, I invoke these commands:
stop <bundleId>
uninstall <bundleId>
install reference:file:../MyBundle/target/classes
start <bundleId>
You can imagine having to invoke 4 commands on the shell each time is not that much fun... So even if you can give me a way to boil this down to less commands to type it would be a great improvement already.
UPDATE
I hacked around a bit and came up with the class below. It's an adaptation of the telnet example with some small changes and a main method with the necessary commands to uninstall a bundle and then re-install and start it. The path to the bundle should be given as an argument to the program and would look like:
reference:file:../MyBundle/target/classes
I still very much welcome answers to this question, as I don't really like this solution at all. I have however verified that this works:
import java.io.IOException;
import java.io.InputStream;
import java.io.PrintStream;
import java.net.SocketException;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.LinkedBlockingQueue;
import org.apache.commons.net.telnet.TelnetClient;
public class GogoDeployer {
static class Responder extends Thread {
private StringBuilder builder = new StringBuilder();
private final GogoDeployer checker;
private CountDownLatch latch;
private String waitFor = null;
private boolean isKeepRunning = true;
Responder(GogoDeployer checker) {
this.checker = checker;
}
boolean foundWaitFor(String waitFor) {
return builder.toString().contains(waitFor);
}
public synchronized String getAndClearBuffer() {
String result = builder.toString();
builder = new StringBuilder();
return result;
}
#Override
public void run() {
while (isKeepRunning) {
String s;
try {
s = checker.messageQueue.take();
} catch (InterruptedException e) {
break;
}
synchronized (Responder.class) {
builder.append(s);
}
if (waitFor != null && latch != null && foundWaitFor(waitFor)) {
latch.countDown();
}
}
System.out.println("Responder stopped.");
}
public String waitFor(String waitFor) {
synchronized (Responder.class) {
if (foundWaitFor(waitFor)) {
return getAndClearBuffer();
}
}
this.waitFor = waitFor;
latch = new CountDownLatch(1);
try {
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
return null;
}
String result = null;
synchronized (Responder.class) {
result = builder.toString();
builder = new StringBuilder();
}
return result;
}
}
static class TelnetReader extends Thread {
private boolean isKeepRunning = true;
private final GogoDeployer checker;
private final TelnetClient tc;
TelnetReader(GogoDeployer checker, TelnetClient tc) {
this.checker = checker;
this.tc = tc;
}
#Override
public void run() {
InputStream instr = tc.getInputStream();
try {
byte[] buff = new byte[1024];
int ret_read = 0;
do {
if (instr.available() > 0) {
ret_read = instr.read(buff);
}
if (ret_read > 0) {
checker.sendForResponse(new String(buff, 0, ret_read));
ret_read = 0;
}
} while (isKeepRunning && (ret_read >= 0));
} catch (Exception e) {
System.err.println("Exception while reading socket:" + e.getMessage());
}
try {
tc.disconnect();
checker.stop();
System.out.println("Disconnected.");
} catch (Exception e) {
System.err.println("Exception while closing telnet:" + e.getMessage());
}
}
}
private static final String prompt = "g!";
private static GogoDeployer client;
private String host;
private BlockingQueue<String> messageQueue = new LinkedBlockingQueue<String>();
private int port;
private TelnetReader reader;
private Responder responder;
private TelnetClient tc;
public GogoDeployer(String host, int port) {
this.host = host;
this.port = port;
}
public void stop() {
responder.isKeepRunning = false;
reader.isKeepRunning = false;
try {
Thread.sleep(10);
} catch (InterruptedException e) {
}
responder.interrupt();
reader.interrupt();
}
public void send(String command) {
PrintStream ps = new PrintStream(tc.getOutputStream());
ps.println(command);
ps.flush();
}
public void sendForResponse(String s) {
messageQueue.add(s);
}
public void connect() throws SocketException, IOException {
tc = new TelnetClient();
tc.connect(host, port);
reader = new TelnetReader(this, tc);
reader.start();
responder = new Responder(this);
responder.start();
}
public String waitFor(String s) {
return responder.waitFor(s);
}
private static String exec(String cmd) {
String result = "";
System.out.println(cmd);
client.send(cmd);
result = client.waitFor(prompt);
return result;
}
public static void main(String[] args) {
try {
String project = args[0];
client = new GogoDeployer("localhost", 6666);
client.connect();
System.out.println(client.waitFor(prompt));
System.out.println(exec("uninstall " + project));
String result = exec("install " + project);
System.out.println(result);
int start = result.indexOf(":");
int stop = result.indexOf(prompt);
String bundleId = result.substring(start + 1, stop).trim();
System.out.println(exec("start " + bundleId));
client.stop();
} catch (SocketException e) {
System.err.println("Unable to conect to Gogo remote shell: " + e.getMessage());
} catch (IOException e) {
System.err.println("Unable to conect to Gogo remote shell: " + e.getMessage());
}
}
}
When I met the same requirement (deploy bundle from target/classes as fast as I can) my first thought was also extending my container with some shell functionality. My second thought was, however, to write a simple bundle that opens up an always-on-top window and I can simply drag-and-drop any project(s) from Eclipse (or total commander or whatever) to that window. The code than checks if the folder(s) that was dropped has a target/classes folder and if it has it will be deployed.
The source code is available at https://github.com/everit-org/osgi-richconsole
The dependency is available from the maven-central.
The dependency is:
<dependency>
<groupId>org.everit.osgi.dev</groupId>
<artifactId>org.everit.osgi.dev.richconsole</artifactId>
<version>1.2.0</version>
</dependency>
You can use the bundle it while you develop and remove it when you set up your live server. However it is not necessary as if the container is running in a headless mode the pop-up window is not shown.
I called it richconsole as I would like to have more features in the future (not just deployment) :)