how properly save data to MongoDb throw Spring-data in no-blocking stack using CompletableFuture - mongodb

The question could be summarized: how properly save data to MongoDb throw Spring-data in no-blocking stack using CompletableFuture (i.e. Spring Webflux + reactive.ReactiveCrudRepository + java.util.concurrent)?
I have struglled for the last three days studing and searcing around and reading several tutorials in order to find a recommended way or at least a "north path" to persist data when someone wants to use CompletableFuture for that. I could reach the code bellow succesfully working but I am not sure if I am doing some weird stuff.
Basically, I want to use CompletableFuture because I want to chain futures. Let say, save firstly in MongoDb and if well-done then "thenAcceptAsync" and finally "thenCombine" them.
Well, ReactiveCrudRepository.save returns Mono<> and I must subscribe in order to effectivelly save it. Additionally Mono<>.subscribe() returns dispose whic I understand I can use to cancel it let's say if the thread takes too long because MongoDb is out for instance or any other exception. SO far so good.
What is unclear to me is if I am not messing up the idea of using using saving the data which blocks in assyncronous method. Since my puporse is leave to "future" resolution am I am blocking during the save method bellow and completely losing the benefitis of saving in different thread and get a future result?
Code saving properly to MongoDb but not clear to me if it is really "no-blocking" approach. Note that completableFuture.get() is commented since I don't need it in onder to effectively save my data
#Async("taskExecutor")
public void transferirDisposableReturnedSupplyAsync(Extrato e) throws InterruptedException, ExecutionException {
CompletableFuture<Disposable> completableFuture = CompletableFuture
.supplyAsync(() -> extratoRepository.save(e).subscribe());
//completableFuture.get(); unnecessary since subscribe() above already saved it
}
In case it is relevant:
Repository:
import org.springframework.data.mongodb.repository.Query;
import org.springframework.data.repository.reactive.ReactiveCrudRepository;
import com.noblockingcase.demo.model.Extrato;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import org.springframework.data.domain.Pageable;
public interface ExtratoRepository extends ReactiveCrudRepository<Extrato, String> {
#Query("{ id: { $exists: true }}")
Flux<Extrato> retrieveAllExtratosPaged(final Pageable page);
}
AsyncConfiguration:
import java.util.concurrent.Executor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
// The #EnableAsync annotation enables Spring’s ability to run #Async methods in a background thread pool.
// The bean taskExecutor helps to customize the thread executor such as configuring number of threads for an application, queue limit size and so on.
// Spring will specifically look for this bean when the server is started.
// If this bean is not defined, Spring will create SimpleAsyncTaskExecutor by default.
#Configuration
#EnableAsync
public class AsyncConfiguration {
private static final Logger LOGGER = LoggerFactory.getLogger(AsyncConfiguration.class);
#Bean(name = "taskExecutor")
public Executor taskExecutor() {
LOGGER.debug("Creating Async Task Executor");
final ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(2);
executor.setMaxPoolSize(2);
executor.setQueueCapacity(100);
executor.setThreadNamePrefix("ExtratoThread-");
executor.initialize();
return executor;
}
}
*** added
import { Injectable, NgZone } from '#angular/core';
import { Observable } from 'rxjs';
import { Extrato } from './extrato';
#Injectable({
providedIn: "root"
})
export class SseService {
extratos: Extrato[] = [];
constructor(private _zone: NgZone) { }
getServerSentEvent(url: string): Observable<any> {
this.extratos = [];
return Observable.create(observer => {
const eventSource = this.getEventSource(url);
eventSource.onmessage = event => {
this._zone.run(() => {
let json = JSON.parse(event.data);
this.extratos.push(new Extrato(json['id'], json['description'], json['value'], json['status']));
observer.next(this.extratos);
});
};
eventSource.onerror = (error) => {
if (eventSource.readyState === 0) {
console.log('The stream has been closed by the server.');
eventSource.close();
observer.complete();
} else {
observer.error('EventSource error: ' + error);
}
}
});
}
private getEventSource(url: string): EventSource {
return new EventSource(url);
}
}

Related

#DynamicPropertySource not being invoked (Kotlin, Spring Boot and TestContainers)

I'm trying to define a #TestConfiguration class that is executed once before all integration tests to run a MongoDB TestContainer in Kotlin in a Spring Boot project.
Here is the code:
import org.springframework.boot.test.context.TestConfiguration
import org.springframework.test.context.DynamicPropertyRegistry
import org.springframework.test.context.DynamicPropertySource
import org.testcontainers.containers.MongoDBContainer
import org.testcontainers.utility.DockerImageName
#TestConfiguration
class TestContainerMongoConfig {
companion object {
#JvmStatic
private val MONGO_CONTAINER: MongoDBContainer = MongoDBContainer(DockerImageName.parse("mongo").withTag("latest")).withReuse(true)
#JvmStatic
#DynamicPropertySource
private fun emulatorProperties(registry: DynamicPropertyRegistry) {
registry.add("spring.data.mongodb.uri", MONGO_CONTAINER::getReplicaSetUrl)
}
init { MONGO_CONTAINER.start() }
}
}
The issue seems to be that emulatorProperties method is not being called.
The regular flow should be that the container is started and then the properties are set.
The first step happens, the second does not.
I know there is an alternative for which I can do this configuration in each functional test class but I don't like it as it adds not needed noise to the test class.
For example, with a Java project that uses Postgres I managed to make it work with the following code:
import javax.sql.DataSource;
import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.context.annotation.Bean;
import org.testcontainers.containers.PostgreSQLContainer;
import org.testcontainers.utility.DockerImageName;
#TestConfiguration
public class PostgresqlTestContainersConfig {
static final PostgreSQLContainer POSTGRES_CONTAINER;
private final static DockerImageName IMAGE = DockerImageName.parse("postgres").withTag("latest");
static {
POSTGRES_CONTAINER = new PostgreSQLContainer(IMAGE);
POSTGRES_CONTAINER.start();
}
#Bean
DataSource dataSource() {
return DataSourceBuilder.create()
.username(POSTGRES_CONTAINER.getUsername())
.password(POSTGRES_CONTAINER.getPassword())
.driverClassName(POSTGRES_CONTAINER.getDriverClassName())
.url(POSTGRES_CONTAINER.getJdbcUrl())
.build();
}
}
I'm trying to achieve the same thing but in Kotlin and using MongoDB.
Any idea on what may be the issue causing the #DynamicPropertySource not being called?
#DynamicPropertySource is part of the Spring-Boot context lifecycle. Since you want to replicate the Java setup in a way, it is not required to use #DynamicPropertySource. Instead you can follow the Singleton Container Pattern, and replicate it in Kotlin as well.
Instead of setting the config on the registry, you can set them as a System property and Spring Autoconfig will pick it up:
init {
MONGO_CONTAINER.start()
System.setProperty("spring.data.mongodb.uri", MONGO_CONTAINER.getReplicaSetUrl());
}
I was able to resolve similar problem in Groovy by:
Having static method annotated with #DynamicPropetySource directly in the test class (probably it would also work in superclass.
But I didn't want to copy the code into every test class that needs MongoDB.
I resolved the issue by using ApplicationContexInitializer
The example is written in groovy
class MongoTestContainer implements ApplicationContextInitializer<ConfigurableApplicationContext>{
static final MongoDBContainer mongoDBContainer = new MongoDBContainer(DockerImageName.parse("mongo:6.0.2"))
#Override
void initialize(ConfigurableApplicationContext applicationContext) {
mongoDBContainer.start()
def testValues = TestPropertyValues.of("spring.data.mongodb.uri="+ mongoDBContainer.getReplicaSetUrl())
testValues.applyTo(applicationContext.getEnvironment())
}
}
To make it complete, in the test class, you just need to add #ContextConfiguration(initializers = MongoTestContainer) to activate context initializer for the test.
For this you could also create custom annotation which would combine #DataMongoTest with previous annotation.
This solution works for me.
Method with #DynamicPropertySource is inside companion object(also added #JvmStatic) and added org.testcontainers.junit.jupiter.Testcontainers on the test class
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.boot.jdbc.DataSourceBuilder
import org.springframework.boot.test.context.TestConfiguration
import org.springframework.context.annotation.Bean
import org.springframework.test.context.ContextConfiguration
import org.springframework.test.context.DynamicPropertyRegistry
import org.springframework.test.context.DynamicPropertySource
import org.springframework.test.context.junit.jupiter.SpringExtension
import org.testcontainers.containers.PostgreSQLContainer
import org.testcontainers.junit.jupiter.Container
import org.testcontainers.junit.jupiter.Testcontainers
import javax.sql.DataSource
#ExtendWith(SpringExtension::class)
#Testcontainers
#TestConfiguration
#ContextConfiguration(classes = [PostgresqlTestContainersConfig::class])
class PostgresqlTestContainersConfig {
#Autowired
var dataSource: DataSource? = null
#Test
internal fun name() {
dataSource!!.connection.close()
}
#Bean
fun dataSource(): DataSource? {
return DataSourceBuilder.create()
.username(POSTGRES_CONTAINER.getUsername())
.password(POSTGRES_CONTAINER.getPassword())
.driverClassName(POSTGRES_CONTAINER.getDriverClassName())
.url(POSTGRES_CONTAINER.getJdbcUrl())
.build()
}
companion object {
#JvmStatic
#Container
private val POSTGRES_CONTAINER: PostgreSQLContainer<*> = PostgreSQLContainer("postgres:9.6.12")
.withDatabaseName("integration-tests-db")
.withUsername("sa")
.withPassword("sa")
#JvmStatic
#DynamicPropertySource
fun postgreSQLProperties(registry: DynamicPropertyRegistry) {
registry.add("db.url") { POSTGRES_CONTAINER.jdbcUrl }
registry.add("db.user") { POSTGRES_CONTAINER.username }
registry.add("db.password") { POSTGRES_CONTAINER.password }
}
}
}

Only apply modifyResponseBody for certain content-type

I am using GatewayFilterSpec.modifyResponseBody (marked as a "BETA" feature) to rewrite JSON payloads. This works well as long as the response payloads are in fact of content-type application/json. In my case, that is unfortunately not always guaranteed, and I would like it to only apply the modifyResponseBody if the reponse has the Content-Type: application/json header, else skip the filter. Is this possible with Spring Cloud Gateway, and how to do this? Thank you.
Now I'm getting this:
org.springframework.web.reactive.function.UnsupportedMediaTypeException: Content type 'text/html' not supported
at org.springframework.web.reactive.function.BodyInserters.lambda$null$11(BodyInserters.java:329)
at java.util.Optional.orElseGet(Optional.java:267)
at org.springframework.web.reactive.function.BodyInserters.lambda$bodyInserterFor$12(BodyInserters.java:325)
Here is a "solution", one that has all sorts of problems:
package my_package;
import org.reactivestreams.Publisher;
import org.springframework.cloud.gateway.filter.GatewayFilter;
import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.filter.factory.rewrite.ModifyResponseBodyGatewayFilterFactory;
import org.springframework.context.annotation.Primary;
import org.springframework.core.io.buffer.DataBuffer;
import org.springframework.http.codec.ServerCodecConfigurer;
import org.springframework.http.server.reactive.ServerHttpResponse;
import org.springframework.http.server.reactive.ServerHttpResponseDecorator;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ServerWebExchange;
import reactor.core.publisher.Mono;
import static org.springframework.http.MediaType.APPLICATION_JSON;
#Component
#Primary
public class JsonOnlyModifyResponseBodyGatewayFilterFactory extends ModifyResponseBodyGatewayFilterFactory {
public JsonOnlyModifyResponseBodyGatewayFilterFactory(ServerCodecConfigurer codecConfigurer) {
super(codecConfigurer);
}
#Override
public GatewayFilter apply(Config config) {
return new MyModifyResponseGatewayFilter(config);
}
public class MyModifyResponseGatewayFilter extends ModifyResponseGatewayFilter {
MyModifyResponseGatewayFilter(Config config) {
super(config);
}
#Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
ServerHttpResponse serverHttpResponse = getServerHttpResponseFromSuper(exchange);
ServerHttpResponseDecorator responseDecorator = new ServerHttpResponseDecorator(exchange.getResponse()) {
#Override
public Mono<Void> writeWith(Publisher<? extends DataBuffer> body) {
if (APPLICATION_JSON.isCompatibleWith(getDelegate().getHeaders().getContentType())) {
return serverHttpResponse.writeWith(body);
}
return super.writeWith(body);
}
};
return chain.filter(exchange.mutate().response(responseDecorator).build());
}
private ServerHttpResponse getServerHttpResponseFromSuper(ServerWebExchange exchange) {
ServerHttpResponse[] serverHttpResponse = new ServerHttpResponse[1];
//noinspection UnassignedFluxMonoInstance
super.filter(exchange, chain -> {
serverHttpResponse[0] = chain.getResponse(); // capture the response when the super sets it
return null;
});
return serverHttpResponse[0];
}
}
}
The chosen approach is in lieu of just changing a copy of the existing ModifyResponseBodyGatewayFilterFactory. This allows version upgrades of Spring Boot Gateway to bring in minor changes of ModifyResponseBodyGatewayFilterFactory. But since JsonOnlyModifyResponseBodyGatewayFilterFactory is very dependent on the implementation of ModifyResponseBodyGatewayFilterFactory, this may easily get broken. Another flaw of this solution is that I had to put an #Primary annotation to avoid a required a single bean, but 2 were found exception, but it overrides the default which would presumably affect other uses of modifyResponseBody. It's ugly to call super.filter and not use its result. And so on. So, while this "works", it doesn't, well, fill me with joy.

How to get JSON data via http get request

I am new to Vaadin.
As in topic I would like to make http get reaquest in order to retieve some JSON data.
How could I do this ?
I have been trying to make this by com.google.gwt.http.client.RequestBuilder, but I have obtained
java.lang.UnsatisfiedLinkError: com.google.gwt.xhr.client.XMLHttpRequest.create().
I think the error is associated to GWT client - side nature.
So how could I make http get request in Vaadin 7 server - side ?
Here is my code:
package com.example.soaclient;
import javax.servlet.annotation.WebServlet;
import com.google.gwt.http.client.Request;
import com.google.gwt.http.client.RequestBuilder;
import com.google.gwt.http.client.RequestCallback;
import com.google.gwt.http.client.RequestException;
import com.google.gwt.http.client.Response;
import com.google.gwt.json.client.JSONObject;
import com.google.gwt.json.client.JSONParser;
import com.google.gwt.json.client.JSONValue;
import com.vaadin.annotations.Theme;
import com.vaadin.annotations.VaadinServletConfiguration;
import com.vaadin.server.VaadinRequest;
import com.vaadin.server.VaadinServlet;
import com.vaadin.ui.Button;
import com.vaadin.ui.Button.ClickEvent;
import com.vaadin.ui.Label;
import com.vaadin.ui.UI;
import com.vaadin.ui.VerticalLayout;
#SuppressWarnings("serial")
#Theme("soaclient")
public class SoaclientUI extends UI {
#WebServlet(value = "/*", asyncSupported = true)
#VaadinServletConfiguration(productionMode = false, ui = SoaclientUI.class)
public static class Servlet extends VaadinServlet {
}
#Override
protected void init(VaadinRequest request) {
final VerticalLayout layout = new VerticalLayout();
layout.setMargin(true);
setContent(layout);
Button button = new Button("Click Me");
button.addClickListener(new Button.ClickListener() {
public void buttonClick(ClickEvent event) {
layout.addComponent(new Label("Thank you for clicking"));
String url = "some URL goes here";
RequestBuilder builder = new RequestBuilder(RequestBuilder.GET, url);
try {
Request request = builder.sendRequest(null, new RequestCallback() {
public void onError(Request request, Throwable exception) {
// Couldn't connect to server (could be timeout, SOP violation, etc.)
}
public void onResponseReceived(Request request, Response response) {
if (200 == response.getStatusCode()) {
// Process the response in response.getText()
} else {
// Handle the error. Can get the status text from response.getStatusText()
}
}
});
} catch (RequestException e) {
// Couldn't connect to server
}
}
});
layout.addComponent(button);
}
}
With Vaadin you should not use anything from com.google.gwt.http.client package. That is only for client side development, e.g. when you make components that need client side counterparts.
Instead of GWT classes you should just stick to generic JDK libraries. E.g. you could simply use java.net.URL.openStream(). But if you are consuming some REST services, you could refer to my recent JAX-RS 2.0 Client article.

Akka-quartz scheduler never triggers my actor

I am trying to use akka-quartz-scheduler to trigger a cleanup event at regular intervals. I get the scheduler to send the message to the actor, but only when there is no Calendar associated with the schedule. Whenever I attach a calendar to a schedule, the actor never receives any messages.
This is the application.conf section relevant for akka-quartz-scheduler. If I remove the line
calendars = ["Minimal"]"
from the config, my actor is triggered. If I leave the line in, no actor gets called.
akka {
quartz {
defaultTimezone = "Europe/Oslo"
schedules {
NowAndThen {
description ="Delete temp files now and then, eg every hour"
expression = "*/10 * * * * ?"
calendars = ["Minimal"]
}
}
calendars {
Minimal {
type = Daily
exclude {
startTime = "15:00"
endTime = "15:01"
}
}
}
}
}
I am initializing the extension from a playframework application, in Global.java:
import akka.actor.ActorRef;
import akka.actor.ActorSystem;
import akka.actor.Extension;
import akka.actor.Props;
import com.typesafe.akka.extension.quartz.QuartzSchedulerExtension;
import play.Application;
import play.GlobalSettings;
import play.Logger;
import play.libs.Akka;
import uttrekk.CleanupRunner;
public class Global extends GlobalSettings {
public void onStart(Application app) {
AkkaStartUp.startup(app);
}
static class AkkaStartUp {
public static void startup(Application app) {
// Starter autoamtiske avelveringer norges eiendommer
ActorSystem system = Akka.system();
ActorRef cleanupRef =system.actorOf(new Props(CleanupRunner.class));
QuartzSchedulerExtension scheduler = (QuartzSchedulerExtension) QuartzSchedulerExtension.get(system);
scheduler.schedule("NowAndThen",cleanupRef,"Clean");
}
}
}
The actor implementation looks something like the following:
package uttrekk;
import akka.actor.UntypedActor;
import play.Logger;
import util.NewProperties;
import java.io.File;
import java.io.FilenameFilter;
public class CleanupRunner extends UntypedActor {
#Override
public void onReceive(Object message) throws Exception {
Logger.info("Running cleanup of temporary files");
}
}
the problem occurs during calendars initialization.
Check line 245 in QuartzSchedulerExtension class:
scheduler.addCalendar(name.toUpperCase, calendar, true, true)
The calendar is added using UpperCase, so Quartz never finds it, producing that no job is triggered then.
If you define your calendar in akka config using UpperCase name it should work.

Play 2.0! [Java] - Generating XML response from the REST API

I am parsing through Play framework documents and trying to figure out if there is anything out of the box available for generating XML response from the given domain object, just like how we have for Json.toJson(Object).
The following code works fine for Json REST API in play framework 2.1.2, can anyone suggest how can XML be generated out of the box here instead of Json?
package controllers;
import java.util.List;
import java.util.concurrent.Callable;
import play.Logger;
import play.libs.F.Function;
import play.libs.F.Promise;
import play.libs.Json;
import play.mvc.Controller;
import play.mvc.Result;
import com.amazonaws.services.simpledb.model.Item;
public class ShowItemsJson extends Controller {
public static Result allItems() {
// Now create the async process to lookup items in simpledb
AllItems<List<Item>> callable = new AllItems<List<Item>>();
Promise<List<Item>> promise = play.libs.Akka.future(callable);
return async(promise.map(new Function<List<Item>, Result>() {
public Result apply(List<Item> rm) throws Throwable {
// Convert the result into json before sending.
// TODO How to do same for XML?
return ok(Json.toJson(rm));
}
}));
}
// One instance of this class should be used for each create request
static class AllItems<V> implements Callable<V> {
#SuppressWarnings("unchecked")
public V call() throws Exception {
try {
return (V) Test.getAllItems();
} catch (Error e) {
// Error is handled here to log NoClassDefFoundError
Logger.error("Error: ", e);
throw e;
}
}
}
}
There is no built in support for generating XML from Java objects in Play 2 as far as I know, there are loads of options in Java-land though.
To name a few:
JAXB for doing it with reflection/annotations - reference implementation http://jaxb.java.net
xom - http://www.xom.nu
jdom - http://www.jdom.org,
dom4j - http://dom4j.sourceforge.net