Only apply modifyResponseBody for certain content-type - spring-cloud

I am using GatewayFilterSpec.modifyResponseBody (marked as a "BETA" feature) to rewrite JSON payloads. This works well as long as the response payloads are in fact of content-type application/json. In my case, that is unfortunately not always guaranteed, and I would like it to only apply the modifyResponseBody if the reponse has the Content-Type: application/json header, else skip the filter. Is this possible with Spring Cloud Gateway, and how to do this? Thank you.
Now I'm getting this:
org.springframework.web.reactive.function.UnsupportedMediaTypeException: Content type 'text/html' not supported
at org.springframework.web.reactive.function.BodyInserters.lambda$null$11(BodyInserters.java:329)
at java.util.Optional.orElseGet(Optional.java:267)
at org.springframework.web.reactive.function.BodyInserters.lambda$bodyInserterFor$12(BodyInserters.java:325)

Here is a "solution", one that has all sorts of problems:
package my_package;
import org.reactivestreams.Publisher;
import org.springframework.cloud.gateway.filter.GatewayFilter;
import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.filter.factory.rewrite.ModifyResponseBodyGatewayFilterFactory;
import org.springframework.context.annotation.Primary;
import org.springframework.core.io.buffer.DataBuffer;
import org.springframework.http.codec.ServerCodecConfigurer;
import org.springframework.http.server.reactive.ServerHttpResponse;
import org.springframework.http.server.reactive.ServerHttpResponseDecorator;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ServerWebExchange;
import reactor.core.publisher.Mono;
import static org.springframework.http.MediaType.APPLICATION_JSON;
#Component
#Primary
public class JsonOnlyModifyResponseBodyGatewayFilterFactory extends ModifyResponseBodyGatewayFilterFactory {
public JsonOnlyModifyResponseBodyGatewayFilterFactory(ServerCodecConfigurer codecConfigurer) {
super(codecConfigurer);
}
#Override
public GatewayFilter apply(Config config) {
return new MyModifyResponseGatewayFilter(config);
}
public class MyModifyResponseGatewayFilter extends ModifyResponseGatewayFilter {
MyModifyResponseGatewayFilter(Config config) {
super(config);
}
#Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
ServerHttpResponse serverHttpResponse = getServerHttpResponseFromSuper(exchange);
ServerHttpResponseDecorator responseDecorator = new ServerHttpResponseDecorator(exchange.getResponse()) {
#Override
public Mono<Void> writeWith(Publisher<? extends DataBuffer> body) {
if (APPLICATION_JSON.isCompatibleWith(getDelegate().getHeaders().getContentType())) {
return serverHttpResponse.writeWith(body);
}
return super.writeWith(body);
}
};
return chain.filter(exchange.mutate().response(responseDecorator).build());
}
private ServerHttpResponse getServerHttpResponseFromSuper(ServerWebExchange exchange) {
ServerHttpResponse[] serverHttpResponse = new ServerHttpResponse[1];
//noinspection UnassignedFluxMonoInstance
super.filter(exchange, chain -> {
serverHttpResponse[0] = chain.getResponse(); // capture the response when the super sets it
return null;
});
return serverHttpResponse[0];
}
}
}
The chosen approach is in lieu of just changing a copy of the existing ModifyResponseBodyGatewayFilterFactory. This allows version upgrades of Spring Boot Gateway to bring in minor changes of ModifyResponseBodyGatewayFilterFactory. But since JsonOnlyModifyResponseBodyGatewayFilterFactory is very dependent on the implementation of ModifyResponseBodyGatewayFilterFactory, this may easily get broken. Another flaw of this solution is that I had to put an #Primary annotation to avoid a required a single bean, but 2 were found exception, but it overrides the default which would presumably affect other uses of modifyResponseBody. It's ugly to call super.filter and not use its result. And so on. So, while this "works", it doesn't, well, fill me with joy.

Related

Resty-GWT custom callback on async start and end

I use resty gwt for all server communication. I would like some indicator that would show the operation is in progress.
I consider 2 aproaches:
progressbar, which will show in progress percentage;
animation, that will be showed while operation is in progress, but without any percantage.
I've assumed that I need to add custom filter with callback.
I would like to fire events like: RestyGwtComunicationStart and RestyGwtComunicationEnd, or callback to fire onComunicationStarted and
onComunicationEnded. I would like to have this declared in one place, RestyGWT Dispatcher configuration. Also if there was an error I would like to fetch the error.
But I don't know where to start. There is no word about it in documentations.
Can I ask You for help? How can I do this?
So if you want to know that a request has been sent it is up to you in your GWT app to treat that. You can send an event when you trigger your request. You have multiple way of doing this.
Have a look at Request Dispatcher inside the doc https://resty-gwt.github.io/documentation/restygwt-user-guide.html
Then if you want to get progress info, as HTTP calls are synchronous. So there is no way to do this easily.
The way I have been doing it is the following:
1) Create a first call to initiate a processing on the backend with a POST, this will return the ID of your processing
2) Then do a GET on your processing ID that will return the progress. Once the progress is 100% it will return the ID of the result
3) GET the result with the result ID
(You can mix 2 and 3 together eventually and return result when progress is 100% in the same DTO)
Another option is to replace 2) by pushing info from backend to front end (html5 websocket)
Someone already did it as a pull-request to resty. Guess you can give it a try:
https://github.com/resty-gwt/resty-gwt/pull/151
Unfortunately "Dispatcher/Callback filters" feature does not described in the official documentation. But I can suggest next solution (this code should be placed in EntryPoint implementation of your module):
public void onModuleLoad() {
//...
//used to show busy indicator before send HTTP request
DispatcherFilter busyIndicatorDispatcherFilter = new DispatcherFilter() {
#Override
public boolean filter(Method method, RequestBuilder builder) {
BusyIndicator.show();
return true;
}
};
//used to show busy indicator after HTTP response recieved
CallbackFilter busyIndicatorCallbackFilter = new CallbackFilter() {
#Override
public RequestCallback filter(Method method, Response response, RequestCallback callback) {
BusyIndicator.hide();
return callback;
}
};
//registering FilterawareDispatcher (and busy indicator filters) as default Dispatcher
Defaults.setDispatcher(new DefaultFilterawareDispatcher(
busyIndicatorDispatcherFilter,
new DefaultDispatcherFilter(new DefaultCallbackFactory(busyIndicatorCallbackFilter))));
//...
}
Unfortunately I did not get adequate answer, So I developed my own solution.
At first I've added Resty configuration RestyGwtConfig to my Module configuration
public class ClientModule extends AbstractPresenterModule {
#Override
protected void configure() {
bind(RestyGwtConfig.class).asEagerSingleton();
install(new DefaultModule.Builder()
.defaultPlace(Routing.HOME.url)
.errorPlace(Routing.ERROR.url)
.unauthorizedPlace(Routing.LOGIN.url)
.tokenFormatter(RouteTokenFormatter.class).build());
install(new AppModule());
install(new GinFactoryModuleBuilder().build(AssistedInjectionFactory.class));
bind(ResourceLoader.class).asEagerSingleton();
}
}
then I've set Custom distpatcher for all my comunication requests of resty gwt.
import org.fusesource.restygwt.client.Defaults;
import org.fusesource.restygwt.client.Resource;
import pl.korbeldaniel.cms.shared.ServiceRouting;
import com.google.gwt.core.client.GWT;
import com.google.inject.Inject;
public class RestyGwtConfig {
#Inject
public RestyGwtConfig(RestyDispatcher dispatcher) {
Defaults.setDispatcher(dispatcher);
}
}
Then I've added custom filter (ProgressIndicatorFilter) to handle communication's start and end callbacks:
import org.fusesource.restygwt.client.Method;
import org.fusesource.restygwt.client.dispatcher.DefaultFilterawareDispatcher;
import com.google.gwt.http.client.Request;
import com.google.gwt.http.client.RequestBuilder;
import com.google.gwt.http.client.RequestException;
import com.google.inject.Inject;
public class RestyDispatcher extends DefaultFilterawareDispatcher {
#Inject
public RestyDispatcher(ProgressIndicatorFilter progressIndicatorFilter) {
addFilter(progressIndicatorFilter);
}
}
in filter class method overriden filter I've added an event trigger (eventBus.fireEvent(new IndicatorEvent("Rest-Gwt Comunication started"));) and registered callback, here is whole code:
import org.fusesource.restygwt.client.Method;
import org.fusesource.restygwt.client.dispatcher.DispatcherFilter;
import pl.korbeldaniel.cms.client.template.progressIndicator.IndicatorEvent;
import com.google.gwt.http.client.RequestBuilder;
import com.google.inject.Inject;
import com.google.web.bindery.event.shared.EventBus;
class ProgressIndicatorFilter implements DispatcherFilter {
private AssistedInjectionFactory factory;
private EventBus eventBus;
#Inject
public ProgressIndicatorFilter(AssistedInjectionFactory factory, EventBus eventBus) {
this.factory = factory;
this.eventBus = eventBus;
}
#Override
public boolean filter(Method method, RequestBuilder builder) {
builder.setCallback(factory.createProgressIndicatorCallback(method));
eventBus.fireEvent(new IndicatorEvent("Resty-Gwt Comunication started"));
return true;
}
}
Registering a callback couldn't be done straight forward, like
new ProgressIndicatorDispatcherCallback()
cause I use dependency injection. So I've created a factory to assist injection as follow:
public interface AssistedInjectionFactory {
ProgressIndicatorDispatcherCallback createProgressIndicatorCallback(Method method);
}
Here and here You can find more Assisted Injection info.
Here is the callback code:
class ProgressIndicatorDispatcherCallback implements RequestCallback {
private RequestCallback requestCallback;
private EventBus eventBus;
#Inject
public ProgressIndicatorDispatcherCallback(#Assisted Method method, EventBus eventBus) {
this.requestCallback = method.builder.getCallback();
this.eventBus = eventBus;
}
#Override
public void onResponseReceived(Request request, Response response) {
endComunicationFireIvent();
requestCallback.onResponseReceived(request, response);
}
#Override
public void onError(Request request, Throwable exception) {
endComunicationFireIvent();
requestCallback.onError(request, exception);
}
private void endComunicationFireIvent() {
eventBus.fireEvent(new IndicatorEvent("Rest-Gwt Comunication ended"));
}
}

Zuul proxy with ribbon + listOfServers and logging requirement

Hi I have configured Zuul (spring-cloud-netflix) to work with ribbon.
I was able to see which request is coming to zuul and what requests is being sent to related endpoint in the logs before adding the ribbon to it.
At some point I have added ribbon & load balancing feature.
Now I can not see outgoing req(from zuul to real endpoint) request in the logs. Things are working fine . There is no error . But I need to know/log incomingReq->outgoingReq information inside the Zuul proxy. I mean I need the route path information (especially with ip information) .
Is there any way to see/get this information ? I want to log it in a filter if possible ?
I found a way to do it something like that ..
Hope it helps somebody else too.
import com.netflix.client.IResponse;
import com.netflix.zuul.ZuulFilter;
import com.netflix.zuul.context.RequestContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import java.net.URI;
#Component
public class PostFilter extends ZuulFilter {
Logger auditLogger = LoggerFactory.getLogger("auditLogger");
#Override
public String filterType() {
return "post";
}
#Override
public int filterOrder() {
return 1;
}
#Override
public boolean shouldFilter() {
return true;
}
#Override
public Object run() {
RequestContext ctx = RequestContext.getCurrentContext();
URI ribbonResponseURI = ((IResponse) ctx.get("ribbonResponse")).getRequestedURI();
AuditBean auditBean = (AuditBean) RequestContext.getCurrentContext().get("auditBean");
auditBean.setIncomingURI(ctx.getRequest().getRequestURL().toString());
auditBean.setOutgoingURI(ribbonResponseURI.toString());
auditLogger.info("{}", auditBean.toJson());
return null;
}
}

How to get JSON data via http get request

I am new to Vaadin.
As in topic I would like to make http get reaquest in order to retieve some JSON data.
How could I do this ?
I have been trying to make this by com.google.gwt.http.client.RequestBuilder, but I have obtained
java.lang.UnsatisfiedLinkError: com.google.gwt.xhr.client.XMLHttpRequest.create().
I think the error is associated to GWT client - side nature.
So how could I make http get request in Vaadin 7 server - side ?
Here is my code:
package com.example.soaclient;
import javax.servlet.annotation.WebServlet;
import com.google.gwt.http.client.Request;
import com.google.gwt.http.client.RequestBuilder;
import com.google.gwt.http.client.RequestCallback;
import com.google.gwt.http.client.RequestException;
import com.google.gwt.http.client.Response;
import com.google.gwt.json.client.JSONObject;
import com.google.gwt.json.client.JSONParser;
import com.google.gwt.json.client.JSONValue;
import com.vaadin.annotations.Theme;
import com.vaadin.annotations.VaadinServletConfiguration;
import com.vaadin.server.VaadinRequest;
import com.vaadin.server.VaadinServlet;
import com.vaadin.ui.Button;
import com.vaadin.ui.Button.ClickEvent;
import com.vaadin.ui.Label;
import com.vaadin.ui.UI;
import com.vaadin.ui.VerticalLayout;
#SuppressWarnings("serial")
#Theme("soaclient")
public class SoaclientUI extends UI {
#WebServlet(value = "/*", asyncSupported = true)
#VaadinServletConfiguration(productionMode = false, ui = SoaclientUI.class)
public static class Servlet extends VaadinServlet {
}
#Override
protected void init(VaadinRequest request) {
final VerticalLayout layout = new VerticalLayout();
layout.setMargin(true);
setContent(layout);
Button button = new Button("Click Me");
button.addClickListener(new Button.ClickListener() {
public void buttonClick(ClickEvent event) {
layout.addComponent(new Label("Thank you for clicking"));
String url = "some URL goes here";
RequestBuilder builder = new RequestBuilder(RequestBuilder.GET, url);
try {
Request request = builder.sendRequest(null, new RequestCallback() {
public void onError(Request request, Throwable exception) {
// Couldn't connect to server (could be timeout, SOP violation, etc.)
}
public void onResponseReceived(Request request, Response response) {
if (200 == response.getStatusCode()) {
// Process the response in response.getText()
} else {
// Handle the error. Can get the status text from response.getStatusText()
}
}
});
} catch (RequestException e) {
// Couldn't connect to server
}
}
});
layout.addComponent(button);
}
}
With Vaadin you should not use anything from com.google.gwt.http.client package. That is only for client side development, e.g. when you make components that need client side counterparts.
Instead of GWT classes you should just stick to generic JDK libraries. E.g. you could simply use java.net.URL.openStream(). But if you are consuming some REST services, you could refer to my recent JAX-RS 2.0 Client article.

Create Scalding Source like TextLine that combines multiple files into single mappers

We have many small files that need combining. In Scalding you can use TextLine to read files as text lines. The problem is we get 1 mapper per file, but we want to combine multiple files so that they are processed by 1 mapper.
I understand we need to change the input format to an implementation of CombineFileInputFormat, and this may involve using cascadings CombinedHfs. We cannot work out how to do this, but it should be just a handful of lines of code to define our own Scalding source called, say, CombineTextLine.
Many thanks to anyone who can provide the code to do this.
As a side question, we have some data that is in s3, it would be great if the solution given works for s3 files - I guess it depends on whether CombineFileInputFormat or CombinedHfs works for s3.
You get the idea in your question, so here is what possibly is a solution for you.
Create your own input format that extends the CombineFileInputFormat and uses your own custom RecordReader. I am showing you Java code, but you could easily convert it to scala if you want.
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileSplit;
import org.apache.hadoop.mapred.InputSplit;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.LineRecordReader;
import org.apache.hadoop.mapred.RecordReader;
import org.apache.hadoop.mapred.Reporter;
import org.apache.hadoop.mapred.lib.CombineFileInputFormat;
import org.apache.hadoop.mapred.lib.CombineFileRecordReader;
import org.apache.hadoop.mapred.lib.CombineFileSplit;
public class CombinedInputFormat<K, V> extends CombineFileInputFormat<K, V> {
public static class MyKeyValueLineRecordReader implements RecordReader<LongWritable,Text> {
private final RecordReader<LongWritable,Text> delegate;
public MyKeyValueLineRecordReader(CombineFileSplit split, Configuration conf, Reporter reporter, Integer idx) throws IOException {
FileSplit fileSplit = new FileSplit(split.getPath(idx), split.getOffset(idx), split.getLength(idx), split.getLocations());
delegate = new LineRecordReader(conf, fileSplit);
}
#Override
public boolean next(LongWritable key, Text value) throws IOException {
return delegate.next(key, value);
}
#Override
public LongWritable createKey() {
return delegate.createKey();
}
#Override
public Text createValue() {
return delegate.createValue();
}
#Override
public long getPos() throws IOException {
return delegate.getPos();
}
#Override
public void close() throws IOException {
delegate.close();
}
#Override
public float getProgress() throws IOException {
return delegate.getProgress();
}
}
#Override
public RecordReader getRecordReader(InputSplit split, JobConf job, Reporter reporter) throws IOException {
return new CombineFileRecordReader(job, (CombineFileSplit) split, reporter, (Class) MyKeyValueLineRecordReader.class);
}
}
Then you need to extend the TextLine class and make it use your own input format you just defined (Scala code from now on).
import cascading.scheme.hadoop.TextLine
import cascading.flow.FlowProcess
import org.apache.hadoop.mapred.{OutputCollector, RecordReader, JobConf}
import cascading.tap.Tap
import com.twitter.scalding.{FixedPathSource, TextLineScheme}
import cascading.scheme.Scheme
class CombineFileTextLine extends TextLine{
override def sourceConfInit(flowProcess: FlowProcess[JobConf], tap: Tap[JobConf, RecordReader[_, _], OutputCollector[_, _]], conf: JobConf) {
super.sourceConfInit(flowProcess, tap, conf)
conf.setInputFormat(classOf[CombinedInputFormat[String, String]])
}
}
Create a scheme for the for your combined input.
trait CombineFileTextLineScheme extends TextLineScheme{
override def hdfsScheme = new CombineFileTextLine().asInstanceOf[Scheme[JobConf,RecordReader[_,_],OutputCollector[_,_],_,_]]
}
Finally, create your source class:
case class CombineFileMultipleTextLine(p : String*) extends FixedPathSource(p :_*) with CombineFileTextLineScheme
If you want to use a single path instead of multiple ones, the change to your source class is trivial.
I hope that helps.
this should do the trick, ya man? - https://wiki.apache.org/hadoop/HowManyMapsAndReduces

Reusable Liferay (6.0.6) service

I am trying to implement resuable Custom Services without using ext and servicebuilder.
I referred this article: http://www.devatwork.nl/2010/04/implementing-a-reusable-liferay-service-without-ext-or-service-builder/ , but I am confused in how should I implement this using eclipse? Following are the steps that I followed to do this:
- Created liferay-plugin project within eclipse.
- Created package containing CustomServices (interface) and CustomServicesUtil.
- Created jar file of package in step 2.
- Placed that jar file in tomcat\lib\ext\
- Then created package (with in same liferay-plugin project), that includes CutomServicesImpl and CustomServicesBaseImpl
- Defined portlet-spring.xml, service.properties, and modified web.xml (as per the article), and finally deployed the project.
On deployment, project is deployed successfully, but when I am trying to use customMethods defined in CustomServicesImpl through CustomServicesUtil.getCustomMethod(), I am getting the following error:
"java.lang.ClassNotFoundException: com.demo.custom.services.CustomServicesUtil"
I configure build path to include customservices.jar file but its not working out, still showing the same error. I don’t know whether this is the correct way to implement resuable services or not. I tried this so that i can make use of custom method in one of my project.
Here is the code for custom services:
CustomServices.java
package com.demo.custom.services;
import com.liferay.portal.model.User;
public interface CustomServices {
String getCustomName(User user);
}
CustomServicesUtil.java
package com.demo.custom.services;
import com.liferay.portal.model.User;
public class CustomServicesUtil {
private static CustomServices services;
public static CustomServices getServices() {
if (services == null) {
throw new RuntimeException("Custom Services not set");
}
return services;
}
public void setServices(CustomServices pServices) {
services = pServices;
}
public static String getCustomName(User user){
return getServices().getCustomName(user);
}
}
CustomServicesBaseImpl.java
package com.demo.custom.services.impl;
import com.demo.custom.services.CustomServices;
import com.liferay.portal.kernel.exception.SystemException;
import com.liferay.portal.service.base.PrincipalBean;
import com.liferay.portal.util.PortalUtil;
public abstract class CustomServicesBaseImpl extends PrincipalBean implements CustomServices {
protected CustomServices services;
public CustomServices getServices() {
return services;
}
public void setServices(CustomServices pServices) {
this.services = pServices;
}
protected void runSQL(String sql) throws SystemException {
try {
PortalUtil.runSQL(sql);
} catch (Exception e) {
throw new SystemException(e);
}
}
}
CustomServicesImpl.java
package com.demo.custom.services.impl;
import com.liferay.portal.model.User;
public class CustomServicesImpl extends CustomServicesBaseImpl {
#Override
public String getCustomName(User user) {
// TODO Auto-generated method stub
if(user == null){
return null;
}else{
return new StringBuffer().append(user.getFirstName()).append(" ").append(user.getLastName()).toString();
}
}
}
Here is the code of controller class of my another portlet, where i am making use of this service.
HelloCustomName.java
package com.test;
import java.io.IOException;
import javax.portlet.PortletException;
import javax.portlet.RenderRequest;
import javax.portlet.RenderResponse;
import com.demo.custom.services.CustomServicesUtil;
import com.liferay.portal.kernel.util.WebKeys;
import com.liferay.portal.model.User;
import com.liferay.portal.theme.ThemeDisplay;
import com.liferay.util.bridges.mvc.MVCPortlet;
public class HelloCustomName extends MVCPortlet {
#Override
public void doView(RenderRequest renderRequest,
RenderResponse renderResponse) throws IOException, PortletException {
System.out.println("--doview----");
ThemeDisplay themeDisplay = (ThemeDisplay)renderRequest.getAttribute(WebKeys.THEME_DISPLAY);
User user = themeDisplay.getUser();
String customName = CustomServicesUtil.getCustomName(user); //getting error here
System.out.println("customName:" + customName);
}
}
Please point me on how to implement resuable services? Any guidance will be really useful.
Thanks.
My mind, you don't need the complexity of services. Simply make utility classes and put this in to tomcat/lib/ext. Be sure that tomcat/lib/ext is correct configured in tomcat/conf/catalina.properties, something like this:
common.loader=${catalina.home}/lib/ext/*.jar