Play! Framework 2.5 form encoding - encoding

After upgrading from Play! 2.4 to Play! 2.5 there seems to be a problem with form-enconding. German umlauts for example really get messed up. For example the german word "Bär" becomes "Bär".
I already changed the way I access form data from
Form.form(Form.class).bindFromRequest();
to the injected way, as described in the migration guide:
formFactory.form(Form.class).bindFromRequest();
Is there a was to fix the character encoding in Play 2.5?

This is a bug in play 2.5 , see github issue: https://github.com/playframework/playframework/pull/5920
I resolve this with a custom body parser:
package com.kashi.ssff.services;
import akka.util.ByteString;
import play.api.http.HttpConfiguration;
import play.core.parsers.FormUrlEncodedParser;
import play.http.HttpErrorHandler;
import play.libs.F;
import play.libs.streams.Accumulator;
import play.mvc.BodyParser;
import play.mvc.BodyParsers;
import play.mvc.Http;
import play.mvc.Result;
import javax.inject.Inject;
import java.util.Map;
/**
* Created by mohsen on 3/21/16.
*/
public class FormBodyParser extends BodyParser.BufferingBodyParser<Map<String, String[]>> {
private final HttpErrorHandler errorHandler;
public FormBodyParser(long maxLength, HttpErrorHandler errorHandler) {
super(maxLength, errorHandler, "Error parsing form");
this.errorHandler = errorHandler;
}
#Inject
public FormBodyParser(HttpConfiguration httpConfiguration, HttpErrorHandler errorHandler) {
super(httpConfiguration, errorHandler, "Error parsing form");
this.errorHandler = errorHandler;
}
#Override
public Accumulator<ByteString, F.Either<Result, Map<String, String[]>>> apply(Http.RequestHeader request) {
return BodyParsers.validateContentType(errorHandler, request, "Expected application/x-www-form-urlencoded",
ct -> ct.equalsIgnoreCase("application/x-www-form-urlencoded"), super::apply);
}
#Override
protected Map<String, String[]> parse(Http.RequestHeader request, ByteString bytes) throws Exception {
String charset = request.charset().orElse("UTF-8");
return FormUrlEncodedParser.parseAsJavaArrayValues(bytes.decodeString(charset), charset);
}
}
and annotate controller with this body parser:
#BodyParser.Of(FormBodyParser.class)
public Result register() {
.
.
.
With special thanks #GregMethvin

in your form try adding the accept-charset="ISO-8859-1"
<form accept-charset="ISO-8859-1" action="whatever" method="post">
tested it for the word Bär and it works fine
EDIT:
For the arabic script the characters are stored as ISO-8859-1 encoded as well. An image to show that at the end all goes well.

Related

Only apply modifyResponseBody for certain content-type

I am using GatewayFilterSpec.modifyResponseBody (marked as a "BETA" feature) to rewrite JSON payloads. This works well as long as the response payloads are in fact of content-type application/json. In my case, that is unfortunately not always guaranteed, and I would like it to only apply the modifyResponseBody if the reponse has the Content-Type: application/json header, else skip the filter. Is this possible with Spring Cloud Gateway, and how to do this? Thank you.
Now I'm getting this:
org.springframework.web.reactive.function.UnsupportedMediaTypeException: Content type 'text/html' not supported
at org.springframework.web.reactive.function.BodyInserters.lambda$null$11(BodyInserters.java:329)
at java.util.Optional.orElseGet(Optional.java:267)
at org.springframework.web.reactive.function.BodyInserters.lambda$bodyInserterFor$12(BodyInserters.java:325)
Here is a "solution", one that has all sorts of problems:
package my_package;
import org.reactivestreams.Publisher;
import org.springframework.cloud.gateway.filter.GatewayFilter;
import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.filter.factory.rewrite.ModifyResponseBodyGatewayFilterFactory;
import org.springframework.context.annotation.Primary;
import org.springframework.core.io.buffer.DataBuffer;
import org.springframework.http.codec.ServerCodecConfigurer;
import org.springframework.http.server.reactive.ServerHttpResponse;
import org.springframework.http.server.reactive.ServerHttpResponseDecorator;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ServerWebExchange;
import reactor.core.publisher.Mono;
import static org.springframework.http.MediaType.APPLICATION_JSON;
#Component
#Primary
public class JsonOnlyModifyResponseBodyGatewayFilterFactory extends ModifyResponseBodyGatewayFilterFactory {
public JsonOnlyModifyResponseBodyGatewayFilterFactory(ServerCodecConfigurer codecConfigurer) {
super(codecConfigurer);
}
#Override
public GatewayFilter apply(Config config) {
return new MyModifyResponseGatewayFilter(config);
}
public class MyModifyResponseGatewayFilter extends ModifyResponseGatewayFilter {
MyModifyResponseGatewayFilter(Config config) {
super(config);
}
#Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
ServerHttpResponse serverHttpResponse = getServerHttpResponseFromSuper(exchange);
ServerHttpResponseDecorator responseDecorator = new ServerHttpResponseDecorator(exchange.getResponse()) {
#Override
public Mono<Void> writeWith(Publisher<? extends DataBuffer> body) {
if (APPLICATION_JSON.isCompatibleWith(getDelegate().getHeaders().getContentType())) {
return serverHttpResponse.writeWith(body);
}
return super.writeWith(body);
}
};
return chain.filter(exchange.mutate().response(responseDecorator).build());
}
private ServerHttpResponse getServerHttpResponseFromSuper(ServerWebExchange exchange) {
ServerHttpResponse[] serverHttpResponse = new ServerHttpResponse[1];
//noinspection UnassignedFluxMonoInstance
super.filter(exchange, chain -> {
serverHttpResponse[0] = chain.getResponse(); // capture the response when the super sets it
return null;
});
return serverHttpResponse[0];
}
}
}
The chosen approach is in lieu of just changing a copy of the existing ModifyResponseBodyGatewayFilterFactory. This allows version upgrades of Spring Boot Gateway to bring in minor changes of ModifyResponseBodyGatewayFilterFactory. But since JsonOnlyModifyResponseBodyGatewayFilterFactory is very dependent on the implementation of ModifyResponseBodyGatewayFilterFactory, this may easily get broken. Another flaw of this solution is that I had to put an #Primary annotation to avoid a required a single bean, but 2 were found exception, but it overrides the default which would presumably affect other uses of modifyResponseBody. It's ugly to call super.filter and not use its result. And so on. So, while this "works", it doesn't, well, fill me with joy.

Camel Rest DSL retrieve HTTP POST multipart File

My Router class looks like below and i am trying to upload a video file and store it to a File location.
SpringBootRouter.java
package com.camelrest;
import java.util.HashMap;
import java.util.Map;
import org.apache.camel.component.restlet.RestletComponent;
import org.apache.camel.spring.boot.FatJarRouter;
import org.restlet.Component;
import org.restlet.ext.spring.SpringServerServlet;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.embedded.ServletRegistrationBean;
import org.springframework.context.annotation.Bean;
#SpringBootApplication
public class MySpringBootRouter extends FatJarRouter {
#Autowired
private MultipartProcessor multipartProcessor;
#Override
public void configure() {
restConfiguration().component("restlet");
rest("/upload").post().to("direct:upload");
from("direct:upload")
.to("file://E:/RestTest");
}
#Bean
public ServletRegistrationBean servletRegistrationBean() {
SpringServerServlet serverServlet = new SpringServerServlet();
ServletRegistrationBean regBean = new ServletRegistrationBean(
serverServlet, "/rest/*");
Map<String, String> params = new HashMap<String, String>();
params.put("org.restlet.component", "restletComponent");
regBean.setInitParameters(params);
return regBean;
}
#Bean
public Component restletComponent() {
return new Component();
}
#Bean
public RestletComponent restletComponentService() {
return new RestletComponent(restletComponent());
}
}
I am trying to upload a video file using postman as per below screenshot :
My contents of the file that i upload are saved with a file name with some random camel ID generated by camel
However i want the filename that is passed in body
SampleVideo_1280x720_10mb.mp4
to be the name of the file and remove the following contents from the body
----------------------------948281627232093197119960
Content-Disposition: form-data; name="file"; filename="SampleVideo_1280x720_10mb.mp4"
Content-Type: video/mp4
So final output can be the video uploaded with the filename used during the upload with postman
You can use MimeMultipartDataFormat to unmarshal Multipart request. Using this, will prepare attachments, to Exchange.
After that you need somehow convert Attachment to InputStream and fill CamelFileName header. With this task can help you small Processor.
Route:
from("direct:upload")
.unmarshal().mimeMultipart().split().attachments()
.process(new PrepareFileFromAttachment())
.to("file://C:/RestTest");
Processor:
class PrepareFileFromAttachment implements Processor {
#Override
public void process(Exchange exchange) throws Exception {
DataHandler dataHandler = exchange.getIn().getBody(Attachment.class).getDataHandler();
exchange.getIn().setHeader(Exchange.FILE_NAME, dataHandler.getName());
exchange.getIn().setBody(dataHandler.getInputStream());
}
}
The approach above does not work in case your form contains only single input in form. This is because MimeMultipartDataFormat marshals first form input into body (without storing file name) and other inputs to attachments where the file name is stored.
In this case you need to create Processor reading InputStream directly:
Route:
from("direct:upload")
.process(new ProcessMultipartRequest())
.to("file:c://RestTest");
Processor
public class ProcessMultipartRequest implements Processor {
#Override
public void process(Exchange exchange) throws Exception {
InputStream is = exchange.getIn().getBody(InputStream.class);
MimeBodyPart mimeMessage = new MimeBodyPart(is);
DataHandler dh = mimeMessage.getDataHandler();
exchange.getIn().setBody(dh.getInputStream());
exchange.getIn().setHeader(Exchange.FILE_NAME, dh.getName());
}
}

Create Scalding Source like TextLine that combines multiple files into single mappers

We have many small files that need combining. In Scalding you can use TextLine to read files as text lines. The problem is we get 1 mapper per file, but we want to combine multiple files so that they are processed by 1 mapper.
I understand we need to change the input format to an implementation of CombineFileInputFormat, and this may involve using cascadings CombinedHfs. We cannot work out how to do this, but it should be just a handful of lines of code to define our own Scalding source called, say, CombineTextLine.
Many thanks to anyone who can provide the code to do this.
As a side question, we have some data that is in s3, it would be great if the solution given works for s3 files - I guess it depends on whether CombineFileInputFormat or CombinedHfs works for s3.
You get the idea in your question, so here is what possibly is a solution for you.
Create your own input format that extends the CombineFileInputFormat and uses your own custom RecordReader. I am showing you Java code, but you could easily convert it to scala if you want.
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileSplit;
import org.apache.hadoop.mapred.InputSplit;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.LineRecordReader;
import org.apache.hadoop.mapred.RecordReader;
import org.apache.hadoop.mapred.Reporter;
import org.apache.hadoop.mapred.lib.CombineFileInputFormat;
import org.apache.hadoop.mapred.lib.CombineFileRecordReader;
import org.apache.hadoop.mapred.lib.CombineFileSplit;
public class CombinedInputFormat<K, V> extends CombineFileInputFormat<K, V> {
public static class MyKeyValueLineRecordReader implements RecordReader<LongWritable,Text> {
private final RecordReader<LongWritable,Text> delegate;
public MyKeyValueLineRecordReader(CombineFileSplit split, Configuration conf, Reporter reporter, Integer idx) throws IOException {
FileSplit fileSplit = new FileSplit(split.getPath(idx), split.getOffset(idx), split.getLength(idx), split.getLocations());
delegate = new LineRecordReader(conf, fileSplit);
}
#Override
public boolean next(LongWritable key, Text value) throws IOException {
return delegate.next(key, value);
}
#Override
public LongWritable createKey() {
return delegate.createKey();
}
#Override
public Text createValue() {
return delegate.createValue();
}
#Override
public long getPos() throws IOException {
return delegate.getPos();
}
#Override
public void close() throws IOException {
delegate.close();
}
#Override
public float getProgress() throws IOException {
return delegate.getProgress();
}
}
#Override
public RecordReader getRecordReader(InputSplit split, JobConf job, Reporter reporter) throws IOException {
return new CombineFileRecordReader(job, (CombineFileSplit) split, reporter, (Class) MyKeyValueLineRecordReader.class);
}
}
Then you need to extend the TextLine class and make it use your own input format you just defined (Scala code from now on).
import cascading.scheme.hadoop.TextLine
import cascading.flow.FlowProcess
import org.apache.hadoop.mapred.{OutputCollector, RecordReader, JobConf}
import cascading.tap.Tap
import com.twitter.scalding.{FixedPathSource, TextLineScheme}
import cascading.scheme.Scheme
class CombineFileTextLine extends TextLine{
override def sourceConfInit(flowProcess: FlowProcess[JobConf], tap: Tap[JobConf, RecordReader[_, _], OutputCollector[_, _]], conf: JobConf) {
super.sourceConfInit(flowProcess, tap, conf)
conf.setInputFormat(classOf[CombinedInputFormat[String, String]])
}
}
Create a scheme for the for your combined input.
trait CombineFileTextLineScheme extends TextLineScheme{
override def hdfsScheme = new CombineFileTextLine().asInstanceOf[Scheme[JobConf,RecordReader[_,_],OutputCollector[_,_],_,_]]
}
Finally, create your source class:
case class CombineFileMultipleTextLine(p : String*) extends FixedPathSource(p :_*) with CombineFileTextLineScheme
If you want to use a single path instead of multiple ones, the change to your source class is trivial.
I hope that helps.
this should do the trick, ya man? - https://wiki.apache.org/hadoop/HowManyMapsAndReduces

Play 2.0! [Java] - Generating XML response from the REST API

I am parsing through Play framework documents and trying to figure out if there is anything out of the box available for generating XML response from the given domain object, just like how we have for Json.toJson(Object).
The following code works fine for Json REST API in play framework 2.1.2, can anyone suggest how can XML be generated out of the box here instead of Json?
package controllers;
import java.util.List;
import java.util.concurrent.Callable;
import play.Logger;
import play.libs.F.Function;
import play.libs.F.Promise;
import play.libs.Json;
import play.mvc.Controller;
import play.mvc.Result;
import com.amazonaws.services.simpledb.model.Item;
public class ShowItemsJson extends Controller {
public static Result allItems() {
// Now create the async process to lookup items in simpledb
AllItems<List<Item>> callable = new AllItems<List<Item>>();
Promise<List<Item>> promise = play.libs.Akka.future(callable);
return async(promise.map(new Function<List<Item>, Result>() {
public Result apply(List<Item> rm) throws Throwable {
// Convert the result into json before sending.
// TODO How to do same for XML?
return ok(Json.toJson(rm));
}
}));
}
// One instance of this class should be used for each create request
static class AllItems<V> implements Callable<V> {
#SuppressWarnings("unchecked")
public V call() throws Exception {
try {
return (V) Test.getAllItems();
} catch (Error e) {
// Error is handled here to log NoClassDefFoundError
Logger.error("Error: ", e);
throw e;
}
}
}
}
There is no built in support for generating XML from Java objects in Play 2 as far as I know, there are loads of options in Java-land though.
To name a few:
JAXB for doing it with reflection/annotations - reference implementation http://jaxb.java.net
xom - http://www.xom.nu
jdom - http://www.jdom.org,
dom4j - http://dom4j.sourceforge.net

How to resolve java.net.SocketException Permission Denied error?

I have already included the Internet Permissions in the andoird manifest page, still the error seems to persist. I am also recieveing an unknown host excption in the similar code. Kindly guide me through! Ty! :)
package name.id;
import android.app.Activity;
import android.content.DialogInterface.OnClickListener;
import android.os.Bundle;
import org.ksoap2.*;
import org.ksoap2.serialization.*;
import org.ksoap2.transport.*;
import android.view.View;
import android.widget.Button;
import android.widget.EditText;
import android.widget.TextView;
public class FirstPage extends Activity implements android.view.View.OnClickListener
{
/** Called when the activity is first created. */
TextView tv;
Button bu;
EditText et;
private static final String SOAP_ACTION="http://lthed.com/GetFullNamefromUserID";
private static final String METHOD_NAME="GetFullNamefromUserID";
private static final String NAMESPACE="http://lthed.com";
private static final String URL="http://vpwdvws09/services/DirectoryService.asmx";
#Override
public void onCreate(Bundle savedInstanceState)
{
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
et=(EditText) findViewById(R.id.editText1);
tv=(TextView) findViewById(R.id.textView2);
bu=(Button) findViewById(R.id.button1);
bu.setOnClickListener((android.view.View.OnClickListener) this);
tv.setText("");
}
public void onClick(View v)
{
// creating the soap request and all its paramaters
SoapObject request= new SoapObject(NAMESPACE, METHOD_NAME);
//request.addProperty("ID","89385815");// hardcoding some random value
//we can also take in a value in a var and pass it there
//set soap envelope,set to dotnet and set output
SoapSerializationEnvelope soapEnvelope = new SoapSerializationEnvelope(SoapEnvelope.VER11);
soapEnvelope.dotNet=true;
soapEnvelope.setOutputSoapObject(request);
HttpTransportSE obj = new HttpTransportSE(URL);
//to make call to server
try
{
obj.call(SOAP_ACTION,soapEnvelope);// in out parameter
SoapPrimitive resultString=(SoapPrimitive)soapEnvelope.getResponse();
//soapObject or soapString
tv.setText("Status : " + resultString);
}
catch(Exception e)
{
tv.setText("Error : " + e.toString());
//e.printStackTrace();
}
}
}
Have you added Internet permission to your manifest:
<uses-permission android:name="android.permission.INTERNET"/>
The error was being thrown because of an error in the URL. The URL needed my IP address to function properly rather than the URL i was providing because that the webservice was not able to understand.
Depending on what KSOAP2 version you are using error may vary.
Its recommend that use KSOAP2 v3+ lib.
Simply use following code to override...
StrictMode.ThreadPolicy policy = new StrictMode.ThreadPolicy.Builder().permitAll().build();
StrictMode.setThreadPolicy(policy);
and also use...