JBoss AS 7.1 - Request Parameters encoding - jboss

I have everything in UTF-8. That includes Content-Type, database, files, java, everything (unless I've missed something).
I follow a lot of stackoverflow answers, JIRAs, blogs, and etc, but, it still failing.
The problem itself is the following:
When I submit, let's suppose, to http://localhost:8080/app/searh?text=café, debugging, my request.getParameter("text") is always wrong, something like café, and request.getCharachterEncoding() gives me null (?).
Looking at the request headers, I got this:
GET http://localhost:8080/app/search?text=caf%C3%A9 HTTP/1.1
Host: localhost:8080
Proxy-Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.69 Safari/537.17
Referer: http://localhost:8080/app/search?text=n%C3%A3o
Accept-Encoding: gzip,deflate,sdch
Accept-Language: pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4
Accept-Charset: UTF-8,*;q=0.5
Cookie: JSESSIONID=OMMITED
And the response headers:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Pragma: No-cache
Cache-Control: no-cache
Expires: Wed, 31 Dec 1969 21:00:00 BRT
Content-Type: text/html;charset=UTF-8
Transfer-Encoding: chunked
Content-Encoding: gzip
Vary: Accept-Encoding
Date: Tue, 19 Mar 2013 14:06:24 GMT
Proxy-Connection: keep-alive
Connection: keep-alive
It's everything UTF-8. I just don't understand.
I tried to pass -Dfile.encoding=UTF-8 -Dfile.io.encoding=UTF-8 -DjavaEncoding=UTF-8 in my standalone.conf JAVA_OPTS variable, tried to put
<property name="org.apache.catalina.connector.URI_ENCODING" value="UTF-8"/>
<property name="org.apache.catalina.connector.USE_BODY_ENCODING_FOR_QUERY_STRING" value="true"/>
in my standalone.xml. Nothing of this solves the issue.
What can I try to do to fix this?
Thanks in advance.
BTW: Is a JBoss AS 7.1.1.

I came into the same issue but in Jboss 5.1, and I solved it adding the URIEncoding attribute to the HTTP Connector (in jbossweb/server.xml) and decoding the URL/GET parameters manually.
But the way to define it in Jboss7 is different from previous versions, but googling a bit I found this link: basically you've to add the following lines in the standalone.xml or domain.xml file after the end of the </extensions> tag (it looks like you've already done this step ;-):
<system-properties>
     <property name="org.apache.catalina.connector.URI_ENCODING" value="UTF-8"/>
     <property name="org.apache.catalina.connector.USE_BODY_ENCODING_FOR_QUERY_STRING" value="true"/>
</system-properties>
Moreover you've to decode manually the URI or parameter with the help of the java.net.URIEncoder object:
String param = URLDecoder.decode(request.getParameter("text"), "UTF-8");
BalusC has an interesing post in his blog about it.
And finally, a second solution, if you want to avoid using the previous options: have you considered using the POST method instead of the GET one?
I hope it helps.

I solved the issue creating a Filter that set both request and response encoding to UTF-8.
Pretty hacky, but works.

I don't have enough reputation to add a comment to the answer by Toni S. Magraner, so I'll write here.
request.getParameter("text") already does the URL decoding. Calling URLDecoder.decode() again will give double decoding which will probably not do what you want. An example:
logger.error("p1:"+request.getParameter("p1"));
Calling it with
http://localhost/test?p1=ku%2fki%44__%33X%C3%A9X
prints:
p1:ku/kiD__3XéX

The standalone.xml or domain.xml configuration doesn't work for me.
On jboss-as-7.1.1.Final just add this line to standalone.conf, this file lives under the directory bin:
JAVA_OPTS="$JAVA_OPTS -Dorg.apache.catalina.connector.URI_ENCODING=UTF-8"
from: JBOSS 7 encoding not working as expected

We had a similar problem in POST requests, with parameters in request body:
p1=v2&text=café
We solved client-side adding a header in request:
Content-Type: application/x-www-form-urlencoded; charset=UTF-8

Related

sapui5 OData.V4.oDataModel with Teiid 11.2 / Wildfly

I am trying to use SAPUI5 in a SPA to display data from a TEIID/Wildfly ODataV4 service. When SAPUI5 ODATA V4 data model is bound to the service, I run into several errors. I thereby connect via proxy (grunt-connect-proxy2) to the odata service. Basic Auth works. The metadata file above the marked service folder in the attached image (screenshot1) is the actual metadata file of the service which, as you can see, is loaded correctly. So no CORS issue or authorization issue.
Screenshot1
The issue seems to be related to an CSRF Token request as far as I understand. Seems that Teiid/Wildfly is not answering the CSRF Token fetch request. Is there a way to configure Wildfly to answer the request or alternatively a way to disable CSRF requests for the odata V4 model? I have seen such an option in the constructor of the odata V2 model. How could a working configuration look like?
The following is a screenshot from the browser log:
I observed a further issue, from which I do not know if it is related to the previous one (there is also a X-CSRF-Token: Fetch involved), or if a have something more missing somewhere. The second issue happens when I use an aggregation binding to bind a odata collection to a sapui5 list. The response looks like
Request URL: http://localhost:9001/odata4/svc/my_nutri_diary/$batch
Request Method: POST Status Code: 406 Not Acceptable Remote Address:
[::1]:9001 Referrer Policy: no-referrer-when-downgrade Response
Headersview source access-control-allow-credentials: true
access-control-allow-origin: http://localhost:9001 cache-control:
no-cache, no-store, must-revalidate connection: close
content-encoding: gzip content-length: 125 content-type:
application/json;odata.metadata=minimal date: Mon, 12 Nov 2018
20:04:30 GMT expires: 0 odata-version: 4.0 pragma: no-cache server:
WildFly/11 x-powered-by: Undertow/1 Request Headersview source Accept:
multipart/mixed Accept-Encoding: gzip, deflate, br Accept-Language: de
Authorization: Basic SU1TVXNlcjpJTVM0Zm9ydW0l Connection: keep-alive
Content-Length: 329 Content-Type: multipart/mixed;
boundary=batch_id-1542053070786-11 Cookie: sidebar_collapsed=false;
cycle_analytics_help_dismissed=1;
__utmz=111872281.1539128843.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=111872281.767670437.1539128843.1541866362.1541870562.42 DNT: 1 Host: localhost:9001 MIME-Version: 1.0 OData-MaxVersion: 4.0
OData-Version: 4.0 Origin: http://localhost:9001 Referer:
http://localhost:9001/ User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS
11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko)
Version/11.0 Mobile/15A372 Safari/604.1 X-CSRF-Token: Fetch
X-Requested-With: XMLHttpRequest Request Payload
--batch_id-1542053070786-11 Content-Type:application/http Content-Transfer-Encoding:binary GET Profile?$skip=0&$top=100 HTTP/1.1
Accept:application/json;odata.metadata=minimal;IEEE754Compatible=true
Accept-Language:de
Content-Type:application/json;charset=UTF-8;IEEE754Compatible=true
--batch_id-1542053070786-11--
Thanks for your advice!
Best regards,
Christoph
Further note regarding search for a workaround: As I am currently searching for a workaround to be able to use the odata.v4 model, I found the following blog post:
https://blogs.sap.com/2015/08/05/disable-csrf-token-for-odata-calls-using-sap-netweaver-gateway/
However, the approach does not seem to work for the odata.v4 model as it has a different interface. There is no setHeaders() function to set custom headers on the datamodel. I therefore tried to set the header up via
$.ajaxSetup({headers: {'X-Requested-With': 'X'}});
Unfortunately, this also does not work. If someone has the odata.v4 Model running with TEIID or Olingo v4 it would be great if he could give me a feedback on how he had worked around this issue.

Axis2 sending multipart response when content-type is "text/xml"

Axis2 by default is send a multipart response even when there are no attachments
Why is axis2 sending a multipart response & how to ask it to send a "application/xml" or "application/soap+xml" how to get rid of multipart?
In the axis2.xml I have
In the response I see
HTTP/1.1 200 OK
Date: Fri, 17 Feb 2017 01:07:08 GMT
Transfer-Encoding: chunked
Content-Type: multipart/related; boundary="MIMEBoundary_87162747c87b279f7caa4e1ab573d5d864a878de7fae1a0b"; type="application/xop+xml"; start="<0.97162747c87b279f7caa4e1ab573d5d864a878de7fae1a0b#apache.org>"; start-info="text/xml"
--MIMEBoundary_87162747c87b279f7caa4e1ab573d5d864a878de7fae1a0b
Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
Content-Transfer-Encoding: binary
Content-ID: <0.97162747c87b279f7caa4e1ab573d5d864a878de7fae1a0b#apache.org>
200<?xml version="1.0" encoding="UTF-8"?>
<List_Wrapper>
<_bp>
<_comments><_comment><content>Again a new comment this is a text type bp comment need to see this in the text json 1</content><published_date>2017-01-18T21:07:15</published_date><published_by>cyril furtado</published_by><company>Chevron Inc.</company></_comment></_comments></_bp>
</List_Wrapper>
--MIMEBoundary_87162747c87b279f7caa4e1ab573d5d864a878de7fae1a0b-
I too faced similar issue for AXIS 2 quite recently , so thought to answer this.
AXIS 2 by default support attachment processing globally, which might not be needed for all types of services. To resolve this issue I disabled MTOM processing globally by modifying axis2.xml file -
<parameter name="enableMTOM">false</parameter>
This capability can now be enabled on need basis per service basis by enabling this property via respective services.xml file

Downloading files from box.com using content api, GZIP

I have a 6mb txt file in Box.com site.
Now i would like to download the file using api. as it takes time to download, i would like to download it as a gzipped file.
As given here https://developers.box.com/docs/ where we have to add accept-encoding header with the values "gzip, deflate". I have added this header but the file is not downloaded as zip file it has the same size as 6mb, if it is zipped then it should be less than one mb in size.
But it is not happening. The following are the headers passed in REST request.
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36
Authorization: Bearer ACCESSTOKEN
Accept: */*
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8,te;q=0.6
The following are the response headers.
Server: nginx
Date: Thu, 24 Jul 2014 16:24:56 GMT
Content-Type: application/octet-stream
Content-Length: 6685772
Connection: keep-alive
Cache-control: private
Accept-Ranges: bytes
Content-Disposition: attachment;filename="abc.log";filename*=UTF-8''
X-Content-Type-Options: nosniff
Accept-Ranges: bytes
Is there anything that I missed here?
I have run into the same problem. However, I have also realized why this may be happening. Quoting the Box SDK documentation:
If the file is available to be downloaded, the response will be a 302
Found to a URL at dl.boxcloud.com. The dl.boxcloud.com URL is not
persistent. Clients will need to follow the redirect in order to
actually download the file. The raw data of the file is returned
unless the file ID is invalid or the user does not have access to it.
The important point to note is that there is a client redirect to a non-persistent URL in order to download the file. When we check the sequence of headers that are passed as a part of the redirect request, you can see that the Accept-Encoding: gzip deflate is missing. I used Fiddler to try this out, you can use any other HTTP proxy or interceptor.
This should be the reason why the files are not getting downloaded using the gzip encoding.
Hope this helps.

Fiddler not capturing WCF traffic from the web server to the application server

I have two possible flows:
ConsoleClient -(1)-> ApplicationServer
or
SilverlightClient -(2)-> WebServer -(3)-> ApplicationServer
Fiddler successfully captures the HTTP traffic on the (1) and the (2), but not on the (3). Here is a sample capture on (1):
POST /WcfDemo/ws HTTP/1.1
Content-Type: application/soap+xml; charset=utf-8
Host: il-mark-lt
Content-Length: 521
Expect: 100-continue
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing"><s:Header><a:Action s:mustUnderstand="1">http://tempuri.org/IWcfDemoService/Add</a:Action><a:MessageID>urn:uuid:d7fde351-12fd-4872-bc26-52ff97f126e9</a:MessageID><a:ReplyTo><a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address></a:ReplyTo><a:To s:mustUnderstand="1">http://il-mark-lt/WcfDemo/ws</a:To></s:Header><s:Body><Add xmlns="http://tempuri.org/"><x>4</x><y>5</y></Add></s:Body></s:Envelope>
HTTP/1.1 200 OK
Content-Length: 399
Content-Type: application/soap+xml; charset=utf-8
Server: Microsoft-HTTPAPI/2.0
Date: Sat, 17 Sep 2011 20:57:16 GMT
<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing"><s:Header><a:Action s:mustUnderstand="1">http://tempuri.org/IWcfDemoService/AddResponse</a:Action><a:RelatesTo>urn:uuid:d7fde351-12fd-4872-bc26-52ff97f126e9</a:RelatesTo></s:Header><s:Body><AddResponse xmlns="http://tempuri.org/"><AddResult>9</AddResult></AddResponse></s:Body></s:Envelope>
And here is an example of (2):
POST /WcfDemoService.svc/ws HTTP/1.1
Host: localhost:56970
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8,application/json
Accept-Language: fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.6,he-IL;q=0.5,he;q=0.4,ru-RU;q=0.3,ru;q=0.1
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Referer: http://localhost:56970/ClientBin/SilverlightClient.xap
Content-Length: 581
Content-Type: application/soap+xml; charset=utf-8
<s:Envelope xmlns:a="http://www.w3.org/2005/08/addressing" xmlns:s="http://www.w3.org/2003/05/soap-envelope"><s:Header><a:Action s:mustUnderstand="1">http://tempuri.org/IWcfDemoService2/Add</a:Action><a:MessageID>urn:uuid:e8420d3e-f568-49ce-bfc7-5631d5bf3fd0</a:MessageID><a:ReplyTo><a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address></a:ReplyTo><a:To s:mustUnderstand="1">http://localhost:56970/WcfDemoService.svc/ws</a:To></s:Header><s:Body><Add xmlns="http://tempuri.org/"><x>11</x><y>22</y><serverChannelKind>ws</serverChannelKind></Add></s:Body></s:Envelope>
HTTP/1.1 200 OK
Server: ASP.NET Development Server/10.0.0.0
Date: Sat, 17 Sep 2011 20:59:23 GMT
X-AspNet-Version: 4.0.30319
Content-Length: 401
Cache-Control: private
Content-Type: application/soap+xml; charset=utf-8
Connection: Close
<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing"><s:Header><a:Action s:mustUnderstand="1">http://tempuri.org/IWcfDemoService2/AddResponse</a:Action><a:RelatesTo>urn:uuid:e8420d3e-f568-49ce-bfc7-5631d5bf3fd0</a:RelatesTo></s:Header><s:Body><AddResponse xmlns="http://tempuri.org/"><AddResult>33</AddResult></AddResponse></s:Body></s:Envelope>
Now, I am absolutely sure the (3) does get through. So, it all boils down to some misconfiguration on the WebServer, but I cannot nail it. The Web server is just a trivial ASP.NET application hosted within IIS. It even has the following lines in the web.config:
<system.net>
<defaultProxy>
<proxy bypassonlocal="false" usesystemdefault="true" />
</defaultProxy>
</system.net>
Still, this does not work.
To further strengthen my suspicion on the web server configuration, I have checked the SilverlightClient --> ApplicationServer flow and it is captured just fine.
I am using the Asp.Net development server.
Edit
Running procmon reveals that the following suspicious registry key is consulted (amongst others):
HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\ProxyBypass
And on my machine it was set to 1. I have changed it to 0 and seems like it solved my issue. The only problem is that when I change it back to 1 Fiddler continues to capture the problematic leg! Very interesting.
Anyway, I am satisfied, for now.
You are calling "localhost" right?
Fiddler is not able to capture the local traffic if you are using "localhost" as hostname.
Solutions:
Use servername (e.g. myserver)
Use ip4.fiddler (e.g. http://ipv4.fiddler:8787)
Not sure if these are causing it ... but,
A few things to check:
In IIS7 the appPool has a loadUserProfile setting. It causes the session to load a user profile which means it can get system proxy settings.
Check the code making the request from the webServer - even if you configure to use the system proxy and bypass onLocal (which only applies to names without dots in it), code making the request can still explicitly set to use or not to use a proxy.
Far fetched but you may want to play with the account the appPool runs as - local account with profile vs. Network Service.
Hope that helps - these network things have a lot of variables between two points :)

Unable to assemble multipart/form-data request

I'm trying to implement file upload functionality in the iPhone app. Server code is tested and works when files are uploaded from the desktop browser, so I moved to implementing the Objective-C client code. I'm assembling HTTP requests body manually, and despite that it looks correct, it is rejected by the server (server handler unable to extract the parts from multipart content). In desperation I've simplified the form to having only one parameter, but it still does not work.
I've captured the network traffic and I could see that Wireshark could not parse my multipart content as well (have a look at screenshots: Firefox request, iPhone request). I'm pasting it below in hope that you could see the errors I can't see.
Thanks in advance.
Firefox:
POST /cubepaint/actions/gallery/post HTTP/1.1
Host: [...]
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-GB; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Authorization: Basic [...]
Content-Type: multipart/form-data; boundary=---------------------------20072377098235644401115438165
Content-Length: 180
-----------------------------20072377098235644401115438165
Content-Disposition: form-data; name="deviceId"
12345
-----------------------------20072377098235644401115438165--
HTTP/1.1 200 OK
Date: Sat, 17 Oct 2009 22:09:21 GMT
Server: Apache/2.2.3 (Debian) DAV/2 SVN/1.4.2 mod_python/3.2.10 Python/2.4.4 mod_ssl/2.2.3 OpenSSL/0.9.8c
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
iPhone:
POST /cubepaint/actions/gallery/post HTTP/1.1
Host: [...]
User-Agent: Copenhagen/1.0 CFNetwork/459 Darwin/9.8.0
Content-Type: multipart/form-data; boundary=----------0E7B16E6-CD3D-4213-9B42-07DA30822C74
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Authorization: Basic [...]
Content-Length: 187
Connection: keep-alive
----------0E7B16E6-CD3D-4213-9B42-07DA30822C74
Content-Disposition: form-data; name="deviceId"
00000000-0000-1000-8000-0016CBCC0B61
----------0E7B16E6-CD3D-4213-9B42-07DA30822C74--
HTTP/1.1 200 OK
Date: Sat, 17 Oct 2009 22:04:07 GMT
Server: Apache/2.2.3 (Debian) DAV/2 SVN/1.4.2 mod_python/3.2.10 Python/2.4.4 mod_ssl/2.2.3 OpenSSL/0.9.8c
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
Your iPhone version indicates keep-alive but doesn't specify a length. Not sure that's enough to cause trouble.
Also, is it possible your server is checking for user-agent strings it recognizes (say, for backward-compatibility mode)?
I'd also compare the two in a text editor that shows CR/LF characters to make sure you're getting proper line endings.
Another thing you could try is create a simple web-page that does a multipart POST and run it from the iPhone browser (instead of the Mac one) then check the headers that go across the wire. Or you could snag a toolkit like ASIHTTPRequest and see what kind of output it generates for multi-part posts (or just use the toolkit instead of trying to write your own).
Good luck
Solved by reading RFC 2046 (MIME specification): boundary between parts of multipart message should contain two leading '-'s, and last boundary should additionally contain two trailing '-'s. The boundary in the request header and request body in the Firefox request differ:
---------------------------20072377098235644401115438165
and
-----------------------------20072377098235644401115438165
The last boundary looks like this:
-----------------------------20072377098235644401115438165--
You really could not see this with the eye when there are so many leading '-'s in the original boundary.