I got a form in my application. I input there russian and chineese symbols. Http request from browser to application looks like this:
POST /payout/process HTTP/1.0
Host: my_host
X-Real-IP: 10.3.4.6
X-Forwarded-For: 10.3.4.6
X-Forwarded-Proto: https
Connection: close
Content-Length: 286
Accept: */*
Origin: https://my_host
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/43.0.2357.130 Chrome/43.0.2357.130 Safari/537.36
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Referer: https://my_host
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8,ru;q=0.6
Cookie: piwikExternalId=; COOKIEPOO="d14ef459332e84ab12f8187181e01e50df3abd61-5d635c8478334352d2865b11dd747d6d3fe84704=1&d6e2c1ad27a45b7d9b8ed7b039ab96c7d3ef8051=1&7cf1199693fa6d40bf3654db17ec4773e0d61f38=1"
group=27&qwipi.customer_name=%D1%84%D1%8B%D0%B2&qwipi.bank_code=%E6%8B%9B%E5%95%86%E9%93%B6%E8%A1%8C&qwipi.bank_branch=%D1%84%D0%B2%D0%B0&customer_purse=%D1%84%D1%8B%D0%B2%D0%B0&qwipi.bank_city=%D1%84%D0%B2%D1%8B%D0%B0&qwipi.bank_province=%E6%8B%9B%E5%95%86%E9%93%B6%E8%A1%8C&amount=546
Check out request body, parameter customer_name. Decoded it will give фыв. Chec out paramter bank_branch. Decoded it will give 招商银行.
I log the request to console and graylog like this:
def logRequest(request: Request[AnyContent]) {
request.body match {
case AnyContentAsFormUrlEncoded(data) => Logger.info(data.toString())
case AnyContentAsText(text) => Logger.info(text + ", ")
case other => Logger.info(other.toString())
}
}
and I get something like
body: 'qwipi.bank_code' => '??????', 'qwipi.bank_province' => '??????', 'qwipi.customer_name' => '??????', 'amount' => '654', 'qwipi.bank_branch' => '??????', 'qwipi.bank_city' => '??????'
i.e., all non-latin symbols turn to question signs.
echo $LANG outputs en_US.UTF-8, so I guess my application's encoding (whatever that means -- application's itself or java's or playframework's or smth else) is different.
This is what I tried:
def logRequest(request: Request[AnyContent]) {
Logger.info(new String(appendRequestData(request).getBytes(StandardCharsets.UTF_8), StandardCharsets.UTF_8))
}
i.e., got the byte array in UTF-8 and turn it into the String which is in UTF-8 as well. But nothing changed.
What I also tried was starting an application passing -Dfile.encoding=UTF8 key -- the result still the same.
So how can I make my stuff logged properly? What things I might have missed?
It didn't help converting my string from one encoding to another on the fly. It didn't help passing UTF-8 to scalac option in Build.scala like that
So I just passed -Dfile.encoding=UTF-8 when starting my java service.
Related
Please help. I'm trying to learn REST concepts and make the first programs using Delphi and REST objects. I came across a problem that I don’t know how to solve. In the database I have a test with special characters from my mother tongue (Bosnian): č, ć, đ, š, ž. When I pass this text via the GET method, objects that parse and display this text in objects on the form display these characters as "?", "æ", "æè" etc.
I researched and tried unsuccessfully to solve the problem. I tried to use the Utf8ToAnsi function and put iso-8859-2 instead of UTF-8 in the RESTClient, RESTRequest and RESTResponce object parameters.
Please for help, instruction or advice on how to solve or investigate this problem.
Source code on server side:
procedure TWebModule1.UsersGet(Request: TWebRequest; Response: TWebResponse);
var
a: TJSONArray;
o: TJSONObject;
i: Integer;
Q1: TADOQuery;
begin
Q1:=TADOQuery.Create(nil);
Q1.Connection:= BasicDBConn.Konekcija;
with Q1 do
begin
Active:=False;
SQL.Clear;
SQL.Add('Select USER_ID, USER_NAME, ROLE_NAME from USERS, ROLES ' +
' where USERS.ROLE_ID=ROLES.ROLE_ID ');
Active:= True;
end;
a := TJSONArray.Create;
if Q1.RecordCount > 0 then
begin
for i:=1 to Q1.RecordCount do
begin
o := TJSONObject.Create;
o.AddPair('USER_ID', Q1.Fields.Fields[0].Value);
o.AddPair('USER_NAME', Q1.Fields.Fields[1].Value);
o.AddPair('ROLE_NAME', Q1.Fields.Fields[2].Value);
Q1.Next;
a.AddElement(o);
end;
end;
Response.ContentType := 'application/json';
Response.Content := a.ToString;
a.DisposeOf;
Q1.Free;
end;
on client side:
procedure TGlavna.Button1Click(Sender: TObject);
begin
RESTRequest.Resource := 'Users';
RESTRequest.Method := TRESTRequestMethod.rmGet;
RESTRequest.Response := RESTResponse;
RESTRequest.Execute;
if assigned(fJSONArray) then fJSONArray.DisposeOf;
fJSONArray := TJSONObject.ParseJSONValue(RESTResponse.Content) as TJSONArray;
if RESTResponse.Content.IsEmpty then Application.MessageBox('Empty', 'Information', MB_OK)
else Memo1.Lines.Add(RESTResponse.Content);
end;
This is what a data in DB looks like
USER_ID USER_NAME ROLE_ID
u1 Test User 2
u2 T_č_ć_š_đ_ž 1
u3 t_Č_Ć_Ž_Đ_Š 1
u4 Bradić Kenan 1
In web browser, responce data is:
[{"USER_ID":"u1","USER_NAME":"Test User","ROLE_ID":"2"},{"USER_ID":"u3","USER_NAME":"t_È_Æ_Ž_Ð_Š","ROLE_ID":"1"},{"USER_ID":"u4","USER_NAME":"Bradiæ Kenan","ROLE_ID":"1"},{"USER_ID":"u2","USER_NAME":"T_è_æ_š_ð_ž","ROLE_ID":"1"}]
Headers in web browser:
General:
Request URL: http://localhost:8080/Users
Request Method: GET
Status Code: 200 OK
Remote Address: [::1]:8080
Referrer Policy: strict-origin-when-cross-origin
Response Headers:
Connection: close
Content-Length: 228
Content-Type: application/json; charset=ISO-8859-1
Date: Mon, 27 Dec 2021 17:07:54 GMT
Request Headers:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Host: localhost:8080
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="96", "Google Chrome";v="96"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36
Thank you...
The HTTP response contains this line:
Content-Type: application/json; charset=ISO-8859-1
So the client handles interprets the Response content as ISO-85591 encoded instead of UTF-8.
Your server side codes contains this line:
Response.ContentType := 'application/json';
Your server side code should clear the charset=ISO-85591 part.
You can encode your value with base64, and create a JSON payload to translate the JSON payload with REST API.
Moreover, if the value is sensitive, you can encrypt it with AES, and then translate it with REST API.
What technology will you use in server side? PHP? or .NET or JAVA? All 3 technologies used AES in native code, but there are little differences between the technologies.
If you used PHP or PHP + Laravel, you might reference my sample project in my GitHub Repo, the description you can read from my article.
I have a problem with my requests using Edge/IE. I'm getting a code 400 code on
preflight OPTIONS.
I'm adding a 2 additional headers to all my requests in application with axios:
Authorization:
if ( token ) {
axios.defaults.headers.common[ 'Authorization' ] = 'Bearer ' + token;
} else {
delete axios.defaults.headers.common[ 'Authorization' ];
}
Accept-Language:
if ( lang ) {
axios.defaults.headers.common[ 'Accept-Language' ] = lang;
} else {
delete axios.defaults.headers.common[ 'Accept-Language' ];
}
Here's how my request headers works in Chrome:
Accept-Language: pl-PL
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJOZXcgUGFzdG9yYWwgQXBwIiwiYXVkIjoiTmV3IHBhc3RvcmFsIHVzZXIiLCJpYXQiOjE1MTM1OTEzODUsImV4cCI6MTUxMzU5ODU4NSwidWlkIjoxMSwidHlwZSI6NCwibGFuZ3VhZ2UiOjEsImRpc3BsYXlOYW1lIjoiVGVzdG8gUHJvYm9zemN6eiJ9.qyFHZuCHC9-NpNqjY7keY7j4LE5zu5JKxK9VyAFwH60
Request headers in Edge:
Accept: */*
Accept-Encoding: gzip, deflate
Access-Control-Request-Headers: content-type
Access-Control-Request-Method: POST
Cache-Control: no-cache
Connection: Keep-Alive
Content-Length: 0
Host: 52.169.186.226
Origin: http://localhost:8080
Referer: http://localhost:8080/
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299
Response headers:
Access-Control-Allow-Headers: X-Requested-With, Content-Type, Accept, Origin, Authorization
Access-Control-Allow-Methods: OPTIONS,POST
Access-Control-Allow-Origin: *
Connection: close
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Date: Mon, 18 Dec 2017 11:29:49 GMT
Server: Apache/2.4.25 (Ubuntu)
When I remove Accept-Language header the application start working in Edge, but problem still exists in IE.
The only messages from browser is code http error 400 and the console log of error is:
description: "Network Error"
message: "Network Error"
stack: "Error: Network Error at createError (http://localhost:8080/index_bundle.js:68811:3) at handleError (http://localhost:8080/index_bundle.js:68656:7)"
Rest of the headers is standard for axios. On the backend we're using Slimp PHP framework.
Any idea how to debug the problem, or similar problems using standard REST requestes with React + Axios? Or what should I check in first order?
I am trying to update database using SLIM php. But the form data values passed in are showing blank. I have been successful in posting (customPOST) the values.
Here is the code:
AngularJs:
UpdateCategoryFormData:function(category,catId){
formData=new FormData();
angular.forEach(category, function (fieldData, field) {
formData.append(field, fieldData);
});
return Restangular.one('category/update',catId)
.withHttpConfig({transformRequest: angular.identity})
.customPUT(formData, undefined, {}, {'Content-Type': undefined});
}
SLIM php:
$app->put('/category/update/:catId',function($catId) use($app){
$data=$app->request->put();
print_r($data);
$catName=$app->request->put('catName');
$catParent=$app->request->put('catParent');
Result is:
array()
echo $catName;
I am getting a blank as the value of $catName;
Here is the copy of the headers:
Request Method:PUT
Status Code:200 OK
Response Headers
view source
Connection:Keep-Alive
Content-Length:10
Content-Type:text/html
Date:Fri, 28 Aug 2015 03:48:28 GMT
Keep-Alive:timeout=5, max=100
Server:Apache/2.4.9 (Win32) PHP/5.5.12
X-Powered-By:PHP/5.5.12
Request Headers
view source
Accept:application/json, text/plain, */*
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Content-Length:249
Content-Type:multipart/form-data; boundary=----WebKitFormBoundary4gUX2gCt7zCICZNN
User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2489.0 Safari/537.36
Request Payload
------WebKitFormBoundary4gUX2gCt7zCICZNN
Content-Disposition: form-data; name="catName"
Some Category
------WebKitFormBoundary4gUX2gCt7zCICZNN
Content-Disposition: form-data; name="catParent"
45
------WebKitFormBoundary4gUX2gCt7zCICZNN--
Can someone help pls!
Attempting to get a spray endpoint negotiating a postman request for content. My XML marshaller seems to let me down, ie, it never resolves the correct marshaller option based on Accept header and charset.
I have the following:
object ResponseVO {
val NodeSeqMarshaller = Marshaller.delegate[ResponseVO, NodeSeq](ContentType(`text/xml`, `UTF-8`)) { respVO => <ussd><type>{ respVO.respType }</type><message>{ respVO.message }</message></ussd> }
val supportedContentTypes = List[ContentType](ContentType(`text/xml`, `UTF-8`))
implicit val marshaller = Marshaller[ResponseVO] { (respVO, ctx) =>
ctx.tryAccept(supportedContentTypes) match {
case Some(ContentType(`text/xml`, `UTF-8`)) => NodeSeqMarshaller(respVO, ctx)
case whatever => println(whatever); ctx.rejectMarshalling(supportedContentTypes);
}
}
}
and the following route:
trait USSDRoute {
this: SimpleRoutingApp with BootStrappedActorSystem =>
val ussdRoute = path("ussd") {
parameters('msisdn.as[Long], 'session.as[String], 'type.as[Int], 'msg.as[String], 'network.as[Int]) { (msisdn, session, reqType, msg, network) =>
complete {
val reqVO = RequestVO(msisdn, session, reqType, msg, network)
println(s"received $reqVO")
ResponseVO(1, "Spirit midget medium escapes from prison, headlines read: Small medium at large!")
}
}
}
}
Unfortunately I never seem to negotiate correctly, ie, my marshaller resolution will dive bomb into the "whatever" block and respond with a 406
"Resource representation is only available with these Content-Types:
text/xml; charset=UTF-8"
I am using Postman and my request headers read:
GET /ussd?msisdn=0794138690&type=1&network=4&msg=hello&session=99 HTTP/1.1
Host: localhost:9999
Connection: keep-alive
Accept: text/xml; charset=UTF-8
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML,
like Gecko) Chrome/40.0.2214.91 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,nl;q=0.6
It's probably something small and silly - hopefully someone can help steer me in the right direction.
Thank you
Have a look at how the case class ContentType is defined:
case class ContentType(mediaType: MediaType, definedCharset: Option[HttpCharset])
Note that definedCharset is defined as an Option[HttpCharset], therefore your pattern match via
case Some(ContentType(`text/xml`, `UTF-8`)) => ...
never can succeed. So you need to use a Some to perform the pattern match successfully:
case Some(ContentType(`text/xml`, Some(`UTF-8`))) => ...
I'm running into a problem with a 302 redirect after my CORS preflight has successfully returned a 200 status. I'm currently building an app using Laravel 4.1 and Angular 1.2 as well as my own OAUTH2 server.
The error that Chrome/FF/Safari are sending back to me is:
XMLHttpRequest cannot load http://localhost.api/api/v1/tracks?$filter=id%20eq%20guid%27d7de10ba-e353-455b-a3cb-ced9b4965141%27&. The request was redirected to 'http://localhost.api/session/invalid', which is disallowed for cross-origin requests that require preflight.
I assume whats causing my issue is the "redirect" thats happening - as my normal CORS requests all work as expected.
My configuration for my Access-Control-Allow-* headers is fairly open right now during testing.
'paths' => array(
'^/api/' => array(
'allow_origin'=> array('*'),
'allow_headers'=> array('Content-Type', 'Authorization'),
'allow_methods'=> array('POST', 'PUT', 'GET', 'DELETE', 'OPTIONS'),
'max_age' => 0
),
'^/session/' => array(
'allow_origin'=> array('*'),
'allow_headers'=> array('Content-Type', 'Authorization'),
'allow_methods'=> array('GET', 'OPTIONS'),
'max_age' => 0
)
The redirect in question is in a pre-filter thats checking the validity of an OAUTH2 access token
public function filter($route, $request, $data = null)
{
// Get the authorization header or fail
if ($authorization = Request::header('Authorization', false)) {
list($type, $token) = explode(' ', $authorization);
if (is_null($auth = OAuth2::token($token)->first())) {
return Redirect::to('session/invalid');
}
$tokenExpiryDate = Carbon::createFromTimeStamp($auth->access_token_expires);
// If we don't have a Bearer authentication header
// or if the token has expired. Then redirect to an
// expired session route
if ( 'bearer' != strtolower($type)
|| Carbon::now()->gt($tokenExpiryDate)
) {
return Redirect::route(
'expiredSession',
array('expiry' => $tokenExpiryDate->timestamp)
);
}
} else {
// The authentication header is invalid, redirect to let the user know.
return Redirect::to('session/invalid');
}
}
All these requests worked when I tried debugging using POSTMAN, but after my research I've basically found that extensions don't necessarily have to play by the same rules. As well as I noticed my requests using POSTMAN never send any preflight OPTIONS requests when I initiate a simple GET, POST, etc..
Here are the headers of both an OPTIONS request as well as a GET request to follow it that is returning my error
OPTIONS Request
Remote Address:127.0.0.1:80
Request URL:http://localhost.api/api/v1/tracks?$filter=id%20eq%20guid%27d7de10ba-e353-455b-a3cb-ced9b4965141%27&
Request Method:OPTIONS
Status Code:200 OK
Request Headers
Accept:*/*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Access-Control-Request-Headers:accept, authorization
Access-Control-Request-Method:GET
Cache-Control:no-cache
Connection:keep-alive
DNT:1
Host:vegas.ine.com
Origin:http://localhost.angular
Pragma:no-cache
Referer:http://localhost.angular/admin/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36
Response Headers
Access-Control-Allow-Headers:content-type, authorization
Access-Control-Allow-Methods:POST, PUT, GET, DELETE, OPTIONS
Access-Control-Allow-Origin:http://localhost.angular
Cache-Control:no-cache
Connection:close
Content-Type:text/html
Date:Thu, 01 May 2014 16:22:19 GMT
Server:Apache/2.2.26 (Unix) DAV/2 PHP/5.4.24 mod_ssl/2.2.26 OpenSSL/0.9.8y
Set-Cookie:laravel_session=eyJpdiI6IktOZjlTM1ZVNUx0TEhoaTczY3dQcDBKRWlvbnppbDA3QTdqSENJdTc2R1U9IiwidmFsdWUiOiJEZ2ltXC9mNm1Qa20rV3BVRlNHTXgySGtUeVlpNjNZcGFudDFBWDJJekl1MEVNVlhSRE5WWk5YZDNxUkZuU0VEVytcL3NLNlVBXC9hZWtJQzdHU2FqVWtMdz09IiwibWFjIjoiYTYxYjEwNjlmYmI2MjMwNmE4MzlkYjIwNGZlNzA4Y2ViZGVkZmU1MTQzMzc5NmU2YzI2ZGExNzYxY2U5ZjdiMCJ9; expires=Thu, 01-May-2014 18:22:19 GMT; path=/; httponly
X-Frame-Options:SAMEORIGIN
X-Powered-By:PHP/5.4.24
GET Request
Remote Address:127.0.0.1:80
Request URL:http://localhost.api/api/v1/tracks?$filter=id%20eq%20guid%27d7de10ba-e353-455b-a3cb-ced9b4965141%27&
Request Method:GET
Status Code:302 Found
Request Headers
Accept:application/json, text/plain, */*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Authorization:Bearer 6Ss4XPrPM5jQD7Es0dz7TPRQ76hGA69vT9K94pst
Cache-Control:no-cache
Connection:keep-alive
DNT:1
Host:vegas.ine.com
Origin:http://localhost.angular
Pragma:no-cache
Referer:http://localhost.angular/admin/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36
Response Headers
Access-Control-Allow-Origin:http://localhost.angular
Cache-Control:no-cache
Connection:Keep-Alive
Content-Type:text/html; charset=UTF-8
Date:Thu, 01 May 2014 16:22:19 GMT
Keep-Alive:timeout=5, max=100
Location:http://localhost.api/session/invalid
Server:Apache/2.2.26 (Unix) DAV/2 PHP/5.4.24 mod_ssl/2.2.26 OpenSSL/0.9.8y
Set-Cookie:laravel_session=eyJpdiI6InlnREVPcUJTcyswMnRLanFDSlZ6QWFBVXZWMGdMNVNLYWxNTHRJVUlkalk9IiwidmFsdWUiOiJ4aXN5U0dcL1NYeGQrcUVzWFhYV3o2MWhcL25hQTlhcVUxbWxkN2R6SG9KZDNKaGNLTkRQY2FyTitpVHNGZzYxVVRtZUhoZGZRWE9GWjZRaDd1VVwvZUZuUT09IiwibWFjIjoiY2EzZTViZGIzZmVlMDcwZjdhMzBjOWQxYTgwZWNlYTJiMDk3ODdlZTk3NTYxMDNmM2YyODJjOGIxMzBmMmJlMiJ9; expires=Thu, 01-May-2014 18:22:20 GMT; path=/; httponly
Transfer-Encoding:chunked
Vary:Authorization
X-Clockwork-Id:1398961340.2239.1349476325
X-Clockwork-Version:1.5
X-Frame-Options:SAMEORIGIN
X-Powered-By:PHP/5.4.24
I have done something like this and it worked fine for me
//pattern to allow origins
$allowedOriginPattern = /** YOUR PATTERNS **/;
$allowedOrigin = "";
if (preg_match($allowedOriginPattern, $_SERVER['HTTP_ORIGIN'])) {
$allowedOrigin = $_SERVER['HTTP_ORIGIN'];
}
/**
* set http content type
*/
header('Content-Type: application/json;charset=UTF-8');
header('Access-Control-Allow-Origin: ' . $allowedOrigin);
header('Access-Control-Allow-Methods: DELETE, HEAD, GET, OPTIONS, POST, PUT');
header('Access-Control-Allow-Headers: Content-Type, Content-Range, Content-Disposition, Content-Description');
header('Access-Control-Max-Age: 1728000');
I have added this code in laravel index.php
Here is the reference for CORS
http://www.w3.org/TR/cors/#cross-origin-request-with-preflight-0
Spent an hour and this trying to redirect to a subdomain
Instead just used javascript after a successful response
window.location.href = resp.data.redirect