Azure Function Request POST Body Encoding - encoding

I'm wondering whether encoding is something I have to explicitly handle in an Azure function (v3) triggered by an http POST.
For example... which approach is correct (s1, s2, or s3):
[Function("MyFancyFunction")]
public async Task<HttpResponseData> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "post")] HttpRequestData req,
FunctionContext executionContext)
{
// What charset/encoding will be used if not specified?
var s1 = req.ReadAsString();
// Surely it can't be save to assume UTF8?
var s2 = req.ReadAsString(Encoding.UTF8);
// Use the charset of the first content-type
var ct = MediaTypeHeaderValue.Parse(req.Headers.GetValues("content-type").First());
var s3 = req.ReadAsString(ct.Encoding);
...
}
Thanks!

I put together a simple test function and simple http POST console app.
The console app explicitly encodes the POST payload as Win-1252 (only chosen because it has characters from 0-255). The payload includes a 0x0080 character (utf-8 start sequence) which should cause problems if not decoded using the appropriate encoding.
Here's what the POST payload program looks like:
var client = new HttpClient();
var req = new HttpRequestMessage(HttpMethod.Post, " http://localhost:7071/api/Function1");
req.Content = new StringContent("d\u0080d");
req.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");
req.Content.Headers.ContentEncoding.Add("win-1252");
client.Send(req);
The test function looks like:
[Function("Function1")]
public static HttpResponseData Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestData req,
FunctionContext executionContext)
{
var s1 = req.ReadAsString(); // s1 == "d\u0080d"... yippie!
var response = req.CreateResponse(HttpStatusCode.OK);
return response;
}
The end result is that YES Azure functions will pay attention to a content-type's charset instruction and use the appropriate Encoding.

which approach is correct (s1, s2, or s3)
Below is the sample code which I used in my environment for encoding data and it worked for me rather than these 3 approaches I suggest to use the below given code.
public static async Task<string> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest req,
ILogger log)
{
log.LogInformation("SendMessage function requested");
string body = string.Empty;
using (var reader = new StreamReader(req.Body, Encoding.UTF8))
{
body = await reader.ReadToEndAsync();
log.LogInformation($"Message body : {body}");
}
log.LogInformation($"SendMessage processed.");
return body;
}
And here is the Output with the Encoded Data

Related

Token mismatch exception when deploying in any VM

Here is my csrf and cors handler of my vertx application
#Log4j2
public class CsrfVerticle extends AbstractVerticle {
private final Set<HttpMethod> httpMethodSet =
new HashSet<>(Arrays.asList(GET, POST, OPTIONS, PUT, DELETE, HEAD));
private final Set<String> headerSet = new HashSet<>(
Arrays.asList("Content-Type", "Authorization", "Origin", "Accept", "X-Requested-With",
"Cookie", "X-XSRF-TOKEN"));
private Connection dbConnection;
private WebClient webClient;
private Vertx vertx;
public void start() throws Exception {
super.start();
HttpServer httpServer = TestService.vertx.createHttpServer();
Router router = Router.router(TestService.vertx);
SessionStore store = LocalSessionStore.create(vertx);
SessionHandler sessionHandler = SessionHandler.create(store)
.setCookieSameSite(CookieSameSite.STRICT)
.setCookieHttpOnlyFlag(false);
router.route().handler(LoggerHandler.create());
if (TestService.serviceConfiguration.isEnableCSRF()) {
router.route()
.handler(CorsHandler.create("*").allowedMethods(httpMethodSet).allowedHeaders(headerSet)
.allowCredentials(true).addOrigin(TestService.serviceConfiguration.getFrontendUrl()));
router.route().handler(
CSRFHandler.create(vertx, csrfSecret()).setCookieHttpOnly(false))
.handler(sessionHandler);
} else {
router.route()
.handler(CorsHandler.create("*").allowedMethods(httpMethodSet).allowedHeaders(headerSet)
.allowCredentials(true)).handler(sessionHandler);
}
dbConnection = createConnection(TestService.serviceConfiguration.getJdbcConfig());
TestAuth testAuth = new TestAuth(TestService.serviceConfiguration.getUsername(),
TestService.serviceConfiguration.getPassword());
AuthenticationHandler basicAuthHandler = BasicAuthHandler.create(testAuth);
router.route("/student/*").handler(basicAuthHandler);
router.route("/student/add").method(HttpMethod.POST).handler(this::handleAddUser);
router.route("/student/get").method(HttpMethod.GET).handler(this::handleGetUser);
router.route("/student/delete").method(HttpMethod.DELETE)
.handler(this::handleDeleteUser);
router.route("/student/update").method(HttpMethod.PUT).handler(this::handleUpdateUser);
httpServer.requestHandler(router).listen(TestService.serviceConfiguration.getPort());
log.info("Console Server Verticle Started Successfully. Listening to {} port",
TestService.serviceConfiguration.getPort());
}
I am able to receive cookies in browser and send it back along with updated X-XSRF-TOKEN attached to the header
Everything works fine in my local but when deploying in VM I get the below error for all post requests
ctx.fail(403, new IllegalArgumentException("Token signature does not match"));
from csrf handler of vertx.
Here are the frontend code to add x-xsrf-token when sending requests to backend
createXsrfHeader(headers: HttpHeaders) {
let xsrfToken = Cookies.get('XSRF-TOKEN')
let sessionToken = Cookies.get('vertx-web.session')
if(xsrfToken)
headers = headers.append('X-XSRF-TOKEN', xsrfToken);
// if(xsrfToken && sessionToken)
// headers = headers.append('Cookie', `XSRF-TOKEN=${xsrfToken}; vertx-web.session=${sessionToken}`);
return headers;
}
[Adding header to post request]
callPostRequest(subUrl: string,reqData: any) {
let headers = new HttpHeaders();
headers = this.createAuthorizationHeader(headers);
headers = this.createXsrfHeader(headers);
return this.http.post<any>(this.basicApiUrl+subUrl, reqData, {
headers: headers,
withCredentials : true
}).pipe(map(resData => {
// console.log(resData);
return resData;
}));
}
[Adding header to put request]
callPutRequest(subUrl: string,reqData: any) {
let headers = new HttpHeaders();
headers = this.createAuthorizationHeader(headers);
headers = this.createXsrfHeader(headers);
return this.http.put<any>(this.basicApiUrl+subUrl, reqData,{
headers: headers,
withCredentials : true
}).pipe(map(resData => {
// console.log(resData);
return resData;
}));
}
[Adding header to delete request]
callDeleteRequest(subUrl: string,reqData?: any) {
let headers = new HttpHeaders();
headers = this.createAuthorizationHeader(headers);
headers = this.createXsrfHeader(headers);
return this.http.delete<any>(this.basicApiUrl+subUrl, {
headers: headers,
withCredentials : true
}).pipe(map(resData => {
// console.log(resData);
return resData;
}));
}
Is there any ways to solve it.
I believe your problem is here:
router.route() // <--- HERE
.handler(
CSRFHandler.create(vertx, csrfSecret())
.setCookieHttpOnly(false))
.handler(sessionHandler);
You are telling the application to create a new CSRF token for each request that is happening, instead of being specific of which end points are really form-based endpoints.
Imagine the following, your form is on /student/form your browser may request:
/student/form (new CSRF token: OK)
/images/some-image-in-the-html.png (new CSRF token: probably Wrong)
/css/styles.css (new CSRF token: probably Wrong)
...
Now the issue is that the 1st call did correctly generated a token, but the following 2+ will generate new tokens too and these won't match the 1st so your tokens are always misaligned.
You probably need to be more specific with the resources you want to protect, from your code I am assuming that you probably want something like:
router.route(""/student/*") // <--- Froms are always here under
.handler(
CSRFHandler.create(vertx, csrfSecret())
.setCookieHttpOnly(false))
.handler(sessionHandler);
Be careful if calling other endpoints would affect the forms too. Note that you can add multiple handlers per route, so you can be more explicit with:
router.route(""/student/add")
// 1st always CSRF checks
.handler(
CSRFHandler.create(vertx, csrfSecret())
.setCookieHttpOnly(false))
// and now we the handler that will handle the form data
.handler(this::handleAddUser)

How to add content header to Flurl

I would like to know how to add a content header to a flurl-statement.
The onedrive implementation requires me to add a content-type header to the content, and tried every possible solution with no luck.
I'm forced to use the regular httpclient with the following code.
Public Async Function UploadFile(folder As String, filepath As String) As Task(Of Boolean) Implements ICloud.UploadFile
Dim data As Byte() = File.ReadAllBytes(filepath)
Dim uploadurl As String = "drive/items/" + folder + ":/" + Path.GetFileName(filepath) + ":/" + "content?access_token=" + Token.access_token
Using client As New HttpClient()
client.BaseAddress = New Uri(ApiUrl)
Dim request As HttpRequestMessage = New HttpRequestMessage(HttpMethod.Put, uploadurl)
request.Content = New ByteArrayContent(data)
request.Content.Headers.Add("Content-Type", "application/octet-stream")
request.Content.Headers.Add("Content-Length", data.Length)
Dim response = Await client.SendAsync(request)
Return response.IsSuccessStatusCode
End Using
End Function
I already tried the regular PutJsonAsync method of Flurl, but with no luck.
It's the only non-flurl piece remaining in my code.
Thanx in advance.
The real issue here is that there's currently no out-of-the-box support for sending streams or byte arrays in Flurl. I plan to add some soon, but with the implementation details you already have it's easy to add this yourself with an extension method. (Forgive the C#, hopefully you can translate to VB.)
public static Task<HttpResponseMessage> PutFileAsync(this FlurlClient client, string filepath)
{
var data = File.ReadAllBytes(filepath);
var content = new ByteArrayContent(data);
content.Headers.Add("Content-Type", "application/octet-stream");
content.Headers.Add("Content-Length", data.Length);
return client.SendAsync(HttpMethod.Put, content: content);
}
The above works if you already have a FlurlClient, but as the docs describe it's a good idea to have corresponding string and Url extensions, which can just delegate to the above method:
public static Task<HttpResponseMessage> PutFileAsync(this Url url, string filepath)
{
return new FlurlClient(url).PutFileAsync(filepath);
}
public static Task<HttpResponseMessage> PutFileAsync(this string url, string filepath)
{
return new FlurlClient(url).PutFileAsync(filepath);
}
Tuck those away in a static helper class and they should work seamlessly with Flurl:
await uploadurl.PutFileAsync(filepath)

Azure REST WebClient PUT Blob

I'm trying to simply upload a new blob to an Azure Storage countainer using WebClient like this :
var sas = "[a new generated sas with Read, Write, List & Delete permissions]";
var sData = "This is a test!";
var sEndPoint = "http://myaccount.blob.core.windows.net/mycontainer/MyTest.txt" + sas;
var clt = new WebClient();
var res = await clt.UploadStringTaskAsync(sEndPoint, "PUT", sData);
This is giving me a "(400) Bad Request." error. Am I doing anything wrong here?
Thanks
(By the way, I need to use REST instead of Client API since I'm in a Silverlight project)
You would need to define a request header (x-ms-blob-type) for blob type and set it's value to BlockBlob. Also for Put requests you would need to define the Content-Length request header as well. I wrote a blog post on Shared Access Signatures and performing some blob operations using that (with both REST API and Storage Client library) which you can read here: http://gauravmantri.com/2013/02/13/revisiting-windows-azure-shared-access-signature/.
and here's the code from that post on uploading blob. It uses HttpWebRequest/HttpWebResponse instead of WebClient:
static void UploadBlobWithRestAPISasPermissionOnBlobContainer(string blobContainerSasUri)
{
string blobName = "sample.txt";
string sampleContent = "This is sample text.";
int contentLength = Encoding.UTF8.GetByteCount(sampleContent);
string queryString = (new Uri(blobContainerSasUri)).Query;
string blobContainerUri = blobContainerSasUri.Substring(0, blobContainerSasUri.Length - queryString.Length);
string requestUri = string.Format(CultureInfo.InvariantCulture, "{0}/{1}{2}", blobContainerUri, blobName, queryString);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(requestUri);
request.Method = "PUT";
request.Headers.Add("x-ms-blob-type", "BlockBlob");
request.ContentLength = contentLength;
using (Stream requestStream = request.GetRequestStream())
{
requestStream.Write(Encoding.UTF8.GetBytes(sampleContent), 0, contentLength);
}
using (HttpWebResponse resp = (HttpWebResponse)request.GetResponse())
{
}
}
When testing against the blob emulator this is the code I need to get it working:
var connection = ConfigurationManager.AppSettings["AzureStorageConnectionString"];
var storageAccount = CloudStorageAccount.Parse(connection);
var client = new WebClient();
client.Headers.Add("x-ms-blob-type", "BlockBlob");
client.Headers.Add("x-ms-version", "2012-02-12");
client.UploadData(string.Format(#"{0}/$root/{1}{2}", storageAccount.BlobEndpoint, myFileName, sharedAccessSignature), "PUT", _content);

Working with Azure in Winrt with Rest API, trouble with signature

I'm trying to work with azure storage in winrt. Since the azure storage client is not compatible with winrt I am trying to use azure's rest API. I am having a heck of a time getting the signature right and I could use another set of eyes to help me see where I'm going wrong.
Azure Account provides a name and key property, this method builds up the request right now simply listing all blobs.
'private async void BuildHTTPRequest(AzureAccount account)
{
System.Net.Http.HttpClient request = new HttpClient();
request.BaseAddress = new Uri(string.Format("http://{0}.blob.core.windows.net/", account.Name));
// Always have to use UTC date/time
request.DefaultRequestHeaders.Add("x-ms-date", DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture));
string fmtStringToSign = "{0}\n{1}\n{2}\n{3:R}\n{4}{5}";
request.DefaultRequestHeaders.Add("x-ms-version", "2011-08-18");
string hdr = CanonicalizeHeaders(request.DefaultRequestHeaders);
string authValue = string.Format(fmtStringToSign, "GET", "", "", "", hdr, "");
byte[] signatureByteForm = System.Text.Encoding.UTF8.GetBytes(authValue);
string hashKey = account.Key;
MacAlgorithmProvider macAlgorithmProvider = MacAlgorithmProvider.OpenAlgorithm("HMAC_SHA256");
BinaryStringEncoding encoding = BinaryStringEncoding.Utf8;
var messageBuffer = CryptographicBuffer.ConvertStringToBinary(authValue, encoding);
IBuffer keyBuffer = CryptographicBuffer.ConvertStringToBinary(hashKey, encoding);
CryptographicKey hmacKey = macAlgorithmProvider.CreateKey(keyBuffer);
IBuffer signedMessage = CryptographicEngine.Sign(hmacKey, messageBuffer);
string hashedString = CryptographicBuffer.EncodeToBase64String(signedMessage);
String authHeader = String.Format(CultureInfo.InvariantCulture, "{0} {1}:{2}", "SharedKey",
account.Name, hashedString);
request.DefaultRequestHeaders.Add("Authorization", authHeader);
// Send the request to the queue
try
{
var test1 = request.GetAsync("?comp=list").Result;
if (test1.IsSuccessStatusCode)
{
}
}
catch (WebException ex) { }
}
This should set the headers up for signing...
public string CanonicalizeHeaders(System.Net.Http.Headers.HttpRequestHeaders hdrCollection)
{
StringBuilder retVal = new StringBuilder();// Look for header names that start with "x-ms-" // Then sort them in case-insensitive manner.
List<string> httpStorageHeaderNameArray = new List<string>();
Dictionary<string, string> ht = new Dictionary<string, string>();
foreach (var key in hdrCollection)
{
if (key.Key.ToLowerInvariant().StartsWith("x-ms-", StringComparison.Ordinal))
{
if (ht.ContainsKey(key.Key.ToLowerInvariant()))
{
ht[key.Key.ToLowerInvariant()] = string.Format("{0},{1}", ht[key.Key.ToLowerInvariant()],
hdrCollection.FirstOrDefault(m => m.Key == key.Key).ToString().Replace("\n", string.Empty).Replace("\r", string.Empty).Trim());
}
else
{
httpStorageHeaderNameArray.Add(key.Key.ToLowerInvariant());
ht.Add(key.Key.ToLowerInvariant(),
hdrCollection.FirstOrDefault(m => m.Key == key.Key).Value.FirstOrDefault().ToString().Replace("\n", string.Empty).Replace("\r", string.Empty).Trim());
}
}
}
httpStorageHeaderNameArray.Sort();// Now go through each header's values in the sorted order and append them to the canonicalized string.
foreach (string key in httpStorageHeaderNameArray)
{
retVal.AppendFormat("{0}:{1}\n", key.Trim(), ht[key]);
}
return retVal.ToString();
}
'
Latest version of storage client library supports WinRT. You can read more about it here: http://blogs.msdn.com/b/windowsazurestorage/archive/2012/10/29/introducing-windows-azure-storage-client-library-2-0-for-net-and-windows-runtime.aspx. What I did was download the source code from Github: https://github.com/WindowsAzure/azure-sdk-for-net, opened the solution in VS 2012 and built RT project to get the necessary winmd files.
Coming to your problem, I believe you're running into this issue because you're passing an empty string for canonicalized resource string:
string authValue = string.Format(fmtStringToSign, "GET", "", "", "", hdr, "")
Please see this link for more details on creating canonicalized resource string: http://msdn.microsoft.com/en-us/library/windowsazure/dd179428.aspx.

REST Windows Phone Photo upload

I'm trying to upload a photo to a REST api in a Windows Phone 7 application using RestSharp for my Gets/Posts.
The post parameters are as follows:
photo:
The photo, encoded as multipart/form-data
photo_album_id:
Identifier of an existing photo album, which may be an event or group
album
I've created my request, but every time I get back "{\"details\":\"missing photo parameter\",\"problem\":\"The API request is malformed\"}\n
My photo parameter looks like this:
"---------------------------8cd9bfbafb3ca00\r\nContent-Disposition: form-data; name=\"filename\"; filename=\"somefile.jpg\"\r\nContent-Type: image/jpg\r\n\r\n(some binary junk listed here)\r\n-----------------------------8cd9bfbafb3ca00--"
I'm not quite sure if it's a problem with how I'm presenting the binary data for the image (currently in my PhotoTaskCompleted event, I read the contents of e.ChosenPhoto into a byte[] and pass that to a helper method to create the form data) or if I just don't create the form correctly.
I'm just trying to do this a simple as possible, then I can refactor once I know how it all works.
void ImageObtained(object sender, PhotoResult e)
{
var photo = ReadToEnd(e.ChosenPhoto);
var form = PostForm(photo);
var request = new RequestWrapper("photo", Method.POST);
request.AddParameter("photo_album_id", _album.album_id);
request.AddParameter("photo", form);
request.Client.ExecuteAsync<object>(request, (response) =>
{
var s = response.Data;
});
}
private string CreateBoundary()
{
return "---------------------------" + DateTime.Now.Ticks.ToString("x");
}
private string PostForm(byte[] data)
{
string boundary = CreateBoundary();
StringBuilder post = new StringBuilder();
post.Append(boundary);
post.Append("\r\n");
post.Append("Content-Disposition: form-data; name=\"filename\"; filename=\"somefile.jpg\"");
post.Append("\r\n");
post.Append("Content-Type: image/jpg");
post.Append("\r\n\r\n");
post.Append(ConvertBytesToString(data));
post.Append("\r\n");
post.Append("--");
post.Append(boundary);
post.Append("--");
return post.ToString();
}
public static string ConvertBytesToString(byte[] bytes)
{
string output = String.Empty;
MemoryStream stream = new MemoryStream(bytes);
stream.Position = 0;
using (StreamReader reader = new StreamReader(stream))
{
output = reader.ReadToEnd();
}
return output;
}
Hammock for Windows Phone makes this real simple.
You just add the file to the request using the AddFile method and pass it the photo stream.
var request = new RestRequest("photo", WebMethod.Post);
request.AddParameter("photo_album_id", _album.album_id);
request.AddFile("photo", filename, e.ChosenPhoto);
Hum are you sure that your PostForm is correct ? The content-* params should be set in the headers of your POST and not in the body ?
var request = (HttpWebRequest)WebRequest.Create(url);
request.Headers.Add(HttpRequestHeader.Authorization,"blabla");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";