Web Api returns garbage for text files unless run from the browser bar - facebook

I am writing a file service using Asp.Net’s Web Api. The service retrieves files (Css, Excel, Csv, etc.) from SQL Server and serves them up in response to Get requests.
My first test case is for Css files. The issue is that, while I can see the correct data on the server side, when the browser retrieves/decodes it, the results are mangled. The issue appears to be related to the encodings.
Here are the request/response headers in FireFox:
When I click on the response tab in FireBug, here’s what it looks like:
The results look like ascii being displayed as utf8. This is the html view in FireBug:
The above example is an iFrame inside a Facebook application which is running ssl.
If I take the url and open it directly in the browser, it works and correctly displays my Css:
In summary, when I retrieve my Css file from a tag inside my Facebook app, I get garbage (encoding issue?). If I retrieve it straight from the browser, it works.
My CssFormatter MediaTypeFormatter code:
public override Task WriteToStreamAsync(Type type, object value, Stream writeStream, HttpContent content, TransportContext transportContext)
{
var taskSource = new TaskCompletionSource<object>();
try
{
var incomingFile = value as FileRestService.Entity.IFile;
var ms = new MemoryStream(incomingFile.DataBuffer);
ms.CopyTo(writeStream);
ms.Flush();
taskSource.SetResult(writeStream);
}
catch (Exception e)
{
taskSource.SetException(e);
}
return taskSource.Task;
}
Am I creating the response stream incorrectly? I noticed that the response headers do not specify the encoding. Is this an issue?

I find the easiest way to handle this is to write something along the lines of (here's the important details):
public class Formatter : MediaTypeFormatter {
// TODO override the constructor to add some mappings or some other way for this formatter to be picked up
// TODO override CanReadType and CanWriteType according to your rules
public override void SetDefaultContentHeaders(Type t, HttpContentHeaders headers, string mediaType) {
base.SetDefaultContentHeaders(t, headers, mediaType);
headers.ContentDisposition = new ContentDispositionHeaderValue("attachment") {
FileName = "SomeName.ext"
};
}
public override Task WriteToStreamAsync(Type t, object value, Stream s, HttpContentHeaders headers, TransportContext context) {
return Task.Factory.StartNew(() => {
// TODO code to write to the output stream, flush it but don't explicitly close it
});
}
}

Related

Handle uncompleted file upload by API POST endpoint in asp.net core MVC

To simplify the problem let's say I have a simple asp.net mvc endpoint which receives a file. In most of the cases it will be a .jpg one:
[HttpPost]
[Route("appraisal/{appraisalID}/files/{fileSubjectCode}")]
[ProducesResponseType(StatusCodes.Status201Created, Type = typeof(IEnumerable<AppraisalFileModel>))]
[ProducesResponseType(StatusCodes.Status400BadRequest, Type = typeof(ModelStateDictionary))]
public async Task<IActionResult> UploadAppraisalFile(int appraisalID, string fileSubjectCode, [FromForm]IFormFile file)
{
file = file ?? Request.Form.Files?.FirstOrDefault();
// intermitent code change to investigate and validate Complete File Size and Actual File Size
var completeFileSizeHeader = Request.Headers["Complete-File-Size"];
int.TryParse(completeFileSizeHeader, out int completeFileSize);
if (file == null || file.Length != completeFileSize)
{
using (var stream = new MemoryStream())
{
await file.CopyToAsync(stream);
stream.Position = 0;
var inputAsString = Convert.ToBase64String(stream.ToArray());
Logger.LogDebug("Complete-File-Size header doesn't much received byteArray size", file.Length, completeFileSize, inputAsString);
}
return StatusCode(StatusCodes.Status411LengthRequired, "Complete-File-Size header doesn't much received byteArray size");
}
// some other logic..
}
I'm trying to prevent an edge case when somebody performs a POST request against my API UploadAppraisalFile endpoint and suddenly loses an internet connection which would result in sending a request with not the full file content.
My idea was to count the file size at the point where the file is uploaded, add the information about the file size as an extra HTTP-HEADER (I called it Complete-File-Size), and then when the request reaches the backend, count if the received file size is exactly the same as the Complete-File-Size.
To produce such an issue/edge case I tried:
uploading a big file(about 16MB) and then suddenly after submitting the HTML form immediately close the browser window.
uploading a file and then, in the Chrome browser, in the Network pane, change the uploading speed to a very minimum, then submit the form and then immediately close the browser.
When I run the debug mode, in each case I found that either: UploadAppraisalFile endpoint was never reached or if it was reached then always the full file was sent. For the 2nd successful case, to be 100% sure I converted the received file into base64 string and then I checked how the file looks like in https://codebeautify.org/base64-to-image-converter.
My question is: Is it even possible that the sent POST request is broken and contains not full file content due to a broken internet connection that happened suddenly during the sending process? If yes, then what's the best way to produce the issue. Cheers
You can pass HttpContext.RequestAborted as a CancellationToken to ALL async methods provided by .NET in "some other logic" part.
Let's use code you provided as an example :
await stream.CopyToAsync(memoryStream, HttpContext.RequestAborted)
I don't have an access to a full method but I assume you save it to some blob storage or file system. Most of these persistence API's accept CancellationToken as a parameter.
Receiving incomplete file
I was able to achieve a "partial" file using this code and Postman. It will basically read chunks from response stream until connection is interrupted.
As soon as I close Postman window TaskCancelledException is raised and stream is closed.
[HttpPost]
public async Task<IActionResult> UploadAppraisalFile([FromForm] IFormFile file)
{
var appraisalfile = file ?? Request.Form.Files.FirstOrDefault();
if (appraisalfile != null)
{
var buffer = ArrayPool<byte>.Shared.Rent(1024);
using var stream = appraisalfile.OpenReadStream();
while (await stream.ReadAsync(buffer, 0, buffer.Length, HttpContext.RequestAborted) > 0)
{
// Do something with buffer
_logger.LogInformation("Total length (bytes) {0}, partial length (bytes) {1}", stream.Length, stream.Position);
}
ArrayPool<byte>.Shared.Return(buffer);
}
return Ok();
}

Why does the Facebook Graph API respond with the same "next" URL over and over?

I am using Volley to submit POST requests to the Facebook Graph API in order to retrieve information about photos and videos from a user account using their BATCH facility so I get it all in one go (rather than making one call for photos, one for videos). The first call works perfectly:
request = new StringRequest(Request.Method.POST,
"https://graph.facebook.com",
future,
new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
Timber.e("Got VolleyError: %s", error.getMessage());
}
}) {
#Override
public String getBodyContentType() {
return "application/x-www-form-urlencoded; charset=" + getParamsEncoding();
}
#Override
protected Map<String, String> getParams() throws AuthFailureError {
Map<String,String> params = new HashMap<>();
JSONArray batchRequest = new JSONArray();
JSONObject photoRequest = new JSONObject();
JSONObject videoRequest = new JSONObject();
try {
photoRequest.put("method", "GET");
photoRequest.put("relative_url",facebookUserID + String.format("?fields=photos.limit(%1$s){id,created_time,images{source},picture}",batchSize));
videoRequest.put("method", "GET");
videoRequest.put("relative_url",facebookUserID + String.format("?fields=videos.limit(%1$s){id,created_time,source,picture}",batchSize));
batchRequest.put(photoRequest);
batchRequest.put(videoRequest);
} catch (JSONException e) {
Timber.d("Lifecycle: Exception constructing batch request: %s", e.getMessage());
return null;
}
//try {
// Timber.d("Lifecycle: batchRequest: %s", batchRequest.toString(2));
//} catch (JSONException e) {
// e.printStackTrace();
//}
params.put("batch", batchRequest.toString());
params.put("include_headers", "false");
params.put(FB_BASE_ACCESSTOKEN_KEY, facebookToken);
return params;
}
};
InTouchUtils.addToRequestQueue(request);
// Using a blocking volley request, this chain has been called on a separate async task
// See SO: https://stackoverflow.com/questions/16904741/can-i-do-a-synchronous-request-with-volley
facebookRetval = future.get(VOLLEY_REQUEST_DEFAULT_TIMEOUT, TimeUnit.SECONDS);
returnResult = parseBatchRequest(facebookRetval);
The returned JSON has all the fields I've requested, as well as the pagination block with cursors, and a "next" and/or "previous" url, per the Facebook documentation.
A "next" URL looks something like:
https://graph.facebook.com/v7.0/FACEBOOK_ID_HERE/photos?access_token=ACCESS_TOKEN_HERE&fields=id,created_time,images{source},picture&limit=5&after=AFTER_TOKEN_HERE
There is one of these that gets passed back from the batch operation for each of the original GET operations (assuming both photos and videos have greater than LIMIT items).
Again, this part works fine.
But when I try and use that "next" URL to create another BATCH call, it fails with an "unsupported GET operation" error. This is true even though I can use a standard Volley GET using that exact same URL and it works perfectly.
I have tried using the "https://graph.facebook.com" portion of the above URL as the root of the POST (like what worked in the initial call), and everything after that as the "relative_url" parameter. No go.
Then I tried parsing out just the "after" portion of the "next" url, and constructing a new relative_url that was exactly like the first one, but tacking on a "&after=" + AFTER_VALUE to it as the relative_url. No go - in fact, while this succeeded in making the call, I keep getting the initial batch over and over and over. It is like it is ignoring the "&after=" parameter.
For now I am back to making two GET calls (one for photos, one for videos) just using the NEXT url as long as it keeps being passed back to me. This works fine, but obviously I'm making two network calls instead of the single batch one.
Ideas?
A little more examination revealed that I had made a string parsing error on the subsequent batch operation, and was inadvertently including a forward slash when I should not have been.
For those new to using the batch API, the lesson is that you need "https://graph.facebook.com" as the POST url (no trailing forward slash), then your relative url should NOT start with a forward slash. So the URL I was trying to utilize on calls 2..N like this:
https://graph.facebook.com/v7.0/FACEBOOK_ID_HERE/photos?access_token=ACCESS_TOKEN_HERE&fields=id,created_time,images{source},picture&limit=5&after=AFTER_TOKEN_HERE
should be broken out as:
photoRequest.put("relative_url", "v7.0/FACEBOOK_ID_HERE/photos?access_token=ACCESS_TOKEN_HERE&fields=id,created_time,images{source},picture&limit=5&after=AFTER_TOKEN_HERE");
The API handles putting in the forward slash between the root and the relative url.

Callback function issues in FB.API for Unity

I am using Unity 5.5.2f1 pro and facebook's SDK v 7.9.4
I have a script which after login (managed in a previous scene) sends an API request to FB asking for friends, name and email and sends that info as a POST to a php website.
code:
[Serializable]
public struct FBData {
public string first_name;
public string email;
public string friends;
public string id;}
public class UserManagement : MonoBehaviour {
string urlSaveUserData="some php website";
public Text testTxt;
FBData parsedData;
// Use this for initialization
void Start () {
//Check if it's the first time the user is opening the app.
if (UserInfo.FIRST_TIME) {
//update text (only used for testing, should be removed in production.)
testTxt.text = "Your user id is: " + UserInfo.ID;
//Perform FB.API call to get User Data.
getUserData ();
//Save in SQL table. (won't get here if line in getUserData() is active)
StartCoroutine ("saveUserData");
} else {
//do something else.
}
note: Since this is meant for iOS I have to test it on a device so I'm using text in the screen to display info (think of it as a badly implemented print statement).
The problem: In my callback function for FB.API I write in the text Gameobject (aka testTxt) the parsed information from the response which is saved in the Custom UserInfo clss. It display's correctly but the code gets stuck there. It doesn't continue to the next function. HOWEVER, if I delete/comment that line and don't display anything in the text field. The codes does continue to the POST function BUT the information from the API call is not passed, i.e my custom class is empty (leading me to believe the callback function is not called at all).
public void getUserData(){
string query = "me?fields=first_name,email,friends";
FB.API (query, HttpMethod.GET, Apicallback, new Dictionary<string, string> ());
}
private void Apicallback(IGraphResult result){
//Parse Graph response into a specific class created for this result.
parsedData = JsonUtility.FromJson<FBData>(result.RawResult);
//Pass each field into UserInfo class.
UserInfo.EMAIL = parsedData.email;
UserInfo.FRIENDS = parsedData.friends;
UserInfo.NAME = parsedData.first_name;
UserInfo.FACEBOOKID = parsedData.id;
/*problem area, if I comment line below, then previous information is apparently not stored. If left as is then testTxt displays correct information but code gets stuck there. */
testTxt.text = "This is the info from USerInfoInside the APICallback: " + UserInfo.EMAIL + UserInfo.FRIENDS + UserInfo.FACEBOOKID;
}
The function below is to send info to php website, is there for illustrative purposes:
code:
public IEnumerator saveUserData() {
//get user info (this information is EMPTY if line in getUserData() is commented.
parsedData.id = UserInfo.FACEBOOKID;
parsedData.friends = UserInfo.FRIENDS;
parsedData.first_name = UserInfo.NAME;
parsedData.email = UserInfo.EMAIL;
//translate data into json
string JsonBodyData = JsonUtility.ToJson (parsedData);
//Custom web request (POST method doesnt seem to work very well, documentation example sends empty form)
var w = new UnityWebRequest(urlSaveUserData, "POST");
byte[] bodyRaw = new System.Text.UTF8Encoding().GetBytes(JsonBodyData);
w.uploadHandler = (UploadHandler) new UploadHandlerRaw(bodyRaw);
w.downloadHandler = (DownloadHandler) new DownloadHandlerBuffer();
w.SetRequestHeader("Content-Type", "application/json");
yield return w.Send();
//work with received data...}
Im stuck here any help is appreciated. Thanks!
Be sure to use EscapeURL when using strings directly for JSON or HTTP POST and GET methods. The lack of this treatment tends to screw things over, particulary in iOS platforms.
From what I can see, this code
string query = "me?fields=first_name,email,friends";
should instead be escaped as
string query = WWW.EscapeURL("me?fields=first_name,email,friends");
so characters like "?" won't get encoded as an URL symbol.
I'm assuming you don't need to do that for your illustrative example, because UnityWebRequest already escapes your POST request strings internally, but I can't fully confirm that.

GWT: Response not coming back to Form onSubmitCompleteEvent when a servlet sends a file in response to browser

Code on client side :
#UiHandler("form")
void onFormSubmission(SubmitCompleteEvent event) {
hideProcessingPopUp();
if (event.getResults().contains("Exception")) {
// handle exception
}
}
Servlet code in doPost method:
response.setContentType("text/csv");
response.setHeader("Content-Disposition", "attachment;filename="
+ exportType + ".csv");
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(
response.getOutputStream(), "UTF-8"));
// Adding content
List<CustomObject> list = (List<CustomObject>) anonymousList;
for (CustomObject eachObject : list) {
writer.append(eachObject.getContent());
writer.newLine();
}
writer.flush();
// Gently close streams/writers.
close(writer);
return fileContent;
Servlet code is fine, as am getting expected data and file. Problem is with response not reaching the client side i.e., SubmitCompleteEvent. Please help me out, thanks in advance.
This is how browsers work; and the javadoc for FormPanel actually warns you about it:
The back-end server is expected to respond with a content-type of 'text/html', meaning that the text returned will be treated as HTML. If any other content-type is specified by the server, then the result HTML sent in the onFormSubmit event will be unpredictable across browsers, and the onSubmitComplete event may not fire at all.

ajaxcontroltoolkit setting hidden value after asyncfileupload has completed

I have an asyncfileupload control that I'm using from the ajaxcontroltoolkit. On the file complete in the code behind I process the file and write the information in the file to a database. I get the id of the record from the database, and this needs to be written to an asp hidden field. I've tried just setting the value:
fldImageID.Value = pimg.IdImageGroup.ToString();
I've tried Registering a script like I've seen in an example on a website:
ScriptManager.RegisterClientScriptBlock(
ImageFileUploader,
ImageFileUploader.GetType(),
"script1",
"alert('hi'); top.document.getElementById('"
+ fldImageID.ClientID
+ "').value='"
+ pimg.IdImageGroup.ToString()
+ "'; top.document.getElementById('"
+ lblError.ClientID
+ "').innerHTML = 'image uploaded'",
true);
I've just tried embedding javascript in a response.Write call from the method I've set to process the uploaded file. Nothing I've done has worked so far. After I've done everything the hidden field still does not contain the required value.
This is pretty easy with jQuery. Have an html hidden input control placed in your page, not the asp:hidden input control. Add a class lets say "hiddenPhoto" to your html hidden control.
so lets say our control html is like this
<input type="hidden" class="hiddenPhoto" runat="server" id="hdPhotoName" />
Now access it using class selector in your OnClientUploadComplete js method and set its value. Have it declared runat="server" in order to access its value on the server side.
Regards
I found an acceptable solution back when I was working on this. And since then I've received emails from people who have had the same problem and have been asking if I found a solution. So I'm presenting it here, stripping out any extraineous code:
From the user control that has the FileUpload control I first set the session variable on the back side in the FileUploadComplete handler:
*in the ascx file (upload_chart.ascx) I have the AsyncFileUpload, what is important is the OnUploadComplete and the OnClientUploadComplete:*
<ajaxToolkit:AsyncFileUpload
OnUploadedComplete="FileUploadComplete1"
OnClientUploadComplete="UploadComplete1"
ID="ImageFileUploader"
runat="server" />
*in the code behind of the ascx file (upload_chart.ascx.cs) I handle the FileUploadComplete:*
public void FileUploadComplete1(object sender, EventArgs e)
{
try
{
if (ImageFileUploader.FileBytes.Length > 0)
{
// File data is in ImageFileUploaded.FileBytes
// Save it however you need to
// I saved it to a database, in a DBImage Object class I created
// DBImage is specific to my application
ODS.Entity.DBImage pimg =
ODS.Data.DataRepository.SaveImageBytes(ImageFileUploaded.FileBytes);
// Set the ImageID1 in the session
Session["ImageID1"] = pimg.IdImageGroup.ToString();
}
else
{
// error handling for an empty file, however you want to handle it
}
}
catch (Exception Ex)
{
// error handling for an unhandled exception, whatever you want to do here
}
}
Javascript and script methods are used to set the value on the page, here is my codebehind for the script method:
// on the aspx page code behind (chartofthedayadmin.aspx.cs) I have the webmethod:
[System.Web.Services.WebMethod]
public static string GetImageID1()
{
System.Web.SessionState.HttpSessionState Session = System.Web.HttpContext.Current.Session;
String retval = Session["ImageID1"].ToString();
Session["ImageID1"] = null;
return retval;
}
Here is the javascript:
// on the aspx front end (chartofthedayadmin.aspx) I have the javascript
// to call the Web method and the javascript failed message:
function UploadComplete1() {
var str = PageMethods.GetImageID1(uploadSuccess1, uploadFailed);
}
function uploadFailed() {
alert('error occurred or some meaningfull error stuff');
}
*// javascript on the user control (upload_chart.ascx) to set the value of the hidden field*
function uploadSuccess1(result) {
document.getElementById('<%= fldImageID.ClientID %>').value = result;
}
note: Make sure your scriptmanager has EnablePageMethods="true".
The better and more simple solution is in code behind:
string script = String.Format("top.document.getElementById('hdnFilename').value='{0}';", safeFilename);
ScriptManager.RegisterClientScriptBlock(this, this.GetType(), "hdnFilenameFromCodeBehind", script, true);
In my case, safeFilename is the unique filename, after handling duplicate filename, i.e. sample_5.png in the 5th upload of sample.png.
See http://forums.asp.net/t/1503989.aspx