For the past couple of years, I've been using Google Cloud Storage to handle storing files for a project that runs for a couple months every year. In the process of testing this year, I've been running into issues failing to upload certain files.
This is the response I've been getting:
<Error>
<Code>InvalidPolicyDocument</Code>
<Message>The content of the form does not meet the conditions specified in the policy document.</Message>
<Details>Missing upload</Details>
</Error>
As additional background, I use plupload, and its flash runtime, to handle the upload functionality and send the form submit to GCS.
The headers for the request do include an upload field
------------KM7gL6Ij5KM7KM7cH2Ij5cH2GI3cH2
Content-Disposition: form-data; name="Upload"
Submit Query
and the upload field is specified in the Policy Document I send to GCS
{
"expiration": "2014-09-25T11:32:54.000Z",
"conditions": [
...
["eq", "$Upload", "Submit Query"]
]
}
The only time I get this error is when I try to upload files greater than 100KB in size (file types tested: jpg, png, mp3).
In short, in cases where the file size is greater than 100KB, I get an error telling me that there is no form field for the "upload" variable referenced in the Policy Document, but if the file size is less than 100KB, it accepts the transaction.
I have tried not supplying the "upload" variable as part of the policy document and get the opposite result.
Has anyone encountered something similar or can anyone shed some light on the issue? I'm unsure at this point if plupload is causing the problem or if GCS in fact ignores the "Upload" (submit) field in certain cases but not in others.
Related
I am trying to add contents and an attachment from a Form to a SharePoint list. However, the Get file content using path action in my flow is failing. The error I'm receiving says "Unauthorized" and in the file content box, I receive the following message:
"status": 401,
"message": "A potentially dangerous Request.Path value was detected from the client (?).",
"source": "apidod.connectorp.svc.ms"
The file path is as follows (minus the front of the path):
sites/HSMWINGATLANTIC_Supply_Requests/Shared%20Documents/Forms/AllItems.aspx?newTargetListUrl=%2Fsites%2FHSMWINGATLANTIC%5FSupply%5FRequests%2FShared%20Documents&viewpath=%2Fsites%2FHSMWINGATLANTIC%5FSupply%5FRequests%2FShared%20Documents%2FForms%2FAllItems%2Easpx&id=%2Fsites%2FHSMWINGATLANTIC%5FSupply%5FRequests%2FShared%20Documents%2FApps%2FMicrosoft%20Forms%20Fairfax%2FVehicle%20Rental%20Request%2FSupporting%20Documents&viewid=55590b8b%2D4994%2D4e8b%2D804b%2D24f4774c21e920220815 - HSM-40 Truck Request for 15 AUG 20_Charles Power 1.pdf
c.d.power
For that Get File content using path you would need a relative path without the site url part. You can actually extract the correct path with an expression.
In the example below I retrieve the link property from the Attachment question answer value. I use a json function to turn it into an array, since Microsoft returns a string value for some reason ;)
After that I use nthindexof to determine at which forward slash (starting position of string) I need to slice with a slice function, in this case the 7th instance, which is index 6.
This should retrieve the part which we need for a get file content using path action. With a decodeUriComponent function I make sure the %20 is turned back into space characters.
Make sure you update the question id to your question id.
decodeUriComponent(slice(json(outputs('Get_response_details')?['body/re67e0cfcd95d488593347d93f2728204'])[0]['link'], nthindexof(json(outputs('Get_response_details')?['body/re67e0cfcd95d488593347d93f2728204'])[0]['link'], '/', 6)))
I found the solution to the issue. This wasn’t working because it is a group form and form responses are sent to the group’s SharePoint site; not the user’s OneDrive. Therefore, the Get file content action should be using the SharePoint connector instead of OneDrive.
I am trying to convert some documents into answer units with Watson's Document Conversion service, using the watson-developer-cloud Javascript library in Node.js. Certain ones (an example is at IBM internal link and is a .DOCX file) return this error:
Error: code:400 error: The supplied data appears to be in the Office
2007+ XML. You are calling the part of POI that deals with OLE2 Office
Documents. You need to call a different part of POI to process this
data (eg XSSF instead of HSSF)
If I try to convert it via the document conversion demo site, it seems to convert without error. My program downloads the file from the source, writes it to disk, and then uploads it to the Document Conversion service via the above mentioned library.
Is there any way around this error? Consider that this conversion is part of a massive automated conversion of thousands of documents, so manual handling for these outliers is out of the question.
The service attempts to autodetect the media type of the uploaded file using the first few bytes of the file, and the file name.
If the file name is unavailable (i.e., not passed in by your user), you could provide the media type of the file you are uploading in the file portion of the convert call:
file: {
value: fs.createReadStream('filename'),
options: {
contentType: 'application/vnd.openxmlformats officedocument.wordprocessingml.document'
}
}
I'm building a simple image file upload form. Programmatically, I'm using the Laravel 5 framework. Through the Input facade (through Illuminate), I can resolve the file object, which in itself is an UploadedFile (through Symfony).
The UploadedFile's API ref page (Symfony docs) says that
public integer | null getClientSize()
Returns the file size. It is extracted from the request from which the
file has been uploaded. It should not be considered as a safe
value. Return Value integer|null The file size
What will be these cases where the uploaded filesize is wrongly reported?
Are there known exploits using this?
How can the admin ensure this is detected (and hence logged as a trespass attempt)?
That method is using the "Content-Length" header, which can easily be forged. You'll want to use the easy construct $_FILES['myfile']['size']. As an answer to another question has already stated: Can $_FILES[...]['size'] be forged?
This value checks the actual size of the file, and is not modified by the provided headers.
If you'd like to check for people misbehaving, you can simply compare the content-length header to your $_FILES['myfile']['size'] value.
I have seen several posts addressing this issue or similar to this issue for requests or GETs. I am not having this problem getting the data from the server, its solely on the POST.
The Errors I get are
The JSON request was too large to be deserialized.
or either
Error during serialization or deserialization using the JSON JavaScriptSerializer. The length of the string exceeds the value set on the maxJsonLength property. Parameter name: input
I haven't been able to consistently determine which actions result in which error, but it is predominately the latter one.
In an effort to get the value of the MaxJsonSize value, on the Index method of the controller, I get this data and dump it into a viewbag to write to console on the client side. Every time it comes back at 10k (102400).
If I reduce the data package size, and still serialize as previously, I get no errors.
In fiddler I can inspect the package and all the JSON is deserializable in fiddler, so I don't see an issue in my JSON. Additionally if I console.log(data) chrome sees no problems with it either.
The VM in the controller is the same for both POST and GET. With the exception there is more data with the POST than the GET. To test this I got a huge data set from the server;
GeoJSON data for all 50 states. Following was the result.
GET Content-Length: 3229309 return 200
POST Content-Length: 2975244 return 500
The POST failed in this scenario and returned the second error listed previously.
I only changed the data minimally (one string) and don't know why when sent back its smaller, but the JSON for both the GET and the POST is virtually identical.
I've tried changing the web.config file:
<system.web.extensions>
<scripting>
<webServices>
<jsonSerialization maxJsonLength="2147483644"/>
</webServices>
</scripting>
</system.web.extensions>
I just added this to the end of my config file just prior to
I've also added a parameter in Settings.config
<add key="aspnet:MaxJsonDeserializerMembers" value="2147483644" />
I have also verified that this param loads as part of the application settings in IIS.
Is there something else I can try to change to allow for these large data sets to be sent in a POST.
As a last resort, I was going to pull all of the GeoJSON data out of the POST. However when a user navigates back and they haven't changed what they were mapping, we'd have to find all the GeoJSON data again, causing undue work on the server etc. I thought if I only had to fetch it once that would be best from an efficiency perspective.
I struggled with this too, nothing I changed in web.config helped, despite several SO answers looking relevant. They helped with returning large JSON data, but the large JSON post kept failing. In the end I found this:
increase maxJsonLength for JSON POST and used the solution there, and it worked for me.
Quoting from there :
the MVC json serializer does not look at the webconfig to get the max length (thats for asp.net web services). you need to use your own serializer. you override ExecuteResult and supply you own json serializer. to override the input, create a new JsonValueProviderFactory, then override ValueProvider in the controller to return your new json factory when its a json request.
OK, here's a goal I've been looking for a while.
As it's known, most advertising and analytics companies use a so called "pixel" code in order to track websites views, transactions, conversion etc.
I do have a general idea on how it works, the problem is how to implement it. The tracking codes consist from few parts.
The tracking code itself.
This is the code that the users inserts on his webpage in the <head> section. The main goal of this code is to set some customer specific variables and to call the *.js file.
*.js file.
This file holds all the magic of CRUD (create/read/update/delete) cookies, track user's events and interaction with the webpage.
The pixel code.
This is an <img> tag with the src atribute pointing to an image *.gif (for example) file that takes all the parameters collected on the page, and stores them in the database.
Example:
WordPress pixel code: <img id="wpstats" src="http://stats.wordpress.com/g.gif?host=www.hostname.com&list_of_cookies_value_pairs;" alt="">
Google Analitycs:
http://www.google-analytics.com/__utm.gif?utmwv=4&utmn=769876874&etc
Now, it's obvious that the *.gif request has to reach a server side scripting language in order to read the parameters data and store them in a db.
Does anyone have an idea how to implement this in Zend?
UPDATE
Another thing I'm interested in is: How to avoid the user's browser to load the cached *.gif ? Will a random parameter value do the trick? Example: src="pixel.gif?nocache=random_number" where the nocache parameter value will be different on every request.
As Zend is built using PHP, it might be worth reading the following question and answer: Developing a tracking pixel.
In addition to this answer and as you're looking for a way of avoiding caching the tracking image, the easiest way of doing this is to append a unique/random string to it, which is generated at runtime.
For example, server-side and with the creation of each image, you might add a random URL id:
<?php
// Generate random id of min/max length
$rand_id = rand(8, 8);
// Echo the image and append a random string
echo "<img src='pixel.php?a=".$vara."&b=".$varb."&rand=".$rand_id."'>";
?>
Just adding my 2 cents to this thread because I think an important, and frequently used, option is missing: you don't necessarily need a scripting language to capture the request. A more efficient approach is to use the web server access log (like apache access log for instance) to log the request and then handle that log with whatever tools you see fit, like ELK stack for instance.
This makes serving the requests much lighter because no scripting language is loaded to prepare the response, just native apache response, which is typically much more efficient.
First of all, the *.gif doesn't need to be that file type, the only thing that is of interest is the Content-Type http header. Set that to image/gif (or any other, appropiate type) in the beginning, execute your code and render some sort of image to the response body.
Well, all of the above codes are correct and is good but to be certain, the guy above mention "g.gif"
You can just add a simple php code to write to an sql or fwrite("file.txt",$opened)
where var $opened serves as the counter++ if someone opened your mail... then save it as "g.gif"
TO DO all of this just add these:
<Files "/thisdirectory">
AddType application/x-httpd-php .gif
</Files>
to your ".htaccess" file but be sure to make a new directory for that g.gif or whatever.gif where the directory only contains g.gif and .htaccess