companion: 2022-09-22T23:31:07.088Z [error] a62da431-f9ce-4fae-b18d-dc59189a53ea root.error PayloadTooLargeError: request entity too large
at readStream (/usr/local/share/.config/yarn/global/node_modules/raw-body/index.js:155:17)
at getRawBody (/usr/local/share/.config/yarn/global/node_modules/raw-body/index.js:108:12)
at read (/usr/local/share/.config/yarn/global/node_modules/body-parser/lib/read.js:77:3)
at jsonParser (/usr/local/share/.config/yarn/global/node_modules/body-parser/lib/types/json.js:135:5)
at Layer.handle [as handle_request] (/usr/local/share/.config/yarn/global/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (/usr/local/share/.config/yarn/global/node_modules/express/lib/router/index.js:317:13)
at /usr/local/share/.config/yarn/global/node_modules/express/lib/router/index.js:284:7
at Function.process_params (/usr/local/share/.config/yarn/global/node_modules/express/lib/router/index.js:335:12)
at next (/usr/local/share/.config/yarn/global/node_modules/express/lib/router/index.js:275:10)
at middleware (/usr/local/share/.config/yarn/global/node_modules/express-prom-bundle/src/index.js:174:5)
::ffff:172.29.0.1 - - [22/Sep/2022:23:31:07 +0000] "POST /s3/multipart/FqHx7wOxKS8ASbAWYK7ZtEfpWFOT2h9KIX2uHTPm2EZ.k1INl8vxfdpH7KBXhLTii1WL7GeDLzLcAKOW0vmxKhfCrcUCRMgHGdxEd5Nwxr._omBrtqOQFuY.Fl9nX.Vy/complete?key=videos%2Fuploads%2Ff86367432cef879b-4d84eb44-thewoods_weddingfilm_1.mp4 HTTP/1.1" 413 211 "http://localhost:8080/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"
It will always give out a 413 on the last request with about 188k of payload describing all the parts.
I've tried anything including:
var bodyParser = require("body-parser");
app.use(bodyParser.json({ limit: "50mb" }));
app.use(bodyParser.urlencoded({limit: "50mb", extended: false}));
but it has no effect. I've been spending months on this problem, read every article, complaint, issue on the internet regarding this and still, no one knows why it's happening and no one knows how to resolve it.
Can anyone help?
This is a known issue with the S3 plugin. It is fixed in the latest version of Uppy, but Companion is still on an older version. You can use the S3 Multipart plugin directly, which is what Companion uses under the hood.
const Uppy = require('#uppy/core')
const AwsS3Multipart = require('#uppy/aws-s3-multipart')
const uppy = new Uppy()
uppy.use(AwsS3Multipart, {
companionUrl: 'https://companion.uppy.io',
companionCookiesRule: 'same-origin',
limit: 5,
getUploadParameters (file) {
return {
method: 'post',
endpoint: 'https://companion.uppy.io/s3/multipart',
fields: {
filename: file.name,
size: file.size,
contentType: file.type
}
}
}
})
uppy.on('upload-success', (file, data) => {
console.log('Upload successful', file, data)
})
uppy.on('upload-error', (file, error) => {
console.log('Upload error', file, error)
})
uppy.addFile({
name: 'test.txt',
type: 'text/plain',
data: new Blob(['hello world'], { type: 'text/plain' })
})
uppy.upload()
The S3 Multipart plugin is a bit more complicated to use than the S3 plugin, but it is more flexible. It allows you to upload files larger than 5GB, and it allows you to upload files in parallel. It also allows you to upload files to S3-compatible services like Minio. The S3 plugin is a bit simpler to use, but it has some limitations. It doesn’t allow you to upload files larger than 5GB, and it doesn’t allow you to upload files in parallel. It also doesn’t allow you to upload files to S3-compatible services like Minio.
Related
I have a script the parse a specific webpage.
When I set headless false, the puppeteer doesn't load page
await page.goto('https://www.google.com', {
waitUntil: 'load',
// Remove the timeout
timeout: 0
});
I tried with a lot of configurations, like:
const args = [
'--no-sandbox',
'--enable-logging',
' --v=1',
'--disable-gpu',
'--disable-extension',
'--disable-setuid-sandbox',
'--disable-infobars',
'--window-position=0,0',
'--ignore-certifcate-errors',
'--ignore-certifcate-errors-spki-list',
'--user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3312.0 Safari/537.36"'
];
const options = {
args,
headless: false, // default is true
userDataDir: "./user_data",
defaultViewport: null,
devtools: true,
ignoreHTTPSErrors: true,
};
But the script stops to await page.goto until timeout.
As I understand you're parsing not the google.com page.
The first thing that should be considered it's the waitUntil: 'load'. What it done, it considers navigation to be finished when the load event is fired.
The load event is fired when the whole webpage (HTML) has loaded fully, including all dependent resources such as JavaScript files, CSS files, and images.
There is a big chance that this event is not firing in your case for a reasonable timeout, so I would suggest not to rely on this waitUntil but use another wait like the presence of some selector, for example
await page.goto('https://www.google.com');
await page.waitForSelector('[name="q"]');
I've got a next.js react app running on a custom Express server with custom routes. I'm working on this project by myself, but I'm hoping I might have a collaborator at some point, and so my main goal is really just to clean things up and make everything more legible.
As such, I've been trying move as much of the Express routing logic as possible to the built in Next.js api routes. I'm also trying to replace all the fetch calls I have with axios requests, since they look less verbose.
// current code
const data = await fetch("/api/endpoint", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ foo: "bar" })
}).then(x => x.json());
// what I'd like
const data = await axios.post( "/api/endpoint", { foo: "bar" });
The problem I've been having is that the dynamic next.js api routes stall as soon as there's JSON data in the body. I'm not even getting an error, the request just gets stuck as "pending" and the await promise never resolved.
I get responses from these calls, but I can't pass in the data I need:
// obviously no data passed
const data = await axios.post( "/api/endpoint");
// req.body = {"{ foo: 'bar' }":""}, which is weird
const data = await axios.post( "/api/endpoint", JSON.stringify({ foo: "bar" }));
// req.body = "{ foo: 'bar' }" if headers omitted from fetch, so I could just JSON.parse here, but I'm trying to get away from fetch and possible parse errors
const data = await fetch("/api/endpoint", {
method: "POST",
// headers: { "Content-Type": "application/json" },
body: JSON.stringify({ foo: "bar" })
}).then(x => x.json());
If I try to call axios.post("api/auth/token", {token: "foo"}), the request just gets stuck as pending and is never resolved.
The Chrome Network panel gives me the following info for the stalled request:
General
Request URL: http://localhost:3000/api/auth/token
Referrer Policy: no-referrer-when-downgrade
Request Headers
Accept: application/json, text/plain, */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,es;q=0.8
Connection: keep-alive
Content-Length: 26
Content-Type: application/json;charset=UTF-8
Cookie: token=xxxxxxxxxxxxxxxxxxxx; session=xxxxxxxxxxxxxxxxxxxxxxx
Host: localhost:3000
Origin: http://localhost:3000
Referer: http://localhost:3000/dumbtest
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36
Request Payload
{token: "foo"}
I've tried looking into what might be causing this, and everything seems to point towards there being an issue with preflight requests, but, since those are related to CORS policies, I don't understand why I'd be encountering those. I'm making a request from http://localhost:3000 to http://localhost:3000/api/auth/token.
Even so, I did try to add cors middleware as shown in the next.js example, but that didn't make a difference. As far as I can tell, the request never even hits the server - I've got a console.log call as the first line in the handler, but it's never triggered by these requests.
Is there something obvious I'm missing? This feels like it should be a simple switch to make, but I've spent the last day and a half trying to figure this out, but I keep reaching the same point with every solution I try - staring at a gray pending request in my Network tab and a console reporting no errors or anything.
After a few more hours searching, I found my answer here
Turns out that since I was using a bodyParser middleware in my express server, I had to disable the Next body parsing by adding this at the top of my file:
export const config = {
api: {
bodyParser: false,
},
}
I am making a POST request to server to upload an image and sending formdata using axios in react-native. i am getting "Network Error". i also try fetch but nothing work.using react native image picker libeary for select image.in postman api working fine
formData.append('title', Title);
formData.append('class_id', selectClass._id)
formData.append('subject_id', checkSelected)
formData.append('teacher_id', userId)
formData.append('description', lecture);
formData.append('type', 'image');
var arr=[];
arr.push(imageSource)
arr.map((file,index)=>{
formData.append('file',{
uri:file.path,
type:file.type,
name:file.name
})
})
axios({
method: 'post',
url: URL + 'admin/assignment/create',
data: data,
headers: {
"content-type": "multipart/form-data",
'x-auth-token': token,
},
})
.then(function (response) {
//handle success
console.log('axios assigment post',response);
})
.catch(function (response) {
//handle error
console.log('axios assigment post',response);
});
Project keeps flipper java file under app > source > debug in react native > 0.62. There is an issue with Flipper Network that causes the problem in your case. If you remove the debug folder, you will not be able to debug Android with Flipper, so the best solution is upgrading Flipper version in android > gradle.properties to 0.46.0 that fixes the problem.
You can change it with this line
FLIPPER_VERSION=0.46.0
react-nativeandroid
The issue that I was facing which is close to what you are mentioning is that I was getting NetworkError when using image-picker and trying to upload the file using axios. It was working perfectly in iOS but not working in android.
This is how I solved the issue.
There are two independent issues at action here. Let’s say we get imageUri from image-picker, then we would use these following lines of code to upload from the frontend.
const formData = new FormData();
formData.append('image', {
uri : imageUri,
type: "image",
name: imageUri.split("/").pop()
});
The first issue is with the imageUri itself. If let’s say photo path is /user/.../path/to/file.jpg. Then file picker in android would give imageUri value as file:/user/.../path/to/file.jpg whereas file picker in iOS would give imageUri value as file:///user/.../path/to/file.jpg.
The solution for the first issue is to use file:// instead of file: in the formData in android.
The second issue is that we are not using proper mime-type. It is working fine on iOS but not on Android. What makes this worse is that the file-picker package gives the type of the file as “image” and it does not give proper mime-type.
The solution is to use proper mime-type in the formData in the field type. Ex: mime-type for .jpg file would be image/jpeg and for .png file would be image/png. We do not have to do this manually. Instead, you can use a very famous npm package called mime.
The final working solution is:
import mime from "mime";
const newImageUri = "file:///" + imageUri.split("file:/").join("");
const formData = new FormData();
formData.append('image', {
uri : newImageUri,
type: mime.getType(newImageUri),
name: newImageUri.split("/").pop()
});
I hope this helps to solve your problem :)
REACT NATIVE SOLUTION
If you are using Axios or Fetch in React Native and you got Network Error when uploading the file or data.
Try to commenting below line from the /android/app/src/main/java/com/{your_project}/MainApplication.java
its located around the 40-50 line
initializeFlipper(this, getReactNativeHost().getReactInstanceManager())
https://github.com/facebook/react-native/issues/28551
I faced the same issue. The following steps worked for me.
update FLIPPER_VERSION=0.52.0 latest
for formData code as below:
let formData = new FormData();
let file = {
uri: brand.uri,
type: 'multipart/form-data',
name: brand.uri
};
formdata.append('logo', file);
The type must be 'multipart/form-data' as the post header.
"react-native": "0.62.1",
"react": "16.11.0",
"axios": "^0.19.2",
weird solution i have to delete debug folder
in android ->app->source->debug
and restart the app again
its solve my problem. i think it's cache problem.
I had this problem and solve it via commenting the 43 line in
android/src/debug/.../.../ReactNativeFlipper.java
// builder.addNetworkInterceptor(new FlipperOkhttpInterceptor(networkFlipperPlugin));
could you test it?
in my case, the solution was to change to
const headers = {
accept: 'application/json',
'content-type': 'multipart/form-data',
};
change this line: form_data.append('file', data);
To form_data.append('file', JSON.stringify(data));
from https://github.com/react-native-image-picker/react-native-image-picker/issues/798
You need to add this uesCleartextTraffic="true" to the AndroidManifest.xml file found inside the dir android/app/src/main/AndroidManifest.xml
<application ... android:usesCleartextTraffic="true"> Then, Because of issue with Flipper Network.
I commented initializeFlipper(this, getReactNativeHost().getReactInstanceManager())
in this file /android/app/src/main/java/com/{your_project}/MainApplication.java
Also, commenting out line number 43 in this file android/app/src/debug/java/com/**/ReactNativeFlipper.java
line43: builder.addNetworkInterceptor(new FlipperOkhttpInterceptor(networkFlipperPlugin));
If using expo and expo-image-picker, then the problem is only with the image type and nothing else.
In the latest updates, they removed the bug related to path (as other answers mention to change the beginning of the path which was correct for the older versions).
Now to remove the problem, we need to change the type only and is mentioned by other answers to use mime which works fine;
import mime from 'mime'
const data = new FormData();
data.append('image', {
uri: image.uri,
name: image.uri.split('/').pop() // getting the text after the last slash which is the name of the image
type: mime.getType(image.uri) // image.type returns 'image' but mime.getType(image.uri) returns 'image/jpeg' or whatever is the type
})
In my case, after debbuging for a while, the issue was in nginx.
The image was "too big".
I Had to add annotations to the Kubernetes ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: 20m
....
It was a bit tricky to debug since the request never got through the load balancer (nginx) to the Api service.
The "Network error" message didn't help a lot either.
I am using Expo SDK 42 (react-native v0.63). And I was using the expo-document-picker library to pick the documents & upload to server.
This is the code I am using to open the picker & get the metadata about the file.
const result = await DocumentPicker.getDocumentAsync({
type: 'image/*',
copyToCacheDirectory: false,
multiple: false,
});
if (result.type === 'success') {
const name_split = result.name.split('.');
const ext = name_split[name_split.length - 1];
// result.uri = 'file://' + result.uri;
result.type = helper.get_mime_type(ext);
delete result.size;
}
(You can write your function to get the mime type from the file extension or use some library from npm)
And this is the code I am using to upload the file to server:
const formdata = new FormData();
formdata.append('custom_param', 'value');
formdata.append('file', result); // 'result' is from previous code snippet
const headers = {
accept: 'application/json',
'content-type': 'multipart/form-data',
};
const opts = {
method: 'POST',
url: 'your backend endpoint',
headers: headers,
data: formdata,
};
return await axios.request(axiosopts);
The above code is the working code. I want to explain what I did wrong initially that was causing the Network Error in axios.
I had set copyToCacheDirectory to true initially and the uri I was getting in the result was in the format /data/path/to/file.jpeg. And I also tried appending file:// to beginning of the uri but it didn't work.
I then set copyToCacheDirectory to false and the uri I was getting in the result was in the format content://path/to/file.jpeg. And this didn't cause any Network Error in axios.
I have faced this error as well. I found out that I got this error because the file path is wrong and axios couldn't find the file. It is a weird error message when the uri is wrong though but that's what actually has happened. So double checking the uri of the file would fix the issue. Mainly consider file://.
I faced the same issue.
After capturing the photo wait for 10sec then start uploading. It worked for me.
Also got the same issue. I spent almost 4 days to find reason.
So in my case it was Content-Type: multipart/form-data. I forgot
indicate it. In Android it should be indicated explicitly...
After two days of searching for a solution, I found that the problem was in my rn-fetch-blob library, Changed it to in package.json dependencies
"rn-fetch-blob": "^0.12.0",
fix my Netowk issue and app crash on uploading. I am using
react-native-image-crop-picker
always send image and file in formdata in post api through axios in react js and react native
Blockquote
REACT NATIVE SOLUTION USING AXIOS
I face the same issue after upgrading react native cli project.
I'm using
"react-native": "0.70.6",
"react": "18.1.0",
"axios": "^1.1.3"
AND FLIPPER_VERSION=0.125.0
The below code solves my issue
const imageData = image;
const form = new FormData();
form.append("ProfileImage", {
type: imageData.mime,
uri: imageData.path,
name: imageData.path.split("/").pop(),
});
axios({
method: "put",
url: `${URL}/api/UploadPhoto`,
data: formData,
headers: {
"Content-Type": "multipart/form-data",
"cache-control": "no-cache",
},
processData: false,
contentType: false,
mimeType: "multipart/form-data",
});
For me, I didn't comment on any line from the /android/app/src/main/java/com/{your_project}/MainApplication.java
initializeFlipper(this, getReactNativeHost().getReactInstanceManager())
Also not changed FLIPPER_VERSION
I'm trying to upload files to a library on sharepoin 2013 on premise by Ajax. I'm using the following code:
function uploadFileee(file) {
// var file = element.files[0];
console.log(file);
var reader = new FileReader();
reader.onload = function (e) {
enviar(e.target.result, file.name);
}
reader.onerror = function (e) {
alert(e.target.error);
}
//reader.readAsArrayBuffer(file);
reader.readAsArrayBuffer(file);
function enviar(file, name) {
var url = String.format(
"{0}/_api/Web/Lists/getByTitle('{1}')/RootFolder/Files/Add(url='{2}', overwrite={3})",
_spPageContextInfo.webAbsoluteUrl, "TreinamentoLib", name, "true");
console.log(url);
jQuery.ajax({
url: url,
type: "POST",
data: file,
processData: false,
headers: {
Accept: "application/json;odata=verbose",
"X-RequestDigest": jQuery("#__REQUESTDIGEST").val()
},
success: function (data) {
console.log("sucesso");
},
error : function(err)
{
console.log("erro");
}
})
}
}
As long the file is below 200mb's, it works just fine, but bigger than that, the browser crashes.
Chunks just work on the online version of sharepoint.. cound't make it work on On Premise.
Already though on creating an webApi in C# to receive the chunks and group it together and upload it to the library..
Anyone have ever done something like it? does anyone have any sugestion?
In SharePoint On Premise, you can try to increase the maxRequestLength="51200" executionTimeout="999999" in the web.config file at "C:\Inetpub\wwwroot\wss\VirtualDirectories\< Virtual Directory >" folder.
Or check maximum upload size for the web application : Central Admin> Application Management> Manage Web Applications> Select the desired web app and click General Settings on the ribbon.
When Ext JS issues a DELETE request from a restful store, it includes an entity body. Although this doesn't seem to be forbidden by the HTTP spec, Google App Engine doesn't accept such requests. So I'd like to know if there is a way to prevent a restful store from including a redundant entity body on DELETE requests.
Details:
Using this sample as reference:
http://www.sencha.com/deploy/dev/examples/restful/restful.html
This is how the store is defined:
var store = new Ext.data.Store({
id: 'user',
restful: true, // <-- This Store is RESTful
proxy: proxy,
reader: reader,
writer: writer
});
After pressing the "Delete" button, this is the request Ext JS sends:
DELETE http://www.sencha.com/deploy/dev/examples/restful/app.php/users/6 HTTP/1.1
Host: www.sencha.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; pt-BR; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: pt-br,pt;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Content-Type: application/json; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: http://www.sencha.com/deploy/dev/examples/restful/restful.html
Content-Length: 10
Cookie: bb_sessionhash=8d75f5e42d576fb695a02bf1d24c9ff1; etc...
{"data":6}
When a request in this format (with the "data" content) is submitted to Google App Engine, it replies with:
400 Bad Request
You can fix this problem, as you guessed, by overriding a method in the HttpProxy class. First, add this code:
// Special HttpProxy that sends no body on DELETE requests
Ext.data.GAEHttpProxy = Ext.extend(Ext.data.HttpProxy, {
doRequest: function(action, rs, params, reader, cb, scope, arg) {
if(this.api[action]['method'].toLowerCase() == "delete") {
delete params.jsonData;
}
Ext.data.GAEHttpProxy.superclass.doRequest.call(this, action, rs, params, reader, cb, scope, arg);
}
});
Then, use this new class ("GAEHttpProxy") instead of HttpProxy in the rest of your code (for instance, when you create the proxy you use in your store shown above). This worked for me, and I hope it works for you!
Although the question is asked 7 years ago and we have sencha 6 now, the problem isn't solved OOTB yet. So here is my working solution:
Ext.define('My.Proxy', {
extend: 'Ext.data.proxy.Rest',
writer: {
type: 'json',
writeAllFields: true, // may be false, as you wish
transform: {
fn: function(data, request) {
return request.config.action === 'destroy' ? null : data;
},
scope: this
}
}
});
We could also do this check: request.config.method === 'DELETE' but for some reason it always returns false. So I recommend to stay with action === 'destroy'