How do I build a cURL command that will follow redirects, but only return the content from the final (200) response? For example, I run curl.exe -LIk securityheaders.com and it returns the 301 response headers and the 200 response headers. I want to assign a variable to that cURL and only have the regex from the 200 response headers to check against a set of conditions.
You are using -LIk that means:
-k, --insecure Allow insecure server connections (OK)
-L, --location Follow redirects (OK)
-I, --head Show document info only (WHY)
When you put -I you are request for the document info only when all you want to is the content, just remove -I.
curl -Lk securityheaders.com
<!DOCTYPE HTML>
<html lang="en">
...
Related
Based on https://github.com/istio/istio/issues/23757 it's been quite some time now that I get no answer about it, so I realized that most likely I don't understand the internals and I need to shift the question in another diretion.
I have a case where vulnerability scan shows that we are vulnerable to custom origin domains. The expectation from the provider is to block request that don't match a predefined ORIGIN within a virtual service allowOrigin setting.
I am trying to send OPTIONS preflights or simple gets, but no matter what I do the mesh always returns 200:
curl -s -H "Origin: http://fake" --verbose http://192.168.223.10:31380/productpage | grep -i "HTTP/1.1 200 OK"
curl -s -H "Origin: http://testit.com" --verbose http://192.168.223.10:31380/productpage | grep -i "HTTP/1.1 200 OK"
curl -s -X OPTIONS -H "Origin: http://testit.com" --verbose http://192.168.223.10:31380/productpage | grep -i "HTTP/1.1 200 OK"
curl -s -X OPTIONS -H "Origin: http://fake" --verbose http://192.168.223.10:31380/productpage | grep -i "HTTP/1.1 200 OK"
Is this something that controls only client blocking (browser) and if so how am I supposed to test it with curl?
I know how to reject a origin like, but it will just return not found then:
- uri:
exact: /productpage
headers:
origin:
regex: "*test.com"
There is an answer to this issue on github:
Hi everyone. Testing CORS using curl can be a bit misleading. CORS is not enforced at the server side; it will not return a 4xx error for example. Instead, headers are returned back which are used by browsers to deny/accept. https://www.envoyproxy.io/docs/envoy/latest/start/sandboxes/cors.html?highlight=cors gives a good demo of this, and https://www.sohamkamani.com/blog/2016/12/21/web-security-cors/#:~:text=CORS%20isn't%20actually%20enforced,header%20in%20all%20its%20responses. is a good explanation.
So Istio's job here is simply to return these headers. I have added a test showing this works: https://github.com/istio/istio/pull/26231
I am trying to convert a curl call to python request . The curl call looks like
$ curl -b $COOKIE_FILE \
-X FIND $URL \
-H 'Content-Type: application/json; charset=utf-8' \
I tried using request to get the correct response via requests.post() but i could not get the correct response. Can someone help me with this
Requests allows using arbitrary request methods with the request function, take a look at the documentation.
For example this is how it would look with a Session() object:
import requests
with requests.Session() as sess:
sess.headers = {
#headers in dict format, you could add cookies here too(or set it to sess.cookies)
}
response = sess.request("FIND", url)
I am looking to test my REST resource on dropwizard using CURL command. I am able to upload the file and get the content and file name. But along with the file I want to pass some list of Ids as well.
How can I pass list of long ids (array of Long data type) to the REST service in the below CURL command?
Can I pass a json structure additionally in the below command if required along with my file content?
CURL command
curl -F 'file=#/cygdrive/c/TestDocument1.txt' http://localhost:8199/test-app/api/upload-documents/1004/documents
REST service to upload file
#POST
#Consumes(MediaType.MULTIPART_FORM_DATA)
#Path("/reconciliation-details/{reconciliationDetailId}/documents")
public Response uploadDocument(#FormDataParam("file") File inventoryDocumentContent,
#FormDataParam("file") FormDataContentDisposition fileDetail,
#FormDataParam("reconciliationIds") List<Long> reconciliationIds) throws Exception {
byte[] documentContent = FileUtils.readFileToByteArray(inventoryDocumentContent);
String documentName = fileDetail.getFileName();
reconDetailsService.uploadDocument(documentName, documentContent, reconciliationIds);
return ResponseHelper.createOkResponse();
}
With multipart, each part can have it's own Content-Type header. By default if it is not specified, text/plain is used. Some clients cannot set the Content-Type for individual parts, as mentioned in this post (for which there is a solution mentioned), but cURL has this capability. You just need to append ;type=<content-type>. For example
-F "reconciliationIds=[1, 2, 3, 4];type=application/json"
Instead of the using the actual JSON array, you could also use a file containing the JSON
-F "reconciliationIds=#path-to-json;type=application/json"
Jersey will see that the Content-Type for this part is application/json and use the deserializer that it would normally use for JSON
Make sure to see the linked post above - Even though this may work, you don't want to limit your users to only using clients with this capability. The bottom of the post gives you the solution, though I have not tested it with a List<Long>. I'm not sure how that would work using the getEntityAs(Class), as you wouldn't be able to pass the generic Long type. You could always wrap it in a POJO instead of just using a list.
You need to set your content-type to application/json. But -d sends the Content-Type application/x-www-form-urlencoded, which is not accepted on Spring's side.
Looking at the curl man page, I think you can use -H:
-H "Content-Type: application/json"
Full example:
curl -H "Content-Type: application/json" -X POST -d '{"username":"xyz","password":"xyz"}' http://localhost:3000/api/login
(-H is short for --header, -d for --data.)
For Json with file
curl -i -X POST -H "Content-Type: multipart/mixed" -F "blob=#/Users/username/Documents/bio.jpg" -F "metadata={\"edipi\":123456789,\"firstName\":\"John\",\"lastName\":\"Smith\",\"email\":\"john.smith#gmail.com\"};type=application/json" http://localhost:8080/api/v1/user/
I have a POST method in my API which uses multipart encoded form data. I have set up the correct header and data settings so that the mashape web interface generated the following curl:
curl -X POST --include 'https://sslavov-text-analytics-v1.p.mashape.com/news' \
-H 'Authorization: Basic ***********' \
-H 'X-Mashape-Key: ************' \
-H 'Content-Type: multipart/form-data' \
-F 'file=#sample.docx' \
-F 'meta={"documentType": "application/vnd.openxmlformats-officedocument.wordprocessingml.document"};type=application/json'
Basically i'm trying to upload a file with a simple paragraph of text for processing. The curious part is that when I run this exact curl in a bash script, everything works smoothly, but when I try to run it through mashape, it says either 400 Bad Request or 500 Internal Server Error
In my particular case, these errors are generated when I don't pass correct form or headers. So my question is: Is there an error in the curl syntax or should I keep looking for the error on server side?
EDIT: I figured out what the problem was. -F 'file=#sample.docx' was passed before -F 'meta....' and that was causing the 500 Internal Server Error So now the question is: Is there any way to specifically arrange the order of the form fields (because mashape rearranges them aplhabetically)?
How to make a request for uploading a file to facebook using graph api in curl
I am making this request but getting Error:
curl https://graph.facebook.com/<id>/photos -F "source=#me.jpg" -d "message=Me" -v
ERR:
Only One Http Request can be Selected
You cannot mix -F and -d options in the same command line. If you want a multipart formpost (which I believe you do) then you need to add all parts with -F, including the "message=Me" part.