Curl Post request to upload .zip file to Sharepoint returning "Compressed (zipped) folder invalid when downloading - rest

I have a curl Post request that attempts to upload the latest documents to our Sharepoint. It appears to work fine, the file I send is valid and opens fine before I send it. Once it arrives on sharepoint the filesize is the same but when I download it appears to be corrupted and wont open.
I have tried manually uploading the zip file to Sharepoint, it works fine this way.
I have also scanned the internet for some solution to the problem but I am unable to find anything.
curl -X POST
"http://12.3.456.78:8080/TEST-2.0/sharepoint?relativePath=Shared%20Documents%2FRelease%20Documents%2Fv1&teamSite=Test"
-H "accept: /" -H "Content-Type: multipart/form-data" -F "file=#Test.zip;type=application/x-zip-compressed"
I expect the file on sharepoint to be valid and open correctly after download but I get the following error.
"Windows cannot open the folder.
The Compressed (zipped) Folder 'C:\Test.zip' is invalid.'

The answer was within the code, although the file was sending, it was obviously not sending correctly which was causing the corruption.
The line that I added that made the difference was the following.
post.setEntity( new FileEntity(file) );
My full method can be found below, feel free to comment if anyone needs further information on this in the future.
public HttpResponse uploadFile( String teamSite,
String relativePath,
String accessToken,
File file ) throws IOException
{
String fileName = null;
HttpClient client = HttpClientBuilder.create().build();
fileName = file.getName().replaceAll( " ", "%20" );
relativePath = relativePath.replaceAll( " ", "%20" );
teamSite = teamSite.replaceAll( " ", "%20" );
HttpPost post = new HttpPost( URL );
post.setHeader( "Accept", "application/json;odata=verbose" );
post.setHeader( "Authorization", "Bearer " + accessToken );
post.setEntity( new FileEntity(file) ); //This line here
HttpResponse response = client.execute( post );
HttpEntity entity = response.getEntity();
#SuppressWarnings("unused")
String responseString = EntityUtils.toString( entity, "UTF-8" );
return response;
}

Related

SoapUI POST to REST with attached file in Groovy

I'm trying to POST to a Sharepoint REST service an attached file with SoapUI Pro. I've tried the examples at: https://support.smartbear.com/readyapi/docs/requests/attachment/rest.html
But with no luck.
It should work with POST with byte-array as body. But how do I do that in SoapUI and Groovy?
In the tool Insomnia it works with "Binary File".
I add these headers:
Accept: application/json;odata=verbose
Content-Type: application/octet-stream
Media type = multipart/mixed and Post QueryString
But the file won't be uploaded to SharePoint.
PowerShell code that works:
$headers = #{
'X-RequestDigest' = 'xxxxxxxxxxxxxxxxxxxxxxx'
'Accept' = 'application/json;odata=verbose'
}
$document = [System.IO.File]::ReadAllBytes('C:\temp\myFile.docx')
Invoke-RestMethod -Method Post -UseDefaultCredentials -Uri "https://xxxx.xxx/add(url='myFile.docx',%20overwrite=true)" -Headers $headers -Body $document
I tried to go through this as well a while ago but I found it easier to use HTTP to do this.
You may try to see if it fits your requirements
My groovy script for attachment :
// upload source file before import
// get uploading request
def source_file = context.expand( '${#TestCase#source_file_path}' )
log.info "upload $source_file"
def aPIToolsTestSuite = context.expand( '${#Project#APIToolsTestSuite}' ) // the test suite that contains the test case with the HTTP request
tc_name = "import - upload resource files"
request = testRunner.testCase.testSuite.project.testSuites[aPIToolsTestSuite].testCases[tc_name].getTestStepByName("Request 1").testRequest
// clear request from any existing attachment
for (a in request.attachments)
{
request.removeAttachment(a)
}
// attach file to upload
def file = new File(source_file)
if (file == null)
{
log.error "bad file name : $source_file"
}
else
{
// attach file and set properties
try{
def attachment = request.attachFile (file, true)
attachment.contentType = "application/octet-stream"
attachment.setPart("upload file '$source_file'")
}
catch (Exception e){
log.error "file ${file.name} : exception $e"
}
}
// now upload file - launch the request
def jsonSlurper = new groovy.json.JsonSlurper()
def TC;
def async = false
TC = testRunner.testCase.testSuite.project.getTestSuiteByName(aPIToolsTestSuite).getTestCaseByName(tc_name)
result = TC.run (context.getProperties(), async)
if (String.valueOf( result.status ) != "PASS")
{
msg = "unexpected failure during $tc_name when uploading $source_file"
testRunner.fail(msg)
log.error msg
}
else
{
// this part is for further processing
// file uploaded, go through the import and properties backup process
resource_to_import = TC.getPropertyValue("testResponse").split('\"')[1]
// file uploaded, go through the import and properties backup process
testRunner.testCase.setPropertyValue("resource_id", resource_to_import)
}
And the HTTP request contained in the test case APIToolsTestSuite/import - upload resource files
first step : get endpoint
def env = testRunner.testCase.testSuite.project.activeEnvironment
rest = env.getRestServiceAt(0)
config = rest.getEndpoint().config
endpoint = new XmlSlurper().parseText(config.toString())
testRunner.testCase.setPropertyValue("endpoint", endpoint.toString())
second step, HTTP request:
POST
with Request tab parameters :
name : metadata
value : {"storageType":"FILESYSTEM","itemName":"my_source_file"}
type : QUERY
media type : multipart/form-data
Post QueryString
Headers : application/json
Good luck :)

Groovy script for Jenkins: execute HTTP request without 3rd party libraries

I need to create a Groovy post build script in Jenkins and I need to make a request without using any 3rd party libraries as those can't be referenced from Jenkins.
I tried something like this:
def connection = new URL( "https://query.yahooapis.com/v1/public/yql?q=" +
URLEncoder.encode(
"select wind from weather.forecast where woeid in " + "(select woeid from geo.places(1) where text='chicago, il')",
'UTF-8' ) )
.openConnection() as HttpURLConnection
// set some headers
connection.setRequestProperty( 'User-Agent', 'groovy-2.4.4' )
connection.setRequestProperty( 'Accept', 'application/json' )
// get the response code - automatically sends the request
println connection.responseCode + ": " + connection.inputStream.text
but I also need to pass a JSON in the POST request and I'm not sure how I can do that. Any suggestion appreciated.
Executing POST request is pretty similar to a GET one, for example:
import groovy.json.JsonSlurper
// POST example
try {
def body = '{"id": 120}'
def http = new URL("http://localhost:8080/your/target/url").openConnection() as HttpURLConnection
http.setRequestMethod('POST')
http.setDoOutput(true)
http.setRequestProperty("Accept", 'application/json')
http.setRequestProperty("Content-Type", 'application/json')
http.outputStream.write(body.getBytes("UTF-8"))
http.connect()
def response = [:]
if (http.responseCode == 200) {
response = new JsonSlurper().parseText(http.inputStream.getText('UTF-8'))
} else {
response = new JsonSlurper().parseText(http.errorStream.getText('UTF-8'))
}
println "response: ${response}"
} catch (Exception e) {
// handle exception, e.g. Host unreachable, timeout etc.
}
There are two main differences comparing to GET request example:
You have to set HTTP method to POST
http.setRequestMethod('POST')
You write your POST body to outputStream:
http.outputStream.write(body.getBytes("UTF-8"))
where body might be a JSON represented as string:
def body = '{"id": 120}'
Eventually it's good practice to check what HTTP status code returned: in case of e.g. HTTP 200 OK you will get your response from inputStream while in case of any error like 404, 500 etc. you will get your error response body from errorStream.

POST request on arduino with ESP8266 using WifiESP library

I am attempting to make RESTful POST request using the WifiESP library (https://github.com/bportaluri/WiFiEsp). I'm able to successfully make the request with curl, but consistently get an error using the Arduino and ESP. I suspect the problem is related to the manual formatting of the POST request the library requires, but I don't see anything wrong. Here my sanitized code:
if (client.connect(server, 80)) {
Serial.println("Connected to server");
// Make a HTTP request
String content = "{'JSON_key': 2.5}"; // some arbitrary JSON
client.println("POST /some/uri HTTP/1.1");
client.println("Host: http://things.ubidots.com");
client.println("Accept: */*");
client.println("Content-Length: " + sizeof(content));
client.println("Content-Type: application/json");
client.println();
client.println(content);
}
The error I get (via serial monitor) is this:
Connected to server
[WiFiEsp] Data packet send error (2)
[WiFiEsp] Failed to write to socket 3
[WiFiEsp] Disconnecting 3
My successful curl requests looks like this:
curl -X POST -H "Content-Type: application/json" -d 'Some JSON' http://things.ubidots.com/some/uri
After some experimentation, here is the solution to the multiple problems.
The JSON object was not correctly formatted. Single quotes were not accepted, so I needed to escape the double quotes.
The host does not need "http://" in a POST request; POST is a HTTP method.
The sizeof() method returns the size, in bytes, of the variable in memory rather than the length of the string. It needs to be replaced by .length().
Appending an integer to a string requires a cast.
This is the corrected code:
if (client.connect(server, 80)) {
Serial.println("Connected to server");
// Make the HTTP request
int value = 2.5; // an arbitrary value for testing
String content = "{\"JSON_key\": " + String(value) + "}";
client.println("POST /some/uri HTTP/1.1");
client.println("Host: things.ubidots.com");
client.println("Accept: */*");
client.println("Content-Length: " + String(content.length()));
client.println("Content-Type: application/json");
client.println();
client.println(content);
}
The code explained by Troy D is right and it's working .I think the error in posting the data to the server is due to this line
client.println("Content-Length: " + sizeof(content));
and the correct way is
client.println("Content-Length: " + String(content.length()));
Now coming to this error
Connected to server
[WiFiEsp] Data packet send error (2)
[WiFiEsp] Failed to write to socket 3
[WiFiEsp] Disconnecting 3
This is the error of library you can ignore it.
The problem with "Data packet send error (2)", "Failed to write to socket 3" and "Disconnecting 3" is not a problem within the WifiEsp library as far as I can see, believe it's more likely to be within the AT firmware. By default the http headers contain a "Connection: close" parameter which in normal cases should be correct. However with this bug the server will get disconnected before the reply is received on the client side and any response from the server will be identified as garbage data. Using the value "Connection: keep-alive" as a workaround will make it possible to receive the acceptance from the server in a proper way.
I'm running my Arduino + ESP8266-07 against a MVC based Web Api that I created on one of my servers and in the controllers Post-method I use a single string as return value, the value I return if everything is ok is simply one of the strings that WifiEsp keeps track of (It will still include the http status code in the response header that it returns)
public async Task<string> Post([FromBody]JObject payload)
{
//Code to handle the data received, in my case I log unit ip, macaddress, datetime and sensordata into a db with entity framework
return "SEND OK";
}
So in your Arduino code try following instead:
String PostHeader = "POST http://" + server + ":" + String(port) + "/api/values HTTP/1.1\r\n";
PostHeader += "Connection: keep-alive\r\n";
PostHeader += "Content-Type: application/json; charset=utf-8\r\n";
PostHeader += "Host: " + server + ":" + String(port) + "\r\n";
PostHeader += "Content-Length: " + String(jsonString.length()) + "\r\n\r\n";
PostHeader += jsonString;
client.connect(server.c_str(), port);
client.println(PostHeader);
client.stop();
In the file debug.h located in the library source code you could alter a define and get more output to your serial console. Open the file and change
#define _ESPLOGLEVEL_ 3
to
#define _ESPLOGLEVEL_ 4
Save the file and recompile/deploy your source code to your Arduino and you will get extensive information about all AT commands the library sends and what the library receives in return.

Unable to upload file via REST request in GRAILS

I am trying to upload a file via REST using GRAILS
curl -X POST -H "Cache-Control: no-cache" -H "Postman-Token: d5d7aef8-3964-311b-8b64-4a7a82c52323" -H "Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW" -F "file1=myfile.jpg" -F "fname=swateek" -F "lname=jena" 'http://localhost:8081/sampleFileREST/document/upload'
Here's how my controller looks like:
class DocumentController extends RestfulController<Document> {
static responseFormats = ['json', 'xml']
def upload() {
def fileLocation="<xml>empty</xml>"
def file = request.getParameter('file1')
def f1 = request.getParameter('fname')
def f2 = "<abc>"+request.getParameter('lname')+"</abc>"
def params = "gf"
if(file.empty) {
fileLocation = "<xml>"+"File cannot be empty"+"</xml><allprm>"+params+"</allprm>"
} else {
def documentInstance = new Document()
documentInstance.filename = file.originalFilename
documentInstance.fullPath = grailsApplication.config.uploadFolder + documentInstance.filename
file.transferTo(new File(documentInstance.fullPath))
documentInstance.save()
fileLocation = "<xml>"+documentInstance.fullPath+"</xml>"
}
/* return "File uploaded to: "+documentInstance.fullPath */
render(text: fileLocation, contentType: "text/xml", encoding: "UTF-8")
}
}
I am able to access the parameters of the request, anything except the file I am sending in the request.
Unable to figure out what's wrong here.
UPDATE
I had used .getParameter() to fetch a file. That's incorrect, the correct way is as below:
request.getFile('<filename>') // without the <>
This might raise an error in IntelliJ as "Symbol Not Found" or "Cannot Resolve Method", please follow the procedure in the answer below.
Damn the IDE that I was using, IntelliJ.
Also, this piece of code while getting the file:
def file = request.getParameter('file1')
should be replaced as
def file = request.getFile('file1')
Now previously, when I was using the request.getFile() method I was getting an "Symbol Not Found" error and it was failing to execute the request.
Solution:
Open IntelliJ
Click on "File"
Find the option "Invalidate Caches/Restart" and wait for IntelliJ to come back again.
If this doesn't work, the other way is mentioned in this answer:
IntelliJ IDEA JDK configuration on Mac OS

Uploading files to Onedrive using REST API

I am trying to upload a file into OneDrive using its REST API. This is what I am trying to accomplish based on documentation available at OneDrive Rest API:
POST https://apis.live.net/v5.0/me/skydrive/files?access_token=ACCESS_TOKEN
Content-Type: multipart/form-data; boundary=A300x
--A300x
Content-Disposition: form-data; name="file"; filename="HelloWorld.txt"
Content-Type: application/octet-stream
Hello, World!
--A300x--
This is what I have:
Uri destination = new Uri(string.Format("https://apis.live.net/v5.0/{0}/files?", folder.ID));
BackgroundUploader uploader = new BackgroundUploader ();
uploader.SetRequestHeader("Authorization", "Bearer " + account.AccessToken);
uploader.SetRequestHeader("Content-Type", "multipart/form-data; boundary=\"foo_bar_baz\"");
List<BackgroundTransferContentPart> parts = new List<BackgroundTransferContentPart>();
BackgroundTransferContentPart metaDataPart = new BackgroundTransferContentPart();
metaDataPart.SetHeader("Content-Disposition", "form-data; name=\"file\";filename=\""+content.Name+"\"");
parts.Add(metaDataPart);
BackgroundTransferContentPart contentPart = new BackgroundTransferContentPart();
contentPart.SetHeader("Content-Type", content.ContentType);
// content is a StorageFile
contentPart.SetFile(content);
response.UploadOperation = await uploader.CreateUploadAsync(destination, parts, "form-data", "foo_bar_baz");
This line below causes an Access violation error and the Windows Store app crashes:
response.UploadOperation = await uploader.CreateUploadAsync(destination, parts, "form-data", "foo_bar_baz");
You are creating two BackgroundTransferContentPart and only adding the fist to your 'List'.
I think you only need one, something like this:
List<BackgroundTransferContentPart> parts = new List<BackgroundTransferContentPart>();
BackgroundTransferContentPart metaDataPart = new BackgroundTransferContentPart();
metaDataPart.SetHeader("Content-Disposition",
"form-data; name=\"file\";filename=\"" + content.Name + "\"");
metaDataPart.SetHeader("Content-Type", content.ContentType);
metaDataPart.SetFile(content);
parts.Add(metaDataPart);
UPDATE: Ok, the above code fixed the Access Violation issue. Why you are getting a 400 error is a mystery.
But another way to upload a file to OneDrive is using the PUT method:
Uri putUri = new Uri(string.Format("https://apis.live.net/v5.0/{0}/files/{1}",
"folder.a4fb14adbccd1917.A4FB14ADBCCD1917!32089",
content.Name));
BackgroundUploader uploader = new BackgroundUploader();
uploader.SetRequestHeader("Authorization", "Bearer " + accessToken);
uploader.Method = "PUT";
UploadOperation putOperation = uploader.CreateUpload(putUri, content);
await putOperation.StartAsync();
Have you tried PUT?