I have this powershell script that uploads a file :
$URL = 'DESTINATION';
function UploadFile {
param (
$path,
$filename
)
$LF = "`r`n";
$fileBytes = [System.IO.File]::ReadAllBytes($path);
$fileEnc = [System.Text.Encoding]::GetEncoding('UTF-8').GetString($fileBytes);
$boundary = [System.Guid]::NewGuid().ToString();
$bodyLines = (
"--$boundary",
"Content-Disposition: form-data; filename=`"$filename`"",
"Content-Type: multipart/form-data$LF",
$fileEnc,
"--$boundary--$LF"
) -join $LF;
Invoke-RestMethod -Uri $URL -Method Post -ContentType "multipart/form-data; boundary=`"$boundary`"" -Body $bodyLines;
}
And my file is a .sqlite database.
The issue is that even if the file is actually uploaded, the .sqlite file is corrupted and can't be read with DB Browser for example. I precise that the original file before upload is not corrupted.
I presume there must be something wrong with the readfile/upload process, but I can't see what is wrong. Can someone please help me?
Related
I'm trying to upload a file to an ASP.NET Core API using PowerShell. My problem is that the API returns a status code 400 saying that form values are missing. Here's my PowerShell code:
add-type -AssemblyName System.Net.Http
$boundary = [System.Guid]::NewGuid().ToString()
$multipartContent = [System.Net.Http.MultipartFormDataContent]::new($boundary)
$multipartContent.Headers.Remove("Content-Type")
$multipartContent.Headers.TryAddWithoutValidation("Content-Type", "multipart/form-data; boundary=`"$boundary`"")
$stringContent = [System.Net.Http.StringContent]::new("This is a file I'm uploading")
$multipartContent.Add($stringContent, "Description")
$FilePath = "c:\path\to\file.json"
$FileStream = [System.IO.File]::OpenRead($FilePath)
$streamContent = [System.Net.Http.StreamContent]::new($FileStream)
$streamContent.Headers.ContentType = "application/json"
$multipartContent.Add($streamContent, "TheFile", "file.json")
$body = $multipartContent
$response = Invoke-RestMethod 'https://myapi.com/uploadfile' -Method 'POST' -Body $body
The code is pretty much migrated from a C# sample I have and is working. Can anyone see what I am doing wrong there?
As per my comment above, the Web cmdlets in PowerShell did not directly support mutlipart/form-data last I checked and most of the working examples create some kind of http template for the -Body parameter.
Here's a working sample:
$boundary = [System.Guid]::NewGuid().ToString()
$FilePath = "c:\path\to\file.json"
$TheFile = [System.IO.File]::ReadAllBytes($FilePath)
$TheFileContent = [System.Text.Encoding]::GetEncoding('iso-8859-1').GetString($TheFile)
$LF = "`r`n"
$bodyLines = (
"--$boundary",
"Content-Disposition: form-data; name=`"Description`"$LF",
"This is a file I'm uploading",
"--$boundary",
"Content-Disposition: form-data; name=`"TheFile`"; filename=`"file.json`"",
"Content-Type: application/json$LF",
$TheFileContent,
"--$boundary--$LF"
) -join $LF
Invoke-RestMethod 'https://myapi.com/uploadfile' -Method POST -ContentType "multipart/form-data; boundary=`"$boundary`"" -Body $bodyLines
Your other option if you want to keep it as close as possible to the C# code you have is to use HttpClient or whatever you are currently using instead of the web cmdlets in PowerShell.
Update: PowerShell 7+ does include a -Form parameter on both Web Cmdlets. Please see the linked examples from the official documentation:
Invoke-RestMethod
Invoke-WebRequest
I'm trying to upload a zip file to Nexus repository after converting the zipe file (1.6gb) to bytes.
The problem is when it readallbytes of the zip file, if the file size is big it throw system.outofmemory error. it doesn't throw the error if it's just 3-600mb, is there a way to read bytes and avoid the oufofmemory error?
Exception calling "ReadAllText" with "1" argument(s): "Exception of type 'System.OutOfMemoryException' was thrown."
$fileBin = [System.IO.File]::ReadAlltext($File)
Function Upload-File-To-Nexus {
[CmdletBinding()]
Param(
[Parameter(Mandatory = $True )][ValidateNotNullOrEmpty()][string]$File, # aka File
[parameter(Mandatory = $True )][ValidateNotNullOrEmpty()][string]$Repository,
[parameter(Mandatory = $True )][ValidateNotNullOrEmpty()][string]$Directory,
[Parameter(Mandatory = $False)][ValidateNotNullOrEmpty()][string]$FileName # destination filename, which can be derived from uploaded file
)
Begin {
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes("fahq-ra-build:tEDlCI=0m9CES8l*lk?b"))
$header = #{authorization = "Basic $token" }
$Uri = "https://nexus-arps.corp.firstam.com/service/rest/v1/components?repository=${Repository}"
$boundary = [System.Guid]::NewGuid().ToString()
$LF = "`r`n"
}
Process {
If (!($FileName -ne $Null -And $FileName -ne "")) {
# if filename is null, then just use the filename from the input file
$FileName = Split-Path $File -Leaf
}
Try { $fileBin = [System.IO.File]::ReadAlltext($File) }
Catch {
throw $_.exception
Throw "Unable to read file $File. Aborted."
}
$bodyLines = (
"--${boundary}",
"Content-Disposition: form-data; name=`"raw.directory`"",
"",
"${Directory}",
"--${boundary}",
"Content-Disposition: form-data; name=`"raw.asset1`"; filename=`"${FileName}`"",
"Content-Type: application/octet-stream",
"",
$file,
"",
"--${boundary}",
"Content-Disposition: form-data; name=`"raw.asset1.filename`"",
"",
"${FileName}",
"--${boundary}--",
""
) -join $LF
$Response = Invoke-WebRequest -Uri $Uri -Method "POST" -Headers $header -ContentType "multipart/form-data; boundary=`"$boundary`"" -Body $bodyLines
If ($Response.StatusCode -ge 200 -And $Response.StatusCode -lt 300) {
$Output = "https://${Server}/repository/${repository}/${directory}/${FileName}" -Replace "//", "/"
Write-Output $Output
}
Else {
Write-Output $Response
}
}
}
You probably meant .ReadAllBytes(), given that your'e not dealing with a text-based files.
However, you don't have to load the file into memory - use Invoke-WebRequest's -InFile parameter to specify a file to upload.
IIRC, the way ReadAllBytes() works is it doesn't know or lookup the file size up front. Instead, it guesses a buffer size for the resulting byte array. When the guess fails, it doubles the guess, allocates a new array, and copies what was read so far. This repeats until the file is loaded.
You get OutOfMemory, then, not only from actually running out of memory but also from running out address space in the current process from the extra buffer, where individual processes are limited by default to only 2GB.
If that sounds bad and inefficient, you're right, and RealAllBytes() is really only useful for small files.
When dealing with larger files, you need to use the streaming APIs, such as File.OpenRead(). Here you can check the actual size of the file, use that to allocate the exact size byte array, and then read in a small chunk (say, 4K) at a time.
Even better, since you're already using Invoke-WebRequest did you know it supports a -InFile argument, which can be used to upload a file without loading the entire thing into memory?
I want to attach a file to an jira issue , i am able to do it with postman, but have tried several ways without break through.
my code looks like this
function ConvertTo-Base64($string) {
$bytes = [System.Text.Encoding]::UTF8.GetBytes($string);
$encoded = [System.Convert]::ToBase64String($bytes);
return $encoded;
}
function Get-HttpBasicHeader([string]$username, [string]$password, $Headers = #{}) {
$b64 = ConvertTo-Base64 "$($username):$($Password)"
$Headers["Authorization"] = "Basic $b64"
$Headers["X-Atlassian-Token"] = "nocheck"
return $Headers
}
$restapiuri = "https://xxxx.xxxx.com/rest/api//2/issue/test-8442/attachments"
$headers = Get-HttpBasicHeader "xxxxxx" "xxxxxxxx"
$myfile = "C:\TEMP\out.txt"
$fileBytes = [System.IO.File]::ReadAllBytes($myfile);
$fileEnc = [System.Text.Encoding]::GetEncoding('UTF-8').GetString($fileBytes);
$boundary = [guid]::NewGuid().ToString()
$LF = "`r`n";
$body = '(
"--$boundary",
"Content-Disposition: form-data; name=`"fil`"; filename=`"out.txt`"",
"Content-Type: application/octet-stream$LF",
$fileEnc,
"--$boundary--$LF"
) -join $LF
'
Invoke-RestMethod -uri $restapiuri -Headers $headers -Method POST -ContentType "multipart/form-data; boundary=`"$boundary`"" -Body $body
In postman i am doing a post request with
Authorization
Basic Auth
headers
header X-Atlassian-Token = o-check
Content-Type = multipart/form-data
Body form-data
key value
file filepath
The powershell doesn't return any errors, but it does not attach any file either
I have tried several examples without luck so if any of you have any ideas on have to do this in powershell i would be glad.
I am on powershell version
Major Minor Build Revision
5 1 14409 1018
I found the following that looks to be a good fit for this scenario.
function Upload-JiraFile($jiraTicket, $filepath, $authorization)
{
$wc = new-object System.Net.WebClient
$wc.Headers.Add("Authorization", $authorization)
$wc.Headers.Add("X-Atlassian-Token", "nocheck")
$wc.UploadFile("$URIPath/issue/$jiraTicket/attachments", $filepath)
}
Here is how you could use it with the bare minimal modification to the code you have today:
$URIPath = "https://xxxx.xxxx.com/rest/api/2"
Upload-JiraFile -JiraTicket test-8442 -FilePath c:\temp\MyJpg.jpg `
-Authorization $Headers["Authorization"]
Excerpted from this thread.
I try to upload a file to a uri with POST. I somehow fail. I got help from
powershell invoke-restmethod multipart/form-data
but they uploaded a text file.
Also the curl command works:
curl -X POST "https://some.place.over.the.rainbow.com/api/v1/dataupdate" -H "accept: */*" -H "Authorization: Basic 123123123123" -F "file=#/c/updates/SomeFile.zip"
In my ps1 Script I tried following:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$Headers = #{
Authorization = "Basic 123123123123"
};
$FilePath = "C:\Users\asd\SomeFile.zip";
$Uri = 'ttps://some.place.over.the.rainbow.com/api/v1/dataupdate';
$LF = "`r`n";
$boundary = [System.Guid]::NewGuid().ToString();
$fileBytes = [System.IO.File]::ReadAllBytes($FilePath);
$fileEnc = [System.Text.Encoding]::GetEncoding('UTF-8').GetString($fileBytes);
$bodyLines = (
"--$boundary",
"Content-Disposition: form-data; name=`"file`"; filename=`"SomeFile.zip`"",
$fileEnc,
"--$boundary--$LF"
) -join $LF
Invoke-RestMethod -Uri $Uri -Method Post -ContentType "multipart/form-data; boundary=`"$boundary`"" -Headers $Headers -Body $bodyLines
On Serverside I get
Servlet.service() for servlet [dispatcherServlet] in context with path [/...] threw exception [Request processing failed; nested exception is
org.springframework.web.multipart.MultipartException: Failed to parse multipart servlet request;
nested exception is java.io.IOException: org.apache.tomcat.util.http.fileupload.FileUploadException:
Header section has more than 10240 bytes (maybe it is not properly terminated)] with root cause
org.apache.tomcat.util.http.fileupload.MultipartStream$MalformedStreamException: Header section has more than 10240 bytes (maybe it is not properly terminated)at
org.apache.tomcat.util.http.fileupload.MultipartStream.readHeaders(MultipartStream.java:523)
~[tomcat-embed-core-8.5.34.jar!/:8.5.34]at org.apache.tomcat.util.http.fileupload.FileUploadBase$FileItemIteratorImpl.findNextItem(FileUploadBase.java:880)
~[tomcat-embed-core-8.5.34.jar!/:8.5.34]at org.apache.tomcat.util.http.fileupload.FileUploadBase$FileItemIteratorImpl.<init>(FileUploadBase.java:845)
I need to execute it on a windows machine where I do not haver curl unfortunately.
Appreciate any help
I'm trying to send a file via Invoke-RestMethod in a similar context as curl with the -F switch.
Curl Example
curl -F FileName=#"/path-to-file.name" "https://uri-to-post"
In powershell, I've tried something like this:
$uri = "https://uri-to-post"
$contentType = "multipart/form-data"
$body = #{
"FileName" = Get-Content($filePath) -Raw
}
Invoke-WebRequest -Uri $uri -Method Post -ContentType $contentType -Body $body
}
If I check fiddler I see that the body contains the raw binary data, but I get a 200 response back showing no payload has been sent.
I've also tried to use the -InFile parameter with no luck.
I've seen a number of examples using a .net class, but was trying to keep this simple with the newer Powershell 3 commands.
Does anyone have any guidance or experience making this work?
The accepted answer won't do a multipart/form-data request, but rather a application/x-www-form-urlencoded request forcing the Content-Type header to a value that the body does not contain.
One way to send a multipart/form-data formatted request with PowerShell is:
$ErrorActionPreference = 'Stop'
$fieldName = 'file'
$filePath = 'C:\Temp\test.pdf'
$url = 'http://posttestserver.com/post.php'
Try {
Add-Type -AssemblyName 'System.Net.Http'
$client = New-Object System.Net.Http.HttpClient
$content = New-Object System.Net.Http.MultipartFormDataContent
$fileStream = [System.IO.File]::OpenRead($filePath)
$fileName = [System.IO.Path]::GetFileName($filePath)
$fileContent = New-Object System.Net.Http.StreamContent($fileStream)
$content.Add($fileContent, $fieldName, $fileName)
$result = $client.PostAsync($url, $content).Result
$result.EnsureSuccessStatusCode()
}
Catch {
Write-Error $_
exit 1
}
Finally {
if ($client -ne $null) { $client.Dispose() }
if ($content -ne $null) { $content.Dispose() }
if ($fileStream -ne $null) { $fileStream.Dispose() }
if ($fileContent -ne $null) { $fileContent.Dispose() }
}
The problem here was what the API required some additional parameters. Initial request required some parameters to accept raw content and specify filename/size. After setting that and getting back proper link to submit, I was able to use:
Invoke-RestMethod -Uri $uri -Method Post -InFile $filePath -ContentType "multipart/form-data"
I found this post and changed it a bit
$fileName = "..."
$uri = "..."
$currentPath = Convert-Path .
$filePath="$currentPath\$fileName"
$fileBin = [System.IO.File]::ReadAlltext($filePath)
$boundary = [System.Guid]::NewGuid().ToString()
$LF = "`r`n"
$bodyLines = (
"--$boundary",
"Content-Disposition: form-data; name=`"file`"; filename=`"$fileName`"",
"Content-Type: application/octet-stream$LF",
$fileBin,
"--$boundary--$LF"
) -join $LF
Invoke-RestMethod -Uri $uri -Method Post -ContentType "multipart/form-data; boundary=`"$boundary`"" -Body $bodyLines
For anyone wondering (like Jelphy) whether David's answer can be used with cookies/credentials, the answer is yes.
First set the session with Invoke-WebRequest:
Invoke-WebRequest -Uri "$LoginUri" -Method Get -SessionVariable 'Session'
Then POST to the Login URL, which stores the authentication cookie in $Session:
$Response = Invoke-WebRequest -Uri "$Uri" -Method Post -Body $Body -WebSession $Session
The steps above are the standard way to deal with session in Powershell. But here is the important part. Before creating the HttpClient, create an HttpClientHandler and set it's CookieContainer property with the cookies from the session:
$ClientMessageHandler = New-Object System.Net.Http.HttpClientHandler
$ClientMessageHandler.CookieContainer = $Session.Cookies
Then pass this object to the HttpClient constructor
$Client = [System.Net.Http.HttpClient]::new($ClientMessageHandler)
Voila, you now have an HttpClient with session cookies set automatically via Invoke-WebRequest. The rest of David's example should work (copied here for completeness):
$MultipartFormData = New-Object System.Net.Http.MultipartFormDataContent
$FileStream = [System.IO.File]::OpenRead($FilePath)
$FileName = [System.IO.Path]::GetFileName($FilePath)
$FileContent = New-Object System.Net.Http.StreamContent($FileStream)
$MultipartFormData.Add($FileContent, $FieldName, $FileName)
$Result = $Client.PostAsync($url, $content).Result
$Result.EnsureSuccessStatusCode()
I had many files to upload with each request, so I factored out this last bit into a lambda function:
function Add-FormFile {
param ([string]$Path, [string]$Name)
if ($Path -ne "")
{
$FileStream = [System.IO.File]::OpenRead($Path)
$FileName = [System.IO.Path]::GetFileName($Path)
$FileContent = [System.Net.Http.StreamContent]::new($FileStream)
$MultipartFormData.Add($FileContent, $Name, $FileName)
}
}