Ftp uploading failed after every two or three succeeded uploads - powershell

I got the the following error message when uploading. The Powershell create a zip file using 7za.exe and call my FTP function to upload the file. What may cause the problem? Will Windows ftp.exe client be more stable?
Exception calling "GetRequestStream" with "0" argument(s): "The remote
server returned an error: (550) File unavailable (e.g., file not
found, no access)."
Update:
It seems the same files always failed in the loop. However, It works if I just run ftpFile file_name_with_full_path. (The file_name_with_full_path is copied from the output of the loop script.
Update 2:
I tried to use webclient ($webclient.UploadFile($uri, $File)) to ftp the files. Same error.
Update 3:
Found this Question. May need to add $ftp.KeepAlive = false. Why?
function ftpFile
{
Param (
[Parameter(Mandatory=$true, ValueFromPipeline=$true)]
[ValidateScript({Test-Path $_})]
[String]
$filePath
,
[Parameter(Mandatory=$false)]
[String]
$ftpUrl = "ftp://10.0.1.1/Data/"
,
[Parameter(Mandatory=$false)]
[String]
$Login = "username"
,
[Parameter(Mandatory=$false)]
[String]
$password = "password"
)
Process {
try {
$ftp = [System.Net.FtpWebRequest]::Create("$ftpUrl/$(Split-Path $filePath -Leaf)")
$ftp = [System.Net.FtpWebRequest]$ftp
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential("$Login","$password")
$ftp.UseBinary = $true
$ftp.UsePassive = $true
# read in the file to upload as a byte array
$content = gc -en byte $filePath
$ftp.ContentLength = $content.Length
# get the request stream, and write the bytes into it
$rs = $ftp.GetRequestStream()
$rs.Write($content, 0, $content.Length)
$rs.Close()
$rs.Dispose()
echo "ftpFile: $filePath size: $($content.Length)"
}
catch {
throw "FTP: $_"
}
}
}

FTP Error 550 is Access Denied and tends to be a username/password conflict.
If you're using a loop and passing the same username and password each time this function is called
AND
it's working on some iterations of the loops and not others
THEN
you need to check the ftp auth/error logs on the server to get a grasp of why you're being denied.
As #AndyArismendi asked, does it always fail on the same file? Without more complete code and understanding of your use, this is hard to lock down to a simple solution.

Found this Question. Need to add $ftp.KeepAlive = false.

Related

Uploading a File via FTP using PowerShell

I am writing a Powershell script that watches a directory, and when a file (or multiple) is uploaded into the directory, it takes those files, copies them to another folder, sends them to an FTP server, and then deletes the file from the original directory.
I am having problems connecting to the FTP server. I am not sure if the problem is the way I am configuring the Web Client, or if the problem is that the ftp URI has spaces in it and I am not escaping them properly.
Here is the code:
$source = "c:/testFtp"
$ftpdestination = "ftp://username:password#ftp.ftpsite.com/folder with space/folder with space"
$webclient = New-Object -TypeName System.Net.WebClient
$files = Get-ChildItem $source
foreach ($file in $files)
{
Write-Host "Uploading $file"
try {
$ftp = "$ftpdestination/$file"
$uri = New-Object -TypeName System.Uri -ArgumentList $ftp
$webclient.UploadFile($uri, $source/$file)
} catch {
Add-content "$logs" -value "There was an error uploading to ftp"
}
}
$webclient.Dispose()
I have tried escaping the folder spaces multiple ways, so I am beginning to think that is not the problem and that I am not configuring the web client properly.
It is also not catching errors very often, so I don't believe it throws an error when the webclient has failure on the upload. Any help is appreciated!
ANSWER:
It turns out the WebClient was connecting properly, but SSL was blocking the files from being sent. I found this out by using C# to compile and run the script, and it gave me better error handling, as I am new to Powershell scripts and cannot seem to get good error handling.
After researching, I could not find a way to enable SSL with WebClient, so I switched over to FtpWebRequest. Here is the successful code (This try catch block does not seem to log errors as I would like, but the tool will successfully send files to the ftp server now:
try {
$ftp = [System.Net.FtpWebRequest]::Create("$ftpsite/$filename")
$ftp = [System.Net.FtpWebRequest]$ftp
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential("$username","$password")
$ftp.UseBinary = $true
$ftp.UsePassive = $true
$ftp.EnableSSL = $true #<-----------------This was the line that made this work
$ftp.KeepAlive = $false
# read in the file to upload as a byte array
$content = [System.IO.File]::ReadAllBytes("$source/$file")
$ftp.ContentLength = $content.Length
# get the request stream, and write the bytes into it
$rs = $ftp.GetRequestStream()
$rs.Write($content, 0, $content.Length)
# be sure to clean up after ourselves
$rs.Close()
$rs.Dispose()
Write-Host "Successfully uploaded: $source/$file"
Copy-Item "$source/$file" -Destination "$copydestination"
Remove-Item "$source/$file"
$logline = "$(Get-Date), Added File: $file to $copydestination"
Add-content "$logs" -value $logline
} catch {
$res = $ftp.GetResponse()
Add-content "$logs" -value "There was an error: $res.StatusCode"
Write-Error -Message "There was an error." -ErrorAction Stop
}

How to download a zip file with PowerShell from request stream

Currently, I make a POST request to a external website then I am supposed to get a zip file in return. I can get the zip file, but it comes in an xml with just the name.zip and nothing is downloaded. I have no idea why it is not downloading. My code is below on the piece where I make the actual request. I am not sure if I am over engineering this or what else I would have to do to get the actual file to download.
$url = "https://thewebsite.net/v6_1?id=$messageID"
Write-Output($url)
$Body = [byte[]][char[]]$xmlMessage
Write-Output($Body)
$Request = [System.Net.HttpWebRequest]::CreateHttp($url);
$Request.Method="POST"
$Request.ContentType = 'text/xml;charset=utf-8'
$Request.ContentLength = $Body.Length
$Request.ClientCertificates.Add($Certificate)
Write-Output($Request.ClientCertificates)
$Stream = $Request.GetRequestStream();
$Stream.Write($Body, 0, $Body.Length);
$Response = $Request.GetResponse()
$totalLength = [System.Math]::Floor($Response.get_ContentLength()/1024)
$responseStream = $Response.GetResponseStream()
$targetStream = New-Object -TypeName System.IO.FileStream -ArgumentList "D:\path\to\save\test.txt", Create
$buffer = new-object byte[] 1GB
$count = $responseStream.Read($buffer,0,$buffer.length)
$downloadedBytes = $count
while ($count -gt 0)
{
[System.Console]::CursorLeft = 0
[System.Console]::Write("Downloaded {0}K of {1}K", [System.Math]::Floor($downloadedBytes/1024), $totalLength)
$targetStream.Write($buffer, 0, $count)
$count = $responseStream.Read($buffer,0,$buffer.length)
$downloadedBytes = $downloadedBytes + $count
Write-Output($count)
}
$targetStream.Flush()
$targetStream.Close()
$targetStream.Dispose()
$responseStream.Dispose()
Unfortunately without certain download URI it's hard to clarify either you case is nontrivial or you just select non optimal way to get remote file. Routine way to get ".zip" (or any other 'octet/stream' file) with Power-Shell is execute the following command
Invoke-WebRequest -uri "https://thewebsite.net/v6_1?id=$messageID" -Method "GET" -Outfile (-join($messageID,".zip"))
then $messageID.zip file would be created in directory from which you execute Power-Shell
Progress would be shown in console window automatically. I test this example just before write the answer and it works independently on method "POST"/"GET" when remote host actually return "octet/stream" in the response. Maybe in you case file is not directly returned after requesting
thewebsite.net/v6_1?id=$messageID
but it is not a point of you original question.
Have you tried using Invoke-WebRequest?
$path = [Environment]::GetFolderPath("MyDocuments")
Invoke-WebRequest "example.com" -OutFile "$path\ZippedFile.zip"
A variable does not have to be used, as the path can be completely defined in the Invoke-WebRequest line if desired.

PowerShell upload file to ftp will upload file twice

UPDATE:
the
$ftprespsonse = [System.Net.FtpWebResponse]$ftp.GetResponse()
in the following code creates an empty file(file with the same name but has size 0), which leads to the duplication of my original question. My question is why the GetRepsonse creates that empty file? My guess right now is the [System.Net.FtpWebRequest]::Create and GetResponse will mess up stuff.
$username="user"
$password="pw"
$ftp = [System.Net.FtpWebRequest]::Create("ftp://xxx.xxx.xxx:{port}/file.txt")
$ftp = [System.Net.FtpWebRequest]$ftp
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential($username,$password)
$ftp.UseBinary = $true
$ftp.UsePassive = $true
$ftp.EnableSsl = $true
$ftp.KeepAlive = $false
$ftprespsonse = [System.Net.FtpWebResponse]$ftp.GetResponse()
$content = [System.IO.File]::ReadAllBytes("c:\file.txt")
$ftp.ContentLength = $content.Length
try
{
$rs = $ftp.GetRequestStream()
$rs.Write($content, 0, $content.Length)
'File Uploaded.'
Write-Host 'Status code: ' + $ftprespsonse.StatusCode
Write-Host 'Status descriptionL: ' + $ftprespsonse.StatusDescription
$ftprespsonse.close()
$ftp.Abort()
$rs.Close()
$rs.Dispose()
}
catch [System.Exception]
{
'Upload failed.'
$ftprespsonse = [System.Net.FtpWebResponse]$ftp.GetResponse()
Write-Host 'Status code: ' + $ftprespsonse.StatusCode
Write-Host 'Status descriptionL: ' + $ftprespsonse.StatusDescription
$ftprespsonse.close()
$ftp.Abort()
}
By running this script, I can see the following output:
File Uploaded.
Status code: + ClosingData
Status descriptionL: + 226- Transfer complete - acknowledgment message is pending.
226- Transfer complete - acknowledgment message is pending.
226 Transfer complete (Batch Number = 30009).
And going to the remote folder, I can see that two files are created, the file names are the same, but one of them has file size 0 and the other is 570kb (which is correct).
any idea what happened?
Use Powershell FTP Module(http://gallery.technet.microsoft.com/scriptcenter/PowerShell-FTP-Client-db6fe0cb), you will sidestep having to implement your own FTP client and likely avoid your curious issue.
It turned out the order of
$ftp = [System.Net.FtpWebRequest]::Create($fileRemotePath)
and
$ftprespsonse = [System.Net.FtpWebResponse]$ftp.GetResponse()
and
$rs = $ftp.GetRequestStream()
messes up the stuff. By running the code line by line, I find that the empty file is created at the first appearance of the $ftp.GetResponse() , after that, the full file is created on line $ftp.GetRequestStream()
therefore, I will move the first GetResponse() to after the GetRequestStream()

ftp via powershell - how to indicate success

I'm using a Powershell script to automate sending a .txt to an FTP site. When I execute it in powershell, nothing happens. The root\prompt just appears...no messages that it was successful. How do I tell if it worked? Here is my script in case it helps.
$localfile = "D:\Export\TESTING.txt"
$remotefile = "/TESTING.txt"
$ftphost = "ftp://ftp.site.com"
$URI = $ftphost + $remotefile
$username="USERNAME"
$password="1234"
function Get-FTPFile
($URI,$localfile,$username,$password){
$credentials=New-Object System.Net.NetworkCredential
($username,$password)
$ftp=[System.Net.FtpWebRequest]::Create($URI)
$ftp.Credentials=$credentials
$ftp.UseBinary=1
$ftp.KeepAlive=0
$response=$ftp.GetResponse()
$responseStream = $response.GetResponseStream()
$file = New-Object
IO.FileStream ($localfile,[IO.FileMode]::Create)
[byte[]]$buffer = New-Object byte[] 1024
$read = 0
do{
$read=$responseStream.Read($buffer,0,1024)
$file.Write($buffer,0,$read)
}
while ($read -ne 0)$file.close()
}
Use WebRequestMethods.Ftp.GetFileSize following completion of the upload to confirm that the uploaded file size matches the local file size.
You could also use a try/catch block to check for exceptions during your read/write operations. A lack of exceptions would give you some confidence that the upload was successful (i.e. no news is good news).

Creating a directory on remote FTP using powershell

I"m able to put a file up to a remote FTP with a modified version of...
$File = "D:\Dev\somefilename.zip"
$ftp = "ftp://username:password#example.com/pub/incoming/somefilename.zip"
"ftp url: $ftp"
$webclient = New-Object System.Net.WebClient
$uri = New-Object System.Uri($ftp)
"Uploading $File..."
$webclient.UploadFile($uri, $File)
I'm running into the problem that I"m trying to upload a file to a directory that doesn't exist, the put fails. So I need to create the target directory first. GET-MEMBER doesn't seem to show any methods I can invoke to create a directory, only file operations.
I use function Create-FtpDirectory
function Create-FtpDirectory {
param(
[Parameter(Mandatory=$true)]
[string]
$sourceuri,
[Parameter(Mandatory=$true)]
[string]
$username,
[Parameter(Mandatory=$true)]
[string]
$password
)
if ($sourceUri -match '\\$|\\\w+$') { throw 'sourceuri should end with a file name' }
$ftprequest = [System.Net.FtpWebRequest]::Create($sourceuri);
$ftprequest.Method = [System.Net.WebRequestMethods+Ftp]::MakeDirectory
$ftprequest.UseBinary = $true
$ftprequest.Credentials = New-Object System.Net.NetworkCredential($username,$password)
$response = $ftprequest.GetResponse();
Write-Host Upload File Complete, status $response.StatusDescription
$response.Close();
}
Taken from Ftp.psm1 where you can find also other functions for FTP.
To others: sorry for not following well known verb-noun pattern. ;)