UPDATE:
the
$ftprespsonse = [System.Net.FtpWebResponse]$ftp.GetResponse()
in the following code creates an empty file(file with the same name but has size 0), which leads to the duplication of my original question. My question is why the GetRepsonse creates that empty file? My guess right now is the [System.Net.FtpWebRequest]::Create and GetResponse will mess up stuff.
$username="user"
$password="pw"
$ftp = [System.Net.FtpWebRequest]::Create("ftp://xxx.xxx.xxx:{port}/file.txt")
$ftp = [System.Net.FtpWebRequest]$ftp
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential($username,$password)
$ftp.UseBinary = $true
$ftp.UsePassive = $true
$ftp.EnableSsl = $true
$ftp.KeepAlive = $false
$ftprespsonse = [System.Net.FtpWebResponse]$ftp.GetResponse()
$content = [System.IO.File]::ReadAllBytes("c:\file.txt")
$ftp.ContentLength = $content.Length
try
{
$rs = $ftp.GetRequestStream()
$rs.Write($content, 0, $content.Length)
'File Uploaded.'
Write-Host 'Status code: ' + $ftprespsonse.StatusCode
Write-Host 'Status descriptionL: ' + $ftprespsonse.StatusDescription
$ftprespsonse.close()
$ftp.Abort()
$rs.Close()
$rs.Dispose()
}
catch [System.Exception]
{
'Upload failed.'
$ftprespsonse = [System.Net.FtpWebResponse]$ftp.GetResponse()
Write-Host 'Status code: ' + $ftprespsonse.StatusCode
Write-Host 'Status descriptionL: ' + $ftprespsonse.StatusDescription
$ftprespsonse.close()
$ftp.Abort()
}
By running this script, I can see the following output:
File Uploaded.
Status code: + ClosingData
Status descriptionL: + 226- Transfer complete - acknowledgment message is pending.
226- Transfer complete - acknowledgment message is pending.
226 Transfer complete (Batch Number = 30009).
And going to the remote folder, I can see that two files are created, the file names are the same, but one of them has file size 0 and the other is 570kb (which is correct).
any idea what happened?
Use Powershell FTP Module(http://gallery.technet.microsoft.com/scriptcenter/PowerShell-FTP-Client-db6fe0cb), you will sidestep having to implement your own FTP client and likely avoid your curious issue.
It turned out the order of
$ftp = [System.Net.FtpWebRequest]::Create($fileRemotePath)
and
$ftprespsonse = [System.Net.FtpWebResponse]$ftp.GetResponse()
and
$rs = $ftp.GetRequestStream()
messes up the stuff. By running the code line by line, I find that the empty file is created at the first appearance of the $ftp.GetResponse() , after that, the full file is created on line $ftp.GetRequestStream()
therefore, I will move the first GetResponse() to after the GetRequestStream()
Related
i have been looking for a way to know the status of my ftp files, there are few log files are being uploading on my ftp server after every 15mints, but few times it fails to upload i just want an alert when ever a file fails to upload.
following code has been tired
function update {
$ftprequest = [System.Net.FtpWebRequest]::Create("ftp://ftpsite.com/Script_Apps/install_firefox.exe")
$ftprequest.Method = [System.Net.WebRequestMethods+Ftp]::GetDateTimestamp
$response = $ftprequest.GetResponse().StatusDescription
$tokens = $response.Split(" ")
$code = $tokens[0]
$localfile = (Get-Item "$dir\Apps\install_firefox.exe").LastWriteTimeUtc
if ($tokens -gt $localfile) {
write-host "Updating Firefox Installer..."
$File = "$dir\Apps\install_firefox.exe"
$ftp = "ftp://ftpsite.com/Script_Apps/install_firefox.exe"
$webclient = New-Object System.Net.WebClient
$uri = New-Object System.Uri($ftp)
$webclient.DownloadFile($uri, $File)
"Updated Firefox" >> $global:logfile
mainmenu
}
else {
Write-Host "Local Copy is Newer."
sleep 3
mainmenu
}
}
I'm going to assume this line deals with downloading/uploading files for your FTP site.
$ftp = "ftp://ftpsite.com/Script_Apps/install_firefox.exe"
$webclient = New-Object System.Net.WebClient
$uri = New-Object System.Uri($ftp)
$webclient.DownloadFile($uri, $File)
If so, then we can lookout for errors and do something when one occurs with try{}catch{} blocks.
You can put any code inside a try block, and when it hits a terminating error it will capture the error as $Error[0] which we can reference in a catch block.
try{
$ftp = "ftp://ftpsite.com/Script_Apps/install_firefox.exe"
$webclient = New-Object System.Net.WebClient
$uri = New-Object System.Uri($ftp)
$webclient.DownloadFile($uri, $File)
}
catch{
#Write out an error to the screen
"failed to download file $($ftp). `n--Ran into error $($Error[0].Exception.Message)"
#Todo - send a notification
}
This will give a result like this, when it fails to download.
Failed to download file ftp://ftpsite.com/Script_Apps/install_firefox.exe.
---Ran into error Exception calling "DownloadFile" with "2" argument(s):
"Unable to connect to the remote server"
Then, you can determine the best way to send a notification to suit your needs. There's way's to send e-mail from PowerShell, or Tweets, or even using a Push message to the Pushbullet app on your phone!
Currently, I make a POST request to a external website then I am supposed to get a zip file in return. I can get the zip file, but it comes in an xml with just the name.zip and nothing is downloaded. I have no idea why it is not downloading. My code is below on the piece where I make the actual request. I am not sure if I am over engineering this or what else I would have to do to get the actual file to download.
$url = "https://thewebsite.net/v6_1?id=$messageID"
Write-Output($url)
$Body = [byte[]][char[]]$xmlMessage
Write-Output($Body)
$Request = [System.Net.HttpWebRequest]::CreateHttp($url);
$Request.Method="POST"
$Request.ContentType = 'text/xml;charset=utf-8'
$Request.ContentLength = $Body.Length
$Request.ClientCertificates.Add($Certificate)
Write-Output($Request.ClientCertificates)
$Stream = $Request.GetRequestStream();
$Stream.Write($Body, 0, $Body.Length);
$Response = $Request.GetResponse()
$totalLength = [System.Math]::Floor($Response.get_ContentLength()/1024)
$responseStream = $Response.GetResponseStream()
$targetStream = New-Object -TypeName System.IO.FileStream -ArgumentList "D:\path\to\save\test.txt", Create
$buffer = new-object byte[] 1GB
$count = $responseStream.Read($buffer,0,$buffer.length)
$downloadedBytes = $count
while ($count -gt 0)
{
[System.Console]::CursorLeft = 0
[System.Console]::Write("Downloaded {0}K of {1}K", [System.Math]::Floor($downloadedBytes/1024), $totalLength)
$targetStream.Write($buffer, 0, $count)
$count = $responseStream.Read($buffer,0,$buffer.length)
$downloadedBytes = $downloadedBytes + $count
Write-Output($count)
}
$targetStream.Flush()
$targetStream.Close()
$targetStream.Dispose()
$responseStream.Dispose()
Unfortunately without certain download URI it's hard to clarify either you case is nontrivial or you just select non optimal way to get remote file. Routine way to get ".zip" (or any other 'octet/stream' file) with Power-Shell is execute the following command
Invoke-WebRequest -uri "https://thewebsite.net/v6_1?id=$messageID" -Method "GET" -Outfile (-join($messageID,".zip"))
then $messageID.zip file would be created in directory from which you execute Power-Shell
Progress would be shown in console window automatically. I test this example just before write the answer and it works independently on method "POST"/"GET" when remote host actually return "octet/stream" in the response. Maybe in you case file is not directly returned after requesting
thewebsite.net/v6_1?id=$messageID
but it is not a point of you original question.
Have you tried using Invoke-WebRequest?
$path = [Environment]::GetFolderPath("MyDocuments")
Invoke-WebRequest "example.com" -OutFile "$path\ZippedFile.zip"
A variable does not have to be used, as the path can be completely defined in the Invoke-WebRequest line if desired.
I've had a script written in Powershell which transferred a file via FTP which worked fine by using:
$content = [System.IO.File]::ReadAllBytes($backup_app_data)
But this stopped working once the file size reached 2Gb, by throwing an error saying that this method is limited to reading files up to this size.
Exception calling "ReadAllBytes" with "1" argument(s): "The file is
too long. This operation is currently limited to supporting files less
than 2 gigabytes in size.
Now, I'm trying to modify my script and use an alternate approach with Read, which from what I understand will allow me to read the file in chunks and then write those chunks and so on.
Unfortunately although the script I've modified seems to be working, since a file is created on my FTP location, there is no data being transferred and no error is thrown by the script.
There is just a 0kb file created in the destination folder and the script ends.
I've tried to do some debugging, but I can't seem to find the issue. Below is the code I'm currently using.
$file = 'G:\Backups\DATA_backup_2016_09_05.zip'
$dest_name = 'DATA_backup_2016_09_05.zip'
$ftp = [System.Net.FtpWebRequest]::Create($ftp+$dest_name)
$ftp = [System.Net.FtpWebRequest]$ftp
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential($user, $pass)
$ftp.UseBinary = $true
$ftp.UsePassive = $true
# determine the size of the file
$file_size = (Get-Item $file).length
$chunk_size = 512mb
$bytes_to_read = $file_size
$iterations = $file_size / $chunk_size
$iter = 0
$fstream = [System.IO.FileStream]
[byte[]] $byte_array
while ($bytes_to_read > 0){
if($iterations > 1) {
$content = $fstream.Read($byte_array, $iter, $chunk_size)
}
else {
$content = $fstream.Read($byte_array, $iter, $bytes_to_read)
}
$ftp.ContentLength = $content.Length
$rs = $ftp.GetRequestStream()
$rs.Write($content, 0, $content.Length)
# keep the loop going
$iter = $iter + 1
$iterations = $iterations - 1
$bytes_to_read = $bytes_to_read - $chunk_size
}
$rs.Close()
$rs.Dispose()
Any help is appreciated.
So, based on the suggestions in the comment section and using as an example the answer from the possible duplicate question, I managed to find the solution myself.
Below is the code I used to transfer via FTP a .zip archive which was 3.74Gb.
$ftp_addr = "ftp://ftp.somerandomwebsite.com/folder1/"
$user = "usr"
$pass = "passwrd"
$bufSize = 256mb
# ... missing code where I identify the file I want to FTP ... #
# Initialize connection to FTP
$ftp = [System.Net.FtpWebRequest]::Create($ftp_addr + $backup_file + ".zip")
$ftp = [System.Net.FtpWebRequest]$ftp
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential($user, $pass)
$ftp.Timeout = -1 #infinite timeout
$ftp.ReadWriteTimeout = -1 #infinite timeout
$ftp.UseBinary = $true
$ftp.UsePassive = $true
$requestStream = $ftp.GetRequestStream()
$fileStream = [System.IO.File]::OpenRead($file_to_ftp)
$chunk = New-Object byte[] $bufSize
while ( $bytesRead = $fileStream.Read($chunk, 0, $bufSize) ){
$requestStream.write($chunk, 0, $bytesRead)
$requestStream.Flush()
}
$fileStream.Close()
$requestStream.Close()
So far, this code has worked flawlessly for about multiple (20+) attempts. I hope it helps others!
I am trying to download a file from reporting services. The first time I run this code it works fine, but the next time it wont overwrite the first ones downloaded file. I have tried adding Remove-Item $file before I create the WebClient object, but when I do this I get the error The process cannot access the file 'D:\Work\RawMaterialCodes.xls' because it is being used by another process., the process being Powershell itself. I have tried calling $webClient.Dispose() thinking this might release the file, but no luck.
Does anyone have any ideas how I can overwrite the downloaded file and/or remove it before the next download attempt?
$reportServer = "http://localhost/ReportServer_TRITON"
$reportName = "RawMaterialCodes"
$file = "D:\Work\RawMaterialCodes.xls"
$startDate = "2014-01-22"
$endDate = "2014-01-24"
$category = "Cat1"
$destination = ""
$reportUrl = $reportServer + "?/" + $reportName + "&StartDate=" + $startDate + "&EndDate=" + $endDate + "&Category=" + $category + "&Destination=" + $destination + "&rs:Format=Excel"
$webClient = new-object System.Net.WebClient
$webClient.Credentials = New-Object System.Net.NetworkCredential($userName, $password, $domain)
$webClient.DownloadFile($reportUrl, $file)
$mailMessage = new-object System.Net.Mail.MailMessage
$mailMessage.From = $emailFrom
$mailMessage.To.Add($emailTo)
$mailMessage.Subject = $emailSubject
$mailMessage.Body = $emailBody
$attachment = new-object System.Net.Mail.Attachment($file, 'text/plain')
$mailMessage.Attachments.Add($attachment)
$smtpClient = New-Object System.Net.Mail.SmtpClient($smtpServer, 25)
$smtpClient.EnableSsl = $enableSsl
if ($smtpAuthUsername -ne "")
{
$smtpClient.Credentials = New-Object System.Net.NetworkCredential($smtpAuthUsername, $smtpAuthPassword)
}
$smtpClient.Send($mailMessage)
$mailMessage is keeping an open handle to your file. Add
$mailMessage.Dispose()
at the end of the script and you should be able to overwrite.
I don't see anything wrong with the code as you have it written. Disposing the WebClient is unnecessary here with respect to freeing up the file as it doesn't keep a handle to it. The most likely cause of this error is another piece of your code or another program which is opening the file and not properly disposing of it.
I would first assume it was my code at fault and carefully audit any other place I manipulated this file and see if I accidentally left a handle to it open
I'm using a Powershell script to automate sending a .txt to an FTP site. When I execute it in powershell, nothing happens. The root\prompt just appears...no messages that it was successful. How do I tell if it worked? Here is my script in case it helps.
$localfile = "D:\Export\TESTING.txt"
$remotefile = "/TESTING.txt"
$ftphost = "ftp://ftp.site.com"
$URI = $ftphost + $remotefile
$username="USERNAME"
$password="1234"
function Get-FTPFile
($URI,$localfile,$username,$password){
$credentials=New-Object System.Net.NetworkCredential
($username,$password)
$ftp=[System.Net.FtpWebRequest]::Create($URI)
$ftp.Credentials=$credentials
$ftp.UseBinary=1
$ftp.KeepAlive=0
$response=$ftp.GetResponse()
$responseStream = $response.GetResponseStream()
$file = New-Object
IO.FileStream ($localfile,[IO.FileMode]::Create)
[byte[]]$buffer = New-Object byte[] 1024
$read = 0
do{
$read=$responseStream.Read($buffer,0,1024)
$file.Write($buffer,0,$read)
}
while ($read -ne 0)$file.close()
}
Use WebRequestMethods.Ftp.GetFileSize following completion of the upload to confirm that the uploaded file size matches the local file size.
You could also use a try/catch block to check for exceptions during your read/write operations. A lack of exceptions would give you some confidence that the upload was successful (i.e. no news is good news).