FTP Download Multiple Files using PowerShell - powershell

Question

I’m new to PowerShell and am trying to convert a batch file that downloads multiple files based on names and extension from a directory on an ftp site. While I’ve found several examples that download a file, I’m struggling to find one that shows how to download multiple files. In a batch I can quite simply use the ftp.exe and the mget command with wildcards??
Can someone please point me in the right direction.
Thanks in advance.
John

Answer 1

There are multiple ways to achieve this. One is to use the System.Net.FtpWebRequest as shown in this example:
http://www.systemcentercentral.com/BlogDetails/tabid/143/IndexID/81125/Default.aspx
Or there are /n Software NetCmdlets you can use:
http://www.nsoftware.com/powershell/tutorials/FTP.aspx

Answer 2

In a batch I can quite simply use the ftp.exe and the mget command
with wildcards??
You can do the same in Powershell if you want to.
For a more Powershell way, you can use the FTPWebRequest. See here: http://msdn.microsoft.com/en-us/library/ms229711.aspx. You can build on the example to download multiple files in a loop.
But bottomline is, you do not have to convert something you have in batch to Powershell. You can, if you want, but what you have in batch, especially when calling external programs, should work just as well.

Answer 3

Another resource you might want to check: PowerShell FTP Client Module
http://gallery.technet.microsoft.com/scriptcenter/PowerShell-FTP-Client-db6fe0cb

Answer 4

Oddly enough there are no built in cmdlets to deal with FTP. I'm not sure why the PowerShell team made that decision but it means you'll have to rely on using .NET code, a third party script/module/snap-in or a Win32 program such as FTP.exe as others have already answered with.
Here's is an example of downloading multiple files (binary and text) using .NET code:
$files = "Firefox Setup 9.0.exe", "Firefox Setup 9.0.exe.asc"
$ftpFolder = 'ftp://ftp.mozilla.org/pub/firefox/releases/9.0/win32/en-US'
$outputFolder = (Resolve-Path "~\My Documents").Path
foreach ($file in $files) {
try {
$uri = $ftpFolder + '/' + $file
$request = [Net.WebRequest]::Create($uri)
$request.Method = [Net.WebRequestMethods+Ftp]::DownloadFile
$responseStream = $request.GetResponse().GetResponseStream()
$outFile = Join-Path $outputFolder -ChildPath $file
$fs = New-Object System.IO.FileStream $outFile, "Create"
[byte[]] $buffer = New-Object byte[] 4096
do {
$count = $responseStream.Read($buffer, 0, $buffer.Length)
$fs.Write($buffer, 0, $count)
} while ($count -gt 0)
} catch {
throw "Failed to download file '{0}/{1}'. The error was {2}." -f $ftpFolder, $file, $_
} finally {
if ($fs) { $fs.Flush(); $fs.Close() }
if ($responseStream) { $responseStream.Close() }
}
}

Answer 5

#Jacob. You need ::ListDirectory method to make a list. After, you have to output it in a text file with the out-file command. After that, you import the list with the get-content command. So with a text file, you can make a collection of objects with a foreach loop (don't forget to skip the last line with the '-cne' condition).
You include in this loop your download-ftp function with the parameter of your loop.
Understood ? Not sure if my explanation is good.
So there's an example from one of my script :
$files = Get-FtpList $ftpSource $ftpDirectory $ftpLogin $ftpPassword | Out-File -Encoding UTF8 -FilePath list.txt
$list = Get-Content -Encoding UTF8 -Path list.txt
foreach ($entry in $list -cne "")
{
Get-FtpFile $ftpSource $ftpDirectory $entry $target $ftpLogin $ftpPassword
Start-Sleep -Milliseconds 10
}
Hope it works now for you.
PS:Get-FtpList and Get-FtpFile are custom functions.

Answer 6

This is what i did.As i needed to download a file based on a pattern i dynamically created a command file and then let ftp do the rest
I used basic powershell commands. i did not need to download any additional components
I first Check if the Requisite number of files exist. if they do i invoke the FTP the second time with an Mget.
I run this from a windows 2008 Server connecting to a windows XP remote server
function make_ftp_command_file($p_file_pattern,$mget_flag)
{
# This function dynamically prepares the FTP file
# The file needs to be prepared daily because the pattern changes daily
# Powershell default encoding is Unicode
# Unicode command files are not compatible with FTP so we need to make sure we create an ASCII File
write-output "USER" | out-file -filepath C:\fc.txt -encoding ASCII
write-output "ftpusername" | out-file -filepath C:\fc.txt -encoding ASCII -Append
write-output "password" | out-file -filepath C:\fc.txt -encoding ASCII -Append
write-output "ASCII" | out-file -filepath C:\fc.txt -encoding ASCII -Append
If($mget_flag -eq "Y")
{
write-output "prompt" | out-file -filepath C:\fc.txt -encoding ASCII -Append
write-output "mget $p_file_pattern" | out-file -filepath C:\fc.txt -encoding ASCII -Append
}
else
{
write-output "ls $p_file_pattern" | out-file -filepath C:\fc.txt -encoding ASCII -Append
}
write-output quit | out-file -filepath C:\fc.txt -encoding ASCII -Append
}
########################### Init Section ###############################
$yesterday = (get-date).AddDays(-1)
$yesterday_fmt = date $yesterday -format "yyyyMMdd"
$file_pattern = "BRAE_GE_*" + $yesterday_fmt + "*.csv"
$file_log = $yesterday_fmt + ".log"
echo $file_pattern
echo $file_log
############################## Main Section ############################
# Change location to folder where the files need to be downloaded
cd c:\remotefiles
# Dynamically create the FTP Command to get a list of files from the Remote Servers
echo "Call function that creates a FTP Command "
make_ftp_command_file $file_pattern N
#echo "Connect to remote site via FTP"
# Connect to Remote Server and get file listing
ftp -n -v -s:C:\Clover\scripts\fc.txt 10.129.120.31 > C:\logs\$file_log
$matches=select-string -pattern "BRAE_GE_[A-Z][A-Z]*" C:\logs\$file_log
# Check if the required number of Files available for download
if ($matches.count -eq 36)
{
# Create the ftp command file
# this time the command file has an mget rather than an ls
make_ftp_command_file $file_pattern Y
# Change directory if not done so
cd c:\remotefiles
# Invoke Ftp with newly created command file
ftp -n -v -s:C:\Clover\scripts\fc.txt 10.129.120.31 > C:\logs\$file_log
}
else
{
echo "Full set of Files not available"
}

Answer 7

It's not Powershell specific. But I've tried many other solutions and so far
The http://ncftp.com/ client works the best. It comes with ncftpls.exe for listing remote files and ncftpget.exe for getting files. Use them with Start-Process -Wait

Answer 8

A file list can be constructed in a variable, and used with a regular FTP command....
$FileList="file1_$cycledate.csv
file2_$cycledate.csv
file3_$cycledate.csv
file4_$cycledate.csv"
"open $FTPServer
user $FTPUser $FTPPassword
ascii
cd report
" +
($filelist.split(' ') | %{ "mget $_" }) | ftp -i -n