An exception occurred during a WebClient request (Powershell) - powershell

Question

I'm trying to copy a directory folder from our HTTP server using Powershell, I would like to copy it's entire contents including subfolders into the local drive of my current server. The point of this is for server deployment automation so that my boss can run my powershell script and have an entire server setup with all our folders copied to its C: drive. This is the code I have
$source = "http://servername/serverupdates/deploy/Program%20Files/"
$destination = "C:\Program Files"
$client = new-object System.Net.WebClient
$client.DownloadFile($source, $destination)
When I run the script in Powershell ISE as admin, I get the error message
"Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
Any suggestions on what could be going on?
I have also tried this block of code, but nothing happens when I run it, no errors or anything.
$source = "http://serverName/serverupdates/deploy/Program%20Files/"
$webclient = New-Object system.net.webclient
$destination = "c:/users/administrator/desktop/test/"
Function Copy-Folder([string]$source, [string]$destination, [bool]$recursive) {
if (!$(Test-Path($destination))) {
New-Item $destination -type directory -Force
}
# Get the file list from the web page
$webString = $webClient.DownloadString($source)
$lines = [Regex]::Split($webString, "<br>")
# Parse each line, looking for files and folders
foreach ($line in $lines) {
if ($line.ToUpper().Contains("HREF")) {
# File or Folder
if (!$line.ToUpper().Contains("[TO PARENT DIRECTORY]")) {
# Not Parent Folder entry
$items =[Regex]::Split($line, """")
$items = [Regex]::Split($items[2], "(>|<)")
$item = $items[2]
if ($line.ToLower().Contains("<dir&gt")) {
# Folder
if ($recursive) {
# Subfolder copy required
Copy-Folder "$source$item/" "$destination$item/" $recursive
} else {
# Subfolder copy not required
}
} else {
# File
$webClient.DownloadFile("$source$item", "$destination$item")
}
}
}
}
}

Answer 1

System.Net.WebClient.DownloadFile expects the second parameter to be a filename, not a directory. It can't download a directory recursively, it can only download a single file.
For the second part, run it line by line and see what happens. But parsing HTML to get paths is prone to error and is generally advised against.
My advice: Don't use http for this. Copy the stuff from a file share, it's only one line and saves you a lot of trouble. If you have to use http, download an archive and extract it in the target directory.

Answer 2

Beside #Gerald Schneider's answer, beware that the same WebException might also occur if the client process does not have needed permission to create output file.
I would suggest you to take the following strategy:
Download file to a unique filename with .tmp (.txt) extensions Windows Temporary Folder to avoid write-permission and other permissions issues
Move temporary file to destination folder
Rename temporary file to destination filename
Hope it helps :-)

Answer 3

In addition to the other answers, the error might occur if you've ran out of disk space.