Problem downloading file using Powershell - powershell

I have this simple script that works on a laptop:
[Net.ServicePointManager]::SecurityProtocol
[enum]::GetNames([Net.SecurityProtocolType])
$url = "https://www.contextures.com/SampleData.zip"
wget -Uri $url -OutFile "C:\temp\temp.zip"
But when I'm trying to run it on a server I'm always getting this error:
wget : The request was aborted: Could not create SSL/TLS secure
channel.
Any ideas what might be causing this? Any help would be appreciated.
I already tried this and still getting the same error message:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

Set it to TLS 1.2:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

You can give a try with this example while adding
$AllProtocols = [System.Net.SecurityProtocolType]'Ssl3,Tls,Tls11,Tls12';
cls
$Folder = "$Env:Temp\DownloadFolder\"
# We create a SubFolder Named "DownloadFolder" in the temporary file %Temp% if it doesn't exists yet !
If ((Test-Path -Path $Folder) -eq 0) { New-Item -Path $Folder -ItemType Directory | Out-Null }
$urls=#('https://www.contextures.com/SampleData.zip',
'https://cdn2.unrealengine.com/Fortnite%2FBoogieDown_GIF-1f2be97208316867da7d3cf5217c2486da3c2fe6.gif'
)
$start_time = Get-Date
$AllProtocols = [System.Net.SecurityProtocolType]'Ssl3,Tls,Tls11,Tls12';
[System.Net.ServicePointManager]::SecurityProtocol = $AllProtocols;
$i = 0
foreach($url in $urls)
{
$i = $i + 1
$output = $Folder + $url.Split("/")[-1]
Try {
Write-Host "$i - Downloading" $output.Split("\")[-1] from $url
(New-Object System.Net.WebClient).DownloadFile($url,$output)
}
Catch
{
Write-Host "Error in : " $url -ForegroundColor Red -BackgroundColor Yellow
Write-Host "Message: [$($_.Exception.Message)"] -ForegroundColor Red -BackgroundColor Yellow
$i = $i - 1
}
}
Write-Output "Time taken: $((Get-Date).Subtract($start_time).Seconds) second(s) to download $i files"
ii $Folder

Related

Need To Install Alien Vault on Client, but Client Window Machines Run PowerShell 2.0. Need to convert to syntax that will work on that machine

Client doesn't want to upgrade, because they don't want anything to
break.
The code that I need to work on their machine is below. Right now
this script will not work because it's made to work for PowerShell 3.0 and above
[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls12; (new-object Net.WebClient).DownloadString("https://api.agent.alienvault.cloud/osquery-api/us-east-1/bootstrap?flavor=powershell") | iex; install_agent -controlnodeid <Agent ID>
How can this be rewritten to accomplish the same thing?
Now if you know of a simpler way to work around this issue, that will help greatly.
new-module -name install_agent -scriptblock {
function AgentDoStart() {
$kServiceName = "osqueryd"
$osquerydService = Get-WmiObject -Class Win32_Service -Filter "Name='$kServiceName'"
if ($osquerydService) {
Start-Service $kServiceName
Write-Host "'$kServiceName' system service is started." -foregroundcolor Cyan
return 1
} else {
Write-Host "'$kServiceName' is not an installed system service." -foregroundcolor Yellow
return 0
}
}
function AgentDoStop() {
$kServiceName = "osqueryd"
$osquerydService = Get-WmiObject -Class Win32_Service -Filter "Name='$kServiceName'"
if ($osquerydService) {
Stop-Service $kServiceName
Start-Sleep -s 1
$proc = Get-Process osqueryd -ErrorAction SilentlyContinue
if ($proc) {
Write-Host "osqueryd still running, killing processes"
Stop-Process -Force -Name osqueryd
}
Write-Host "'$kServiceName' system service is stopped." -foregroundcolor Cyan
return 1
} else {
Write-Host "'$kServiceName' is not an installed system service." -foregroundcolor Yellow
return 0
}
}
Function Install-Project() {
param(
[string]$apikey="",
[string]$controlnodeid="",
[string]$hostid="",
[string]$assetid=""
)
Install-Project-Internal -apikey $apikey -controlnodeid $controlnodeid -hostid $hostid -assetid $assetid
Write-Host "See install.log for details" -ForegroundColor Cyan
}
Function Download-And-Install-Sysmon() {
#===================================================
#1. Download Sysmon
#===================================================
$source = "https://download.sysinternals.com/files/Sysmon.zip"
Write-Host "Downloading Sysmon from $source" -ForegroundColor Cyan
$file = "$($env:TEMP)\Sysmon.zip"
Invoke-WebRequest $source -OutFile $file
#===================================================
#2. Clean & Prepare Sysmon installation target
#===================================================
$targetondisk = "$($env:USERPROFILE)\Documents\Sysmon\"
Write-Host "Preparing Sysmon target path $($targetondisk)" -ForegroundColor Cyan
Remove-Item $targetondisk -Recurse -ErrorAction Ignore
# Suppress output, but not errors:
[void](New-Item -ItemType Directory -Force -Path $targetondisk)
If (-Not (Test-Path -Path $targetondisk)) {
Write-Error "Skipping Sysmon... Destination path $($targetondisk) does not exist."
} Else {
#===================================================
#3. Unzip Sysmon
#===================================================
Unblock-File -Path $file
Write-Host "Uncompressing the Zip file to $($targetondisk)" -ForegroundColor Cyan
$FoundExtractionAssembly = 0
try {
# Load preferred extraction method's assembly (.NET 4.5 or later)
# Write-Host "Using preferred extraction method..."
Add-Type -As System.IO.Compression.FileSystem -ErrorAction Stop
$FoundExtractionAssembly = 1
}
catch [System.Exception] {
# Write-Host "Preferred extraction method not found. Attempting fall-back method..."
}
If ($FoundExtractionAssembly) {
[IO.Compression.ZipFile]::ExtractToDirectory($file, $targetondisk)
} Else {
# Fall-back method, may fail in sessions lacking access to interactive shell
$continue_flag = 1
try {
$shell_app = New-Object -COMObject "Shell.Application"
} catch {
Write-Error "Could not create Shell.Application object"
$continue_flag = 0
}
if ($continue_flag) {
$zip_file = $shell_app.namespace($file)
$destination = $shell_app.namespace($targetondisk)
if ($destination -ne $null) {
$destination.Copyhere($zip_file.items(), 0x10)
}
}
}
}
#===================================================
#3. Download Sysmon Config File
#===================================================
$source = "https://www.alienvault.com/documentation/resources/downloads/sysmon_config_schema4_0.xml"
Write-Host "Downloading Sysmon config file from $source" -ForegroundColor Cyan
$destination = [System.IO.Path]::GetTempFileName()
Invoke-WebRequest $source -OutFile $destination
#===================================================
#3. Install Sysmon
#===================================================
Write-Host "Installing Sysmon from $source" -ForegroundColor Cyan
If ( (get-childitem $destination).length -eq 0 ) {
$command = "& '$targetondisk\sysmon' -accepteula -h md5 -n -l -i"
Write-Host "Not using an additional Sysmon configuration file" -ForegroundColor Cyan
}
Else {
$command = "& '$targetondisk\sysmon' -accepteula -h md5 -n -l -i '$destination'"
Write-Host "Sysmon configuration file to use $destination" -ForegroundColor Cyan
}
Write-Host "Installing Sysmon with command $command" -ForegroundColor Cyan
iex $command
}
Function Install-Project-Internal() {
param(
[string]$apikey="",
[string]$controlnodeid="",
[string]$hostid="",
[string]$assetid=""
)
If (-NOT ([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator")) {
Write-Error "You do not have Administrator rights to run this script!`nPlease re-run this script as an Administrator!"
Return
}
If ($PSVersionTable.PSVersion.Major -lt 3) {
Write-Error "This script must be run using Powershell version 3 or higher. You have version $PSVersionTable.PSVersion.Major installed"
Return
}
$kServiceName = "osqueryd"
$BASE = "$($env:SYSTEMDRIVE)\Program Files\osquery"
$OLDBASE = "$($env:SYSTEMDRIVE)\ProgramData\osquery"
$secretfile = $(Join-Path $BASE "secret")
$flagfile = $(Join-Path $BASE "osquery.flags")
if ([string]::IsNullOrEmpty($hostid)) {
$hostid = $assetid
}
if ([string]::IsNullOrEmpty($apikey)) {
$apikey = $controlnodeid
}
if ([string]::IsNullOrEmpty($apikey)) {
if ([System.IO.File]::Exists("$secretfile")) {
$apikey = [IO.File]::ReadAllText("$secretfile").Trim()
}
}
if ([string]::IsNullOrEmpty($apikey)) {
# check old location in ProgramData
$oldsecretfile = $(Join-Path $OLDBASE "secret")
if ([System.IO.File]::Exists("$oldsecretfile")) {
$apikey = [IO.File]::ReadAllText("$oldsecretfile").Trim()
}
}
if ([string]::IsNullOrEmpty($apikey)) {
Write-Warning "You must supply either the -apikey or -controlnodeid parameters to identify your agent account"
return
}
# use TLS 1.2
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Download-And-Install-Sysmon
#===================================================
#4. Download and install osquery
#===================================================
try {
AgentDoStop
} catch {
Write-Error "Did not stop osqueryd service. Hopefully, this is fine."
}
Write-Host "Downloading installer"
$webclient = New-Object System.Net.WebClient
$webclient.DownloadFile("https://prod-usm-saas-agent-config.s3.amazonaws.com/repo/windows/alienvault-agent-20.01.0203.0301.msi", "$env:TEMP\alienvault-agent.msi")
Write-Host "Installing"
try {
Start-Process C:\Windows\System32\msiexec.exe -ArgumentList "/i $env:TEMP\alienvault-agent.msi ALLUSERS=1 /qn /l*v .\install.log" -wait
echo "INSTALLATION SUCCESSFULLY COMPLETED" >> .\install.log
} catch {
echo "INSTALLATION ERROR (ERRORLEVEL=%ERRORLEVEL%)" >> .\install.log
Write-Error "INSTALLATION ERROR (ERRORLEVEL=%ERRORLEVEL%)"
Return
}
# If the install directory doesn't exist, bail
if (![System.IO.Directory]::Exists("$BASE")) {
echo "Installation directory does not exist: $BASE" >> .\install.log
Write-Error "Installation directory does not exist: $BASE"
Return
}
# $osquerydService = Get-WmiObject -Class Win32_Service -Filter "Name='osqueryd'"
# if ($osquerydService) {
# Write-Host "Service exists, uninstalling"
# try {
# Stop-Service $kServiceName
# AgentDoStop
#
# Write-Host "Found '$kServiceName', stopping the system service..."
# Start-Sleep -s 5
# Write-Host "System service should be stopped."
# $osquerydService.Delete()
# Write-Host "System service '$kServiceName' uninstalled." -foregroundcolor Cyan
# } catch {
# Write-Error "Did not uninstall osqueryd service. Hopefully, it's not already installed."
# }
# }
Write-Host "Writing secret"
[IO.File]::WriteAllLines("$secretfile", $apikey)
# if hostid is not specified, try to extract from flag file
if ([string]::IsNullOrEmpty($hostid)) {
if ([System.IO.File]::Exists($flagfile)) {
$match = (Select-String -Path $flagfile -Pattern "specified_identifier=(.*)")
if ($match.Matches.Groups.success) {
$hostid = $match.Matches.Groups[1].Value.Trim()
Write-Host "Detected and re-using previously selected host id from ${flagfile}: $hostid"
} else {
Write-Host "Existing host id not found in ${flagfile}"
}
}
}
# if still not found, check old ProgramData location
if ([string]::IsNullOrEmpty($hostid)) {
$oldflagfile = $(Join-Path $OLDBASE "osquery.flags")
if ([System.IO.File]::Exists($oldflagfile)) {
$match = (Select-String -Path $oldflagfile -Pattern "specified_identifier=(.*)")
if ($match.Matches.Groups.success) {
$hostid = $match.Matches.Groups[1].Value.Trim()
Write-Host "Detected and re-using previously selected host id from ${oldflagfile}: $hostid"
} else {
Write-Host "Existing host id not found in ${oldflagfile}"
}
}
}
echo "Creating flag file"
copy $BASE\osquery.flags.example $flagfile
Write-Host "Setting host identifier"
# if still no hostid, use generated default
if ([string]::IsNullOrEmpty($hostid)) {
$hostid="00000000-8019-46ae-b324-685a63cb327a"
}
$output = "--tls_hostname=api.agent.alienvault.cloud/osquery-api/us-east-1", "--host_identifier=specified", "--specified_identifier=$hostid"
[IO.File]::AppendAllLines([string]$flagfile, [string[]]$output)
# add customer certs if present
$custpem = "$($env:SYSTEMROOT)\System32\drivers\etc\osquery_customer_certs.pem"
if ([System.IO.File]::Exists($custpem)) {
Write-Host "Adding customer certs"
type "$custpem" >> "$BASE\certs\certs.pem"
}
# start service
if (-NOT (AgentDoStop)) {
return
}
AgentDoStart
Write-Host "Deleting installer"
del $env:TEMP\alienvault-agent.msi
if (($BASE -ne $OLDBASE) -And [System.IO.Directory]::Exists($OLDBASE)) {
Write-Host "renaming old ProgramData/osquery directory"
move "$OLDBASE" "$($OLDBASE).renamed"
}
}
set-alias install_agent -value Install-Project
export-modulemember -alias 'install_agent' -function 'Install-Project'
}
If you remove this from "Function Install-Project-Internal"
If ($PSVersionTable.PSVersion.Major -lt 3) {
Write-Error "This script must be run using Powershell version 3 or higher. You have version $PSVersionTable.PSVersion.Major installed"
Return
}
Does it work? And if not, what is the error?

Extract http link from html element

Symantec recently changed their download page which moved to broadcom. Since then Invoke-WebRequest cannot grab the http url for the v5i64.exe file.
However the http url can be found when using Developer Tools in the browser looking at the Elements level, inside the body section of the page.
Does anyone have an idea on how this daily-changed url can be extracted with PowerShell?
$webreq = Invoke-WebRequest "https://www.broadcom.com/support/security-center/definitions/download/detail?gid=sep"
$webreq.Links | Select href
Use IE via ComObject
$ie = new-object -ComObject "InternetExplorer.Application"
$ie.visible=$True
while($ie.Busy) { Start-Sleep -Milliseconds 100 }
$IE.navigate2("https://www.broadcom.com/support/security-center/definitions/download/detail?gid=sep")
while ($IE.busy) {
start-sleep -milliseconds 1000 #wait 1 second interval to load page
}
Then find elementy by $ie.Document.IHTMLDocument3_getElementsByTagName("element name")
The following PowerShell script will prompt you to download the links containing the text v5i64.exe and HTTPS. This works on PowerShell 5.1 for Windows. It does not work for PowerShell 6 or 7 (PowerShell Core).
Tested on Windows 10.0.18363.657, Internet Explorer 11.657.18362, PowerShell 5.1.18362.628
$url = "https://www.broadcom.com/support/security-center/definitions/download/detail?gid=sep"
$outfile = "./v5i64.exe"
$ie = New-Object -ComObject "InternetExplorer.Application"
$ie.visible=$True
while($ie.Busy) {
Start-Sleep -Milliseconds 100
}
$ie.navigate2($url)
while($ie.ReadyState -ne 4 -or $ie.Busy) {
Start-Sleep -milliseconds 500
}
$ie.Document.getElementsByTagName("a") | % {
if ($_.ie8_href -like "*v5i64.exe") {
if ($_.ie8_href -like "https://*") {
$len = (Invoke-WebRequest $_.ie8_href -Method Head).Headers.'Content-Length'
Write-Host "File:" $_.ie8_href
Write-Host "Size:" $len
$confirm = Read-Host "Download file? [y/n]"
if ($confirm -eq "y") {
Write-Host "Downloading" $_.ie8_href
Invoke-WebRequest -Uri $_.ie8_href -OutFile $outfile
}
}
}
}
$ie.Stop()
$ie.Quit()
thanks for the proposed solutions. However here is my final code I am using:
$SEP_last_link = ("http://definitions.symantec.com/defs/"+($SEP_last | Select-String release -NotMatch | select -Last 1))
$Symantec_folder = "C:\Download for DVD\Symantec"
$Symantec_filepath = "$Symantec_folder\$SEP_last"
if (!(Test-Path "$Symantec_filepath" -PathType Leaf)) {
Write-Host "`rStart to download Symantec $SEP_last file: $(Get-Date)`r"
$start_time = (Get-Date)
$webclient = New-Object System.Net.WebClient
$WebClient.DownloadFile($SEP_last_link, $Symantec_filepath)
Write-Host "`r$SEP_last file has been downloaded successfully`r" -ForegroundColor Green
$end_time = $(get-date) - $start_time
$total_time = "{0:HH:mm:ss}" -f ([datetime]$end_time.Ticks)
Write-Host "`rTime to download Symantec $SEP_last file: $total_time`r"
} else {
Write-Host "`rSymantec $SEP_last file already exists!`r" -ForegroundColor Yellow
}
Get-ChildItem -Path "$Symantec_Folder\*-v5i64.exe" -Exclude "$SEP_last" -Verbose –Force | Remove-Item

Download files from set of folders on FTP/SFTP server with WinSCP .NET assembly in PowerShell and email results

I have this PowerShell script that I'm working on. CSV file is imported to get source and destination paths. The goal is to move files from a SFTP/FTP server into a destination and send an email report.
Task scheduler will run this code every hour. And if there's a new file, as email will be sent out.
It's almost done, but two things are missing:
Check if the file already exists and Body email seems empty: Getting the following error: Cannot validate argument on parameter 'Body'. The argument is null or empty. Provide an argument that is not
null or empty, and then try the command again.
I would like some assistance on how to check if the file exists and how to get this email if a new file was dropped and copied to the destination list
$SMTPBody = ""
$SMTPMessage = #{
"SMTPServer" = ""
"From" = ""
"To" = ""
"Subject" = "New File"
}
try {
# Load WinSCP .NET assembly
Add-Type -Path "C:\Program Files (x86)\WinSCP\WinSCPnet.dll"
# Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::sftp
HostName = ""
UserName = ""
Password = ""
PortNumber = "22"
FTPMode = ""
GiveUpSecurityAndAcceptAnySshHostKey = $true
}
$session = New-Object WinSCP.Session
try
{
# Connect
$session.Open($sessionOptions)
# Download files
$transferOptions = New-Object WinSCP.TransferOptions
$transferOptions.TransferMode = [WinSCP.TransferMode]::Binary
Import-Csv -Path "D:\FILESOURCE.csv" -ErrorAction Stop | foreach {
$synchronizationResult = $session.SynchronizeDirectories(
[WinSCP.SynchronizationMode]::Local, $_.Destination, $_.Source, $False)
$synchronizationResult.Check()
foreach ($download in $synchronizationResult.Downloads ) {
Write-Host "File $($download.FileName) downloaded" -ForegroundColor Green
$SMTPBody +=
"`n Files: $($download.FileName -join ', ') `n" +
"Current Location: $($_.Destination)`n"
Send-MailMessage #SMTPMessage -Body $SMTPBody
}
$transferResult =
$session.GetFiles($_.Source, $_.Destination, $False, $transferOptions)
#Find the latest downloaded file
$latestTransfer =
$transferResult.Transfers |
Sort-Object -Property #{ Expression = { (Get-Item $_.Destination).LastWriteTime }
} -Descending |Select-Object -First 1
}
if ($latestTransfer -eq $Null) {
Write-Host "No files found."
$SMTPBody += "There are no new files at the moment"
}
else
{
$lastTimestamp = (Get-Item $latestTransfer.Destination).LastWriteTime
Write-Host (
"Downloaded $($transferResult.Transfers.Count) files, " +
"latest being $($latestTransfer.FileName) with timestamp $lastTimestamp.")
$SMTPBody += "file : $($latestTransfer)"
}
Write-Host "Waiting..."
Start-Sleep -Seconds 5
}
finally
{
Send-MailMessage #SMTPMessage -Body $SMTPBody
# Disconnect, clean up
$session.Dispose()
}
}
catch
{
Write-Host "Error: $($_.Exception.Message)"
}
I believe your code has more problems than you think.
Your combination of SynchronizeDirectories and GetFiles is suspicious. You first download only the new files by SynchronizeDirectories and then you download all files by GetFiles. I do not think you want that.
On any error the .Check call will throw and you will not collect the error into your report.
You keep sending partial reports by Send-MailMessage in the foreach loop
This is my take on your problem, hoping I've understood correctly what you want to implement:
$SMTPBody = ""
Import-Csv -Path "FILESOURCE.csv" -ErrorAction Stop | foreach {
Write-Host "$($_.Source) => $($_.Destination)"
$SMTPBody += "$($_.Source) => $($_.Destination)`n"
$synchronizationResult =
$session.SynchronizeDirectories(
[WinSCP.SynchronizationMode]::Local, $_.Destination, $_.Source, $False)
$downloaded = #()
$failed = #()
$latestName = $Null
$latest = $Null
foreach ($download in $synchronizationResult.Downloads)
{
if ($download.Error -eq $Null)
{
Write-Host "File $($download.FileName) downloaded" -ForegroundColor Green
$downloaded += $download.FileName
$ts = (Get-Item $download.Destination).LastWriteTime
if ($ts -gt $latest)
{
$latestName = $download.FileName;
$latest = $ts
}
}
else
{
Write-Host "File $($download.FileName) download failed" -ForegroundColor Red
$failed += $download.FileName
}
}
if ($downloaded.Count -eq 0)
{
$SMTPBody += "No new files were downloaded`n"
}
else
{
$SMTPBody +=
"Downloaded $($downloaded.Count) files:`n" +
($downloaded -join ", ") + "`n" +
"latest being $($latestName) with timestamp $latest.`n"
}
if ($failed.Count -gt 0)
{
$SMTPBody +=
"Failed to download $($failed.Count) files:`n" +
($failed -join ", ") + "`n"
}
$SMTPBody += "`n"
}
It will give you a report like:
/source1 => C:\dest1`
Downloaded 3 files:
/source1/aaa.txt, /source1/bbb.txt, /source1/ccc.txt
latest being /source1/ccc.txt with timestamp 01/29/2020 07:49:07.
/source2 => C:\dest2
Downloaded 1 files:
/source2/aaa.txt
latest being /source2/aaa.txt with timestamp 01/29/2020 07:22:37.
Failed to download 1 files:
/source2/bbb.txt
To check and make sure the csv file exists before you process the entire thing, you can use the Test-Path,
...
if (!(Test-Path D:\FileSource.csv)) {
Write-Output "No File Found"
$SMTPBody += "There are no new files at the moment"
return; # Dont run. file not found. Exit out.
}
Import-Csv -Path "D:\FILESOURCE.csv" -ErrorAction Stop | foreach {
...
and for the Body error you are getting, it is coming from the finally loop because there are cases where $SMTPBody would be null. This will no longer be an issue because $SMTPBody will have some text when file is not found at the beginning.
Even though you are using return in the if statement to check if the file exists, finally will always get executed. Since we updated $smtpbody, your Send-MailMessage will no longer error out.
Update
If you want to check if the file you are downloading already exists, you can use the if statement like this,
foreach ($download in $synchronizationResult.Downloads ) {
if (!(Test-Path Join-Path D: $download.FileName) {
$SMTPBody += "File $($download.Filename) already exists, skipping."
continue # will go to the next download...
}
Write-Host "File $($download.FileName) downloaded" -ForegroundColor Green
...
If you do get the error regarding body, thats mostly because your script came across an exception and was sent straight over to finally statement. Finally statement sends the email with empty body because it was never set (due to exception). I would recommend using the debugger (step through) and see which step causes the exception and look into adding steps to make sure script doesnt fail.

PowerShell script ignores files with <whitespace>H

I inherited this script from my precursor, which is supposed to copy a bunch of files from one location into a JFrog Artifactory.
It workes fine except for files with followed by an H in the name.
For example Test My Client.exe will be copied correctly but Test My Host.exe will be ignored by the script.
Anybody has an idea what the reason for this strange behaviour is?
param
(
[string]$RootPath="D:\_temp\upload\",
[string]$Repo="repo-snapshot",
[string]$Product="SEARCHER",
[string]$Version="1.23.45678.9012",
[string]$ArtifactoryRoot
)
$AF_USER = "myUser"
$AF_PWD = "pA55VVoRd" #ConvertTo-SecureString "pA55VVoRd" -AsPlainText -Force
$CREDS = New-Object System.Management.Automation.PSCredential ($AF_USER, $AF_PWD)
$MajorMinor = $Version.Split('.')[0] + "." + $Version.Split('.')[1]
$BuildRevision = $Version.Split('.')[2] + "." + $Version.Split('.')[3]
$uri = "https://artifactory.test.com/artifactory/$Repo/TEST/$Product/$MajorMinor/$BuildRevision/$ArtifactoryRoot/"
$curl = "$PSScriptRoot\..\..\bin\ReleaseU\curl.exe"
Write-Host $MajorMinor
write-host $BuildRevision
$filesToUpload = Get-ChildItem $RootPath -Recurse -File
foreach($file in $filesToUpload)
{
$artifactoryAddPath = $file.FullName.Remove(0,$RootPath.Length).Replace('\','/')
$completeUrl = $uri+$artifactoryAddPath
write-host $completeUrl
$FileSize = (Get-Item $file.FullName).Length /1MB
Write-Host "Filesize: "$FileSize" MB"
$preTime = Get-Date
write-host "$curl --user $AF_USER`:$AF_PWD --upload-file " $file.FullName "$completeUrl --insecure"
. $curl --user $AF_USER`:$AF_PWD --upload-file $file.FullName $completeUrl --insecure
$postTime = Get-Date
Write-Host "Upload finished"
$timeDiff = $postTime - $preTime
$avgSpeed = $FileSize / $timeDiff.TotalSeconds
Write-Host -Object ("Upload speed is: {0:N2}MB/sec" -f ($avgSpeed));
}

Upload files with FTP using PowerShell

I want to use PowerShell to transfer files with FTP to an anonymous FTP server. I would not use any extra packages. How?
I am not sure you can 100% bullet proof the script from not hanging or crashing, as there are things outside your control (what if the server loses power mid-upload?) - but this should provide a solid foundation for getting you started:
# create the FtpWebRequest and configure it
$ftp = [System.Net.FtpWebRequest]::Create("ftp://localhost/me.png")
$ftp = [System.Net.FtpWebRequest]$ftp
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential("anonymous","anonymous#localhost")
$ftp.UseBinary = $true
$ftp.UsePassive = $true
# read in the file to upload as a byte array
$content = [System.IO.File]::ReadAllBytes("C:\me.png")
$ftp.ContentLength = $content.Length
# get the request stream, and write the bytes into it
$rs = $ftp.GetRequestStream()
$rs.Write($content, 0, $content.Length)
# be sure to clean up after ourselves
$rs.Close()
$rs.Dispose()
There are some other ways too. I have used the following script:
$File = "D:\Dev\somefilename.zip";
$ftp = "ftp://username:password#example.com/pub/incoming/somefilename.zip";
Write-Host -Object "ftp url: $ftp";
$webclient = New-Object -TypeName System.Net.WebClient;
$uri = New-Object -TypeName System.Uri -ArgumentList $ftp;
Write-Host -Object "Uploading $File...";
$webclient.UploadFile($uri, $File);
And you could run a script against the windows FTP command line utility using the following command
ftp -s:script.txt
(Check out this article)
The following question on SO also answers this: How to script FTP upload and download?
I'm not gonna claim that this is more elegant than the highest-voted solution...but this is cool (well, at least in my mind LOL) in its own way:
$server = "ftp.lolcats.com"
$filelist = "file1.txt file2.txt"
"open $server
user $user $password
binary
cd $dir
" +
($filelist.split(' ') | %{ "put ""$_""`n" }) | ftp -i -in
As you can see, it uses that dinky built-in windows FTP client. Much shorter and straightforward, too. Yes, I've actually used this and it works!
Easiest way
The most trivial way to upload a binary file to an FTP server using PowerShell is using WebClient.UploadFile:
$client = New-Object System.Net.WebClient
$client.Credentials =
New-Object System.Net.NetworkCredential("username", "password")
$client.UploadFile(
"ftp://ftp.example.com/remote/path/file.zip", "C:\local\path\file.zip")
Advanced options
If you need a greater control, that WebClient does not offer (like TLS/SSL encryption, etc), use FtpWebRequest. Easy way is to just copy a FileStream to FTP stream using Stream.CopyTo:
$request = [Net.WebRequest]::Create("ftp://ftp.example.com/remote/path/file.zip")
$request.Credentials =
New-Object System.Net.NetworkCredential("username", "password")
$request.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$fileStream = [System.IO.File]::OpenRead("C:\local\path\file.zip")
$ftpStream = $request.GetRequestStream()
$fileStream.CopyTo($ftpStream)
$ftpStream.Dispose()
$fileStream.Dispose()
Progress monitoring
If you need to monitor an upload progress, you have to copy the contents by chunks yourself:
$request = [Net.WebRequest]::Create("ftp://ftp.example.com/remote/path/file.zip")
$request.Credentials =
New-Object System.Net.NetworkCredential("username", "password")
$request.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$fileStream = [System.IO.File]::OpenRead("C:\local\path\file.zip")
$ftpStream = $request.GetRequestStream()
$buffer = New-Object Byte[] 10240
while (($read = $fileStream.Read($buffer, 0, $buffer.Length)) -gt 0)
{
$ftpStream.Write($buffer, 0, $read)
$pct = ($fileStream.Position / $fileStream.Length)
Write-Progress `
-Activity "Uploading" -Status ("{0:P0} complete:" -f $pct) `
-PercentComplete ($pct * 100)
}
$ftpStream.Dispose()
$fileStream.Dispose()
Uploading folder
If you want to upload all files from a folder, see
PowerShell Script to upload an entire folder to FTP
I recently wrote for powershell several functions for communicating with FTP, see https://github.com/AstralisSomnium/PowerShell-No-Library-Just-Functions/blob/master/FTPModule.ps1. The second function below, you can send a whole local folder to FTP. In the module are even functions for removing / adding / reading folders and files recursively.
#Add-FtpFile -ftpFilePath "ftp://myHost.com/folder/somewhere/uploaded.txt" -localFile "C:\temp\file.txt" -userName "User" -password "pw"
function Add-FtpFile($ftpFilePath, $localFile, $username, $password) {
$ftprequest = New-FtpRequest -sourceUri $ftpFilePath -method ([System.Net.WebRequestMethods+Ftp]::UploadFile) -username $username -password $password
Write-Host "$($ftpRequest.Method) for '$($ftpRequest.RequestUri)' complete'"
$content = $content = [System.IO.File]::ReadAllBytes($localFile)
$ftprequest.ContentLength = $content.Length
$requestStream = $ftprequest.GetRequestStream()
$requestStream.Write($content, 0, $content.Length)
$requestStream.Close()
$requestStream.Dispose()
}
#Add-FtpFolderWithFiles -sourceFolder "C:\temp\" -destinationFolder "ftp://myHost.com/folder/somewhere/" -userName "User" -password "pw"
function Add-FtpFolderWithFiles($sourceFolder, $destinationFolder, $userName, $password) {
Add-FtpDirectory $destinationFolder $userName $password
$files = Get-ChildItem $sourceFolder -File
foreach($file in $files) {
$uploadUrl ="$destinationFolder/$($file.Name)"
Add-FtpFile -ftpFilePath $uploadUrl -localFile $file.FullName -username $userName -password $password
}
}
#Add-FtpFolderWithFilesRecursive -sourceFolder "C:\temp\" -destinationFolder "ftp://myHost.com/folder/" -userName "User" -password "pw"
function Add-FtpFolderWithFilesRecursive($sourceFolder, $destinationFolder, $userName, $password) {
Add-FtpFolderWithFiles -sourceFolder $sourceFolder -destinationFolder $destinationFolder -userName $userName -password $password
$subDirectories = Get-ChildItem $sourceFolder -Directory
$fromUri = new-object System.Uri($sourceFolder)
foreach($subDirectory in $subDirectories) {
$toUri = new-object System.Uri($subDirectory.FullName)
$relativeUrl = $fromUri.MakeRelativeUri($toUri)
$relativePath = [System.Uri]::UnescapeDataString($relativeUrl.ToString())
$lastFolder = $relativePath.Substring($relativePath.LastIndexOf("/")+1)
Add-FtpFolderWithFilesRecursive -sourceFolder $subDirectory.FullName -destinationFolder "$destinationFolder/$lastFolder" -userName $userName -password $password
}
}
Here's my super cool version BECAUSE IT HAS A PROGRESS BAR :-)
Which is a completely useless feature, I know, but it still looks cool \m/ \m/
$webclient = New-Object System.Net.WebClient
Register-ObjectEvent -InputObject $webclient -EventName "UploadProgressChanged" -Action { Write-Progress -Activity "Upload progress..." -Status "Uploading" -PercentComplete $EventArgs.ProgressPercentage } > $null
$File = "filename.zip"
$ftp = "ftp://user:password#server/filename.zip"
$uri = New-Object System.Uri($ftp)
try{
$webclient.UploadFileAsync($uri, $File)
}
catch [Net.WebException]
{
Write-Host $_.Exception.ToString() -foregroundcolor red
}
while ($webclient.IsBusy) { continue }
PS. Helps a lot, when I'm wondering "did it stop working, or is it just my slow ASDL connection?"
You can simply handle file uploads through PowerShell, like this.
Complete project is available on Github here https://github.com/edouardkombo/PowerShellFtp
#Directory where to find pictures to upload
$Dir= 'c:\fff\medias\'
#Directory where to save uploaded pictures
$saveDir = 'c:\fff\save\'
#ftp server params
$ftp = 'ftp://10.0.1.11:21/'
$user = 'user'
$pass = 'pass'
#Connect to ftp webclient
$webclient = New-Object System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass)
#Initialize var for infinite loop
$i=0
#Infinite loop
while($i -eq 0){
#Pause 1 seconde before continue
Start-Sleep -sec 1
#Search for pictures in directory
foreach($item in (dir $Dir "*.jpg"))
{
#Set default network status to 1
$onNetwork = "1"
#Get picture creation dateTime...
$pictureDateTime = (Get-ChildItem $item.fullName).CreationTime
#Convert dateTime to timeStamp
$pictureTimeStamp = (Get-Date $pictureDateTime).ToFileTime()
#Get actual timeStamp
$timeStamp = (Get-Date).ToFileTime()
#Get picture lifeTime
$pictureLifeTime = $timeStamp - $pictureTimeStamp
#We only treat pictures that are fully written on the disk
#So, we put a 2 second delay to ensure even big pictures have been fully wirtten in the disk
if($pictureLifeTime -gt "2") {
#If upload fails, we set network status at 0
try{
$uri = New-Object System.Uri($ftp+$item.Name)
$webclient.UploadFile($uri, $item.FullName)
} catch [Exception] {
$onNetwork = "0"
write-host $_.Exception.Message;
}
#If upload succeeded, we do further actions
if($onNetwork -eq "1"){
"Copying $item..."
Copy-Item -path $item.fullName -destination $saveDir$item
"Deleting $item..."
Remove-Item $item.fullName
}
}
}
}
You can use this function :
function SendByFTP {
param (
$userFTP = "anonymous",
$passFTP = "anonymous",
[Parameter(Mandatory=$True)]$serverFTP,
[Parameter(Mandatory=$True)]$localFile,
[Parameter(Mandatory=$True)]$remotePath
)
if(Test-Path $localFile){
$remoteFile = $localFile.Split("\")[-1]
$remotePath = Join-Path -Path $remotePath -ChildPath $remoteFile
$ftpAddr = "ftp://${userFTP}:${passFTP}#${serverFTP}/$remotePath"
$browser = New-Object System.Net.WebClient
$url = New-Object System.Uri($ftpAddr)
$browser.UploadFile($url, $localFile)
}
else{
Return "Unable to find $localFile"
}
}
This function send specified file by FTP.
You must call the function with these parameters :
userFTP = "anonymous" by default or your username
passFTP = "anonymous" by default or your password
serverFTP = IP address of the FTP server
localFile = File to send
remotePath = the path on the FTP server
For example :
SendByFTP -userFTP "USERNAME" -passFTP "PASSWORD" -serverFTP "MYSERVER" -localFile "toto.zip" -remotePath "path/on/the/FTP/"
Goyuix's solution works great, but as presented it gives me this error: "The requested FTP command is not supported when using HTTP proxy."
Adding this line after $ftp.UsePassive = $true fixed the problem for me:
$ftp.Proxy = $null;
Simple solution if you can install curl.
curl.exe -p --insecure "ftp://<ftp_server>" --user "user:password" -T "local_file_full_path"