Upload file to SFTP using PowerShell - powershell

We were asked to set up an automated upload from one of our servers to an SFTP site. There will be a file that is exported from a database to a filer every Monday morning and they want the file to be uploaded to SFTP on Tuesday. The current authentication method we are using is username and password (I believe there was an option to have key file as well but username/password option was chosen).
The way I am envisioning this is to have a script sitting on a server that will be triggered by Windows Task scheduler to run at a specific time (Tuesday) that will grab the file in question upload it to the SFTP and then move it to a different location for backup purposes.
For example:
Local Directory: C:\FileDump
SFTP Directory: /Outbox/
Backup Directory: C:\Backup
I tried few things at this point WinSCP being one of them as well as SFTP PowerShell Snap-In but nothing has worked for me so far.
This will be running on Windows Server 2012R2.
When I run Get-Host my console host version is 4.0.
Thanks.

You didn't tell us what particular problem do you have with the WinSCP, so I can really only repeat what's in WinSCP documentation.
Download WinSCP .NET assembly.
The latest package as of now is WinSCP-5.21.7-Automation.zip;
Extract the .zip archive along your script;
Use a code like this (based on the official PowerShell upload example):
# Load WinSCP .NET assembly
Add-Type -Path "WinSCPnet.dll"
# Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Sftp
HostName = "example.com"
UserName = "user"
Password = "mypassword"
SshHostKeyFingerprint = "ssh-rsa 2048 xxxxxxxxxxx...="
}
$session = New-Object WinSCP.Session
try
{
# Connect
$session.Open($sessionOptions)
# Upload
$session.PutFiles("C:\FileDump\export.txt", "/Outbox/").Check()
}
finally
{
# Disconnect, clean up
$session.Dispose()
}
You can have WinSCP generate the PowerShell script for the upload for you:
Login to your server with WinSCP GUI;
Navigate to the target directory in the remote file panel;
Select the file for upload in the local file panel;
Invoke the Upload command;
On the Transfer options dialog, go to Transfer Settings > Generate Code;
On the Generate transfer code dialog, select the .NET assembly code tab;
Choose PowerShell language.
You will get a code like above with all session and transfer settings filled in.
(I'm the author of WinSCP)

There isn't currently a built-in PowerShell method for doing the SFTP part. You'll have to use something like psftp.exe or a PowerShell module like Posh-SSH.
Here is an example using Posh-SSH:
# Set the credentials
$Password = ConvertTo-SecureString 'Password1' -AsPlainText -Force
$Credential = New-Object System.Management.Automation.PSCredential ('root', $Password)
# Set local file path, SFTP path, and the backup location path which I assume is an SMB path
$FilePath = "C:\FileDump\test.txt"
$SftpPath = '/Outbox'
$SmbPath = '\\filer01\Backup'
# Set the IP of the SFTP server
$SftpIp = '10.209.26.105'
# Load the Posh-SSH module
Import-Module C:\Temp\Posh-SSH
# Establish the SFTP connection
$ThisSession = New-SFTPSession -ComputerName $SftpIp -Credential $Credential
# Upload the file to the SFTP path
Set-SFTPFile -SessionId ($ThisSession).SessionId -LocalFile $FilePath -RemotePath $SftpPath
#Disconnect all SFTP Sessions
Get-SFTPSession | % { Remove-SFTPSession -SessionId ($_.SessionId) }
# Copy the file to the SMB location
Copy-Item -Path $FilePath -Destination $SmbPath
Some additional notes:
You'll have to download the Posh-SSH module which you can install to your user module directory (e.g. C:\Users\jon_dechiro\Documents\WindowsPowerShell\Modules) and just load using the name or put it anywhere and load it like I have in the code above.
If having the credentials in the script is not acceptable you'll have to use a credential file. If you need help with that I can update with some details or point you to some links.
Change the paths, IPs, etc. as needed.
That should give you a decent starting point.

I am able to sftp using PowerShell as below:
PS C:\Users\user\Desktop> sftp user#aa.bb.cc.dd
user#aa.bb.cc.dd's password:
Connected to user#aa.bb.cc.dd.
sftp> ls
testFolder
sftp> cd testFolder
sftp> ls
taj_mahal.jpeg
sftp> put taj_mahal_1.jpeg
Uploading taj_mahal_1.jpeg to /home/user/testFolder/taj_mahal_1.jpeg
taj_mahal_1.jpeg 100% 11KB 35.6KB/s 00:00
sftp> ls
taj_mahal.jpeg taj_mahal_1.jpeg
sftp>
I do not have installed Posh-SSH or anything like that. I am using Windows 10 Pro PowerShell. No additional modules installed.

Using PuTTY's pscp.exe (which I have in an $env:path directory):
pscp -sftp -pw passwd c:\filedump\* user#host:/Outbox/
mv c:\filedump\* c:\backup\*

$FilePath = "C:\Backup\xxx.zip"
$SftpPath = '/Cloud_Deployment/Backup'
$SftpIp = 'mercury.xxx.xx.uk' #Or IP
$Password = 'password'
$userroot = 'username'
$Password = ConvertTo-SecureString $Password -AsPlainText -Force
$Credential = New-Object System.Management.Automation.PSCredential ($userroot, $Password)
Install-Module -Name Posh-SSH #rus as Admin
$SFTPSession = New-SFTPSession -ComputerName $SftpIp -Credential $Credential
#Download file
#Get-SFTPItem -SessionId $SFTPSession.SessionId -Path $SftpPath/test.txt -Destination c:\temp
#Upload file
Set-SFTPItem -SessionId $SFTPSession.SessionId -Path $FilePath -Destination $SftpPath
#Disconnect all SFTP Sessions
Remove-SFTPSession -SFTPSession $SFTPSession
#or
Get-SFTPSession | % { Remove-SFTPSession -SessionId ($_.SessionId) }
Ref : powershell-sftp
If any how you face error "PackageManagement\Install-Package : No match was found for the specified search criteria and module name 'Posh-SSH'"
Then please visit Here

Well, while using powershell 7, we can simply upload files using sftp with following command
echo "put localpath/file.txt destinationpath/file.txt" | sftp username#server
make sure to add these double quotes.

Related

Upload file using SFTP in PowerShell

I am trying the below command but didn't get any output:
echo "put C:\Users\abhishek.chawla\Desktop\AbhishekC_bills.zip deployment/AbhishekC_bills.zip" | sftp TESTSHELTER#shelter-ftp.outlinesys.com
Do I need to add the password too? If yes, then where to add the password?
One of the workaround you can follow ,
To provide the password into the command for upload files to SFTP using PowerShell through Winscp below is the example of code.
cmdltes:-
Add-Type -Path "WinSCPnet.dll"
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Sftp
HostName = "example.com"
UserName = "user"
Password = "mypassword"
SshHostKeyFingerprint = "ssh-rsa 2048 xx...="
}
$session = New-Object WinSCP.Session
try
{
$session.Open($sessionOptions)
$session.PutFiles("C:\users\export.txt","/Outbox/").Check()
}
finally
{
$session.Dispose()
}
For complete information please refer this BLOG|Use WinSCP to Upload Files to SFTP Site With PowerShell .
For more information regarding similar issue please refer the below links:-
SO THREAD| Power Shell File Upload to SFTP with File Existence Check & Upload file to SFTP using PowerShell .

how to delete files on ftp server older then 10 days [duplicate]

I have to write a script which accesses an FTP server, and then deletes all *.zip files which are older than X days.
As clarification: The script can't run on the FTP server.
This is what I have so far:
$ftpServer = "RandomFTPServer"
$ftpUser = "Username"
$ftpPassword = Read-Host "Password" -AsSecureString
$credentials = New-ObjectSystem.Net.NetworkCredential($ftpUser, $ftpPassword)
function Get-FtpRequest($ftpPath) {
$ftpRequest = [System.Net.FtpWebRequest]::Create("$ftpServer/$ftpPath")
$ftpRequest.Credentials = $credentials
$ftpRequest.UseBinary = $true
$ftpRequest.KeepAlive = $true
$ftpRequest.UsePassive = $true
return $ftpRequest
}
Any tips on what I need to do next?
You have to retrieve timestamps of remote files to select the old ones.
Unfortunately, there's no really reliable and efficient way to retrieve timestamps using features offered by .NET framework/PowerShell as it does not support FTP MLSD command.
So either you use:
ListDirectoryDetails method (FTP LIST command) to retrieve details of all files in a directory and then you deal with FTP server specific format of the details (*nix format similar to ls *nix command is the most common, drawback is that the format may change over time, as for newer files "May 8 17:48" format is used and for older files "Oct 18 2009" format is used)
GetDateTimestamp method (FTP MDTM command) to individually retrieve timestamps for each file. Advantage is that the response is standardized by RFC 3659 to YYYYMMDDHHMMSS[.sss]. Disadvantage is that you have to send a separate request for each file, what can be quite inefficient.
Some references:
C# class to parse WebRequestMethods.Ftp.ListDirectoryDetails FTP response
Parsing FtpWebRequest ListDirectoryDetails line
Retrieving creation date of file (FTP)
PowerShell FTP download files and subfolders
C# Download all files and subdirectories through FTP
Though Microsoft does not recommend FtpWebRequest for a new development anyway.
Alternatively you can use a 3rd party FTP client implementation that supports the modern MLSD command and/or has built-in support for parsing different formats of the LIST command.
For example, WinSCP .NET assembly supports both.
# Load WinSCP .NET assembly
Add-Type -Path "WinSCPnet.dll"
# Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Ftp
HostName = "ftp.example.com"
UserName = "username"
Password = "password"
}
try
{
# Connect
$session = New-Object WinSCP.Session
$session.Open($sessionOptions)
# List files
$remotePath = "/remote/path"
$directoryInfo = $session.ListDirectory($remotePath)
# Find old files
$limit = (Get-Date).AddDays(-15)
$oldFiles =
$directoryInfo.Files |
Where-Object { -Not $_.IsDirectory } |
Where-Object { $_.LastWriteTime -lt $limit }
# Delete them
foreach ($oldFileInfo in $oldFiles)
{
$session.RemoveFile($oldFileInfo.FullName).Check()
}
Write-Host "Done"
}
finally
{
# Disconnect, clean up
$session.Dispose()
}
If you can do with a plain batch file, it's actually even easier with WinSCP scripting:
winscp.com /ini=nul /log=delete.log /command ^
"open ftp://username:password#ftp.example.com/" ^
"rm /remote/path/*<15D" ^
"exit"
See file masks with time constraints.
(I'm the author of WinSCP)
Note that WinSCP does not require any installation. So you can just have its binaries copied around with your batch file or PowerShell script.
I'm currently doing this using FTPUSE, a freeware command-line tool which maps a FTP folder to a windows drive letter, together with a batch file in the following way:
: delete files older than 7 days from ftp://my.ftpsite.net/folder/subfolder
ftpuse F: my.ftpsite.net password /USER:username
timeout /t 5
forfiles -p "F:\folder\subfolder" -s -m *.* -d -7 -c "cmd /C DEL #File /Q"
ftpuse F: /DELETE
The software is compatible with all major versions of windows: Windows XP, Vista, 7, Server 2003, Server 2008, Windows 8, Server 2012 and Windows 10 (32-bit, 64-bit).
For further info, you can also read this post I wrote about FTPUSE (I'm not the author in any way, I just find it very useful for these kind of tasks).

Reference WinSCP.exe from PowerShell script executed from SSIS

I am trying to execute a PowerShell script from within SSIS. My script starts with the Add-Type -Path "WinSCPnet.dll" and it is erroring out because it cannot find the WinSCP.exe in the folder that houses my PowerShell script. Come to find out the server admin did NOT install WinSCP into the GAC. Is this creating my problem?
If so, how and where can I reference the WinSCP.exe in my script using $session.ExecutablePath? Any help/direction would be appreciated. Thanks.
Here is my script below:
# Load WinSCP .NET assembly
Add-Type -Path "WinSCPnet.dll"
# Declare variables
$date = Get-Date
$dateStr = $date.ToString("yyyyMMdd")
#$fileDirectory = "\\abicfs2\apps\CoverageVerifier\"
#$filePath = "\\abicfs2\apps\CoverageVerifier\cvgver." + $dateStr + ".0101"
$filePath = "\\empqaapp1\coverageverifier_scripts\CoverageVerifier\cvgver.20190121.0101"
# Write-Output $filePath
# Set up session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Sftp
HostName = "secureftp.iso.com"
UserName = "account"
Password = "password"
SshHostKeyFingerprint = "ssh-rsa 2048 8C1lwAjxCNRF6B4kbPIeW52/GB+98FmLMt0AJNf/Sf4="
}
#$sessionOptions.AddRawSettings("FSProtocol", "2")
$session = New-Object WinSCP.Session
# $session.SessionLogPath = "\\share\apps\CoverageVerifier\UploadLog.log"
try
{
# Connect
$session.Open($sessionOptions)
# Transfer files
$session.PutFiles($filePath,"/").Check()
}
finally
{
$session.Dispose()
}
I am trying to execute a Powershell script from within SSIS
It seems you believe you need to have WinSCP .NET assembly in GAC, so that you can execute it from a PowerShell script executed from SSIS. I do not think it's true. You need an assembly in GAC, only if you directly use it from an SSIS code. What is not your case.
You can simply store the WinSCPnet.dll and WinSCP.exe to your PowerShell script directory.
Anyway to answer your question:
If so, how and where can I reference the WinSCP.exe in my script using $session.ExecutablePath?
$session = New-Object WinSCP.Session
$session.ExecutablePath = "C:\path\WinSCP.exe"
(but as per above, I do not think you need it)
Come to find out the server admin did NOT install WinSCP into the GAC.
You cannot install .exe file to GAC.
Excerpt from WinSCP assembly installation instruction (https://winscp.net/eng/docs/library_install):
The package includes the assembly itself (winscpnet.dll) and a
required dependency, WinSCP executable winscp.exe.
The binaries interact with each other and must be kept in the same
folder for the assembly to work. In rare situations this is not
possible (e.g. when installing the assembly to GAC), make use of the
Session.ExecutablePath property to force the assembly to look for the
winscp.exe in a different location.

PowerShell - File transfer from windows to unix

I am working on UIpath automation for which I need some files to be transferred back and forth between Windows and Unix machines (only through PowerShell). Kindly provide your inputs as I'm a newbie.
I am using plink in my PowerShell script to connect to a Unix server. Though it works fine, is there any other better way to connect to a Unix server (HP UX) from Windows (through a PowerShell script).
Struggling to find a good module and sample scripts to do a secure copy between the Unix and Windows servers. I came across Posh SSH /WinSCP, sftp etc. but I'm not able to implement any as I do not find the right sample scripts. Also Install-Module does not work (not recognized).
Your help on this would be much appreciated.
Thanks in advance!
If you want to use SFTP I am using the code below to upload some files automatically to an ftp site:
First of all you have to download the winscp SFTP powershell libraries.
https://winscp.net/eng/download.php
then extract the contents at the same location the script is located.
Then in your script you must add:
# Load WinSCP .NET assembly
# Give the path the dll file is located so powershell can call it.
Add-Type -Path "C:\Path where the dll file is located\WinSCPnet.dll"
# Setup session options
# Add all the properties the session needs to be established such as username password hostname and fingerprint.
# The username and password must be in plaintext.
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Sftp
HostName = "HostName"
UserName = "UserName"
Password = "Password"
SshHostKeyFingerprint = "SSH fingerprint"
Then after the session with those credentials is up you must put your next step of copying the files.
# Open a session to the Host
# Try to connect to the Host with the credentials and information provided previously.
# Upload the file from a specific path to the path on the host.
# And then close the session and clean up the session trace data.
# $session.Dispose() -> If session was opened, closes it, terminates underlying WinSCP process, deletes XML log file and disposes object.
$session = New-Object WinSCP.Session
Try
{
# Connect to the SFTP site
$session.Open($sessionOptions)
# Upload the files from local disk to the destination
$session.PutFiles("Path of the file you want to upload", "/import/").Check()
}
Finally
{
# Disconnect, clean up
$session.Dispose()
}
Probably there is an easier way with Power Shell 6 that can do more with the Unix/Linux operating systems but at this point of answering I haven't used it.
Once you get Posh-SSH installed, something like this will probably get you down the road. There are ways to keep the password in plain text out of your script.
$SecurePassword = ConvertTo-SecureString -AsPlainText 'thepassword' -Force
$Credentials = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList 'me',$SecurePassword
$Session = New-SFTPSession -ComputerName 'zenith' -Credential $Credentials -ConnectionTimeout 30
$r = Set-SFTPFile -Session $Session. -LocalFile 'C:\Users\me\t.txt' -RemotePath '/home/me' -Overwrite
Remove-SFTPSession -SFTPSession $session | Out-Null

Powershell - Copying File to Remote Host and Executing Install exe using WMI

EDITED: Here is my code now. The install file does copy to the remote host. However, the WMI portion does not install the .exe file, and no errors are returned. Perhaps this is a syntax error with WMI? Is there a way to just run the installer silently with PsExec? Thanks again for all the help sorry for the confusion:
#declare params
param (
[string]$finalCountdownPath = "",
[string]$slashes = "\\",
[string]$pathOnRemoteHost = "c:\temp\",
[string]$targetJavaComputer = "",
[string]$compname = "",
[string]$tempPathTarget = "\C$\temp\"
)
# user enters target host/computer
$targetJavaComputer = Read-Host "Enter the name of the computer on which you wish to install Java:"
[string]$compname = $slashes + $targetJavaComputer
[string]$finalCountdownPath = $compname + $tempPathTarget
#[string]$tempPathTarget2 =
#[string]$finalCountdownPath2 = $compname + $
# say copy install media to remote host
echo "Copying install file and running installer silently please wait..."
# create temp dir if does not exist, if exist copy install media
# if does not exist create dir, copy dummy file, copy install media
# either case will execute install of .exe via WMII
#[string]$finalCountdownPath = $compname + $tempPathTarget;
if ((Test-Path -Path $finalCountdownPath) )
{
copy c:\hdatools\java\jre-7u60-windows-i586.exe $finalCountdownPath
([WMICLASS]"\\$targetJavaComputer\ROOT\CIMV2:win32_process").Create("cmd.exe /c c:\temp\java\jre-7u60-windows-i586.exe /s /v`" /qn")
}
else {
New-Item -Path $finalCountdownPath -type directory -Force
copy c:\hdatools\dummy.txt $finalCountdownPath
copy "c:\hdatools\java\jre-7u60-windows-i586.exe" $finalCountdownPath
([WMICLASS]"\\$targetJavaComputer\ROOT\CIMV2:win32_process").Create("cmd.exe /c c:\temp\java\jre-7u60-windows-i586.exe /s /v`" /qn")
}
I was trying to get $Job = Invoke-Command -Session $Session -Scriptblock $Script to allow me to copy files on a different server, because I needed to off load it from the server it was running from. I was using the PowerShell Copy-Item to do it. But the running PowerShell script waits until the file is done copying to return.
I want it to take as little resources as possible on the server that the powershell is running to spawn off the process on another server to copy the file. I tried to user various other schemes out there, but they didn't work or the way I needed them to work. (Seemed kind of kludgey or too complex to me.) Maybe some of them could have worked? But I found a solution that I like that works best for me, which is pretty easy. (Except for some of the back end configuration that may be needed if it is is not already setup.)
Background:
I am running a SQLServer Job which invokes Powershell to run a script which backups databases, copies backup files, and deletes older backup files, with parameters passed into it. Our server is configured to allow PowerShell to run and under the pre-setup User account with SQL Server Admin and dbo privileges in an Active Directory account to allow it to see various places on our Network as well.
But we don't want it to take the resources away from the main server. The PowerShell script that was to be run would backup the database Log file and then use the another server to asynchronously copy the file itself and not make the SQL Server Job/PowerShell wait for it. We wanted it to happen right after the backup.
Here is my new way, using WMI, using Windows Integrate Security:
$ComputerName = "kithhelpdesk"
([Wmiclass]'Win32_Process').GetMethodParameters('Create')
Invoke-WmiMethod -ComputerName RemoteServerToRunOn -Path win32_process -Name create -ArgumentList 'powershell.exe -Command "Copy-Item -Path \\YourShareSource\SQLBackup\YourDatabase_2018-08-07_11-45.log.bak -Destination \\YourShareDestination\YourDatabase_2018-08-07_11-45.log.bak"'
Here is my new way using passed in Credentials, and building arg list variable:
$Username = "YouDomain\YourDomainUser"
$Password = "P#ssw0rd27"
$ComputerName = "RemoteServerToRunOn"
$FromFile = "\\YourShareSource\SQLBackup\YourDatabase_2018-08-07_11-45.log.bak"
$ToFile = "\\YourShareDestination\SQLBackup\YourDatabase_2018-08-07_11-45.log.bak"
$ArgumentList = 'powershell.exe -Command "Copy-Item -Path ' + $FromFile + ' -Destination ' + $ToFile + '"'
$SecurePassWord = ConvertTo-SecureString -AsPlainText $Password -Force
$Cred = New-Object -TypeName "System.Management.Automation.PSCredential" -ArgumentList $Username, $SecurePassWord
([Wmiclass]'Win32_Process').GetMethodParameters('Create')
Invoke-WmiMethod -ComputerName $ComputerName -Path win32_process -Name create -ArgumentList $ArgumentList -Credential $Cred
We think that this above one is the preferred one to use.
You can also run a specific powershell that will do what you want it to do (even passing in parameters to it):
Invoke-WmiMethod -ComputerName RemoteServerToRunOn -Path win32_process -Name create -ArgumentList 'powershell.exe -file "C:\PS\Test1.ps1"'
This example could be changed to pass in parameters to the Test1.ps1 PowerShell script to make it more flexible and reusable. And you may also want to pass in a Credential like we used in a previous example above.
Help configuring WMI:
I got the main gist of this working from: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/invoke-wmimethod?view=powershell-5.1
But it may have also needed WMI configuration using:
https://helpcenter.gsx.com/hc/en-us/articles/202447926-How-to-Configure-Windows-Remote-PowerShell-Access-for-Non-Privileged-User-Accounts?flash_digest=bec1f6a29327161f08e1f2db77e64856b433cb5a
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/enable-psremoting?view=powershell-5.1
Powershell New-PSSession Access Denied - Administrator Account
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/invoke-wmimethod?view=powershell-5.1 (I used to get how to call Invoke-WmiMethod).
https://learn.microsoft.com/en-us/powershell/scripting/core-powershell/console/powershell.exe-command-line-help?view=powershell-6 (I used to get syntax of command line)
I didn't use this one, but could have: How to execute a command in a remote computer?
I don't know for sure if all of the steps in the web articles above are needed, I suspect not. But I thought I was going to be using the Invoke-Command PowerShell statement to copy the files on a remote server, but left my changes from the articles above that I did intact mostly I believe.
You will need a dedicated User setup in Active Directory, and to configure the user accounts that SQL Server and SQL Server Agent are running under to give the main calling PowerShell the privileges needed to access the network and other things to, and can be used to run the PowerShell on the remote server as well. And you may need to configure SQLServer to allow SQL Server Jobs or Stored Procedures to be able to call PowerShell scripts like I did. But this is outside the scope of this post. You Google other places on the internet to show you how to do that.