I have previously using .bat scripts for this process and am relatively new to writing powershell scripts. I am conducting a transfer from an FTP location and need to delete each file from the FTP server after transfer. What is the correct syntax for this? Code is as follows:-
$a=Get-Date -format MMMM
try
{
# Load WinSCP .NET assembly
Add-Type -Path "C:\Program Files (x86)\WinSCP\WinSCPnet.dll"
# Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Sftp
HostName = "xxx.xxx.xxx.xxx"
UserName = "joe"
Password = "joebloggs"
SshHostKeyFingerprint = "ssh-rsa 1024 01:07:da:11:22:33:44:55:66"
FtpMode = [WinSCP.FtpMode]::Active
}
$session = New-Object WinSCP.Session
try
{
# Connect
$session.Open($sessionOptions)
# Download files
$transferOptions = New-Object WinSCP.TransferOptions
$transferOptions.TransferMode = [WinSCP.TransferMode]::Binary
$transferResult = $session.GetFiles("/*", "C:\Users\User\Dropbox\Rockar\Data\System Data\Raw Feeds\Stock\"+$a+"\", $False, $transferOptions)
# Throw on any error
$transferResult.Check()
# Print results
foreach ($transfer in $transferResult.Transfers)
{
Write-Host ("Download of {0} succeeded" -f $transfer.FileName)
}
}
finally
{
# Disconnect, clean up
$session.Dispose()
}
Discovered the answer with a bit of further reading on the WinSCP site about getfiles parameters winscp.net/eng/docs/library_session_getfiles#parameters the remove parameter just needs setting to $true from $false
You should call $session.RemoveFiles, here's the method description:
https://winscp.net/eng/docs/library_session_removefiles
Related
I am using the following slightly modified script (from https://winscp.net/eng/docs/script_local_move_after_successful_upload) to upload files to an SFTP site on AWS...and in a PRD env I will have to have this run through approx 0.5M small files...
param (
$localPath = "C:\FTP\*.DAT",
$remotePath = "/",
$backupPath = "C:\FTP\Complete\"
)
try
{
# Load WinSCP .NET assembly
#Add-Type -Path "WinSCPnet.dll"
$ScriptPath = $(Split-Path -Parent $MyInvocation.MyCommand.Definition)
[Reflection.Assembly]::UnsafeLoadFrom( $(Join-Path $ScriptPath "WinSCPnet.dll") ) | Out-Null
# Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Sftp
HostName = "somewhere.com"
UserName = "user"
SshHostKeyFingerprint = "ssh-rsa 2048 XXXXXXXI"
SshPrivateKeyPath = "C:\blahblah.ppk"
}
$session = New-Object WinSCP.Session
$transferOptions = New-Object WinSCP.TransferOptions
# Look to ignore any file property change errors
$transferOptions.FilePermissions = $Null # This is default
$transferOptions.PreserveTimestamp = $False
try
{
# Connect
$session.Open($sessionOptions)
# Upload files, collect results
$transferResult = $session.PutFiles($localPath, $remotePath)
# Iterate over every transfer
foreach ($transfer in $transferResult.Transfers)
{
# Success or error?
if ($Null -eq $transfer.Error)
{
Write-Host "Upload of $($transfer.FileName) succeeded, moving to backup"
# Upload succeeded, move source file to backup
Move-Item $transfer.FileName $backupPath
}
else
{
Write-Host "Upload of $($transfer.FileName) failed: $($transfer.Error.Message)"
}
}
}
finally
{
# Disconnect, clean up
$session.Dispose()
}
exit 0
}
catch
{
Write-Host "Error: $($_.Exception.Message)"
exit 1
}
The scripts works great, but unfortunately only loads one file before disconnecting. The script fails if there is an error loading any of the files, so it is working as expected.
But the error I am getting, is
"Upload of file 'somefile.DAT' was successful, but error occurred while setting the permissions and/or timestamp.
If the problem persists, turn off setting permissions or preserving timestamp. Alternatively you can turn on 'Ignore permission errors' option.
The server does not support the operation.
Error code: 8
Error message from server (US-ASCII): SETSTAT unsupported"
I think I have the following settings possibly configured incorrectly, but I'm not sure what I am doing wrong here....thoughts?
$transferOptions.FilePermissions = $Null # This is default
$transferOptions.PreserveTimestamp = $False
I've actually managed to get this to work by modifying the session and transfer options..
$session.Open($sessionOptions)
$transferOptions = New-Object WinSCP.TransferOptions
# Look to ignore any file property change errors
$transferOptions.PreserveTimeStamp = $false
$transferOptions.FilePermissions = $Null
$transferOptions.AddRawSettings("IgnorePermErrors", "1")
# Upload files, collect results
$transferResult = $session.PutFiles($localPath, $remotePath, $False, $transferOptions)
Can you embed the private key into a PowerShell WinSCP script instead of calling for the .ppk file? Maybe something like this?
# Load WinSCP .NET assembly
Add-Type -Path "WinSCPnet.dll"
# Set up session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Scp
HostName = "domain.com"
UserName = "username"
SshHostKeyFingerprint = "ssh-rsa 2048 38741934871934871293471293487"
SecurePrivateKeyPassphrase = "AAABAHH1......................"
}
$session = New-Object WinSCP.Session
try
{
# Connect
$session.Open($sessionOptions)
# Your code
}
finally
{
$session.Dispose()
}
WinSCP .NET assembly supports reading a key from a file only.
But you can create the key file by your script:
$privateKeyPath = New-TemporaryFile
Set-Content -Path $privateKeyPath "PuTTY-User-Key-File-2: ssh-rsa
Encryption: none
Comment: ...
Public-Lines: 6
...
Private-Lines: 14
...
Private-MAC: ...
"
# Set up session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
...
SshPrivateKeyPath = $privateKeyPath
}
$session = New-Object WinSCP.Session
try
{
# Connect
$session.Open($sessionOptions)
# Your code
}
finally
{
Remove-Item $privateKeyPath
$session.Dispose()
}
Trying to run the following script. Connect to a ftp / sftp server and get the files from a remotepath to a localpath
Since there are a list of sources and destinations, I decided to create a csv file and pass the value using $.Source and $.Destination
Session.getfiles () seems to not be working when I import my csv file.
But if I hard code the path, it seems to work.
$session.GetFiles("c:\source\test", "c:\destination\", $False, $transferOptions)
Goal: Script reads the list of Sources and move files according to its destination. Also go back to a # amount of days and downdload the latest file -
Error: "No such File" or "cant get attributes of file"
try{
Add-Type -Path "WinSCPnet.dll"
# Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::sftp
HostName = "xxxxx"
UserName = "xxxxr"
Password = "xxxxxxx"
PortNumber="xx"
FTPMode="Active"
GiveUpSecurityAndAcceptAnySshHostKey = $true
}
$session = New-Object WinSCP.Session
try
{
# Connect
$session.Open($sessionOptions)
# Download files
$transferOptions = New-Object WinSCP.TransferOptions
$transferOptions.TransferMode = [WinSCP.TransferMode]::Binary
Import-Csv -Path "C:\movefiles.csv" -ErrorAction Stop
$transferResult =
$session.GetFiles($_.Source, $_.Destination, $False, $transferOptions)
# Throw on any error
$transferResult.Check()
# Print results
foreach ($transfer in $transferResult.Transfers)
{
Write-Host "Download of $($transfer.FileName) succeeded"
}
}
finally
{
# Disconnect, clean up
$session.Dispose()
}
#exit 0
}
catch
{
Write-Host "Error: $($_.Exception.Message)"
#exit 1
}
By commenting the line out
$transferresult.check()
Script seems to work. But still missing how to get only new files
I currently have a working script that upload files to a SFTP remote sirectory. The problem I am having is that there will be 3 files and they should be uploaded in sequence at different intervals. I have already thought about using Windows Task Scheduler as to take care of the frequency of the upload but there is another issue. I have identified that the files differ in naming based on one keyword. Is there a way I can modify my code to look for the files in the directory by a particular name? For example it searches in the directory for a file with "customer" in its name. Based on that keyword/name it then uploads that particular file. Please see current working script:
# Load WinSCP .NET assembly
Add-Type -Path "WinSCPnet.dll"
# Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Sftp
HostName = "server"
UserName = "username"
Password = "password"
SshHostKeyFingerprint = "key"
}
$session = New-Object WinSCP.Session
try
{
# Connect
$session.Open($sessionOptions)
# Upload files
$transferOptions = New-Object WinSCP.TransferOptions
$transferOptions.TransferMode = [WinSCP.TransferMode]::Binary
$transferResult =
$session.PutFiles("E:\CMBPAID", "/NESAMSCARIMED", $False, $transferOptions)
# Throw on any error
$transferResult.Check()
# Print results
foreach ($transfer in $transferResult.Transfers)
{
Write-Host "Upload of $($transfer.FileName) succeeded"
}
}
finally
{
# Disconnect, clean up
$session.Dispose()
}
Use file mask *customer*:
$transferResult =
$session.PutFiles("E:\CMBPAID\*customer*", "/NESAMSCARIMED/", $False, $transferOptions)
(note the slash added to the end of the target path)
Hi All i am new to power shell scripting, i am trying to download all files in root directory and files in sub folder based on file name from FTP server. As i am looking to download files which got matched on files names in like, as my path will always point to root directory and root directory is having text files and sub folders, the sub folders contains text files.
I found an example to get text files to download to local from ftp server, but the example i found is only downloading text files from root directory it is not scanning sub folders and not getting files in sub folders.
Here is the example i tried
# $url = "ftp://XXX.com/"
$user = 'UserName'
$pass = 'Password'
$folder = 'FTP_Folder'
$target = "C:\Folder\Folder1\"
#SET CREDENTIALS
$credentials = new-object System.Net.NetworkCredential($user, $pass)
function Get-FtpDir ($url,$credentials) {
$request = [Net.WebRequest]::Create($url)
$request.Method = [System.Net.WebRequestMethods+FTP]::ListDirectory
if ($credentials) { $request.Credentials = $credentials }
$response = $request.GetResponse()
$reader = New-Object IO.StreamReader $response.GetResponseStream()
$reader.ReadToEnd()
$reader.Close()
$response.Close()
}
#SET FOLDER PATH
$folderPath= $ftp + "/" + $folder + "/"
$Allfiles=Get-FTPDir -url $folderPath -credentials $credentials
$files = ($Allfiles -split "`r`n")
$files
$webclient = New-Object System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass)
$counter = 0
foreach ($file in ($files | where {$_ -like "*.txt"})){
$source=$folderPath + $file
$destination = $target + $file
$webclient.DownloadFile($source, $target+$file)
#PRINT FILE NAME AND COUNTER
$counter++
$counter
$source
}
I tried a module PSFTP module functions like Get-ftpchilditem function with recursive, but it not worked for me.
Please help me in this to get files from root folder and sub folders of FTP server to local system.
I have done it in the past using WinSCP and SFTP. The example is below. This uses SFTP SCP which is basically over SSH. I am sure you can change the WInSCP protocol to ftp and try different combination. Hope this helps you. Change the FileMasks to only include .txt and this will go through sub folders. Also for this to work you will require WinSCPnet.dll and WinSCP.exe which you can easily download from WinSCP site. Download the ".Net Assembly/COM Library" and keep the script and the required files in the same folder and update the path in the script. Good Luck.
# Load WinSCP .NET assembly
Add-Type -Path "C:\Users\administrator.AD\Documents\WinSCP Script\WinSCPnet.dll"
$hostname = "ftp.xyz.net"
$user = "me"
$pass = "mypassword"
$sfolder = "/home/me"
$dfolder = "C:\tmp"
$sshHostKeyFingerprint = "ssh-rsa 2048 eb:2c:f9:ab:19:2e:0c:22:02:d4:8f:64:44:75:ec:04"
$mask = "*.jpg;*.jpeg;*.png;*.tiff;*.txt"
####################### END USER CONFIGURABLE SETTINGS #########################
# Main script
# Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Scp
HostName = $hostname
UserName = $user
Password = $pass
SshHostKeyFingerprint = $sshHostKeyFingerprint
}
$session = New-Object WinSCP.Session
$transferOptions = New-Object WinSCP.TransferOptions
$transferOptions.FileMask = "*.jpg; *.jpeg; *.png; *.tiff;"
try
{
# Will continuously report progress of synchronization
$session.add_FileTransferred( { FileTransferred($_) } )
# Connect
$session.Open($sessionOptions)
# Synchronize files
$synchronizationResult = $session.SynchronizeDirectories(
[WinSCP.SynchronizationMode]::Local,$dfolder, $sfolder, $False, $False, 1, $transferOptions)
# Throw on any error
$synchronizationResult.Check()
}
finally
{
# Disconnect, clean up
$session.Dispose()
}