I am using Transferetto module to download SFTP files, it is able to connect to server and list the server files but throwing error while downloading with below error:-
Receive-SFTPFile : A parameter cannot be found that matches parameter name 'RemoteFile'.
My script is like below:-
Set-ExecutionPolicy RemoteSigned -Scope Process -Force
Import-Module Transferetto
$SftpClient = Connect-SFTP -Server '100.100.00.00' -Username 'FRG_100_22' -Password 'dfggrty'
$List = Get-SFTPList -SftpClient $SftpClient -Path '/FRG'
Get-Childitem -Path "\\data\files" | Remove-Item
$downloadedFiles = #()
# Download multiple files into directory
foreach($originalSFtpFile in $list) {
$downloadedFile = Receive-SFTPFile -SftpClient $SftpClient -RemoteFile $originalSFtpFile -LocalPath "\\data\files" -LocalExists Overwrite -VerifyOptions Retry, None
Write-Host "Downloaded file $($originalSFtpFile.Name) in folder $($downloadedFile.LocalPath)"
$downloadedFiles += ($downloadedFile)
}
Disconnect-SFTP -SftpClient $SftpClient
I used RemotePath instead of RemoteFile and it solved the problem.
Related
I wrote one script that does file downloads from FTP server but now due to change in servers, it should be translated for SFTP.
I checked many post but most of them using .net script.
I need to do it using powershell.
Below is the working script that downloads from FTP, please help me translate it for SFTP server.
SFTP server has hostname,password and username.
Script:-----------
function Dld-FtpFiles {
param
(
$FtpHost,
$FtpUser,
$FtpPassword,
$FtpFolder,
$DownloadDirectory
)
$Client = Connect-FTP -Server $FtpHost -Verbose -Username $FtpUser -Password $FtpPassword
$List = Get-FTPList -Client $Client -Path $FtpFolder
Get-Childitem -Path $DownloadDirectory | Remove-Item
$downloadedFiles = #()
foreach($originalFtpFile in $list) {
$downloadedFile = Receive-FTPFile -Client $Client -RemoteFile $originalFtpFile -LocalPath $DownloadDirectory -LocalExists Overwrite -VerifyOptions Retry, None
Write-Host "Downloaded file $($originalFtpFile.Name) in folder $($downloadedFile.LocalPath)"
$downloadedFiles += ($downloadedFile)
}
Disconnect-FTP -Client $Client
$downloadedFiles = $downloadedFiles | Where-Object {$_}
return $downloadedFiles
}
Set-ExecutionPolicy RemoteSigned -Scope Process -Force
Import-Module Transferetto
$ftpDetails = #(
(New-Object PSOBject -Property #{
FtpHost = 'eu.data.com'
FtpUser = 'FT_34436'
FtpPassword = 'V64445h3'
FtpFolder = '/File'
DownloadDirectory = "\\Srcfile\data"
})
)
foreach($ftpDetail in $ftpDetails) {
$downloadedFiles = Dld-FtpFiles -FtpHost $ftpDetail.FtpHost -FtpUser $ftpDetail.FtpUser -FtpPassword $ftpDetail.FtpPassword -FtpFolder $ftpDetail.FtpFolder -DownloadDirectory $ftpDetail.DownloadDirectory
}
While trying to delete files from ftp server using above script it loops on finding ./././././ directories which don't exist. Is there any error on the script or it's a bug in psftpd? Can anyone help me with the issue? Here is the looping lines copied from powershell console:
VERBOSE: Performing the operation "Remove item from ftp location" on target "ftp:/domain.com/.cagefs".
VERBOSE: Use original path: domain.com/.cagefs
VERBOSE: Remove additonal slash: domain.com/.cagefs
VERBOSE: Path is directory
VERBOSE: Use WebRequestMethods: LIST
VERBOSE: Use original path: domain.com/.cagefs.
VERBOSE: Remove additonal slash: domain.com/.cagefs.
VERBOSE: Path is directory
VERBOSE: Use WebRequestMethods: LIST
VERBOSE: Use original path: domain.com/.cagefs/..
VERBOSE: Remove additonal slash: domain.com/.cagefs/..
Code:
`Import-Module PSFTP`
`$FTPServer = 'domain.com'`
`$FTPUsername = 'username'`
`$FTPPassword = 'password'`
`$FTPSecurePassword = ConvertTo-SecureString -String $FTPPassword -asPlainText -Force`
`$FTPCredential = New-Object System.Management.Automation.PSCredential($FTPUsername,$FTPSecurePassword)`
`Set-FTPConnection -Credentials $FTPCredential -Server $FTPServer -Session MySession -UsePassive`
`$Session = Get-FTPConnection -Session MySession`
`$output = Get-FTPChildItem -Session $Session -Path '/' | Select ModifiedDate, Name | Where-Object {($_.ModifiedDate -lt (Get-Date).AddDays(-7))}|select name `
`echo $output`
foreach ( $file in $output ) {
Remove-FTPItem $file.name -Recurse -Session $Session -Verbose
}
I want to update 2 workspaces from two different tfs in one script using powershell.
The first Workspace is updating without any Problems. After the update is finished powershell connects to the second Workspace, but isn't updating the local data like the first time.
I guess the old Connection might still block the pipe or something like that, but I haven't found any cmd to clean the pipe. My code looks like this:
param(
[string]$TestTFS = "http://TestTFS",
[string]$ProdTFS = "http://ProdTFS",
[string]$Teamproject="$\TeamprojectPath",
[string]$LocalTestWorkspace="C:\LocalTestWorkspacePath",
[string]$LocalProdWorkspace="C:\LocalProdWorkspacePath"
)
# Import Microsoft.TeamFoundation.PowerShell Snapin
Add-PSSnapin Microsoft.TeamFoundation.PowerShell
# Connect to production-TFS
$ProdEnvServer = Get-TfsServer -Name $ProdTFS
Write-Host "tfsConnect ="$ProdEnvServer
# Get prod teamprojekt
Get-TfsChildItem $Teamprojekt -Server $ProdEnvServer
# Update files in local prod workspace
Update-TfsWorkspace -Force -Recurse $LocalProdWorkspace
# Connect to test-TFS
$TestEnvServer = Get-TfsServer -Name $TestTFS
Write-Host "tfsConnect ="$TestEnvServer
# Get test teamprojekt
Get-TfsChildItem $Teamprojekt -Server $TestEnvServer
# Update files in local test workspace
Update-TfsWorkspace -Force -Recurse $LocalTestWorkspace
After 3 months and noone coming up with an answer. I just assume that the Cmdlets don't work as they should. The only option here seems to be a Workaround.
# Copy Team Project from Prod to Test TFS
param([string]$TestTFS = "http://TestTFS",
[string]$ProdTFS = "http://ProdTFS",
[String]$Teamproject="$/Teamproject",
[String]$LocalTestWorkspace="C:\LocalTestWorkspacePath",
[String]$LocalProdWorkspace="C:\LocalProdWorkspacePath")
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client")
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.VersionControl.Client")
try
{
clear
$LocalTestProjectPath = $LocalTestWorkspace + $Teamproject.Substring(1)
$LocalProdProjectPath = $LocalProdWorkspace + $Teamproject.Substring(1)
# Connect to production-TFS
Write-Host "Getting latest of $ProdTFS"
$tfsColProd = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($ProdTFS)
[Microsoft.TeamFoundation.VersionControl.Client.VersionControlServer] $vcsProd = $tfsColProd.GetService([type] "Microsoft.TeamFoundation.VersionControl.Client.VersionControlServer")
# TryGetWorkspace is sometimes buggy and doesn't return an existing workspace
# Delete existing workspace manually before if that happens
$workspaceProd = $vcsProd.TryGetWorkspace($LocalProdWorkspace)
$isProdTempWorkspace = $false
# create Workspace if it doesn't exists
if (-not $workspaceProd) {
Write-Host "No workspace found, creating temporary for prod"
$workspaceProd = $vcsProd.CreateWorkspace("Temp_" + [System.Guid]::NewGuid().ToString())
$workspaceProd.Map($Teamproject, $LocalProdProjectPath)
$isProdTempWorkspace = $true
}
$itemSpecFullTeamProj = New-Object Microsoft.TeamFoundation.VersionControl.Client.ItemSpec($Teamproject, "Full")
$fileRequest = New-Object Microsoft.TeamFoundation.VersionControl.Client.GetRequest($itemSpecFullTeamProj,
[Microsoft.TeamFoundation.VersionControl.Client.VersionSpec]::Latest)
$workspaceProd.Get($fileRequest, [Microsoft.TeamFoundation.VersionControl.Client.GetOptions]::GetAll)
if ($isProdTempWorkspace) {
Write-Host "Deleting temporary workspace for prod"
$workspaceProd.Delete()
}
Write-Host "Getting latest of $TestTFS"
$tfsColTest = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($TestTFS)
$vcsTest = $tfsColTest.GetService([type] "Microsoft.TeamFoundation.VersionControl.Client.VersionControlServer")
# TryGetWorkspace is sometimes buggy and doesn't return an existing workspace
# Delete existing workspace manually before if that happens
[Microsoft.TeamFoundation.VersionControl.Client.Workspace] $workspaceTest = $vcsTest.TryGetWorkspace($LocalTestWorkspace)
$isTestTempWorkspace = $false
# create Workspace if it doesn't exists
if (-not $workspaceTest) {
Write-Host "No workspace found, creating temporary for test"
$workspaceTest = $vcsTest.CreateWorkspace("Temp_" + [System.Guid]::NewGuid().ToString())
$workspaceTest.Map($Teamproject, $LocalTestProjectPath)
$isTestTempWorkspace = $true
}
$workspaceTest.Get($fileRequest, [Microsoft.TeamFoundation.VersionControl.Client.GetOptions]::GetAll)
# Remove local test folder and copy prod folder into test workspace
Write-Host "Copying over Prod to Test"
# Delete updated test project folder
Remove-Item -Path $LocalTestProjectPath -Force -Recurse
# Copy prod folder to test workspace
Copy-Item -Path $LocalProdProjectPath -Destination $LocalTestProjectPath -Force -Recurse
# Calling tfpt is the only thing that works
Write-Host "Comparing for changes"
$ps = new-object System.Diagnostics.Process
$ps.StartInfo.Filename = $env:TFSPowerToolDir + "tfpt.exe"
$ps.StartInfo.Arguments = "online /adds /deletes /diff /noprompt /recursive $LocalTestProjectPath"
$ps.StartInfo.RedirectStandardOutput = $false # careful, only output works, has hanging problems (2k Buffer limit)
$ps.StartInfo.RedirectStandardError = $false
$ps.StartInfo.UseShellExecute = $false
$ps.Start()
$ps.WaitForExit()
# Check in new test project folder into test environment
$wsCheckinParams = New-Object Microsoft.TeamFoundation.VersionControl.Client.WorkspaceCheckInParameters(
#($itemSpecFullTeamProj),"Update project to production environment version")
# CheckIn better manually to check for errors
$workspaceTest.CheckIn($wsCheckinParams)
if ($isTestTempWorkspace) {
Write-Host "Deleting temporary workspace for test"
$workspaceTest.Delete()
Remove-Item -Path D:\Development -Force -Recurse
}
}
catch [System.Exception]
{
Write-Host "Exception: " ($Error[0]).Exception
EXIT $LASTEXITCODE
}
My approach is very similar to Zittelrittel. Just send the path and it will automatically figure out the workspace.
This will not work in PowerShell ISE (x86), I had to use the 64-bit version!
Add-PSSnapin Microsoft.TeamFoundation.PowerShell
Write-Host "Updating Workspace1, please wait..."
Update-TfsWorkspace -item C:\dev\Workspace1\code -Recurse | Format-Table
Write-Host "Updating Workspace2, please wait..."
Update-TfsWorkspace -item C:\dev\Workspace1\code -Recurse | Format-Table
In your calls to update TFS workspace, pipe the result to out-null. This should effectively remove any data that would otherwise be stored in the pipeline.
Update-TfsWorkspace -Force -Recurse $LocalProdWorkspace | Out-Null
Update-TfsWorkspace -Force -Recurse $LocalTestWorkspace | Out-Null
I have the powershell script built and I'm getting a "Random" bit of output into the CSV file. The string is MailboxExport(and a number). It looks like a value that (Get-MailboxExportRequest).name would return but I can't see where I would pull something like that or how it is being inserted. I think I may have just been staring at it too long and I may just need a fresh pair of eyes to spot my mistake. I would go into what the script is trying to do but I've put quite a few notes in the script that should explain it fairly well.
################################################## PST Extraction Script ##################################################
# Completed October 2013 by Trey Nuckolls
#
# This script is meant to extract PST files from the Site 1 Exchange server at the Site2 site and deliver those PST
# files to a share on the Site2 network. The script will change the input CSV file to keep track of which PSTfiles have been
# extracted and when that occoured. The script will also set security on the PST file so only the user and IT administraion
# can access the PST file.
#
# To run this script, enter the username of the Site 1 domain account that you want to target for extraction of a PST file then
# Run the script. Can be run from any machine on the network as long as it is run by someone with domain admin rights on the
# Site 2 network. Powershell v2 or v3 is required to run the script.
#
#############################################################################################################################
$InPstPath = '\\Site1_Server\PST_Store'
$OutPstPath = '\\Site2_Server\PST_Store'
$AdminPath = '\\Site2_Server\PST_Store\Admin\'
#Container for Site1 username
$User = Get-Content $AdminPath'login.txt'
#Container for encrypted Site1 Password
$PWord = Cat $AdminPath'pass.txt' | ConvertTo-SecureString
#Credential package for accessing Site1 resouces
$Credentials = New-Object –TypeName System.Management.Automation.PSCredential –ArgumentList $User, $PWord
#Creation of Powershell Drives for use during session
New-PSDrive -Name Site1Share -PSProvider FileSystem -Root $InPstPath -Credential $Credentials
New-PSDrive -Name Site2Share -PSProvider FileSystem -Root $OutPstPath
#Container for Powershell session to Exchange server
$PSSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri http://Site1_Server/powershell -Credential $Credentials
#Creation of Powershell session to Site1 Exchange server, including import of exchange commandlets
Import-PSSession $PSSession
#Import of the CSV file that lists users to be targeted
$In_List = Invoke-Command {Import-Csv "\\Site1_Server\PST_Store\To_Be_Exported.csv"} -computername Site1_Server -Credential $Credentials
$Processed = foreach ($objUser in $In_List) {
if ($objUser.Completed -ne "Yes") {
$TargetUser = $objUser.name
$ShortDate = (Get-Date).toshortdatestring()
$SourceFile = "Site1Share:\$TargetUser.pst"
$DestinationFile = "Site2Share:\$TargetUser.pst"
#Export Mailbox to PST File
New-MailboxExportRequest -Mailbox $TargetUser -Filepath $InPstPath\$TargetUser.pst
do {Start-Sleep -Seconds 10}
until((Get-MailboxExportRequest -Status InProgress).count -eq 0)
#Copy PST File to PST Share
Copy-Item -Path $SourceFile -Destination $DestinationFile
#Add Security access on PST file (Target_User-Modify). Domain Admin-Full is inherited from parent.
$Acl = Get-Acl $DestinationFile
$Permission = "Site2_Domain\$TargetUser","Modify","Allow"
$AccessRule = New-Object System.Security.AccessControl.FileSystemAccessRule $Permission
$Acl.SetAccessRule($AccessRule)
$Acl | Set-Acl $DestinationFile
#Remove PST file From Temporary area
Remove-Item -Path $SourceFile -Force
#Write back to checklist for new items that have just been processed
[PSCustomObject]#{Name=$TargetUser;Completed="Yes";Date=$ShortDate}
} else { if ($objUser.Completed -eq "Yes") {
#Passthrough of items that have already been completed
[PSCustomObject]#{Name=$objUser.name;Completed=$objUser.Completed;Date=$objUser.Date}}
}}
#Output the new version of the checklist
$Processed | export-csv -Path C:\TEMP\processed.csv
#Overwrite the old version checklist with the new one
Move-Item -Path C:\TEMP\processed.csv -Destination Site1Share:\To_Be_Exported.csv -force
#Cleanup PsDrives and PsSessions
Remove-PSDrive -Name Site1Share
Remove-PSDrive -Name Site2Share
Remove-PSSession -Session (Get-PSSession)
Input CSV is...
"Name","Completed","Date"
"User1","Yes","10/8/2013"
"User2","Yes","10/11/2013"
"User3",,
and output is...
"Name","Completed","Date"
"User1","Yes","10/8/2013"
"User2","Yes","10/11/2013"
"MailboxExport7",,
"User3","Yes","10/11/2013"
It is indeed very likely that the issue is caused by New-MailboxExportRequest, as you already suspected. The cmdlet prints information about the created object, which lumped together with the rest of the output you create in the loop, and then assigned to the variable $Processed.
To avoid this you can suppress the cmdlet output like this:
New-MailboxExportRequest -Mailbox ... | Out-Null
or like this:
New-MailboxExportRequest -Mailbox ... >$null
Assigning the output to a variable should work as well:
$exportRequest = New-MailboxExportRequest -Mailbox ...
On you Export-CSV, try adding the flag: "-NoTypeInformation"
I think this may be some sort of name space crossover issue between the custom object and another existing object (probably the mailboxexportrequest object on the exchange server). After messing around with this for a while I was able to get it to fail in a new way where the resultant csv file was full of details from the mailbox exports and their was a 'name' column that also had listed the usernames. I changed the hashes on the input csv from 'name to 'username' and the resultant MailboxExport entries have ceased. There are now blank row but I'm certainly willing to live with that imperfection as it doesn't break this (short lived) process.
If anyone has any insight into the root cause I'd certainly love to hear what it is but I think I've figured out a solution to the point that I can live with.
I just burned a couple of hours searching for a solution to send files over an active PSSession. And the result is nada, niente. I'm trying to invoke a command on a remote computer over an active session, which should copy something from a network storage. So, basically this is it:
icm -Session $s {
Copy-Item $networkLocation $PCLocation }
Because of the "second hop" problem, I can't do that directly, and because I'm running win server 2003 I cant enable CredSSP. I could first copy the files to my computer and then send/push them to the remote machine, but how? I tried PModem, but as I saw it can only pull data and not push.
Any help is appreaciated.
This is now possible in PowerShell / WMF 5.0
Copy-Item has -FromSession and -toSession parameters. You can use one of these and pass in a session variable.
eg.
$cs = New-PSSession -ComputerName 169.254.44.14 -Credential (Get-Credential) -Name SQL
Copy-Item Northwind.* -Destination "C:\Program Files\Microsoft SQL Server\MSSQL10_50.SQL2008R2\MSSQL\DATA\" -ToSession $cs
See more examples at here, or you can checkout the official documentation.
If it was a small file, you could send the contents of the file and the filename as parameters.
$f="the filename"
$c=Get-Content $f
invoke-command -session $s -script {param($filename,$contents) `
set-content -path $filename -value $contents} -argumentlist $f,$c
If the file is too long to fit in whatever the limits for the session are, you could read the file in as chunks, and use a similar technique to append them together in the target location
PowerShell 5+ has built-in support for doing this, described in David's answer.
I faced the same problem a while ago and put together a proof-of-concept for sending files over a PS Remoting session. You'll find the script here:
https://gist.github.com/791112
#requires -version 2.0
[CmdletBinding()]
param (
[Parameter(Mandatory=$true)]
[string]
$ComputerName,
[Parameter(Mandatory=$true)]
[string]
$Path,
[Parameter(Mandatory=$true)]
[string]
$Destination,
[int]
$TransferChunkSize = 0x10000
)
function Initialize-TempScript ($Path) {
"<# DATA" | Set-Content -Path $Path
}
function Complete-Chunk () {
#"
DATA #>
`$TransferPath = `$Env:TEMP | Join-Path -ChildPath '$TransferId'
`$InData = `$false
`$WriteStream = [IO.File]::OpenWrite(`$TransferPath)
try {
`$WriteStream.Seek(0, 'End') | Out-Null
`$MyInvocation.MyCommand.Definition -split "``n" | ForEach-Object {
if (`$InData) {
`$InData = -not `$_.StartsWith('DATA #>')
if (`$InData) {
`$WriteBuffer = [Convert]::FromBase64String(`$_)
`$WriteStream.Write(`$WriteBuffer, 0, `$WriteBuffer.Length)
}
} else {
`$InData = `$_.StartsWith('<# DATA')
}
}
} finally {
`$WriteStream.Close()
}
"#
}
function Complete-FinalChunk ($Destination) {
#"
`$TransferPath | Move-Item -Destination '$Destination' -Force
"#
}
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version Latest
$EncodingChunkSize = 57 * 100
if ($EncodingChunkSize % 57 -ne 0) {
throw "EncodingChunkSize must be a multiple of 57"
}
$TransferId = [Guid]::NewGuid().ToString()
$Path = ($Path | Resolve-Path).ProviderPath
$ReadBuffer = New-Object -TypeName byte[] -ArgumentList $EncodingChunkSize
$TempPath = ([IO.Path]::GetTempFileName() | % { $_ | Move-Item -Destination "$_.ps1" -PassThru}).FullName
$Session = New-PSSession -ComputerName $ComputerName
$ReadStream = [IO.File]::OpenRead($Path)
$ChunkCount = 0
Initialize-TempScript -Path $TempPath
try {
do {
$ReadCount = $ReadStream.Read($ReadBuffer, 0, $EncodingChunkSize)
if ($ReadCount -gt 0) {
[Convert]::ToBase64String($ReadBuffer, 0, $ReadCount, 'InsertLineBreaks') |
Add-Content -Path $TempPath
}
$ChunkCount += $ReadCount
if ($ChunkCount -ge $TransferChunkSize -or $ReadCount -eq 0) {
# send
Write-Verbose "Sending chunk $TransferIndex"
Complete-Chunk | Add-Content -Path $TempPath
if ($ReadCount -eq 0) {
Complete-FinalChunk -Destination $Destination | Add-Content -Path $TempPath
Write-Verbose "Sending final chunk"
}
Invoke-Command -Session $Session -FilePath $TempPath
# reset
$ChunkCount = 0
Initialize-TempScript -Path $TempPath
}
} while ($ReadCount -gt 0)
} finally {
if ($ReadStream) { $ReadStream.Close() }
$Session | Remove-PSSession
$TempPath | Remove-Item
}
Some minor changes would allow it to accept a session as a parameter instead of it starting a new one. I found the memory consumption on the Remoting service on the destination computer could grow quite large when transferring large files. I suspect PS Remoting wasn't really designed to be used this way.
NET USE allows you to add a local drive letter for the remote system, which then then allows you to use the drive letter in your PSSession, or even without a PSSession. This is helpful if you don't have Powershell v5.0, and even if you do,
You may use the remote machine name or its IP address as part of the remote UNC path and you can specify the username and password credentials on the same line:
NET USE Z: \\192.168.1.50\ShareName /USER:192.168.1.50\UserName UserPassword
Another example:
NET USE Z: \\RemoteSystem\ShareName /USER:RemoteSystem\UserName UserPassword
OR
NET USE Z: \\RemoteSystem\ShareName /USER:Domain\UserName UserPassword
If you don't supply the user credentials on the same line, you will be prompted for them:
>NET USE Z: \\192.168.1.50\ShareName
Enter the user name for '192.168.1.50': 192.168.1.50\UserName
Enter the password for 192.168.1.50: *****
The command completed successfully.
You may remove the drive letter when you're finished with the following:
NET USE Z: /delete
You can get the full syntax with NET USE /?
>net use /?
The syntax of this command is:
NET USE
[devicename | *] [\\computername\sharename[\volume] [password | *]]
[/USER:[domainname\]username]
[/USER:[dotted domain name\]username]
[/USER:[username#dotted domain name]
[/SMARTCARD]
[/SAVECRED]
[[/DELETE] | [/PERSISTENT:{YES | NO}]]
NET USE {devicename | *} [password | *] /HOME
NET USE [/PERSISTENT:{YES | NO}]
NET is a standard external .exe command in the system folder and works in Powershell just fine.
$data = Get-Content 'C:\file.exe' -Raw
Invoke-Command -ComputerName 'server' -ScriptBlock { $using:data | Set-Content -Path 'D:\filecopy.exe' }
Don't actually know what the maximum file size limitation is.