Download files over PSSession for PS<5 - powershell

How do I download files from a remote server over a PSSession? I'm aware that PS5 introduced Copy-Item -FromSession, but both local and remote may be not running PS5. My files are also quite large so a simple Get-Content may be problematic.

You can read file on remote side as sequence of byte arrays, stream them thru remoting, and then assemble them back to file locally:
function DownloadSingleFile {
param(
[System.Management.Automation.Runspaces.PSSession] $FromSession,
[string] $RemoteFile,
[string] $LocalFile,
[int] $ChunkSize = 1mb
)
Invoke-Command -Session $FromSession -ScriptBlock {
param(
[string] $FileName,
[int] $ChunkSize
)
$FileInfo = Get-Item -LiteralPath $FileName
$FileStream = $FileInfo.OpenRead()
try {
$FileReader = New-Object System.IO.BinaryReader $FileStream
try {
for() {
$Chunk = $FileReader.ReadBytes($ChunkSize)
if($Chunk.Length) {
,$Chunk
} else {
break;
}
}
} finally {
$FileReader.Close();
}
} finally {
$FileStream.Close();
}
} -ArgumentList $RemoteFile, $ChunkSize | ForEach-Object {
$FileName = $ExecutionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath($LocalFile)
$FileStream = [System.IO.File]::Create($FileName)
} {
$FileStream.Write($_, 0, $_.Length)
} {
$FileStream.Close()
}
}

Related

How to programatically set Shortcuts TargetPath to a website?

I want to use powershell to modify the TargetPath of a shortcut to open a website
I found the below script that almost works
function Set-Shortcut {
param(
[Parameter(ValueFromPipelineByPropertyName=$true)]
$LinkPath,
$Hotkey,
$IconLocation,
$Arguments,
$TargetPath
)
begin {
$shell = New-Object -ComObject WScript.Shell
}
process {
$link = $shell.CreateShortcut($LinkPath)
$PSCmdlet.MyInvocation.BoundParameters.GetEnumerator() |
Where-Object { $_.key -ne 'LinkPath' } |
ForEach-Object { $link.$($_.key) = $_.value }
$link.Save()
}
}
However
Set-Shortcut -LinkPath "C:\Users\user\Desktop\test.lnk" -TargetPath powershell start process "www.youtube.com"
Will default out to attaching a default path if you do not define one to look like:
"C:\Users\micha\Desktop\powershell start process "www.youtube.com""
how do I get rid of that default file path?
BONUS:
I'd be appreciative if someone broke down this line of code:
ForEach-Object { $link.$($_.key) = $_.value }
have you tried to change the properties of the shortcut object directly?
Try this and let me know:
function Set-Shortcut {
[CmdletBinding()]
param (
[Parameter(Mandatory, Position = 0, ValueFromPipeline)]
[System.String]$FilePath,
[Parameter(Mandatory, Position = 1)]
[System.String]$TargetPath
)
Begin {
$shell = new-object -ComObject WScript.Shell
}
Process {
try {
$file = Get-ChildItem -Path $FilePath -ErrorAction Stop
$shortcut = $shell.CreateShortcut($file.FullName)
$shortcut.TargetPath = $TargetPath
$shortcut.Save()
}
catch {
throw $PSItem
}
}
End {
while ($result -ne -1) {
$result = [System.Runtime.InteropServices.Marshal]::ReleaseComObject($shell)
}
}

Powershell logging to lost network location: FileStream won't reconnect/flush

I am working on a script logging solution for an install script that has many tasks and long processing times, and I am trying to address the issue of network dropout. I am also moving to a StreamWriter based approach vs my old Add-Content approach for performance reasons.
The problem I am having is that once the network drops out, the StreamWriter doesn't reconnect. So, first question is, CAN I reconnect, or is this a limitation of StreamWriter? The fact that the StreamWriter has a cache that can be Flushed makes me think I may be doing a bunch of work to recreate functionality that is already there.
And second, I am starting to think a simpler/better solution is simply to write the log to a local folder, so the log is always complete, then simply attempt to copy that to the network location for progress review. I had been thinking about implementing parallel logs, so a loss of the network would still leave the local copy complete. Curious if anyone else looks at this and says "Well, YEAH, dufus, obviously."
function Get-PxLogFile {
return $script:pxLogFile
}
function Set-PxLogFile {
param (
[string]$path
)
if (Test-Path $path) {
Remove-Item $path -force
}
[string]$script:pxLogFile= $path
}
function Get-PxLogWriter {
$logFile = Get-PxLogFile
if (-not $script:pxFileStream) {
$script:pxFileStream = New-Object io.fileStream $logFile, 'Append', 'Write', 'Read'
$script:pxlogWriter = New-Object io.streamWriter $script:pxFileStream
} elseif ($script:pxFileStream.name -ne $logFile) {
$script:pxFileStream = New-Object io.fileStream $logFile, 'Append', 'Write', 'Read'
$script:pxlogWriter = New-Object io.streamWriter $script:pxFileStream
}
return $script:pxlogWriter
}
function Dispose-PxLogFile {
$script:pxlogWriter.Dispose()
$script:pxFileStream.Dispose()
$script:pxFileStream = $null
$script:pxlogWriter = $null
}
# Shared
function Get-PxDeferredLog {
return ,$script:deferredLog # , keeps PS from unrolling a single item array into a string
}
function Set-PxDeferredLog {
param (
[collections.arrayList]$deferredLog
)
[collections.arrayList]$script:deferredLog = $deferredLog
}
function Get-PxDeferredLogTimes {
return ($script:deferredLogTimes -join ', ')
}
function Finalize-PxLogFile {
$logWriter = Get-PxLogWriter
if (($deferredLog = Get-PxDeferredLog).count -gt 0) {
$abandonTime = (Get-Date) + (New-TimeSpan -seconds:3)
$logWriter = Get-PxLogWriter
:lastChanceLogWindow do {
$deferredLog = Get-PxDeferredLog
:deferredItemsWrite do {
try {
$logWriter.WriteLine($deferredLog[0])
if ($deferredLog.count -gt 0) {
$deferredLog.RemoveAt(0)
} else {
break :lastChanceWindow
}
} catch {
break :deferredItemsWrite
}
} while ($deferredLog.count -gt 0)
Set-PxDeferredLog $deferredLog
if ($deferredLog.count -eq 0) {
break :lastChanceWindow
}
if ((Get-Date) -gt $abandonTime) {
break :lastChanceLogWindow
}
} while ((Get-Date) -lt $abandonTime)
if ($deferredLog) {
Write-Host "Failed to write all log entries"
}
}
$script:deferredLogItems = $script:deferredLogTimes = $null
}
function Add-PxLogFileContent {
param (
[string]$string
)
$logWriter = Get-PxLogWriter
# Nested Functions
function Add-PxDeferredLogItem {
param (
[string]$item
)
if (-not $script:deferredLog) {
[collections.arrayList]$script:deferredLog = New-Object collections.arrayList
}
if ($script:deferredLog.count -eq 0) {
Start-PxDeferredLogTime
}
[void]$script:deferredLog.Add($item)
}
function Start-PxDeferredLogTime {
if (-not $script:deferredLogTimes) {
[collections.arrayList]$script:deferredLogTimes = New-Object collections.arrayList
}
if ((-not $script:deferredLogTimes) -or (-not $script:deferredLogTimes[-1].EndsWith('-'))) {
[void]$script:deferredLogTimes.Add("$((Get-Date).ToString('T'))-")
}
}
function Stop-PxDeferredLogTime {
if ($script:deferredLogTimes[-1].EndsWith('-')) {
$script:deferredLogTimes[-1] = "$($script:deferredLogTimes[-1])$((Get-Date).ToString('T'))"
}
}
$deferredLogProcessed = $false
if ([collections.arrayList]$deferredLog = Get-PxDeferredLog) {
$deferredLogProcessed = $true
:deferredItemsWrite do {
try {
$logWriter.WriteLine($deferredLog[0])
$logWriter.Flush()
$deferredLog.RemoveAt(0)
} catch {
break :deferredItemsWrite
}
} while ($deferredLog.count -gt 0)
if ($deferredLog.count -eq 0) {
$deferredLogPending = $false
} else {
$deferredLogPending = $true
}
Set-PxDeferredLog $deferredLog
} else {
$deferredLogPending = $false
}
if (-not $deferredLogPending) {
try {
if ($logWriter.WriteLine($string)) {
$logWriter.Flush()
}
if ($deferredLogProcessed) {Stop-PxDeferredLogTime}
} catch {
Add-PxDeferredLogItem $string
Write-Host "Failed: $(Get-Date)`n$($_.Exception.Message)"
}
} else {
Add-PxDeferredLogItem $string
}
}
### MAIN
Clear-Host
$script:deferredLogItems = $script:deferredLogTimes = $null
$logPath = '\\px\Content'
Write-Host 'logTest1.txt'
$startTime = Get-Date
$endTime = $startTime + (New-TimeSpan -minutes:5)
#Set-PxLogFile "$([System.IO.Path]::GetFullPath($env:TEMP))\logTest1.txt"
Set-PxLogFile "$logPath\logTest1.txt"
do {
Add-PxLogFileContent "logged: $(Get-Date)"
Start-SLeep -s:10
} while ((Get-Date) -lt $endTime)
Finalize-PxLogFile
Write-Host 'logTest2.txt'
$startTime = Get-Date
$endTime = $startTime + (New-TimeSpan -minutes:5)
#Set-PxLogFile "$([System.IO.Path]::GetFullPath($env:TEMP))\logTest2.txt"
Set-PxLogFile "$logPath\logTest2.txt"
do {
Add-PxLogFileContent "logged: $(Get-Date)"
Start-SLeep -s:10
} while ((Get-Date) -lt $endTime)
if ([string]$deferredLogTimes = Get-PxDeferredLogTimes) {
Add-PxLogFileContent "Deferred logging time ranges: $deferredLogTimes"
}
Finalize-PxLogFile
Dispose-PxLogFile

Powershell File Lock function is not opening excel File

I've written a function to check if an excel file is being used/locked by another process/user in a shared network drive, and if used, to pause the script and keep checking till it's available as the next action is to move it out of it's folder. However when I'm using System.IO to read the file, it does not open the file. I've tested on my local drive and this does open the file, but does this not work in Network Drives?
$IsLocked = $True
Function Test-IsFileLocked {
[cmdletbinding()]
Param (
[parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True)]
[Alias('FullName','PSPath')]
[string[]]$Path
)
Process {
while($isLocked -eq $True){
If ([System.IO.File]::Exists($Path)) {
Try {
$FileStream = [System.IO.File]::Open($Path,'Open','Write')
$FileStream.Close()
$FileStream.Dispose()
$IsLocked = $False
} Catch {
$IsLocked = $True
echo "file in use, trying again in 10 secs.."
Start-Sleep -s 10
}
}
}
}
}
This is where the code does not pick up/open the excel file in my function
$FileStream = [System.IO.File]::Open($Path,'Open','Write')
This is where the program calls the function.Loop through a folder of items in the network drive and if the item matches the pattern, then the function will be called to check if the file is in use:
$DirectoryWithExcelFile = Get-ChildItem -Path "Z:\NetworkDriveFolder\"
$DestinationFolder = "Z:\DestinationFolder\"
$pattern = "abc"
foreach($file in $DirectoryWithExcelFile){
if($file.Name -match $pattern){
Test-IsFileLocked -Path $file
$destFolder = $DestinationFolder+$file.Name
Move-item $file.FullName -destination $destFolder
break
}
}
You have to place the close and dispose in the finally part of the try so if it throws an exception it disposes of the lock. There's no guarantee it's a file lock exception either so you are better to throw the exception, catch the exception you're expecting or write it out
$IsLocked = $True
Function Test-IsFileLocked {
[cmdletbinding()]
Param (
[parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True)]
[Alias('FullName','PSPath')]
[string]$Path
)
Process {
while($isLocked -eq $True){
If ([System.IO.File]::Exists($Path)) {
Try {
$FileStream = [System.IO.File]::Open($Path,'Open','Write')
$IsLocked = $False
} Catch [System.IO.IOException] {
$IsLocked = $True
echo "file in use, trying again in 10 secs.."
Start-Sleep -s 10
} Catch {
# source https://stackoverflow.com/questions/38419325/catching-full-exception-message
$formatstring = "{0} : {1}`n{2}`n" +
" + CategoryInfo : {3}`n" +
" + FullyQualifiedErrorId : {4}`n"
$fields = $_.InvocationInfo.MyCommand.Name,
$_.ErrorDetails.Message,
$_.InvocationInfo.PositionMessage,
$_.CategoryInfo.ToString(),
$_.FullyQualifiedErrorId
$formatstring -f $fields
write-output $formatstring
} finally {
if($FileStream) {
$FileStream.Close()
$FileStream.Dispose()
}
}
}
}
}
}
Edit:
Path should be a string, not a string array unless you have multiple paths in which case rename to $Paths and loop through them $Paths| % { $Path = $_; #do stuff here }

Unable to gather error info from halting Powershell script

I'm writing a script that will watch a directory for any new mp4 files, then convert the file using HandBrake's CLI tool. The logic that watches the directory for changes works by itself, but if I drop a large video into the "watched" directory the conversion fails since it kicks off as soon as it sees a new file, before the file has time to finish copying.
I'm using a do until loop to check if the file is locked/downloading and then continues once the file is unlocked/writable. The loop works as a stand-alone script, but when used inside the filesystem watcher the script it will halt without any errors on this line:
[System.IO.FileStream] $fs = $convertingFile.OpenWrite();
This occurs regardless if $ErrorActionPreference = "SilentlyContinue" is commented out. I've been unable to gather any log output to see why the script is halting using Start-Transcript or Out-File.
How should I best gather error info about why the script halts once it hits this line?
Bonus: Why might this script not provide error information?
$ErrorActionPreference = "SilentlyContinue"
function Start-FileSystemWatcher {
[CmdletBinding()]
param(
[Parameter()]
[string]$Path,
[Parameter()]
[ValidateSet('Changed','Created','Deleted','Renamed')]
[string[]]$EventName,
[Parameter()]
[string]$Filter,
[Parameter()]
[System.IO.NotifyFilters]$NotifyFilter,
[Parameter()]
[switch]$Recurse,
[Parameter()]
[scriptblock]$Action
)
#region Build FileSystemWatcher
$FileSystemWatcher = New-Object System.IO.FileSystemWatcher
if (-not $PSBoundParameters.ContainsKey('Path')) {
$Path = $PWD
}
$FileSystemWatcher.Path = $Path
if ($PSBoundParameters.ContainsKey('Filter')) {
$FileSystemWatcher.Filter = $Filter
}
if ($PSBoundParameters.ContainsKey('NotifyFilter')) {
$FileSystemWatcher.NotifyFilter = $NotifyFilter
}
if ($PSBoundParameters.ContainsKey('Recurse')) {
$FileSystemWatcher.IncludeSubdirectories = $True
}
if (-not $PSBoundParameters.ContainsKey('EventName')) {
$EventName = 'Changed','Created','Deleted','Renamed'
}
if (-not $PSBoundParameters.ContainsKey('Action')) {
$Action = {
switch ($Event.SourceEventArgs.ChangeType) {
'Renamed' {
$Object = "{0} was {1} to {2} at {3}" -f $Event.SourceArgs[-1].OldFullPath,
$Event.SourceEventArgs.ChangeType,
$Event.SourceArgs[-1].FullPath,
$Event.TimeGenerated
}
Default {
$Object = "{0} was {1} at {2}" -f $Event.SourceEventArgs.FullPath,
$Event.SourceEventArgs.ChangeType,
$Event.TimeGenerated
}
}
$WriteHostParams = #{
ForegroundColor = 'Green'
BackgroundColor = 'Black'
Object = $Object
}
Write-Host #WriteHostParams
}
}
$ObjectEventParams = #{
InputObject = $FileSystemWatcher
Action = $Action
}
foreach ($Item in $EventName) {
$ObjectEventParams.EventName = $Item
$ObjectEventParams.SourceIdentifier = "File.$($Item)"
Write-Verbose "Starting watcher for Event: $($Item)"
$Null = Register-ObjectEvent #ObjectEventParams
}
}
$FileSystemWatcherParams = #{
Path = 'X:\share\scripts\ps\converter\input'
Recurse = $True
NotifyFilter = 'FileName'
Verbose = $True
Action = {
$Item = Get-Item $Event.SourceEventArgs.FullPath
$WriteHostParams = #{
ForegroundColor = 'Green'
BackgroundColor = 'Black'
}
$inputFile = "${PWD}\input\$($Item.Name)".trim()
$outputFile = "${PWD}\output\$($Item.Name)".trim()
$logFile = "${PWD}\log\$($Item.Name).txt"
$testLogFile = "${PWD}\log\$($Item.Name)(t).txt"
function mp4-Func {
Start-Transcript -path $logFile
Write-Host "New mp4 file detected..."
$convertingFile = New-Object -TypeName System.IO.FileInfo -ArgumentList $inputFile
$locked = 1
do {
[System.IO.FileStream] $fs = $convertingFile.OpenWrite();
if (!$?) {
Write-Host "Can't convert yet, file appears to be loading..."
sleep 2
}
else {
$fs.Dispose()
$locked = 0
}
} until ($locked -eq 0)
Write-Host "File unlocked and ready for conversion."
HandBrake
Stop-Transcript
$WriteHostParams.Object = "Finished converting: $($Item.Name)"
}
function HandBrake {
.\HandBrakeCLI.exe --input "$inputFile" `
--output "$outputFile" `
--format av_mp4 `
--encoder x264 `
--vb 1700 `
--two-pass `
--aencoder copy:aac `
--ab 320 `
--arate 48 `
--mixdown stereo
}
switch -regex ($Item.Extension) {
'\.mp4' { mp4-Func }
}
Write-Host #WriteHostParams
}
}
#( 'Created') | ForEach-Object {
$FileSystemWatcherParams.EventName = $_
Start-FileSystemWatcher #FileSystemWatcherParams
}
I think you'll find that $ErrorActionPreference only impact errors at the level of cmdlets, while you hitting a problem in non-cmdlet code. For that, you'll probably need a try/catch construct.
Based on Burt_Harris' answer (please upvote his answer over this one), I changed the "do while" loop so that it would use try/catch rather than an if/else statement. By using $.Exception.Message and $.Exception.ItemName I was able to better understand why the script was halting at that particular line.
Working code:
Do {
Try {
[System.IO.FileStream] $fs = $convertingFile.OpenWrite()
$fs.Dispose()
$locked = 0
}
Catch {
$ErrorMessage = $_.Exception.Message
$FailedItem = $_.Exception.ItemName
Write-Host $ErrorMessage
Write-Host $FailedItem
Write-Host "Can't convert yet, file appears to be loading..."
sleep 5
continue
}
} until ($locked -eq 0)

Reference credential in CMD called by DSC

I was wondering how I can reference credential in CMD called by DSC.
This is the configuration that I'm trying to deploy, but it doesn't receive credentials.
configuration SQLCMD
{
param
(
[Parameter(Mandatory=$true)]
[ValidateNotNullorEmpty()]
[PSCredential]
$Credential
)
Import-DscResource -ModuleName xSqlServer
Node localhost
{
LocalConfigurationManager
{
ConfigurationMode = 'ApplyOnly'
RebootNodeIfNeeded = $true
ActionAfterReboot = 'ContinueConfiguration'
AllowModuleOverwrite = $true
}
Script DeployDBmoveTempDB
{
SetScript = {
$SourceFile = 'C:\DatabaseTest.dacpac'
$TargetServerName = 'localhost'
$TargetDatabaseName = 'TestDB1'
$databaseSizeSQLCMD = '200MB'
$databaseLogSizeSQLCMD = '20MB'
$tempdbSizeSQLCMD = '1900MB'
$tempdbLogSizeSQLCMD = '1900MB'
trap {
Write-Error $_
Exit 1
}
$args = #('/Action:Publish'
,"/SourceFile:$SourceFile"
,"/TargetServerName:$TargetServerName"
,"/TargetUser:$Credential.UserName"
,"/TargetPassword:$Credential"
,"/TargetDatabaseName:$TargetDatabaseName"
,"/v:databaseSizeSQLCMD=$databaseSizeSQLCMD"
,"/v:databaseLogSizeSQLCMD=$databaseLogSizeSQLCMD"
,"/v:tempdbSizeSQLCMD=$databaseSizeSQLCMD"
,"/v:tempdbLogSizeSQLCMD=$databaseLogSizeSQLCMD"
,'/p:BlockOnPossibleDataLoss=false'
)
try {
& "C:\Program Files (x86)\Microsoft SQL Server\130\DAC\bin\SqlPackage.exe" $args
}
catch {
Write-Host $_ ;
}
}
TestScript = {
Test-Path D:\TestDB1_primary.mdf
}
GetScript = { <# This must return a hash table #> }
}
}
}
However, the following configuration works fine:
configuration SQLCMD
{
param
(
[Parameter(Mandatory=$true)]
[ValidateNotNullorEmpty()]
[PSCredential]
$Credential
)
Import-DscResource -ModuleName xSqlServer
Node localhost
{
LocalConfigurationManager
{
ConfigurationMode = 'ApplyOnly'
RebootNodeIfNeeded = $true
ActionAfterReboot = 'ContinueConfiguration'
AllowModuleOverwrite = $true
}
Script DeployDBmoveTempDB
{
SetScript = {
$ErrorActionPreference = "Stop"
$SourceFile = 'C:\DatabaseTest.dacpac'
$TargetServerName = 'localhost'
$user_name = 'mySqlAdmin'
$user_pwd = 'blabla'
$TargetDatabaseName = 'TestDB1'
$databaseSizeSQLCMD = '200MB'
$databaseLogSizeSQLCMD = '20MB'
$tempdbSizeSQLCMD = '1900MB'
$tempdbLogSizeSQLCMD = '1900MB'
trap {
Write-Error $_
Exit 1
}
$args = #('/Action:Publish'
,"/SourceFile:$SourceFile"
,"/TargetServerName:$TargetServerName"
,"/TargetUser:$user_name"
,"/TargetPassword:$user_pwd"
,"/TargetDatabaseName:$TargetDatabaseName"
,"/v:databaseSizeSQLCMD=$databaseSizeSQLCMD"
,"/v:databaseLogSizeSQLCMD=$databaseLogSizeSQLCMD"
,"/v:tempdbSizeSQLCMD=$databaseSizeSQLCMD"
,"/v:tempdbLogSizeSQLCMD=$databaseLogSizeSQLCMD"
,'/p:BlockOnPossibleDataLoss=false'
)
try {
& "C:\Program Files (x86)\Microsoft SQL Server\130\DAC\bin\SqlPackage.exe" $args
}
catch {
Write-Host $_ ;
}
}
TestScript = {
Test-Path D:\TestDB1_primary.mdf
}
GetScript = { <# This must return a hash table #> }
}
}
}
DSC-configurations store scripts as strings in the generated mof and does not expand variables by default since it wouldn't know which to expand and which to keep as part of the script. However, by specifying the $using:-Scope, you can include variables defined in the configuration. During mof-compilcation, the variables are then added at the start of each of the Get-/Set-/TestScript scriptblocks.
Ex:
configuration SQLCMD
{
param
(
[Parameter(Mandatory=$true)]
[ValidateNotNullorEmpty()]
[PSCredential]
$Credential
)
Import-DscResource -ModuleName xSqlServer
$user_name = $Credential.UserName
$user_pwd = $Credential.GetNetworkCredential().Password
Node localhost
{
Script DeployDBmoveTempDB
{
SetScript = {
$TargetDatabaseName = 'TestDB1'
$args = #(,"/TargetUser:$using:user_name"
,"/TargetPassword:$using:user_pwd"
,"/TargetDatabaseName:$TargetDatabaseName")
try {
& "C:\Program Files (x86)\Microsoft SQL Server\130\DAC\bin\SqlPackage.exe" $args
}
catch { Write-Host $_ }
}
TestScript = { Test-Path "D:\TestDB1_primary.mdf" }
GetScript = { <# This must return a hash table #> }
}
}
}
Be aware that the password will be stored in plain text in the mof-file. Ex:
SetScript = "$user_name ='User1'\n$user_pwd ='Password1'\n \n\n $TargetDatabaseName = 'TestDB1'\n $args = #(,\"/TargetUser:$user_name\"\n
,\"/TargetPassword:$user_pwd\"\n ,\"/TargetDatabaseName:$TargetDatabaseName\") \n\n try {\n & \"C:\\Program File
s (x86)\\Microsoft SQL Server\\130\\DAC\\bin\\SqlPackage.exe\" $args\n }\n catch { Write-Host $_ } \n\n ";