Consider the following command:
7z.exe a -t7z folder.7z folder
I have the following two stripped down powershell scripts
File 1: common.ps1
function Archive-Folder ($src, $dest_path, $archive_name) {
$script_dir = split-path $script:MyInvocation.MyCommand.Path
if ((test-path $src) -eq $false) {
write-error "$src is not a valid source directory"
#return
return $false
}
if ((test-path $dest_path) -eq $false) {
write-error "$dest_path is not a valid destination directory"
#return
return $false
}
if ([string]::IsNullOrWhiteSpace($archive_name) -eq $true) {
write-error "$archive_name is not a valid archive name"
#return
return $false
}
write-verbose "archiving the folder"
$archive_command = "$script_dir\7z.exe a -t7z $dest_path\$archive_name $src"
$exe = "$script_dir\7z.exe"
$arguments = #('a', '-t7z', "$dest_path\$archive_name", "$src")
iex $archive_command
# this doesn't stream the output. it prints it all at once.
# & $exe $arguments | write-verbose
return $true
}
File 2: script.ps1
$script_dir = split-path $script:MyInvocation.MyCommand.Path
. "$script_dir\common.ps1"
$VerbosePreference = "Continue"
$src = 'C:\some\source'
$backup_path = 'C:\some\destination'
$date_format = 'yyyy_MM_dd_HHmm'
$date = get-date
$date_str = $date.tostring($date_format)
$date_ticks = $date.ticks
$archive_name = "backup-$date_str-$date_ticks.7z"
# this prints the output streamed. The output ends with `True`
archive-folder $src $backup_path $archive_name
# the following however doesn't output anything. in order to separate the command output from my function output,
# i was printing the command output using write-verbose
$isSuccess = archive-folder $src $backup_path $archive_name
if ($isSuccess -eq $true) {
#proceed with the rest of the code
}
With inputs from #Christian & #zdan, I was able to isolate the issue to the capturing of the return value. Similar to archive-folder, I have other functions that execute some commandline tool. I was thinking that each of these functions can return a true or false depending on whether the function was called with the right operations and the commandline tool executed properly.
However, if I capture the return value of my archive-folder function, then the output of the command doesn't get printed to the console. Also, my return value doesn't consist of a true or false value. It consists of the entire output of the command.
My first attempt at solving this was to write the command execution statement as iex $archive_command | write-verbose, but this did not stream the output.
I suppose I can check for side effects the commandline tool has in case of success (like presence of the archive file) to determine whether my function executed successfully, but am not sure if I will be able to do this for all functions that I may end up creating.
Is there a way to return a value and also stream the output of a commandline tool?
EDIT 2
With regards to why am I diving the code into two separate files/functions, my actual use scenario is as follows
The script.ps1 will be coordinating this flow. Backup the database (mongodb generates files for each collection of the db). Archive the database backup. Upload the archive to S3. Each of these steps will be done by a separate function in common.ps1. The script.ps1 will only contain the glue code. Posting all this might have complicated the question and I felt wasn't needed to understand the issue am facing
EDIT 1
If the folder being compressed has 5 files, 7zip will first output the copyright. Then it will output the text Scanning. Then it will output a line Creating archive at some location. Then it will process each file, outputting the progress for each file, one by one. This way, we get constant feedback about the progress of the operation.
If I execute the powershell function, then I see no output for the duration of the operation and then all the output at once. I get no feedback from 7zip. I would like to simulate the behaviour that 7zip shows when ran as a standalone exe.
This works for me:
& 7z.exe a -t7z -bsp1 $archive_name $src 2>&1 | Out-Host
The -bsp1 switch redirects the progress information stream to the stdout stream. This is a feature of 7z. Also look at the -bb switch.
The 2>&1 redirects the error stream to stdout. This is a feature of PowerShell.
-bs (Set output stream for output/error/progress line) switch Syntax
Syntax
-bs{o|e|p}{0|1|2}
{id} | Stream Type
................................
o | standard output messages
e | error messages
p | progress information
{N} | Stream Destination
................................
0 | disable stream
1 | redirect to stdout stream
2 | redirect to stderr stream
It seems to me that you should be able to just do:
&7z.exe a -t7z $archive_name $src | write-verbose
Unless there is something I am missing.
why not this way? If you need
function archive-folder ($src, $archive_name) {
$origcolor = [console]::ForegroundColor
[console]::ForegroundColor = "yellow"
"archiving $src"
$command = "7z.exe a -t7z $archive_name $src"
iex $command
[console]::ForegroundColor = $origcolor
}
My trivial caller function:
Function k
{
dir c:\ps\ita
Write-Host "Starting 7zippping from K function"
archive-folder -archive_name c:\ps\pippo.7z c:\ps\ita
Write-Host "7zipping from function k ended"
}
and what I see in powershell console ( in yellow the output from archive-folder)
k
Directory: C:\ps\ita
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 08/06/2011 19:26 5,502 MB ita.txt
-a--- 28/05/1994 16:59 165,624 KB ITALIANO.A
-a--- 28/05/1994 16:54 53,903 KB ITALIANO.B
-a--- 28/05/1994 17:00 165,541 KB ITALIANO.C
-a--- 08/06/2011 11:06 98,609 KB ITALIANO.D
-a--- 28/05/1994 17:00 72,077 KB ITALIANO.E
-a--- 28/05/1994 16:54 80,813 KB ITALIANO.F
-a--- 28/05/1994 16:55 78,312 KB ITALIANO.G
-a--- 28/05/1994 16:55 2,412 KB ITALIANO.H
-a--- 08/06/2011 11:07 298,609 KB ITALIANO.I
-a--- 28/05/1994 16:55 1,033 KB ITALIANO.J
-a--- 28/05/1994 16:55 1,777 KB ITALIANO.K
-a--- 28/05/1994 17:01 71,553 KB ITALIANO.L
-a--- 08/06/2011 10:59 162,084 KB ITALIANO.M
-a--- 28/05/1994 16:56 47,123 KB ITALIANO.N
-a--- 28/05/1994 16:56 72,973 KB ITALIANO.O
-a--- 08/06/2011 19:37 264,109 KB ITALIANO.P
-a--- 28/05/1994 16:56 10,512 KB ITALIANO.Q
-a--- 08/06/2011 19:38 327,348 KB ITALIANO.R
-a--- 08/06/2011 19:40 566,512 KB ITALIANO.S
-a--- 08/06/2011 10:57 184,719 KB ITALIANO.T
-a--- 28/05/1994 16:57 19,378 KB ITALIANO.U
-a--- 28/05/1994 16:57 61,552 KB ITALIANO.V
-a--- 28/05/1994 16:57 1,334 KB ITALIANO.W
-a--- 28/05/1994 16:57 1,368 KB ITALIANO.X
-a--- 28/05/1994 16:57 533 B ITALIANO.Y
-a--- 28/05/1994 17:01 7,054 KB ITALIANO.Z
Starting 7zippping from K funztion
archiving c:\ps\ita
7-Zip 4.65 Copyright (c) 1999-2009 Igor Pavlov 2009-02-03
Scanning
Updating archive c:\ps\pippo.7z
Compressing ITA\ITALIANO.H
Compressing ITA\ITALIANO.C
Compressing ITA\ITALIANO.F
Compressing ITA\ita.txt
Compressing ITA\ITALIANO.O
Compressing ITA\ITALIANO.A
Compressing ITA\ITALIANO.B
Compressing ITA\ITALIANO.D
Compressing ITA\ITALIANO.E
Compressing ITA\ITALIANO.G
Compressing ITA\ITALIANO.I
Compressing ITA\ITALIANO.J
Compressing ITA\ITALIANO.K
Compressing ITA\ITALIANO.L
Compressing ITA\ITALIANO.M
Compressing ITA\ITALIANO.N
Compressing ITA\ITALIANO.P
Compressing ITA\ITALIANO.Q
Compressing ITA\ITALIANO.R
Compressing ITA\ITALIANO.S
Compressing ITA\ITALIANO.T
Compressing ITA\ITALIANO.U
Compressing ITA\ITALIANO.V
Compressing ITA\ITALIANO.W
Compressing ITA\ITALIANO.X
Compressing ITA\ITALIANO.Y
Compressing ITA\ITALIANO.Z
Everything is Ok
7zipping from function k ended
Related
I have a csv file with extension and description.
I want to import that file and use it as the filter parameter in a gci command.
But I get no results.
I expect to get a list of the jpg files but get no results.
$extensions=Import-CSV -Path c:\scripts\Media-extension-foto.csv
#$extensions=Import-CSV -Path c:\scripts\Media-extension-foto.csv -header extension
$extensions.extension
$src = "c:\scripts\"
#gci c:\scripts\ -Include $Extensions.extension #-Force -recurse
#gci c:\scripts\ -filter $Extensions.extension #-Force -recurse
gci c:\scripts\|where{$_ -like $extensions.extension}`
my csv file looks like this (just made a small file for testing)
extension,"description"
*.JPEG,JPEG Image
*.JPF,JPEG 2000 Image
*.JPG,JPEG Image
*.JPG_LARGE,Twitter Large JPEG Image
There are jpg files in that folder :
Directory: C:\Scripts
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 23/11/2022 11:02 509592 nieuw9754560_02-10.jpg
-a--- 23/11/2022 11:02 576486 nieuw9754560_02-15.jpg
-a--- 23/11/2022 11:02 641802 nieuw9754560_02-20.jpg
-a--- 23/11/2022 11:01 705702 nieuw9754560_02-25.jpg
-a--- 23/11/2022 11:01 763249 nieuw9754560_02-30.jpg
I've just tested this - I think all you're missing is changing the $_ to $_.Extension for the Get-ChildItem, on the last line.
Hope that helps.
Doesn't directly fix the code but here's another way of getting the result using foreach to iterate through each extension in the array:
foreach($e in $extensions.extension) {
gci c:\scripts\ | where {$_ -like $e}
}
I have some text files, basically called jobs, where a job can contain another job or series of jobs, which can contain another job or series of jobs, etc... Essentially calling the job it contains.
The idea is to iterate through the top most file, step into the next lower file and so on until we reach the "lowest level".
The script I currently use is a loop, within a loop, within a loop. If I need to reach further down, I need to add another loop. In this example, we ignore the contents of the file having "JOB" in it.
Is there a better way to loop or iterate through all this without just adding more ForEach-Objects within ForEach-Objects?
cd C:\Files
$files = Get-ChildItem -Name
$files | ForEach-Object {
$filename1 = $_
write-host "$filename1"
$step1 = Get-Content $filename1
$step1 | ForEach-Object {
$filename2 = $_
write-host "`t$filename2" #We tab to show 2nd level job is inside 1st level job
$step2 = Get-Content $filename2
$step2 | ForEach-Object {
$filename3 = $_
$write-host "`t`t$filename3"
$step3 = Get-Content $filename3
$step3 | ForeEach-Object {#keep adding loops}
}
}
}
The jobs/files are contained all in a single directory as:
FILE1A
FILE1B
FILE1C
FILE2
FILE3
FILE4
etc..
Content of each file may look like:
JOB FILE1B
No limit to how many "JOB ???" may be contained inside the file.
There really is no convention to the naming of these files/jobs. When running the script, the result should look like this:
FILE1A
FILE1B
FILE1C
FILE2
FILE3
FILEXYZ
FILEZYX
FILE123
FILEABC
As the others in comments have mentioned, one might use recursion to accomplish this.
I hope I understood the question in my mock up.
I have some files that look like this below, some containing lines with paths to other files and some not (0 length empty files)
Directory: C:\temp\powershell\recursive_tests
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 18.08.2021 19:31 133 A1.txt
-a--- 18.08.2021 19:14 0 A1a.txt
-a--- 18.08.2021 19:31 133 A1b.txt
-a--- 18.08.2021 19:18 0 A1b1.txt
-a--- 18.08.2021 19:18 0 A1b2.txt
-a--- 18.08.2021 19:18 0 A1b3.txt
-a--- 18.08.2021 19:14 0 A1c.txt
-a--- 18.08.2021 19:31 133 B1.txt
-a--- 18.08.2021 19:31 133 B1a.txt
-a--- 18.08.2021 19:18 0 B1a1.txt
-a--- 18.08.2021 19:18 0 B1a2.txt
-a--- 18.08.2021 19:18 0 B1a3.txt
-a--- 18.08.2021 19:14 0 B1b.txt
-a--- 18.08.2021 19:31 133 B1c.txt
-a--- 18.08.2021 19:18 0 B1c1.txt
-a--- 18.08.2021 19:18 0 B1c2.txt
-a--- 18.08.2021 19:18 0 B1c3.txt
As an example A1.txt looks like this
C:\temp\powershell\recursive_tests\A1a.txt
C:\temp\powershell\recursive_tests\A1b.txt
C:\temp\powershell\recursive_tests\A1c.txt
I imagine the code to process through them could look something like this
function Get-FileInFile {
param(
[string[]]$Paths,
$Level = 0
)
foreach ($path in $Paths) {
Write-Host "$("`t" * $Level)$Path"
$lines = Get-Content -Path $Path
$newLevel = $Level + 1
$lines | ForEach-Object {
get-fileinfile -Path $_ -Level $newLevel
}
}
}
Output
PS C:\temp\powershell\recursive_tests> get-fileinfile 'C:\temp\powershell\recursive_tests\A1.txt', 'C:\temp\powershell\recursive_tests\B1.txt'
C:\temp\powershell\recursive_tests\A1.txt
C:\temp\powershell\recursive_tests\A1a.txt
C:\temp\powershell\recursive_tests\A1b.txt
C:\temp\powershell\recursive_tests\A1b1.txt
C:\temp\powershell\recursive_tests\A1b2.txt
C:\temp\powershell\recursive_tests\A1b3.txt
C:\temp\powershell\recursive_tests\A1c.txt
C:\temp\powershell\recursive_tests\B1.txt
C:\temp\powershell\recursive_tests\B1a.txt
C:\temp\powershell\recursive_tests\B1a1.txt
C:\temp\powershell\recursive_tests\B1a2.txt
C:\temp\powershell\recursive_tests\B1a3.txt
C:\temp\powershell\recursive_tests\B1b.txt
C:\temp\powershell\recursive_tests\B1c.txt
C:\temp\powershell\recursive_tests\B1c1.txt
C:\temp\powershell\recursive_tests\B1c2.txt
C:\temp\powershell\recursive_tests\B1c3.txt
Here I pass in only the 2 top level files A1.txt and B1.txt. Since all the files exist in the same folder I cannot use Get-ChildItem and just foreach through every file or files will be processed more than once, first being called initially by me and then again when their path is encountered in the sublevel files, so instead I just pass in the 2 that I know are the roots.
I am trying to implement the code at this highly rated page https://blogs.technet.microsoft.com/gbordier/2009/05/05/powershell-and-writing-files-how-fast-can-you-write-to-a-file/ using "Method 4."
When I run the code below, it does appear to take a few seconds, but no t.txt file appears to be created. What am I missing?
PS L:\users\lit\t> Get-Content .\ttt.ps1
$s = "some text"
$stream = [System.IO.StreamWriter] "t.txt"
1..10000 | % {
$stream.WriteLine($s)
}
$stream.close()
PS L:\users\lit\t> $PSVersionTable["PSVersion"]
Major Minor Build Revision
----- ----- ----- --------
5 0 10586 117
Many thanks to #Ansgar Wiechers. The only way I could place the file into the correct directory was to produce a full path from a root. Since I wanted to put it into the current directory, (Get-Location).Path provided the directory path to which I appended the file name.
Without specifying the full path, System.IO.StreamWriter wanted to put the file into my home directory.
PS C:\Users\lit\t\booo> Get-Content .\ttt.ps1
[cmdletbinding()]
Param()
$FullPath = Join-Path (Get-Location).Path "t.txt"
Write-Verbose $FullPath
$stream = [System.IO.StreamWriter] $FullPath
Write-Verbose $stream.BaseStream.Name
1..10 | % { $stream.WriteLine($_) }
$stream.close()
PS C:\Users\lit\t\booo> .\ttt.ps1 -Verbose
VERBOSE: C:\Users\lit\t\booo\t.txt
VERBOSE: C:\Users\lit\t\booo\t.txt
PS C:\Users\lit\t\booo> Get-ChildItem
Directory: C:\Users\lit\t\booo
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 1/11/2017 5:22 PM 31 t.txt
-a---- 1/11/2017 5:21 PM 251 ttt.ps1
I have a directory that I want to recursively copy to anoth location for processing. It seems like the first directory is not getting created. The part of the program that does the copy is below:
# Read the source files to convert.
$lAllFiles = Get-ChildItem $lSrc -Exclude $lExcludedFiles
# Remove excluded directories
foreach ( $lExclusion in $lAllExcludedDirectories ) {
# If we have a wildcard use the -notlike operator to filter the list.
# Otherwise use an -ne operator.
if ( ($lExclusion -like '*' ) -or ($lExclusion -like '?' ) ) {
$lAllFiles = [array]$lAllFiles -notlike $lExclusion
} else {
$lAllFiles = [array]$lAllFiles -ne $lExclusion
}
}
# Start the log.
Start-Transcript -Path "$lJobLog"
# Copy the data to be converted to the conversion root.
ForEach ( $lFile in $lAllFiles ) {
Copy-Item $lFile $lCnv -Recurse -Force -ErrorAction SilentlyContinue -ErrorVariable +JobErrors -Verbose
}
.
.
.
When I compare the source and destination directories, it looks as though the first subdirectory is not created in the destination and the contents of the first sub directory are written to the destination.
As an example:
The Source:
Mode LastWriteTime Length Name
---- ------------- ------ ----
d---- 1/6/2013 9:30 AM AP-LEDGER
d---- 3/12/2014 9:28 AM AP.DETAIL
d---- 8/20/2014 9:33 PM AR-LEDGER
d---- 9/3/2011 9:58 AM lost+found
-a--- 5/13/2016 9:21 PM 32768 AP-BATCH-CTRL
-a--- 5/13/2016 8:39 PM 291293184 AP-LEDGER-XREF
-a--- 9/28/2015 3:14 PM 8425472 AP.JDE
-a--- 5/13/2016 8:39 PM 150700032 AP.LINK
-a--- 5/2/2016 3:30 PM 52224 AP.QUEUES
-a--- 5/13/2016 8:17 PM 743018496 AP.SUSP
-a--- 4/30/2016 9:11 PM 51222528 ARROLLARC
-a--- 4/30/2016 9:11 PM 102404096 ARROLLFWD
-a--- 5/14/2016 3:29 AM 1016950784 A_Pa
-a--- 5/14/2016 12:37 AM 238280704 A_Ra
-a--- 5/14/2016 3:16 AM 61423616 GL-CROSS
-a--- 5/14/2016 3:16 AM 175235072 GL-INDEX
-a--- 5/14/2016 3:16 AM 21512192 G_La
-a--- 5/14/2016 3:16 AM 224661504 X_AP.SUSP
-a--- 5/14/2016 3:29 AM 150089728 X_A_Pa
-a--- 5/14/2016 12:37 AM 63578112 X_A_Ra
The Destination:
Mode LastWriteTime Length Name
---- ------------- ------ ----
d---- 5/26/2016 10:53 PM AP.DETAIL
d---- 5/26/2016 10:53 PM AR-LEDGER
-a--- 5/13/2016 8:38 PM 1073737728 dat001
-a--- 5/13/2016 8:39 PM 682541056 dat002
-a--- 5/13/2016 8:39 PM 1004023808 idx001
-a--- 5/13/2016 8:37 PM 41234432 over001
Note the files in the directory should reside in the folder AP-LEDGER.
What is going on?
Does the -Recurse -Force not create the directories?
Check your Copy-Item, I've used something like this function to copy files/directories from one location to the next with exclusions.
Function Copy-Folder($Path,$DestinationPath,$Exclude = $exclude) {
# Get all the files to copy
$files = Get-ChildItem -Path $Path -Exclude $Exclude
# Copy files to destination folder.
foreach($file in $files) {
Copy-Item $file.FullName $DestinationPath -Recurse -Force
$filename = $file.Name
$copy = "$DestinationPath\$filename"
Write-Host "Copied: $filename to $copy"
}
}
You should also take a look at your logic for the removing excluded sub-directories
I would like to change the creation date of the files that I generate with this script :
$clientname = Read-Host "Enter the client name"
$path = Read-Host "Enter the complete path of .bak files"
$time = "01-00-00"
$space = " "
for ($i = 0; $i -lt 7;$i++)
{
$date = (Get-Date).AddDays(-1-$i).ToString('yyyy-MM-dd')
New-Item -ItemType file $path$clientname$space$date$space$time.bak
}
So it gives me theses files :
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 16/08/2013 16:55 0 client 2013-08-15 01-00-00.bak
-a--- 16/08/2013 16:55 0 client 2013-08-14 01-00-00.bak
-a--- 16/08/2013 16:55 0 client 2013-08-13 01-00-00.bak
-a--- 16/08/2013 16:55 0 client 2013-08-12 01-00-00.bak
-a--- 16/08/2013 16:55 0 client 2013-08-11 01-00-00.bak
-a--- 16/08/2013 16:55 0 client 2013-08-10 01-00-00.bak
-a--- 16/08/2013 16:55 0 client 2013-08-09 01-00-00.bak
I want to modify the LastWriteTime property of each files, I want it to be the same as the date in the file name.
Example for this file "client 2013-08-15 01-00-00.bak" the LastWriteTime will be "15/08/2013 01:00"
I'm stuck and I do not how can we do that
Thank you
Not tested, but try this in your loop after you call New-Item:
$file = Get-ChildItem $path$clientname$space$date$space$time.bak
$file.LastWriteTime = (Get-Date).AddDays(-1-$i)
If you want to see a full listing of things you can do with FileInfo objects, try calling $file | gm in your powershell console. You could also view the docs on MSDN.
for ($i=0; $i -lt 7;$i++)
{
$date = (Get-Date).AddDays(-1-$i)
$filedate = $date.ToString('yyyy-MM-dd')
$file = New-Item -ItemType File $path$clientname$space$filedate$space$time.bak
$file.LastWriteTime = $date
}