Powershell pattern matching - powershell

I have a requirement to create directories based on a numeric 9 digit with leading zeros primary key from an order entry database on a remote server.
Each morning a new directory should be created. The directory name will be the primary key number of the new customer. I already have the sql statement extracting the primary key information into a text file using BCP that preserves the leading zeros.
This file is transferred from the database server to the local directory where the folders need to be created. Using some PowerShell code that I think I found I am trying to create the folders from the text file that I have been modifying.
I need the leading zeros preserved in the folder name so I can reference back to the database later in the project. My problem is that when I run the PowerShell script, no folders are created. I think I have the problem isolated to the pattern definition, but don't understand what is wrong.
Input txt file example
001132884
001454596
001454602
001454605
001454606
001454601
001107119
001454600
001454608
PowerShell script
$folder="Customerdocuments"; # Directory to place the new folders in.
$txtFile="E:\dirtext.txt"; # File with list of new folder-names
$pattern="\d+.+"; # Pattern that lines must match
Get-Content $txtFile | %{
if($_ -match $pattern)
{
mkdir "$folder\$_";
}
}

I'm missing a clear question/error report.
Your pattern takes all digits (greedy) from input \d+
and requires at least one other (any) character .+ which isn't present in your sample file.
So your issue isn't related to leading zeroes.
Better specify exactly 9 digits and put that into a Where-object,
The path from $folder will be relative to the current folder and should be build using Join-Path
As mkdir is just a wrapper function for New-Item and it supports piped input I'd use it directly.
$folder="Customerdocuments"; # Directory to place the new folders in.
$txtFile="E:\dirtext.txt"; # File with list of new folder-names
$pattern="^\d{9}$" # Pattern that lines must match
Get-Content $txtFile | Where-Object {$_ -match $pattern}|
New-Item -Path {Join-Path $folder $_} -ItemType Directory | Out-Null
}

Related

Why does my powershell script loop not work when run, but when stepped through in debugger it works fine?

So I have the below script for a project at work:
# This script will look for a CSV in the script folder
# If found it will split the CSV based on a change in a column header
# It will then create seperate CSV files based on the column data
# get script directory and add it to a variable
Set-Location $PSScriptRoot
$baseDir = $PSScriptRoot
# get a list of csv file names
$csvfile = Get-ChildItem -Path $baseDir -Filter *.csv
# If multiple CSV files loop through all of them
foreach ($i in $csvfile) {
# Import and split the original csv
# Change the value after -Property to match the column header name of the column used to split on value change -
# Header names with spaces require surrounding quotes -
# This value will also be used to name the resulting CSV file
Import-Csv $i | Group-Object -Property "Submission ID" |
Foreach-Object {$path="Output\"+$_.name+".csv" ; $_.group |
Export-Csv -Path $path -NoTypeInformation}
# get the current time and date
$procdte = Get-Date -uformat "%m-%d-%Y %H %M %S"
# rename the original file so that it's not processed again
Rename-Item $i -NewName "Processed $procdte.txt"
}
# End processing loop
Important: some parts of this script are commented out - like the Rename-Item line is half-commented. Dunno why, think it's a stackoverflow markdown issue. It isn't like that in ISE
I tested it out with two csv files in the current directory. It's supposed to rename both files at the end to a .txt so they don't get processed again. When I just run the script, it misses the 2nd csv file. But when I step through it with debugger in ISE, it renames both files fine. What gives?
Ran powershell script, expecting both CSV files to be renamed, however it only renamed one of them. Thought there was an issue with the script, but when I run it in debug, it renames both files fine. Have managed to replicate multiple times.
Edit: to specify - this is not designed to work with two csv files specifically. It's supposed to work with one or more. I was just using two to test.

Can't delete Windows folder that contains a subfolder named "..." which contains a subfolder named "..." etc

I'm stumped.
I have a folder called c:/data/ToDelete. It used to contain files as well as a single subfolder called ..., which contained the same files as well as single subfolder named ... and this went on ad nauseum.
I moved the contents of the ToDelete folder elsewhere, but the subfolder/subfolder structure remains, and I can't get rid of it.
This is what it looks like in Explorer
This PC > Data > ToDelete > ... > ... > ... > ... > etc
clicking on the above reveals the path: C:\Data\ToDelete\
I was able to rename the first folder (from c:/data/temp/ to c:/data/ToDelete/) but whenever I try to rename the subfolder I get the "This folder is already open" (probably because it is just referring back to it's parent folder).
Things I've tried (none worked)
the basics: Delete key, Shift Delete, drag to recycle bin, send to recycle bin
command prompt delete, including trying the ASCII code for . just in case.
I tried zipping it with 7zip
I did a chkdsk and got no errors
I tried removing with Revo Uninstaller Pro (the Delete folder option).
I tried renaming the subfolders with this powershell script that I found online thinking I could then delete them but got an error:
Get-ChildItem -Recurse | where {$_.attributes -eq "Directory"} | Where-Object {$_.Name -match '...'} | Rename-Item -NewName { $_.Name -replace '...' , 'ToDelete' }
The funny thing is I found the folder in my backup directory as well, so somehow the entire thing can get copied (which means I have to remove it in multiple places now).
You need to do two things to enable PowerShell to process files or folders literally named ...:
Use -LiteralPath instead of -Path parameter where possible, to prevent PowerShell from parsing the path.
Prepend \\?\ to the full path to tell the underlying Windows APIs to disable all string parsing and to send the string that follows it straight to the file system.
Example:
# For demonstration, create two nested sub folders literally named '...'
New-Item -ItemType Directory -Path '\\?\c:\data\ToDelete\...\...'
# Rename all '...' sub folders to 'ToDelete'
Get-ChildItem -LiteralPath '\\?\c:\data\ToDelete' -Directory -Recurse -Filter '...' |
Sort-Object { $_.Fullname.Length } -Desc |
Rename-Item -NewName ToDelete
Before renaming the folders, we must sort by path length (descending) to make sure that deepest nested folders will be renamed first. Otherwise paths enumerated by Get-ChildItem would be incorrect after renaming the first folder.
For what you have tried:
Note that -match '...' doesn't find folders named literally like "...". The -match operator's RHS operand is a regular expression pattern and as such would match the first sequence of any three characters in the string. The same can be said about -replace '...'. To literally match "...", use -eq '...' or escape the dots like this when used in RegEx pattern: \.\.\.

Expanding Environmental Variable using CSV file into Powershell script

i am trying to create a backup script with csv file which will contain all the local (where backup will be stored) and backup (folder to backup) loaction and run robocopy to copy files from backup to local folder. i want to import environmental variable from csv file that are used for default folders (e.g. env:APPDATA) as location to be use for backing up some files and folders in user documents or pulbic documents. When i use the env variable directly or use full path address in the script the copy action works fine.
robocopy "&env:localappdata\steam" "$backup"
but when import from csv, the script does not see it as env variable. robocopy shows the error because it picks up the locations like this
Source : C:\WINDOWS\system32\$env:LOCALAPPDATA\steam\
Dest : D:\backup\steam\
Below is the full code i am using.
$path = "$PSScriptRoot\backup\"
$locations = import-csv "$PSScriptRoot\backup\local.csv" -Delimiter "," -Header 'Local','Backup','Display' | Select-Object Local,Backup,display
foreach($location in $locations){
$source = $location.Local
$source = $source.ToString()
$destination = $location.Backup
$destination = $destination.tostring()
$Name = $location.Display
$Name = $name.tostring()
Write-host "Copying $Name, please wait...." -foregroundColor Yellow
robocopy "$destination" "$path$source" \s \e
}
And my CSV file looks like this
Steam, $env:LOCALAPPDATA\steam, Steam files Backup
As you have added a path in the csv as $env:LOCALAPPDATA\steam, and knowing that whatever you read from file using Import-Csv is a string, you need to use regex to convert the $env:VARNAME into %VARNAME% in order to be able to resolve that environment variable into a path.
Instead of
$source = $source.ToString()
do this
$source = [Environment]::ExpandEnvironmentVariables(($source -replace '\$env:(\w+)', '%$1%'))
$source wil now have a valid path like C:\Users\Aman\AppData\Local\Steam
Do the same with $destination
Regex details:
\$ Match the character “$” literally
env: Match the characters “env:” literally
( Match the regular expression below and capture its match into backreference number 1
\w Match a single character that is a “word character” (letters, digits, etc.)
+ Between one and unlimited times, as many times as possible, giving back as needed (greedy)
)
P.S. Instead of combining a path with string concatenation like you do in "$path$source", it is much safer to use the Join-Path cmdlet for that

Renaming files from a text list using powershell

I have seen many different postings here on how to approach this task. I have tried them all with minor variations of them to no avail. Every time I run the powershell code, I still get the same error message: "cannot rename because the file does not exist..."
I just simply want to rename a bunch of files in one folder from the list of filenames I have in a text file.
Here is one version of code:
$inputFile1=#()
$filesTochange=#()
$inputFile1 = get-content H:\dev\outputfile.txt
$filesToChange = Get-ChildItem H:\dev\extractedFiles1 | Foreach -Begin
{$i=0}
-process {Rename-Item $filesToChange[$i] -newName ($inputFile1[$i] -f $i++)
}
Here is another version:
$inputFile1=#()
$filesTochange=#()
$inputFile1 = get-content H:\dev\outputfile.txt
$filesToChange = Get-ChildItem H:\dev\extractedFiles1
$i=0
foreach ($fil in $filesToChange) {Rename-Item $fil -NewName
$inputFile1[$i++]}
Not entirely sure what's your setup or desired output is but give this a whirl bud. Not the most elegant looking solutions but hopefully this is what you are looking for? Do be mindful with the sorting of the filenames in your outputfile.txt and how the folders are listed when you get the childitem.
$BasePath = 'C:\Test'
$FilesToChangeFolderName = 'extractedFiles1'
$filestochange = Get-ChildItem -Path "$BasePath\$FilesToChangeFolderName"
$FileNames = Get-Content "$BasePath\outputfile.txt"
if($filestochange.FullName.Count -eq $FileNames.Count)
{
for($i=0; $i -lt $FileNames.Count; $i++)
{
write-host "Renaming file $($filestochange.Name[$i]) to $($FileNames[$i]+".txt")"
Rename-Item -Path $filestochange.FullName[$i] -NewName ($FileNames[$i]+".txt")
}
}
Setup -
outputfile.txt contains:
renametoA
renametoB
renametoC
renametoD
renametoE
renametoF
Folder structure:
Results:
Renaming file renameto1.txt to renametoA.txt
Renaming file renameto2.txt to renametoB.txt
Renaming file renameto3.txt to renametoC.txt
Renaming file renameto4.txt to renametoD.txt
Renaming file renameto5.txt to renametoE.txt
Renaming file renameto6.txt to renametoF.txt
Explanation [TLDR]:
The script below takes a text file input which contains the name that should be used to rename each text file you have.
Example:
outputfile.txt file contains the names below:
renametoA
renametoB
renametoC
renametoD
renametoE
renametoF
There are 6 text files inside the "extractedFiles1" folder which you can see in the image.
The script actually renames the files in the "extractedFiles1" folder according to the names from the output.txt file.
Thus it follows this logic:
Renaming file renameto1.txt to renametoA.txt
Renaming file renameto2.txt to renametoB.txt
Renaming file renameto3.txt to renametoC.txt
Renaming file renameto4.txt to renametoD.txt
Renaming file renameto5.txt to renametoE.txt
Renaming file renameto6.txt to renametoF.txt
So after all the script runs your "extractedFiles1" folder's gonna look something like this:
Despite this being an older thread, I wanted to offer another "brute force" option.
CodeNagi's reply works well in PowerShell, although it took me a bit to get working.
If you already have a list with file names (output.txt) consider using excel (or OpenOffice) and the command prompt cmd.
This video is a step by step guide for some of this solution:
https://youtu.be/znhqGrF4gVQ
Open cmd with administrator privileges.
Create a list of your current (old) file names:
cd H:\dev\extractedFiles1
dir > input.txt
Open the input.txt (e.g. in Notepad). It should look like this:
Volume in drive H is Windows
Volume Serial Number is C123-4567
Directory of H:\dev\extractedFiles1
05/12/2022 11.24 .
05/12/2022 11.24 ..
05/12/2022 09.34 5,255,248 Filename1.txt
05/12/2022 09.34 5,255,952 Filename2.txt
...
Copy the lines with filenames and timestamps into an Excel sheet
Use Data > Text to columns to split the filenames from the time stamps
Copy or import your target/new filenames from output.txt next to the old filenames
Make a new column with the formula
= "ren"&" "&A1&" "&B1
resulting in something like
ren Filename1.txt FilenameA.txt
Copy all formulas and paste them in cmd. Make sure you are in the right directory.
Notes: If you have spaces in the file names, you will need to wrap each file name first in apostrophes ". Since the concatenation formula in excel doesn't accept """ (triple apostropes), make yourself a column with only " (here C) and then refer to it in the concatenation: = "ren "&C1&A1&C1&" "&C1&B1&C1&.
Further, if you have duplicate files or want to make sure they are copied correclty, you can use the MOVE function instead of rename (ren).
Instead of point 6. above do the following:
Make a new column with the formula
= "MOVE"&" "&A1&" "&"H:\dev\extractedFiles1\Renamed"&B1
Copy the created command into cmd
It will move and rename the files according to the names in B1.
This example make a copy and rename file to a list csv
Import-CSV LISTA.csv -Header newFileName | % { Copy-Item -Path archivo_convert.pdf -Destination "$($_.newfilename).pdf" }

How to do a copy /b in a powershell script to insert a BOM marker, but as a batch for files that match a filter and changes the ext on output?

REASONS WHY THIS IS NOT A DUPLICATE
Since 3 people have already voted to close, I guess I should explain why this question is not a duplicate:
I cannot use cat or >> as these mess up the encoding of the files, which are UTF8 on input and need to be UTF8-BOM on output.
The linked question does not show how to loop through all files that match a given pattern in a directory, and concatenate a single file to each of the matching files on output, plus give the new file a different extension.
Using Set-Content is not Powershell 6 future-proof, since Set-Content will NOT add a BOM marker. In Powershell 5 and below, it sometimes adds a BOM marker and sometimes not, depending on the configuration settings of the executing user. See 'quick note on encoding' at the end of this article.
So in conclusion I am looking for a solution that uses copy (hence the question title) and does NOT use Cat or Set-Content.
I need to loop through certain files in a given directory and run the following on each file:
copy /b BOMMarker.txt+InputFile.dat OutputFile.txt
This inserts the contents of the BOMMarker.txt file at the start of the InputFile.dat and writes the output to OutputFile.txt
I found this question which explains how I can loop through the folder to load each file into Powershell, but how do I apply the "copy /b" command so that I can get the BOM marker at the start of each file?
EDIT
The comment from Jeroen indicates I can just do Set-Content on the output file, as Powershell will automatically add the BOM at the start.
But I also need to change the extension. So the output filename needs to be the same as the input filename, just with a changed extension (from .dat to .txt) and including the BOM.
I am guessing I can use Path.ChangeExtension somehow to do this, but not sure how to combine that with also adding the BOM.
EDIT - for Bounty
The example answer I posted does not work in all environments I tested it, and I do not know why (possibly different default Powershell setttings) but also, it is not future proof since Powershell 6 will not output BOM by default.
From the given directory, I need to process all files that match the filter (DIL_BG_TXN*.dat).
For each of those files, I need to copy it with a BOM at the start but the resultant new file needs to be the same name but with the extension .txt instead of .dat.
This solutions uses streams, that reliably read and write as-is:
$bomStream = [IO.File]::OpenRead('BOMMarker.txt')
$location = "" # set this to the folder location
$items = Get-ChildItem -Path $location -Filter DIL_BG_TXN*.dat
foreach ($item in $items) {
$sourceStream = [IO.File]::OpenRead($item.FullName)
$targetStream = [IO.File]::OpenWrite([IO.Path]::ChangeExtension($item.FullName, '.txt'))
$bomStream.CopyTo($targetStream)
$sourceStream.CopyTo($targetStream)
$targetStream.Flush()
$targetStream.Close()
$sourceStream.Close()
$bomStream.Position = 0
}
$bomStream.Close()
Of course please write the absolute path of BOMMarker.txt (1st line) according to its location.
This finally worked:
$Location = "C:\Code\Bulgaria_Test"
$items = Get-ChildItem -Path $Location -Filter DIL_BG_TXN*.dat
ForEach ($item in $items) {
Write-Host "Processing file - " $item
cmd /c copy /b BOMMarker.txt+$item ($item.BaseName + '.txt')
}
Description:
Set the directory location where all the .dat files are.
Load only those files that match the filter into the array $items.
Loop through each $item in the array.
With each $item, call cmd shell with the copy /b command and concatenate the bom marker file with the $item file and write the result to the basename of $item plus the new extension.