Goal: Update text entry on one line within many files distributed on a server
Summary: As part of an application migration between datacenters the .rdp files on end-user desktops need to be updated to point to the new IP address of their Remote Desktop Server. All the .rdp files reside on Windows servers in a redirected folders SMB share where I have Administrative access.
Powershell experience: minimal. Still trying to wrap my head around the way variables, output and piping work.
Was originally trying to make a single line of powershell code to complete this task but got stuck and had to make script file with the two lines of code below.
-Line 1: Search for all .rdp files in the folder structure and store the full path with file name in a variable. Every file will be checked since the users tend to accidentally change file names, eliminating absolute predictability.
-Line 2: I want to make one pass through all the files to replace only instances of two particular IP addresses with the new address. Then write the changes into the original file.
$Path = ls 'C:\Scripts\Replace-RDP\TESTFILES\' -Include *.rdp -Recurse -Force -ErrorAction SilentlyContinue | foreach fullname
$Path | (Get-Content -Path $Path) -Replace 'IPserver1','newIPserver1' -Replace 'IPserver2','newIPserver2' | Set-Content $Path -Force
Have found most of the solution with Powershell but have a problem with the results. The second line of code when output to the screen changes contents correctly in memory. The content written to file however resulted in the new server IP address being written into ALL rdp files even if the source rdp file's target IP address doesn't match the -Replace criterion.
Text inside a .rdp on the relevant line is:
full address:s:192.168.1.123
changes to:
full address:s:172.16.1.23
Thank you for all assistance in reaching the endpoint. Have spent hours learning from various sites and code snippets.
You need to keep track of each file that you are reading so that you can save changes to that file. Foreach-Object makes this process easy. Inside of the Foreach-Object script block, the current object $_ is the FullName value for each of your files.
$CurrentIP1 = '192\.168\.1\.123'
$CurrentIP2 = '192\.168\.1\.124'
$NewIP1 = '172.16.1.23'
$NewIP2 = '172.16.1.24'
$files = (Get-ChildItem 'C:\Scripts\Replace-RDP\TESTFILES\' -Filter *.rdp -Recurse -Force -File -ErrorAction SilentlyContinue).FullName
$files | Foreach-Object {
if (($contents = Get-Content $_) -match "$CurrentIP1|$CurrentIP2") {
$contents -replace $CurrentIP1,$NewIP1 -replace $CurrentIP2,$NewIP2 |
Set-Content $_
}
}
Note that using the -File switch on Get-ChildItem (alias ls) outputs only files. Since -replace uses regex to do matching, you must backslash escape literal . characters.
I am attempting to search a directory of perl scripts and compile a list of all the other perl scripts executed from those files(intentionally trying to do this through Powershell). A simplistic dependency mapper, more or less.
With the below line of code I get output of every line where a reference to a perl file is found, but what I really need is same output AND the file in which each match was found.
Get-Content -Path "*.pl" | Select-String -Pattern '\w+\.pl' | foreach {Write-Host "$_"}
I have succeeded using some more complicated code but I think I can simplify it and accomplish most of the work through a couple lines of code(The code above accomplishes half of that).
Running this on a windows 10 machine powershell v5.1
I do things like this all the time. You don't need to use get-content.
ls -r *.pl | Select-String \w+\.pl
file.pl:1:file2.pl
You don't need to use ls or Get-ChildItem either; Select-String can take a path parameter:
Select-String -Pattern '\w+\.pl' -Path *.pl
which shortens to this in the shell:
sls \w+\.pl *.pl
(if your regex is more complex it might need spaces around it).
For the foreach {write-host part, you're writing a lot of code to turn useful objects back into less-useful strings, and forcibly writing them to the host instead of the standard output stream. You can pick out the data you want with:
sls \w+\.pl *.pl | select filename, {$_.matches[0]}
which will keep them as objects with properties, but render by default as a table.
I am attempting to find all file names in a root directory that contain a ~ so they can be changed.
I have tried using the escape character (-filter "~" instead of -filter "~"), however when doing that it grabs nothing.
I have also tried putting the ~ in a string variable, but that also did not work, it just pulled the entire root folder again
I know there are about 200 files that have ~ in their name in the test directory, so this should grab 200 files.
[string] $rootfolder = "C:\Documents\Powershell-Test",
[string] $folder = "",
[string] $subfolder = "",
#Place Illegal Character Here
[string] $illegalCharacter = "~",
#place replacement Phrase here
[string] $replacementPhrase = "",
$files = Get-ChildItem -Path $rootfolder -Recurse -Filter *$illegalCharacter*
I use the same script I used to find other unwanted characters. When I ran this script for # and &, it grabbed all the files with the character, remo and printed them to a csv file.
When I ran the ~, it grabbed every file in the root folder.
I did not include the code to print the file because that code is not altered when changing what is being searched for.
-Filter qualifies the path parameter. This means you need wildcards if you don't know the exact string name of target directory or file. If you don't know where the ~ exists in the file names, you need to use *~* when qualifying the path. Per Matthias R. Jessen, -Filter "*~*" will not produce the desired result because *~* matches every single file with 8 or more characters in the name because PowerShell offloads filtering to Windows, and Windows applies the filter to both the real name and the 8dot3 name of the file system items.
The following will produce the desired result. Note this was tested on PowerShell v5.1.
$rootfolder = "C:\Documents\Powershell-Test"
$illegalCharacter = '~'
$files = Get-ChildItem -Path $rootfolder -Recurse |
Where-Object {$_.Name -like "*$IllegalCharacter*"}
Putting the wildcards in the path works for me. The -filter *~* matching short filenames thing doesn't happen in PS 6, so it appears to be a bug.
get-childitem -path $rootfolder\*$illegalCharacter* -recurse
It even matches the number in the short filename.
cmd /c dir /x
07/17/2019 02:19 PM 10 VERYLO~2.TXT verylongname.txt
ls . *2*
-a---- 7/17/2019 2:19 PM 10 verylongname.txt
"get-childitem -filter" seems like something to be avoided, like ">".
I am looking for a reliable command-line method of getting SHA256 hashes for files in Windows. My understanding is that the way to do this is via Microsoft's Get-FileHash cmdlet under PowerShell. I have seen several web sites with examples and reviewed Microsoft's own documentation. It appears that the following syntax should work on Windows Server 2012:
Get-FileHash myfile.txt -Algorithm SHA256
The command runs without error, but there is no output. If I send the output to a file, the file is created with no content. I have also seen examples which pipe the output to Format-List; I tried that, but still nothing. I have also tried running the command with invalid arguments, and again nothing.
I am open to using a different program, but due to business requirements, it would need to be a supported download.
I'm using PowerShell 4.0 and I just encountered the same problem of null output from Get-FileHash. The cause of my problem is different than the OP but I have found a solution to my problem and I figured I would post my findings for anyone who came to this page trying to solve the problem of null output (or seemingly incorrect output) from Get-FileHash.
The problem only happens (for me) when the path to the target file contains brackets [ ] and those brackets contain either zero characters or 2 or more characters.
EDIT: I now understand WHY this happens. The string is interpreted as Regular Expression (RegEx) so the square brackets [ ] take on their special RegEx meaning. The -LiteralPath tells PowerShell to interpret the string as a simple match (no RegEx).
Consider the following paths which refer to 4 existing text files (hypothetically):
C:\Test\My Text.txt
C:\Test\My [Text].txt
C:\Test\My [Te]xt.txt
C:\Test\My Text[].txt
The following command produces normal output:
Get-FileHash "C:\Test\My Text.txt"
but there will be null output if using the following commands:
Get-FileHash "C:\Test\My [Text].txt"
Get-FileHash "C:\Test\My [Te]xt.txt"
Get-FileHash "C:\Test\My Text[].txt"
This can be solved by using the -LiteralPath switch. For example:
Get-FileHash -LiteralPath "C:\Test\My [Text].txt"
Variables are expanded normally when using the -LiteralPath switch. For example:
(Get-ChildItem C:\Test).FullName | ForEach {
Get-FileHash -LiteralPath $_
}
If there is exactly 1 character between the brackets, the brackets will be ignored when using Get-FileHash.
Consider the following paths which refer to 3 existing text files (hypothetically), each with unique hash values:
C:\Test\My Text.txt
C:\Test\My Tex[t].txt
C:\Test\My[ ]Text.txt
Get-FileHash interprets all three of the following commands in exactly the same way ( the path is interpreted as C:\Test\My Text.txt ) and therefore each command has the exact same output despite each file having it's own unique hash value:
Get-FileHash "C:\Test\My Text.txt"
Get-FileHash "C:\Test\My Tex[t].txt"
Get-FileHash "C:\Test\My[ ]Text.txt"
P.S. I'm a very new programmer, please forgive me for any poor usage of terminology.
Get-FileHash, requires Windows PowerShell 4.0
Based on your comments you are at version 3, which is default on Win 2012 (non R2) Here how to check you PS version
You can update PS on Win 2012 (non R2) to version 4.0 or use Win 2012 R2
If you just run Get-FileHash on a PS version 3 system you should get
PS C:\> Get-FileHash
Get-FileHash : The term 'Get-FileHash' is not recognized as the name of a cmdlet, function, script file, or operable
program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Get-FileHash
+ ~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (Get-FileHash:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
On lower PS version you can use this function
Function Get-FileHashTSO([String] $FileName,$HashName = "SHA1")
{
$FileStream = New-Object System.IO.FileStream($FileName,[System.IO.FileMode]::Open)
$StringBuilder = New-Object System.Text.StringBuilder
[System.Security.Cryptography.HashAlgorithm]::Create($HashName).ComputeHash($FileStream)|%{[Void]$StringBuilder.Append($_.ToString("x2"))}
$FileStream.Close()
$StringBuilder.ToString()
}
store it as .ps (e.g. Get-FileHashTSO.ps1) file and call it like this
powershell -command "& { . C:\myScripts\Get-FileHashTSO.ps1 ; Get-FileHashTSO "C:\someLocation\someFile.iso" "SHA1" }"
I calculated the hash of all files on a drive and exported them to a .csv by using this:
Get-ChildItem C: -Recurse |
Get-FileHash |
Export-Csv -Path C:\Users\yourname\Documents\Output\hashes.csv -NoTypeInformation
Import-Csv -Path C:\Users\yourname\Documents\Output\hashes.csv
Why it works (I think):
Get-ChildItem <--gets everything under your path, in this case C:, and -Recurse gets all the files within folders. You can add limitations if needed.
Get-FileHash <--after you've said get these files, you're saying calculate the hashes
Export-Csv <--says, we're sending your hashes out as comma separated values file, which is crazy helpful, and -Path says put it HERE, -NoTypeInformation just removes the #TYPE row from the top of the .csv file, and versions of PowerShell before 6 need this.
Import-Csv <--says, bring that data into the file at this -Path
Be sure to have the .csv file created in that location before you run the script. The script won't create the container for you. There's no need to clear the data from the .csv file between runs, it clears itself.
I'm not really a programmer, hence the annoyingly lengthy explanation. Hope it helps others. Wouldn't have figured it out without Stack Overflow forums! Thanks everyone!!
I'm using PowerShell on Windows 7, and writing a script to copy a bunch of files from one folder structure to another. Kind of like compiling. The PowerShell Copy-Item cmdlet thinks that square brackets, [ ], are wildcards of some kind, and I am not able to escape them for some reason.
I can't use -LiteralPath, because I want to use an asterisk * wildcard since the filename has a date as part of the filename, and the date changes. The date is used as a version number.
This post was helpful, but no amount of ticks (2x or 4x per bracket) escapes the square brackets.
I am not receiving an error; PowerShell behaves the same as if I entered in the wrong filename.
This is the specific line I'm working on:
#to Fusion Server
Copy-item -Path $FSG\$SW\0.RoomView.Notes\starter\"[RoomView] Versions explained*.pdf" -Destination $FSG\$containerFolder\$rootFolder\"Fusion Server"\
And this is the whole thing:
# Compiles the Fusion packet for distribution
###############################
###########Variables###########
###############################
#folder structure
$FSG = "F:\FSG"
$containerFolder = "Packet.Fusion for IT and AV Professionals"
$rootFolder = "Fusion for IT and AV pros $(Get-Date -format “MM-dd-yyyy”)"
$subRoot1 = "Fusion Server"
$subRoot2 = "Scheduling Enhancement and Panels"
$subRoot2sub1 = "Scheduling Panels"
$subRoot3 = "SQL Server"
#source folders
$HW = "0.Hardware"
$3SMDoc = "0.Hardware\TPMC-3SM.Documentation"
$4SMDoc = "0.Hardware\TPMC-4SM.Documentation"
$4SMDDoc = "0.Hardware\TPMC-4SM-FD.Documentation"
$730Doc = "0.Hardware\TSW-730.Documentation"
$730OLH = "0.Hardware\TSW-730.OLH"
$CENRVS = "0.Hardware\CEN-RVS.Notes"
$ProjMgmt = "0.Project Management"
$SW = "0.Software"
$RVLicensing = "0.Software\0.RoomView.License"
$RVNotes = "0.Software\0.RoomView.Notes"
$SQLLicensing = "0.Software\database.SQL.Licensing"
$SQLNotes = "0.Software\database.SQL.Notes"
$FRVMarketing = "0.Software\Fusion RV.Marketing"
$FRVNetworking = "0.Software\Fusion RV.Networking"
$FRVNotes = "0.Software\Fusion RV.Notes"
###############################
#create the directory structure
###############################
md -Path $FSG\$containerFolder -Name $rootFolder
cd $FSG\$containerFolder\$rootFolder
md "eControl and xPanels"
md "Fusion Server" #$subRoot1
md "Getting Started as a User"
md "Project Management"
md "RoomView Connected Displays"
md "Scheduling Enhancement and Panels" #$subRoot2
md "SQL Server" #$subRoot3
cd $FSG\$containerFolder\$rootFolder\$subRoot1
md "CEN-RVS"
md "Licenseing Information"
md "Networking"
md "Official Documentation"
md "Prerequisites, including powerShell script"
md "Product Info"
md "Requirements"
md "Tech Info"
md "Windows Authentication to Fusion RV"
cd $FSG\$containerFolder\$rootFolder\$subRoot2
md "Outlook Add-in"
md "Scheduling Panels" #$subRoot2sub1
cd $FSG\$containerFolder\$rootFolder\$subRoot2\$subRoot2sub1
md "TPMC-3SM"
md "TPMC-4SM"
md "TPMC-4SM-FD"
md "TSW-730"
cd $FSG\$containerFolder\$rootFolder\$subRoot3
md "Multi-database model only"
md "SQL Licensing"
cd $FSG\$containerFolder
#reset current folder
###############################
#copy the files
###############################
#Copy-Item -Path C:\fso\20110314.log -Destination c:\fsox\mylog.log
#To the root
Copy-item -Path $FSG\$ProjMgmt\starter\"Fusion Support Group Contact info*.pdf" -Destination $FSG\$containerFolder\$rootFolder\
Copy-item -Path $FSG\$containerFolder\"Fusion for IT and AV professionals release notes.txt" -Destination $FSG\$containerFolder\$rootFolder\
#to eControl and xPanels
Copy-item -Path $FSG\$SW\xpanel.Notes\starter\*.* -Destination $FSG\$containerFolder\$rootFolder\"eControl and xPanels"\
#to Fusion Server
Copy-item -Path $FSG\$SW\0.RoomView.Notes\starter\"[RoomView] Versions explained*.pdf" -Destination $FSG\$containerFolder\$rootFolder\"Fusion Server"\
What can I do to escape the square brackets and still use a wildcard filename part of the Copy-Item cmdlet?
In this situation, you have to use double-backticks with single quotes in order to escape the brackets. You can also use quadruple backticks when you use double quoted strings.
So the fixed line of code is:
Copy-item -Path $FSG\$SW\0.RoomView.Notes\starter\'``[RoomView``] Versions explained*.pdf' -Destination $FSG\$containerFolder\$rootFolder\'Fusion Server'\
Another good resource on file paths and wired characters etc. is to read this article: Taking Things (Like File Paths) Literally
EDIT
Thanks to #mklement0 for highlighting that the true cause of this
inconsistency is because of a bug currently in
PowerShell1.
This bug causes escaping of wildcard characters, as well as backticks
with the default -Path parameter to behave differently than other
parameters e.g. the -Include and -Filter parameters.
To expand on #mklement0's excellent answer, and comments, and other answers below:
To better understand why we need single quotes and two back ticks in this situation; (and to highlight the bug and inconsistencies) let's run through some examples to demonstrate what is going on:
Get-Item, and associated cmdlets (Get-ChildItem, Copy-Item, etc.), handle the -Path parameter differently when dealing with a combination of escaped wildcard characters and unescaped wildcard characters *at the same time***!
TLDR: The underlying reason that we need a combination of single quotes and double backticks is how the underlying PowerShell provider parses the -Path parameter string for wildcards. It appears to parse it once for the escape characters, and a second time for the evaluation of the wildcard.
Let's go through some examples to demonstrate this odd outcome:
First, let's create two files to test with called File[1]a.txt and File[1]b.txt
"MyFile" | Set-Content '.\File`[1`]a.txt'
"MyFriend" | Set-Content '.\File`[1`]b.txt'
We'll try different ways to get the file. We know that Square brackets [ ] are wildcards, and so we need to escaped them with the backtick character.
We will try to get one file explicitly.
Let's start by using single quoted literal strings:
PS C:\> Get-Item 'File[1]a.txt'
PS C:\> Get-Item 'File`[1`]a.txt'
Directory: C:\
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 2019-09-06 5:42 PM 8 File[1]a.txt
PS C:\> Get-Item 'File``[1``]a.txt'
Directory: C:\
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 2019-09-06 5:42 PM 8 File[1]a.txt
For single quoted strings, one backtick is all that is required to retrieve the file, but two backticks also work.
Using Double quoted strings we get:
PS C:\> Get-Item "File[1]a.txt"
PS C:\> Get-Item "File`[1`]a.txt"
PS C:\> Get-Item "File``[1``]a.txt"
Directory: C:\
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 2019-09-06 5:42 PM 8 File[1]a.txt
For double quoted strings, as expected, we can see that we need two backticks to make it work.
Now, we want to retrieve both files and use a wildcard.
Let's start with single quotes:
PS C:\> Get-Item 'File[1]*.txt'
PS C:\> Get-Item 'File`[1`]*.txt'
PS C:\> Get-Item 'File``[1``]*.txt'
Directory: C:\
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 2019-09-06 5:42 PM 8 File[1]a.txt
-a---- 2019-09-06 5:49 PM 10 File[1]b.txt
With the single quotes, when we have a wildcard character, we need two sets of backticks. One to escape the bracket, and a second backtick to escape the backtick that we used to escape the bracket when the wildcard is evaluated.
Similarly for double quotes:
PS C:\> Get-Item "File[1]*.txt"
PS C:\> Get-Item "File`[1`]*.txt"
PS C:\> Get-Item "File``[1``]*.txt"
PS C:\> Get-Item "File```[1```]*.txt"
PS C:\> Get-Item "File````[1````]*.txt"
Directory: C:\
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 2019-09-06 5:42 PM 8 File[1]a.txt
-a---- 2019-09-06 5:49 PM 10 File[1]b.txt
With double quotes it's a little more verbose to evaluate with a wildcard. In this case, we need four sets of back ticks. For double quotes we need two backticks to escape the bracket, and another two backticks to escape the escape characters once it comes to evaluation of the star wildcard.
EDIT
As #mklement0 mentions, this behavior with the -Path parameter is inconsistent, and behaves differently than the -Include parameter, where only a single backtick is required to properly escape the brackets. This may be "fixed" in a later version of PowerShell.
1 As of Windows PowerShell v5.1 / PowerShell Core 6.2.0-preview.3
An overview and some background information:
In order to effectively escape a character that you want to be interpreted verbatim as part of a wildcard expression, it must be `-escaped as seen by the target cmdlet (its underlying PowerShell drive provider).
Ensuring that can get tricky, because ` (backtick) is also used as the escape character in double-quoted strings ("...") and unquoted command arguments (which for the most part behave like double-quoted strings).
Note: The scenario in the question doesn't allow use of -LiteralPath, but in cases where you know a path to be a concrete, literal path, use of the -LiteralPath (which can be shorted to -lp in PowerShell Core) is the best choice - see this answer.
When passing an argument to the wildcard-supporting -Path parameter of a PowerShell drive provider-related cmdlet (Get-ChildItem, Copy-Item, Get-Content, ...) and you want [ and ] to be treated verbatim rather than as a character set/range expression:
String-literal representations:
'file`[1`].txt'
` chars. are preserved as-is inside '...', so the target cmdlet sees them, as intended.
"file``[1``].txt"
``, i.e. doubling is needed inside "..." in order to preserve a single ` in the resulting string (the first ` is the (double-quoted) string-internal escape character, and the second ` is the character it escapes, to be passed through).
file``[1``].txt
Ditto for unquoted command arguments, which (for the most part) act like "..."
Caveat: Due to a bug - see this GitHub issue - mixing (unescaped) ? or * with escaped [ and ] requires the latter to be doubly escaped (with ``, as seen by the target cmdlet / provider):
If you wanted to match literal filename file[1].txt with a wildcard pattern that matches [ and ] literally while also containing special character * (to match any run of characters), instead of the expected 'file`[1`]*', you'll have to use 'file``[1``]*' (sic); with a double-quoted or unescaped argument you then have to effectively use quadruple backticks: "file````[1````]*" / file````[1````]* - see this answer for more.
Note that direct use of wildcards with the -like operator is not affected:
'a[b' -like 'a`[*' is - correctly - $true,
whereas 'a[b' -like 'a``[*' - rightfully - complains about an invalid pattern.
Similarly, parameters -Include and -Exclude are not affected.
-Filter plays by different rules to begin with: [...] as a construct isn't supported at all, and [ and ] chars. are always considered literals (again, see this answer).
To escape a path string programmatically, via a variable, use:
$literalName = 'file[1].txt'
$escapedName = [WildcardPattern]::Escape($literalName) # -> 'file`[1`].txt'
I use this:
Copy-Item $file.fullname.replace("[", "``[").replace("]", "``]") $DestDir
The way that Powershell automatically tab-completes the filename is usually the best way,
Example:
copy-item '.\file`[test`].txt'
On PowerShell v 2.0 and up the escape character to use is the backslash. For example, if we want to remove the brackets from this string "[Servername: QA01]" which is the sort of output we get from the Exchange Admin PowerShell cmdlet activity in System Center Orchestrator, we use the following logic:
$string -replace '\[','' -replace '\]',''
>Servername: QA01
This is pretty weird. See, you have to use a single-quote (which normally implies in PowerShell 'evaluate this precisely as written', so this is very odd syntax).
Don't feel bad for not figuring this out on your own, this is very odd syntax.
Apparently, square brackets need double-backticks to escape, which is unusual. Reference here.
You're sure that doesn't work? I've seen it referred to a few times.
Edit: Yes, it works, you used double quotes instead of backticks.
Double quote is above the apostrophe character, next to the Enter key. Backtick is right underneath the Escape key, sharing the key with the tilde, ~.
One option is to get the filenames using the legacy dir, which will let you use the * wildcard character, but doesn't try to "blob" the square brackets. Then feed that list to move-item using -literalpath
cmd /c dir *]* /b |
foreach { Move-Item -LiteralPath $_ -Destination <destination path> }
Assuming nothing else matches, you can use ? instead of the brackets. A file named "a[h-j]", copying to directory "foo":
copy-item a?h-j? foo
There's a difference between ' and `:
The first is the single quote that is the non-shift character on the " key.
The second is the backtick that I thought I was using but actually wasn't. It's the nonshift character on the ~ key.
This works:
# to Fusion Server
Copy-item -Path $FSG\$SW\0.RoomView.Notes\starter\'``[RoomView``] Versions explained*.pdf' -Destination $FSG\$containerFolder\$rootFolder\"Fusion Server"\