Retrieving second part of a line when first part matches exactly - powershell

I used the below steps to retrieve a string from file
$variable = 'abc#yahoo.com'
$test = $variable.split('#')[0];
$file = Get-Content C:\Temp\file1.txt | Where-Object { $_.Contains($test) }
$postPipePortion = $file | Foreach-Object {$_.Substring($_.IndexOf("|") + 1)}
This results in all lines that contain $test as a substring. I just want the result to contain only the lines that exactly matches $test.
For example, If a file contains
abc_def|hf#23$
abc|ohgvtre
I just want the text ohgvtre

If I understand the question correctly you probably want to use Import-Csv instead of Get-Content:
Import-Csv 'C:\Temp\file1.txt' -Delimiter '|' -Header 'foo', 'bar' |
Where-Object { $_.foo -eq $test } |
Select-Object -Expand bar

To address the exact matching, you should be testing for equality (-eq) rather than substring (.Contains()). Also, there is no need to parse the data multiple times. Here is your code, rewritten to to operate in one pass over the data using the -split operator.
$variable = 'abc#yahoo.com'
$test = $variable.split('#')[0];
$postPipePortion = (
# Iterate once over the lines in file1.txt
Get-Content C:\Temp\file1.txt | foreach {
# Split the string, keeping both parts in separate variables.
# Note the backslash - the argument to the -split operator is a regex
$first, $second = ($_ -split '\|')
# When the first half matches, output the second half.
if ($first -eq $test) {
$second
}
}
)

Related

PowerShell - Find and replace multiple patterns to anonymize file

I need you help. I have a log.txt file with various data in it which I have to anonymize.
I would like to retrieve all these "strings" matching a predefined patterns, and replace these by another values for each of them. What is important is that each new string from the same pattern (and with different value from the previous) should be replaced by the predefined value increased by +1 (e.g. "orderID = 123ABC" becomes "orderID = order1" and "orderID=456ABC" becomes "orderID=order2").
The patterns to search for are more than 20 so it is not possible to put them all in single line.
My idea is:
Define "patterns.txt" file
Define "replace.txt" file ("pattern" value and replacement value)
Search for all "patterns" in the log file, the result will be ARRAY
Find the unique entries in that ARRAY
Get the "replacement" value for each unique entry in the ARRAY
Replace all occurrences in log.txt. The tricky part here is that any occurrence of the same type (but different value from the previous one) needs to be incremented by (+1) in order to be different from the one before.
Example of what I have :
requestID>qwerty1-qwerty2-qwerty3</requestID
requestID>12345a-12345b-12345c</requestID
requestID>qwerty1-qwerty2-qwerty3</requestID
requestID>qwerty1-qwerty2-qwerty3</requestID
orderID>012345ABCDE</orderID
orderID>012345ABCDE</orderID
orderID>ABCDE012345</orderID
orderID>ABCDE012345</orderID
keyId>XYZ123</keyId
keyId>ABC987</keyId
keyId>XYZ123</keyId
Desired result:
requestID>Request-1</requestID
requestID>Request-2</requestID
requestID>Request-1</requestID
requestID>Request-1</requestID
orderID>Order-1</orderID
orderID>Order-1</orderID
orderID>Order-2</orderID
orderID>Order-2</orderID
keyId>Key-1</keyId
keyId>Key-2</keyId
keyId>Key-1</keyId
For the moment I managed only to find the unique values per type:
$N = "C:\FindAndReplace\input.txt"
$Patterns = "C:\FindAndReplace\pattern.txt"
(Select-String $N -Pattern 'requestID>\w{6}-\w{6}-\w{6}</requestID>').Matches.Value | Sort-Object -Descending -Unique
(Select-String $N -Pattern '<orderID>\w{20}</orderID>').Matches.Value | Sort-Object -Descending -Unique
(Select-String $N -Pattern '<keyId>\w{8}</keyId>').Matches.Value | Sort-Object -Descending -Unique
Thanks in advance for any suggestion on how to progress.
Your patterns don't match your sample data. I've corrected the patterns to accommodate the actual sample data.
It seems a simple hash table per type would fulfill the need to keep track of matches and counts. If we process the log file with a switch statement using the -Regex and -File parameters we can work on each line at a time. The logic for each is
Check if the current match exists in the specific type's match array.
If not, add it with it's replacement value (type-count) and increment count.
If it does exist, use the already defined replacement value.
Capture all the output in a variable and then write it out to file when done.
Create the example log file
$log = New-TemporaryFile
#'
<requestID>qwerty1-qwerty2-qwerty3</requestID> -match
<requestID>12345a-12345b-12345c</requestID>
<requestID>qwerty1-qwerty2-qwerty3</requestID>
<requestID>qwerty1-qwerty2-qwerty3</requestID>
<orderID>012345ABCDE</orderID>
<orderID>012345ABCDE</orderID>
<orderID>ABCDE012345</orderID>
<orderID>ABCDE012345</orderID>
<keyId>XYZ123</keyId>
<keyId>ABC987</keyId>
<keyId>XYZ123</keyId>
'# | Set-Content $log -Encoding UTF8
Define "tracker" variables for each type containing the count and a matches array
$Request = #{
Count = 1
Matches = #()
}
$Order = #{
Count = 1
Matches = #()
}
$Key = #{
Count = 1
Matches = #()
}
Read and process the log file line by line
$output = switch -Regex -File $log {
'<requestID>(\w{6,7}-\w{6,7}-\w{6,7})</requestID>' {
if(!$Request.matches.($matches.1))
{
$Request.matches += #{$matches.1 = "Request-$($Request.count)"}
$Request.count++
}
$_ -replace $matches.1,$Request.matches.($matches.1)
}
'<orderID>(\w{11})</orderID>' {
if(!$Order.matches.($matches.1))
{
$Order.matches += #{$matches.1 = "Order-$($Order.count)"}
$Order.count++
}
$_ -replace $matches.1,$Order.matches.($matches.1)
}
'<keyId>(\w{6})</keyId>' {
if(!$Key.matches.($matches.1))
{
$Key.matches += #{$matches.1 = "Key-$($Key.count)"}
$Key.count++
}
$_ -replace $matches.1,$Key.matches.($matches.1)
}
default {$_}
}
$output | Set-Content $log -Encoding UTF8
The $log file now contains
<requestID>Request-1</requestID>
<requestID>Request-2</requestID>
<requestID>Request-1</requestID>
<requestID>Request-1</requestID>
<orderID>Order-1</orderID>
<orderID>Order-1</orderID>
<orderID>Order-2</orderID>
<orderID>Order-2</orderID>
<keyId>Key-1</keyId>
<keyId>Key-2</keyId>
<keyId>Key-1</keyId>

Compare the contents of two files and output the the differences in contents along with line numbers

I came upon the problem where we need to compare contents two files a.txt and b.txt line by line and output the result if any difference found along with content and line number.
We should not use Compare-Object in this scenario. Do we have any alternative?
I tried using for loops but unable to get desired result
For ex : a.txt:
Hello = "Required"
World = 5678
Environment = "new"
Available = 9080.90
b.txt"
Hello = "Required"
World = 5678.908
Environment = "old"
Available = 6780.90
I need to get the output as:
Line number 2:World is not matching
Line number 3:Environment is not matching
Line number 4:Available is not matching
I tried with the following code snippet but was unsuccessful
$file1 = Get-Content "C:\Users\Desktop\a.txt"
$file2 = Get-Content "C:\Users\Desktop\b.txt"
$result = "C:\Users\Desktop\result.txt"
$file1 | foreach {
$match = $file2 -match $_
if ( $match ){
$match | Out-File -Force $result -Append
}
}
As you seem to have an adverse reaction to Compare-Object, lets try this extremely janky set-up. As you have little to no requirements listed, this will give you the bare minimum to meet your conditions of 'any difference found'.
Copy and paste more If statements should you have more lines.
$a = get-content C:\a.txt
$b = get-content C:\b.txt
If($a[0] -ne $b[0]) {
"Line number 1:Hello is not matching" | Out-Host
}
If($a[1] -ne $b[1]) {
"Line number 2:World is not matching" | Out-Host
}
If($a[2] -ne $b[2]) {
"Line number 3:Environment is not matching" | Out-Host
}
If($a[3] -ne $b[3]) {
"Line number 4:Available is not matching" | Out-Host
}
Get-Content returns the file content as an array of strings with a zero based index.
The array variable has an automatic property .Count/.Length
you can use to iterate the arrays with a simple counting for.
You need to split the line at the = to separate name and content.
Use -f format operator to output the results.
## Q:\Test\2019\05\21\SO_56231110.ps1
$Desktop = [environment]::GetFolderPath('Desktop')
$File1 = Get-Content (Join-Path $Desktop "a.txt")
$File2 = Get-Content (Join-Path $Desktop "b.txt")
for ($i=0;$i -lt $File.Count;$i++){
if($File1[$i] -ne $File2[$i]){
"Line number {0}:{1} is not matching" -f ($i+1),($File1[$i] -split ' = ')[0]
}
}
Sample output:
Line number 2:World is not matching
Line number 3:Environment is not matching
Line number 4:Available is not matching

Splitting in Powershell

I want to be able to split some text out of a txtfile:
For example:
Brackets#Release 1.11.6#Path-to-Brackets
Atom#v1.4#Path-to-Atom
I just want to have the "Release 1.11.6" part. I am doing a where-object starts with Brackets but I don't know the full syntax. Here is my code:
"Get-Content -Path thisfile.txt | Where-Object{$_ < IM STUCK HERE > !
You could do this:
((Get-Content thisfile.txt | Where-Object { $_ -match '^Brackets' }) -Split '#')[1]
This uses the -match operator to filter out any lines that don't start with Brackets (the ^ special regex character indicates that what follows must be at the beginning of the line). Then it uses the -Split operator to split those lines on # and then it uses the array index [1] to get the second element of the split (arrays start at 0).
Note that this will throw an error if the split on # doesn't return at least two elements and it assumes that the text you want is always the second of those elements.
$bracketsRelease = Get-Content -path thisfile.txt | foreach-object {
if ( $_ -match 'Brackets#(Release [^#]+)#' )
{
$Matches[1]
}
}
or
(select-string -Path file.txt -Pattern 'Brackets#(Release [^#]+)#').Matches[0].Groups[1].value

powershell split keeping only the 2 first part of string

i'm new in powershell
i have a file that contains OU paths separates by "/"
i'd like to only keep the 2 first element of each string
example:
paris/sales/salers/14tharrdt
london/comptability/office1
would give
paris/sales
london/comptability
i googled and found many things, but no way to do this simple thing
thanks for help
You could use -split to split the string on the \ character, select the first two elements and join them together:
$content = Get-Content 'your_file_path'
$content | foreach {
($_ -split '/')[0, 1] -join '/'
}
$content | Set-Content 'your_file_path'
Or, if you prefer using regex here a solution without split (demo here):
$content = Get-Content 'your_file_path'
$content | foreach {
$_ -replace '(.*?\/[^\/]+).*', '$1'
}
$content | Set-Content 'your_file_path'
Like:
$a = "paris/sales/salers/14tharrdt"
$b = $a.Split("/")[0] + "/" + $a.Split("/")[1]
$b
Improvement would be if call Split once and save it to a variable

Remove Duplicate Group of Data in Text file

I have a text file formatted similar to the following:
Description1: Data-123<br>
Description2: Data-ABC<br>
Description3: Data-789<br>
Description4: Data-EFG<br>
Description5: Data-XYZ<br>
Description1: Data-123<br>
Description2: Data-ABC<br>
Description3: Data-789<br>
Description4: Data-EFG<br>
Description5: Data-XYZ<br>
Description1: Data-123<br>
Description2: Data-ABC<br>
Description3: Data-789<br>
Description4: Data-EFG<br>
Description5: Data-584<br>
I need PowerShell to compare each group (5 lines of data) as a whole and remove any duplicate groups, leaving only the unique groups of data. I can get it to remove single duplicate lines with the code below, but no luck comparing each group.
get-content TextFile.txt | sort-object | get-unique > NewTextFile.txt
Maybe this can work, you need to create the output file based on the result of last line of code, anyway I give no explanation because you don't show us any code you have so far.
$a = gc mylist.txt
$b = [string]::Empty
$c = #()
$a | % {if ( $_ -ne [string]::Empty )
{ $b += "$_`n" }
else
{ $c += $b
$b = [string]::Empty
}
}
$c += $b
$c | select -Unique | out-file .\mynew.txt
Split the file content on double new line characters (that should match the end of the line right before the empty line + the empty line right after it), split each object returned (remove the empty line) and then join it back, add new line and write the results to a new file.
(Get-Content TextFile.txt | Out-String) -split "`r`n`r`n" | ForEach-Object{
($_.Split("`r`n",[System.StringSplitOptions]::RemoveEmptyEntries) -join "`r`n") + "`n"
} | Select-Object -Unique | Out-File NewTextFile.txt