How to delete lines containing a specific character in a text file with PowerShell?
Input file example:
12314+000000 00000000
20 300 1238238948023
A+A+000000 00000000
20 500 1238238948023
Output file example:
20 300 1238238948023
20 500 1238238948023
So it will delete lines containing + or specific characters or words.
The simplest way is to use new file to write data in it:
Get-Content "D:\test.txt" | Where-Object {$_ -notmatch '\+'} | Set-Content "D:\out.txt"
That will give:
20 300 1238238948023
20 500 1238238948023
In out.txt.
This is used in this answer and this
Related
I need to read multiple files and count the no. of lines in each file. Then write all this information in a single CSV file with the file path information.
I could write a code (with help from forums) that will help to read all files and print the information as required.
Script:
Get-ChildItem -Path C:\Logs\test *.log | Select-Object -Property FullName,#{Name="LineCount";$Expression={#(Get-Content -Path $_.FullName).Length}}
Output of script:
FullName LineCount
-------- ---------
C:\Logs\test\ANCD065.log 296
C:\Logs\test\ANCE066.log 287
Problems:
The problem is not able to write the above information in a CSV file.
Write "what parameter need pass not clear to me" | Out-File C:\Log\OP.CSV
The script reads all lines, is there a way to start reading line after certain lines? An example in the given file need is count lines from line 9 (start) to the end(could be 300).
1. 3.03 MET DATA RINEX VERSION / TYPE
2. cnvtToRINEX 3.14.0 convertToRINEX OPR 20220511 072939 UTC PGM / RUN BY / DATE
3. ----------------------------------------------------------- COMMENT
4. XYXTY3 MARKER NAME
5. UBWTZ3 MARKER NUMBER
6. 3 PR TD HR # / TYPES OF OBSERV
7. 0083019.4060 0025010.0967 0253356.6176 0.0000 PR SENSOR POS XYZ/H
8. END OF HEADER
9. 19 03 02 00 00 00 946.0 8.5 93.0
10. 19 03 02 00 05 00 946.0 8.4 93.4
11. 19 03 02 00 10 00 946.0 8.4 93.4
12. 19 03 02 00 15 00 946.0 8.4 94.2
13. 19 03 02 00 20 00 946.0 8.5 93.1
14. 19 03 02 00 25 00 946.1 8.7 90.2
15. 19 03 02 00 30 00 946.2 8.4 92.0
16. 19 03 02 00 35 00 946.3 8.3 93.2
The problem is not able to write the above information in a CSV file.
This should be as simple as appending an Export-Csv command to the pipeline:
Get-ChildItem -Path C:\Logs\test *.log |
Select-Object -Property #(
'FullName'
#{ Name = "LineCount"; Expression = {
#(Get-Content -Path $_.FullName).Length
}}
) |
Export-Csv output.csv -NoTypeInformation
Note that my Select-Object changes are just for code readability.
The script reads all lines, is there a way to start reading line after
certain lines? An example in the given file need is count lines from
line 9 (start) to the end(could be 300).
You may use Select-Object -Skip n to skip given number of lines from the beginning. Another improvement is to use Measure-Object to count number of items. In your current code, you have to read the whole log file into an array in memory, before being able to count lines. With Measure-Object, the file will be streamed, requiring much less memory for processing.
(Get-Content -Path $_.FullName | Select-Object -Skip 8 | Measure-Object).Count
Complete solution:
Get-ChildItem -Path C:\Logs\test *.log |
Select-Object -Property #(
'FullName'
#{ Name = "LineCount"; Expression = {
(Get-Content -Path $_.FullName | Select-Object -Skip 8 | Measure-Object).Count
}}
) |
Export-Csv output.csv -NoTypeInformation
I have been using the script below to replace words in multiple text files stored in the same directory but when I want to change numbers (phone numbers +31 85 888 3156 replacing with +1 415 390 5045) I get a specific different number as output +49 800 723 84 68 ???
The words change and replace without any problem but the numbers I cannot figure out.
I am not a programmer but would like to get some feedback on how to find and replace numbers with the script below.
I used a scripts I found on the internet
Get-ChildItem 'path on computer*.html' -Recurse |
ForEach {
(Get-Content $_ | ForEach { $_ -replace '+31 85 888 3156', '+1 415 390 5045'}) | Set-Content $_
}
Every time the same end result. Text is replaced correctly but with the phone numbers I seem to get a calculation.
I have a file which has header at first row and then other data at remaining rows. I want to check whether all rows has equal no of data with header.
For example: if header has 10 count then I want all my remaining rows to have 10 data each so there will be no error while loading the data.
Suppose in line 5 and 6 there are only 5 data each.So,i want to combine these two rows in such case.
My expected output is(Row 5 has the merged data)
There may such breakable data in many rows of the file.So, i just want to scan whole file and will merge the two rows when such cases are seen.
So, I tried using:
$splitway=' '
$firstLine = Get-Content -Path $filepath -TotalCount 1
$firstrowheader=$firstLine.split($splitway,System.StringSplitOptions]::RemoveEmptyEntries)
$requireddataineachrow=$firstrowheader.Count
echo $requireddataineachrow
The above code will give me 10 since my header is having 10 data.
For ($i = 1; $i -lt $totalrows; $i++) {
$singleline=Get-Content $filepath| Select -Index $i
$singlelinesplit=$singleline.split($splitway,[System.StringSplitOptions]::RemoveEmptyEntries)
if($singlelinesplit.Count -lt $requireddataineachrow){
$curr=Get-Content $filepath| Select -Index $i
$next=Get-Content $filepath| Select -Index $i+1
Write-Host (-join($curr, " ", $next))
}
echo $singlelinesplit.Count
}
I tested using Write-Host (-join($curr, " ", $next)) to join two lines but it's not giving the correct output.
echo $singlelinesplit.Count is showing correct result:
My whole data is:
billing summary_id offer_id vendor_id import_v system_ha rand_dat mand_no sad_no cad_no
11 23 44 77 88 99 100 11 12 500
1111 2333 4444 6666 7777777 8888888888 8888888888888 9999999999 1111111111111 2000000000
33333 444444 As per new account ddddddd gggggggggggg wwwwwwwwwww bbbbbbbbbbb qqqqqqqqqq rrrrrrrrr 5555555
22 33 44 55 666<CR>
42 65 66 55 244
11 23 44 76 88 99 100 11 12 500
1111 2333 new document 664466 7777777 8888888888 8888888888888 9999999999 111111144111 200055000
My whole code if needed is:
cls
$filepath='D:\test.txt'
$splitway=' '
$totalrows=#(Get-Content $filepath).Length
write-host $totalrows.gettype()
$firstLine = Get-Content -Path $filepath -TotalCount 1
$firstrowheader=$firstLine.split($splitway,[System.StringSplitOptions]::RemoveEmptyEntries)
$requireddataineachrow=$firstrowheader.Count
For ($i = 1; $i -lt $totalrows; $i++) {
$singleline=Get-Content $filepath| Select -Index $i
$singlelinesplit=$singleline.split($splitway,[System.StringSplitOptions]::RemoveEmptyEntries)
if($singlelinesplit.Count -lt $requireddataineachrow){
$curr=Get-Content $filepath| Select -Index $i
$next=Get-Content $filepath| Select -Index $i+1
Write-Host (-join($curr, " ", $next))
}
echo $singlelinesplit.Count
}
Update: It seems that instances of string <CR> are a verbatim part of your input file, in which case the following solution should suffice:
(Get-Content -Raw sample.txt) -replace '<CR>\s*', ' ' | Set-Content sample.txt
Here's a solution that makes the following assumptions:
<CR> is just a placeholder to help visualize an actual newline in the input file.
Only data rows with fewer columns than the header row require fixing (as Mathias points out, your data is ambiguous, because a column value such as As per new account technically comprises three values, due to its embedded spaces).
Such a data row can blindly be joined with the subsequent line (only) to form a complete data row.
# Create a sample file.
#'
billing summary_id offer_id vendor_id import_v system_ha rand_dat mand_no sad_no cad_no
11 23 44 77 88 99 100 11 12 500
1111 2333 4444 6666 7777777 8888888888 8888888888888 9999999999 1111111111111 2000000000
33333 444444 As per new account ddddddd gggggggggggg wwwwwwwwwww bbbbbbbbbbb qqqqqqqqqq rrrrrrrrr 5555555
22 33 44 55 666
42 65 66 55 244
11 23 44 76 88 99 100 11 12 500
1111 2333 new document 664466 7777777 8888888888 8888888888888 9999999999 111111144111 200055000
'# > sample.txt
# Read the file into the header row and an array of data rows.
$headerRow, $dataRows = Get-Content sample.txt
# Determine the number of whitespace-separated columns.
$columnCount = (-split $headerRow).Count
# Process all data rows and save the results back to the input file:
# Whenever a data row with fewer columns is encountered,
# join it with the next row.
$headerRow | Set-Content sample.txt
$joinWithNext = $false
$dataRows |
ForEach-Object {
if ($joinWithNext) {
$partialRow + ' ' + $_
$joinWithNext = $false
}
elseif ((-split $_).Count -lt $columnCount) {
$partialRow = $_
$joinWithNext = $true
}
else {
$_
}
} | Add-Content sample.txt
I am new to PowerShell and would like to seek your inputs. I have an "Incoming" csv file from which I need to create an "Outgoing" csv file by doing the following.
Copy the contents the "Incoming" file to the "Outgoing" file as-is.
Loop through the column "ID" and for each unique ID, copy the row and append the "Outgoing" file.
Incoming File Sample:
Outgoing File Sample:
Is this doable from PowerShell?
This does what you described:
# Import your data
$incoming = Import-Csv 'c:\folder\incoming.csv'
# 1. Copy Incoming onto Outgoing:
$incoming | Select *,NewColumn | Export-Csv C:\temp\test.csv -Append -NoTypeInformation
# 2. Append the first Incoming row with a unique ID to Outgoing:
$incoming |
Group-Object ID | ## Group rows by ID
Foreach {$_.Group[0]} | ## Select first row per ID
Select *,#{l='NewColumn';e={"Added via script"}} | ## Add column
Export-Csv 'c:\folder\outgoing.csv' -Append -NoTypeInformation
# example incoming:
ID Invoice Paid
-- ------- ----
100 100 150
100 50 150
101 200 200
102 500 850
102 250 850
102 100 850
# creates this outgoing (assuming EMPTY outgoing file to start with):
ID Invoice Paid NewColumn
-- ------- ---- ---------
100 100 150
100 50 150
101 200 200
102 500 850
102 250 850
102 100 850
100 100 150 Added via script
101 200 200 Added via script
102 500 850 Added via script
Note that the Outgoing csv file must already have the NewColumn field added for -Append to work properly if using an existing file. Otherwise your new column will not be exported.
I am collecting some user info from AD and trying to sort it, then remove duplicates, but I can't seem to get sort -unique to work:
foreach ($pute in $puters){
$user = get-loggedinuser -computer $pute.name | select -expandproperty username
$fullname = get-aduser $user | select -expandproperty name
$fullnames += $fullname
}
$fullnames = $fullnames | sort -unique
What this gives me is something like:
Adam Warlock
Adam Warlock
Bruce Biggalow
Frank Coutel
Frank Coutel
Tony Shaloub
Where the output is sorted, but the duplicates remain.
Update: The problem turned out to be the lack of initialization of $fullnames as an array, with $fullnames = #() (in the absence of this initialization, += perform simple string concatenation instead of appending an element to an array, conceptually speaking).
The answer below may still be of interest for diagnosing how identical-looking strings may differ.
Since your Sort-Object inputs are strings (and the name-per-line output implies that $fullnames is indeed an array, even though you don't show its initialization as such), the implication is that the ostensible duplicates must differ by whitespace (type) or invisible control characters.
You can try the following technique to discover such invisible discrepancies:
'Adam Warlock',
'Adam Warlock ', # trailing space
'Frank Coutel',
('Frank{0}Coutel' -f [char] 0xa0) | # no-break space instead of regular space
ForEach-Object { $_ + "`t" + [int[]] [char[]] $_ } |
Sort-Object -Unique
Output:
Adam Warlock 65 100 97 109 32 87 97 114 108 111 99 107
Adam Warlock 65 100 97 109 32 87 97 114 108 111 99 107 32 # Note the extra 32
Frank Coutel 70 114 97 110 107 160 67 111 117 116 101 108 # Note the 160
Frank Coutel 70 114 97 110 107 32 67 111 117 116 101 108
The numbers are the Unicode code points of the characters that make up each string, in decimal format.
Alternatively, for a better visualization of the differences, use helper function Debug-String from this Gist:
Note: The following snippet downloads and defines the function automatically. I can personally assure you that doing so is safe, but you you should always check the source code yourself first.
# Download and define helper function Debug-String
# * See comments above.
# * To see instructions on how to make the function available in
# *future* sessions, remove >3 $null and re-run this command directly
# from the command line.
irm https://gist.github.com/mklement0/7f2f1e13ac9c2afaf0a0906d08b392d1/raw/Debug-String.ps1 | iex 3>$null
'Adam Warlock',
'Adam Warlock ', # trailing space
'Frank Coutel',
('Frank{0}Coutel' -f [char] 0xa0) | # no-break space instead of regular space
Debug-String |
Sort-Object -Unique
Output:
Adam·Warlock
Adam·Warlock· # Note the trailing ·
Frank·Coutel
Frank`u{a0}Coutel # Note the `u{a0} escape sequence
Debug-String visualizes regular spaces as ·, and formatting/control characters outside the ASCII range as Unicode escape sequences, e.g. `u{a0}, where a0 is the hex form of the character's Unicode code point, namely of the no-break space ( U+00A0) character, in this case.