Copy specific lines from a text file to separate file using powershell - powershell

I am trying to get all the lines from an Input file starting with %% and paste it into Output file using powershell.
Used the following code, however I am only getting last line in Output file starting with %% instead of all the lines starting with %%.
I have only started to learn powershell, please help
$Clause = Get-Content "Input File location"
$Outvalue = $Clause | Foreach {
if ($_ -ilike "*%%*")
{
Set-Content "Output file location" $_
}
}

You are looping over the lines in the file, and setting each one as the whole content of the file, overwriting the previous file each time.
You need to either switch to using Add-Content instead of Set-Content, which will append to the file, or change the design to:
Get-Content "input.txt" | Foreach-Object {
if ($_ -like "%%*")
{
$_ # just putting this on its own, sends it on out of the pipeline
}
} | Set-Content Output.txt
Which you would more typically write as:
Get-Content "input.txt" | Where-Object { $_ -like "%%*" } | Set-Content Output.txt
and in the shell, you might write as
gc input.txt |? {$_ -like "%%*"} | sc output.txt
Where the whole file is filtered, and then all the matching lines are sent into Set-Content in one go, not calling Set-Content individually for each line.
NB. PowerShell is case insensitive by default, so -like and -ilike behave the same.

For a small file, Get-Content is nice. But if you start trying to do this on heavier files, Get-Content will eat your memory and leave you hanging.
Keeping it REALLY simple for other Powershell starters out there, you'll be better covered (and with better performance). So, something likes this would do the job:
$inputfile = "C:\Users\JohnnyC\Desktop\inputfile.txt"
$outputfile = "C:\Users\JohnnyC\Desktop\outputfile.txt"
$reader = [io.file]::OpenText($inputfile)
$writer = [io.file]::CreateText($outputfile)
while($reader.EndOfStream -ne $true) {
$line = $reader.Readline()
if ($line -like '%%*') {
$writer.WriteLine($line);
}
}
$writer.Dispose();
$reader.Dispose();

Related

Powershell, replace string line by line in a textfile

I have a file called "file123624.TXT" that contains this information:
FKHOGU1100;;;;;;;;;;;;;randomdata;1;0;2;1234
YJKMRI1101;;;;;;;;;;;;;randomdata;1;0;2;1234
FWPCYY1113;GV;randomdata;5;;;;;;;6018;randomdata;GU;;1;0;2;1234
VOBYTM1100;;;;;;;;;;;;;randomdata;1;0;2;1234
ZSOKHW1160;GV;randomdata;53;;;;;;;7353;randomdata;GU;;1;0;2;1234
YCHQHS1123;GV;randomdata;4;;;;;;;5063;randomdata;GU;;1;0;2;1234
YXRCZO1105;GV;randomdata;39;;;;;;;9510;randomdata;GU;;1;0;2;1234
XVDUEM1100;GV;randomdata;14;;;;;;;9901;randomdata;GU;;1;0;2;1234
CHECKSUM;0000008
All i want to do is add an - after the first six characters in the file, except for the last line "CHECKSUM; 0000008"
I have made a small powershell script that almost does the trick:
$file = Get-Content "C:\Users\usr\Desktop\file*.txt"
foreach ($i in $file)
{
if($i -notmatch "CHECKSUM*")
{$I.Insert(6,'-')}
}
This script output the lines i need to be changed, but i cant replace them line for line.
The result i want in the "file123624.txt" after running the script is this:
FKHOGU-1100;;;;;;;;;;;;;randomdata;1;0;2;1234
YJKMRI-1101;;;;;;;;;;;;;randomdata;1;0;2;1234
FWPCYY-1113;GV;randomdata;5;;;;;;;6012;randomdata;GU;;1;0;2;1234
VOBYTM-1100;;;;;;;;;;;;;randomdata;1;0;2;1234
ZSOKHW-1160;GV;randomdata;53;;;;;;;7653;randomdata;GU;;1;0;2;1234
YCHQHS-1123;GV;randomdata;4;;;;;;;5463;randomdata;GU;;1;0;2;1234
YXRCZO-1105;GV;randomdata;39;;;;;;;9210;randomdata;GU;;1;0;2;1234
XVDUEM-1100;GV;randomdata;14;;;;;;;9401;randomdata;GU;;1;0;2;1234
CHECKSUM;0000008
Any solutions or tips on this would be appreciated
You can do the following, which utilizes the Foreach-Object and has a similar structure to your current code.
$file = Get-Content file123624.TXT | Foreach-Object {
if ($_ -notmatch '^CHECKSUM') {
$_.Insert(6,'-')
}
else {
$_
}
}
$file | Set-Content file123624.TXT

How to make changes to file content and save it to another file using powershell?

I want to do this
read the file
go through each line
if the line matches the pattern, do some changes with that line
save the content to another file
For now I use this script:
$file = [System.IO.File]::ReadLines("C:\path\to\some\file1.txt")
$output = "C:\path\to\some\file2.txt"
ForEach ($line in $file) {
if($line -match 'some_regex_expression') {
$line = $line.replace("some","great")
}
Out-File -append -filepath $output -inputobject $line
}
As you can see, here I write line by line. Is it possible to write the whole file at once ?
Good example is provided here :
(Get-Content c:\temp\test.txt) -replace '\[MYID\]', 'MyValue' | Set-Content c:\temp\test.txt
But my problem is that I have additional IF statement...
So, what could I do to improve my script ?
You could do it like that:
Get-Content -Path "C:\path\to\some\file1.txt" | foreach {
if($_ -match 'some_regex_expression') {
$_.replace("some","great")
}
else {
$_
}
} | Out-File -filepath "C:\path\to\some\file2.txt"
Get-Content reads a file line by line (array of strings) by default so you can just pipe it into a foreach loop, process each line within the loop and pipe the whole output into your file2.txt.
In this case Arrays or Array List(lists are better for large arrays) would be the most elegant solution. Simply add strings in array until ForEach loop ends. After that just flush array to a file.
This is Array List example
$file = [System.IO.File]::ReadLines("C:\path\to\some\file1.txt")
$output = "C:\path\to\some\file2.txt"
$outputData = New-Object System.Collections.ArrayList
ForEach ($line in $file) {
if($line -match 'some_regex_expression') {
$line = $line.replace("some","great")
}
$outputData.Add($line)
}
$outputData |Out-File $output
I think the if statement can be avoided in a lot of cases by using regular expression groups (e.g. (.*) and placeholders (e.g. $1, $2 etc.).
As in your example:
(Get-Content .\File1.txt) -Replace 'some(_regex_expression)', 'great$1' | Set-Content .\File2.txt
And for the good example" where [MYID\] might be somewhere inline:
(Get-Content c:\temp\test.txt) -Replace '^(.*)\[MYID\](.*)$', '$1MyValue$2' | Set-Content c:\temp\test.txt
(see also How to replace first and last part of each line with powershell)

Delete lines from multiple textfiles in PowerShell

I am trying to delete lines with a defined content from multiple textfiles.
It works in the core, but it will rewrite every file even if no changes are made, which is not cool if you are just modifying 50 out of about 3000 logonscripts.
I even made a if statement but it seems like it doesn't work.
Alright this is what I already have:
#Here $varFind will be escaped from potential RegEx triggers.
$varFindEscaped = [regex]::Escape($varFind)
#Here the deletion happens.
foreach ($file in Get-ChildItem $varPath*$varEnding) {
$contentBefore = Get-Content $file
$contentAfter = Get-Content $file | Where-Object {$_ -notmatch $varFindEscaped}
if ($contentBefore -ne $contentAfter) {Set-Content $file $contentAfter}
}
What the variables mean:
$varPath is the path in which the logonscripts are.
$varEnding is the file ending of the files to modify.
$varFind is the string that triggers the deletion of the line.
Any help would be highly appreciated.
Greetings
Löwä Cent
You have to read the file regardless but some improvement on your change condition could help.
#Here the deletion happens.
foreach ($file in Get-ChildItem $varPath*$varEnding) {
$data = (Get-Content $file)
If($data -match $varFindEscaped){
$data | Where-Object {$_ -notmatch $varFindEscaped} | Set-Content $file
}
}
Read the file into $data. Check to see if the pattern $varFindEscaped is present in the file. If it is than filter out those matching the same pattern. Else we move onto the next file.

Powershell Find and replace first line of CSV only

I need to read in a CSV file and find replace certain characters from the first line of the file only. I have used foreach-object however this processes the entire file. Any thought on how this can best be achieved.
Here is the code :
Get-Content c:\output.csv | ForEach-Object { $_ -replace "objectGUID", 'StudentID' } | Set-Content c:\output2.csv
This won't fix the problem of having to process the entire file, but should substantially reduce the time it takes to do it if it's a substantially large file.
$Updated = $false
Get-Content c:\output.csv -ReadCount 1000 |
ForEach-Object {
if ($Updated)
{
$_ | Add-Content c:\output2.csv
}
else {
$_[0] = $_[0] -replace "objectGUID", 'StudentID'
$_ | Set-Content c:\output2.csv
$Updated = $true
}
}
Edit: if it's only 3000 rows this should be sufficient:
$FileContent = Get-Content c:\output.csv
$FileContent[0] = $FileContent[0] -replace 'objectGUID', 'StudentID'
$FileContent | Set-Content c:\output2.csv
Ok, Get-Content makes this simple enough if all you want to do is change the first line of a text file.
GC c:\output.csv|select -first 1|%{$_ -replace "objectGUID", 'StudentID'}|Out-File C:\Output2.csv
GC C:\output.csv -readcount 1000|Select -skip 1|Out-File C:\Output2.csv -Append
That will pull the first line only, replacing the text you wanted and write it to a new file (assuming you don't already have an Output2.csv file). After that it reads in the rest of the file skipping the first line and adds that to the same file. You can delete the original file after that and rename the output file if you feel the need.

Powershell. Writing out lines based on string within the file

I'm looking for a way to export all lines from within a text file where part of the line matches a certain string. The string is actually the first 4 bytes of the file and I'd like to keep the command to only checking those bytes; not the entire row. I want to write the entire row. How would I go about this?
I am using Windows only and don't have the option to use many other tools that might do this.
Thanks in advance for any help.
Do you want to perform a simple "grep"? Then try this
select-string .\test.txt -pattern "\Athat" | foreach {$_.Line}
or this (very similar regex), also writes to an outfile
select-string .\test.txt -pattern "^that" | foreach {$_.Line} | out-file -filepath out.txt
This assumes that you want to search for a 4-byte string "that" at the beginning of the string , or beginning of the line, respectively.
Something like the following Powershell function should work for you:
function Get-Lines {
[cmdletbinding()]
param(
[string]$filename,
[string]$prefix
)
if( Test-Path -Path $filename -PathType Leaf -ErrorAction SilentlyContinue ) {
# filename exists, and is a file
$lines = Get-Content $filename
foreach ( $line in $lines ) {
if ( $line -like "$prefix*" ) {
$line
}
}
}
}
To use it, assuming you save it as get-lines.ps1, you would load the function into memory with:
. .\get-lines.ps1
and then to use it, you could search for all lines starting with "DATA" with something like:
get-lines -filename C:\Files\Datafile\testfile.dat -prefix "DATA"
If you need to save it to another file for viewing later, you could do something like:
get-lines -filename C:\Files\Datafile\testfile.dat -prefix "DATA" | out-file -FilePath results.txt
Or, if I were more awake, you could ignore the script above, use a simpler solution such as the following one-liner:
get-content -path C:\Files\Datafile\testfile.dat | select-string -Pattern "^DATA"
Which just uses the ^ regex character to make sure it's only looking for "DATA" at the beginning of each line.
To get all the lines from c:\somedir\somefile.txt that begin with 'abcd' :
(get-content c:\somedir\somefile.txt) -like 'abcd*'
provided c:\somedir\somefile.txt is not an unusually large (hundreds of MB) file. For that situation:
get-content c:\somedir\somefile.txt -readcount 1000 |
foreach {$_ -like 'abcd*'}