Powershell. Writing out lines based on string within the file - powershell

I'm looking for a way to export all lines from within a text file where part of the line matches a certain string. The string is actually the first 4 bytes of the file and I'd like to keep the command to only checking those bytes; not the entire row. I want to write the entire row. How would I go about this?
I am using Windows only and don't have the option to use many other tools that might do this.
Thanks in advance for any help.

Do you want to perform a simple "grep"? Then try this
select-string .\test.txt -pattern "\Athat" | foreach {$_.Line}
or this (very similar regex), also writes to an outfile
select-string .\test.txt -pattern "^that" | foreach {$_.Line} | out-file -filepath out.txt
This assumes that you want to search for a 4-byte string "that" at the beginning of the string , or beginning of the line, respectively.

Something like the following Powershell function should work for you:
function Get-Lines {
[cmdletbinding()]
param(
[string]$filename,
[string]$prefix
)
if( Test-Path -Path $filename -PathType Leaf -ErrorAction SilentlyContinue ) {
# filename exists, and is a file
$lines = Get-Content $filename
foreach ( $line in $lines ) {
if ( $line -like "$prefix*" ) {
$line
}
}
}
}
To use it, assuming you save it as get-lines.ps1, you would load the function into memory with:
. .\get-lines.ps1
and then to use it, you could search for all lines starting with "DATA" with something like:
get-lines -filename C:\Files\Datafile\testfile.dat -prefix "DATA"
If you need to save it to another file for viewing later, you could do something like:
get-lines -filename C:\Files\Datafile\testfile.dat -prefix "DATA" | out-file -FilePath results.txt
Or, if I were more awake, you could ignore the script above, use a simpler solution such as the following one-liner:
get-content -path C:\Files\Datafile\testfile.dat | select-string -Pattern "^DATA"
Which just uses the ^ regex character to make sure it's only looking for "DATA" at the beginning of each line.

To get all the lines from c:\somedir\somefile.txt that begin with 'abcd' :
(get-content c:\somedir\somefile.txt) -like 'abcd*'
provided c:\somedir\somefile.txt is not an unusually large (hundreds of MB) file. For that situation:
get-content c:\somedir\somefile.txt -readcount 1000 |
foreach {$_ -like 'abcd*'}

Related

How to prepend to a file in PowerShell?

I'm generating two files, userscript.meta.js and userscript.user.js. I need the output of userscript.meta.js to be placed at the very beginning of userscript.user.js.
Add-Content doesn't seem to accept a parameter to prepend and Get-Content | Set-Content will fail because userscript.user.js is being used by Get-Content.
I'd rather not create an intermediate file if it's physically possible to have a clean solution.
How to achieve this?
The Subexpression operator $( ) can evaluate both Get-Content statements which are then enumerated and passed through the pipeline to Set-Content:
$(
Get-Content userscript.meta.js -Raw
Get-Content userscript.user.js -Raw
) | Set-Content userscript.user.js
Consider using the Absolute Path of the files if your current directory is not where those files are.
An even more simplified approach than the above would be to put the paths in the desired order since both, the -Path and -LiteralPath parameters can take multiple values:
(Get-Content userscript.meta.js, userscript.user.js -Raw) |
Set-Content userscript.user.js
And in case you want to get rid of excess leading or trailing white-space, you can include the String.Trim Method:
(Get-Content userscript.meta.js, userscript.user.js -Raw).Trim() |
Set-Content userscript.user.js
Note that in above examples the grouping operator ( ) is mandatory as we need to consume all output from Get-Content before being passed through the pipeline to Set-Content. See Piping grouped expressions for more details.
For future folks, here's a snippet if you need to prepend the same thing to multiple files:
example: prepending an #include directive to a bunch of auto-generated C++ files so it works with my Windows environment.
Get-ChildItem -Path . -Filter *.cpp | ForEach-Object {
$file = $_.FullName
# the -Raw param was important for me as it didn't read the entire
# file properly without it. I even tried [System.IO.File]::ReadAllText
# and got the same thing, so there must have been some characater that
# caused the file read to return prematurely
$content = Get-Content $file -Raw
$prepend = '#include "stdafx.h"' + "`r`n"
#this could also be from a file: aka
# $prepend = Get-Content 'path_to_my_file_used_for_prepending'
$content = $prepend + $content
Set-Content $file $content
}

Copy specific lines from a text file to separate file using powershell

I am trying to get all the lines from an Input file starting with %% and paste it into Output file using powershell.
Used the following code, however I am only getting last line in Output file starting with %% instead of all the lines starting with %%.
I have only started to learn powershell, please help
$Clause = Get-Content "Input File location"
$Outvalue = $Clause | Foreach {
if ($_ -ilike "*%%*")
{
Set-Content "Output file location" $_
}
}
You are looping over the lines in the file, and setting each one as the whole content of the file, overwriting the previous file each time.
You need to either switch to using Add-Content instead of Set-Content, which will append to the file, or change the design to:
Get-Content "input.txt" | Foreach-Object {
if ($_ -like "%%*")
{
$_ # just putting this on its own, sends it on out of the pipeline
}
} | Set-Content Output.txt
Which you would more typically write as:
Get-Content "input.txt" | Where-Object { $_ -like "%%*" } | Set-Content Output.txt
and in the shell, you might write as
gc input.txt |? {$_ -like "%%*"} | sc output.txt
Where the whole file is filtered, and then all the matching lines are sent into Set-Content in one go, not calling Set-Content individually for each line.
NB. PowerShell is case insensitive by default, so -like and -ilike behave the same.
For a small file, Get-Content is nice. But if you start trying to do this on heavier files, Get-Content will eat your memory and leave you hanging.
Keeping it REALLY simple for other Powershell starters out there, you'll be better covered (and with better performance). So, something likes this would do the job:
$inputfile = "C:\Users\JohnnyC\Desktop\inputfile.txt"
$outputfile = "C:\Users\JohnnyC\Desktop\outputfile.txt"
$reader = [io.file]::OpenText($inputfile)
$writer = [io.file]::CreateText($outputfile)
while($reader.EndOfStream -ne $true) {
$line = $reader.Readline()
if ($line -like '%%*') {
$writer.WriteLine($line);
}
}
$writer.Dispose();
$reader.Dispose();

Powershell Import-csv with return character

I tried the following to turn a text file into a document by leveraging import-csv where each item in the original document was a new line
Sample file.txt
James Cameron
Kirk Cobain
Linda Johnson
Code:
$array = import-csv file.txt | ConvertFrom-Csv -Delim `r
foreach ($Data in $array)
{
if (sls $Data Master.txt -quiet)
{Add-Content file.txt $Data}
}
It never created the document
Import-Csv takes a CSV and outputs PSCustomObjects. It's intended for when the file has a header row, and it reads that as the properties of the objects. e.g.
FirstName,LastName
James,Cameron
Kirk,Cobain
# ->
#{FirstName='James';LastName='Cameron'}
#{FirstName='Kirk';LastName='Cobain'}
etc.
If your file has no header row, it will take the first row and then ruin everything else afterwards. You need to provide the -Header 'h1','h2',... parameter to fix that. So you could use -Header Name, but your data only has one property, so there's not much benefit.
ConvertFrom-Csv is intended to do the same thing, but from CSV data in a variable instead of a file. They don't chain together usefully. It will try, but what you end up with is...
A single object, with a property called '#{James=Kirk}' and a value of '#{James=Linda}', where 'James' was taken from line 1 as a column header, and the weird syntax is from forcing those objects through a second conversion.
It's not at all clear why you are reading in from file.txt and adding to file.txt. But since you don't have a CSV, there's no benefit from using the CSV cmdlets.
$lines = Get-Content file.txt
$master = Get-Content master.txt
foreach ($line in $lines)
{
if ($master -contains $line)
{
Add-Content file2.txt $line
}
}
or just
gc file.txt |? { sls $_ master.txt -quiet } | set-content file2.txt
Auto-generated PS help links from my codeblock (if available):
gc is an alias for Get-Content (in module Microsoft.PowerShell.Management)
? is an alias for Where-Object
sls is an alias for Select-String (in module Microsoft.PowerShell.Utility)
Set-Content (in module Microsoft.PowerShell.Management)

Powershell V2 find and replace

I am trying to change dates programmatically in a file. The line I need to fix looks like this:
set ##dateto = '03/15/12'
I need to write a powershell V2 script that replaces what's inside the single quotes, and I have no idea how to do this.
The closest I've come looks like this:
gc $file | ? {$_ -match "set ##dateto ="} | % {$temp=$_.split("'");$temp[17]
=$CorrectedDate;$temp -join ","} | -outfile newfile.txt
Problems with this: It gives an error about the index 17 being out of range. Also, the outfile only contains one line (The unmodified line). I'd appreciate any help with this. Thanks!
You can do something like this ( though you may want to handle the corner cases) :
$CorrectedDate = '10/09/09'
gc $file | %{
if($_ -match "^set ##dateto = '(\d\d/\d\d/\d\d)'") {
$_ -replace $matches[1], $CorrectedDate;
}
else {
$_
}
} | out-file test2.txt
mv test2.txt $file -force

Powershell equivalent to sponge in moreutils?

There's a GNU program called sponge that soaks up input before writing to a file so you can do something like this: cat myFile | grep "myFilter" | sponge myFile
Is there a powershell equivalent, so I can work on a file in place, without having to pipe to a temporary file?
Thanks
In Powershell, judicious use of parentheses will force an operation to completely finish before passing data to the next command in the pipeline. The default for piping Get-Content is to pipe line by line to the next command, but with parentheses it must form a complete data set (e.g., load all lines) before continuing:
(Get-Content myFile) | Select-String 'MyFilter' | Set-Content myFile
An alternative that may use less memory (I have not benchmarked it) is to only force the results of Select-String to complete before continuing:
(Get-Content myFile | Select-String 'MyFilter') | Set-Content myFile
You could also assign things to a variable as an additional step. Any technique will load the contents into the Powershell session's memory, so be careful with big files.
Addendum: Select-String returns MatchInfo objects. Using Out-File adds pesky extra blank lines due to the way it tries to format the results as a string, but Set-Content correctly converts each object to its own string as it writes, producing better output. Being that you're coming from *nix and are used to everything returning strings (whereas Powershell returns objects), one way to force string output is to pipe them through a foreach that converts them:
(Get-Content myFile | Select-String 'MyFilter' | foreach { $_.tostring() }) | Set-Content myFile
You can try this :
(Get-content myfile) | where {$_ -match "regular-expression"} | Set-content myfile
or
${full-path-file-name-of-myfile} | where {$_ -match "regular-expression"} | add-content Anotherfile
more easier to keep in mind
two other ways come to mind - they are both the same really just one is a function the other is on the command line. (I don't know sponge on unix so I can't say for certain they mimic it).
here's the first on the command line
Get-Content .\temp.txt |
Select-String "grep" |
foreach-object -begin { [array] $out = #()} -process { $out = $out + ($_.tostring())} -end {write-output $out}
and the second is two create a function to do it
function sponge {
[cmdletbinding()]
Param(
[Parameter(
Mandatory = $True,
ValueFromPipeline = $True)]
[string]$Output
)
Begin {
[array] $out = #()
}
Process {
$out = $out + $Output
}
End {
Write-Output $Out
}
}
Get-Content .\temp2.txt | Select-String "grep" | sponge
HTH,
Matt