Hi I'm completely new to Powershell so pardon me if this question has a really simple answer.
I would like to use Powershell to look thru a textfile, get all values and prefix and post fix these values with a character.
How can this be done?
Thanks
To add a prefix and suffix to each line of a file (with default command aliases this would be a lot shorter, but full names are clearer):
get-content input-file | `
foreach-object -process { "Prefix" + $_ + "Suffix" } | `
out-file output-file
Add -encoding UTF8 to the out-file to override the default encoding (UTF-16).
To do this in place output-file will need to be a temporary file and then replace the input-file after the processing has completed or read the input file into memory.
Related
I have four text files in the following directory that have varying EOL characters:
C:\Sandbox 1.txt, 2.txt, 3.txt, 4.txt
I would like to write a powershell script that will loop through all files in the directory and find the EOL characters that are being used for each file and print them into a new file named EOL.txt
Sample contents of EOL.txt:
1.txt UNIX(LF)
2.txt WINDOWS(CRLF)
3.txt WINDOWS(CRLF)
4.txt UNIX(LF)
I know to loop through files I will need something like the following, but I'm not sure how to read the file EOL:
Get-ChildItem "C:\Sandbox" -Filter *.txt |
Foreach-Object {
}
OR
Get-Content "C:\Sandbox\*" -EOL | Out-File -FilePath "C:\Sandbox\EOL.txt"
##note that EOL is not a valid Get-Content command
Try the following:
Get-ChildItem C:\Sandbox\*.txt -Exclude EOL.txt |
Get-Content -Raw |
ForEach-Object {
$newlines = [regex]::Matches($_, '\r?\n').Value | Select-Object -Unique
$newLineDescr =
switch ($newlines.Count) {
0 { 'N/A' }
2 { 'MIXED' }
default { ('UNIX(LF)', 'WINDOWS(CRLF)')[$newlines -eq "`r`n"] }
}
# Construct and output a custom object for the file at hand.
[pscustomobject] #{
Path = $_.PSChildName
NewlineFormat = $newLineDescr
}
} # | Out-File ... to save to a file - see comments below.
The above outputs something like:
FileName NewlineFormat
-------- -------------
1.txt UNIX(LF)
2.txt WINDOWS(CRLF)
3.txt N/A
4.txt MIXED
N/A means that no newlines are present, MIXED means that both CRLF and LF newlines are present.
You can save the output:
directly in the for-display format shown above by appending a > redirection or piping (|) to Out-File, as in your question.
alternatively, using a structured text format better suited to programmatic processing, such CSV; e.g.:
Export-Csv -NoTypeInformation -Encoding utf8 C:\Sandbox\EOL.txt
Note:
Short of reading the raw bytes of a text file one by one or in batches, the only way to analyze the newline format is to read the file in full and search for newline sequences. Get-Content -Raw reads a given file in full.
[regex]::Matches($_, '\r?\n').Value extracts all newline sequences - whether CRLF or LF - from the file's content, and Select-Object -Unique reduces them to the set of distinct sequences.
('UNIX(LF)', 'WINDOWS(CRLF)')[$newlines -eq "`r`n"] is a convenient, but somewhat obscure emulation of the following ternary conditional:
$newlines -eq "`r`n" ? 'WINDOWS(CRLF)' : 'UNIX(LF)', which could be used in PowerShell (Core) 7+ as-is, but, unfortunately isn't supported in Windows PowerShell.
The technique relies on a [bool] value getting coerced to an [int] value when used as an array index ($true -> 1, $false -> 0), thereby selecting the appropriate element from the input array.
If you don't mind the verbosity, you can use a regular if statement as an expression (i.e., you can assign its output directly to a variable: $foo = if ...), which works in both PowerShell editions:
if ($newlines -eq "`r`n") { 'WINDOWS(CRLF)' } else { 'UNIX(LF)' }
Simpler alternative via WSL, if installed:
WSL comes with the file utility, which analyzes the content of files and reports summary information, including newline formats.
While you get no control over the output format, which invariably includes additional information, such as the file's character encoding, the command is much simpler:
Set-Location C:\Sandbox
wsl file *.txt
Caveats:
This approach is fundamentally limited to files on local drives.
If changing to the target dir. is not an option, relative paths would need their \ instances translated to /, and full paths would need drive specs. such as C: translated to /mnt/c (lowercase!).
Interpreting the output:
If the term line terminators (referring to newlines) is not mentioned in the output (for text files), Unix (LF) newlines only are implied.
Windows (CRLF) newlines only are implied if you see with CRLF line terminators
In case of a mix of LF and CRLF, you'll see with CRLF, LF line terminators
In the absence of newlines you'll see with no line terminators
I have a small problem when using PowerShell to output a txt file with a list of files names in a directory with the word file appended to the start of each line(file dave.acc). When i use this one liner at a prompt(foreach ($i in Get-ChildItem .\*.aac) {echo "file '$i'" | Out-File -encoding ascii mylist.txt} i only have one entry in my new text file after this is run (seems to be just the last entry in the dir. There is five files in the dir and my expected output in my new txt file should read file
file dave.mp3
file steve.mp3
etc etc any help would be great.
The thing is, for each file corresponding to your filter, you overwrite "mylist.txt".
Try :
foreach ($i in Get-ChildItem .\*.aac) {
Add-Content .\mylist.txt "file $i"
}
Add-Content will append new content to your file for each file in your Get-ChildItem output. :)
Edit :
Also, you should not use "echo" which is an alias of "Write-Output". If you don't want to use "Add-Content" you can do :
foreach ($i in Get-ChildItem .\*.aac) {
Write-Output "file $i" | Out-File -encoding ascii -Append .\mylist.txt
}
But as you can see, it is longer to write. :)
"Add-Content" essentially do the ascii encoding and the "Append" option by default.*
*edit: It's true for PS 5.1 but not for PS 7 which default the encoding to UTF-8 as #zett42 mentioned it in the comments.
I'm attempting to replace the hostname from inside a CSV file with a blank space " ", but I am struggling to understand how this works, as when I attempt to call $env:computername it takes it as a string rather than the variable as per below:
$var = $env:computername
[io.file]::readalltext("C:\test\test.csv").replace("$var"," ") | Out-File c:\test\test-new.csv -Encoding ascii –Force
I have tested this and can see that the script is looking for the string $var in the CSV rather than the variable. Ultimately I would like to run this against a directory of CSV files that contain different hostnames in them and output the newly edited files to a separate location also.
I removed the quotes and tested this script and it worked for my test csv file
The quotes should be expanding the variable but if not you can simply pass the environmental variable directly like so:
[io.file]::readalltext("C:\test\test.csv").replace($env:ComputerName,"") | Out-File c:\test\test-new.csv -Encoding ascii –Force
I have a text file containing names of people seperated by . how to replace the . with new lines so that each name is in new line with powershell
If you have the latest version of PSCX (3.2.0) http://pscx.codeplex.com, we just added a new command to simplify this type of task a bit:
Edit-File -Path c:\path\to\file.txt -Pattern '\.' -Replacement "`r`n"
Or using positional params:
Edit-File c:\path\to\file.txt '\.' "`r`n"
This command also handles taking care that the file's original encoding is preserved. Using Out-File will output using Unicode unless you override with the -Encoding parameter which of course requires that you know the file's encoding in the first place. :-)
You can do a simple replace...
(gc c:\path\to\file.txt) -replace "\.","`n" | out-file c:\path\to\newfile.txt
I am trying to replicate the functionality of the cat command in Unix.
I would like to avoid solutions where I explicitly read both files into variables, concatenate the variables together, and then write out the concatenated variable.
Simply use the Get-Content and Set-Content cmdlets:
Get-Content inputFile1.txt, inputFile2.txt | Set-Content joinedFile.txt
You can concatenate more than two files with this style, too.
If the source files are named similarly, you can use wildcards:
Get-Content inputFile*.txt | Set-Content joinedFile.txt
Note 1: PowerShell 5 and older versions allowed this to be done more concisely using the aliases cat and sc for Get-Content and Set-Content respectively. However, these aliases are problematic because cat is a system command in *nix systems, and sc is a system command in Windows systems - therefore using them is not recommended, and in fact sc is no longer even defined as of PowerShell Core (v7). The PowerShell team recommends against using aliases in general.
Note 2: Be careful with wildcards - if you try to output to inputFiles.txt (or similar that matches the pattern), PowerShell will get into an infinite loop! (I just tested this.)
Note 3: Outputting to a file with > does not preserve character encoding! This is why using Set-Content is recommended.
Do not use >; it messes up the character encoding. Use:
Get-Content files.* | Set-Content newfile.file
In cmd, you can do this:
copy one.txt+two.txt+three.txt four.txt
In PowerShell this would be:
cmd /c copy one.txt+two.txt+three.txt four.txt
While the PowerShell way would be to use gc, the above will be pretty fast, especially for large files. And it can be used on on non-ASCII files too using the /B switch.
You could use the Add-Content cmdlet. Maybe it is a little faster than the other solutions, because I don't retrieve the content of the first file.
gc .\file2.txt| Add-Content -Path .\file1.txt
To concat files in command prompt it would be
type file1.txt file2.txt file3.txt > files.txt
PowerShell converts the type command to Get-Content, which means you will get an error when using the type command in PowerShell because the Get-Content command requires a comma separating the files. The same command in PowerShell would be
Get-Content file1.txt,file2.txt,file3.txt | Set-Content files.txt
I used:
Get-Content c:\FileToAppend_*.log | Out-File -FilePath C:\DestinationFile.log
-Encoding ASCII -Append
This appended fine. I added the ASCII encoding to remove the nul characters Notepad++ was showing without the explicit encoding.
If you need to order the files by specific parameter (e.g. date time):
gci *.log | sort LastWriteTime | % {$(Get-Content $_)} | Set-Content result.log
You can do something like:
get-content input_file1 > output_file
get-content input_file2 >> output_file
Where > is an alias for "out-file", and >> is an alias for "out-file -append".
Since most of the other replies often get the formatting wrong (due to the piping), the safest thing to do is as follows:
add-content $YourMasterFile -value (get-content $SomeAdditionalFile)
I know you wanted to avoid reading the content of $SomeAdditionalFile into a variable, but in order to save for example your newline formatting i do not think there is proper way to do it without.
A workaround would be to loop through your $SomeAdditionalFile line by line and piping that into your $YourMasterFile. However this is overly resource intensive.
To keep encoding and line endings:
Get-Content files.* -Raw | Set-Content newfile.file -NoNewline
Note: AFAIR, whose parameters aren't supported by old Powershells (<3? <4?)
I think the "powershell way" could be :
set-content destination.log -value (get-content c:\FileToAppend_*.log )