I'm trying to write a PowerShell script which will take particular extension files from different servers. To pass many extensions I know we can use # followed by extension. When I pick them from an input file and pass it on the script, it doesn't work.
$ServerName=Get-content "D:\HOMEWARE\BLRMorningCheck\Jerry\servername.txt"
foreach ($server in $ServerName)
{
$server_host=echo $server | %{$data = $_.split(";"); Write-Output "$($data[0])"}
$Targetfolder=echo $server | %{$data = $_.split(";"); Write-Output "$($data[1])"}
$Ext=echo $server | %{$data = $_.split(";"); Write-Output "$($data[2])"}
$Extension =#($Ext)
$Targetfolder=$Targetfolder.Trim('"')
$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse
echo $Files
}
My extensions are *.log, *.log*7z, *.txt*7z, *.txt*.
$Ext contains a string, probably with a comma-separated list of extensions. However, a comma-separated string doesn't turn into an array (which is what the -Include parameter expects) just because you put it in #(). You need to split the string at the delimiter character:
PS C:\> $Ext = ".log,.log*7z,*.txt*7z,.txt"
PS C:\> $Ext
.log,.log*7z,*.txt*7z,.txt
PS C:\> $Extension = $Ext -split ','
PS C:\> $Extension
.log
.log*7z
*.txt*7z
.txt
Also, like I said in my answer to your previous question, you're probably better off using Import-Csv for reading your input file:
$filename = 'D:\HOMEWARE\BLRMorningCheck\Jerry\servername.txt'
Import-Csv $filename -Delimiter ';' -Header 'ComputerName', 'TargetFolder', 'Ext' |
select TargetFolder, #{n='Extensions';e={$_.Ext -split ','}} |
% { Get-Childitem $_.TargetFolder -Include $_.Extensions -Recurse }
#Ansgar has a the correct approach but in case you're new to powershell the more basic syntax below might be easier to understand. If you already know what file extensions you're looking for you don't need to get them from the file, just create an array that contains them.
$exts = "*.log",".log*7z","*.txt*7z","*.txt"
$servers = Get-Content "D:\HOMEWARE\BLRMorningCheck\Jerry\servername.txt"
foreach ($server in $servers) {
$host = $($server.split(';'))[0]
$targetFolder = $($server.split(';'))[1]
$files = Get-Childitem $targetFolder -Include $exts -Recurse
foreach($f in $files) {
Write-Output $f.fullname
}
}
Related
I am trying to construct a script that moves through specific folders and the log files in it, and filters the error codes. After that it passes them into a new file.
I'm not really sure how to do that with for loops so I'll leave my code bellow.
If someone could tell me what I'm doing wrong, that would be greatly appreciated.
$file_name = Read-Host -Prompt 'Name of the new file: '
$path = 'C:\Users\user\Power\log_script\logs'
Add-Type -AssemblyName System.IO.Compression.FileSystem
function Unzip
{
param([string]$zipfile, [string]$outpath)
[System.IO.Compression.ZipFile]::ExtractToDirectory($zipfile, $outpath)
}
if ([System.IO.File]::Exists($path)) {
Remove-Item $path
Unzip 'C:\Users\user\Power\log_script\logs.zip' 'C:\Users\user\Power\log_script'
} else {
Unzip 'C:\Users\user\Power\log_script\logs.zip' 'C:\Users\user\Power\log_script'
}
$folder = Get-ChildItem -Path 'C:\Users\user\Power\log_script\logs\LogFiles'
$files = foreach($logfolder in $folder) {
$content = foreach($line in $files) {
if ($line -match '([ ][4-5][0-5][0-9][ ])') {
echo $line
}
}
}
$content | Out-File $file_name -Force -Encoding ascii
Inside the LogFiles folder are three more folders each containing log files.
Thanks
Expanding on a comment above about recursing the folder structure, and then actually retrieving the content of the files, you could try something line this:
$allFiles = Get-ChildItem -Path 'C:\Users\user\Power\log_script\logs\LogFiles' -Recurse
# iterate the files
$allFiles | ForEach-Object {
# iterate the content of each file, line by line
Get-Content $_ | ForEach-Object {
if ($_ -match '([ ][4-5][0-5][0-9][ ])') {
echo $_
}
}
}
It looks like your inner loop is of a collection ($files) that doesn't yet exist. You assign $files to the output of a ForEach(...) loop then try to nest another loop of $files inside it. Of course at this point $files isn't available to be looped.
Regardless, the issue is you are never reading the content of your log files. Even if you managed to loop through the output of Get-ChildItem, you need to look at each line to perform the match.
Obviously I cannot completely test this, but I see a few issues and have rewritten as below:
$file_name = Read-Host -Prompt 'Name of the new file'
$path = 'C:\Users\user\Power\log_script\logs'
$Pattern = '([ ][4-5][0-5][0-9][ ])'
if ( [System.IO.File]::Exists( $path ) ) { Remove-Item $path }
Expand-Archive 'C:\Users\user\Power\log_script\logs.zip' 'C:\Users\user\Power\log_script'
Select-String -Path 'C:\Users\user\Power\log_script\logs\LogFiles\*' -Pattern $Pattern |
Select-Object -ExpandProperty line |
Out-File $file_name -Force -Encoding ascii
Note: Select-String cannot recurse on its own.
I'm not sure you need to write your own UnZip function. PowerShell has the Expand-Archive cmdlet which can at least match the functionality thus far:
Expand-Archive -Path <SourceZipPath> -DestinationPath <DestinationFolder>
Note: The -Force parameter allows it to over write the destination files if they are already present. which may be a substitute for testing if the file exists and deleting if it does.
If you are going to test for the file that section of code can be simplified as:
if ( [System.IO.File]::Exists( $path ) ) { Remove-Item $path }
Unzip 'C:\Users\user\Power\log_script\logs.zip' 'C:\Users\user\Power\log_script'
This is because you were going to run the UnZip command regardless...
Note: You could also use Test-Path for this.
Also there are enumerable ways to get the matching lines, here are a couple of extra samples:
Get-ChildItem -Path 'C:\Users\user\Power\log_script\logs\LogFiles' |
ForEach-Object{
( Get-Content $_.FullName ) -match $Pattern
# Using match in this way will echo the lines that matched from each run of
# Get-Content. If nothing matched nothing will output on that iteration.
} |
Out-File $file_name -Force -Encoding ascii
This approach will read the entire file into an array before running the match on it. For large files it may pose a memory issue, however it enabled the clever use of -match.
OR:
Get-ChildItem -Path 'C:\Users\user\Power\log_script\logs\LogFiles' |
Get-Content |
ForEach-Object{ If( $_ -match $Pattern ) { $_ } } |
Out-File $file_name -Force -Encoding ascii
Note: You don't need the alias echo or its real cmdlet Write-Output
UPDATE: After fuzzing around a bit and trying different things I finally got it to work.
I'll include the code below just for demonstration purposes.
Thanks everyone
$start = Get-Date
"`n$start`n"
$file_name = Read-Host -Prompt 'Name of the new file: '
Out-File $file_name -Force -Encoding ascii
Expand-Archive -Path 'C:\Users\User\Power\log_script\logs.zip' -Force
$i = 1
$folders = Get-ChildItem -Path 'C:\Users\User\Power\log_script\logs\logs\LogFiles' -Name -Recurse -Include *.log
foreach($item in $folders) {
$files = 'C:\Users\User\Power\log_script\logs\logs\LogFiles\' + $item
foreach($file in $files){
$content = Get-Content $file
Write-Progress -Activity "Filtering..." -Status "File $i of $($folders.Count)" -PercentComplete (($i / $folders.Count) * 100)
$i++
$output = foreach($line in $content) {
if ($line -match '([ ][4-5][0-5][0-9][ ])') {
Add-Content -Path $file_name -Value $line
}
}
}
}
$end = Get-Date
$time = [int]($end - $start).TotalSeconds
Write-Output ("Runtime: " + $time + " Seconds" -join ' ')
I'm trying to loop through 40 CSV files in a path and remove any characters that are not numeric,alphabets and space values only in the headers.
Below is my code i tried working on, This is working for headers in the files but also its replacing all the data in the file and i can see only headers without special characters in it, i'm just a beginner in power shell, not sure how to proceed further any help is much appreciated.
$path = "C:\AllFiles\"
Get-ChildItem -path $path -Filter *.csv |
Foreach-Object {
$content = Get-Content $_.FullName
$content[0] = $content[0] -replace '[^0-9a-zA-Z, ]'|Set-Content $_.FullName
}
The -replace operator requires two values, the first value is what you are looking for, and the second value is what to replace the first value with.
EXAMPLE:
"John Jones" -replace "Jones","Smith"
This will replace "Jones" with the text "Smith" creating a new string "John Smith"
In your example, instead of creating a regex of what you want to keep, create a regex of what you want to replace.
EXAMPLE:
$path = "C:\AllFiles\"
Get-ChildItem -path $path -Filter *.csv |
Foreach-Object {
$content = Get-Content -Path $path
$content[0] = $content[0] -replace '[regex for special chars]',""
Set-Content $path -value $content -force
}
This will replace the whole string, with a string where you've replaced the regex values with ""
This should do the trick and should be the fastest method:
$path = 'C:\AllFiles\'
$collection = Get-ChildItem -path $path -Filter *.csv'
foreach( $file in $collection ) {
$content = [System.IO.File]::ReadAllLines( $file.FullName )
$content[0] = $content[0] -replace '[^0-9a-zA-Z, ]'
[System.IO.File]::WriteAllLines( $file.FullName, $content ) | Out-Null
}
Pretty close, try it like this instead:
$path = "C:\temp"
Get-ChildItem -path $path -Filter *.csv |
Foreach-Object {
$content = Get-Content $_
$content[0] = $content[0] -replace '[^a-zA-Z0-9, ]',''
$content | Out-File $_
}
This will only clear special characters on the first line but leaves the rest of the file untouched.
Try this:
dir "C:\AllFiles" -Filter *.csv | % {
(Get-Content $_.FullName)[0] -replace '[\W]', '' | Set-Content $_.FullName -Force
}
I try to read big data log file, in folder C: \ log \ 1 \ i put 2 txt files, i need open-> read all file .txt and find with filter some text like whis: [text]
# Filename: script.ps1
$Files = Get-ChildItem "C:\log\1\" -Filter "*.txt"
foreach ($File in $Files)
{
$StringMatch = $null
$StringMatch = select-string $File -pattern "[Error]"
if ($StringMatch) {out-file -filepath C:\log\outputlog.txt -inputobject $StringMatch}
}
# end of script
not work
Would doing something like a select-string work?
Select-String C:\Scripts\*.txt -pattern "SEARCH STRING HERE" | Format-List
Or if there are multiple files you are wanting to parse maybe use the same select-string but within a loop and output the results.
$Files = Get-ChildItem "C:\log\1\" -Filter "*.txt"
foreach ($File in $Files)
{
$StringMatch = $null
$StringMatch = select-string $File -pattern "SEARCH STRING HERE"
if ($StringMatch) {out-file -filepath c:\outputlog.txt -inputobject $StringMatch}
}
This will print out the file name along with the line number in the file. I hope this is what you are looking for.
Remove-Item -Path C:\log\outlog.txt
$Files = Get-ChildItem "C:\log\1\" -Filter "*.txt"
foreach ($File in $Files)
{
$lineNumber = 0
$content = Get-Content -Path "C:\log\1\$File"
foreach($line in $content)
{
if($line.Contains('[Error]'))
{
Add-Content -Path C:\log\outlog.txt -Value "$File -> $lineNumber"
}
$lineNumber++
}
}
Code below works
It selects strings in txt files in your folder based on -SimpleMatch and then appends it to new.txt file.
Though i do not know how to put two simple matches in one line. Maybe someone does and can post it here
Select-String -Path C:\log\1\*.txt -SimpleMatch "[Error]" -ca | select -exp line | out-file C:\log\1\new.txt -Append
Select-String -Path C:\log\1\*.txt -SimpleMatch "[File]" -ca | select -exp line | out-file C:\log\1\new.txt -Append
Regards
-----edit-----
If you want to you may not append it anywhere just display - simply dont pipe it to out-file
use index then check it :
New-Item C:\log\outputlog.txt
$Files = Get-ChildItem "C:\log\1\" -Include "*.txt"
foreach ($File in $Files)
{
$StringMatch = $null
$StringMatch = Get-Content $File
if($StringMatch.IndexOf("[Error]") -ne -1)
{
Add-Content -Path C:\log\outputlog.txt -Value ($StringMatch+"
-------------------------------------------------------------
")
}
}
# end of script
How can I change the following code to look at all the .log files in the directory and not just the one file?
I need to loop through all the files and delete all lines that do not contain "step4" or "step9". Currently this will create a new file, but I'm not sure how to use the for each loop here (newbie).
The actual files are named like this: 2013 09 03 00_01_29.log. I'd like the output files to either overwrite them, or to have the SAME name, appended with "out".
$In = "C:\Users\gerhardl\Documents\My Received Files\Test_In.log"
$Out = "C:\Users\gerhardl\Documents\My Received Files\Test_Out.log"
$Files = "C:\Users\gerhardl\Documents\My Received Files\"
Get-Content $In | Where-Object {$_ -match 'step4' -or $_ -match 'step9'} | `
Set-Content $Out
Give this a try:
Get-ChildItem "C:\Users\gerhardl\Documents\My Received Files" -Filter *.log |
Foreach-Object {
$content = Get-Content $_.FullName
#filter and save content to the original file
$content | Where-Object {$_ -match 'step[49]'} | Set-Content $_.FullName
#filter and save content to a new file
$content | Where-Object {$_ -match 'step[49]'} | Set-Content ($_.BaseName + '_out.log')
}
To get the content of a directory you can use
$files = Get-ChildItem "C:\Users\gerhardl\Documents\My Received Files\"
Then you can loop over this variable as well:
for ($i=0; $i -lt $files.Count; $i++) {
$outfile = $files[$i].FullName + "out"
Get-Content $files[$i].FullName | Where-Object { ($_ -match 'step4' -or $_ -match 'step9') } | Set-Content $outfile
}
An even easier way to put this is the foreach loop (thanks to #Soapy and #MarkSchultheiss):
foreach ($f in $files){
$outfile = $f.FullName + "out"
Get-Content $f.FullName | Where-Object { ($_ -match 'step4' -or $_ -match 'step9') } | Set-Content $outfile
}
If you need to loop inside a directory recursively for a particular kind of file, use the below command, which filters all the files of doc file type
$fileNames = Get-ChildItem -Path $scriptPath -Recurse -Include *.doc
If you need to do the filteration on multiple types, use the below command.
$fileNames = Get-ChildItem -Path $scriptPath -Recurse -Include *.doc,*.pdf
Now $fileNames variable act as an array from which you can loop and apply your business logic.
Other answers are great, I just want to add... a different approach usable in PowerShell:
Install GNUWin32 utils and use grep to view the lines / redirect the output to file http://gnuwin32.sourceforge.net/
This overwrites the new file every time:
grep "step[49]" logIn.log > logOut.log
This appends the log output, in case you overwrite the logIn file and want to keep the data:
grep "step[49]" logIn.log >> logOut.log
Note: to be able to use GNUWin32 utils globally you have to add the bin folder to your system path.
I have trouble to use several commands in one Foreach or Foreach-Object loop
My situation is -
I have many text files, about 100.
So they are read Get-ChildItem $FilePath -Include *.txt
Every file's structure is same only key information is different.
Example
User: Somerandomname
Computer: Somerandomcomputer
With -Replace command I remove "User:" and "Computer:" so $User = Somerandomname and $computer = "Somerandomcomputer.
In each circle $user and $Computer with -Append should be written to one file. And then next file should be read.
foreach-object { $file = $_.fullname;
should be used, but I can not figure out the right syntax for it. Could someone help me with it?
Assuming you've defined $FilePath, $user, and/or $computer elsewhere, try something like this.
$files = Get-ChildItem $FilePath\*.txt
foreach ($file in $files)
{
(Get-Content $file) |
Foreach-Object { $content = $_ -replace "User:", "User: $user" ; $content -replace "Computer:", "Computer: $computer" } |
Set-Content $file
}
You can use ; to delimit additional commands in within the Foreach-Object, for example if you wanted to have separate commands for your user and computer name. If you don't enclose the Get-Content cmdlet with parenthesis you will get an error because that process will still have $file open when Set-Content tries to use it.
Also note that with Powershell, strings in double quotes will evaluate variables, so you can put $user in the string to do something like "User: $user" if you so desired.
Try this:
gci $FilePath -Include *.txt | % {
gc $_.FullName | ? { $_ -match '^(?:User|Computer): (.*)' } | % { $matches[1] }
} | Out-File 'C:\path\to\output.txt'
If User and Computer are on separate lines, you need to read the lines two at a time. The ReadCount parameter of Get-Content allows you to do that.
Get-ChildItem $FilePath -Include *.txt `
| Get-Content -ReadCount 2 `
| %{ $user = $_[0] -replace '^User: ', ''; $computer = $_[1] -replace '^Computer: ', ''; "$user $computer" } `
| Out-File outputfile.txt
This makes the assumption that every file contains only lines of the exact form
User: someuser
Computer: somecomputer
User: someotheruser
Computer: someothercomputer
...
If this is not the case, you will need to provide whatever is the exact file format.