I'm using the variable $listing as the file name as given below.
Get-Content BHS_output.txt | Select-String -pattern $listing| sort | Foreach-Object {$_ -replace "$listing", ""} > $listing
Where $listing is the variable from the first row of the file on first iteration. On the second iteration
$listing = Get-Content .\file.txt | select-object -First 1 .
$listing must have BHS_E_CNBY20150622035126.CSV .. Not BHS_E_BHSA20150622035126.CSV .
File name : TestFile1_sorted.txt
BHS_E_BHAA20150622035126.CSV
BHS_E_BHSA20150622035126.CSV
BHS_E_CNBY20150622035126.CSV
BHS_E_PACS20150622035126.CSV
The error you would be getting would be about saving to the same file you are reading from which is the product of the pipeline you created. Each line is processed one at a time so the second line is being written to file while the files is still open to keep reading.
(Get-Content $path) | Select-Object -Skip 1 | Set-Content $path
Putting the (Get-Content $path) in brackets will process the entire file into memory. Beware if the file is large. Then the rest of your code will work as normally.
have you tried just "select" instead of "select-object"
Get-Content TestFile1_sorted.txt | Select -Skip 1
Someone answered a question about this before:
Remove Top Line of Text File with Powershell
get-content $file |
select -Skip 1 |
set-content "$file-temp"
move "$file-temp" $file -Force
you can do this:
$a = Get-Content C:\temp.txt
$a[1..-1] | Out-File c:\newfile.txt
if you want to overwrite the file replace the destination
what it does it creates new array and select it from the 2nd to the end
Related
I have a folder of pipe delimited text files that I need to remove the last column on. I'm not seasoned in PS but I found enough through searches to help. I have two pieces of code. The first creates new text files in my destination path, keeps the pipe delimiter, but doesn't remove the last column. There are 11 columns. Here is that script:
$OutputFolder = "D:\DC_Costing\Vendor Domain\CostUpdate_Development_Load_To_IMS"
ForEach ($File in (Get-ChildItem "D:\DC_Costing\Vendor Domain\CostUpdate_Development_Stage_To_IMS\*.txt"))
{
(Get-Content $File) | Foreach-Object { $_.split()[0..9] -join '|' } | Out-File $OutputFolder\$($File.Name)
}
Then this second code I tried creates the new text files on my destination path, it DOES get rid of the last column, but it loses the pipe delimiter. Ugh.
$OutputFolder = "D:\DC_Costing\Vendor Domain\CostUpdate_Development_Load_To_IMS"
ForEach ($File in (Get-ChildItem "D:\DC_Costing\Vendor Domain\CostUpdate_Development_Stage_To_IMS\*.txt"))
{
Import-Csv $File -Header col1,col2,col3,col4,col5,col6,col7,col8,col9,col10,col11 -Delimiter '|' |
Foreach-Object {"{0} {1} {2} {3} {4} {5} {6} {7} {8} {9}" -f $_.col1,$_.col2,$_.col3,$_.col4,$_.col5,$_.col6,$_.col7,$_.col8,$_.col9,$_.col10} | Out-File $destination\$($File.Name)
}
I have no clue on what I'm doing wrong. I have no preference in which way I get this done but I need to keep the delimiter and the have the last column removed. Any help would be greatly appreciated.
In your plain-text processing attempt with Get-Content, you simply need to split each line by | first (.Split('|')), before extracting the fields of interest with a range operation (..) and joining them back with |:
Get-Content $File |
Foreach-Object { $_.Split('|')[0..9] -join '|' } |
Out-File $OutputFolder\$($File.Name)
In your Import-Csv-based attempt, you can take advantage of the fact that it will only read as many columns as you supply column names for, via -Header:
# Pass only 10 column names to -Header
Import-Csv $File -Header (0..9).ForEach({ 'col' + $_ }) -Delimiter '|' |
ConvertTo-Csv -Delimiter '|' | # convert back to CSV with delimiter '|'
Select-Object -Skip 1 | # skip the header row
Out-File $destination\$($File.Name)
Note that ConvertTo-Csv, just like Export-Csv by default double-quotes each field in the resulting CSV data / file.
In Windows PowerShell, you cannot avoid this, but in PowerShell (Core) 7+ you can control this behavior with -UseQuotes Never, for instance.
You can give this a try, should be more efficient than using Import-Csv, however note, this should always exclude the last column of your files no matter how many columns they have and assuming they're pipe delimited:
$OutputFolder = "D:\DC_Costing\Vendor Domain\CostUpdate_Development_Load_To_IMS"
foreach ($File in (Get-ChildItem "D:\DC_Costing\Vendor Domain\CostUpdate_Development_Stage_To_IMS\*.txt")) {
[IO.File]::ReadAllLines($File.FullName) | & {
process{
-join ($_ -split '(?=\|)' | Select-Object -SkipLast 1)
}
} | Set-Content (Join-Path $OutputFolder -ChildPath $File.Name)
}
How can I run one single PowerShell script that does the following in series?
Adds a the filename of all csv files in a directory as a column in the end of each file using this script:
Get-ChildItem *.csv | ForEach-Object {
$CSV = Import-CSV -Path $_.FullName -Delimiter ","
$FileName = $_.Name
$CSV | Select-Object *,#{N='Filename';E={$FileName}} | Export-CSV $_.FullName -NTI -Delimiter ","}
Merges all csv files in the directory into a single csv file
Keeping only a header (first row) only from first csv and excluding all other first rows from files.
Similiar to what kemiller2002 has done here, except one script with csv inputs and a csv output.
Bill's answer allows you to combine CSVs, but doesn't tack file names onto the end of each row. I think the best way to do that would be to use the PipelineVariable common parameter to add that within the ForEach loop.
Get-ChildItem \inputCSVFiles\*.csv -PipelineVariable File |
ForEach-Object { Import-Csv $_ | Select *,#{l='FileName';e={$File.Name}}} |
Export-Csv \outputCSVFiles\newOutputFile.csv -NoTypeInformation
That should accomplish what you're looking for.
This is the general pattern:
Get-ChildItem \inputCSVFiles\*.csv |
ForEach-Object { Import-Csv $_ } |
Export-Csv \outputCSVFiles\newOutputFile.csv -NoTypeInformation
Make sure the output CSV file has a different filename pattern, or use a different directory name (like in this example).
If your csv files dont have always same header you can do it :
$Dir="c:\temp\"
#get header first csv file founded
$header=Get-ChildItem $Dir -file -Filter "*.csv" | select -First 1 | Get-Content -head 1
#header + all rows without header into new file
$header, (Get-ChildItem $Dir -file -Filter "*.csv" | %{Get-Content $_.fullname | select -skip 1}) | Out-File "c:\temp\result.csv"
Hello my question is is there a faster way to search for a phrase in a file other than select-string. I need to find a certain phrase in the first line of about 60k files but the current way i am doing it is too slow for what i need to have done.
I have tried doing
(Select-String "Phrase I am looking for" (cat mylist1)).Filename > mylist2
which gave me a result of 2 minutes and 30 seconds and then i tried
cat mylist1| %{ if ((cat $_ -first 1) -match "Phrase I am looking for") {echo $_}} > mylist2
which gave me a result of 2 minute and 57 seconds. Is there another method of searching for a string through a large amount of files that can bring the search time down?
Since you have at least PowerShell 3.0 then you could use .Where with Get-Content's -TotalCount and that should help some. -TotalCount defines how many lines of the file are being read in. I see that you are already using its alias -First though so there won't be any big changes here for that.
$path = "d:\temp"
$matchingPattern = "function"
(Get-ChildItem $path -File).Where{(Get-Content $_ -TotalCount 1) -match $matchingPattern }
I will try and test this against 60K of files and see what I can get in the mean htim. The above would return file objects where the first line contains "function". My test ran against 60K of files but my lines were likely shorter. Still did it in 44 seconds so perhaps that will help you
StreamReader will usually beat out Get-Content as well but since we are only getting one line I don't think it will be more efficient. This uses a streamreader in the where clause and reads the first line.
(Get-ChildItem $path -File).Where{([System.IO.File]::OpenText($_.Fullname).ReadLine()) -match $matchingPattern }
Note that the above code could contain a memory leak but it finished in 8 seconds compared to my first test. Writing to file added a second or two. Your mileage will vary.
Note that -match supports regex so you would need to escape regex meta characters if present.
You can do simply it:
$yoursearch = "PowerShell is cool!"
get-content "c:\temp\*.*" -TotalCount 1 | where { $_ -ilike "*$yoursearch*"} | select PSPath, #{N="Founded";E={$_}}
or A short version for non-purists:
gc "c:\temp\*.*" -To 1 | ? { $_ -ilike "*$yoursearch*"} | select PSPath, {$_}
If you want export your result:
$yoursearch = "PowerShell is cool!"
get-content "c:\temp\*.*" -TotalCount 1 | where { $_ -ilike "*$yoursearch*"} | select PSPath, #{N="Founded";E={$_}} |
export-csv "c:\temp\yourresult.csv" -notype
If you want a better filter for files input :
Get-ChildItem "c:\temp" -File |
Where {$firstrow= (Get-Content $_.FullName -TotalCount 1); $firstrow -ilike "*$yoursearch*"} |
Select fullName, #{N="Founded";E={$firstrow}} |
Export-Csv "c:\temp\yourresult.csv" -notype
or A short version for non-purists:
gci "c:\temp" -File | ? {$r= (gc $_.FullName -TotalCount 1); $r -ilike "*$yoursearch*"} |
Select f*, #{N="Founded";E={$r}} |
epcsv "c:\temp\yourresult.csv" -notype
Note: -file option exist only in PowerShell V5 (or +), else use psiscontainer propertie into where instruction
Note2: You can use option -list of select-string, seach all in file but stop when 1 row is founded
$yoursearch = "PowerShell where are you"
Select-String -Path "c:\temp\*.*" -Pattern $yoursearch -list | select Path, Line | export-csv "C:\temp\result.csv" -NoTypeInformation
A quick way to write to a file is to use the StreamWriter object. Assuming the files are in a folder:
$writer = [System.IO.StreamWriter] "selection.txt"
$files = gci -Path $path
$pattern ="Phrase"
$files | %{gc -Path $_.FullName | select -First 1 | ?{$_ -match $pattern}} | %{$writer.WriteLine($_)}
An example on how I would do it would be something like
Get-ChildItem -Path $path | Where-Object{$_.Name -contains "My String"}
This is generally a pretty fast way of achieving this however be advised if you -recurse through the entire C:\ drive then regardless you will be sitting for a minute unless you multi-thread
I have a sample.txt file where I would like to remove the content, except for it first top line.
My sample.txt file looks like this:
1.dslkjfladsdjgmvjfgmldskbm;sldkvmg,;sdlmg;msj,;sdrl
2.dlkjfsadfjmsal;jsalv;dsvmdsfkgmrg,pvtpvhtphb[h.[y
3.fljsafckg,ksd,v;vyklt;vlkbmyulnmktr,ouf,f/.n,
4.dflcajsmglsdv;ks'ítb;pjk.'gpjnuk. uo.hulmk,vgjhumk.'l
I would like the output result to look like following:
1.dslkjfladsdjgmvjfgmldskbm;sldkvmg,;sdlmg;msj,;sdrl
I have tried these commands but output clears the entire file.
Clear-Content C:\sample.txt | where { $_.length -gt 2}| Set-Content C:\sample.txt
Clear-Content C:\sample.txt | Select -first 2 | Set-Content C:\sample.txt
Could someone please advise what am I missing?
Thanks
From the documentation for the Clear-Content cmdlet:
The Clear-Content cmdlet deletes the contents of an item, such as deleting the text from a file, but it does not delete the item. As a result, the item exists, but it is empty.
By using Clear-Content you are deleting the contents of sample.txt without reading it. Instead, you want to use Get-Content:
(Get-Content C:\sample.txt -First 1) | Set-Content C:\sample.txt
Using the -First parameter (alias of -TotalCount) of the Get-Content cmdlet works much the same as Select-Object -First but allows Get-Content to stop reading the file as soon as we have the number of lines we need; 1, not 2, is passed to retrieve only the first line. Note that the invocation of Get-Content must be surrounded in parentheses otherwise Set-Content will complain that the file is already open. Alternatively, you can read the contents of the file into a variable first...
$firstLine = Get-Content C:\sample.txt -First 1
$firstLine | Set-Content C:\sample.txt
...or do a "safe overwrite" by outputting the filtered text to another file first, then overwrite the original...
Get-Content C:\sample.txt -First 1 | Set-Content C:\sample.txt.tmp
Move-Item C:\sample.txt.tmp C:\sample.txt -Force
(gc C:\sample.txt |select -first 1)|out-file C:\sample.txt
Thanks for the help, I finally succeed applying the following codes:
(Get-Content C:/sample.txt -TotalCount 1) | Set-Content C:\sample.txt
Thanks
Manu
Have this code:
gci D:\Files\Folder |
sort LastWriteTime | select -Last 1 |
foreach-object {$line -replace "\<", ""}
Not working. Tried many variations. Need to replace the "<" character in the file last modified in Folder. Managed to have the correct file selected and written to powershell console. Just cannot remove the "<" character from the file with LastWriteTime.
Windows doesn't allow the < character in file names so I guess you want to modify the file contents removing all occurrences. If so, there are many ways to do that. Example:
# Getting the name of the last modified file.
$file_name = Get-ChildItem D:\Files\Folder | Sort-Object LastWriteTime `
| ? { ! $_.PSIsContainer } | Select-Object -Last 1 | % {$_.FullName }
# Reading the file into a single string.
$string = Get-Content $file_name | Out-String
# Modifying the string and writing the output back to the file.
$string -replace "<", "" | Out-File $file_name
The problem with your initial code is that $line is not defined anywhere. You need to read text from the file first.
If you're trying to replace the < in the filename then this should work:
foreach-object {$_.Name.Replace("<", "")}
To edit the contents of the file you could do this:
$file = gci D:\Files\Folder | sort LastWriteTime | select -Last 1
$temp = $file | gc | foreach-object {$_.Replace("<", "")}
$temp | Out-File $file.FullName