I have a sample.txt file where I would like to remove the content, except for it first top line.
My sample.txt file looks like this:
1.dslkjfladsdjgmvjfgmldskbm;sldkvmg,;sdlmg;msj,;sdrl
2.dlkjfsadfjmsal;jsalv;dsvmdsfkgmrg,pvtpvhtphb[h.[y
3.fljsafckg,ksd,v;vyklt;vlkbmyulnmktr,ouf,f/.n,
4.dflcajsmglsdv;ks'ítb;pjk.'gpjnuk. uo.hulmk,vgjhumk.'l
I would like the output result to look like following:
1.dslkjfladsdjgmvjfgmldskbm;sldkvmg,;sdlmg;msj,;sdrl
I have tried these commands but output clears the entire file.
Clear-Content C:\sample.txt | where { $_.length -gt 2}| Set-Content C:\sample.txt
Clear-Content C:\sample.txt | Select -first 2 | Set-Content C:\sample.txt
Could someone please advise what am I missing?
Thanks
From the documentation for the Clear-Content cmdlet:
The Clear-Content cmdlet deletes the contents of an item, such as deleting the text from a file, but it does not delete the item. As a result, the item exists, but it is empty.
By using Clear-Content you are deleting the contents of sample.txt without reading it. Instead, you want to use Get-Content:
(Get-Content C:\sample.txt -First 1) | Set-Content C:\sample.txt
Using the -First parameter (alias of -TotalCount) of the Get-Content cmdlet works much the same as Select-Object -First but allows Get-Content to stop reading the file as soon as we have the number of lines we need; 1, not 2, is passed to retrieve only the first line. Note that the invocation of Get-Content must be surrounded in parentheses otherwise Set-Content will complain that the file is already open. Alternatively, you can read the contents of the file into a variable first...
$firstLine = Get-Content C:\sample.txt -First 1
$firstLine | Set-Content C:\sample.txt
...or do a "safe overwrite" by outputting the filtered text to another file first, then overwrite the original...
Get-Content C:\sample.txt -First 1 | Set-Content C:\sample.txt.tmp
Move-Item C:\sample.txt.tmp C:\sample.txt -Force
(gc C:\sample.txt |select -first 1)|out-file C:\sample.txt
Thanks for the help, I finally succeed applying the following codes:
(Get-Content C:/sample.txt -TotalCount 1) | Set-Content C:\sample.txt
Thanks
Manu
Related
Data mapping project, in house system to new vendor system. First step is find all the occurrences of current database field names (or column names to be precise) in the C# .cs source files. Trying to use Powershell. Have recently created PS searches with Get-ChildItem and Select-String that work well but the search string array was small and easily hard coded inline. But the application being ported has a couple hundred column names and significant amounts of code. So armed with a text file of all the column names Pipleline would seem like a god tool to create a the basic cross ref for further analysis. However, I was not able to get the Pipeline to work with an external variable anyplace other than first step. Trying using -PipelineVariable, $_. and global variable. Did not find anything specific after lots of searching. P.S. This is my first question to StackoOverflow, be kind please.
Here is what I hoped would work but do dice so far.
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr = [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force |
Select-String $s | Select-Object Path, LineNumber, line | Export-csv $outputfile
}
Did find that this will print the list one time but not twice. In fact it seems using the variable in this way results in processing simply skipping any further pipeline steps.
foreach ($s in $arr) {Write-Host $s | Write $s}
If it isn't possible to do this in Powershell easily my fallback is to do with C# although would much rather get the level up with PowerShell if anyone can point me to the correct understanding of how to do things in the Pipepline, or alternatively construct an equivalent function. Seems like such a natural fit for Powershell.
Thanks.
You're calling Export-csv $outputfile in a loop, which rewrites the whole file in every iteration, so that only the last iteration's output will end up in the file.
While you could use -Append to iteratively append to the output file, it is worth aking a step back: Select-String can accept an array of patterns, causing a line that matches any of them to be considered a match.
Therefore, your code can be simplified as follows:
$inputFile = 'C:\DataColumnsNames.txt'
$outputFile = 'C:\DataColumnsUsages.txt'
Get-ChildItem C:\ProjectFolder -Filter *.cs -Recurse -Force -ea SilentlyContinue |
Select-String -Pattern (Get-Content $inputFile) |
Select-Object Path, LineNumber, line |
Export-csv $outputfile
-Pattern (Get-Content $inputFile) passes the lines of input file $inputFile as an array of patterns to match.
By default, these lines are interpreted as regexes (regular expressions); to ensure that they're treated as literals, add -SimpleMatch to the Select-String call.
This answer to a follow-up question shows how to include the specific pattern among the multiple ones passed to -Pattern that matched on each line in the output.
I think you want to append each occurrence to the csv file. And you need to get the content of the file. Try this:
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force | Foreach {
Get-Content "$_.Fullname" | Select-String $s | Select-Object Path, LineNumber, line | Export-csv -Append -Path "$outputfile"
}
}
-Append was not introduced before powershell v3.0 (Windows 8) then try this:
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force | Foreach {
Get-Content "$_.Fullname" | Select-String $s | Select-Object Path, LineNumber, line | ConvertTo-CSV -NoTypeInformation | Select-Object -Skip 1 | Out-File -Append -Path "$outputfile"
}
}
How can I run one single PowerShell script that does the following in series?
Adds a the filename of all csv files in a directory as a column in the end of each file using this script:
Get-ChildItem *.csv | ForEach-Object {
$CSV = Import-CSV -Path $_.FullName -Delimiter ","
$FileName = $_.Name
$CSV | Select-Object *,#{N='Filename';E={$FileName}} | Export-CSV $_.FullName -NTI -Delimiter ","}
Merges all csv files in the directory into a single csv file
Keeping only a header (first row) only from first csv and excluding all other first rows from files.
Similiar to what kemiller2002 has done here, except one script with csv inputs and a csv output.
Bill's answer allows you to combine CSVs, but doesn't tack file names onto the end of each row. I think the best way to do that would be to use the PipelineVariable common parameter to add that within the ForEach loop.
Get-ChildItem \inputCSVFiles\*.csv -PipelineVariable File |
ForEach-Object { Import-Csv $_ | Select *,#{l='FileName';e={$File.Name}}} |
Export-Csv \outputCSVFiles\newOutputFile.csv -NoTypeInformation
That should accomplish what you're looking for.
This is the general pattern:
Get-ChildItem \inputCSVFiles\*.csv |
ForEach-Object { Import-Csv $_ } |
Export-Csv \outputCSVFiles\newOutputFile.csv -NoTypeInformation
Make sure the output CSV file has a different filename pattern, or use a different directory name (like in this example).
If your csv files dont have always same header you can do it :
$Dir="c:\temp\"
#get header first csv file founded
$header=Get-ChildItem $Dir -file -Filter "*.csv" | select -First 1 | Get-Content -head 1
#header + all rows without header into new file
$header, (Get-ChildItem $Dir -file -Filter "*.csv" | %{Get-Content $_.fullname | select -skip 1}) | Out-File "c:\temp\result.csv"
What I need to do is to take a number of text files (csv, all in one directory) and create files where the entire output is reversed line by line, but keeping the header at the top instead of ending up at the bottom.
I am able to take one file (by name) and copy the first line and create a new file with just that line in it. Then I take the original file minus the first line, read it into an array and reverse it. I then append that to the file that only has the header. It works fine, except for the output name, which I'd like to be [file-REV.csv] but so far I've only gotten to [file.csv-REV]...
So, once I had that working, I thought it was time to have the program find all of the csv's in the directory and loop through them, creating a reverse file for each.
This is what I have so far:
cd c:\users\$([Environment]::UserName)\working
$Path = "c:\users\$([Environment]::UserName)\working\"
ForEach ($file in Get-ChildItem -Filter *.csv) {
Get-Content $file -totalcount 1 | Set-Content .\$file-REV.csv
$flip = (get-content | select -Skip 1)
[array]::Reverse($flip)
$flip | add-content "$file-REV.csv"
}
Here is the message I receive when executing the script:
cmdlet Get-Content at command pipeline position 1
Supply values for the following parameters:
Path[0]:
I've tried to put in the entire path Get-Content -Path c:\users\jmurphy\working\ and then it complains that it can't find the entire path.
Couple things. First you are defining the folder to work in a variable, so use that in the Get-ChildItem. (I change the name to $Folder out of habit because $Path is already used variable in the Environment scope. Also $env: is a quicker way to read the variables out of the Environment scope.).
$Folder = "C:\Users\$env:UserName\working\"
$Files = Get-ChildItem $Folder -Filter *.csv
Second, you'll just want to use the Fullname property from what's returned from Get-ChildItem because that's the full path of each file.
ForEach ($File in $Files) {
So you'll want to use that full path to the file in your Get-Content
Get-Content $File.Fullname -totalcount 1 | Set-Content .\$($file.Basename)-REV.csv
and you'll want to use $File again as the path to the file when you call Get-Content again:
$Flip = (Get-Content $File.Fullname | Select -Skip 1)
Basename is the property from Get-Childitem with just the filename without the extension. You can force an evaluation of a property of a variable inside double quotes by enclosing in $().
$Flip | Add-Content "$($file.Basename)-REV.csv"
All together the script should look like this:
$Folder = "C:\Users\$env:UserName\working\"
$Files = Get-ChildItem $Folder -Filter *.csv
ForEach ($File in $Files) {
Get-Content $File.Fullname -totalcount 1 | Set-Content .\$($file.Basename)-REV.csv
$Flip = (Get-Content $File.Fullname | select -Skip 1)
[array]::Reverse($Flip)
$Flip | Add-Content "$($file.basename)-REV.csv"
}
Joe, you are doing great at learning PowerShell. You just need to harness the power of object orientation a little more.
You already know that Get-ChildItem will return an object about the file(s). What you need to know are the members included in that object. Use Get-Member to find that out.
$file = Get-ChildItem .\t.csv
$file | Get-Member
Once you find an interesting member, see what its value is.
$file.Name
$file.Extension
$file.BaseName
From this, you can construct a new file name.
$newFilename = $file.Basename + '-REV' + $file.Extension
Put this before the Get-Content line. Then use $newFilename with Add-Content.
Hello my question is is there a faster way to search for a phrase in a file other than select-string. I need to find a certain phrase in the first line of about 60k files but the current way i am doing it is too slow for what i need to have done.
I have tried doing
(Select-String "Phrase I am looking for" (cat mylist1)).Filename > mylist2
which gave me a result of 2 minutes and 30 seconds and then i tried
cat mylist1| %{ if ((cat $_ -first 1) -match "Phrase I am looking for") {echo $_}} > mylist2
which gave me a result of 2 minute and 57 seconds. Is there another method of searching for a string through a large amount of files that can bring the search time down?
Since you have at least PowerShell 3.0 then you could use .Where with Get-Content's -TotalCount and that should help some. -TotalCount defines how many lines of the file are being read in. I see that you are already using its alias -First though so there won't be any big changes here for that.
$path = "d:\temp"
$matchingPattern = "function"
(Get-ChildItem $path -File).Where{(Get-Content $_ -TotalCount 1) -match $matchingPattern }
I will try and test this against 60K of files and see what I can get in the mean htim. The above would return file objects where the first line contains "function". My test ran against 60K of files but my lines were likely shorter. Still did it in 44 seconds so perhaps that will help you
StreamReader will usually beat out Get-Content as well but since we are only getting one line I don't think it will be more efficient. This uses a streamreader in the where clause and reads the first line.
(Get-ChildItem $path -File).Where{([System.IO.File]::OpenText($_.Fullname).ReadLine()) -match $matchingPattern }
Note that the above code could contain a memory leak but it finished in 8 seconds compared to my first test. Writing to file added a second or two. Your mileage will vary.
Note that -match supports regex so you would need to escape regex meta characters if present.
You can do simply it:
$yoursearch = "PowerShell is cool!"
get-content "c:\temp\*.*" -TotalCount 1 | where { $_ -ilike "*$yoursearch*"} | select PSPath, #{N="Founded";E={$_}}
or A short version for non-purists:
gc "c:\temp\*.*" -To 1 | ? { $_ -ilike "*$yoursearch*"} | select PSPath, {$_}
If you want export your result:
$yoursearch = "PowerShell is cool!"
get-content "c:\temp\*.*" -TotalCount 1 | where { $_ -ilike "*$yoursearch*"} | select PSPath, #{N="Founded";E={$_}} |
export-csv "c:\temp\yourresult.csv" -notype
If you want a better filter for files input :
Get-ChildItem "c:\temp" -File |
Where {$firstrow= (Get-Content $_.FullName -TotalCount 1); $firstrow -ilike "*$yoursearch*"} |
Select fullName, #{N="Founded";E={$firstrow}} |
Export-Csv "c:\temp\yourresult.csv" -notype
or A short version for non-purists:
gci "c:\temp" -File | ? {$r= (gc $_.FullName -TotalCount 1); $r -ilike "*$yoursearch*"} |
Select f*, #{N="Founded";E={$r}} |
epcsv "c:\temp\yourresult.csv" -notype
Note: -file option exist only in PowerShell V5 (or +), else use psiscontainer propertie into where instruction
Note2: You can use option -list of select-string, seach all in file but stop when 1 row is founded
$yoursearch = "PowerShell where are you"
Select-String -Path "c:\temp\*.*" -Pattern $yoursearch -list | select Path, Line | export-csv "C:\temp\result.csv" -NoTypeInformation
A quick way to write to a file is to use the StreamWriter object. Assuming the files are in a folder:
$writer = [System.IO.StreamWriter] "selection.txt"
$files = gci -Path $path
$pattern ="Phrase"
$files | %{gc -Path $_.FullName | select -First 1 | ?{$_ -match $pattern}} | %{$writer.WriteLine($_)}
An example on how I would do it would be something like
Get-ChildItem -Path $path | Where-Object{$_.Name -contains "My String"}
This is generally a pretty fast way of achieving this however be advised if you -recurse through the entire C:\ drive then regardless you will be sitting for a minute unless you multi-thread
I'm using the variable $listing as the file name as given below.
Get-Content BHS_output.txt | Select-String -pattern $listing| sort | Foreach-Object {$_ -replace "$listing", ""} > $listing
Where $listing is the variable from the first row of the file on first iteration. On the second iteration
$listing = Get-Content .\file.txt | select-object -First 1 .
$listing must have BHS_E_CNBY20150622035126.CSV .. Not BHS_E_BHSA20150622035126.CSV .
File name : TestFile1_sorted.txt
BHS_E_BHAA20150622035126.CSV
BHS_E_BHSA20150622035126.CSV
BHS_E_CNBY20150622035126.CSV
BHS_E_PACS20150622035126.CSV
The error you would be getting would be about saving to the same file you are reading from which is the product of the pipeline you created. Each line is processed one at a time so the second line is being written to file while the files is still open to keep reading.
(Get-Content $path) | Select-Object -Skip 1 | Set-Content $path
Putting the (Get-Content $path) in brackets will process the entire file into memory. Beware if the file is large. Then the rest of your code will work as normally.
have you tried just "select" instead of "select-object"
Get-Content TestFile1_sorted.txt | Select -Skip 1
Someone answered a question about this before:
Remove Top Line of Text File with Powershell
get-content $file |
select -Skip 1 |
set-content "$file-temp"
move "$file-temp" $file -Force
you can do this:
$a = Get-Content C:\temp.txt
$a[1..-1] | Out-File c:\newfile.txt
if you want to overwrite the file replace the destination
what it does it creates new array and select it from the 2nd to the end