From this answer I can create multiple files a.txt, b.txt, ... , z.txt. in Bash with:
touch {a..z}.txt
Or 152 with:
touch {{a..z},{A..Z},{0..99}}.txt
How can I do this in Powershell?
I know New-Item a.txt, but If I want multiple files as above?
For curiosity, what are the equivalent commands in Command Prompt (cmd.exe)?
For Powershell:
1..5 | foreach { new-item -path c:\temp\$_.txt }
The foreach loop will run for each number in 1 to 5, and generate a file in the desired path with the name of that number passed to the command (represented by the $_)
You could also write it as:
%{1..5} | new-item c:\temp\$_.txt
For cmd:
for /L %v in (1,1,5) do type nul > %v.txt
More information here: cmd/batch looping
Not quite as concise as bash, but it can be done.
#(97..(97+25)) + #(48..(48+9)) |
ForEach-Object { New-Item -Path "$([char]$_).txt" -WhatIf }
Another way...
#([int][char]'a'..[int][char]'z') + #([int][char]'0'..[int][char]'9') |
ForEach-Object { New-Item -Path "$([char]$_).txt" -WhatIf }
And one more...
function rng { #($([int][char]$args[0])..$([int][char]$args[1])) }
(rng 'a' 'z') + (rng '0' '9') |
ForEach-Object { New-Item -Path "$([char]$_).txt" -WhatIf }
If you are desperate to do this in a cmd.exe shell, this might work. When it looks like the correct commands are produced, delete or comment out the echo line and remove the rem from the next line.
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET "CLIST=abcdefghijklmnopqrstuvwxyz0123456789"
FOR /L %%i IN (0,1,35) DO (
CALL SET "S=%%CLIST:~%%i,1%%.txt"
echo TYPE NUL ^>"!S!"
rem TYPE NUL >"!S!"
)
#Emilson Inoa Your solution is very ingenuous; there's a typo in the names, am sure you meant to exclude the extensions in the array. You'll end up with
("Text1", "Text2", "Text3") | % {ni -Path "/path/to/dir" -Name "$_.txt"}
For letters, in PowerShell, use:
97..( 97+25 ) | foreach { new-item $env:temp\$( [char]$_ ).txt }
Very simple, take a look:
("Text1.txt","Text2.txt", "Text3.txt") | foreach { New-Item -Path "X" -Name "$_.txt" }
You will replace X of course with the path where you want the files to be created.
If further explanation is required, let me know.
The following command in powershell will create multiple files (20) named Doc_.txt in the directory that the command is executed.
new-item $(1..20 | %{"Doc$_.txt"})
I think this command is the closest to the bash equivalent:
touch Doc{1..20}.txt
This doesn't directly answer your question (as this method requires each file extension be provided), but a more rudimentary way is of course:
New-Item {a..z}.txt, {A..Z}.txt, {0..9}.txt
After a few failed attempts, I mashed a couple of the answers above to create files titled [char][int].txt: 1..5 | foreach {New-Item -Path 'X' -Name "abc$_.txt"}, where X is the path. Also, just to thank the original writer of the question, as it described really succinctly exactly the problem I was trying to solve.
1..1000000 | foreach {new-item -path C:\Testdata$_.txt}
This worked fine for me. Created 1M files in the C:\Testdata folder.
In the PowerShell, you can use New-Item
New-Item a.txt, b.txt, c.txt
then hit Enter
Related
Hello I'm looking for powershell script which would merge all csv files in a directory into one text file (.txt) . All csv files have same header which is always stored in a first row of every file. So I need to take header from the first file, but in rest of the files the first row should be skipped.
I was able to find batch file which is doing exactly what I need, but I have more than 4000 csv files in a single directory and it takes more than 45 minutes to do the job.
#echo off
ECHO Set working directory
cd /d %~dp0
Deleting existing combined file
del summary.txt
setlocal ENABLEDELAYEDEXPANSION
set cnt=1
for %%i in (*.csv) do (
if !cnt!==1 (
for /f "delims=" %%j in ('type "%%i"') do echo %%j >> summary.txt
) else (
for /f "skip=1 delims=" %%j in ('type "%%i"') do echo %%j >> summary.txt
)
set /a cnt+=1
)
Any suggestion how to create powershell script which would be more efficient than this batch code?
Thank you.
John
If you're after a one-liner you can pipe each csv to an Import-Csv and then immediately pipe that to Export-Csv. This will retain the initial header row and exclude the remaining files header rows. It will also process each csv one at a time rather than loading all into memory and then dumping them into your merged csv.
Get-ChildItem -Filter *.csv | Select-Object -ExpandProperty FullName | Import-Csv | Export-Csv .\merged\merged.csv -NoTypeInformation -Append
This will append all the files together reading them one at a time:
get-childItem "YOUR_DIRECTORY\*.txt"
| foreach {[System.IO.File]::AppendAllText
("YOUR_DESTINATION_FILE", [System.IO.File]::ReadAllText($_.FullName))}
# Placed on seperate lines for readability
This one will place a new line at the end of each file entry if you need it:
get-childItem "YOUR_DIRECTORY\*.txt" | foreach
{[System.IO.File]::AppendAllText("YOUR_DESTINATION_FILE",
[System.IO.File]::ReadAllText($_.FullName) + [System.Environment]::NewLine)}
Skipping the first line:
$getFirstLine = $true
get-childItem "YOUR_DIRECTORY\*.txt" | foreach {
$filePath = $_
$lines = $lines = Get-Content $filePath
$linesToWrite = switch($getFirstLine) {
$true {$lines}
$false {$lines | Select -Skip 1}
}
$getFirstLine = $false
Add-Content "YOUR_DESTINATION_FILE" $linesToWrite
}
Try this, it worked for me
Get-Content *.csv| Add-Content output.csv
This is pretty trivial in PowerShell.
$CSVFolder = 'C:\Path\to\your\files';
$OutputFile = 'C:\Path\to\output\file.txt';
$CSV = Get-ChildItem -Path $CSVFolder -Filter *.csv | ForEach-Object {
Import-Csv -Path $_
}
$CSV | Export-Csv -Path $OutputFile -NoTypeInformation -Force;
Only drawback to this approach is that it does parse every file. It also loads all files into memory, so if we're talking about 4000 files that are 100 MB each you'll obviously run into problems.
You might get better performance with System.IO.File and System.IO.StreamWriter.
Your Batch file is pretty inefficient! Try this one (you'll be surprised :)
#echo off
ECHO Set working directory
cd /d %~dp0
ECHO Deleting existing combined file
del summary.txt
setlocal
for %%i in (*.csv) do set /P "header=" < "%%i" & goto continue
:continue
(
echo %header%
for %%i in (*.csv) do (
for /f "usebackq skip=1 delims=" %%j in ("%%i") do echo %%j
)
) > summary.txt
How this is an improvement
for /f ... in ('type "%%i"') requires to load and execute cmd.exe in order to execute the type command, capture its output in a temporary file and then read data from it, and this is done with each input file. for /f ... in ("%%i") directly reads data from the file.
The >> redirection opens the file, appends data at end and closes the file, and this is done with each output *line*. The > redirection keeps the file open all the time.
If you need to scan folder recursively then you can use the approach below
Get-ChildItem -Recurse -Path .\data\*.csv | Get-Content | Add-Content output.csv
what this basically does is:
Get-ChildItem -Recurse -Path .\data\*.csv Find the requested files recursively
Get-Content Get content for each
Add-Content output.csv append it to output.csv
Here is a version also using System.IO.File,
$result = "c:\temp\result.txt"
$csvs = get-childItem "c:\temp\*.csv"
#read and write CSV header
[System.IO.File]::WriteAllLines($result,[System.IO.File]::ReadAllLines($csvs[0])[0])
#read and append file contents minus header
foreach ($csv in $csvs) {
$lines = [System.IO.File]::ReadAllLines($csv)
[System.IO.File]::AppendAllText($result, ($lines[1..$lines.Length] | Out-String))
}
Get-ChildItem *.csv|select -First 1|Get-Content|select -First 1|Out-File -FilePath .\input.csv -Force #Get the header from one of the CSV Files, write it to input.csv
Get-ChildItem *.csv|foreach {Get-Content $_|select -Skip 1|Out-File -FilePath .\Input.csv -Append} #Get the content of each file, excluding the first line and append it to input.csv
stinkyfriend's helpful answer shows an elegant, PowerShell-idiomatic solution based on Import-Csv and Export-Csv.
Unfortunately,
it is quite slow because it involves ultimately unnecessary round-trip conversion to and from objects.
also, even though it shouldn't matter to a CSV parser, the specific format of the files can get altered in the process, because Export-Csv double-quotes all column values, invariably so in Windows PowerShell, by default in PowerShell (Core) 7+, which now offers opt-in control via -UseQuotes and -QuoteFields).
When performance matters, a plain-text solution is required, which also avoids any inadvertent format alteration (just like the linked answer it assumes that all input CSV files have the same column structure).
The following PSv5+ solution:
reads each input file's content into memory in full, as a single multi-line string, using Get-Content -Raw (which is much faster than the default line-by-line reading),
skips the header line for all but the first file with -replace '^.+\r?\n', using the regex-based -replace operator,
and saves the results to the target file with Set-Content -NoNewLine.
Character-encoding caveat:
PowerShell never preserves the input character encoding of files, so you may have to use the -Encoding parameter to override Set-Content's default encoding (the same applies to Export-Csv and any other file-writing cmdlets; in PowerShell (Core) 7+ all cmdlets now consistently default to BOM-less UTF-8; but not only do Windows PowerShell cmdlets not default to UTF-8, they use varying encodings - see the bottom section of this answer).
# Determine the output file and remove a preexisting one, if any.
$outFile = 'summary.csv'
if (Test-Path $outFile) { Remove-Item -ErrorAction Stop $outFile }
# Process all *.csv files in the current folder and merge their contents,
# skipping the header line for all but the first file.
$first = $true
Get-ChildItem -Filter *.csv |
Get-Content -Raw |
ForEach-Object {
$content =
if ($first) { # first file: output content as-is
$_; $first = $false
} else { # subsequent file: skip the header line.
$_ -replace '^.+\r?\n'
}
# Make sure that each file content ends in a newline
if (-not $content.EndsWith("`n")) { $content += [Environment]::NewLine }
$content # Output
} |
Set-Content -NoNewLine $outFile # add -Encoding as needed.
The modern Powershell 7 answer:
(Assuming all csv files are on the same directory and have the same amount of fields.)
#(Get-ChildItem -Filter *.csv).fullname | Import-Csv |Export-Csv ./merged.csv -NoTypeInformation
First part of the pipeline gets all the .csv files and parses the fullname (Path + filename + extension), then import CSV takes each and creates an object and then each object gets merged into a single CSV file with only one header.
I found the previous solutions quite inefficient for large csv-files in terms of performance, so here is a performant alternative.
Here is an alternative which simply appends the files:
cmd /c copy ((gci "YOUR_DIRECTORY\*.csv" -Name) -join '+') "YOUR_OUTPUT_FILE.csv"
Thereafter, you probably want to get rid of the multiple csv-headers.
The following batch script is very fast. It should work well as long as none of your CSV files contain tab characters, and all source CSV files have fewer than 64k lines.
#echo off
set "skip="
>summary.txt (
for %%F in (*.csv) do if defined skip (
more +1 "%%F"
) else (
more "%%F"
set skip=1
)
)
The reason for the restrictions is that MORE converts tabs into a series of spaces, and redirected MORE hangs at 64k lines.
#Input path
$InputFolder = "W:\My Documents\... input folder"
$FileType = "*.csv"
#Output path
$OutputFile = "W:\My Documents\... some folder\merged.csv"
#Read list of files
$AllFilesFullName = #(Get-ChildItem -LiteralPath $InputFolder -Filter $FileType | Select-Object -ExpandProperty FullName)
#Loop and write
Write-Host "Merging" $AllFilesFullName.Count $FileType "files."
foreach ($FileFullName in $AllFilesFullName) {
Import-Csv $FileFullName | Export-Csv $OutputFile -NoTypeInformation -Append
Write-Host "." -NoNewline
}
Write-Host
Write-Host "Merge Complete"
$pathin = 'c:\Folder\With\CSVs'
$pathout = 'c:\exported.txt'
$list = Get-ChildItem -Path $pathin | select FullName
foreach($file in $list){
Import-Csv -Path $file.FullName | Export-Csv -Path $pathout -Append -NoTypeInformation
}
type *.csv >> folder\combined.csv
first of all: I'm on a Windows 7 machine ;).
I got a folder with several dozens files. Each file contains about 240.000 rows. But only half of that rows are needed.
What I would like to do is: have a script that runs over these files, filters out every row that contains the string "abcd" and have it either saved in a new directory or just saved in the same file.
I would try using Powershell as below:
$currentPath = "the path these files currently in"
$newPath = "the path you want to put the new files"
$files = Get-ChildItem $currentPath
foreach ($item in $files) {
Get-Content $item | Where-Object {$_ -notmatch 'abcd'} |Set-Content $newPath\$item
}
#echo off
setlocal enableextensions
set "_where=c:\some\where\*.txt"
set "_filter=abcd"
rem find files which needs filter
for /f "tokens=*" %%f in ('findstr /m "%_filter%" "%_where%"') do (
rem generate a temporary file with the valid content
findstr /v /c:"%_filter%" "%%~ff" > "%%~ff.tmp"
rem rename original file to .old
ren "%%~ff" *.old > nul
rem rename temporary file as original file
ren "%%~ff.tmp" "%%~nxf" > nul
)
rem if needed, delete *.old files
you might use sed for Windows
sed -i.bak "/abcd/!d" *.txt
Find all abcd containing rows in .txt files, make a backup file .bak and store the the detected lines in the original file.
#echo on
For %%a in (*.txt) do (CALL:FILTER "%%a")
echo/Done.&pause>nul&exit/b
:FILTER
type "%~1"|find "abcd" 1>nul 2>nul
if %errorlevel% EQU 0 find /n "abcd" "%~1">"Found abcd in %~1.txt"
The command Find returns error level = 0 if him find something
If the files are that big, I'd so something like this:
$Folder = 'C:\MyOldFiles'
$NewFolder = 'C:\MyNewFiles'
New-Item -ItemType Directory -Path $NewFolder -Force
foreach ($file in Get-ChildItem $Folder)
{
Get-Content $file -ReadCount 1500 |
foreach { $_ -notmatch 'abcd' } |
Add-Content "$NewFolder\$($file.name)"
}
I am trying to execute powershell if-else from cmd.
For example to check the number of files, that has "temp" in its name in D: drive,I used,
if(($i=ls D:\* | findstr /sinc:Temp).count -ne 0 ) {Write-Host $i}
This works fine from PS windows
But if want to do the same from cmd, How do i do it?
I tried
powershell -noexit if(($i=ls D:\* | findstr /sinc:Temp).count -ne 0 ) {Write-Host $i}
which did not work unfortunately.
Just put the command in double quotes:
powershell "if(($i=ls D:\* | findstr /sinc:Temp).count -ne 0 ) {Write-Host $i}"
I also think you don't need the -NoExit switch here. This switch prevents powershell from exiting after running the command. If you want to return back to cmd, remove this switch.
I know this doesn't answer how to run the command(others have covered it already), but why would you want to combine cmd and powershell, when both can do the job alone?
Ex powershell:
#Get all files on D: drive with temp in FILEname(doesn't check if a foldername was temp)
Get-ChildItem D:\ -Recurse -Filter *temp* | where { !$_.PSIsContainer } | foreach { $_.fullname }
#Check if fullpath includes "temp" (if folder in path includes temp, all files beneath are shown)
Get-ChildItem D:\ -Recurse | where { !$_.PSIsContainer -and $_.FullName -like "*temp*" } | foreach { $_.fullname }
Ex cmd:
#Get all files with "temp" in filename
dir d:\*temp* /s /a:-d /b
Just another solution for you problem without using powershell:
dir /b D:\*Temp* | find /v /c "::"
This will print just the number of files or folders on D: that have "Temp" in their names. The double-colon here is just a string that should not be in the output of dir /b, so find /v /c "::" counts all lines of the output of dir /b.
#utapyngo's double quote solution works.
Also another way for #utapyngo's another way to make it in cmd:
dir /b D:\* | find /c "*Temp*"
And to Bill: there should not be a opening double quote before & in your first code, I guess?
powershell -noexit "& "C:\........\run_script.ps1"
see these -- http://poshoholic.com/2007/09/27/invoking-a-powershell-script-from-cmdexe-or-start-run/
http://www.leeholmes.com/blog/2006/05/05/running-powershell-scripts-from-cmd-exe/
or for V2
Powershell.exe -File C:\.........\run_script.ps1
I've been trying to find a script that recursively prints all files and folders within a directory like this where the backslash is used to indicate directories:
Source code\
Source code\Base\
Source code\Base\main.c
Source code\Base\print.c
List.txt
I'm using PowerShell 3.0 and most other scripts I've found do not work (though they didn't anything like what I'm asking).
Additionally: I need it to be recursive.
What you are likely looking for is something to help distinguish a file from a folder. Luckily there is a property call PSIsContainer that is true for folder and false for files.
dir -r | % { if ($_.PsIsContainer) { $_.FullName + "\" } else { $_.FullName } }
C:\Source code\Base\
C:\Source code\List.txt
C:\Source code\Base\main.c
C:\Source code\Base\print.c
If the leading path information is not desirable, you can remove it easily enough using -replace:
dir | % { $_.FullName -replace "C:\\","" }
Hopefully this gets you headed off in the right direction.
It could be like:
$path = "c:\Source code"
DIR $path -Recurse | % {
$_.fullname -replace [regex]::escape($path), (split-path $path -leaf)
}
Following the #Goyuix idea:
$path = "c:\source code"
DIR $path -Recurse | % {
$d = "\"
$o = $_.fullname -replace [regex]::escape($path), (split-path $path -leaf)
if ( -not $_.psiscontainer) {
$d = [string]::Empty
}
"$o$d"
}
dir | % {
$p= (split-path -noqualifier $_.fullname).substring(1)
if($_.psiscontainer) {$p+'\'} else {$p}
}
This one shows full paths, as some of the other answers do, but is shorter:
ls -r | % { $_.FullName + $(if($_.PsIsContainer){'\'}) }
However, the OP I believe asked for relative paths (i.e. relative to the current directory) and only #C.B.'s answer addressed that point. So by just adding a substring we have this:
ls -r | % { $_.FullName.substring($pwd.Path.length+1) + $(if($_.PsIsContainer){'\'}) }
PowerShell Command For Directory List into Txt File:
For Full Path Directory List (Folder & File) to text file:
ls -r | % { $_.FullName + $(if($_.PsIsContainer){'\'}) } > filelist.txt
For Relative Path Directory List (Folder & File) to text file:
ls -r | % { $_.FullName.substring($pwd.Path.length+1) + $(if($_.PsIsContainer){'\'}) } > filelist.txt
Not powershell, but you can use the following within command prompt to recursively list files into a textfile:
dir *.* /s /b /a:-d > filelist.txt
You can achieve this through the get-childitem command in PowerShell. Refer to the below syntax:
Get-ChildItem "Folder name or Path" -Recurse | select FullName > list.txt
This will help you write all the plain files and folders names recursively onto a file called list.txt
Refer to this for more information. https://ss64.com/ps/get-childitem.html
Answering late, but it might help someone!
(ls $path -r).FullName | % {if((get-item "$_").psiscontainer){"$_\"}else{$_}}
Only use in PS 3.0
I made an improved version of the code submitted (since the code output are inside powershell which has an output limit)
Shift + Right Click in the folder you're trying to scan files and folder on
copy and paste this (just edit your_pc_name)
dir -r | % { if ($.PsIsContainer) { $.FullName + "" } else { $_.FullName } } | Out-File -FilePath c:\users\your_pc_name\desktop\OUTPUT.txt
This will print all the files and folders into a txt file in your dekstop.
extra tips:
copy the output and paste it in excel
use ctrl + f to search for the filename you wanted.
This help me a lot in searching for a large database of files.
OK what I am trying to do is make a script that will read each line of a text file, "directory.txt" and export every line that is to a file and not to a directory. Example below.
I'm just trying to remove the paths to directories like "C:\users\"
and keep any path that is to a file like "C:\users\file.txt"
In the test file, Direcory.txt" there will be the following:
C:\path\path\folder\
C:\path\path\file.ext
C:\path\path\path\path\folder
The script will need to read the text file above and export the following line to a new text file.
C:\path\path\file.ext
The batch script equivalent would be the following:
#ECHO OFF
FOR /F %%A IN (directory.txt) DO CALL:NoDir "%%A"
pause
EXIT /B
:NoDir
IF "%~x1" NEQ "" ECHO %~1>>nodir.txt
EXIT /B
Batch script can't handle a file of 400mb so need to use powershell to do it o.0
FTR: The condition if "%~x1" neq "" does not do what you seem to expect. It will match not only folders, but also files without an extension.
Anyway, in PowerShell you'd probably do something like this to list only items that are not directories:
Get-Content \PATH\TO\directory.txt `
| Get-Item `
| Where-Object { -not $_.PSIsContainer } `
| Select-Object FullName
I'm not sure I understand the question (batch example shows all paths), maybe that's what you're looking for:
Get-Content directory.txt | Where-Object {$_ -eq 'C:\path\path\file.ext'}
You didn't say whether you're concerned if they are valid paths or not. The following outputs to the host the lines that are either valid files (using the '-PathType leaf' parameter of the Test-Path cmdlet), or if the last 4 characters of the last item in the path are a dot followed by 3 letters.
$Lines = Get-Content C:\Path\to\file.txt
foreach ( $line in $Lines )
{
if ( (Test-Path $line -PathType leaf) -OR ($line -match "\.\w{3}$") )
{
Write-Host $line
}
}
If you find your file extensions are longer than 3 letters, you can change the regex appropriately ("\.\w{3,4}$" for 4 character extensions, or "\.[\w\d]{3,4}$ to match extensions that are 3 or 4 characters long and might include numbers)
And the one-liner:
Get-Content C:\Path\to\file.txt | % { if ((Test-Path $_ -PathType Leaf) -OR ($_ -match "\.\w{3}$")) { $_ } }