I would like to use PowerShell to add a line break for every tilde it finds in a file.
The source could contain main .in files which contain tildes.
I have this script so far, and could benefit by some assistance in how to tweak it.
This will work for one file, but not for many:
(Get-Content -Path '.\amalgamatedack.in') |
ForEach-Object {$_.Replace('~', "~`r`n")} |
Set-Content -Path '.\amalgamatedack.in'
You can use Get-ChildItem to find all your .in files, then follow the same logic, just replace the input and output hardcoded file name for the absolute path of each file (.FullName property).
Your code could also benefit by using Get-Content -Raw, assuming these files are not very big and they fit in memory, reading the content as single multi-line string is always faster.
# If you need to search recursively for the files use `-Recurse`
Get-ChildItem path\to\sourcefolder -Filter *.in | ForEach-Object {
($_ | Get-Content -Raw).Replace('~', "~`r`n") |
Set-Content -Path $_.FullName
}
Related
Thanks in advance for the help.
I have a folder with multiple CSV files. I’d like to be able to extract the first line of each of the files and store the results in a separate CSV file. The newly created CSV file will have the first column as the file name and the second column to be the first line of the file.
The output should look something like this (as an exported CSV File):
FileName,FirstLine
FileName1,Col1,Col2,Col3
FileName2,Col1,Col2,Col3
Notes:
There are other files that should be ignored. I’d like the code to loop through all CSV files which match the name pattern. I’m able to locate the files using the below code:
$targetDir ="C:\CSV_Testing\"
Get-ChildItem -Path $targetDir -Recurse -Filter "em*"
I’m also able to read the first line of one file with the below code:
Get-Content C: \CSV_Testing\testing.csv | Select -First 1
I guess I just need someone to help with looping through the files and exporting the results. Is anyone able to assist?
Thanks
You basically need a loop, to enumerate each file, for this you can use ForEach-Object, then to construct the output you need to instantiate new objects, for that [pscustomobject] is the easiest choice, then Export-Csv will convert those objects into CSV.
$targetDir = "C:\CSV_Testing"
Get-ChildItem -Path $targetDir -Recurse -Filter "em*.csv" | ForEach-Object {
[pscustomobject]#{
FileName = $_.Name
FirstLine = $_ | Get-Content -TotalCount 1
}
} | Export-Csv path\to\theResult.csv -NoTypeInformation
I have assumed the files actually have the .Csv extension hence changed your filter to -Filter "em*.csv", if that's not the case you could use the filter as you currently have it.
I'm searching through directories recursively. I want to find files that contain text i'm looking for OR the text is in the content of the file.
For example, if I search for "hello", i'd normally do this:
Find matching file names:
get-childitem -filter "*hello*"
Find files that have text in them:
get-childitem -recurse | select-string -pattern "*hello*"
But I want to do both at the same time. Which means you could have files that don't have "hello" in the name but it does appear in the contents of the file. Or visa versa.
EDIT: I considered using where-object with an -or but having trouble figuring out how to construct that properly.
EDIT: My mistake, meant to include select-string in the example.
Ideas?
Thanks
I don't think its possible to use -Filter because you might be excluding those files which's content could contain the word you're looking for.
An easy approach I could think of, is looping through all files in $startPath recursively and, if the word is contained in the file's name, go to the next iteration with continue or break if you want to stop the loop at first finding, and of course, use -Raw for Get-Content:
$startPath = 'C:\path\to\startFolder'
$magicWord = 'hello'
foreach($file in Get-ChildItem $startPath -Recurse -File)
{
if($file.Name -match $magicWord)
{
$file
continue
# or break here if you want to stop the loop
}
if((Get-Content $file.FullName -Raw) -match $magicWord)
{
$file
# break here if you want to stop the loop
}
}
Not sure if using this would be faster or not:
if([system.io.file]::ReadAllText($file.FullName) -match $magicWord)
{
...
}
If I execute:
Get-ChildItem *.ext -recurse
the output consists of a series of Directory sections followed by one or more columns of info for each matching file separated by said directory sections. Is there something like the Unix find command? In which each matching file name appears on a single line with its full relative path?
Get-Childitem by default outputs a view for format-table defined in a format xml file somewhere.
get-childitem | format-table
get-childitem | format-list *
shows you the actual properties in the objects being output. See also How to list all properties of a PowerShell object . Then you can pick and choose the ones you want. This would give the full pathname:
get-childitem | select fullname
If you want an output to be just a string and not an object:
get-childitem | select -expand fullname
get-childitem | foreach fullname
Resolve-Path with the -Relative switch can be used to display the relative paths of a set of paths. You can collect the full path names (FullName property) from the Get-ChildItem command and use the member access operator . to grab the path values only.
Resolve-Path -Path (Get-ChildItem -Filter *.ext -Recurse).FullName -Relative
Note: The relative paths here only accurately reflect files found within the current directory (Get-ChildItem -Path .), i.e. Get-ChildItem -Path NotCurrentDirectory could have undesirable results.
Get-ChildItem's -Name switch does what you want:
It outputs the relative paths (possibly including subdir. components) of matching files as strings (type [string]).
# Lists file / dir. paths as *relative paths* (strings).
# (relative to the input dir, which is implicitly the current one here).
Get-ChildItem -Filter *.ext -Recurse -Name
Note that I've used -Filter, which significantly speeds up the traversal.
Caveat: As of PowerShell 7.0, -Name suffers from performance problems and behavioral quirks; see these GitHub issues:
https://github.com/PowerShell/PowerShell/issues/9014
https://github.com/PowerShell/PowerShell/issues/9119
https://github.com/PowerShell/PowerShell/issues/9126
https://github.com/PowerShell/PowerShell/issues/9122
https://github.com/PowerShell/PowerShell/issues/9120
I am having some problem passing the path plus filename to a parser. There are about 90 files of 1 GB each involved in my task. Each of the file is contained in a folder of its own. All of the folders are contained under a parent folder.
Goal: Ideally, I would like to parse 20 files simultaneously for multitasking and continue to the next 20 until all 90 files are done.
This would mean that I would like to spawn some concurrent parsing of 20 files in a batch at any one given time. In carrying out the parsing, I would like to use measure-command to time the work from beginning to finish.
Script I have used:
Get-ChildItem –Path "E:\\OoonaFTP\\input\\Videos3\\" -Filter *.mp4 -recurse | select -expand fullname
Foreach-Object {
Measure-Command { "E:\OoonaFTP\Ooona_x64_ver_2.5.13\OoonaParser.exe -encode -dat -drm $_.FullName" } | Select-Object -Property TotalSeconds
}
===============================
I have this working batch script with a for statement but doing each iteration one after another. This is not what is the ideal case though. I would really like to accomplish this in PowerShell and with simultaneous tasks.
Could someone please suggest some ways by which I could accomplish this?
Thank you very much!
Thanks for the various suggestions. I'm curious that some of them lead to empty output in my Powershell (PSVersion: 5.1.18362.145).
I tried a number of these and, inspired by some of them, found the best answer for my case at the moment:
Get-ChildItem *.ext -recurse | Select-Object -property fullname
(When I made the window wide enough I got all the info I needed; in general I suppose I might need to do more to get the formatting I want.)
I need to search through 350,000 files to find any that contains certain patterns in the filename. However, the list of patterns (id numbers) that it needs to match is 1000! So I would very much like to be able to script this, because they were originally planning on doing it manually...
So to make it clearer:
Check each File in folder and all subfolders.
If the filename contains any of the IDs in the text file then move it to another file
Otherwise, ignore it.
So I have the basic code that works with a single value:
$name = Get-Content 'C:\test\list.txt'
get-childitem -Recurse -path "c:\test\source\" -filter "*$name*" |
move-item -Destination "C:\test\Destination"
If I change $name to point to a single ID, it works, if I have a single ID in the txt file, it works. Multiple items in a list:
1111111
2222222
3333333
It fails. What am I doing wrong? How can I get it to work? I'm still new to powershell so please be a little more descriptive in any answers.
Your test fails because it is effectively trying to do this (using your test data).
Get-ChildItem -Recurse -Path "c:\test\source\" -filter "*1111111 2222222 3333333*"
Which obviously does not work. It is squishing the array into one single space delimited string. You have to account for the multiple id logic in a different way.
I am not sure which of these will perform better so make sure you test both of these with your own data to get a better idea of execution time.
Cycle each "filter"
$filters = Get-Content 'C:\test\list.txt'
# Get the files once
$files = Get-ChildItem -Recurse -Path "c:\test\source" -File
# Cycle Each ID filter manually
$filters | ForEach-Object{
$singleFilter
$files | Where-Object{$_.Name -like "*$singleFilter*"}
} | Move-Item -Destination "C:\test\Destination"
Make one larger filter
$filters = Get-Content 'C:\test\list.txt'
# Build a large regex alternative match pattern. Escape each ID in case there are regex metacharacters.
$regex = ($filters | ForEach-Object{[regex]::Escape($_)}) -join "|"
# Get the files once
Get-ChildItem -Recurse -path "c:\test\source" -File |
Where-Object{$_.Name -match $regex} |
Move-Item -Destination "C:\test\Destination"
try following this tutorial on how to use get-content function. Looks like when you have a multiple line file, you get an array back. you then have to iterate through your array and use the logic you used for only one item
I'm trying to teach myself enough powershell or batch programming to figure out to achieve the following (I've had a search and looked through a couple hours of Youtube tutorials but can't quite piece it all together to figure out what I need - I don't get Tokens, for example, but they seem necessary in the For loop). Also, not sure if the below is best achieved by robocopy or xcopy.
Task:
Define a list of files to retrieve in a csv (file name will be listed as a 13 digit number, extension will be UNKNOWN, but will usually be .jpg but might occasionally be .png - could this be achieved with a wildcard?)
list would read something like:
9780761189931
9780761189988
9781579657159
For each line in this text file, do:
Search a network folder and all subfolders
If exact filename is found, copy to an arbitrary target (say a new folder created on desktop)
(Not 100% necessary, but nice to have) Once the For loop has completed, output a list of files copied into a text file in the newly created destination folder
I gather that I'll maybe need to do a couple of things first, like define variables for the source and destination folders? I found the below elsewhere but couldn't quite get my head around it.
set src_folder=O:\2017\By_Month\Covers
set dst_folder=c:\Users\%USERNAME&\Desktop\GetCovers
for /f "tokens=*" %%i in (ISBN.txt) DO (
xcopy /K "%src_folder%\%%i" "%dst_folder%"
)
Thanks in advance!
This solution is in powershell, by the way.
To get all subfiles of a folder, use Get-ChildItem and the pipeline, and you can then compare the name to the insides of your CSV (which you can get using import-CSV, by the way).
Get-ChildItem -path $src_folder -recurse | foreach{$_.fullname}
I'd personally then use a function to edit the name as a string, but I know this probably isn't the best way to do it. Create a function outside of the pipeline, and have it return a modified path in such a way that you can continue the previous line like this:
Get-ChildItem -path $src_folder -recurse | foreach{$_.CopyTo (edit-path $_.fullname)}
Where "edit-directory" is your function that takes in the path, and modifies it to return your destination path. Also, you can alternatively use robocopy or xcopy instead of CopyTo, but Copy-Item is a powershell native and doesn't require much string manipulation (which in my experience, the less, the better).
Edit: Here's a function that could do the trick:
function edit-path{
Param([string] $path)
$modified_path = $dst_folder + "\"
$modified_path = $path.substring($src_folder.length)
return $modified_path
}
Edit: Here's how to integrate the importing from CSV, so that the copy only happens to files that are written in the CSV (which I had left out, oops):
$csv = import-csv $CSV_path
Get-ChildItem -path $src_folder -recurse | where-object{$csv -contains $_.name} | foreach{$_.CopyTo (edit-path $_.fullname)}
Note that you have to put the whole CSV path in the $CSV_path variable, and depending on how the contents of that file are written, you may have to use $_.fullname, or other parameters.
This seems like an average enough problem:
$Arr = Import-CSV -Path $CSVPath
Get-ChildItem -Path $Folder -Recurse |
Where-Object -FilterScript { $Arr -contains $PSItem.Name.Substring(0,($PSItem.Length - 4)) } |
ForEach-Object -Process {
Copy-Item -Destination $env:UserProfile\Desktop
$PSItem.Name | Out-File -FilePath $env:UserProfile\Desktop\Results.txt -Append
}
I'm not great with string manipulation so the string bit is a bit confusing, but here's everything spelled out.