I have a list of file Id's
and I want to find the system feed files containing any of these numbers from a vast directory using powershell.
I was using Get-Content Cash* -totalcount 1 > cash_Check_outputfile.txt
but as each file contains numerous headers this was not working as I hoped.
Can anyone point me in the right direction?
Much appreciated.
Use the Get-ChildItem cmdlet to retrieve all files and use the Select-String cmdlet to find the files containing any number within your dictionary ($test in this example). Finally, use the Select-Object cmdlet to get the path.
$test = #(
707839
709993
)
$pathToSearch = 'C:\test'
Get-ChildItem $pathToSearch | Select-String $test | Select-Object -ExpandProperty Path
Related
Thanks in advance for the help.
I have a folder with multiple CSV files. I’d like to be able to extract the first line of each of the files and store the results in a separate CSV file. The newly created CSV file will have the first column as the file name and the second column to be the first line of the file.
The output should look something like this (as an exported CSV File):
FileName,FirstLine
FileName1,Col1,Col2,Col3
FileName2,Col1,Col2,Col3
Notes:
There are other files that should be ignored. I’d like the code to loop through all CSV files which match the name pattern. I’m able to locate the files using the below code:
$targetDir ="C:\CSV_Testing\"
Get-ChildItem -Path $targetDir -Recurse -Filter "em*"
I’m also able to read the first line of one file with the below code:
Get-Content C: \CSV_Testing\testing.csv | Select -First 1
I guess I just need someone to help with looping through the files and exporting the results. Is anyone able to assist?
Thanks
You basically need a loop, to enumerate each file, for this you can use ForEach-Object, then to construct the output you need to instantiate new objects, for that [pscustomobject] is the easiest choice, then Export-Csv will convert those objects into CSV.
$targetDir = "C:\CSV_Testing"
Get-ChildItem -Path $targetDir -Recurse -Filter "em*.csv" | ForEach-Object {
[pscustomobject]#{
FileName = $_.Name
FirstLine = $_ | Get-Content -TotalCount 1
}
} | Export-Csv path\to\theResult.csv -NoTypeInformation
I have assumed the files actually have the .Csv extension hence changed your filter to -Filter "em*.csv", if that's not the case you could use the filter as you currently have it.
I am using Beyond Compare and I have gotten it to ouput if there are any differences into a txt file. However, I want to script this in powershell so that if there are no differences in files, the script will continue and do something else. However, I am not sure this is possible. I have looked through Select-String, but unsure if that will be able to do what I am looking for. Attached is what the txt file looks like when there are no differences as well as what it looks like when there are differences.
Is it possible to convert name, size, or modified in the txt file into a variable and then do a condition on whether it is null? Or is there is another way I can do what I am trying to achieve in Powershell? Thanks in advance.
If all you need to know is whether any differences were found, the following should do:
$noDiffs = '' -eq ((Get-Content -Raw report.txt) -split '\r?\n-+\r?\n')[1].Trim()
(Get-Content -Raw report.txt) -split '\r?\n-+\r?\n splits the entire input file by the the divider line (----...) following the table-header line, using a regex (regular expression).
[1] looks a the 2nd element of the resulting array, i.e., whatever comes after the divider line, trims any leading and trailing whitespace, and the result is tested for being the empty string.
With all respect to the question and the answer from #mklement0.
Using Beyond Compare in a PowerShell script is putting the cart before the horse.
There are a lot of cmdlets in PowerShell which would let you easily compare folders (and a lot more) without doing any text scraping.
As using Compare-Object togehter with Get-ChildItem for the given example:
Compare-Object (Get-ChildItem .\Test) (Get-ChildItem .\Test1) -Property Name, Length, LastWriteTime
If you want to do a recursive compare on the relative path, you can do:
Compare-Object (Get-ChildItem .\Test -Recurse -Name) (Get-ChildItem .\Test1 -Recurse -Name)
Note that the -Name parameter will only list relative path strings, if you also want to compare Length and LastWriteTime, You can do:
$TestFolder = 'C:\Test'
$Test1Folder = 'C:\Test1'
$TestFiles = Get-ChildItem $TestFolder -File -Recurse |
Select-Object *,#{N='RelativePath'; E={$_.FullName.SubString($TestFolder.Length)}}
$Test1Files = Get-ChildItem $Test1Folder -File -Recurse |
Select-Object *,#{N='RelativePath'; E={$_.FullName.SubString($Test1Folder.Length)}}
Compare-Object $TestFiles $Test1Files -Property RelativePath,Length,LastWriteTime
If I execute:
Get-ChildItem *.ext -recurse
the output consists of a series of Directory sections followed by one or more columns of info for each matching file separated by said directory sections. Is there something like the Unix find command? In which each matching file name appears on a single line with its full relative path?
Get-Childitem by default outputs a view for format-table defined in a format xml file somewhere.
get-childitem | format-table
get-childitem | format-list *
shows you the actual properties in the objects being output. See also How to list all properties of a PowerShell object . Then you can pick and choose the ones you want. This would give the full pathname:
get-childitem | select fullname
If you want an output to be just a string and not an object:
get-childitem | select -expand fullname
get-childitem | foreach fullname
Resolve-Path with the -Relative switch can be used to display the relative paths of a set of paths. You can collect the full path names (FullName property) from the Get-ChildItem command and use the member access operator . to grab the path values only.
Resolve-Path -Path (Get-ChildItem -Filter *.ext -Recurse).FullName -Relative
Note: The relative paths here only accurately reflect files found within the current directory (Get-ChildItem -Path .), i.e. Get-ChildItem -Path NotCurrentDirectory could have undesirable results.
Get-ChildItem's -Name switch does what you want:
It outputs the relative paths (possibly including subdir. components) of matching files as strings (type [string]).
# Lists file / dir. paths as *relative paths* (strings).
# (relative to the input dir, which is implicitly the current one here).
Get-ChildItem -Filter *.ext -Recurse -Name
Note that I've used -Filter, which significantly speeds up the traversal.
Caveat: As of PowerShell 7.0, -Name suffers from performance problems and behavioral quirks; see these GitHub issues:
https://github.com/PowerShell/PowerShell/issues/9014
https://github.com/PowerShell/PowerShell/issues/9119
https://github.com/PowerShell/PowerShell/issues/9126
https://github.com/PowerShell/PowerShell/issues/9122
https://github.com/PowerShell/PowerShell/issues/9120
I am having some problem passing the path plus filename to a parser. There are about 90 files of 1 GB each involved in my task. Each of the file is contained in a folder of its own. All of the folders are contained under a parent folder.
Goal: Ideally, I would like to parse 20 files simultaneously for multitasking and continue to the next 20 until all 90 files are done.
This would mean that I would like to spawn some concurrent parsing of 20 files in a batch at any one given time. In carrying out the parsing, I would like to use measure-command to time the work from beginning to finish.
Script I have used:
Get-ChildItem –Path "E:\\OoonaFTP\\input\\Videos3\\" -Filter *.mp4 -recurse | select -expand fullname
Foreach-Object {
Measure-Command { "E:\OoonaFTP\Ooona_x64_ver_2.5.13\OoonaParser.exe -encode -dat -drm $_.FullName" } | Select-Object -Property TotalSeconds
}
===============================
I have this working batch script with a for statement but doing each iteration one after another. This is not what is the ideal case though. I would really like to accomplish this in PowerShell and with simultaneous tasks.
Could someone please suggest some ways by which I could accomplish this?
Thank you very much!
Thanks for the various suggestions. I'm curious that some of them lead to empty output in my Powershell (PSVersion: 5.1.18362.145).
I tried a number of these and, inspired by some of them, found the best answer for my case at the moment:
Get-ChildItem *.ext -recurse | Select-Object -property fullname
(When I made the window wide enough I got all the info I needed; in general I suppose I might need to do more to get the formatting I want.)
I have a list of pdf file names which i have stored in a text file line by line as shown below.
322223491.pdf
322223492.pdf
322223493.pdf
the name of the text file is inclusions.txt.
I am passing this to a variable:
$inclusion = get-content .\inclusions.txt
Now i want to check and display if a folder (C:\users\xyz) contains files passed in $inclusions variable how do i do that?
PS C:\users\xyz> Get-childitem
command displays the following
Mode LastWritetime Name
The Name column is the property of interest to me where in i want to compare my inclusions list and if its found in the list then i want to display them in the Powershell terminal
PS C:\users\xyz> Get-childitem | Select-Object -Property Name
is as far as i have got my head around it i dont know how to proceed further to display the filtered data by comparing it with the $inclusions variable.
Furthermore i want to delete the files if they match the name in the inclusions text file. So that is the final Objective to traverse through all the files in the folder compare if the name is same as in the inclusions list and if yes then delete those particular files.
Any help would be appreciated.
Many thanks
Use Where-Object to filter on a condition:
Get-ChildItem |Where-Object { $inclusion -contains $_.Name }
If you are only interested in the file names themselves, use Select-Object -ExpandProperty to grab just the name of each file:
Get-ChildItem |Where-Object { $inclusion -contains $_.Name } |Select-Object -ExpandProperty Name
If you want to do something based on whether a certain folder contains one or more of the files in $inclusion, use an if statement:
$folderPath = "C:\users\xyz\folder\of\interest"
if(#(Get-ChildItem $folderPath |Where-Object {$inclusion -contains $_.Name}).Count -ge 1)
{
Write-Host "Folder $folderPath contains at least one of the inclusion files"
}
I know how to get the data and search through it using some pattern. But that is not what I need.
Get-ChildItem -recurse -Filter *.xml | Get-Content | Select-String -pattern "something here"
I am searching through 100's of GPO xml files and we are trying to remove GPO's that perform the same thing over and over again. I want to find the unique values and combine them in one big happy gpo and get rid of all the redundant ones.
My goal :
1) Get all information from all *.xml files from 100's of sub folders and combine them into one file.
2) Find all lines that contain the same string and get a count of that string. I need a count for all strings in the combined file.
3) My goal is to find the lines that are unique and save them to a file, for further use.
Here's a quick-and-dirty approach using a Hashtable. Since the Hashtable setter performs an "update or create", you'll end up with a distinct list:
$ht = #{}
Get-ChildItem -recurse -Filter *.xml | Get-Content | %{$ht[$_] = $true}
$ht.Keys
Edit: Just saw you wanted counts as well. You can do this:
$ht = #{}
Get-ChildItem -recurse -Filter *.xml | Get-Content | %{$ht[$_] = $ht[$_]+1}
$ht
To export to CSV:
$ht.GetEnumerator() | select key, value | Export-Csv D:\output.csv