I have several txt files distributed in several sub-folders.
This is what a file looks like:
Data file. version 01.10.
1
8
*
DAT\Trep\Typ10
DAT\Trep\Typ12
DAT\Trep\Typ13
what I would like to do is to extract only the part after the last "\" in order to get something like this:
Typ10 FileName.txt Path
Typ12 FileName.txt Path
Typ13 FileName.txt Path
...
I tried the following
Get-ChildItem -Path 'D:\MyData\*.txt' -Recurse | ForEach-Object {Write-Output $_; $data=Get-Content $_}
$data = $data -replace '.*\\'
$data
it works well for a single file but not with several (-recurse).
Being a powershell beginner I can't figure out how to improve my script.
I also tried to add this to get the result shown above in my post, but that doesn't work either.
Select-Object -Property #{Name = 'Result list'; Expression = { $data }}, Filename, Path
Thanks in advance for your kind help
Use Select-String:
Get-ChildItem -Path D:\MyData\*.txt -Recurse -File |
Select-String '^.+\\(.+)' |
ForEach-Object {
[pscustomobect] #{
Result = $_.Matches.Groups[1].Value
FileName = $_.FileName
Path = $_.Path
}
}
As for your desire to exclude certain folders during recursive traversal:
Unfortunately, Get-ChildItem -Exclude only excludes the matching folders themselves, not also their content. There are two relevant feature requests to potentially overcome this limitation in the future:
GitHub issue #4126 asks for path patterns to be supported too in the future.
GitHub issue #15159 proposes a new subtree-exclusion parameter, such as
-ExcludeRecursive.
For now, a different approach with post-filtering based on Where-Object is required, using folder names Folder1 and Folder2 as examples:
Get-ChildItem -Path D:\MyData\*.txt -Recurse |
Where-Object FullName -NotLike *\Folder1\* |
Where-Object FullName -NotLike *\Folder2\* |
Select-String '^.+\\(.+)' |
ForEach-Object {
[pscustomobect] #{
Result = $_.Matches.Groups[1].Value
FileName = $_.FileName
Path = $_.Path
}
}
For a more flexible, cross-platform approach based on regex matching (which is invariably more complex), see the bottom section of this answer.
Related
Ive been running around like crazy lately with this script that Im trying to modify it to suit my needs. I recently found out that deleting the files using "LastWriteTime" is not what Im after..
What I need my script to do is to delete the files that are older than 30 days using the "CreationTime" property, the problem is that after I modify the script to use this it deletes the entire folder structure?
How can this small modification change the behavior of the entire script?
This is what Im using:
$limit = (Get-Date).AddDays(-30)
$del30 = "D:\CompanyX_ftp\users"
$ignore = Get-Content "C:\Users\UserX\Documents\Scripts\ignorelist.txt"
Get-ChildItem $del30 -Recurse |
Where-Object {$_.CreationTime -lt $limit } |
Select-Object -ExpandProperty FullName |
Select-String -SimpleMatch -Pattern $ignore -NotMatch |
Select-Object -ExpandProperty Line |
Remove-Item -Recurse
So if I were to replace the "CreationTime" property with "LastWriteTime" the script will run and do what its supposed to but if I use "CreationTime" it just deletes everything under the folder structure including the folders themselves and the paths that its supposed to ignore.
UPDATE: The script is working for me now for the actual deletion of the files but for the script that Im using to just get a report on the actual files that the script will delete is actually including the paths of the ignorelist.txt file?
Please see below script:
$limit = (Get-Date).AddDays(-30)
$del30 = "D:\CompanyX_ftp\users"
#Specify path for ignore-list
$ignore = Get-Content "C:\Users\UserX\Documents\Scripts\ignorelist.txt"
Get-ChildItem $del5 -File -Recurse |
Where-Object {$_.CreationTime -lt $limit } |
Select-Object -ExpandProperty FullName |
Select-String -SimpleMatch -Pattern $ignore -NotMatch |
Select-Object -ExpandProperty Line |
Get-ChildItem -Recurse | Select-Object FullName,CreationTime
ignorelist.txt sample data:
D:\CompanyX_ftp\users\ftp-customerA\Customer Downloads
D:\CompanyX_ftp\users\ftp-customerB\Customer Downloads
D:\CompanyX_ftp\users\ftp-customerC\Customer Downloads
D:\CompanyX_ftp\users\ftp-customerD\Customer Downloads
D:\CompanyX_ftp\users\ftp-customerE\Customer Downloads
D:\CompanyX_ftp\users\ftp-customerF\Customer Downloads
D:\CompanyX_ftp\users\ftp-customerG\Customer Downloads
D:\CompanyX_ftp\users\ftp-customerH\Customer Downloads\
Any ideas on why its including the paths that I have mentioned on the ignorelist.txt? (I will also provide an image for better illustration).
Thanks in advance for any help or guidance with this.
//Lennart
I see two problems with the updated code:
Duplicate recursion. First Get-ChildItem iterates over contents of directory recursively. Later in the pipeline another recursive iteration starts on items returned by the first Get-ChildItem, causing overlap.
When filtering by $ignore, only paths that exactly match against the $ignore paths are being ignored. Paths that are children of items in the ignore list are not ignored.
Here is how I would do this. Create a function Test-IgnoreFile that matches given path against an ignore list, checking if the current path starts with any path in the ignore list. This way child paths are ignored too. This enables us to greatly simplify the pipeline.
Param(
[switch] $ReportOnly
)
# Returns $true if $File.Fullname starts with any path in $Ignore (case-insensitive)
Function Test-IgnoreFile( $File, $Ignore ) {
foreach( $i in $Ignore ) {
if( $File.FullName.StartsWith( $i, [StringComparison]::OrdinalIgnoreCase ) ) {
return $true
}
}
$false
}
$limit = (Get-Date).AddDays(-30)
$del30 = "D:\CompanyX_ftp\users"
$ignore = Get-Content "C:\Users\UserX\Documents\Scripts\ignorelist.txt"
Get-ChildItem $del30 -File -Recurse |
Where-Object { $_.CreationTime -lt $limit -and -not ( Test-IgnoreFile $_ $ignore ) } |
ForEach-Object {
if( $ReportOnly) {
$_ | Select-Object FullName, CreationTime
}
else {
$_ | Remove-Item -Force
}
}
I am using below Powershell script which successfully traverses through all my case folders within the main folder named Test. What it is incapable of doing is to rename each sub folder, if required, as can be seen in current and desired output. Script should first sort the sub folders based on current numbering and then give them proper serial numbers as folder name prefix by replacing undesired serial numbers.
I have hundreds of such cases and their sub folders which need to be renamed properly.
The below output shows two folders named "352" and "451" (take them as order IDs for now) and each of these folders have some sub-folders with a 2 digit prefix in their names. But as you can notice they are not properly serialized.
$Search = Get-ChildItem -Path "C:\Users\User\Desktop\test" -Filter "??-*" -Recurse -Directory | Select-Object -ExpandProperty FullName
$Search | Set-Content -Path 'C:\Users\User\Desktop\result.txt'
Below is my current output:
C:\Users\User\Desktop\test\Case-352\02-Proceedings
C:\Users\User\Desktop\test\Case-352\09-Corporate
C:\Users\User\Desktop\test\Case-352\18-Notices
C:\Users\User\Desktop\test\Case-451\01-Contract
C:\Users\User\Desktop\test\Case-451\03-Application
C:\Users\User\Desktop\test\Case-451\09-Case Study
C:\Users\User\Desktop\test\Case-451\14-Violations
C:\Users\User\Desktop\test\Case-451\21-Verdict
My desired output is as follows:
C:\Users\User\Desktop\test\Case-352\01-Proceedings
C:\Users\User\Desktop\test\Case-352\02-Corporate
C:\Users\User\Desktop\test\Case-352\03-Notices
C:\Users\User\Desktop\test\Case-451\01-Contract
C:\Users\User\Desktop\test\Case-451\02-Application
C:\Users\User\Desktop\test\Case-451\03-Case Study
C:\Users\User\Desktop\test\Case-451\04-Violations
C:\Users\User\Desktop\test\Case-451\05-Verdict
Thank you so much. If my desired functionality can be extended to this script, it will be of great help.
Syed
You can do the following based on what you have posted:
$CurrentParent = $null
$Search = Get-ChildItem -Path "C:\Users\User\Desktop\test" -Filter '??-*' -Recurse -Directory | Where Name -match '^\d\d-\D' | Foreach-Object {
if ($_.Parent.Name -eq $CurrentParent) {
$Increment++
} else {
$CurrentParent = $_.Parent.Name
$Increment = 1
}
$CurrentNumber = "{0:d2}" -f $Increment
Join-Path $_.Parent.FullName ($_.Name -replace '^\d\d',$CurrentNumber)
}
$Search | Set-Content -Path 'C:\Users\User\Desktop\result.txt'
I added Where to filter more granularly beyond what -Filter allows.
-match and -replace both use regex to perform the matching. \d is a digit. \D is a non-digit. ^ matches the position at the beginning of the string.
The string format operator -f is used to maintain the 2-digit requirement. If you happen to reach 3-digit numbers, then 3 digit numbers will be output instead.
You can take this further to perform a rename operation:
$CurrentParent = $null
Get-ChildItem . -Filter '??-*' -Recurse -Directory | Where Name -match '^\d\d-\D' | Foreach-Object {
if ($_.Parent.Name -eq $CurrentParent) {
$Increment++
} else {
$CurrentParent = $_.Parent.Name
$Increment = 1
}
$CurrentNumber = "{0:d2}" -f $Increment
$NewName = $_.Name -replace '^\d\d',$CurrentNumber
$_ | Where Name -ne $NewName | Rename-Item -NewName $NewName -WhatIf
}
$NewName is used to simply check if the new name already exists. If it does, a rename will not happen for that object. Remove the -WhatIf if you are happy with the results.
Data mapping project, in house system to new vendor system. First step is find all the occurrences of current database field names (or column names to be precise) in the C# .cs source files. Trying to use Powershell. Have recently created PS searches with Get-ChildItem and Select-String that work well but the search string array was small and easily hard coded inline. But the application being ported has a couple hundred column names and significant amounts of code. So armed with a text file of all the column names Pipleline would seem like a god tool to create a the basic cross ref for further analysis. However, I was not able to get the Pipeline to work with an external variable anyplace other than first step. Trying using -PipelineVariable, $_. and global variable. Did not find anything specific after lots of searching. P.S. This is my first question to StackoOverflow, be kind please.
Here is what I hoped would work but do dice so far.
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr = [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force |
Select-String $s | Select-Object Path, LineNumber, line | Export-csv $outputfile
}
Did find that this will print the list one time but not twice. In fact it seems using the variable in this way results in processing simply skipping any further pipeline steps.
foreach ($s in $arr) {Write-Host $s | Write $s}
If it isn't possible to do this in Powershell easily my fallback is to do with C# although would much rather get the level up with PowerShell if anyone can point me to the correct understanding of how to do things in the Pipepline, or alternatively construct an equivalent function. Seems like such a natural fit for Powershell.
Thanks.
You're calling Export-csv $outputfile in a loop, which rewrites the whole file in every iteration, so that only the last iteration's output will end up in the file.
While you could use -Append to iteratively append to the output file, it is worth aking a step back: Select-String can accept an array of patterns, causing a line that matches any of them to be considered a match.
Therefore, your code can be simplified as follows:
$inputFile = 'C:\DataColumnsNames.txt'
$outputFile = 'C:\DataColumnsUsages.txt'
Get-ChildItem C:\ProjectFolder -Filter *.cs -Recurse -Force -ea SilentlyContinue |
Select-String -Pattern (Get-Content $inputFile) |
Select-Object Path, LineNumber, line |
Export-csv $outputfile
-Pattern (Get-Content $inputFile) passes the lines of input file $inputFile as an array of patterns to match.
By default, these lines are interpreted as regexes (regular expressions); to ensure that they're treated as literals, add -SimpleMatch to the Select-String call.
This answer to a follow-up question shows how to include the specific pattern among the multiple ones passed to -Pattern that matched on each line in the output.
I think you want to append each occurrence to the csv file. And you need to get the content of the file. Try this:
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force | Foreach {
Get-Content "$_.Fullname" | Select-String $s | Select-Object Path, LineNumber, line | Export-csv -Append -Path "$outputfile"
}
}
-Append was not introduced before powershell v3.0 (Windows 8) then try this:
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force | Foreach {
Get-Content "$_.Fullname" | Select-String $s | Select-Object Path, LineNumber, line | ConvertTo-CSV -NoTypeInformation | Select-Object -Skip 1 | Out-File -Append -Path "$outputfile"
}
}
I am seeking help creating a PowerShell script which will search a specified path for multiple .xml files within the same folder.
The script should provide the full path of the file(s) if found.
The script should also provide a date.
Here's my code:
$Dir = Get-ChildItem C:\windows\system32 -Recurse
#$Dir | Get-Member
$List = $Dir | where {$_.Extension -eq ".xml"}
$List | Format-Table Name
$folder = "C:\Windows\System32"
$results = Get-ChildItem -Path $folder -File -Include "*.xml" | Select Name, FullName, LastWriteTime
This will return all xml files only and display the file name, full path to the file and last time it was written to. The "-File" switch is only available in Powershell 4 and up. So if doing it off a Windows 7 or Windows 2008 R2 Server, you will have to make sure you updated your WMF to 4 or higher. Without file the second like will look like.
#Powershell 2.0
$results = Get-ChildItem -Path $folder -Include "*.xml" | Where {$_.PSIsContainer -eq $false} | Select Name, FullName, LastWriteTime
I like the Select method mentioned above for the simpler syntax, but if for some reason you just want the file names with their absolute path and without the column header that comes with piping to Select (perhaps because it will be used as input to another script, or piped to another function) you could do the following:
$folder = 'C:\path\to\folder'
Get-ChildItem -Path $folder -Filter *.xml -File -Name | ForEach-Object {
[System.IO.Path]::GetFullPath($_)
}
I'm not sure if Select lets you leave out the header.
You could also take a look at this answer to give you some more ideas or things to try if you need the results sorted, or the file extension removed:
https://stackoverflow.com/a/31049571/10193624
I was able to make a few changes exporting the results to a .txt file, but though it provides the results I only want to isolate the same .xml files.
$ParentFolder = "C:\software"
$FolderHash = #{}
$Subfolders = Get-ChildItem -Path $ParentFolder
foreach ($EventFolder in $Subfolders) {
$XMLFiles = Get-ChildItem -Path $EventFolder.fullname -Filter *.xml*
if ($XMLFiles.Count -gt 1) {
$FolderHash += #{$EventFolder.FullName = $EventFolder.LastWriteTime}
}
}
$FolderHash
Judging from your self-answer you want a list of directories that contain more than one XML file without recursively searching those directories. In that case your code could be simplified to something like this:
Get-ChildItem "${ParentFolder}\*\*.xml" |
Group-Object Directory |
Where-Object { $_.Count -ge 2 } |
Select-Object Name, #{n='LastWriteTime';e={(Get-Item $_.Name).LastWriteTime}}
For a specific folder, I need to list all files with extension .js even if nested in subfolders at any level.
The result for the output console should be a list of file names with no extension line by line to be easily copy and pasted in another application.
At the moment I am trying this, but in output console I get several meta information and not a simple list.
Get-ChildItem -Path C:\xx\x-Recurse -File | sort length –Descending
Could you please provide me some hints?
If sorting by Length is not a necessity, you can use the -Name parameter to have Get-ChildItem return just the name, then use [System.IO.Path]::GetFileNameWithoutExtension() to remove the path and extension:
Get-ChildItem -Path .\ -Filter *.js -Recurse -File -Name| ForEach-Object {
[System.IO.Path]::GetFileNameWithoutExtension($_)
}
If sorting by length is desired, drop the -Name parameter and output the BaseName property of each FileInfo object. You can pipe the output (in both examples) to clip, to copy it into the clipboard:
Get-ChildItem -Path .\ -Filter *.js -Recurse -File| Sort-Object Length -Descending | ForEach-Object {
$_.BaseName
} | clip
If you want the full path, but without the extension, substitute $_.BaseName with:
$_.FullName.Remove($_.FullName.Length - $_.Extension.Length)
The simple option is to use the .Name property of the FileInfo item in the pipeline and then remove the extension:
Get-ChildItem -Path "C:\code\" -Filter *.js -r | % { $_.Name.Replace( ".js","") }
There are two methods for filtering files: globbing using an Wildcard, or using a Regular Expression (Regex).
Warning: The globbing method has the drawback that it also matches files which should not be matched, like *.jsx.
# globbing with Wildcard filter
# the error action prevents the output of errors
# (ie. directory requires admin rights and is inaccessible)
Get-ChildItem -Recurse -Filter '*.js' -ErrorAction 'SilentlyContinue'
# filter by Regex
Where-Object { $_.Name -Match '.*\.js$' }
You then can sort by name or filesize as needed:
# sort the output
Sort-Object -PropertyName 'Length'
Format it a simple list of path and filename:
# format output
Format-List -Property ('Path','Name')
To remove the file extension, you can use an select to map the result:
Select-Item { $_.Name.Replace( ".js", "") }
Putting it all together, there is also a very short version, which you should not use in scripts, because it's hardly readable:
ls -r | ? { $_.Name -matches '.*\.js' } | sort Length | % { $_.Name.Replace( ".js", "") | fl
If you like brevity, you can remove the ForEach-Object and quotes. -Path defaults to the current directory so you can omit it
(Get-ChildItem -Filter *.js -Recurse).BaseName | Sort length -Descending
The above Answers works fine. However in WIndows there is a alias called ls the same as on linux so another shorter command that works too would be ls -Filter *.exe
Use BaseName for the file name without the file extension.
Get-ChildItem -Path ".\*.js" | Sort-Object Length -Descending | ForEach-Object {
$_.BaseName
}
I always used cygwin for this in the past. My last employer locked down our environments and it wasn't available. I like to review the latest files I've modified often. I created the following environment variable named LatestCode to store the script. I then execute it with: iex $env:latest code.
Here is the script: get-childitem “.” -recurse -include *.ts, *.html , *.sass, *.java, *.js, *.css | where-object {$_.mode -notmatch “d”} | sort lastwritetime -descending | Select-Object -First 25 | format-table lastwritetime, fullname -autosize