I would like to somehow keep the last x amount of directories in the selected directory.
Ie. directory structure:
d:\test\
a
1
2
3
4
b
1
2
3
4
c
1
2
3
4
d
1
2
3
4
CASE 1: If I give this: d:\test or d:\test\* with 2 last versions then the result should be:
c
1
2
3
4
d
1
2
3
4
CASE 2: If I give this: d:\test\*\* with 2 last versions then the result should be:
a
3
4
b
3
4
c
3
4
d
3
4
CASE 3: If I give this: d:\test\*\*\* with 2 last versions then similarly to the previous case, the parents need to stay and only the subfolders need to be removed.
Until now I've found this:
Get-ChildItem -Path D:\test -Directory | Sort-Object -Property CreationTime | Select-Object -SkipLast 2 | Remove-Item
This does work for case 1, but not for cases 2 and 3.
Ok, it seems I've found a version that seems to work with all cases, but I don't know if there is a more efficient way to do this. Here is my version:
$group_dirs = Get-ChildItem -Path $path -Directory -Force | Group-Object -Property Parent
foreach ($group_dir in $group_dirs) {
$group_dir.Group | Sort-Object -Property CreationTime | Select-Object -SkipLast $leftCount | Remove-Item -Force
}
Related
Is it possible to sort numbers in a file in ascending or descending? How is it done?
Get-ChildItem |Sort-Object .\sorting.txt\ -Descending
Get-Content |Sort-Object .\sorting.txt\ -Descending
I tried all of these and even measure-object but none of them gave me what I wanted- numbers in a file sorted in ascend/descend
Consider the following
sorting.txt
9
11
45
12
3
101
Then run:
Get-Content .\sorting.txt | Sort-Object { $_ -as [int] } -Descending >> ./sorted.txt
sorted.txt
101
45
12
11
9
3
I am using PowerShell to collect lists of names from multiple text files. May of the names in these files are similar / repeating. I am trying to ensure that PowerShell returns a single text file with all of the unique items. In looking at the data it looks like the script is gathering 271/296 of the unique items. I'm guessing that some of the data is being flagged as duplicates when it shouldn't, any suggestions?
#Take content of each file (all names) and add unique values to text file
#for each unique value, create a row & check to see which txt files contain
function List {
$nofiles = Read-Host "How many files are we pulling from?"
$data = #()
for ($i = 0;$i -lt $nofiles; $i++)
{
$data += Read-Host "Give me the file name for file # $($i+1)"
}
return $data
}
function Aggregate ($array) {
Get-Content $array | Sort-Object -unique | Out-File newaggregate.txt
}
#SCRIPT BODY
$data = List
aggregate ($data)
I was expecting this code to catch everything, but it's missing some items that look very similar. List of missing names and their similar match:
CORPINZUTL16 MISSING FROM OUTFILE
CORPINZTRACE MISSING FROM OUTFILE
CORPINZADMIN Found In File
I have about 20 examples like this one. Apparently the Get-Content -Unique is not checking every character in a line. Can anyone recommend a better way of checking each line or possibly forcing the get-character to check full names?
Just for demonstration this line creates 3 txt files with numbers
for($i=1;$i -lt 4;$i++){set-content -path "$i.txt" -value ($i..$($i+7))}
1.txt | 2.txt | 3.txt | newaggregate.txt
1 | | | 1
2 | 2 | | 2
3 | 3 | 3 | 3
4 | 4 | 4 | 4
5 | 5 | 5 | 5
6 | 6 | 6 | 6
7 | 7 | 7 | 7
8 | 8 | 8 | 8
| 9 | 9 | 9
| | 10 | 10
Here using Get-Content with a range [1-3] of files
Get-Content [1-3].txt | Sort-Object {[int]$_} -Unique | Out-File newaggregate.txt
$All = Get-Content .\newaggregate.txt
foreach ($file in (Get-ChildItem [1-3].txt)){
Compare-Object $All (Get-Content $file.FullName) |
Select-Object #{n='File';e={$File}},
#{n="Missing";e={$_.InputObject}} -ExcludeProperty SideIndicator
}
File Missing
---- -------
Q:\Test\2019\05\07\1.txt 9
Q:\Test\2019\05\07\1.txt 10
Q:\Test\2019\05\07\2.txt 1
Q:\Test\2019\05\07\2.txt 10
Q:\Test\2019\05\07\3.txt 1
Q:\Test\2019\05\07\3.txt 2
there are two ways to achieve this one is using select-object -Unique which works when data is not sorted and can be used for small data or lists.
When dealing with large files we can use get-Unique command which works with sorted input, if input data is not sorted then it will give wrong results.
Get-ChildItem *.txt | Get-Content | measure -Line #225949
Get-ChildItem *.txt | Get-Content | sort | Get-Unique | measure -Line #119650
Here is my command for multiple files :
Get-ChildItem *.txt | Get-Content | sort | Get-Unique >> Unique.txt
Consider the following directory tree
ROOT
BAR001
foo_1.txt
foo_2.txt
foo_ignore_this_1.txt
BAR001_a
foo_3.txt
foo_4.txt
foo_ignore_this_2.txt
foo_ignore_this_3.txt
BAR001_b
foo_5.txt
foo_ignore_this_4.txt
BAR002
baz_1.txt
baz_ignore_this_1.txt
BAR002_a
baz_2.txt
baz_ignore_this_2.txt
BAR002_b
baz_3.txt
baz_4.txt
baz_5.txt
baz_ignore_this_3.txt
BAR002_c
baz_ignore_this_4.txt
BAR003
lor_1.txt
The structure will always be like this, so no deeper subfolders. I'm working on a script to count the number of files:
for each BARXXX folder
for each BARXXX_Y folder
textfiles with "ignore_this" in the name, should be ignored in the count
For the example above, this would result into:
Folder Filecount
---------------------
BAR001 2
BAR001_a 2
BAR001_b 1
BAR002 1
BAR002_a 1
BAR002_b 3
BAR002_c 0
BAR003 1
I now have:
Function Filecount {
param(
[string]$dir
)
$childs = Get-ChildItem $dir | where {$_.Attributes -eq 'Directory'}
Foreach ($childs in $child) {
Write-Host (Get-ChildItem $dir | Measure-Object).Count;
}
}
Filecount -dir "C:\ROOT"
(Not ready yet but building) This however, does not work. $child seems to be empty. Please tell me what I'm doing wrong.
Well, to start, you're running ForEach ($childs in $child), this syntax is backwards, so that will cause you some issues! If you swap it, so that you're running:
ForEach ($child in $childs)
You'll get the following output:
>2
>2
>1
>1
>1
>3
>0
Alright, I'm back now with the completed answer. For one, instead of using Write-Out, I'm using a PowerShell custom object to let PowerShell do the hard work for me. I'm setting FolderName equal to the $child.BaseName, and then running a GCI on the $Child.FullName to get the file count. I've added an extra parameter called $ignoreme, that should have an asterisk value for the values you want to ignore.
Here's the complete answer now. Keep in mind that my file structure was a bit different than yours, so my file count is different at the bottom as well.
Function Filecount {
param(
[string]$dir="C:\TEMP\Example",
[string]$ignoreme = "*_*"
)
$childs = Get-ChildItem $dir | where {$_.Attributes -eq 'Directory'}
Foreach ($child in $childs) {
[pscustomobject]#{FolderName=$child.Name;ItemCount=(Get-ChildItem $child.FullName | ? Name -notlike $ignoreme | Measure-Object).Count}
}
}
>Filecount | ft -AutoSize
>FolderName ItemCount
>---------- ---------
>BAR001 2
>BAR001_A 1
>BAR001_b 2
>BAR001_C 0
>BAR002 0
>BAR003 0
If you're using PowerShell v 2.0, use this method instead.
Function Filecount {
param(
[string]$dir="C:\TEMP\Example",
[string]$ignoreme = "*_*"
)
$childs = Get-ChildItem $dir | where {$_.Attributes -eq 'Directory'}
Foreach ($child in $childs) {
$ObjectProperties = #{
FolderName=$child.Name
ItemCount=(Get-ChildItem $child.FullName | ? Name -notlike $ignoreme | Measure-Object).Count}
New-Object PSObject -Property $ObjectProperties
}
}
I like that way of creating an object 1RedOne, haven't seen that before, thanks.
We can improve the performance of the code in a few of ways. By using the Filter Left principle, which states that the provider for any cmdlet is inherently more efficient than running things through PowerShell, by performing fewer loops and by removing an unnecessary step:
Function Filecount
{
param
(
[string]$dir = ".",
[parameter(mandatory=$true)]
[string]$ignoreme
)
Get-ChildItem -Recurse -Directory -Path $dir | ForEach-Object `
{
[pscustomobject]#{FolderName=$_.Name;ItemCount=(Get-ChildItem -Recurse -Exclude "*$ignoreme*" -Path $_.FullName).count}
}
}
So, firstly we can use the -Directory switch of Get-Childitem in the top-level directory (I know this is available in v3.0 and above, not sure about v2.0).
Then we can pipe the output of this directly in to the next loop, without storing it first.
Then we can replace another Where-Object with a provider -Exclude.
Finally, we can remove the Measure-Object as a simple count of the array will do:
Filecount "ROOT" "ignore_this" | ft -a
FolderName ItemCount
---------- ---------
BAR001 2
BAR001_a 2
BAR001_b 1
BAR002 1
BAR002_a 1
BAR002_b 3
BAR002_c 0
BAR003 1
Cheers Folks!
I've been able to track down basic head/tail functionality:
head -10 myfile <==> cat myfile | select -first 10
tail -10 myfile <==> cat myfile | select -last 10
But if I want to list all lines except the last three or all lines except the first three, how do you do that? In Unix, I could do "head -n-3" or "tail -n+4". It is not obvious how this should be done for PowerShell.
Useful information is spread across other answers here, but I think it is useful to have a concise summary:
All lines except the first three
1..10 | Select-Object -skip 3
returns (one per line): 4 5 6 7 8 9 10
All lines except the last three
1..10 | Select-Object -skip 3 -last 10
returns (one per line): 1 2 3 4 5 6 7
That is, you can do it with built-in PowerShell commands, but there's that annoyance of having to specify the size going in. A simple workaround is to just use a constant larger than any possible input and you will not need to know the size a priori:
1..10 | Select-Object -skip 3 -last 10000000
returns (one per line): 1 2 3 4 5 6 7
A cleaner syntax is to use, as Keith Hill suggested, the Skip-Object cmdlet from PowerShell Community Extensions (the Skip-Last function in Goyuix's answer performs equivalently but using PSCX saves you from having to maintain the code):
1..10 | Skip-Object -last 3
returns (one per line): 1 2 3 4 5 6 7
First three lines
1..10 | Select-Object –first 3
returns (one per line): 1 2 3
Last three lines
1..10 | Select-Object –last 3
returns (one per line): 8 9 10
Middle four lines
(This works because the -skip is processed before the -first, regardless of the order of parameters in the invocation.)
1..10 | Select-Object -skip 3 -first 4
returns (one per line): 4 5 6 7
Like the -First and -Last parameters, there is also a -Skip parameter that will help. It is worth noting that -Skip is 1 based, not zero.
# this will skip the first three lines of the text file
cat myfile | select -skip 3
I am not sure PowerShell has something that gives you back everything except the last n lines pre-built. If you know the length you could just subtract n from the line count and use the -First parameter from select. You could also use a buffer that only passes lines through when it is filled.
function Skip-Last {
param (
[Parameter(Mandatory=$true,ValueFromPipeline=$true)][PsObject]$InputObject,
[Parameter(Mandatory=$true)][int]$Count
)
begin {
$buf = New-Object 'System.Collections.Generic.Queue[string]'
}
process {
if ($buf.Count -eq $Count) { $buf.Dequeue() }
$buf.Enqueue($InputObject)
}
}
As a demo:
# this would display the entire file except the last five lines
cat myfile | Skip-Last -count 5
If you're using the PowerShell Community Extensions, there is a Take-Object cmdlet that will pass thru all output except the last N items e.g.:
30# 1..10 | Skip-Object -Last 4
1
2
3
4
5
6
All but the last n can be done with
... | select -skiplast $n
You can do it like this:
[array]$Service = Get-Service
$Service[0] #First Item
$Service[0..2] #First 3 Items
$Service[3..($Service.Count)] #Skip the first 3 lines
$Service[-1] #Last Item
$Service[-3..-1] #Last 3 Items
$Service[0..($Service.Count -4)] #Skip the last 3 lines
All but the first n can be done with
... | Select -skip $n
However all "but the last m" has nothing inbuilt. It is doable loading the whole input into an array to get the length – of course for large input that can put unreasonable demands on memory.
I have got text file:
1 2 4 5 6 7
1 3 5 6 7 8
1 2 3 4 5 6
1 2 4 5 6 7
Here first and last line are simmilar. I have a lot of files that have double lines. I need to delete all dublicate.
All these seem really complicated. It is as simple as:
gc $filename | sort | get-unique > $output
Using actual file names instead of variables:
gc test.txt| sort | get-unique > unique.txt
To get unique lines:
PS > Get-Content test.txt | Select-Object -Unique
1 2 4 5 6 7
1 3 5 6 7 8
1 2 3 4 5 6
To remove the duplicate
PS > Get-Content test.txt | group -noelement | `
where {$_.count -eq 1} | select -expand name
1 3 5 6 7 8
1 2 3 4 5 6
If order is not important:
Get-Content test.txt | Sort-Object -Unique | Set-Content test-1.txt
If order is important:
$set = #{}
Get-Content test.txt | %{
if (!$set.Contains($_)) {
$set.Add($_, $null)
$_
}
} | Set-Content test-2.txt
Try something like this:
$a = #{} # declare an arraylist type
gc .\mytextfile.txt | % { if (!$a.Contains($_)) { $a.add($_)}} | out-null
$a #now contains no duplicate lines
To set the content of $a to mytextfile.txt:
$a | out-file .\mytextfile.txt
$file = "C:\temp\filename.txt"
(gc $file | Group-Object | %{$_.group | select -First 1}) | Set-Content $file
The source file now contains only unique lines
The already posted options did not work for me for some reason