I am trying count the rows containing values in a bunch of CSV in a folder. I managed to get the code to count it but I can't seem to find a way to export the results to a CSV. All I got is a blank CSV.
What am I missing here?
$FOLDER_ROOT = "C:\Test\2019"
$OUTPUT_CSV = "C:\Test\2019\Count.csv"
Get-ChildItem $FOLDER_ROOT -re -in "*.csv" | ForEach-Object {
$filestats = Get-Content $_.Fullname | Measure-Object -Line
$linesInFile = $filestats.Lines - 1
Write-Host "$_,$linesInFile"
} | Export-Csv -Path $OUTPUT_CSV -NoType
There are several issues with your code:
Use Get-ChildItem -Filter '*.csv' instead of Get-ChildItem -Include '*.csv'. The former is faster than the latter.
Write-Host most likely causes the output to go directly to the host console. I've been told that this was changed in recent versions (so that host output goes to the new information stream), but for versions at least prior to v5 it's still a reality.
Export-Csv expects object input, since it outputs the properties of the objects as the fields of the CSV (taking the column titles from the property names of the first object). Feeding it strings ("$_,$linesInFile") will result in a CSV that contains only a column "Length", since that is the only property of the string objects.
Use a calculated property for creating a CSV with the filename and line count of the input files:
Get-ChildItem $FOLDER_ROOT -Recurse -Filter '*.csv' |
Select-Object Name, #{n='LineCount';e={(Get-Content $_.Fullname | Measure-Object -Line).Lines - 1}} |
Export-Csv $OUTPUT_CSV -NoType
Write-Host writes only to the host! Most probably you see the output into the PowerShell Console?
Use Write-Output, which could be piped to Export-CSV.
Related
I need to search through 350,000 files to find any that contains certain patterns in the filename. However, the list of patterns (id numbers) that it needs to match is 1000! So I would very much like to be able to script this, because they were originally planning on doing it manually...
So to make it clearer:
Check each File in folder and all subfolders.
If the filename contains any of the IDs in the text file then move it to another file
Otherwise, ignore it.
So I have the basic code that works with a single value:
$name = Get-Content 'C:\test\list.txt'
get-childitem -Recurse -path "c:\test\source\" -filter "*$name*" |
move-item -Destination "C:\test\Destination"
If I change $name to point to a single ID, it works, if I have a single ID in the txt file, it works. Multiple items in a list:
1111111
2222222
3333333
It fails. What am I doing wrong? How can I get it to work? I'm still new to powershell so please be a little more descriptive in any answers.
Your test fails because it is effectively trying to do this (using your test data).
Get-ChildItem -Recurse -Path "c:\test\source\" -filter "*1111111 2222222 3333333*"
Which obviously does not work. It is squishing the array into one single space delimited string. You have to account for the multiple id logic in a different way.
I am not sure which of these will perform better so make sure you test both of these with your own data to get a better idea of execution time.
Cycle each "filter"
$filters = Get-Content 'C:\test\list.txt'
# Get the files once
$files = Get-ChildItem -Recurse -Path "c:\test\source" -File
# Cycle Each ID filter manually
$filters | ForEach-Object{
$singleFilter
$files | Where-Object{$_.Name -like "*$singleFilter*"}
} | Move-Item -Destination "C:\test\Destination"
Make one larger filter
$filters = Get-Content 'C:\test\list.txt'
# Build a large regex alternative match pattern. Escape each ID in case there are regex metacharacters.
$regex = ($filters | ForEach-Object{[regex]::Escape($_)}) -join "|"
# Get the files once
Get-ChildItem -Recurse -path "c:\test\source" -File |
Where-Object{$_.Name -match $regex} |
Move-Item -Destination "C:\test\Destination"
try following this tutorial on how to use get-content function. Looks like when you have a multiple line file, you get an array back. you then have to iterate through your array and use the logic you used for only one item
I am attempting to output full directory path and lastaccesstime in one line.
Needed --
R:\Directory1\Directory2\Directory3, March 10, 1015
What I am getting --
R:\Directory1\Directory2\Directory3
March 10, 1015
Here is my code, It isn't that complicated, but it is beyond me.
Get-ChildItem -Path "R:\" -Directory | foreach-object -process{$_.FullName, $_.LastAccessTime} | Where{ $_.LastAccessTime -lt [datetime]::Today.AddYears(-2) } | Out-File c:\temp\test.csv
I have used foreach-object in the past in order to ensure I do not truncate the excessively long directory names and paths, but never used it when pulling two properties. I would like the information to be on all one line, but haven't been successful. Thanks in advance for the assist.
I recommend filtering (Where-Object) before selecting the properties you want. Also I think you want to replace ForEach-Object with Select-Object, and lastly I think you want Export-Csv rather than Out-File. Example:
Get-ChildItem -Path "R:\" -Directory |
Where-Object { $_.LastAccessTime -lt [DateTime]::Today.AddYears(-2) } |
Select-Object FullName,LastAccessTime |
Export-Csv C:\temp\test.csv -NoTypeInformation
We can get your output on one line pretty easily, but to make it easy to read we may have to split your script out to multiple lines. I'd recommend saving the script below as a ".ps1" which would allow you to right click and select "run with powershell" to make it easier in the future. This script could be modified to play around with more inputs and variables in order to make it more modular and work in more situations, but for now we'll work with the constants you provided.
$dirs = Get-ChildItem -Path "R:\" -Directory
We'll keep the first line you made, since that is solid and there's nothing to change.
$arr = $dirs | Select-Object {$_.FullName, $_.LastAccessTime} | Where-Object{ $_.LastAccessTime -lt [datetime]::Today.AddYears(-2) }
For the second line, we'll use "Select-Object" instead. In my opinion, it's a lot easier to create an array this way. We'll want to deal with the answers as an array since it'll be easiest to post the key,value pairs next to each other this way. I've expanded your "Where" to "Where-Object" since it's best practice to use the full cmdlet name instead of the alias.
Lastly, we'll want to convert our "$arr" object to csv before putting in the temp out-file.
ConvertTo-CSV $arr | Out-File "C:\Temp\test.csv"
Putting it all together, your final script will look like this:
$dirs = Get-ChildItem -Path "C:\git" -Directory
$arr = $dirs | Select-Object {$_.FullName, $_.LastAccessTime} | Where{ $_.LastAccessTime -lt [datetime]::Today.AddYears(-2) }
ConvertTo-CSV $arr | Out-File "C:\Temp\test.csv"
Again, you can take this further by creating a function, binding it to a cmdlet, and creating parameters for your path, output file, and all that fun stuff.
Let me know if this helps!
I have a script that searches for the lastest modified log file. It then is suppose to read that text file and pick up a key phrase then display the line after it.
So far i have this
$logfile = get-childitem 'C:\logs' | sort {$_.lastwritetime} | where {$_ -notmatch "X|Zr" }| select -last 1
$error = get-content $logfile | select-string -pattern "Failed to Modify"
an example line it reads is this
20150721 12:46:26 398fbb92 To CV Failed to Modify
CN=ROLE-x-USERS,OU=Role Groups,OU=Groups,DC=gyp,DC=gypuy,DC=net
MDS_E_BAD_MEMBERSHIP One or more members do not exist in the directory
They key bit of information im trying to get here is
Can anyone help?
Thanks
Try this:
$error = get-content $logfile |
Where-Object { $_ -like "*Failed to Modify*" } |
Select-Object -First 1
This is provided you are looking for the first match in the file. The Select-String cmdlet returns a MatchInfo object. Depending on your requirements there might be no reason to add that level of complexity if you're just looking to pull the first occurrence of this error in the file.
Failing this, my recommendation would be to debug this and step through it. Break on the Get-Content call and see what $logfile is. Run Get-Content $logfile and see what that content looks like. Then do your Select-String on that output. See what MatchInfo.ToString() looks like. Maybe you'll see some disconnect.
Again, my recommendation would be to just parse manually through the file and work with the Where-Object cmdlet at this point.
This shoul work:
get-childitem 'c:\logs' | where {$_.Name -notmatch "X|Zr" } | sort {$_.lastwritetime} | select -last 1 | select-string "Failed to Modify"
But I don't like "X|Zr" part. If your log files have .txt extension, it'll not list them because you're saying you don't want any file containing "x" or "zr" in entire name. Use $_.BaseName (name without extension), or modify regular expression.
I know how to get the data and search through it using some pattern. But that is not what I need.
Get-ChildItem -recurse -Filter *.xml | Get-Content | Select-String -pattern "something here"
I am searching through 100's of GPO xml files and we are trying to remove GPO's that perform the same thing over and over again. I want to find the unique values and combine them in one big happy gpo and get rid of all the redundant ones.
My goal :
1) Get all information from all *.xml files from 100's of sub folders and combine them into one file.
2) Find all lines that contain the same string and get a count of that string. I need a count for all strings in the combined file.
3) My goal is to find the lines that are unique and save them to a file, for further use.
Here's a quick-and-dirty approach using a Hashtable. Since the Hashtable setter performs an "update or create", you'll end up with a distinct list:
$ht = #{}
Get-ChildItem -recurse -Filter *.xml | Get-Content | %{$ht[$_] = $true}
$ht.Keys
Edit: Just saw you wanted counts as well. You can do this:
$ht = #{}
Get-ChildItem -recurse -Filter *.xml | Get-Content | %{$ht[$_] = $ht[$_]+1}
$ht
To export to CSV:
$ht.GetEnumerator() | select key, value | Export-Csv D:\output.csv
I am trying to report on the number of pdf files in a directory. The below code works fine, however i have added Export-Csv into it, and the output does not work. The file is created, but the count is wrong. I get "#TYPE System.Int32" in cell 1A of the output file instead of the file count.... not sure why.
(get-ChildItem C:\Test\* -Filter *.pdf -Recurse).Count | Export-Csv C:\TEMP\Test.csv
Export-CSV works better when you have an object or hashtable with properties and values. All you have is a number in this case and it has no idea what the column heading should be. If all you want is a number in a file, try this:
(get-ChildItem C:\Test\* -Filter *.pdf -Recurse).Count | Set-Content C:\TEMP\Test.csv
But if you really want a csv file or an example for other projects, try this:
$HashTable = #{NumberOfPDFFiles = ((get-ChildItem C:\Test\* -Filter *.pdf -Recurse).Count)}
$HashTable | Export-csv C:\TEMP\Test.csv -NoTypeInformation
Or something like this to stick with the one line idea
(get-ChildItem C:\Test\* -Filter *.pdf -Recurse).Count |
Select-Object #{n='PdfCount';e={$_}} |
Export-CSV C:\TEMP\Test.csv -NoTypeInformation
To compliment kevmar's answer since it does address the issue but doesn't explain why.
From TechNet
By default, the first line of the CSV file contains "#TYPE " followed
by the fully-qualified name of the type of the object.
That is why your first line is: #TYPE System.Int32 and why -NoTypeInformation removes it. If all you are doing is outputting a count then Set-Content makes more sense.