I have a CSV file that contains multiple vendors (Cisco, Redhat, vmware..etc), I need a PowerShell script to read this column (vendor) and add separate column "Multiple1 or Multiple2" (Remark) if the CSV contains multiple entries of same vendors.
I have attached screen shot of the sample file.
I tried from end to get this done, but it didn't work.
Okay, after Spikeys last comment I had a go at guessing what he might want to achieve.
I created a CSV file:
Product,Template
Microsoft Windows,
RedHat Enterprise,
Apple Safari,
Microsoft Windows,
RedHat Enterprise,
RedHat Enterprise,
and then wrote the following script. It's commented and produces the following output:
Product Template
------- --------
Microsoft Windows Multiple2
RedHat Enterprise Multiple3
Apple Safari Multiple1
Microsoft Windows Multiple2
RedHat Enterprise Multiple3
RedHat Enterprise Multiple3
Code:
$Csv = Import-Csv -Path "C:\Book1.csv"
#Hastables have key - value pairs. Example "Microsoft Windows" = 1. Here 'Microsoft Windows' is the key and '1' is the value
[hashtable]$ProductCount = #{}
#Go through each line in the CSV. This returns the product name e.g. Microsoft Windows
ForEach ($Product in $Csv.Product)
{
#If there us no key for the current product in hashtable $Productcount, then add it with value 1
If ($ProductCount.Keys -notcontains $Product)
{
$ProductCount.Add($Product, 1)
}
#If the above does not apply, then increase the value (effectively the count) by 1
Else
{
$ProductCount[$Product] = $ProductCount[$Product] + 1
}
}
#Go through each row in the CSV file. Each row is returned as it's own object with a 'Product' and 'Template' property
ForEach ($Row in $Csv)
{
#Extract the count for the current product from hastable $ProductCount
$Count = $ProductCount[$Row.Product]
#Set the 'Template' property for the current row object to multipile + the count we got earlier
$Row.Template = "Multiple$Count"
}
#Save the changes to the CSV file as a new CSV. You can also overwrite your old one if you like
$Csv | Export-Csv -Path "C:\Book2.csv"
I don't quite understand your question, but here are some techniques I find useful when working with CSV files.
Example CSV:
Name,City
Bob,BlackPool
Alice,Dover
Carl,Manchester
Presume you assign the CSV file to a variable like so
$CSV = Import-CSV -Path "C:\Stuff.csv"
1. You can access all rows in a column by typing the variable dot(.) column header, so
$CSV.Name
returns:
Bob
Alice
Carl
2. To access a row in a CSV file you need to use indexing, so
$CSV[1]
returns:
Name City
---- ----
Alice Dover
3. An easy way to replace a property of a specific row is to filter it using Where-Object. Say I want to change Carl's city to London.
$($CSV | Where-Object {$_.Name -like "Carl"}).City = "London"
Here is what happens:
What's in the parentheses is processed first, so we are selecting a row where the Name property is like "Carl" (You can use wilcard here, so "Ca*" would have worked too). Then, outside of the parentheses, we are setting the city property to "London".
Note: $_ represents the data currently in a pipeline, in this case that's the row containing Carl.
There is more stuff to know, but this might help you the most.
Don't forget to save your changes by using the Export-CSV cmdlet!
$CSV | Export-CSV -Path "C:\new.csv" -NoTypeInformation
Related
I am filtering an export from power cli on my vcenters. I need to produce two csv files from the export Vmware creates from powercli. When I export, Esxi creates a column called GUEST then populates it with "servername.somedomain.com:Microsoft Windows Servers 2016" in the column and separates it with semicolon depending on the OS type.
I need to produce csv report that has all the servers with the same OS's and the associated columns as well as a report of all the like domain servers and associated columns.
Import csv doesn't support wildcards so I can't filter out somedomain.com nor can I filter on 2016 on input. I'm at a loss which direction to turn next and would appreciate or a concise reading on better powershell techniques to manipulate data in csv files for dummies? Below is what I use to do most of the filtering but I'm stuck on the cell described above.
Import-CSV -Path "E:\esxi.csv | ? Folder -notlike *SharePoint*| ? Notes -notlike *Physical* | ? Notes -notlike *Sharepoint* | Export-Csv "E:\filtered.csv" -NoTypeInformation
Here is a link to a sample csv.
https://drive.google.com/file/d/1DdqtVFVcZ0aFnoUDqB2ErNX_yA9LBO8H/view?usp=sharing
I need to produce csv report that has all the servers with the same OS
You'll want Group-Object - it'll group input objects based on some property expression, which is exactly what you need:
# Import CSV data
$data = Import-CSV -Path "E:\esxi.csv"
# Group by OS label (the substring following the last `:` in the GUEST column)
$OSGroups = $data |Group-Object { $_.GUEST -replace '^.*:(?=[^:]+$)'}
# Loop through the resulting groups and output a new CSV for each
foreach($OSGroup in $OSGroups){
# Extract the OS name that we grouped on
$OSName = $OSGroup.Name
# Export relevant records to new CSV
$OSGroup.Group |Export-Csv "PerOSReport_${OSName}.csv" -NoTypeInformation
}
Group,
First time posting a question so please be patient. Attempting to use Powershell to read 1 .csv file and search a second .csv for specific data associated with what I passed from the first .csv file. Lets say that both files contain a column with a heading of 'name' with a list of PC names in that column. The 2nd CSV file in addition has a column with a heading named 'id' and that contains a unique ID associated with that PC name (and to complicate matters there may be duplicate PC Names but each one has a unique ID). I want to work through all the PC names in the first .csv file searching for the corresponding unique id of that PC name in the second .csv file and then set that as a variable so that I can perform some action against it. The code below is what I have so far, I'm manually providing the PC name as I cannot seem to get-content of the first file and pass it to the 2nd file. It will get the unique ID but I cannot seem to get it into a variable. Any help would be appreciated.
$computer = "T4763110"
$csv = Import-Csv "C:\Devices.csv"
$csv | Where-Object { $_.name -eq $computer } | % id
1st CSV file
First CSV file
2nd CSV file
Second CSV file
You forgot to specify a >Delimiter< in your import-csv. You can look your delimiter up, when you open your .csv file with the basic Microsoft-Editor.
Mine looks like this:
ID;Name
1;Alienware
2;Lexware
3;One
4;Chip
5;Stack
6;Linux
I made a new table named dev.csv for myself with following content:
When executing this code:
> $csv = Import-Csv "C:/Users/Desktop/dev.csv" -Delimiter ';'
> $csv
Output:
ID Name
-- ----
1 Alienware
2 Lexware
3 One
4 Chip
5 Stack
6 Linux
Then execute this:
> $csv | where-object {$_.name -eq "Alienware"} | % id
1
To save it to a variable, simply put brackets around it.
> $idvar = ($csv | where-object {$_.name -eq "Alienware"} | % id)
> $idvar
1
This is my first post within Stackoverflow, I have for many years just read many fantastic questions, answers and other various posts. I have learned a lot a lot from this fantastic community. I hope now that I have taken the brave step to really sink my teeth into powershell and join this community that I may be able to contribute in someway!
So I have started working on a project which at it's basic core level - list all files that are older than 7 years so they can then review and delete where possible.
I have however broken the whole entire script up into several stages. I am currently stuck at a step in stage 2.
I have been stuck for about 2 days on what to many of you powershell genius's out there may only take 10mins to figure out!
I must apologise for my stupidity and lack of knowledge, my experience with powershell scripting is limited to literally 5 working days, I am currently diving in and learning with books, but I also have a job to do so don't get to learn the easy way!
My script essentially has 3 steps;
Runs a Get-ACL of top Level DATA folders to create a listing of all Groups that have permissions on particular folder. I want to then either export this data or simply hold it for the next step.
Filter this gathered information based off a CSV which contains a Column labelled Role (Role will contain a group that the Folder Manager is exclusively in), and check the inherent member of this exlcusive group (maybe this last bit needs to be another step as well?)
Stores or Exports this list of exclusive members with their relevant folder to then later use as a variable for to send an email with a list of files that need to be deleted.
With the script below I am essentially stuck on Step 2 and how to create a filter from the CSV (or stored variables?) and apply it to the GET-ACL foreach loop. I may be going about this the whole wrong way using regex, and to be honest most of this is copy and paste and reading around the internet where people have done similar tasks. SO again I apologise if this is just a dumb way to go about it from the start!
I want to thank everyone in advance for all help, opinions and advice, I will listen to it all and I will try and take on-board as much as my brain can handle - I promise!
#$RoleList = import-csv "C:\DATA\scripts\GB_CMS\CSV\datafolders_rolelist.csv"
#foreach ($Manager in $RoleList) {
#$FolderManager = $RoleList.Role
$FolderManagers = Import-Csv C:\DATA\scripts\GB_CMS\CSV\datafolders_rolelist.csv | foreach {
New-Object PSObject -prop #{
Folder = $_.Folder;
Manager = $_.'Folder Manager';
Role = $_.Role
}
}
$Role = $FolderManagers.Role
$Role
gci "c:\DATA" | Where {$_.PSIsContainer} | get-acl |
ForEach $_.Name {
[regex]$regex="\w:\\\S+"
$path=$regex.match($_.Path).Value
$_ | select -expand access |
$
where {$_.identityreference -like "$Role"} |
Select #{Name="Path";Expression={$Path}},IdentityReference
}
Thanks,
Daniel.
Bit of a guess at what you want here. e.g. if you have folders
C:\Data\Accounts
C:\Data\Marketing
C:\Data\Sales
You might have permissions
C:\Data\Accounts {'FolderManagers-Accounts', 'Accounts', 'Directors'}
C:\Data\Marketing {'FolderManagers-Marketing', 'Marketing', 'Sales'}
C:\Data\Sales {'FolderManagers-Sales', 'Sales', 'Directors'}
and your CSV is
Name, Role, Email
Alice, FolderManagers-Accounts, alice#example.com
Bob, FolderManagers-Marketing, bob#example.com
And there will be a clear mapping of one (1) row in the CSV to one of the groups in the ACLs.
And you want, from your script:
Identify who to email about "C:\Data\Accounts"
How close am I?
# Import the managers. This will turn the CSV into an array of objects
# no need to do that explicitly
$FolderManagers = Import-Csv C:\DATA\scripts\GB_CMS\CSV\datafolders_rolelist.csv
# This will be a hashtable pairing up folder names with people
# e.g. 'C:\DATA\Accounts' -> Alice
$FolderMap = #{}
# Run through all the folders
GetChildItem -Path "C:\Data" -Directory | ForEach-Object {
# Run through the group/user ACL entries on the folder
foreach ($group in (Get-Acl $_.FullName).Access.IdentityReference)
{
# Look for a matching row in the CSV
$CsvRow = $FolderManagers | Where-Object {$_.Role -match $group}
if (-not $CsvRow)
{
Write-Error "No manager found for folder $_"
}
else
{
# Add to the map
# $_ converts to folder path, C:\DATA\Accounts
# $CsvRow is the person, #{Name=Alice, Role=..., Email=...}
$FolderMap[$_.FullName] = $CsvRow
}
}
}
Then it (the FolderMap) will be
Name Value
---- -----
C:\Data\Accounts {Name='Alice';Role=...
C:\Data\Marketing {Name='Bob';Role=...
you can query it with
$person = $FolderMap["c:\data\Accounts"]
$person.email
and if you really want to export it, maybe
$FolderMap | ConvertTo-Json | Set-Content foldermanagers.json
Nb. I wrote most of this off the top of my head, and it probably won't just run. And that's a problem with big, not very specific questions on StackOverflow.
Auto-generated PS help links from my codeblock (if available):
Import-Csv (in module Microsoft.PowerShell.Utility)
ForEach-Object
Get-Acl (in module Microsoft.PowerShell.Security)
Where-Object
Write-Error (in module Microsoft.PowerShell.Utility)
I have a drop down list that's being populated by a CSV file. There are many columns in the CSV, but it only pulls from the Name column. Here's what I have that works just fine in most Win 7, and all Win 8+ PC's I've tried it on.
$customers = Import-CSV "$dir\Apps\customers.csv"
$List = $customers.name | Sort-Object
After that there's a ForEach loop to put each item from the list into the menu.
Lately I've been noticing an issue on a couple Win 7 PC's that I can't figure out. The import option doesn't work unless I specify all the headers with the -Header option. I get this error:
After getting it to import correctly by adding all the headers I can't get it to save $customers.name into the $List variable, with or without the sorting. However, if I provide an index number (ex $customers[2].name) it works.
To work around this I've looked at ways to measure the number of rows in the CSV by using the following options after $customers:
$csvlength = Get-Content "$dir\Apps\customers.csv" | Measure-Object - lines
or
$csvlength = Import-CSV "$dir\Apps\customers.csv" | Measure-Object
From there I can see the length by looking at $csvlength.lines or $csvlength.count.
Is there a way to use that information to save the list of names into $List? I've tried something like this with no success:
$List = $customers[0-$csvlength.count] | Sort-Object
Also, I've noticed that when importing the headers it includes Name in the list. If at all possible I'd like to not include the header. I also have a line at the end of the CSV that has other info in it, but no name. That shows up as a blank line. If possible I'd like that to be removed as well.
PowerShell v2 $array.Foo only allows access to a property Foo of the array object itself, not to a property Foo of the array elements. Allowing access to element properties via the array variable is one of the major changes that was introduced with PowerShell v3.
To work around this limitiation in v2 you need to expand the property before sorting:
$List = $customers | Select-Object -Expand name | Sort-Object
Question
How do I reformat an unknown CSV column name according to a formula or subroutine (e.g. rename column " Arbitrary Column Name " to "Arbitrary Column Name" by running a trim or regex or something) while maintaining data?
Goal
I'm trying to more or less sanitize columns (the names) in a hand-produced (or at least hand-edited) csv file that needs to be processed by an existing PowerShell script. In this specific case, the columns have spaces that would be removed by a call to [String]::Trim(), or which could be ignored with an appropriate regex, but I can't figure a way to call or use those techniques when importing or processing a CSV.
Short Background
Most files and columns have historically been entered into the CSV properly, but recently a few columns were being dropped during processing; I determined it was because the files contained a space (e.g., Select-Object was being told to get "RFC", but Import-CSV retrieved "RFC ", so no matchy-matchy). Telling the customer to enter it correctly by hand (though preferred and much simpler) is not an option in this case.
Options considered
I could manually process the text of the file, but that is a messy and error prone way to re-invent the wheel. I wonder if there's a syntax with Select-Object that would allow a softer match for column names, but I can't find that info.
The closest I have come conceptually is using a calculated property in the call to Select-Object to rename the column, but I can only find ways to rename a known column to another known column. So, this would require enumerating the columns and matching them exactly (preferred) or a softer match (like comparing after trimming or matching via regex as a fallback) with expected column names, then creating a collection of name mappings to use in constructing calculated properties from that information to select into a new object.
That seems like it would work, but more it's work than I'd prefer, and I can't help but hope that there's a simpler way I haven't been able to find via Google. Maybe I should try Bing?
Sample File
Let's say you have a file.csv like this:
" RFC "
"1"
"2"
"3"
Code
Now try to run the following:
$CSV = Get-Content file.csv -First 2 | ConvertFrom-Csv
$FixedHeaders = $CSV.PSObject.Properties.Name.Trim(' ')
Import-Csv file.csv -Header $FixedHeaders |
Select-Object -Skip 1 -Property RFC
Output
You will get this output:
RFC
---
1
2
3
Explanation
First we use Get-Content with parameter -First 2 to get the first two lines. Piping to ConvertFrom-Csv will allow us to access the headers with PSObject.Properties.Name. Use Import-Csv with the -Header parameter to use the trimmed headers. Pipe to Select-Object and use -Skip 1 to skip the original headers.
I'm not sure about comparisons in terms of efficiency, but I think this is a little more hardened, and imports the CSV only once. You might be able to use #lahell's approach and Get-Content -raw, but this was done and it works, so I'm gonna leave it to the community to determine which is better...
#import the CSV
$rawCSV = Import-Csv $Path
#get actual header names and map to their reformatted versions
$CSVColumns = #{}
$rawCSV |
Get-Member |
Where-Object {$_.MemberType -eq "NoteProperty"} |
Select-Object -ExpandProperty Name |
Foreach-Object {
#add a mapping to the original from a trimmed and whitespace-reduced version of the original
$CSVColumns.Add(($_.Trim() -replace '(\s)\s+', '$1'), "$_")
}
#Create the array of names and calculated properties to pass to Select-Object
$SelectColumns = #()
$CSVColumns.GetEnumerator() |
Foreach-Object {
$SelectColumns += {
if ($CSVColumns.values -contains $_.key) {$_.key}
else { #{Name = $_.key; Expression = $CSVColumns[$_.key]} }
}
}
$FormattedCSV = $rawCSV |
Select-Object $SelectColumns
This was hand-copied to a computer where I don't have the rights to run it, so there might be an error - I tried to copy it correctly
You can use gocsv https://github.com/DataFoxCo/gocsv to see the headers of the csv, you can then rename the headers, behead the file, swap columns, join, merge, any number of transformations you want