I am filtering an export from power cli on my vcenters. I need to produce two csv files from the export Vmware creates from powercli. When I export, Esxi creates a column called GUEST then populates it with "servername.somedomain.com:Microsoft Windows Servers 2016" in the column and separates it with semicolon depending on the OS type.
I need to produce csv report that has all the servers with the same OS's and the associated columns as well as a report of all the like domain servers and associated columns.
Import csv doesn't support wildcards so I can't filter out somedomain.com nor can I filter on 2016 on input. I'm at a loss which direction to turn next and would appreciate or a concise reading on better powershell techniques to manipulate data in csv files for dummies? Below is what I use to do most of the filtering but I'm stuck on the cell described above.
Import-CSV -Path "E:\esxi.csv | ? Folder -notlike *SharePoint*| ? Notes -notlike *Physical* | ? Notes -notlike *Sharepoint* | Export-Csv "E:\filtered.csv" -NoTypeInformation
Here is a link to a sample csv.
https://drive.google.com/file/d/1DdqtVFVcZ0aFnoUDqB2ErNX_yA9LBO8H/view?usp=sharing
I need to produce csv report that has all the servers with the same OS
You'll want Group-Object - it'll group input objects based on some property expression, which is exactly what you need:
# Import CSV data
$data = Import-CSV -Path "E:\esxi.csv"
# Group by OS label (the substring following the last `:` in the GUEST column)
$OSGroups = $data |Group-Object { $_.GUEST -replace '^.*:(?=[^:]+$)'}
# Loop through the resulting groups and output a new CSV for each
foreach($OSGroup in $OSGroups){
# Extract the OS name that we grouped on
$OSName = $OSGroup.Name
# Export relevant records to new CSV
$OSGroup.Group |Export-Csv "PerOSReport_${OSName}.csv" -NoTypeInformation
}
Related
I have two files/outputs.
$file1 contains:
Name,Address,Number
$file2 contains:
Name
I'm looking to match those two files on Name and output the entire line from $file1
Assuming that your files are CSV files:
Combine Import-Csv with Where-Object to filter those lines, with the help of the -inoperator:
Import-Csv $file1 | # Parse the CSV file into objects
Where-Object Name -in (Import-Csv $file2).Name # Filter
Note:
This is a conceptually simple, but potentially slow solution, depending on the size of the input files. More efficient - but more elaborate - solutions are possible.
The filtered CSV rows are output as objects (of type [pscustomobject]), as parsed by Import-Csv; you could convert them back to CSV via ConvertTo-Csv (in memory) or save them back to a file with Export-Csv.
I have a csv with columns A, B, C. I would like to import that to powershell and only on column C, remove any rows that have the word "Unknown" listed. If Column A or B has "Unknown", they stay, however, if Column C has it, the entire row gets deleted. Per the picture below, Row 4 would be deleted.
Can someone please provide a sample script to do this?
Thanks!
So, you have 3 problems you need to solve:
Import the data from the CSV file
Filter it based on the value of column C
Export the filtered data to a file again
To import, use the aptly named Import-Csv cmdlet:
$data = Import-Csv .\path\to\file.csv
Import-Csv will parse the CSV file and for each row it reads, it will output 1 object with properties corresponding to the column names in the header row.
To filter these objects based on the value of their C property, use the Where-Object cmdlet:
$filteredData = $data |Where-Object C -ne 'Unknown'
Where-Object will test whether the C property on each object does not have the value 'Unknown' (-ne = not equals), and discard any object for which that's not the case.
To re-export the filtered data, use the Export-Csv cmdlet:
$filteredData |Export-Csv .\path\to\output.csv -NoTypeInformation
You can also combine all three statements into a single pipeline expression:
Import-Csv .\path\to\file.csv |Where-Object C -ne 'Unknown' |Export-Csv .\path\to\output.csv -NoTypeInformation
This "one-liner" approach might be preferable if you're working on large CSV files (> hundreds of thousands of records), as it doesn't require reading the entire CSV file into memory at once.
$Data = Get-Content "C:\file.csv" | ConvertFrom-Csv
$Data | Where-Object {$_.C-ne 'Unknown'} | Export-Csv "C:\file_New.csv"
I have a CSV file that contains multiple vendors (Cisco, Redhat, vmware..etc), I need a PowerShell script to read this column (vendor) and add separate column "Multiple1 or Multiple2" (Remark) if the CSV contains multiple entries of same vendors.
I have attached screen shot of the sample file.
I tried from end to get this done, but it didn't work.
Okay, after Spikeys last comment I had a go at guessing what he might want to achieve.
I created a CSV file:
Product,Template
Microsoft Windows,
RedHat Enterprise,
Apple Safari,
Microsoft Windows,
RedHat Enterprise,
RedHat Enterprise,
and then wrote the following script. It's commented and produces the following output:
Product Template
------- --------
Microsoft Windows Multiple2
RedHat Enterprise Multiple3
Apple Safari Multiple1
Microsoft Windows Multiple2
RedHat Enterprise Multiple3
RedHat Enterprise Multiple3
Code:
$Csv = Import-Csv -Path "C:\Book1.csv"
#Hastables have key - value pairs. Example "Microsoft Windows" = 1. Here 'Microsoft Windows' is the key and '1' is the value
[hashtable]$ProductCount = #{}
#Go through each line in the CSV. This returns the product name e.g. Microsoft Windows
ForEach ($Product in $Csv.Product)
{
#If there us no key for the current product in hashtable $Productcount, then add it with value 1
If ($ProductCount.Keys -notcontains $Product)
{
$ProductCount.Add($Product, 1)
}
#If the above does not apply, then increase the value (effectively the count) by 1
Else
{
$ProductCount[$Product] = $ProductCount[$Product] + 1
}
}
#Go through each row in the CSV file. Each row is returned as it's own object with a 'Product' and 'Template' property
ForEach ($Row in $Csv)
{
#Extract the count for the current product from hastable $ProductCount
$Count = $ProductCount[$Row.Product]
#Set the 'Template' property for the current row object to multipile + the count we got earlier
$Row.Template = "Multiple$Count"
}
#Save the changes to the CSV file as a new CSV. You can also overwrite your old one if you like
$Csv | Export-Csv -Path "C:\Book2.csv"
I don't quite understand your question, but here are some techniques I find useful when working with CSV files.
Example CSV:
Name,City
Bob,BlackPool
Alice,Dover
Carl,Manchester
Presume you assign the CSV file to a variable like so
$CSV = Import-CSV -Path "C:\Stuff.csv"
1. You can access all rows in a column by typing the variable dot(.) column header, so
$CSV.Name
returns:
Bob
Alice
Carl
2. To access a row in a CSV file you need to use indexing, so
$CSV[1]
returns:
Name City
---- ----
Alice Dover
3. An easy way to replace a property of a specific row is to filter it using Where-Object. Say I want to change Carl's city to London.
$($CSV | Where-Object {$_.Name -like "Carl"}).City = "London"
Here is what happens:
What's in the parentheses is processed first, so we are selecting a row where the Name property is like "Carl" (You can use wilcard here, so "Ca*" would have worked too). Then, outside of the parentheses, we are setting the city property to "London".
Note: $_ represents the data currently in a pipeline, in this case that's the row containing Carl.
There is more stuff to know, but this might help you the most.
Don't forget to save your changes by using the Export-CSV cmdlet!
$CSV | Export-CSV -Path "C:\new.csv" -NoTypeInformation
Question
How do I reformat an unknown CSV column name according to a formula or subroutine (e.g. rename column " Arbitrary Column Name " to "Arbitrary Column Name" by running a trim or regex or something) while maintaining data?
Goal
I'm trying to more or less sanitize columns (the names) in a hand-produced (or at least hand-edited) csv file that needs to be processed by an existing PowerShell script. In this specific case, the columns have spaces that would be removed by a call to [String]::Trim(), or which could be ignored with an appropriate regex, but I can't figure a way to call or use those techniques when importing or processing a CSV.
Short Background
Most files and columns have historically been entered into the CSV properly, but recently a few columns were being dropped during processing; I determined it was because the files contained a space (e.g., Select-Object was being told to get "RFC", but Import-CSV retrieved "RFC ", so no matchy-matchy). Telling the customer to enter it correctly by hand (though preferred and much simpler) is not an option in this case.
Options considered
I could manually process the text of the file, but that is a messy and error prone way to re-invent the wheel. I wonder if there's a syntax with Select-Object that would allow a softer match for column names, but I can't find that info.
The closest I have come conceptually is using a calculated property in the call to Select-Object to rename the column, but I can only find ways to rename a known column to another known column. So, this would require enumerating the columns and matching them exactly (preferred) or a softer match (like comparing after trimming or matching via regex as a fallback) with expected column names, then creating a collection of name mappings to use in constructing calculated properties from that information to select into a new object.
That seems like it would work, but more it's work than I'd prefer, and I can't help but hope that there's a simpler way I haven't been able to find via Google. Maybe I should try Bing?
Sample File
Let's say you have a file.csv like this:
" RFC "
"1"
"2"
"3"
Code
Now try to run the following:
$CSV = Get-Content file.csv -First 2 | ConvertFrom-Csv
$FixedHeaders = $CSV.PSObject.Properties.Name.Trim(' ')
Import-Csv file.csv -Header $FixedHeaders |
Select-Object -Skip 1 -Property RFC
Output
You will get this output:
RFC
---
1
2
3
Explanation
First we use Get-Content with parameter -First 2 to get the first two lines. Piping to ConvertFrom-Csv will allow us to access the headers with PSObject.Properties.Name. Use Import-Csv with the -Header parameter to use the trimmed headers. Pipe to Select-Object and use -Skip 1 to skip the original headers.
I'm not sure about comparisons in terms of efficiency, but I think this is a little more hardened, and imports the CSV only once. You might be able to use #lahell's approach and Get-Content -raw, but this was done and it works, so I'm gonna leave it to the community to determine which is better...
#import the CSV
$rawCSV = Import-Csv $Path
#get actual header names and map to their reformatted versions
$CSVColumns = #{}
$rawCSV |
Get-Member |
Where-Object {$_.MemberType -eq "NoteProperty"} |
Select-Object -ExpandProperty Name |
Foreach-Object {
#add a mapping to the original from a trimmed and whitespace-reduced version of the original
$CSVColumns.Add(($_.Trim() -replace '(\s)\s+', '$1'), "$_")
}
#Create the array of names and calculated properties to pass to Select-Object
$SelectColumns = #()
$CSVColumns.GetEnumerator() |
Foreach-Object {
$SelectColumns += {
if ($CSVColumns.values -contains $_.key) {$_.key}
else { #{Name = $_.key; Expression = $CSVColumns[$_.key]} }
}
}
$FormattedCSV = $rawCSV |
Select-Object $SelectColumns
This was hand-copied to a computer where I don't have the rights to run it, so there might be an error - I tried to copy it correctly
You can use gocsv https://github.com/DataFoxCo/gocsv to see the headers of the csv, you can then rename the headers, behead the file, swap columns, join, merge, any number of transformations you want
I have powershell script which connects to database & exports result in csv file.
However there is one column of date which size needs to be manually increased after opening csv file.
Do we have some command/property which will make columns AutoFit?
export-csv $OutFile -NoTypeInformation
I can't export excel instead CSV, cause I don't have excell installed on my machine.
This is what I have tried latest.
$objTable | Select Store,RegNo,Date,#{L="Amount";E={($_.Amount).PadLeft(50," ")}},TranCount
$objTable | export-csv $OutFile -NoTypeInformation
But even after adding PadLeft() output is same, Date column is short in width (showing ###, need to increase value manually)
When you say you need to increase one of your column sizes all the comments were right about how everything is formatted based on the object content. If you really need the cells to be a certain length you need to change the data before it is exported. Using the string methods .PadLeft() and .PadRight() I think you will get what you need.
Take this example using output from Get-ChildItem which uses a calculated property to pad the "column" so that all the data takes up at least 20 characters.
$path = "C:\temp"
$results = Get-ChildItem $path
$results | Select LastWriteTime,#{L="Name";E={($_.Name).PadLeft(20," ")}} | Export-CSV C:\temp\file.csv -NoTypeInformation
If that was then exported the output file would look like this (Notice the whitespace):
"LastWriteTime","Name"
"2/23/2015 7:33:55 PM"," folder1"
"2/23/2015 7:48:02 PM"," folder2"
"2/23/2015 7:48:02 PM"," Folder3"
"1/8/2015 10:37:45 PM"," logoutput"