I can only get the length when exporting this to csv, how should it be done properly.
$redo = Import-CSV c:\temp\testimport.txt | Group-Object email |
foreach { "{0} ,{1}" -f $_.Name, (($_.Group | foreach { $_.group }) -join ', ')
}
$redo | Export-CSV c:\temp\test.csv -NoTypeInformation
#
"Length" "46" "59" "110" "47" "149" "38" "69" "32" "62" "29" "49" "31"
"27" "48" "55" "42"
Export-Csv expects an object (or a list of objects) with properties, whereas your command pipeline produces an array of strings. If you feed this array into Export-Csv the cmdlet takes the properties of each given item (which is only Length for strings) and writes those properties to the output file.
You need to build a list of objects with the desired properties instead, e.g.:
Import-CSV c:\temp\testimport.txt `
| Group-Object email `
| select #{n="Name";e={$_.Name}},#{n="Group";e={($_.Group | %{$_.group}) -join ', '}} `
| Export-CSV c:\temp\test.csv -NoTypeInformation
This was exactly what I was looking for. So I'm just going to add, my comment to be sure everyone understands.
This does NOT work:
$str_list = #('Mark','Henry','John')
$str_list | Export-Csv .\ExportStrList.csv -NoType
Because Export-Csv takes Objects and outputs properties. The only properties for a String[ ] is Length, so the CSV file only contains Lengths.
To fix this, like the last guy said, we need to change the String[ ] into an Object[ ]. The simplest way is with Select-Object.
Put each String into the Name property of a new Object[ ], like this:
$str_list = #('Mark','Henry','John')
$str_list = $str_list | Select-Object #{Name='Name';Expression={$_}}
$str_list | Export-Csv .\ExportStrList.csv -NoType
Just to re-iterate, Select-Object outputs a custom PSObject that can easily be manipulated. This is very powerful information, use it wisely.
The method shown for converting data to objects is a very long-hand approach, at least in my circumstance.
I'm developing reports integrating with an IBM V7000 SAN storage subsystem and it's CLI, which I can call from PS using putty plink, returns either tabular output (which can be CSV) or list depending upon the query.
From either I desire to export the data as CSV.
From the list output I'm looping through the result set (one row = one field) and assembling the fields into a string, separating values with commas. (For the tabular output I get to skip this tedious step.)
The following works to write the output to a CSV file which I can then open as a spreadsheet.
$fhStream = [System.IO.StreamWriter] "20150527_QALUNTable.csv"
$fhStream.WriteLine($stColumnHeadings)
$fhStream.WriteLine($stColumnValues)
$fhStream.Close()
Import-Csv works to return the input as an object that I can easily use to prepare my reports (which are assembled from many such files of output, each gathered at a separate point in time -- hence the datestamp prefix).
There are 57 columns of data here so by converting to a CSV I avoid preparing 57 object statements.
(Found .Net technique for writing output (fastest) at http://blogs.technet.com/b/gbordier/archive/2009/05/05/powershell-and-writing-files-how-fast-can-you-write-to-a-file.aspx)
Related
I have two files. One is called allper.csv
institutiongroup,studentid,iscomplete
institutionId=22343,123,FALSE
institutionId=22343,456,FALSE
institutionId=22343,789,FALSE
The other one is called actswithpersons.csv
abc,123;456
def,456
ghi,123
jkl,123;456
Note: The actswithpersons.csv does not have headers - they are going to be added in later via an excel power query so don't want them in there now. The actswithpersons csv columns are delimited with commas - there are only two columns, and the second one contains multiple personids - again Excel will deal with this later.
I want to remove all rows from allper.csv where the personid doesn't appear in actswithpersons.csv, and export them to another csv. So in the desired outcome, allper.csv would look like this
institutiongroup,studentid,iscomplete
institutionId=22343,123,FALSE
institutionId=22343,456,FALSE
and the export.csv would look like this
institutiongroup,studentid,iscomplete
institutionId=22343,789,FALSE
I've got as far as the below, which will put into the shell whether the personid is found in the actswithpersons.csv file.
$donestuff = (Get-Content .\ActsWithpersons.csv | ConvertFrom-Csv); $ids=(Import-Csv .\allper.csv);foreach($id in $ids.personid) {echo $id;if($donestuff -like "*$id*" )
{
echo 'Contains String'
}
else
{
echo 'Does not contain String'
}}
However, I'm not sure how to go the last step, and export & remove the unwanted rows from allper.csv
I've tried (among many things)
$donestuff = (Get-Content .\ActsWithpersons.csv | ConvertFrom-Csv);
Import-Csv .\allper.csv |
Where-Object {$donestuff -notlike $_.personid} |
Export-Csv -Path export.csv -NoTypeInformation
This took a really long time and left me with an empty csv. So, if you can give any guidance, please help.
Since your actswithpersons.csv doesn't have headers, in order for you to import as csv, you can specify the -Header parameter in either Import-Csv or ConvertFrom-Csv; with the former cmdlet being the better solution.
With that said, you can use any header name for those 2 columns then filter by the given column name (ID in this case) after your import of allper.csv using Where-Object:
$awp = (Import-Csv -Path '.\actswithpersons.csv' -Header 'blah','ID').ID.Split(';')
Import-Csv -Path '.\allper.csv' | Where-Object -Property 'Studentid' -notin $awp
This should give you:
institutiongroup studentid iscomplete
---------------- --------- ----------
institutionId=22343 789 FALSE
If you're looking to do it with Get-Content you can split by the delimiters of , and ;. This should give you just a single row of values which you can then compare the entirety of variable ($awp) using the same filter as above which will give you the same results:
$awp = (Get-Content -Path '.\actswithpersons.csv') -split ",|;"
Import-Csv -Path '.\allper.csv' | Where-Object -Property 'Studentid' -notin $awp
I'm trying to replace some specific parts of a selected string but am only returning the length property. Here's my code:
Get-ChildItem "StartPath/Something/Files" -Recurse -File | Select "FullName | Foreach {$_.FullName -replace "StartPath",""} | Export-Csv "ResultPath.csv"
If I omit the foreach bit, this works in that it spits out the full path. I'd like to trim the full path as I'm iterating over tons of files. I'm trying to replace a bit of the path in the beginning of the string but my code above just spits out a CSV file with just string lengths.
Looks like:
"123"
"12"
"52"
and so forth.
The intended result would be a csv file with instead of:
StartPath/Something/Files1
StartPath/Something/Files2
I'd have
Something/Files1
Something/Files2
I've tried a number of things and can't seem to figure it out. Any help is appreciated.
If you pass a string to select / Select-Object (to its positionally implied -Property parameter), it must be a property name.[1]
If you want to perform open-ended operations and/or produce open-ended output for each input object, you must use the ForEach-Object cmdlet:
Get-ChildItem "StartPath/Something/Files" -Recurse -File |
ForEach-Object {
[pscustomobject] #{ FullName = $_.FullName -replace 'StartPath' }
} |
Export-Csv "ResultPath.csv"
Note the use of a [pscustomobject] wrapper that defines a FullName property, so that Export-Csv creates a CSV with that property as its (only) column.
If you pipe [string] instances directly to Export-Csv, their properties are serialized to the output file - and a [string]'s only (public) property is its length (.Length), which is what you saw.
[1] There's also a way to create properties dynamically, using so-called calculated properties, which are defined via hash tables.
How to find the count of unique numbers in a CSV file? When I use the following command in PowerShell ISE
1,2,3,4,2 | Sort-Object | Get-Unique
I can get the unique numbers but I'm not able to get this to work with CSV files. If for example I use
$A = Import-Csv C:\test.csv | Sort-Object | Get-Unique
$A.Count
it returns 0. I would like to count unique numbers for all the files in a given folder.
My data looks similar to this:
Col1,Col2,Col3,Col4
5,,7,4
0,,9,
3,,5,4
And the result should be 6 unique values (preferably written inside the same CSV file).
Or would it be easier to do it with Notepad++? So far I have found examples only on how to count the unique rows.
You can try the following (PSv3+):
PS> (Import-CSV C:\test.csv |
ForEach-Object { $_.psobject.properties.value -ne '' } |
Sort-Object -Unique).Count
6
The key is to extract all property (column) values from each input object (CSV row), which is what $_.psobject.properties.value does;
-ne '' filters out empty values.
Note that, given that Sort-Object has a -Unique switch, you don't need Get-Unique (you need Get-Unique only if your input already is sorted).
That said, if your CSV file is structured as simply as yours, you can speed up processing by reading it as a text file (PSv2+):
PS> (Get-Content C:\test.csv | Select-Object -Skip 1 |
ForEach-Object { $_ -split ',' -ne '' } |
Sort-Object -Unique).Count
6
Get-Content reads the CSV file as a line of strings.
Select-Object -Skip 1 skips the header line.
$_ -split ',' -ne '' splits each line into values by commas and weeds out empty values.
As for what you tried:
Import-CSV C:\test.csv | Sort-Object | Get-Unique:
Fundamentally, Sort-Object emits the input objects as a whole (just in sorted order), it doesn't extract property values, yet that is what you need.
Because no -Property argument is passed to Sort-Object to base the sorting on, it compares the custom objects that Import-Csv emits as a whole, by their .ToString() values, which happen to be empty[1]
, so they all compare the same, and in effect no sorting happens.
Similarly, Get-Unique also determines uniqueness by .ToString() here, so that, again, all objects are considered the same and only the very first one is output.
[1] This may be surprising, given that using a custom object in an expandable string does yield a value: compare $obj = [pscustomobject] #{ foo ='bar' }; $obj.ToString(); '---'; "$obj". This inconsistency is discussed in this GitHub issue.
I filtered by date this file data1.csv
2017.11.1,09:55,1.1,1.2,1.3,1.4,1
2017.11.2,09:55,1.5,1.6,1.7,1.8,2
I don't get a header with -NoTypeInformation:
$CutOff = (Get-Date).AddDays(-2)
$filePath = "data1.csv"
$Data = Import-Csv $filePath -Header Date,Time,A,B,C,D,E
$Data2 = $Data | Where-Object {$_.Date -as [datetime] -gt $Cutoff} | convertto-csv -NoTypeInformation -Delimiter "," | % {$_ -replace '"',''}
But when rewriting with Out-File
$Data2 | Out-File "data2.csv" -Encoding utf8 -Force
I get header back as data2.csv contains:
Date,Time,A,B,C,D,E
2017.11.2,09:55,1.5,1.6,1.7,1.8,2
Why do I have Date,Time,A,B,C,D,E ?
-NoTypeInformation is not about the header but the data type of the rows in the file. Remove it to see what shows up. From Microsoft
Omits the type information header from the output. By default, the string in the output contains #TYPE followed by the fully-qualified name of the object type.
Emphasis mine.
CSVs need headers. That is why it is making one. If you don't want to see the header in the output use Select-Object -Skip 1 to remove it.
$Data |
Where-Object {$_.Date -as [datetime] -gt $Cutoff} |
ConvertTo-CSV -NoTypeInformation -Delimiter "," |
Select-Object -Skip 1 |
% {$_ -replace '"'}
I would not pipe Out-File to itself. You could pipe to Set-Content here just as well.
I am guessing this whole process is to keep the source file in the same state just with some lines filtered out based on date. You could skip most of this just by parsing the date out in each line.
$threshold = (Get-Date).AddDays(-2)
$filePath = "c:\temp\bagel.txt"
(Get-Content $filePath) | Where-Object{
$date,$null=$_.Split(",",2)
[datetime]$date -gt $threshold
} | Set-Content $filePath
Now you don't have to worry about PowerShell CSV object structure or output since we act on the raw data of the file itself.
That will take each line of the input file and filter it out if the parsed date does not match the threshold. Change encoding on the input output cmdlets as you see necessary. What $date,$null=$_.Split(",",2) is doing is splitting the line
on the comma into 2 parts. First of which becomes $date and since this is just a filtering condition we dump the rest of the line into $null.
Properly-formed CSV files must have column headers. Your use of -NoTypeInformation in generating the CSV does not affect column headers; instead, it affects whether the PowerShell object type information is included. If you Export-CSV without -NoTypeInformation, the first line of your CSV file will have a line that looks like #TYPE System.PSCustomObject, which you don't want if you're going to open the CSV in a spreadsheet program.
If you subsequently Import-CSV, the headers (Date, Time, A, B, C) are used to create the fields of a PSObject, so that you can refer to them using the standard dot notation (e.g., $CSV[$line].Date).
The ability to specify -Header on Import-CSV is essentially a "hack" to allow the cmdlet to handle files that are comma-separated, but which did not include column headers.
Since, i am a beginner, i 've no much hands-on to the powershell programming.Although
i had a script developed to insert data from an array to the csv file as follows:
#Following is the array
$InventoryReport = New-Object -TypeName PSobject -Property #{
ComputerName = "1myComputerName"
DomainName = "2myComputerDomain"
Manufacturer = "3myComputerManufacturer"
}
#Now to export the data to csv, i am using following:
$InventoryReport |Select-Object -Property ComputerName, DomainName, Manufacturer | Export-Csv -Path "c:\abc.csv" -NoTypeInformation -ErrorAction Stop
#This works fine
and the output of above is :
"ComputerName","DomainName","Manufacturer"
"1myComputerName","2myComputerDomain","3myComputerManufacturer"
....
Now, i don't want this , i want the ouput to appear in columnar fashion i.e.
"ComputerName","1myComputerName"
"DomainName","2myComputerDomain"
"Manufacturer","3myComputerManufacturer"
What code changes should be done to achieve this. ?
Either you want CSV, which you already have, or you want a custom txt-file. If you want the latter, try this:
$comp = gwmi win32_computersystem
#"
"ComputerName","$($comp.Name)"
"DomainName","$($comp.Domain)"
"Manufacturer","$($comp.Manufacturer)"
"# | Out-File test.txt
sample of test.txt output below. I've got a non-domain, custom built pc, so don't worry about the values.
"ComputerName","GRAIMER-PC"
"DomainName","WORKGROUP"
"Manufacturer","System manufacturer"
EDIT I suggest you learn what CSV is. Remember that CSV is not a fileformat, it's a formatting-style used in a normal textfile. The .csv extension is just cosmetic to let people know that the textfile uses the csv-style. Check out Wikipedia and Technet
In the CSV file, each object is represented by a comma-separated list
of the property values of the object. The property values are
converted to strings (by using the ToString() method of the object),
so they are generally represented by the name of the property value.
Export-CSV does not export the methods of the object.
The format of an exported file is as follows:
-- The first line of the CSV file contains the string '#TYPE ' followed by the fully qualified name of the object, such as #TYPE
System.Diagnostics.Process. To suppress this line, use the
NoTypeInformation parameter.
-- The next line of the CSV file represents the column headers. It contains a comma-separated list of the names of all the properties of
the first object.
-- Additional lines of the file consist of comma-separated lists of the property values of each object.
You could try something like this:
$InventoryReport | Format-List ComputerName, DomainName, Manufacturer `
| Out-String -Stream `
| ? { $_ -ne '' } `
| % { $_ -replace '\s+:\s+', '","' -replace '(^|$)', '"' }