Using Powershell to Export-Csv - powershell

below is the code I am using to find duplicates in a CSV and total them up.
I cant figure out how to Export-CSV correctly
the code below shows what I need in the terminal window
any help would be much appreciated
Import-Csv (Get-ChildItem "E:\Bill3\output.csv") |
Group-Object -Property Code |
Select-Object -Property #{Name='Code';Expression={$_.Name}},
#{Name='Quantity';Expression={
($_.Group| Measure-Object -Sum -Property Quantity ).Sum
}
}
this is how the terminal looks, this is what i would like my exported csv to look like

You can do the following:
Import-Csv (Get-ChildItem "E:\Bill3\output.csv") |
Group-Object -Property Code |
Select-Object -Property #{Name='Code';Expression={$_.Name}},
#{Name='Quantity';Expression={
($_.Group| Measure-Object -Sum -Property Quantity ).Sum
}
} | Export-Csv file.csv -NoType
an empty pipeline element is not allowed
The reason you received the error message above is because of syntax issues with line continuation. The closing } does not continue the line. Therefore, you cannot add your | Export-Csv command on the next line following the last }. They must be on the same line. You can simply run the following code to generate your same error:
PS > | export-csv o.csv
At line:1 char:1
+ | export-csv o.csv
+ ~
An empty pipe element is not allowed.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : EmptyPipeElement
See Natural Line Continuators for an explanation on line continuation.

Related

Using powershell to read a file and split the text, but its throwing an error

I've written a powershell script to scan the network, and put those results into a file, sort and edit those results.
I can do all of that, except some of the pc names are coming back as NAME.DOMAIN.COM and others are just NAME, so I want to split the text, and just get the NAME.
However I'm getting this error for each line in the file:
You cannot call a method on a null-valued expression.
At C:\PowershellScripts\ComputerListV2.ps1:31 char:25
+ $text | Foreach-Object {$_.PCCOMPUTERNAMES.split(".")[0]} | Out-File ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
Here is my script:
[System.Console]::Clear();
#Delete old output file
Write-Output "Removing old output file"
Remove-Item C:\PowershellScripts\ComputerListSorted.txt
#Declare IP range
$range = 1..254
$address = "192.168.0."
#status
Write-Output "Scanning active PCs"
#Scan ip range and get pc names
$range |
ForEach-Object {
Write-Progress 'Scanning Network' $address$_ -PercentComplete (($_ / $range.Count) * 100)
Start-Sleep -Milliseconds 100
Get-WmiObject Win32_PingStatus -Filter "Address='192.168.0.$_' and Timeout=200 and ResolveAddressNames='true' and StatusCode=0" |
Select-Object -Property #{Name="PCCOMPUTERNAMES";Expression={ [Net.DNS]::GetHostByAddress($_.Address).HostName+'.' }}
} | Out-File C:\PowershellScripts\ComputerList.txt
#Sort list
Write-Output "Sort list"
Get-Content -Path C:\PowershellScripts\ComputerList.txt | Sort-Object -Unique | Out-File C:\PowershellScripts\ComputerListRaw.txt
#Edit work file
Get-Content -Path C:\PowershellScripts\ComputerListRaw.txt | Select-Object -Skip 3 | Out-File C:\PowershellScripts\ComputerListEdited.txt
$text = Import-Csv C:\PowershellScripts\ComputerListEdited.txt
$text | Foreach-Object {$_.PCCOMPUTERNAMES.split(".")[0]} | Out-File C:\PowershellScripts\ComputerListEdited2.txt
#Rename file
Rename-Item -Path "C:\PowershellScripts\ComputerListEdited2.txt" -NewName "C:\PowershellScripts\ComputerListSorted.txt"
#Delete old output file
Write-Output "Removing work files"
Remove-Item C:\PowershellScripts\ComputerList.txt
Remove-Item C:\PowershellScripts\ComputerListRaw.txt
#Final statement
Write-Output "Final list in ComputerListSorted.txt"
Here is my sample file ComputerListEdited.txt:
NAME1.
NAME2.DOMAIN.COM
NAME3
NAME4
NAME5.DOMAIN.COM
As zett42 mentioned, your "Import-CSV" function is likely the cause. Running the same code on my computer outputs the error
You cannot call a method on a null-valued expression.
At line:1 char:25
$text | Foreach-Object {$_.PCCOMPUTERNAMES.split(".")[0]} | Out-File ...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CategoryInfo : InvalidOperation: (:) [], RuntimeException
FullyQualifiedErrorId : InvokeMethodOnNull
This is because using "Import-CSV" makes PowerShell treat the first line as the header.
NAME1.
-----
NAME2.DOMAIN.COM
NAME3
NAME4
NAME5.DOMAIN.COM
As the first line is "Name1.", this is the header value for the column of computer names. (Unfortunately, this means it is skipping the first computer as well). Changing this to Get-Content resolves this problem.
Additionally, you're referring to the columns as if they have a header PCCOMPUTERNAMES, which doesn't exist. Remove this reference as well to fix the problem. Try the below in its place.
$text = Get-Content C:\PowershellScripts\ComputerListEdited.txt
$text | Foreach-Object {$_.split(".")[0]} | Out-File C:\PowershellScripts\ComputerListEdited2.txt
A side note as well, you're getting info, exporting to a text file, then reading the text file, then deleting the text file. Keeping this information in a variable inside the script could lead to slightly better performance.

Try hard to learn PowerShell error "The hash literal was incomplete."

Get-Process s* |
where {s$_.Path} |
dir |
sort LastWriteTime |
Format-Table fullname, name,#{label="LastWriteTime";Expr={$_.LastWriteTime}
Error:
The hash literal was incomplete.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : IncompleteHashLiteral
Could You please give me a hint how to rewrite, please?
You missed a curly brace. But there's more problems in your code.
It is not possible to pipe a System.Diagnostics.Process object to 'dir'
I made this, which I think gives the output you want:
Get-Process s* |where {$_.Path} | ForEach-Object {Get-Item $_.Path } |
Sort-Object LastWriteTime | Format-Table fullname, name,LastWriteTime
What it does:
Get all processes where the name start with an s and the returned object has the Path property defined
Get the file object of each process
sort the file objects by LastWriteTime
Format the output

Comparing two CSV files in powershell and creating an output

I'm currently working on a script to import 2 csv files, compare the fqdn columns and output the results to a file.
The issue is after many hours of testing I'm at the point that it looks like my script is working up until the point of getting the path for each file that needs to be imported but I can't seem to get the import-csv command to do what I need it to.
I'd appreciate any guidance you can provide.
My script so far and the error I'm getting are below:
$CMDB_Installed = Get-ChildItem -Path C:\Users\nha1\Desktop\Reports\CMDBInstall | Sort CreationTime -Descending | Select -expa FullName -First 1 | Out-String
$SCOM_AgentList = Get-ChildItem -Path C:\Users\nha1\Desktop\Reports\SCOMUAT | Sort CreationTime -Descending | Select -expa FullName -First 1 | Out-String
$SL = Import-Csv $SCOM_AgentList
$CL = Import-Csv $CMDB_Installed
Compare-Object -ReferenceObject $CL -DifferenceObject $SL -Property fqdn |
Export-Csv -NoTypeInformation -Path = C:\Users\nha1\Desktop\Reports\AuditOutput\UATNeedsAgent+SCOM\UATHosts-NotinSCOM$(get-date -format yyyy-MM-dd).csv
Error Message:
import-csv : Illegal characters in path.
At line:4 char:7
+ $SL = import-csv $SCOM_AgentList
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OpenError: (:) [Import-Csv], ArgumentException
+ FullyQualifiedErrorId : FileOpenFailure,Microsoft.PowerShell.Commands.ImportCsvCommand
import-csv : Illegal characters in path.
At line:5 char:7
+ $CL = import-csv $CMDB_Installed
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OpenError: (:) [Import-Csv], ArgumentException
+ FullyQualifiedErrorId : FileOpenFailure,Microsoft.PowerShell.Commands.ImportCsvCommand
$CMDB_Installed = ... | Select -expa FullName -First 1 | Out-String
Don't use Out-String, unless you want a for-display, multi-line string representation.
Your file-path variables therefore contain newlines (line breaks), which Import-Csv complains about, given that newlines in file names are illegal in NTFS (on Windows).
Simply omit the Out-String call, given that Select -expa FullName -First 1 by definition already outputs a string (given that the .FullName property on the objects output by Get-ChildItem is [string]-typed).
To recreate the problem:
PS> Import-Csv "foo`n" # illegal line break in file path
Import-Csv : Illegal characters in path.
To demonstrate that Out-String produces a multi-line string even with a single-line string as input:
PS> ('foo' | Out-String).EndsWith("`n")
True

Select CSV columns in Powershell where header name contains a specific string

I have a data file of about 10-15 columns from which I want to extract specific columns. Some of the columns I know the exact column header and others I only know that the first two letters will always be "FC".
How do I select only the columns where I know the column header and those that start with "FC"?
Starting with just the "FC" columns, I have tried like this:
$myCSV = Import-CSV "mydata.txt" -Delimiter "`t"
$FCcols = $myCSV[0].psobject.Properties | foreach { $_.Name } | Where {$_ -match "FC"}
$myCSV | select $FCcols
But I just get an error:
Select-Object : Cannot convert System.Management.Automation.PSObject to one of
the following types {System.String, System.Management.Automation.ScriptBlock}.
At line:3 char:16
+ $myCSV | select <<<< $FCcols
+ CategoryInfo : InvalidArgument: (:) [Select-Object], NotSupport
edException
+ FullyQualifiedErrorId : DictionaryKeyUnknownType,Microsoft.PowerShell.Co
mmands.SelectObjectCommand
Then, if I try:
$myCSV = Import-CSV "mydata.txt" -Delimiter "`t"
$FCcols = [System.Collections.ArrayList]#()
$myCSV[0].psobject.Properties | foreach { $_.Name } | Where {$_ -match "FC"} | %{$FCcols.Add($_)}
$myCSV | select $FCcols
I get the output I want except that it is in "column header : value" format, like this:
FC1839 : 0
FC1842 : 1
FC1843 : 6
FC1844 : 12
FC1845 : 4
FC1839 : 0
FC1842 : 0
FC1843 : 19
FC1844 : 22
FC1845 : 14
I am probably just missing something simple, but how do I get to the point that I am able to select these matching columns and then output them to another .txt file (without the header : value format)?
First things first: Mathias R. Jessen's helpful tip not only solves your problem, but significantly simplifies the approach (and also works in PSv2):
$myCSV | Select-Object FC*
The (implied) -Property parameter supports wildcard expressions, so FC* matches all property (column names) that start with FC.
As for the output format you're seeing: Because you're selecting 5 properties, PowerShell defaults to implicit Format-List formatting, with each property name-value pair on its own line.
To fix this display problem, pipe to Format-Table explicitly (which is what PowerShell would do implicitly if you had selected 4 or fewer properties):
$myCSV | Select-Object FC* | Format-Table
To re-export the results to a CSV (TSV) file:
Import-Csv mydata.txt -Delimiter "`t" | Select-Object FC* |
Export-Csv myresults.txt -Encoding Utf8 -Delimiter "`t" -NoTypeInformation
To do so without a header line:
Import-Csv mydata.txt -Delimiter "`t" | Select-Object FC* |
ConvertTo-Csv -Delimiter "`t" -NoTypeInformation | Select-Object -Skip 1 |
Set-Content myresults.txt -Encoding Utf8
As for your specific symptom:
The problem occurs only in PSv2, and it smells like a bug to me.
The workaround is make your column-name array a strongly typed string array ([string[]]):
[string[]] $FCcols = $myCSV[0].psobject.Properties | % { $_.Name } | ? { $_ -match '^FC' }
Note that, for brevity, I've used built-in alias % in lieu of ForEach-Object and ? in lieu of Where-Object.
Also note that the regex passed to -match was changed to ^FC to ensure that only columns that start with FC are matched.
Your code works as-is in PSv3+, but can be simplified:
$FCcols = $myCSV[0].psobject.Properties.Name -match "^FC"
Note how .Name is applied directly to .psobject.Properties, which in v3+ causes the .Name member to be invoked on each item of the collection, a feature called member-access enumeration.
I would use Get-Member to get your columns, something like this:
$myCSV = Import-CSV "mydata.txt" -Delimiter "`t"
$myCSV | select ($myCSV | gm -MemberType NoteProperty | ? {$_.Name -match 'FC'}).Name
Mathias's helpful comment is best way to go for selecting; simple and elegant - dind't know it was an option.
$myCSV | Select *FC*,ColumnIKnowTheNameOf
I believe you need to add Export-Csv to answer your last question. Here's another approach I'd already worked on that makes use of Get-Member and NoteProperty if you need to interrogate csv/similar objects in future.
$myCSV = Import-CSV "mydata.txt" -Delimiter "`t"
# you can get the headings by using Get-Member and Selecting NoteProperty members.
$FCcols = $myCSV |
Get-Member |
Where-Object {$_.MemberType -eq "NoteProperty" -and $_.Name -match "FC"} |
Select-Object -ExpandProperty Name
# you add names to this array.
$FCcols += "ColumnIKnowTheNameOf"
$myCSV | Select-Object $FCcols
# to get a tab-delimited file similar to the one you imported, use Export-Csv
$myCSV | Export-csv "myresults.txt" -Delimiter "`t" -NoTypeInformation
I finally came up with a "quick and dirty" solution which I'm disappointed to not have figured out earlier.
$myCSV = Import-CSV "mydata.txt" -Delimiter "`t" | select FC*
for ($i = 0; $i -lt $myCSV.count; $i++){
$writeline = ($myCSV[$i] | %{$_.PSObject.Properties | %{$_.Value}}) -join "`t"
ac "myresults.txt" $writeline -Encoding utf8}
The first line gives me the columns I want, then the for loop gets the value properties of each column and joins them as tabulated lines, finally each line is appended to a text file.
This may not be the pedagogically correct way to achieve the result, but it works so far.
Thanks to everyone for their input!

PowerShell - Copy specific lines from multiple specific nested files

So, the folder structure looks like this:
RootFolder
RootFolder
SubFolder1
SubSubFolder1
totals.txt
SubSubFolder2
totals.txt
SubFolder2
SubSubFolder1
totals.txt
SubSubFolder2
totals.txt
What I want to do is recursively walk through these Subfolders for the totals.txt file. Read content, and copy lines 22,26,30,34,38,and 42 (with first line being 0 not 1)into a single combined file.
I started with this code:
Get-ChildItem -Recurse | Where-Object {$_ -like "totals.txt" } | Get-Content | Select-Object -Index 22,26,30,34,38,42 | Add-Content "DataInportFile.txt"
However this only finds RootFolder\SubFolder\SubSubFolder\totals.txt and then exits script. Not what I'm looking for...
What I need is the above script to continue searching recursively for the next file and next until all directories have been searched in structure. So I used this:
Get-ChildItem -Recurse | ForEach-Object {$_ -like "totals.txt" } | Get-Content | Select-Object -Index 22,26,30,34,38,42 | Add-Content "DataInportFile.txt"
However, this script errors
C:\Users\user1\scripts\Untitled1.ps1:1 char:69
+ ... urse | ForEach-Object {$_ -like "totals.txt" } | $_.filename #| Get- ...
+ ~~~~~~~~~~~
Expressions are only allowed as the first element of a pipeline.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : ExpressionsMustBeFirstInPipeline
I could use some help figuring out the rest of this powershell script. So close but no joy. Thanks for the help.
Realized I needed to walk through the array
For other's reference, this code works as required:
Get-ChildItem -Recurse | Where-Object {$_ -like "totals.txt" } |
%{
(Get-Content $_.FullName) | Select-Object -Index 22,26,30,34,38,42 | Add-Content "DataInportFile.txt"
}