Comparing two CSV files in powershell and creating an output - powershell

I'm currently working on a script to import 2 csv files, compare the fqdn columns and output the results to a file.
The issue is after many hours of testing I'm at the point that it looks like my script is working up until the point of getting the path for each file that needs to be imported but I can't seem to get the import-csv command to do what I need it to.
I'd appreciate any guidance you can provide.
My script so far and the error I'm getting are below:
$CMDB_Installed = Get-ChildItem -Path C:\Users\nha1\Desktop\Reports\CMDBInstall | Sort CreationTime -Descending | Select -expa FullName -First 1 | Out-String
$SCOM_AgentList = Get-ChildItem -Path C:\Users\nha1\Desktop\Reports\SCOMUAT | Sort CreationTime -Descending | Select -expa FullName -First 1 | Out-String
$SL = Import-Csv $SCOM_AgentList
$CL = Import-Csv $CMDB_Installed
Compare-Object -ReferenceObject $CL -DifferenceObject $SL -Property fqdn |
Export-Csv -NoTypeInformation -Path = C:\Users\nha1\Desktop\Reports\AuditOutput\UATNeedsAgent+SCOM\UATHosts-NotinSCOM$(get-date -format yyyy-MM-dd).csv
Error Message:
import-csv : Illegal characters in path.
At line:4 char:7
+ $SL = import-csv $SCOM_AgentList
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OpenError: (:) [Import-Csv], ArgumentException
+ FullyQualifiedErrorId : FileOpenFailure,Microsoft.PowerShell.Commands.ImportCsvCommand
import-csv : Illegal characters in path.
At line:5 char:7
+ $CL = import-csv $CMDB_Installed
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OpenError: (:) [Import-Csv], ArgumentException
+ FullyQualifiedErrorId : FileOpenFailure,Microsoft.PowerShell.Commands.ImportCsvCommand

$CMDB_Installed = ... | Select -expa FullName -First 1 | Out-String
Don't use Out-String, unless you want a for-display, multi-line string representation.
Your file-path variables therefore contain newlines (line breaks), which Import-Csv complains about, given that newlines in file names are illegal in NTFS (on Windows).
Simply omit the Out-String call, given that Select -expa FullName -First 1 by definition already outputs a string (given that the .FullName property on the objects output by Get-ChildItem is [string]-typed).
To recreate the problem:
PS> Import-Csv "foo`n" # illegal line break in file path
Import-Csv : Illegal characters in path.
To demonstrate that Out-String produces a multi-line string even with a single-line string as input:
PS> ('foo' | Out-String).EndsWith("`n")
True

Related

Using powershell to read a file and split the text, but its throwing an error

I've written a powershell script to scan the network, and put those results into a file, sort and edit those results.
I can do all of that, except some of the pc names are coming back as NAME.DOMAIN.COM and others are just NAME, so I want to split the text, and just get the NAME.
However I'm getting this error for each line in the file:
You cannot call a method on a null-valued expression.
At C:\PowershellScripts\ComputerListV2.ps1:31 char:25
+ $text | Foreach-Object {$_.PCCOMPUTERNAMES.split(".")[0]} | Out-File ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
Here is my script:
[System.Console]::Clear();
#Delete old output file
Write-Output "Removing old output file"
Remove-Item C:\PowershellScripts\ComputerListSorted.txt
#Declare IP range
$range = 1..254
$address = "192.168.0."
#status
Write-Output "Scanning active PCs"
#Scan ip range and get pc names
$range |
ForEach-Object {
Write-Progress 'Scanning Network' $address$_ -PercentComplete (($_ / $range.Count) * 100)
Start-Sleep -Milliseconds 100
Get-WmiObject Win32_PingStatus -Filter "Address='192.168.0.$_' and Timeout=200 and ResolveAddressNames='true' and StatusCode=0" |
Select-Object -Property #{Name="PCCOMPUTERNAMES";Expression={ [Net.DNS]::GetHostByAddress($_.Address).HostName+'.' }}
} | Out-File C:\PowershellScripts\ComputerList.txt
#Sort list
Write-Output "Sort list"
Get-Content -Path C:\PowershellScripts\ComputerList.txt | Sort-Object -Unique | Out-File C:\PowershellScripts\ComputerListRaw.txt
#Edit work file
Get-Content -Path C:\PowershellScripts\ComputerListRaw.txt | Select-Object -Skip 3 | Out-File C:\PowershellScripts\ComputerListEdited.txt
$text = Import-Csv C:\PowershellScripts\ComputerListEdited.txt
$text | Foreach-Object {$_.PCCOMPUTERNAMES.split(".")[0]} | Out-File C:\PowershellScripts\ComputerListEdited2.txt
#Rename file
Rename-Item -Path "C:\PowershellScripts\ComputerListEdited2.txt" -NewName "C:\PowershellScripts\ComputerListSorted.txt"
#Delete old output file
Write-Output "Removing work files"
Remove-Item C:\PowershellScripts\ComputerList.txt
Remove-Item C:\PowershellScripts\ComputerListRaw.txt
#Final statement
Write-Output "Final list in ComputerListSorted.txt"
Here is my sample file ComputerListEdited.txt:
NAME1.
NAME2.DOMAIN.COM
NAME3
NAME4
NAME5.DOMAIN.COM
As zett42 mentioned, your "Import-CSV" function is likely the cause. Running the same code on my computer outputs the error
You cannot call a method on a null-valued expression.
At line:1 char:25
$text | Foreach-Object {$_.PCCOMPUTERNAMES.split(".")[0]} | Out-File ...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CategoryInfo : InvalidOperation: (:) [], RuntimeException
FullyQualifiedErrorId : InvokeMethodOnNull
This is because using "Import-CSV" makes PowerShell treat the first line as the header.
NAME1.
-----
NAME2.DOMAIN.COM
NAME3
NAME4
NAME5.DOMAIN.COM
As the first line is "Name1.", this is the header value for the column of computer names. (Unfortunately, this means it is skipping the first computer as well). Changing this to Get-Content resolves this problem.
Additionally, you're referring to the columns as if they have a header PCCOMPUTERNAMES, which doesn't exist. Remove this reference as well to fix the problem. Try the below in its place.
$text = Get-Content C:\PowershellScripts\ComputerListEdited.txt
$text | Foreach-Object {$_.split(".")[0]} | Out-File C:\PowershellScripts\ComputerListEdited2.txt
A side note as well, you're getting info, exporting to a text file, then reading the text file, then deleting the text file. Keeping this information in a variable inside the script could lead to slightly better performance.

Trying pipe in a list of CSV in to import CSV

So I am trying to pipe in a file list into import CSV:
ls *.csv | select FullName | where FullName -NotMatch "fixed" | ForEach-Object {
Import-Csv -Path %($_.FullName) -Delimiter ";"
}
But I am getting this error:
Import-Csv : A positional parameter cannot be found that accepts argument
'C:\***\Documents\Daraz Order\order.list.export 2019-11-13.csv'.
At line:2 char:1
+ Import-Csv -Path %($_.FullName) -Delimiter ";"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Import-Csv], ParameterBindingException
+ FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.ImportCsvCommand
It gives the same error for all the files in the list. I have tried the ToString() method. But the error persist:
+ Import-Csv -Path %($_.FullName.ToString()) -Delimiter ";"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Import-Csv], ParameterBindingException
+ FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.ImportCsvCommand
What might be causing this?
The answer Patrick gave you is correct. However you still get the error. You should try different methods of troubleshooting (using Powershell_ISE obviously).
$import = #()
ls *.csv| select FullName | where FullName -NotMatch "fixed" | foreach-object {
$path = $_.FullName
$path.GetType()
$path
try{
Get-Content -Path $path
}catch{
Write-Host "Path does not exist"
}
$import += Import-Csv -path $path -delimiter ";"
}
In the above example, to make sure we don't get distracted, we put all the content in the import array. Next look at the path, does it exist? Can you obtain the content of the file? Let's look at the type of the file by trying .GetType().
The error basically says that Import-CSV does not except your argument. So try fixing Import-CSV with a valid path to see if you get it to work first.
PS: Your script does actually work on my system (PowerShell 5.1).
It seems some idiosyncrasy of PowerShell 6. I just decided to use a code block and just pipes. The code that worked for me was:
ls *.csv| select FullName,BaseName | where FullName -NotMatch "fixed"| foreach {
$file = $_.FullName.ToString()
$bname = $_.BaseName.ToString()
$bnamepath = ".\$bname-fixed.csv"
Import-Csv $file -Delimiter ';' | Export-Csv $bnamepath -Delimiter ','
}
I still had to do a pipe at the end because keeping the data in a variable causes the output file just have the object properties and not the values of the other CSV. This script will work if you wan to process the files in Excel without changing your system's locale.

Try hard to learn PowerShell error "The hash literal was incomplete."

Get-Process s* |
where {s$_.Path} |
dir |
sort LastWriteTime |
Format-Table fullname, name,#{label="LastWriteTime";Expr={$_.LastWriteTime}
Error:
The hash literal was incomplete.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : IncompleteHashLiteral
Could You please give me a hint how to rewrite, please?
You missed a curly brace. But there's more problems in your code.
It is not possible to pipe a System.Diagnostics.Process object to 'dir'
I made this, which I think gives the output you want:
Get-Process s* |where {$_.Path} | ForEach-Object {Get-Item $_.Path } |
Sort-Object LastWriteTime | Format-Table fullname, name,LastWriteTime
What it does:
Get all processes where the name start with an s and the returned object has the Path property defined
Get the file object of each process
sort the file objects by LastWriteTime
Format the output

Powershell: how to replace a value within format-table

I want to shorten Directory with relative path:
$Dir = get-childitem C:\temp -recurse
$List = $Dir | where {$_.extension -eq ".txt"}
$List | format-table name, Directory -replace "C:\temp", ""
I get this error:
Format-Table : A parameter cannot be found that matches parameter name 'replace'.
At line:3 char:38
+ $List | format-table name, Directory -replace "C:\temp", ""
+ ~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Format-Table], ParameterBindingException
+ FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.PowerShell.Commands.FormatTableCommand
What's the right syntax ?
You can use a calculated property. Example:
$List | Format-Table name,
#{Name = "Directory"; $Expression = {$_.FullName -replace "C:\\temp", ""}}
A calculated property is simply a hashtable that dictates the content of the property. Calculated properties are available with formatting cmdlets that select properties and output a new custom object (e.g, Select-Object, Format-List, etc.).
(As an aside: The -replace operator uses a regular expression, so you would need to write C:\\temp instead of just C:\temp.)
If your goal is to output file system item directory names: Directory is not a property of all file system objects. Is this what you mean?
Get-ChildItem C:\Temp\*.txt -Recurse | Format-Table Name,
#{Name = "Directory"; Expression = {$_.FullName -replace 'C:\\temp', ''}}
Note how this command takes advantage of the pipeline (no need for the intermediate $List or $Dir variables).
To add to #Bill_Stewart's Answer.
$Dir = get-childitem C:\temp -recurse
$List = $Dir | where {$_.extension -eq ".txt"}
$List | format-table name, #{Label="Directory"; Expression={$_.Directory -replace "C:\\temp", ""}}

'Cannot find drive' powershell error after invoking Get-Date command

Why is it that this command is able to export and create a file in the current path but when I add the Get-Date cmdlet it suddenly fails?
Is the Get-Date cmdlet invoking some type of new environment?
Working command -
Get-Process | Sort-Object WorkingSet64 | Select-Object Name,#{Name='WorkingSet';Expression={($_.WorkingSet64/1MB)}} | Export-Csv -Path "processes64.csv" -Delimiter ","
Command breaks -
$Date = Get-Date -Format "MM-dd-yy-HH:MM"
Get-Process | Sort-Object WorkingSet64 | Select-Object Name,#{Name='WorkingSet';Expression={($_.WorkingSet64/1MB)}} | Export-Csv -Path "processes64$Date.csv" -Delimiter ","
Error message -
Export-Csv : Cannot find drive. A drive with the name 'processes64-06-28-16-15' does not exist.
At line:3 char:120
... Set64/1MB)}} | Export-Csv -Path "processes64-$Date.csv" -Delimiter ","
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CategoryInfo : ObjectNotFound: (processes6406-28-16-15:String) [Export-Csv], DriveNotFoundException
FullyQualifiedErrorId : DriveNotFound,Microsoft.PowerShell.Commands.ExportCsvCommand
I'd just like to either export to a directory on the C drive or to the current working directory..
I believe I figured it out, it's because of the colon in the date format. File names can't have colons..
But I'm still curious as to what Powershell is interpreting, the error message doesn't seem to have anything to do with an invalid file name.