Related
Need to take my $sw CSV file and use foreach to compare that against a hash translation table $swtranslation, Key field, then output matches including the hash table's values that match into a text file.
Problem I have is it runs the search for a few minutes and returns the sw_names.txt output file with nothing in it. It should have well over 1074+ matches. My guess is my syntax or something is not right.
See code for what I have going so far.
# This is the CSV file listing all the network switches I need to run against the translation table.
$sw = Import-Csv .\AllDeviceForExport.csv -Header Host_Name, IP_Address
# Compile the switch translation table for processing and convert to hash //
$swtranslation = #{};
Import-Csv .\sw_translation.csv -Header Host_Name, DataSpace_ID | % {
$swhash[$_.Host_Name] = $_.DataSpace_ID
}
# Run the Switch listing $sw against the translation table $swtranslation
# matching the DataSpace_ID and merging DataSpace_ID and Host name and
# all other switch fields together in output //
foreach ($key in $swhash.Keys) {
$sw | Select-Object #{n="Name";e={$outputhash[$swhash.Keys($_.Host_Name).Value]}},* |
Where-Object { $_.Name -ne $null } |
Foreach { $_ -replace '--' } |
Out-File ./sw_names.txt -Force
}
Expected results:
Host_Name DataSpace_ID
ABC-123-3750-SW1 1
DEF-234-2950-SW1 5
DEF-234-2950-SW2 5
GHI-567-4510-SW1 6
GHI-567-4510-SW2 6
It's unclear what you are after.
You have two csv files without headers,
.\AllDeviceForExport.csv -Header Host_Name, IP_Address
.\sw_translation.csv -Header Host_Name, DataSpace_ID
Usually one builds a hash table from one file and iterates the other to check if there are matching properties or not.
What your code tries to do is building the hash table, iterate the keys of it and then (very inefficiently) on each key search the whole other file thwarting the whole idea.
Not knowing which files Host_Name property should be checked I suggest a different approach:
Use Compare-Object
## Q:\Test\2019\08\15\SO_57515952.ps1
# simulate $swtrans = Import-Csv .\sw_translation.csv -Header Host_Name, DataSpace_ID
$swtrans = #"
ABC-123-3750-SW1,1
DEF-234-2950-SW1,5
DEF-234-2950-SW2,5
GHI-567-4510-SW1,6
GHI-567-4510-SW2,6
"# -split '\r?\n' | ConvertFrom-Csv -Header Host_Name, DataSpace_ID
# simulate $sw = Import-Csv .\AllDeviceForExport.csv -Header Host_Name, IP_Address
$sw = #"
DEF-234-2950-SW1,192.168.234.1
DEF-234-2950-SW2,192.168.234.2
GHI-567-4510-SW1,192.168.567.1
GHI-567-4510-SW2,192.168.567.2
GHI-567-4510-SW3,192.168.567.3
"# -split '\r?\n' | ConvertFrom-Csv -Header Host_Name, IP_Address
Compare-Object -Ref $swtrans -Diff $sw -Property Host_Name -PassThru -IncludeEqual
This yields:
> Q:\Test\2019\08\15\SO_57515952.ps1
Host_Name DataSpace_ID SideIndicator
--------- ------------ -------------
DEF-234-2950-SW1 5 ==
DEF-234-2950-SW2 5 ==
GHI-567-4510-SW1 6 ==
GHI-567-4510-SW2 6 ==
GHI-567-4510-SW3 =>
ABC-123-3750-SW1 1 <=
The SideIndicator Property can be used to specify which lines to output and itself suppressed.
So I have a CSV file which I need to manipulate a bit, select the data I need and export to another CSV file.
The code I have is:
$rawCSV = "C:\Files\raw.csv"
$outputCSV = "C:\Files\output.csv"
Import-Csv -Header #("a","b","c","d") -Path $rawCSV |
select -Skip 7 |
Where-Object { $_.b.length -gt 1 } |
ft b,a,c,d |
Out-File $outputCSV
So this code uses the Import-Csv command to allow me to select just the columns I need, add some headers in the order I want and then I am simply putting the output in to a CSV file called $outputCSV. The contents of this output file look something like this:
b a c d
- - - -
john smith 29 England
mary poopins 79 Walton
I am not sure what the delimiter is in this output and rather than these columns being treated as individuals, they are treated as just one column. I have gone on further to replace all the spaces with a comma using the code:
$b = foreach ($line in $a)
{
$fields = $line -split '`n'
foreach ($field in $fields)
{
$field -replace " +",","
}
}
Which produces a file that looks like this:
b,a,c,d
john,smith,29,England
mary,poppins,79,Walton
But these are all still treated as one column instead of four separate columns as I need.
* UPDATE *
Using the answer given by #, I now get a file looking like this:
Don't use ft to reorder your columns - it's intended to format output for the screen, not really suitable for CSV.
"Manual" solution:
$rawCSV = "C:\Files\raw.csv"
$outputCSV = "C:\Files\output.csv"
# Import and filter your raw data
$RawData = Import-Csv -Header #("a","b","c","d") -Path $rawCSV
$Data = $RawData | Select -Skip 7 | Where-Object { $_.b.length -gt 1 }
# Write your headers to the output file
"b","a","c","d" -join ',' | Out-File $outputCSV -Force
$ReorderedData = foreach($Row in $Data){
# Reorder the columns in each row
'{0},{1},{2},{3}' -f $Row.b , $Row.a , $Row.c, $Row.d
}
# Write the reordered rows to the output file
$ReorderedData | Out-File $outputCSV -Append
Using Export-Csv:
As of PowerShell 3.0, you could also push the rows into a [pscustomobject] and pipe that to Export-Csv (pscustomobject preserves the order in which you supply the properties):
$rawCSV = "C:\Files\raw.csv"
$outputCSV = "C:\Files\output.csv"
# Import and filter your raw data
$RawData = Import-Csv -Header #("a","b","c","d") -Path $rawCSV
$Data = $RawData | Select -Skip 7 | Where-Object { $_.b.length -gt 1 }
# Take the columns you're interested in, put them into new custom objects and export to CSV
$Data | ForEach-Object {
[pscustomobject]#{ "b" = $_.b; "a" = $_.a; "c" = $_.c; "d" = $_.d }
} | Export-Csv -NoTypeInformation $outputCSV
Export-Csv will take care of enclosing strings in quotes to escape ',' properly (one thing less for you to worry about)
First of all, what your raw CSV file looks like? If it's already like this
john,smith,29,England
mary,poppins,79,Walton
then import-csv will give you an array of objects which you can easily manipulate (and objects are the main reason to use PowerShell ;). For example, to check what you have after import:
$r = Import-Csv -Path $rawCSV -Header #("b","a","c","d")
$r.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
$r[0] | get-member
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
a NoteProperty System.String a=smith
b NoteProperty System.String b=john
c NoteProperty System.String c=29
d NoteProperty System.String d=England
For now you have array of objects with properties named "a","b","c","d". To manipulate objects you have select-object cmdlet:
$r | Select-Object a,b,c,d
a b c d
- - - -
smith john 29 England
poppins mary 79 Walton
And after all use export-csv to set the output file:
$r | where { $_.b.length -gt 1 } |
select a,b,c,d |
Export-Csv -NoTypeInformation -Encoding utf8 -path $outputCSV
I could think of two possible reasons why your data teated as one column:
consuming application expect different encoding and can't find
delimiters
delimiters are not commas but something else
Is there any easy way how to change column position? I'm looking for a way how to move column 1 from the beginning to the and of each row and also I would like to add zero column as a second last column. Please see txt file example below.
Thank you for any suggestions.
File sample
TEXT1,02/10/2015,55.930,57.005,55.600,56.890,1890
TEXT2,02/10/2015,51.060,52.620,50.850,52.510,4935
TEXT3,02/10/2015,50.014,50.74,55.55,52.55,5551
Output:
02/10/2015,55.930,57.005,55.600,56.890,1890,0,TEXT1
02/10/2015,51.060,52.620,50.850,52.510,4935,0,TEXT2
02/10/2015,50.014,50.74,55.55,52.55,5551,0,TEXT3
Another option:
#Prepare test file
(#'
TEXT1,02/10/2015,55.930,57.005,55.600,56.890,1890
TEXT2,02/10/2015,51.060,52.620,50.850,52.510,4935
TEXT3,02/10/2015,50.014,50.74,55.55,52.55,5551
'#).split("`n") |
foreach {$_.trim()} |
sc testfile.txt
#Script starts here
$file = 'testfile.txt'
(get-content $file -ReadCount 0) |
foreach {
'{1},{2},{3},{4},{5},{6},0,{0}' -f $_.split(',')
} | Set-Content $file
#End of script
#show results
get-content $file
02/10/2015,55.930,57.005,55.600,56.890,1890,0,TEXT1
02/10/2015,51.060,52.620,50.850,52.510,4935,0,TEXT2
02/10/2015,50.014,50.74,55.55,52.55,5551,0,TEXT3
Sure, split on commas, spit the results back minus the first result joined by commas, add a 0, and then add the first result to the end and join the whole thing with commas. Something like:
$Input = #"
TEXT1,02/10/2015,55.930,57.005,55.600,56.890,1890
TEXT2,02/10/2015,51.060,52.620,50.850,52.510,4935
TEXT3,02/10/2015,50.014,50.74,55.55,52.55,5551
"# -split "`n"|ForEach{$_.trim()}
$Input|ForEach{
$split = $_.split(',')
($Split[1..($split.count-1)]-join ','),0,$split[0] -join ','
}
I created file test.txt to contain your sample data. I Assigned each field a name, "one","two","three" etc so that i could select them by name, then just selected and exported back to csv in the order you wanted.
First, add the zero to the end, it will end up as second last.
gc .\test.txt | %{ "$_,0" } | Out-File test1.txt
Then, rearrange order.
Import-Csv .\test.txt -Header "one","two","three","four","five","six","seven","eight" | Select-Object -Property two,three,four,five,six,seven,eight,one | Export-Csv test2.txt -NoTypeInformation
This will take the output file and get rid of quotes and header line if you would rather not have them.
gc .\test2.txt | %{ $_.replace('"','')} | Select-Object -Skip 1 | out-file test3.txt
id like to format the output of CSV Records in a specific way.
Lets say, i have a csv file with these records:
Field1,Field2
blah,blah
bluh,bluh
Now, what i want to achieve is an output (into console or file) with this format
('blah','blah')
('bluh','bluh')
This [1] does not work! Powershell simply writes nothing into out.txt.
[1]
Import-Csv .\test.csv | Select-Object Field1,Field2 | ForEach-Object {Write-Host "('"$_.Field1"', '"$_.Field2"')"} > .\out.txt
Use the format operator:
import-csv .\test.csv | %{ "({0},{1})" -f $_.Field1, $_.Field2; }
To output to file, simply pipe the output there; i.e.
import-csv .\test.csv | %{ "({0},{1})" -f $_.Field1, $_.Field2; } | out-file .\out.txt
For more info on the format operator: http://ss64.com/ps/syntax-f-operator.html
Try this if work
...{Write-output "('$($_.Field1)', '$($_.Field2)')"} > .\out.txt
Dont use write-host
I'm using PowerShell To import a TAB separated file with headers. The generated file has a few empty strings "" on the end of first line of headers. PowerShell fails with an error:
"Cannot process argument because the
value of argument "name" is invalid.
Change the value of the "name"
argument and run the operation again"
because the header's require a name.
I'm wondering if anyone has any ideas on how to manipulate the file to either remove the double quotes or enumerate them with a "1" "2" "3" ... "10" etc.
Ideally I would like to not modify my original file. I was thinking something like this
$fileContents = Get-Content -Path = $tsvFileName
$firstLine = $fileContents[0].ToString().Replace('`t""',"")
$fileContents[0] = $firstLine
Import-Csv $fileContents -Delimiter "`t"
But Import-Csv is expecting $fileContents to be a path. Can I get it to use Content as a source?
You can either provide your own headers and ignore the first line of the csv, or you can use convertfrom-csv on the end like Keith says.
ps> import-csv -Header a,b,c,d,e foo.csv
Now the invalid headers in the file is just a row that you can skip.
-Oisin
If you want to work with strings instead use ConvertFrom-Csv e.g.:
PS> 'FName,LName','John,Doe','Jane,Doe' | ConvertFrom-Csv | Format-Table -Auto
FName LName
----- -----
John Doe
Jane Doe
I ended up needing to handle multiple instances of this issue. Rather than use the -Header and manually setting up each import instance I wrote a more generic method to apply to all of them. I cull out all of the `t"" instances of the first line and save the file to open as a $filename + _bak and import that one.
$fileContents = Get-Content -Path $tsvFileName
if( ([string]$fileContents[0]).ToString().Contains('""') )
{
[string]$fixedFirstLine = $fileContents[0].ToString().Replace('`t""',"")
$fileContents[0] = $fixedFirstLine
$tsvFileName = [string]::Format("{0}_bak",$tsvFileName
$fileContents | Out-File -FilePath $tsvFileName
}
Import-Csv $tsvFileName -Delimiter "`t"
My Solution if you have much columns :
$index=0
$ColumnsName=(Get-Content "C:\temp\yourCSCFile.csv" | select -First 1) -split ";" | %{
$index++
"Header_{0:d5}" -f $index
}
import-csv "C:\temp\yourCSCFile.csvv" -Header $ColumnsName