PowerShell script to export to single CSV - powershell

I need help to get the following PowerShell script to output too just one .CSV file instead of the current two. Can someone please help? I'm wanting to get the SNMP settings that are set on remote servers. The information I'm after is held in the registry. $DellAdmKey is one key that holds SubKeys and refers to the "communityNames". Within each of the SubKeys I get the "Value Data" which refers to the "TrapDestinations". $DellAdmKey2 is a Key called "ValidCommunities" but does not have Subkeys, it just has DWORD values that refer to "AcceptedCommunityNames" and the "Rights". So the foreach on the 1st line can't be used as "ValidCommunities" does not contain SubKeys
$servers = "ServerName"
$BaseKey = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey(‘LocalMachine’, $server)
$SubKey= $BaseKey.OpenSubKey("SYSTEM\\CurrentControlSet\\Services\\SNMP\Parameters",$true)
$DellAdmKey = $BaseKey.OpenSubKey("SYSTEM\\CurrentControlSet\\Services\\SNMP\\Parameters\\TrapConfiguration\\",$true)
$DellAdmKey2 = $BaseKey.OpenSubKey("SYSTEM\\CurrentControlSet\\Services\\SNMP\\Parameters\\ValidCommunities\\",$true)
$DellAdmKey.GetSubKeyNames() | foreach {
$action = $_
$subkey = $DellAdmKey.openSubKey($_)
$subkey.GetValueNames() | foreach {$_ | Select
#{name="ServerName";Expression={$server}},
#{Name="CommunityNames";Expression={$action}},
#{name="TrapDestinations";Expression={$subkey.getvalue($_)}}
} | Export-Csv c:\temp\1.csv -NoTypeInformation
}
$action = $_
$subkey = $DellAdmKey2.openSubKey($_)
$subkey.GetValueNames() | foreach {
$_ | Select #{name="ServerName";Expression={$server}},
#{name="AcceptedCommunityNames";Expression={$_}},
#{name="Rights";Expression={$subkey.getvalue($_)}}
} | Export-Csv c:\temp\2.csv -NoTypeInformation

If I understand your question (and your code) correctly, the third group of instructions is supposed to go inside the $DellAdmKey.GetSubKeyNames() | foreach { ... }. In that case you simply need to move the | Export-Csv outside that loop to capture all output in one CSV. You need to make sure all objects have the same set of properties, though.
$DellAdmKey.GetSubKeyNames() | foreach {
$action = $_
$subkey = $DellAdmKey.openSubKey($action)
$subkey.GetValueNames() | Select #{name="ServerName";Expression={$server}},
#{Name="CommunityNames";Expression={$action}},
#{name="TrapDestinations";Expression={$subkey.getvalue($_)}}
#{name="AcceptedCommunityNames";Expression={}},
#{name="Rights";Expression={}}
$subkey = $DellAdmKey2.openSubKey($action)
$subkey.GetValueNames() | Select #{name="ServerName";Expression={$server}},
#{Name="CommunityNames";Expression={}},
#{name="TrapDestinations";Expression={}},
#{name="AcceptedCommunityNames";Expression={$_}},
#{name="Rights";Expression={$subkey.getvalue($_)}}
} | Export-Csv 'c:\temp\out.csv' -NoTypeInformation
Edit: If you really just want to run the code as you posted (which seems somewhat dubious to me, since $_ in line 17 should be empty), you could change the second Export-Csv instruction to
... | Export-Csv 'c:\temp\1.csv' -Append -NoTypeInformation
provided you're running PowerShell v3. Prior to that something like this should do:
... | ConvertTo-Csv -NoTypeInformation | select -Skip 1 |
Out-File 'c:\temp\1.csv' -Append
Either way you should make sure all the objects you're exporting have the same set of properties, though. Otherwise you might end up with Rights values in the TrapDestination column or some such.

I have resorted to using this http://psrr.codeplex.com/ it make life easier. Thanks for the replies Ansgar.

Related

PowerShell Export-CSV - Missing Columns [duplicate]

This question already has an answer here:
Not all properties displayed
(1 answer)
Closed 1 year ago.
This is a follow-up question from PowerShell | EVTX | Compare Message with Array (Like)
I changed the tactic slightly, now I am collecting all the services installed,
$7045 = Get-WinEvent -FilterHashtable #{ Path="1system.evtx"; Id = 7045 } | select
#{N=’Timestamp’; E={$_.TimeCreated.ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')}},
Id,
#{N=’Machine Name’; E={$_.MachineName}},
#{N=’Service Name’; E={$_.Properties[0].Value}},#{N=’Image Path’;E=$_.Properties[1].Value}},
#{N=’RunAsUser’; E={$_.Properties[4].Value}},#{N=’Installed By’; E={$_.UserId}}
Now I match each object for any suspicious traits and if found, I add a column 'Suspicious' with the value 'Yes'. This is because I want to leave the decision upto the analyst and pretty sure the bad guys might use something we've not seen before.
foreach ($Evt in $7045)
{
if ($Evt.'Image Path' -match $sus)
{
$Evt | Add-Member -MemberType NoteProperty -Name 'Suspicious' -Value 'Yes'
}
}
Now, I'm unable to get PowerShell to display all columns unless I specifically Select them
$7045 | Format-Table
Same goes for CSV Export. The first two don't include the Suspicious Column but the third one does but that's because I'm explicitly asking it to.
$7045 | select * | Export-Csv -Path test.csv -NoTypeInformation
$7045 | Export-Csv -Path test.csv -NoTypeInformation
$7045 | Select-Object Timestamp, Id, 'Machine Name', 'Service Name', 'Image Path', 'RunAsUser', 'Installed By', Suspicious | Export-Csv -Path test.csv -NoTypeInformation
I read the Export-CSV documentation on MS. Searched StackOverFlow for some tips, I think it has something to do with PS checking the first Row and then compares if the property exists for the second row and so on.
Thank you
The issue you're experiencing is partially because of how objects are displayed to the console, the first object's Properties determines the displayed Properties (Columns) to the console.
The bigger problem though, is that Export-Csv will not export those properties that do not match with first object's properties unless they're explicitly added to the remaining objects or the objects are reconstructed, for this one easy way is to use Select-Object as you have pointed out in the question.
Given the following example:
$test = #(
[pscustomobject]#{
A = 'ValA'
}
[pscustomobject]#{
A = 'ValA'
B = 'ValB'
}
[pscustomobject]#{
C = 'ValC'
D = 'ValD'
E = 'ValE'
}
)
Format-Table will not display the properties B to E:
$test | Format-Table
A
-
ValA
ValA
Format-List can display the objects properly, this is because each property with it's corresponding value has it's own console line in the display:
PS /> $test | Format-List
A : ValA
A : ValA
B : ValB
C : ValC
D : ValD
E : ValE
Export-Csv and ConvertTo-Csv will also miss properties B to E:
$test | ConvertTo-Csv
"A"
"ValA"
"ValA"
You have different options as a workaround for this, you could either add the Suspicious property to all objects and for those events that are not suspicious you could add $null as Value.
Another workaround is to use Select-Object explicitly calling the Suspicious property (this works because you know the property is there and you know it's Name).
If you did not know how many properties your objects had, a dynamic way to solve this would be to discover their properties using the PSObject intrinsic member.
using namespace System.Collections.Generic
function ConvertTo-NormalizedObject {
[CmdletBinding()]
param(
[Parameter(ValueFromPipeline, Mandatory)]
[object[]] $InputObject
)
begin {
$list = [List[object]]::new()
$props = [HashSet[string]]::new([StringComparer]::InvariantCultureIgnoreCase)
}
process {
foreach($object in $InputObject) {
$list.Add($object)
foreach($property in $object.PSObject.Properties) {
$null = $props.Add($property.Name)
}
}
}
end {
$list | Select-Object ([object[]] $props)
}
}
Usage:
# From Pipeline
$test | ConvertTo-NormalizedObject | Format-Table
# From Positional / Named parameter binding
ConvertTo-NormalizedObject $test | Format-Table
Lastly, a pretty easy way of doing it thanks to Select-Object -Unique:
$prop = $test.ForEach{ $_.PSObject.Properties.Name } | Select-Object -Unique
$test | Select-Object $prop
Using $test for this example, the result would become:
A B C D E
- - - - -
ValA
ValA ValB
ValC ValD ValE
Continuing from my previous answer, you can add a column Suspicious straight away if you take out the Where-Object filter and simply add another calculated property to the Select-Object cmdlet:
# create a regex for the suspicious executables:
$sus = '(powershell|cmd|psexesvc)\.exe'
# alternatively you can join the array items like this:
# $sus = ('powershell.exe','cmd.exe','psexesvc.exe' | ForEach-Object {[regex]::Escape($_)}) -join '|'
$7045 = Get-WinEvent -FilterHashtable #{ LogName = 'System';Id = 7045 } |
Select-Object Id,
#{N='Timestamp';E={$_.TimeCreated.ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')}},
#{N='Machine Name';E={$_.MachineName}},
#{N='Service Name'; E={$_.Properties[0].Value}},
#{N='Image Path'; E={$_.Properties[1].Value}},
#{N='RunAsUser'; E={$_.Properties[4].Value}},
#{N='Installed By'; E={$_.UserId}},
#{N='Suspicious'; E={
if ($_.Properties[1].Value -match $sus) { 'Yes' } else {'No'}
}}
$7045 | Export-Csv -Path 'X:\Services.csv' -UseCulture -NoTypeInformation
Because you have many columns, this will not fit the console width anymore if you do $7045 | Format-Table, but the CSV file will hold all columns you wanted.
I added switch -UseCulture to the Export-Csv cmdlet, which makes sure you can simply double-click the csv file so it opens correctly in your Excel.
As sidenote: Please do not use those curly so-called 'smart-quotes' in code as they may lead to unforeseen errors. Straighten these ’ thingies and use normal double or single quotes (" and ')

How to solve the "'IndexOf'." error in PowerShell script

While running the below script I am getting an error:
"Method invocation failed because [System.Management.Automation.PSCustomObject] doesn't contain a method named 'IndexOf'".
Please help me to find out a solution to avoid the above error while running the below script.
Code:
$serverlist_csv1 = Import-Csv -Path $file1
$serverlist_temp1 = $serverlist_csv1
$exclude_serverlist_csv = Import-Csv -Path $file2
foreach ($server in $exclude_serverlist_csv) {
$servers1 = ($serverlist_csv1.'SourceHostName' | Select-Object -ExcludeProperty 'SourceHostName')
if ($servers1 -contains $server.'Exclude Server') {
$server_object1 = ($serverlist_csv1 | Where-Object {$_.SourceHostName -eq $server.'Exclude Server'})
$serverindex1 = $serverlist_csv1.IndexOf($server_object1)
$dataResizable1 = {$serverlist_csv1}.Invoke()
$dataResizable1.RemoveAt($serverindex1)
$serverlist_csv1 = $dataResizable1
}
}
You are getting that error because you used select-object on the pipeline invocation of $serverlist_csv1, which creates a layer over the object effectively creating a new object a pscustomobject and that makes it lose the indexOf() method.
if you avoid this step and instead try for instance to exclude that on the import of the csv or modify the member to nothing if that property isn't necessary to you. The point is like people before me said that does not seem like the better way to solve whatever problem this is.
Build a list of the server names from the second file:
$exclude = Import-Csv $file2 | Select-Object -Expand 'Exclude Server'
Then filter the first file for rows whose SourceHostName column isn't one of those names and write the result back to a file:
Import-Csv $file1 | Where-Object {
$exclude -notcontains $_.SourceHostName
} | Export-Csv 'C:\output.csv' -NoType

Replace One CSV Field With Another Through PowerShell

In PowerShell I'm importing a CSV SAMTemp2 which will have a field called SO. Sometimes SO will be populated with "NW" and in these cases I just want to pull the field called ProdProj from the same line and replace the data in SO with the data in ProdProj then export it the data in that condition.
$RepNW = Import-Csv $SAMTemp2
foreach($d in $data){
If($d.SO -eq "NW"){($d.SO).Replace($d.ProdProj)}}
$RepNW | Export-Csv $SAMTemp -NoTypeInformation
I don't get an error, but this doesn't seem to do anything, either. Can anyone assist me, please?
Update
Per Matt below, I tried:
$RepNW = Import-Csv $SAMTemp2
foreach($d in $RepNW){
If($d.SO -eq "NW"){$d.SO = ($d.SO).Replace($d.ProdProj)}}
$RepNW | Export-Csv $SAMTemp -NoTypeInformation
But I'm not seeing any change. Any assistance is appreciated.
As LotPings pointed out in this line foreach($d in $data){, you haven't defined $data and it seems that you mean it to be foreach($d in $RepNW){
Secondly, rather than using Replace() you can just set one property to be equal to the other.
Last, this probably easiest to do all in the pipeline with ForEach-Object
Import-Csv $SAMTemp2 | ForEach-Object {
If($_.SO -eq "NW"){
$_.SO = $_.ProdProj
}
$_
} | Export-Csv $SAMTemp -NoTypeInformation

Powershell Searching within a text file and exporting horizontally to CSV

I have a big file which content somme blocks of text data like this.
[SERVER1]
LIBELLE=DATA, SOMME DATA
VARIABLES=A,B,C,D,E
PHYSICAL NAME=E:\SOMME\PATH\FILE.INI
ARTICLE SIZE=50
MACHINE=SOME SERVER
PARAMETER =
OPTION =
[SERVER2]
LIBELLE=DATA2, SOMME DATA2
VARIABLES=A,B,C,D,E
PHYSICAL NAME=Z:\SOMME\PATH\FILE2.INI
ARTICLE SIZE=150
MACHINE=SOME SERVER XY
PARAMETER =
OPTION 1 = VOID
OPTION 2 =
OPTION 3 =
OPTION 4 =
OPTION 5 =
What i would like to do is retrieving every block [SERVERX] and put it into a CSV file in this format (horizontally)
ColumnA__ | ColumnB (LIBELLE)___ | ColumnC (VARIABLES) | ColumnD ect...
[SERVER1] | DATA1, SOMME DATA1 | A,B,C,D,E___________ | Ect...
[SERVER2] | DATA2, SOMME DATA2 | A,B,C,D,E___________ | Ect...
I've tried this, the output work as i want but it need to be automated and exported to scv, which doesn't work for me.
$mydata = Get-Content my_file.txt
write-Host $mydata[0] $mydata[1] $mydata[2] $mydata[3] $mydata[4] $mydata[5] | Export-Csv -Path $rep\results.csv -Force -UseCulture -NoTypeInformation
Tried also somthing with select-string but i dont know if this is the right way to do my job..
select-String -path $my_file -Pattern '\[*\]', 'IDENTIFIANTS=','LIBELLE=','VARIABLES=' | Select-Object -Property LineNumber, Line | Export-Csv -Path $rep\results.csv -Force -UseCulture -NoTypeInformation
Thanks for your advices.
So, I would read the whole file in as a multi-line string using the -Raw switch for Get-Content. Then split the file up based on the [ character to denote records. The get the properties from the ConvertFrom-StringData cmdlet (have to prepend "SERVER=" to each record), and make an object from it. Then we find out what all properties any given record can have, make sure to add them all to the first record if it doesn't have it (this is done because when you export to CSV it bases the columns off of the first entries' property list). Then you can export a CSV.
$Data = (Get-Content my_file.txt -Raw) -split "(\[[^[]+)" | ?{![string]::IsNullOrWhiteSpace($_)}
$Records = $Data -replace '\\','\\'|%{$Record="SERVER="+$_.trim()|ConvertFrom-StringData;New-Object PSObject -Prop $Record}
$Props = $Records|%{$_.psobject.properties.name}|select -Unique
$Props | Where{$_ -notin $Records[0].PSObject.Properties.Name}|%{Add-Member -InputObject $Records[0] -NotepropertyName $_ -NotepropertyValue $Null}
$Records|Export-CSV .\my_file.csv -notype
Edit: For those of you out there running PowerShell 2.0 (3 versions out of date at this point in time), you can't use the -Raw parameter. Here's the alternative:
$Data = (Get-Content my_file.txt) -Join "`r`n" -split "(\[[^[]+)" | ?{![string]::IsNullOrWhiteSpace($_)}
Alternative: Thanks #Matt for the suggestion, it is always good to have a different point of view on these things. As Matt suggested, you can use Out-String to combine the array of strings that Get-Content generates, and end up with a single multi-line string. Here's the usage!
$Data = (GC my_file.txt | Out-String) -split "(\[[^[]+)" | ?{![string]::IsNullOrWhiteSpace($_)}

Append existing column in csv by matching values with array and with condition

I will do my best to break this down as simply as I can.
what I have so far that is working:
Currently I have two csv files...
test1.csv
test1ColumnN,test1ColumnI,test1ColumnD,selectDomainOne,selectDomainTwo,selectDomainThree
asdf,asdf,asdf,,,
nValue1,iValue1,dValue1,sValue1,,
qwer,asdf,zxcv,,,
nValue2,iValue2,dValue2,,,
qwer,zxcv,asdf,lkjh,,
nValue3,iValue3,dValue3,sValue3,,
zxcv,qwer,asdf,,poiu,
nValue1,iValue1,dValue1,,sValue1,
nValue4,iValue4,dValue4,,sValue4,
asdf,qwer,zxcv,fghj,mnbv,
nValue5,iValue5,dValue5,,,
asdf,cvbn,erty,,,uytr
nValue7,iValue7,dValue7,,,sValue7
nValue8,iValue8,dValue8,,,sValue8
nValue9,iValue9,dValue9,,,sValue9
qwer,asdf,zxcv,poiu,lkjh,mnbv
test2.csv
DomainCatagories,test2ColumnS,test2ColumnA,test2ColumnN,test2ColumnI,test2ColumnD
DomainOne,sValue1,aValue1,nValue1,,dValueN
DomainOne,sValue2,aValue2,,iValue2,dValue2
DomainOne,sValue3,aValue2,nValue3,iValue3,dValue3
DomainTwo,sValue1,aValue2,,iValue1,dValueN
DomainTwo,sValue4,aValue1,nValue4,,dValueN
DomainTwo,sValue5,aValue1,nValue5,iValue5,dValue5
DomainThree,sValue7,aValue2,nValue7,iValue7,dValue7
DomainThree,sValue8,aValue1,nValue8,iValue8,dValue8
DomainThree,sValue9,aValue2,nValue9,iValue9,dValue9
Now I want to add a column (inside test2.csv) to match the sValue# from both test1.csv and test2.csv with the condition of ($_.DomainCatagories='DomainOne' from test2.csv) and ($_.selectDomainOne from test1.csv)
To do this, I am using the following code...
#Create Column
$domainNameOne = #{}
$domainNameOne = Import-Csv 'C:\Scripts\Tests\test1.csv' | Where-Object {$_.selectDomainOne} | Select-Object -Expand 'selectDomainOne'
(Import-Csv 'C:\Scripts\Tests\test2.csv') |
Select-Object -Property *, #{n='Test1sValues';e={
if($_.DomainCatagories -eq 'DomainOne'){
if(($domainNameOne -contains $_.test2ColumnS) -and ($_.test2ColumnS)){
$_.test2ColumnS
} Else {
'Not found in test1'
}}}} | Export-Csv "C:\Scripts\Tests\test2-Temp" -NoType
Move-Item "C:\Scripts\Tests\test2-Temp" 'C:\Scripts\Tests\test2.csv' -Force
After the code is run, I get the following test2.csv (isCorrect)...
"DomainCatagories","test2ColumnS","test2ColumnA","test2ColumnN","test2ColumnI","test2ColumnD","Test1sValues"
"DomainOne","sValue1","aValue1","nValue1","","dValueN","sValue1"
"DomainOne","sValue2","aValue2","","iValue2","dValue2","Not found in test1"
"DomainOne","sValue3","aValue2","nValue3","iValue3","dValue3","sValue3"
"DomainTwo","sValue1","aValue2","","iValue1","dValueN",""
"DomainTwo","sValue4","aValue1","nValue4","","dValueN",""
"DomainTwo","sValue5","aValue1","nValue5","iValue5","dValue5",""
"DomainThree","sValue7","aValue2","nValue7","iValue7","dValue7",""
"DomainThree","sValue8","aValue1","nValue8","iValue8","dValue8",""
"DomainThree","sValue9","aValue2","nValue9","iValue9","dValue9",""
What I have that is not working:
Next I run the following code...
#Append Column
$domainNameThree = #{}
$domainNameThree = Import-Csv 'C:\Scripts\Tests\test1.csv' | Where-Object {$_.selectDomainThree} | Select-Object -Expand 'selectDomainThree'
(Import-Csv 'C:\Scripts\Tests\test2.csv') | % {
if($_.DomainCatagories -eq 'DomainThree'){
if(($domainNameThree -contains $_.test2ColumnS) -and ($_.test2ColumnS)){
$_.Test1sValues = $_.test2ColumnS
} Else {
$_.Test1sValues = 'Not found in test1'
}}} | Export-Csv "C:\Scripts\Tests\test2-Temp" -NoType
Move-Item "C:\Scripts\Tests\test2-Temp" 'C:\Scripts\Tests\test2.csv' -Force
Instead of adding the values in the correct rows, it completely blanks out the whole file and saves it as an empty file.
End Goal
What I want the code to produce, is this (notice values filled in on last 3 rows in the last column)...
"DomainCatagories","test2ColumnS","test2ColumnA","test2ColumnN","test2ColumnI","test2ColumnD","Test1sValues"
"DomainOne","sValue1","aValue1","nValue1","","dValueN","sValue1"
"DomainOne","sValue2","aValue2","","iValue2","dValue2","Not found in test1"
"DomainOne","sValue3","aValue2","nValue3","iValue3","dValue3","sValue3"
"DomainTwo","sValue1","aValue2","","iValue1","dValueN",""
"DomainTwo","sValue4","aValue1","nValue4","","dValueN",""
"DomainTwo","sValue5","aValue1","nValue5","iValue5","dValue5",""
"DomainThree","sValue7","aValue2","nValue7","iValue7","dValue7","sValue7"
"DomainThree","sValue8","aValue1","nValue8","iValue8","dValue8","sValue8"
"DomainThree","sValue9","aValue2","nValue9","iValue9","dValue9","sValue9"
What am I doing wrong in that 2nd code snippet?
The example you show from What I have that is not working: is missing a key portion. Export-Csv will take everything piped into it to populate the CSV but you are not providing any.
Problem is that you are not passing anything through the pipe. Merely just updating one property. The simplest thing to do is add $_ after the if statement. Or you could just use a calculated property which you have done before in another one of your questions. The example below from Compare dates with different formats in csv file even uses an if statement.
Import-Csv $csvFile | Select-Object *, #{n='MatchDates';e={ if((([datetime]$_.Date1).Date -eq $_.Date3) -and (([datetime]$_.Date2).Date -eq $_.Date3) -and (([datetime]$_.Date1).Date -eq $_.Date2)){ 'Match Found' }Else{ 'No Match Found' }}} |
Export-Csv "$csvFile-results.csv" -NoTypeInformation -Force