How to pass csv column object through function arg - powershell

I have a lot of duplicate code I would like to clean up. I know the following syntax is not correct, but can you let me know the right way? Lets say I have the following code...
Function GetColumnOneValues
{
Param ($csvFile, $ValuesWithoutNullFile)
Import-Csv $csvFile | Where-Object {$_.test1ColumnOne} |
Export-Csv $ValuesWithoutNullFile -NoTypeInformation -Force
}
Function GetColumnTwoValues
{
Param ($csvFile, $ValuesWithoutNullFile)
Import-Csv $csvFile | Where-Object {$_.test2ColumnTwo} |
Export-Csv $ValuesWithoutNullFile -NoTypeInformation -Force
}
Function GetColumnThreeValues
{
Param ($csvFile, $ValuesWithoutNullFile)
Import-Csv $csvFile | Where-Object {$_.test3ColumnThree} |
Export-Csv $ValuesWithoutNullFile -NoTypeInformation -Force
}
^Notice the duplicate code
Function Main
{
$test1CsvFile = "C:\Scripts\Tests\test1.csv"
$test1CsvFileResults = "C:\Scripts\Tests\test1Results.csv"
$test2CsvFile = "C:\Scripts\Tests\test2.csv"
$test2CsvFileResults = "C:\Scripts\Tests\test2Results.csv"
$test3CsvFile = "C:\Scripts\Tests\test3.csv"
$test3CsvFileResults = "C:\Scripts\Tests\test3Results.csv"
GetColumnOneValues $test1CsvFile $test1CsvFileResults
GetColumnTwoValues $test2CsvFile $test2CsvFileResults
GetColumnThreeValues $test3CsvFile $test3CsvFileResults
}
Main
Instead, it should be something like this...
Function GetColumnValues
{
Param ($csvFile, $ValuesWithoutNullFile, $ColumnName)
Import-Csv $csvFile | Where-Object {$ColumnName} |
Export-Csv $ValuesWithoutNullFile -NoTypeInformation -Force
}
Function Main
{
$test1CsvFile = "C:\Scripts\Tests\test1.csv"
$test1CsvFileResults = "C:\Scripts\Tests\test1Results.csv"
$test2CsvFile = "C:\Scripts\Tests\test2.csv"
$test2CsvFileResults = "C:\Scripts\Tests\test2Results.csv"
$test3CsvFile = "C:\Scripts\Tests\test3.csv"
$test3CsvFileResults = "C:\Scripts\Tests\test3Results.csv"
$column1 = $_.test1ColumnOne
$column2 = $_.test2ColumnTwo
$column3 = $_.test3ColumnThree
GetColumnValues $test1CsvFile $test1CsvFileResults $column1
GetColumnValues $test2CsvFile $test2CsvFileResults $column2
GetColumnValues $test3CsvFile $test3CsvFileResults $column3
}
Main
However, instead of the function printing out the non-null value, it prints a blank csv file.
Use these csv Files to test the code...
csvFile1...
test1ColumnOne,test1ColumnTwo,test1ColumnThree
qwer,,
,qwer,
,,qwer
csvFile2...
test2ColumnOne,test2ColumnTwo,test2ColumnThree
asdf,,
,asdf,
,,asdf
csvFile3...
test3ColumnOne,test3ColumnTwo,test3ColumnThree
zxcv,,
,zxcv,
,,zxcv
Results should be...
test1Results.csv...
"test1ColumnOne","test1ColumnTwo","test1ColumnThree"
"qwer","",""
test2Results.csv...
"test2ColumnOne","test2ColumnTwo","test2ColumnThree"
"","asdf",""
test3Results.csv...
"test3ColumnOne","test3ColumnTwo","test3ColumnThree"
"","","zxcv"

If you want to clear empty values from a CSV column, retaining a list of non-empty values of that column, I'd do it with a generic function like this:
function Remove-EmptyValues {
Param(
[Parameter(
Mandatory=$true,
Position=0,
ValueFromPipeline=$true,
ValueFromPipelineByPropertyName=$true
)]
[object[]]$Csv,
[Parameter(Mandatory=$true, Position=1)]
[string]$ColumnTitle
)
Process {
$Csv | Select-Object -Expand $ColumnTitle | Where-Object { $_ }
}
}
so it can be used like this
$csv = Import-Csv 'C:\testScripts\test.csv'
$noEmpty = Remove-EmptyValues $csv 'column A'
or like this:
$csv = Import-Csv 'C:\testScripts\test.csv'
$noEmpty = $csv | Remove-EmptyValues -ColumnTitle 'column A'
Edit: If you want to remove all lines where the value in a particular column is empty, pass input and output filename as well as the name of the column as string parameters:
function Remove-EmptyRecords {
Param(
[Parameter(Mandatory=$true, Position=0)]
[string]$Source,
[Parameter(Mandatory=$true, Position=1)]
[string]$Destination,
[Parameter(Mandatory=$true, Position=2)]
[string]$ColumnTitle
)
Import-Csv $Source |
Where-Object { $_.$ColumnTitle } |
Export-Csv $Destination -NoType
}
Remove-EmptyRecords test1.csv test1Results.csv test1ColumnOne
Remove-EmptyRecords test2.csv test2Results.csv test2ColumnTwo
Remove-EmptyRecords test3.csv test3Results.csv test3ColumnThree

Not sure exactly what you are getting at hear but you GetColumnValues so lets build a function with that goal. You have a columnName argument. Should add a -path one as well.
Function Get-ColumnValues{
Param ($Path, $ColumnName)
$csvData = Import-Csv $Path
If($ColumnName -in ($csvData | Get-Member -MemberType "NoteProperty").Name){
($csvData).$ColumnName | Where-Object{![string]::IsNullOrEmpty($_)}
} Else {
Throw "There is no column present called $ColumnName in the file $Path"
}
}
There is more error prevention that could be included here but simply put this will import a csv and return the values of the requested column assuming it exists.
As an aside the name of your function is misleading if you are exporting the results. You are not only getting them but setting them as well.

Related

Powershell ConvertFrom-StringData inverse operation ConvertTo-StringData [duplicate]

This question already has an answer here:
Convert hashtable back to string data in efficient way
(1 answer)
Closed 2 years ago.
I have found following code for reading hashtable from a text file with key=value pairs.
$value = Get-Content $Path | Out-String | ConvertFrom-StringData
I have not found the inverse operation though how to write the hashtable back to the text file in the same format. I use the following code
$values.GetEnumerator() | ForEach-Object { "{0}={1}" -f $_.Name,$_.Value } | Set-Content $Path
It works fine. I just wonder if there is some more straight forward solution.
Not that I know. This has been asked before.
Your solution looks fine. You could turn your code into a script cmdlet:
function ConvertTo-StringData {
[CmdletBinding()]
param(
[Parameter(Mandatory, Position = 0, ValueFromPipeline)]
[HashTable[]]$HashTable
)
process {
foreach ($item in $HashTable) {
foreach ($entry in $item.GetEnumerator()) {
"{0}={1}" -f $entry.Key, $entry.Value
}
}
}
}
Example:
$hashtable = Get-Content $path | ConvertFrom-StringData
$hashtable | ConvertTo-StringData | Set-Content $path
# or
ConvertTo-StringData $hashtable | Set-Content $path

Function from imported script doesn't execute

i write a script with a function.
here is the script with the function:
function GenerateHashesForProjects(){
[array]$result=#()
Write-Output "Genrate Hash Values"
$dependencyFolder = Join-Path -Path $PSScriptRoot -ChildPath "..\..\Sources\_Dependencies"
#get all folder in a list below the dependency folder expect the "Modules" folder
$dependencyContent = Get-ChildItem -Path $dependencyFolder | where {$_.PSIsContainer -and ($_.Name -notlike "*Modules*")}
#Fill the result array with the project file name and the depending hash value of this file
foreach ($item in $dependencyContent) {
$denpencyProjects = Get-ChildItem -Path $item.Fullname | where { ($_ -like "*.csproj") }
$hashValue = (Get-FileHash $denpencyProjects.FullName -Algorithm MD5).Hash
$name = $denpencyProjects.Name
Write-Output "name: $name `nvalue: $hashValue"
$result += #($denpencyProjects.Name, $hashValue)
}
return $result
}
That script works fine.
Now i want to use this function also in another script. So i import the script and define a variable with that function. Here is the issue if a call the function without the variable it works fine but with the variable definition not, why?
Here is the second script with the import:
. Join-Path -Path $PSScriptroot -ChildPath "..\..\Build\Tools\GenerateHashesForProjects.ps1"
[array]$dependencyFileValues = GenerateHashesForProjects
This test works fine:
. Join-Path -Path $PSScriptroot -ChildPath "..\..\Build\Tools\GenerateHashesForProjects.ps1"
GenerateHashesForProjects
since you didn't post any responses to questions [grin], here is one way to rewrite your code.
what it does ...
creates an advanced function
uses the recommended name format for such
does not supply the "otta be there" Comment Based Help [grin]
defines the parameters
only the $Path is required.
defines but does not use a begin {} block
defines a process {} block
grabs a list of the dirs that branch from the source path
filters out the dirs that are in the $ExcludeDirList
gets the files in those dirs that match the $FileFilter
iterates thru that list
builds a [PSCustomObject] for each file with the desired details
you can add or remove them as needed.
sends that PSCO out to the calling code
the line that calls the function stores the entire set of results into the $Result variable and then shows that on screen.
a few notes ...
i had to change a lot of your details since i have no csproj files
there are no "what is happening" lines
if you need that, you can easily add such. i would NOT use Write-Output, tho, since that will pollute your output data.
there is no error detection OR error handling
here's the code ...
function Get-ProjectFileHash
{
<#
CommentBasedHelp goes here
#>
[CmdletBinding ()]
Param
(
[Parameter (
Mandatory,
Position = 0
)]
[string]
$Path,
[Parameter (
Position = 1
)]
[ValidateSet (
'MD5',
'MACTripleDES',
'RIPEMD160',
'SHA1',
'SHA256',
'SHA384',
'SHA512'
)]
[string]
$Algorithm = 'MD5',
[Parameter (
Position = 2
)]
[string[]]
$ExcludeDirList,
[Parameter (
Position = 3
)]
[string]
$FileFilter
)
begin {}
process
{
$ProjDirList = Get-ChildItem -LiteralPath $Path -Directory |
Where-Object {
# the "-Exclude" parameter of G-CI is wildly unreliable
# this avoids that problem [*grin*]
# build a regex OR listing to exclude
$_.Name -notmatch ($ExcludeDirList -join '|')
}
$FileList = Get-ChildItem -LiteralPath $ProjDirList.FullName -File -Filter $FileFilter
foreach ($FL_Item in $FileList)
{
[PSCustomObject]#{
FileName = $FL_Item.Name
DirName = $FL_Item.Directory
Algorithm = $Algorithm
Hash = (Get-FileHash -LiteralPath $FL_Item.FullName -Algorithm $Algorithm).Hash
}
}
}
end {}
} # end >>> function Get-ProjectFileHash
$Source = 'C:\ProgramData\chocolatey\lib'
$NotWanted = 'choco', '7zip', 'kb', 'bad', 'bkp'
$Filter = '*.nupkg'
$Result = Get-ProjectFileHash -Path $Source -Algorithm MD5 -ExcludeDirList $NotWanted -FileFilter $Wanted
$Result
truncated output ...
FileName DirName Algorithm Hash
-------- ------- --------- ----
autohotkey.nupkg C:\ProgramData\chocolatey\lib\autohotkey MD5 35A1B894AEA7D3473F3BBCBF5788D2D6
autohotkey.install.nupkg C:\ProgramData\chocolatey\lib\autohotkey.install MD5 EFE8AD812CBF647CFA116513AAD4CC15
autohotkey.portable.nupkg C:\ProgramData\chocolatey\lib\autohotkey.portable MD5 D31FA1B5496AAE266E4B0545835E9B19
[*...snip...*]
vcredist2015.nupkg C:\ProgramData\chocolatey\lib\vcredist2015 MD5 56321731BC0AEFCA3EE5E547A7A25D5E
vlc.nupkg C:\ProgramData\chocolatey\lib\vlc MD5 8177E24675461BDFF33639BF1D89784B
wiztree.nupkg

Powershell function-get multple values

I have file 1.csv
number, name # column name
1,john
2,mike
3,test
4,test2
...
I created function for returning all values from this csv (number,name)
Function Get-CSV {
[CmdletBinding()]
param (
# CSV File path
[Parameter(Mandatory=$true)][string]$path
)
#Create an hashtable variable
[hashtable]$return = #{}
Import-Csv $path |
ForEach-Object {
$number = $_.number
$name = $_.name
$return.name = $name
$return.number = $number
return $return
}
# return $return un-commenting this line don't change output
}
# calling function
$a = Get-CSV "C:\Users\1.csv"
$a.number
$a.name
I get only one (last row from CSV) - a.name = test2 and a.number = 4
How to get all rows from CSV when calling this function ?
You need to construct an array of hashtables for this to work. Even better, I would create an array of objects because it gives you control over the property names. You can change the function definition to:
Function Get-CSV {
[CmdletBinding()]
param (
# CSV File path
[Parameter(Mandatory=$true)][string]$path
)
#Create an empty array variable
$return = #()
Import-Csv $path |
ForEach-Object {
$return += ,(New-Object PSObject -property #{'name'= $_.name; 'number'= $_.number})
}
return $return
}
This gives you:
$a = Get-CSV "C:\Users\1.csv"
$a
name number
---- ------
john 1
mike 2
test 3
test2 4
Note
I'm not sure on your exact use case, but Import-Csv already gives you the information back as an object, so there may not be a need to create a separate one.
After lot of googling and try/errors found solution:
Function Get-CSV {
[CmdletBinding()]
param (
# CSV File path
[Parameter(Mandatory=$true)][string]$path
)
Import-Csv $path |
ForEach-Object {
$number = $_.number
$name = $_.name
[pscustomobject]#{
name = $name
number = $number
}
}
}

Powershell Convert a string to datatable

I have the below string which is returned from an Invoke-RestMethod call. How can I transform the results into an object or data table to be able to use it? The first row would be the properties.
I tried the Out-File and Export-CSV first but the only length is added to the file.
Then I tried the format table which did nothing.
Any ideas?
Name,Main No, Sec No
Alex,34.6743,22.7800
Tom,33.8798,21.9098
Tim,34.6743,41.7800
Mark,33.8798,21.9098
The ConvertFrom-Csv cmdlet is smart enough to exclude the empty lines we see in your example.
To output as CSV, simply use:
$data | ConvertFrom-Csv
#output to csv file:
$data | ConvertFrom-Csv | Export-Csv -Path 'D:\test.csv' -NoTypeInformation
Your CSV file will look like this:
"Name","Main No","Sec No"
"Alex","34.6743","22.7800"
"Tom","33.8798","21.9098 "
"Tim","34.6743","41.7800"
"Mark","33.8798","21.9098"
If you want it converted to datatable, you can use below function:
function ConvertTo-DataTable {
[CmdletBinding()]
Param(
[Parameter(Position=0, Mandatory=$true, ValueFromPipeline = $true)]
[PSObject[]]$InputObject
)
Begin {
$dataTable = New-Object System.Data.DataTable
$first = $true
function _GetSafeTypeName($type) {
# internal helper function to return the correct typename for a datatable
$types = #('System.Boolean', 'System.Byte', 'System.SByte', 'System.Char', 'System.Datetime',
'System.TimeSpan', 'System.Decimal', 'System.Double', 'System.Guid', 'System.Single')
$ints = #('System.Int16', 'System.Int32', 'System.Int64')
$uints = #('System.UInt16', 'System.UInt32', 'System.UInt64')
if ($types -contains $type) { return "$type" }
# if the type is Int or UInt, always return the largest variety
if ($ints -contains $type) { return 'System.Int64' }
if ($uints -contains $type) { return 'System.UInt64' }
return 'System.String'
}
}
Process {
foreach ($object in $InputObject) {
$dataRow = $dataTable.NewRow()
foreach($property in $object.PSObject.Properties) {
# read the data type for this property and make sure it is a valid type for a DataTable
$dataType = _GetSafeTypeName $property.TypeNameOfValue
# ensure the property name does not contain invalid characters
$propertyName = $property.Name -replace '[\W\p{Pc}-[,]]', '_' -replace '_+', '_'
if ($first) {
$dataColumn = New-Object System.Data.DataColumn $propertyName, $dataType
$dataTable.Columns.Add($dataColumn)
}
if ($property.Gettype().IsArray -or ($property.TypeNameOfValue -like '*collection*')) {
$dataRow.Item($propertyName) = $property.Value | ConvertTo-XML -As String -NoTypeInformation -Depth 1
}
else {
$value = if ($null -ne $property.Value) { $property.Value } else { [System.DBNull]::Value }
$dataRow.Item($propertyName) = $value -as $dataType
}
}
$dataTable.Rows.Add($dataRow)
$first = $false
}
}
End {
Write-Output #(,($dataTable))
}
}
Then use it like this:
$data | ConvertFrom-Csv | ConvertTo-DataTable
P.S. $data is the result of your Invoke-RestMethod call.

delete a matched string in a condition, PowerShell

the idea is to import a csv, and then if the value "infohostname" contains a nullorwithespace delete the entire line
Function Last_NAS_Parse {
$Import_IP = Import-Csv -Path "$destination_RAW_NAS\audit_nas_8_$((Get-Date).ToString('yyyy-MM-dd')).txt" -Header #("date","infohostname","Version","SMTP","Value_1","Value_2","Value_3","Value_4","Value_5","Value_6","Value_7","Value_8","Value_9")
$Import_IP | ForEach-Object {
if ( [string]::IsNullOrWhiteSpace($_.infohostname)
}
But don't know how can i delete the line after this is selected, thanks.
IMO you don't need a function just a Where-Object:
$Header = ("date","infohostname","Version","SMTP","Value_1","Value_2","Value_3","Value_4","Value_5","Value_6","Value_7","Value_8","Value_9")
$Import_IP = Import-Csv -Path "$destination_RAW_NAS\audit_nas_8_$((Get-Date).ToString('yyyy-MM-dd')).txt" -Header $Header |
Where-Object {![string]::IsNullOrWhiteSpace($_.infohostname)}
But of course you could wrap that in a function
(but a function without passed parameters and returned values isn't a real function)
Loop over the results and store only objects where that boolean evaluation is true, and create a new file. In order to delete the row of the existing file, I believe you'd have to convert it to XLS/X and access it as a COM object
$results = #()
Function Last_NAS_Parse {
$Import_IP = Import-Csv -Path C:\path\to\file.csv
$Import_IP | ForEach-Object {
if ( [string]::IsNullOrWhiteSpace($_.infohostname))
{
Out-Null
}
else
{
$results += $_
}
}
}
Last_NAS_Parse
$results | Export-CSV "C:\export\path\of\newfile.csv"