the idea is to import a csv, and then if the value "infohostname" contains a nullorwithespace delete the entire line
Function Last_NAS_Parse {
$Import_IP = Import-Csv -Path "$destination_RAW_NAS\audit_nas_8_$((Get-Date).ToString('yyyy-MM-dd')).txt" -Header #("date","infohostname","Version","SMTP","Value_1","Value_2","Value_3","Value_4","Value_5","Value_6","Value_7","Value_8","Value_9")
$Import_IP | ForEach-Object {
if ( [string]::IsNullOrWhiteSpace($_.infohostname)
}
But don't know how can i delete the line after this is selected, thanks.
IMO you don't need a function just a Where-Object:
$Header = ("date","infohostname","Version","SMTP","Value_1","Value_2","Value_3","Value_4","Value_5","Value_6","Value_7","Value_8","Value_9")
$Import_IP = Import-Csv -Path "$destination_RAW_NAS\audit_nas_8_$((Get-Date).ToString('yyyy-MM-dd')).txt" -Header $Header |
Where-Object {![string]::IsNullOrWhiteSpace($_.infohostname)}
But of course you could wrap that in a function
(but a function without passed parameters and returned values isn't a real function)
Loop over the results and store only objects where that boolean evaluation is true, and create a new file. In order to delete the row of the existing file, I believe you'd have to convert it to XLS/X and access it as a COM object
$results = #()
Function Last_NAS_Parse {
$Import_IP = Import-Csv -Path C:\path\to\file.csv
$Import_IP | ForEach-Object {
if ( [string]::IsNullOrWhiteSpace($_.infohostname))
{
Out-Null
}
else
{
$results += $_
}
}
}
Last_NAS_Parse
$results | Export-CSV "C:\export\path\of\newfile.csv"
Related
In my CSV file I have "SharePoint Site" column and a few other columns. I'm trying to split the ID from "SharePoint Site" columns and put it to the new column call "SharePoint ID" but not sure how to do it so I'll be really appreciated If I can get any help or suggestion.
$downloadFile = Import-Csv "C:\AuditLogSearch\New folder\Modified-Audit-Log-Records.csv"
(($downloadFile -split "/") -split "_") | Select-Object -Index 5
CSV file
SharePoint Site
Include:[https://companyname-my.sharepoint.com/personal/elksn7_nam_corp_kl_com]
Include:[https://companyname-my.sharepoint.com/personal/tzksn_nam_corp_kl_com]
Include:[https://companyname.sharepoint.com/sites/msteams_c578f2/Shared%20Documents/Forms/AllItems.aspx?id=%2Fsites%2Fmsteams%5Fc578f2%2FShared%20Documents%2FBittner%2DWilfong%20%2D%20Litigation%20Hold%2FWork%20History&viewid=b3e993a1%2De0dc%2D4d33%2D8220%2D5dd778853184]
Include:[https://companyname.sharepoint.com/sites/msteams_c578f2/Shared%20Documents/Forms/AllItems.aspx?id=%2Fsites%2Fmsteams%5Fc578f2%2FShared%20Documents%2FBittner%2DWilfong%20%2D%20Litigation%20Hold%2FWork%20History&viewid=b3e993a1%2De0dc%2D4d33%2D8220%2D5dd778853184]
Include:[All]
After spliting this will show it under new Column call "SharePoint ID"
SharePoint ID
2. elksn
3. tzksn
4. msteams_c578f2
5. msteams_c578f2
6. All
Try this:
# Import csv into an array
$Sites = (Import-Csv C:\temp\Modified-Audit-Log-Records.csv).'SharePoint Site'
# Create Export variable
$Export = #()
# ForEach loop that goes through the SharePoint sites one at a time
ForEach($Site in $Sites){
# Clean up the input to leave only the hyperlink
$Site = $Site.replace('Include:[','')
$Site = $Site.replace(']','')
# Split the hyperlink at the fifth slash (Split uses binary, so 0 would be the first slash)
$SiteID = $Site.split('/')[4]
# The 'SharePoint Site' Include:[All] entry will be empty after doing the split, because it has no 4th slash.
# This If statement will detect if the $Site is 'All' and set the $SiteID as that.
if($Site -eq 'All'){
$SiteID = $Site
}
# Create variable to export Site ID
$SiteExport = #()
$SiteExport = [pscustomobject]#{
'SharePoint ID' = $SiteID
}
# Add each SiteExport to the Export array
$Export += $SiteExport
}
# Write out the export
$Export
A concise solution that appends a Sharepoint ID column to the existing columns by way of a calculated property:
Import-Csv 'C:\AuditLogSearch\New folder\Modified-Audit-Log-Records.csv' |
Select-Object *, #{
Name = 'SharePoint ID'
Expression = {
$tokens = $_.'SharePoint Site' -split '[][/]'
if ($tokens.Count -eq 3) { $tokens[1] } # matches 'Include:[All]'
else { $tokens[5] -replace '_nam_corp_kl_com$' }
}
}
Note:
To see all resulting column values, pipe the above to Format-List.
To re-export the results to a CSV file, pipe to Export-Csv
You have 3 distinct patterns you are trying to extract data from. I believe regex would be an appropriate tool.
If you are wanting the new csv to just have the single ID column.
$file = "C:\AuditLogSearch\New folder\Modified-Audit-Log-Records.csv"
$IdList = switch -Regex -File ($file){
'Include:.+(?=/(\w+?)_)(?<=personal)' {$matches.1}
'Include:(?=\[(\w+)\])' {$matches.1}
'Include:.+(?=/(\w+?)/)(?<=sites)' {$matches.1}
}
$IdList |
ConvertFrom-Csv -Header "Sharepoint ID" |
Export-Csv -Path $newfile -NoTypeInformation
If you want to add a column to your existing CSV
$file = "C:\AuditLogSearch\New folder\Modified-Audit-Log-Records.csv"
$properties = ‘*’,#{
Name = 'Sharepoint ID'
Expression = {
switch -Regex ($_.'sharepoint Site'){
'Include:.+(?=/(\w+?)_)(?<=personal)' {$matches.1}
'Include:(?=\[(\w+)\])' {$matches.1}
'Include:.+(?=/(\w+?)/)(?<=sites)' {$matches.1}
}
}
}
Import-Csv -Path $file |
Select-Object $properties |
Export-Csv -Path $newfile -NoTypeInformation
Regex details
.+ Match any amount of any character
(?=...) Positive look ahead
(...) Capture group
\w+ Match one or more word characters
? Lazy quantifier
(?<=...) Positive look behind
This would require more testing to see if it works well, but with the input we have it works, the main concept is to use System.Uri to parse the strings. From what I'm seeing, the segment you are looking for is always the third one [2] and depending on the previous segments, perform a split on _ or trim the trailing / or leave the string as is if IsAbsoluteUri is $false.
$csv = Import-Csv path/to/test.csv
$result = foreach($line in $csv)
{
$uri = [uri]($line.'SharePoint Site' -replace '^Include:\[|]$')
$id = switch($uri)
{
{-not $_.IsAbsoluteUri} {
$_
break
}
{ $_.Segments[1] -eq 'personal/' } {
$_.Segments[2].Split('_')[0]
break
}
{ $_.Segments[1] -eq 'sites/' } {
$_.Segments[2].TrimEnd('/')
}
}
[pscustomobject]#{
'SharePoint Site' = $line.'SharePoint Site'
'SharePoint ID' = $id
}
}
$result | Format-List
I am new to PowerShell or any scripting stuff.
I have here a DataResources(CSV File) which contains of bunch of data's that needs to be inserted into a Results.CSV
Here is my DataResources
For my Result.csv looks like this.
My goal is to export a NEW csv file based from a results.csv as a template
and if a HostName/Host match the data contains a value equivalent to UA and PWD will be inserted/updated from ExportResources CSV file to Exported new CSV File
after executing scripts that I have created it only modifies 1 row not all.
This is what I've done so far.
$DataSource = Import-Csv -Path D:\coding\Powershell\csv\Resources\ExportResources.csv
$DataResults = Import-Csv -Path D:\coding\Powershell\csv\Result\results.csv
foreach ($ItemDataResults in $DataResults)
{
$HostName = $ItemDataResults.HostName
$UA = $ItemDataResults.UA
$PASSWD = $ItemDataResults.PWD
}
$ItemDataSource = $DataSource | ? {$_.Host -eq $HostName}
if ($UA -eq "" -and $PASSWD -eq "")
{
$ItemDataResults.UA = $ItemDataSource.UA
$ItemDataResults.PWD = $ItemDataSource.Passwd
}
$DataResults | Export-Csv D:\coding\Powershell\csv\new.csv -NoTypeInformation
The result almost meet my expectation but the problem here that it only fills one hostname the rest are empty.
The issue with your script is that you loop through $DataResults once, but you have to iterate through it once for each item in $DataSource if you use the method that you're employing. I believe that a better method is to create a hashtable from one CSV, using the host name as the key, and the whole object as the value, then loop through the second array updating the value for each host. That would look something like this:
$DataSource = Import-Csv -Path D:\coding\Powershell\csv\Resources\ExportResources.csv
$DataResults = Import-Csv -Path D:\coding\Powershell\csv\Result\results.csv
$DataHT = #{}
$DataResults | ForEach-Object { $DataHT.Add($_.HostName,$_) }
ForEach( $Record in $DataSource ){
$DataHT[$Record.Host].UA = $Record.UA
$ItemDataResults.PWD = $Record.Passwd
}
$DataHT.Values | Export-Csv D:\coding\Powershell\csv\new.csv -NoTypeInformation
I'm receiving an automated report from a system that cannot be modified as a CSV. I am using PowerShell to split the CSV into multiple files and parse out the specific data needed. The CSV contains columns that may contain no data, 1 value, or multiple values that are comma separated within the CSV file itself.
Example(UPDATED FOR CLARITY):
"Group","Members"
"Event","362403"
"Risk","324542, 340668, 292196"
"Approval","AA-334454, 344366, 323570, 322827, 360225, 358850, 345935"
"ITS","345935, 358850"
"Services",""
I want the data to have one entry per line like this (UPDATED FOR CLARITY):
"Group","Members"
"Event","362403"
"Risk","324542"
"Risk","340668"
"Risk","292196"
#etc.
I've tried splitting the data and I just get an unknown number of columns at the end.
I tried a foreach loop, but can't seem to get it right (pseudocode below):
Import-CSV $Groups
ForEach ($line in $Groups){
If($_.'Members'.count -gt 1, add-content "$_.Group,$_.Members[2]",)}
I appreciate any help you can provide. I've searched all the stackexchange posts and used Google but haven't been able to find something that addresses this exact issue.
Import-Csv .\input.csv | ForEach-Object {
ForEach ($Member in ($_.Members -Split ',')) {
[PSCustomObject]#{Group = $_.Group; Member = $Member.Trim()}
}
} | Export-Csv .\output.csv -NoTypeInformation
# Get the raw text contents
$CsvContents = Get-Content "\path\to\file.csv"
# Convert it to a table object
$CsvData = ConvertFrom-CSV -InputObject $CsvContents
# Iterate through the records in the table
ForEach ($Record in $CsvData) {
# Create array from the members values at commas & trim whitespace
$Record.Members -Split "," | % {
$MemberCount = $_.Trim()
# Check if the count is greater than 1
if($MemberCount -gt 1) {
# Create our output string
$OutputString = "$($Record.Group), $MemberCount"
# Write our output string to a file
Add-Content -Path "\path\to\output.txt" -Value $OutputString
}
}
}
This should work, you had the right idea but I think you may have been encountering some syntax issues. Let me know if you have questions :)
Revised the code as per your updated question,
$List = Import-Csv "\path\to\input.csv"
foreach ($row in $List) {
$Group = $row.Group
$Members = $row.Members -split ","
# Process for each value in Members
foreach ($MemberValue in $Members) {
# PS v3 and above
$Group + "," + $MemberValue | Export-Csv "\path\to\output.csv" -NoTypeInformation -Append
# PS v2
# $Group + "," + $MemberValue | Out-File "\path\to\output.csv" -Append
}
}
I have a lot of duplicate code I would like to clean up. I know the following syntax is not correct, but can you let me know the right way? Lets say I have the following code...
Function GetColumnOneValues
{
Param ($csvFile, $ValuesWithoutNullFile)
Import-Csv $csvFile | Where-Object {$_.test1ColumnOne} |
Export-Csv $ValuesWithoutNullFile -NoTypeInformation -Force
}
Function GetColumnTwoValues
{
Param ($csvFile, $ValuesWithoutNullFile)
Import-Csv $csvFile | Where-Object {$_.test2ColumnTwo} |
Export-Csv $ValuesWithoutNullFile -NoTypeInformation -Force
}
Function GetColumnThreeValues
{
Param ($csvFile, $ValuesWithoutNullFile)
Import-Csv $csvFile | Where-Object {$_.test3ColumnThree} |
Export-Csv $ValuesWithoutNullFile -NoTypeInformation -Force
}
^Notice the duplicate code
Function Main
{
$test1CsvFile = "C:\Scripts\Tests\test1.csv"
$test1CsvFileResults = "C:\Scripts\Tests\test1Results.csv"
$test2CsvFile = "C:\Scripts\Tests\test2.csv"
$test2CsvFileResults = "C:\Scripts\Tests\test2Results.csv"
$test3CsvFile = "C:\Scripts\Tests\test3.csv"
$test3CsvFileResults = "C:\Scripts\Tests\test3Results.csv"
GetColumnOneValues $test1CsvFile $test1CsvFileResults
GetColumnTwoValues $test2CsvFile $test2CsvFileResults
GetColumnThreeValues $test3CsvFile $test3CsvFileResults
}
Main
Instead, it should be something like this...
Function GetColumnValues
{
Param ($csvFile, $ValuesWithoutNullFile, $ColumnName)
Import-Csv $csvFile | Where-Object {$ColumnName} |
Export-Csv $ValuesWithoutNullFile -NoTypeInformation -Force
}
Function Main
{
$test1CsvFile = "C:\Scripts\Tests\test1.csv"
$test1CsvFileResults = "C:\Scripts\Tests\test1Results.csv"
$test2CsvFile = "C:\Scripts\Tests\test2.csv"
$test2CsvFileResults = "C:\Scripts\Tests\test2Results.csv"
$test3CsvFile = "C:\Scripts\Tests\test3.csv"
$test3CsvFileResults = "C:\Scripts\Tests\test3Results.csv"
$column1 = $_.test1ColumnOne
$column2 = $_.test2ColumnTwo
$column3 = $_.test3ColumnThree
GetColumnValues $test1CsvFile $test1CsvFileResults $column1
GetColumnValues $test2CsvFile $test2CsvFileResults $column2
GetColumnValues $test3CsvFile $test3CsvFileResults $column3
}
Main
However, instead of the function printing out the non-null value, it prints a blank csv file.
Use these csv Files to test the code...
csvFile1...
test1ColumnOne,test1ColumnTwo,test1ColumnThree
qwer,,
,qwer,
,,qwer
csvFile2...
test2ColumnOne,test2ColumnTwo,test2ColumnThree
asdf,,
,asdf,
,,asdf
csvFile3...
test3ColumnOne,test3ColumnTwo,test3ColumnThree
zxcv,,
,zxcv,
,,zxcv
Results should be...
test1Results.csv...
"test1ColumnOne","test1ColumnTwo","test1ColumnThree"
"qwer","",""
test2Results.csv...
"test2ColumnOne","test2ColumnTwo","test2ColumnThree"
"","asdf",""
test3Results.csv...
"test3ColumnOne","test3ColumnTwo","test3ColumnThree"
"","","zxcv"
If you want to clear empty values from a CSV column, retaining a list of non-empty values of that column, I'd do it with a generic function like this:
function Remove-EmptyValues {
Param(
[Parameter(
Mandatory=$true,
Position=0,
ValueFromPipeline=$true,
ValueFromPipelineByPropertyName=$true
)]
[object[]]$Csv,
[Parameter(Mandatory=$true, Position=1)]
[string]$ColumnTitle
)
Process {
$Csv | Select-Object -Expand $ColumnTitle | Where-Object { $_ }
}
}
so it can be used like this
$csv = Import-Csv 'C:\testScripts\test.csv'
$noEmpty = Remove-EmptyValues $csv 'column A'
or like this:
$csv = Import-Csv 'C:\testScripts\test.csv'
$noEmpty = $csv | Remove-EmptyValues -ColumnTitle 'column A'
Edit: If you want to remove all lines where the value in a particular column is empty, pass input and output filename as well as the name of the column as string parameters:
function Remove-EmptyRecords {
Param(
[Parameter(Mandatory=$true, Position=0)]
[string]$Source,
[Parameter(Mandatory=$true, Position=1)]
[string]$Destination,
[Parameter(Mandatory=$true, Position=2)]
[string]$ColumnTitle
)
Import-Csv $Source |
Where-Object { $_.$ColumnTitle } |
Export-Csv $Destination -NoType
}
Remove-EmptyRecords test1.csv test1Results.csv test1ColumnOne
Remove-EmptyRecords test2.csv test2Results.csv test2ColumnTwo
Remove-EmptyRecords test3.csv test3Results.csv test3ColumnThree
Not sure exactly what you are getting at hear but you GetColumnValues so lets build a function with that goal. You have a columnName argument. Should add a -path one as well.
Function Get-ColumnValues{
Param ($Path, $ColumnName)
$csvData = Import-Csv $Path
If($ColumnName -in ($csvData | Get-Member -MemberType "NoteProperty").Name){
($csvData).$ColumnName | Where-Object{![string]::IsNullOrEmpty($_)}
} Else {
Throw "There is no column present called $ColumnName in the file $Path"
}
}
There is more error prevention that could be included here but simply put this will import a csv and return the values of the requested column assuming it exists.
As an aside the name of your function is misleading if you are exporting the results. You are not only getting them but setting them as well.
I have a csv file in the form:
Address,L0,L1,L2,L3,L4
01,Species,,,,
01.01,,Mammals,,,
01.01.01,,,Threatened,,
...
I want to use it to create a matching directory structure. I'm new to scripting and PowerShell, and in this case I'm not sure if I'm on totally the wrong track. Should I use a separate array to store each level's Name/Address pairs and then use those arrays like a lookup table to build the path? If so, I guess I'm stuck on how to set up if-thens based on a row's Address. This is as far as I've got so suggestions on general strategy or links to similar kinds of problem would be really welcome:
$folders = Import-Csv "E:\path\to\file.csv"
$folders | foreach {
$row = new-object PSObject -Property #{
Address = $_.Address;
Level = ([regex]::Matches($_.Address, "\." )).count;
L0 = $_.L0
L1 = $_.L1
L2 = $_.L2
L3 = $_.L3
}
$array += $row
}
#top level directories
$0 = $array | ?{$_.Level -eq 0} |
Select-Object #{n="Address";e={$_.Address;}},#{n="Name";e={$_.L0}}
#2nd level directories
$1 = $array | ?{$_.Level -eq 1} |
Select-Object #{n="Number";e={$_.Address.split(".")[-1];}},#{n="Name";e={$_.L1}}
Not tested, but I think this might do what you want:
$root = 'E:\SomeDirName'
Switch -Regex (Get-Content "E:\path\to\file.csv")
{
'^01,(\w+),,,,$' { $L1,$L2,$L3,$L4 = $null; $L0=$matches[1];mkdir "$root\$L0" }
'^01\.01,,(\w+),,,$' { $L1=$matches[1];mkdir "$root\$L0\$L1" }
'^01\.01\.01,,,(\w+),,$' { $L2=$matches[1];mkdir "$root\$L0\$L1\$L2" }
'^01\.01\.01\.01,,,,(\w+),$' { $L3=$matches[1];mkdir "$root\$L0\$L1\$L2\$L3" }
'^01\.01\.01\.01\.01,,,,,(\w+)$' { $L4=$matches[1];mkdir "$root\$L0\$L1\$L2\$L3\$L4" }
}
To solve that kind of problem a programming concept called recursion is often used.
In short a recursive function is a function that call itself.
I successfully tested this code with you CSV input:
$csvPath = 'C:\Temp\test.csv'
$folderRoot = 'C:\Temp'
$csv = Import-Csv $csvPath -Delimiter ',' | Sort-Object -Property Address
# Recursive function
function Recurse-Folder( $folderAddress, $basePath )
{
# Getting the list of current folder subfolders
$childFolders = $null
if( $folderAddress -like '' )
{
$childFolders = $csv | Where-Object { $_.Address -like '??' }
}
else
{
$childFolders = $csv | Where-Object { $_.Address -like $( $folderAddress + '.??' ) }
}
# For each child folder
foreach( $childFolder in $childFolders )
{
# Get the folder name
$dotCount = $childFolder.Address.Split('.').Count - 1
$childFolderName = $childFolder.$('L'+$dotCount)
if( $childFolderName -ne '')
{
$childFolderPath = $basePath + '\' + $childFolderName
# Creating child folder and calling recursive function for it
New-Item -Path $childFolderPath -ItemType Directory
Recurse-Folder $childFolder.Address $childFolderPath
}
}
}
Recurse-Folder '' $folderRoot