I'm trying to loop through a series of txt files extracting information into arrays based a : delimiter.
All values i'm taking from the text files fit on a single line, except for one. (The "ad text" value). This cuts off the information after line 1 in the final output. When I remove line carriages and breaks beforehand, none of the fields are inputted correctly.
How would I specify wanting my array to accept multi-line inputs for the "ad text" field?
Below is the code i'm working with:
$files = ls "*.txt"
$dictionary = #{}
[System.Collections.Generic.List[String]]$list = #()
foreach($f in $files){$in = Get-Content $f
$in.Split([Environment]::NewLine) | ForEach-Object { $key,$value = $_.Split(':')
$dictionary[$key] = $value
}
[void]$list.Add( $dictionary['Ad ID'] + ',' + $dictionary['Ad Text'] + ',' + $dictionary['Ad Landing Page'] + ',' + $dictionary['Ad Targeting Location'] + ',' + $dictionary['Age'] + ',' + $dictionary['Language'] + ',' + $dictionary['Placements'] + ',' + $dictionary['Interests'] + ',' + $dictionary['Behaviors'] + ',' + $dictionary['Ad Impressions'] + ',' + $dictionary['Ad Clicks'] + ',' + $dictionary['Ad Spend'] + ',' + $dictionary['Ad Creation Date'] + ','+ $dictionary['Friends'] + ',' + $dictionary['Ad End Date'] + ',' + $dictionary['Excluded Connections'] + ',' + $dictionary['Image'] + ',' + $dictionary['Ad Targeting Location']+','+$dictionary[‘File Name’] )
}
$list | Out-File -FilePath '.\trial.csv' -Append
Assuming that the additional lines following Ad Text:-prefixed lines do not contain : chars themselves:
# Create sample text file t.txt
#'
Ad ID:10
Ad Text:one
two
Ad Landing Page:http://example.org
'# > t.txt
# Split the lines into field names and values by ":" and
# build a dictionary.
$dict = [ordered] #{}
$i = 0
(Get-Content -Raw t.txt) -split '(?m)^([^\n:]+):' -ne '' | ForEach-Object {
if ($i++ % 2 -eq 0) {
$key = $_
} else {
$dict[$key] = $_
}
}
# Output the dictionary, showing each entry as a list.
$dict | Format-List
The output is as follows, showing that the Ad Text entry comprises two lines:
Name : Ad ID
Value : 10
Name : Ad Landing Page
Value : http://example.org
Name : Ad Text
Value : one
two
Related
I have a PowerShell script where I use the Call operator to run mkvmerge.exe with a set of arguments/options. I wanted to collect the options in a string so that the string could be expanded based on inputs earlier in the script, but when the destination file is in this string mkvmerge does not recognize it for some reason. I am unsure if this is because of variable expansion or what the issue is
The code below illustrates the problem. I have also tried adding the options to an array and using the # operator, but with no luck
$mkvmerge = "C:\Program Files\MKVToolNix\mkvmerge.exe"
$applyoptions = $true
$subtitleformat = "srt"
$isolanguage = "eng"
foreach ($file in Get-ChildItem -File -Include *.mkv) {
<# Make the string with options for mkvmerge #>
$mkvmergeparams = ""
if ($applyoptions -eq $true) {
$mkvmergeinstructions += ('`' + "#options.json")
}
$mkvmergeparams += ("-o " + "`"" + $file.DirectoryName + "\Remux-" + $file.BaseName + ".mkv`" " + "`"" + $file + "`"")
if (-not($subtitleformat -eq "")) {
$mkvmergeinstructions += (" --default-track " + $subtitledefault) + " --language " + "0:" + $isolanguage + " " + "`"" + $file.DirectoryName + "\" + $file.BaseName + "." + $subtitleformat + "`""
}
<# Check the string #>
Write-Host $mkvmergeparams
<# This does not work, but I would like it to #>
& $mkvmerge $mkvmergeparams
<# This works, but would require a separate line for each set of possible options #>
#& $mkvmerge `#options.json -o ($file.DirectoryName + "\" + "Remux-" + $file.BaseName + ".mkv") $file --default-track $subtitledefault --language ("0:" + $isolanguage) ($file.DirectoryName + "\" + $file.BaseName + "." + $subtitleformat)
}
Read-Host -Prompt "Press enter to exit"
In the second example the variables are expanded and mkvmerge recognizes the options given, but this requires having a separate mkvmerge line for each set of possible inputs
When checking the string it seems to expand to look exactly as it should
Found the answer. I had to create an array with the arguments
<# Build array with options for mkvmerge #>
$mkvmergeparams = $null
if ($applyoptions -eq $true) {$mkvmergeparams += #('#options.json')}
$mkvmergeparams += #("-o", ($file.DirectoryName + "\Remux-" + $file.BaseName + ".mkv"), $file)
if (-not($subtitleformat -eq "")) {$mkvmergeparams += #("--default-track", $subtitledefault, "--language", ("0:" + $isolanguage), ($file.DirectoryName + "\" + $file.BaseName + "." + $subtitleformat))}
I want each row to be populated with the data retrieved from each file. Currently, the 2nd and 3rd column entries are being written to a newline.CSV file output I have tried using "export-csv" and the "-nonewline" command. Perhaps there is a regex command that would solve this?
#Column headings
$headings = "Source file, Review file existence, Review Result, Compilation Check Result, Static Analysis Result, Review up-to-date, Reviewed code version, Latest code version"
# Create array with $headings as first input
$output = #($headings)
$SourceParentDir = "C:\Document"
$Files = get-childitem -Path $SourceParentDir -Recurse -Filter '*.?pp' | % { $_.FullName }
foreach ($File in $Files)
{
$BaseName = [System.IO.Path]::GetFileName($File)
# Populate each row for each file
$output += $BaseName
$output += ", Review Exists" # writes to a newline
$output += ", " + $Result + "," + $Compilation + "," + $StaticAnalysis + "," + $UpToDateFlag + "," + $ReviewFileVersionNumber + "," + $SourceFileVersionNumber + ","
}
# write output to a csv file
$output | Out-File -FilePath Documents\Example-csv.csv -encoding utf8
You can do things that way, but there's definitely a more-Powershelley way:
Get-ChildItem -Path $SourceParentDir -Recurse -Filter '*.?pp' |
ForEach-Object {
$File = $_
# Put your other code here
# This will output an object to the stream
[PSCustomObject]#{
'Source file' = $File.Name
'Review file existence' = 'Review Exists'
'Review Result' = $Result
'Compilation Check Result' = $Compilation
'Static Analysis Result' = $StaticAnalysis
'Review up-to-date' = $UpToDateFlag
'Reviewed code version' = $ReviewFileVersionNumber
'Latest code version' = $SourceFileVersionNumber
}
} | Export-Csv Example-csv.csv -NoTypeInformation
The big drawback here is that you don't get a lot of formatting choices about the CSV. Every field is quoted, for example.
Alternately, if you really want really detailed control of the $output string, you should use a StringBuilder instead of a String. StringBuilder is one of the most potent and widely used classes in C#. This is because strings in C# and Powershell are immutable, so when you += a String you create a new string, copy everything over with the new bit, then throw the old string away. It can be very memory intensive with large operations. StringBuilder lets you get around all that. It's a class that's designed to let you append stuff to strings and format them however you want.
You instance it like so:
$output = [System.Text.StringBuilder]::new()
And then you typically call one of two methods to add text. Append($string) appends the string, AppendLine($string) appends the line and then adds a newline. You can also call AppendLine() with no argument to just add a newline. To get your final string, you call the ToString() method. The append methods do return a status when you call them which you can prevent from outputting pretty easily with a [void], or by saving it to another variable if you need it.
$output = [System.Text.StringBuilder]::new()
[void]$output.AppendLine($headings)
$SourceParentDir = "C:\StarTeam\00011114-JSENS_TRS\ATR\04_SW_Implementation\Operational"
$Files = get-childitem -Path $SourceParentDir -Recurse -Filter '*.?pp' | % { $_.FullName }
foreach ($File in $Files)
{
$BaseName = [System.IO.Path]::GetFileName($File)
# Populate each row for each file
[void]$output.Append($BaseName)
[void]$output.Append(", Review Exists")
[void]$output.Append(", $Result,$Compilation,$StaticAnalysis,$UpToDateFlag,$ReviewFileVersionNumber,$SourceFileVersionNumber,")
[void]$output.AppendLine()
}
$output.ToString() | Out-File -FilePath Documents\Example-csv.csv -encoding utf8
$output is an array, so each of those += inside the loop is a new entry in the array, and therefore a new line in the file.
You can fix this by using a temporary string variable in the middle of the loop, and appending it to $output once at the end of each iteration:
foreach ($File in $Files)
{
$row = [System.IO.Path]::GetFileName($File)
$row += ", Review Exists"
$row += ", " + $Result + "," + $Compilation + "," + $StaticAnalysis + "," + $UpToDateFlag + "," + $ReviewFileVersionNumber + "," + $SourceFileVersionNumber + ","
$output += $row
}
or by putting everything in one big string interpolation:
foreach ($File in $Files)
{
$BaseName = [System.IO.Path]::GetFileName($File)
$output += "$BaseName, Review Exists, $Result, $Compilation, $StaticAnalysis, $UpToDateFlag, $ReviewFileVersionNumber, $SourceFileVersionNumber"
}
But I agree with the other answer that building up an array of custom objects for the Export-Csv commandlet is more idiomatic for PowerShell.
The issue is how you're populating $Output. Since it is defined as an array, each time you're adding new information it's creating a new entry rather than just adding the additional information to a string.
If you make your $output into one line with all required fields it should correct it without you changing the array.
$output += $BaseName + ", Review Exists" + ", " + $Result + "," + $Compilation + "," + $StaticAnalysis + "," + $UpToDateFlag + "," + $ReviewFileVersionNumber + "," + $SourceFileVersionNumber + ","
I have a script I have been working on to generatie some xml files it works as expected when I run it with only 2 rows in the CSV when I add more rows it just combines the values instead of looping. Example the last line is to save the file with 12345678.xml file and instead names it with the numbers from all rows. example
12345678
25789654
45658965
46982268.xml
It does this with all lines in the loop.
I need help getting to loop through each row and making 1 file per row instead of combing the data from all rows. It should loop 4 times saving 4 files instead of 1 file with all the data combined.
#Define the source CSV with the list of MAC address and Phone Numbers
#$sourceCSV = Import-Csv "C:\users\bob\Desktop\phones.csv"
$xmltemp = Get-Content C:\users\bob\Desktop\xmltemp.xml
#Pull first 13 lines of the XML Temp as variable chunk1
$chunk1 = $xmltemp | Select-Object -First 11
#Select lines 15 to 20 of the XML Temp as variable chunk2
$chunk2 = $tdcxmltemp | Select-Object -First 6 -skip 10
#Select lines 22 to 116 of the XML Temp as variable chunk3
$Chunk3 = $tdcxmltemp | Select-Object -First 95 -skip 11
#Select lines 120 to eof of the XML Temp as variable chunk4
$Chunk4 = $tdcxmltemp | Select-Object -skip 119
#Loop to combine the data, and generate a MACaddress.xml for each MAC listed in the CSV file
Import-Csv "C:\users\bob\Desktop\phones.csv" | ForEach-Object {
foreach ($phone in $_.PSObject.Properties.mac){
$phonenum = $sourceCSV.number
$phonemac = $sourceCSV.mac
$update = $sourceCSV.update_server
# combine the first 13 lines of the script and Append phone number on line 13, add lines 15-20 into variable "xml2"
$xml2 = $chunk1 + ("<!-- Configuration file for Line " + $phonenum + " -->") + $chunk2
#Add Update server to xml2 variable and create xml3 variable, combine with chunk 3
$xml3 = $xml2 + (' <update_server source="STATIC">' + $updater + "</update_server>") + $Chunk3
#combine xml3 veriable with line 117 - append phone number (line_id) into variable xml4
$xml4 = $xml3 + (' <line_id>' + $phonenumr + "</line_id>")
#combine xml4 veriable with line 118 - append phone number (label) into variable xml5
$xml5 = $xml4 + (' <label>' + $phonenum + "</label>")
#combine xml5 veriable with line 119 - append phone number (sip_name) into variable xml6
$xml6 = $xml5 + (' <sip_name>' + $phonenumr + "</sip_name>")
#Combine xml6 veriable with chunk4 into variable xml7
$xml7 = $xml6 + $chunk4
#save the xml file and name with macadress
$xml7 | Out-File ('C:\users\bob\Desktop\' + $phonemac + '.xml')
}
}
CSV FIle
mac,number,update_server
12345678,444-2223,192.168.1.1
25789654,1234567,192.168.1.1
45658965,7654321,192.168.1.1
46982268,99999999,192.168.1.1
I'm not totally sure, but I think that instead of this ...
Import-Csv "C:\users\bob\Desktop\phones.csv" | ForEach-Object {
foreach ($phone in $_.PSObject.Properties.mac){
$phonenum = $sourceCSV.number
$phonemac = $sourceCSV.mac
$update = $sourceCSV.update_server
...
#save the xml file and name with macadress
$xml7 | Out-File ('C:\users\bob\Desktop\' + $phonemac + '.xml')
}
}
... you might just want this:
Import-Csv "C:\users\bob\Desktop\phones.csv" | ForEach-Object {
$phonenum = $_.number
$phonemac = $_.mac
$update = $_.update_server
...
#save the xml file and name with macadress
$xml7 | Out-File ('C:\users\bob\Desktop\' + $phonemac + '.xml')
}
Note that in your original example, $sourceCSV.mac is an array containing the value of the mac property from each record in the csv file - i.e. #(12345678, 25789654, 45658965, 46982268) - so when PowerShell evaluates 'C:\users\bob\Desktop\' + $phonemac + '.xml' it takes the array and lists every item with a space between them:
$csv = #"
mac,number,update_server
12345678,444-2223,192.168.1.1
25789654,1234567,192.168.1.1
45658965,7654321,192.168.1.1
46982268,99999999,192.168.1.1
"#
$data = $csv | ConvertFrom-CSV
$phonemac = $data.mac
$phonemac
# 12345678
# 25789654
# 45658965
# 46982268
"C:\users\bob\Desktop\" + $phonemac + ".xml"
# C:\users\bob\Desktop\12345678 25789654 45658965 46982268.xml
It was explained to me that foreach assigns the data for $phone to each iteration.
changing $sourceCSV.number to $phone.number resolved the issue.
more detail:
foreach ($phone in $sourceCSV){
Changing
$phonenum = $sourceCSV.number
$phonemac = $sourceCSV.mac
$update = $sourceCSV.update_server
to
$phonenum = $phone.number
$phonemac = $phone.mac
$update = $phone.update_serve
resolved the issue and it works correctly now
I have a CSV file with delimiter as | , the values between the delimiter are not enclosed between quotes or any character. I am trying out a way to accept this file as an input and export a new CSV file with all the column to be enclosed in ^
Example:
Input:
1|Test|3
Output:
^1^|^Test^|^3^
Have tried below code but in vain:
get-content C:\ip.csv | foreach-object { $_ -replace '|' ,'^'} | set-content C:\op.csv
Try this:
$cont = get-content "D:\path\file.csv"
$cont | % {
$info = "^" + $_.Split("|")[0] + "^" + $_.Split("|")[1] + "^" + $_.Split("|")[2] + "^"
$info
}
I have a csv file containing rows of the following extract:
"EmployeeID","FirstName","LastName","Location","Department","TelephoneNo","Email"
"000001 ","abc ","def ","Loc1"," "," ","name1#company.com "
"000023 ","ghi ","jkl ","Loc2"," "," ","name2#company.com "
"000089 ","mno ","pqr ","Loc2"," "," ","name3#company.com "
How do I keep the quotes and sort and save as a csv file?
I have the following powershell source script which works with csv files not having double quotes for the columns:
Get-Content $Source -ReadCount 1000 |
ConvertFrom-Csv -Delimiter $Delimiter |
Sort-Object -Property $NamesOfColumns -Unique |
ForEach-Object {
# Each of the values in $ColumnValueFormat must be executed to get the property from the loop variable ($_).
$values = foreach ($value in $ColumnValueFormat) {
Invoke-Expression $value
}
# Then the values can be passed in as an argument for the format operator.
$ShowColsByNumber -f $values
} |
Add-Content $Destination;
The $Source, $Delimiter, $NamesOfColumns and $ColumnValueFormat are given or built dynamically.
$ColumnValueFormat with a non quoted csv file contains:
$_.EmployeeID.Trim()
$_.FirstName.Trim()
$_.LastName.Trim()
$_.Location.Trim()
$_.Department.Trim()
$_.TelephoneNo.Trim()
$_.Email.Trim()
$ColumnValueFormat with a quoted csv file contains:
$_."EmployeeID".Trim()
$_."FirstName".Trim()
$_."LastName".Trim()
$_."Location".Trim()
$_."Department".Trim()
$_."TelephoneNo".Trim()
$_."Email".Trim()
The problem seems to be based around the $ColumnValueFormat that is placing the column headers with the double quotes. (If I remove them I am not sure the internals of the cmdlet will recognize the column headings when it is processing the rows)
I am having two problems:
The column heading surrounded by the double quotes. The problem seems to be based around the $ColumnValueFormat that is placing the column headers with the double quotes as it does not process the rows. (If I remove the double quotes then it does not recognize the column headings when it is processing the rows).
Another problem I came across last minute is if the last column is blank it thinks it's a null and when the Invoke-Expression $value executes (where $value holds the last column expression of $_.Email.Trim() - on a non quoted CSV file) it bombs. If I try to place the statement in a try/catch block it simply ignore it the last column is not added to the $values array and again bombs.
Quotes around property names are used syntactically to access names with spaces, not to write quotes to the output.
Export-Csv cmdlet doesn't have an option to force quotes so we'll have to export the CSV manually. And we'll have to process empty values that are $Null after ConvertFrom-Csv with an empty string. In case only some fields are needed we'll use Select cmdlet with -index parameter.
Get-Content $Source |
ConvertFrom-Csv |
%{ $header = $false } {
if (!$header) {
$header = $true
'"' + (
($csv[0].PSObject.Properties.Name.trim() |
select -index 1,6
) -join '","'
) + '"'
}
'"' + (
($_.PSObject.Properties.Value |
%{ if ($_) { $_.trim() } else { '' } } |
select -index 1,6
) -join '","'
) + '"'
} | Out-File $Destination
The above code is great for pass-through processing of large CSV files because it doesn't keep the entire file in memory. Otherwise it's possible to simplify the code a bit:
$csv = Get-Content $Source | ConvertFrom-Csv
$csv | %{
'"' + (
($csv[0].PSObject.Properties.Name.trim() |
select -index 1,6
) -join '","'
) + '"'
} {
'"' + (
($_.PSObject.Properties.Value |
%{ if ($_) { $_.trim() } else { '' } } |
select -index 1,6
) -join '","'
) + '"'
) | Out-File $Destination