I have one row of temperature data in a text file that I would like to convert to a single column and save as a CSV file using a PowerShell script. The temperatures are separated by commas and look like this:
21,22,22,22,22,22,22,20,19,18,17,16,15,14,13,12,11,10,9,9,9,8,8,9,8,8,8,9,9,8,8,8,9,9,9,8,8,8,8,8,9,10,12,14,15,17,19,20,21,21,21,21,21,21,21,21,21,21,21,20,20,20,20,20,22,24,25,26,27,27,27,28,28,28,29,29,29,28,28,28,28,28,28,27,27,27,27,27,29,30,32,32,32,32,33,34,35,35,34,33,32,32,31,31,30,30,29,29,28,28,27,28,29,31,33,34,35,35,35,36,36,36,36,36,36,36,36,36,37,37,37,37,37,37,38,39,40,42,43,43,43,43,43,42,42,42,41,41,41,41,40,39,37,36,35,34,33,32,31,31,31,31,31,31,31,31,31,31,
I have tried several methods based on searches in this forum I thought this might work but it returns an error: Transpose rows to columns in PowerShell
This is the modified code I tried that returns: Error: "Input string was not in a correct format."
$txt = Get-Content 'C:myfile.txt' | Out-String
$txt -split '(?m)^,\r?\n' | ForEach-Object {
# create empty array
$row = #()
$arr = $_ -split '\r?\n'
$k = 0
for ($n = 0; $n -lt $arr.Count; $n += 2) {
$i = [int]$arr[$n]
# if index from record ($i) is greater than current index ($k) append
# required number of empty fields
for ($j = $k; $j -lt $i-1; $j++) { $row += $null }
$row += $arr[$n+1]
$k = $i
}
$row -join '|'
}
This seems like it should be simple to do with only one row of data. Are there any suggestions on how to convert this single row of numbers to one column?
Try this:
# convert row to column data
$header = 'TEMPERATURE'
$values = $(Get-Content input.dat) -split ','
$header, $values | Out-File result.csv
#now test the result
Import-Csv result.csv
The header is the first line (or record) in the CSV file. In this case it's a single word, because there is only one column.
The values are the items between commas in the input. In this case, the -split on commas generates an array of strings. Note that, if comma is a separator, there will be no comma after the last temperature. Your data doesn't look like that, but I have assumed that the real data does.
Then, we just write the header and the array out to a file. But what happened to all the commas? It turns out that, for a single column CSV file, there are no commas separating fields. So the result is a simple CSV file.
Last, there is a test of the output using Import-csv to read the result and display it in table format.
This isn't necessarily the best way to code it, but it might help a beginner get used to powershell.
Assuming I'm understanding your intent correctly, based on your verbal description (not your own coding attempt):
# Create simplified sample input file
#'
21,22,23,
'# > myfile.txt
# Read the line, split it into tokens by ",", filter out empty elements
# with `-ne ''` (to ignore empty elements, such as would
# result from the trailing "," in your sample input),
# and write to an output CSV file with a column name prepended.
(Get-Content myfile.txt) -split ',' -ne '' |
ForEach-Object -Begin { 'Temperatures' } -Process { $_ } |
Set-Content out.csv
More concise alternative, using an expandable (interpolating) here-string:
# Note: .TrimEnd(',') removes any trailing "," from the input.
# Your sample input suggests that this is necessary.
# If there are no trailing "," chars., you can omit this call.
#"
Temperatures
$((Get-Content myfile.txt).TrimEnd(',') -split ',' -join [Environment]::NewLine)
"# > out.csv
out.csv then contains:
Temperatures
21
22
23
I have been grabbing the file data from student_data.dat and trying to display it into tabular form.
The first 3 lines of the file are written like this:
Jamie Zawinski,78.8,81.0,77.3,80.0,80.0,77.0
Adam Douglas,86.2,69.0,77.8,81.0,87.5,88.0
Wallace Steven,66.2,68.0,91.3,78.6,80.3,86.4
I wish to set it up into a table with headers of Student Name, Assignment-1, Assignment-2, etc. I will later be manipulating the data to calculate the class averages for each assignment and the students overall average in the course so far. Each method of setting up the table results in the file being displayed as:
#{Name=Jamie Zawinski; Assignment-1=78.8; Assignment-2=81.0; Assignment-3=77.3; Assignment-4=80.0;
Midterm_Exam=80.0; Final_Exam=77.0} #{Name=Adam Douglas; Assignment-1=86.2; Assignment-2=69.0;
Assignment-3=77.8; Assignment-4=81.0; Midterm_Exam=87.5; Final_Exam=88.0} #{Name=Wallace Steven;
Assignment-1=66.2; Assignment-2=68.0; Assignment-3=91.3; Assignment-4=78.6; Midterm_Exam=80.3;
Final_Exam=86.4}
My coding has looked like this:
$file = Import-Csv C:\**real path**\student_data.dat -Delimiter ',' -Header 'Name', 'Assignment-1','Assignment-2','Assignment-3','Assignment-4','Midterm_Exam','Final_Exam'
Write-Host $file
I tried adding:
foreach ($line in $file){
$data += [pscustomobject]#{
Name = $line.name
Assignment-1 = $line.Assignment-1
Assignment-2 = $line.Assignment-2
}
}
and writing instead:
$filedata = Get-Content ./student_data.dat
$newline = $filedata.Split("`n")
$newline.count
foreach ($l in $newline){
$Names = $l.Split(",")[0].Trim()
$Assignment-1 = $l.Split(",").Trim()
[pscustomObject]#{
Names = $Names;
Assignment-1 = $Assignment-1
}
}
but errors occurred.
First of all, note that you get a more readable output if you use Write-Output instead of Write-Host, which tries to push the entire input into a single string. (Or just remove it altogether, $file is equivalent to Write-Output $file)
As Matthias R. Jessen commented, you are probably looking for the Format-Table command:
$path = "C:\**real path**\student_data.dat"
$header = 'Name', 'Assignment-1','Assignment-2','Assignment-3','Assignment-4','Midterm_Exam','Final_Exam'
$data = Import-Csv $path -Delimiter ',' -Header $header
$data | Format-Table
Note this just "pretty-prints" the data for display in the console. The data is not changed and you should not use this for any kind of file output. And it's not really necessary for working with the data. Once you're done manipulating your data, you can just export it back as CSV:
$outpath = "C:\**real path**\student_data_result.csv"
$data | Export-Csv $outpath -Delimiter "," -NoTypeInformation
I am creating the below script to search through and replace data in a set of files. The problem I'm running into is I need to ONLY match if it's the beginning of the line, and I'm not sure how/where would I use regex in the below example (e.g. ^A, ^B) when doing the comparison? I tried putting the caret in front of the name values in the table, but that didn't work...
$lookupTable = #{
'A'='1';
'B'='2'
#etc
}
Get-ChildItem 'c:\windows\system32\dns' -Filter *.dns |
Foreach-Object {
$file = $_
Write-Host "$file"
(Get-Content -Path $file -Raw) | ForEach-Object {
$line = $_
$lookupTable.GetEnumerator() | ForEach-Object {
$line = $line -replace $_.Name, $_.Value
}
$line
} | Set-Content -Path $file
}
The -replace operator accepts Regex. Just $line = $line -replace "^$($_.Name)", "$_.Value".
the way that regex works makes getting a proper "start of line" marker into the regex pattern along with the $VarName a tad iffy. so i broke it out into it's own line and used the -f string format operator to build the regex pattern.
then i used the way that -replace works on an array of strings that one usually gets from Get-Content to work on the whole array at each pass.
note that the strings have lower case items where they otta be replaced, and uppercase items where the item should NOT be replaced. [grin]
$LookUpTable = #{
A = 'Wizbang Shadooby'
Z = '666 is the number of the beast'
}
$LineList = #(
'a sdfq A er Z xcv'
'qwertyuiop A'
'z xcvbnm'
'z A xcvbnm'
'qwertyuiop Z'
)
$LookUpTable.GetEnumerator() |
ForEach-Object {
$Target = '^{0}' -f $_.Name
$LineList = $LineList -replace $Target, $_.Value
}
$LineList
output ...
Wizbang Shadooby sdfq A er Z xcv
qwertyuiop A
666 is the number of the beast xcvbnm
666 is the number of the beast A xcvbnm
qwertyuiop Z
# Here is a complete, working script that beginners can read.
# This thread
# Using regex in a key/value lookup table in powershell?
# https://stackoverflow.com/questions/57277282/using-regex-in-a-key-value-lookup-table-in-powershell
# User-modifiable variables.
# substitutions
# We need to specify what we're looking for (keys).
# We need to specify our substitutions (values).
# Example: Looking for A and substituting 1 in its place.
# Add as many pairs as you like.
# Here I use an array of objects instead of a Hashtable so that I can specify upper- and lowercase matches.
# Use the regular expression caret (^) to match the beginning of a line.
$substitutions = #(
[PSCustomObject]#{ Key = '^A'; Value = '1' },
[PSCustomObject]#{ Key = '^B'; Value = '2' },
[PSCustomObject]#{ Key = '^Sit'; Value = '[Replaced Text]' }, # Example for my Latin placeholder text.
[PSCustomObject]#{ Key = 'nihil'; Value = '[replaced text 2]' }, # Lowercase example.
[PSCustomObject]#{ Key = 'Nihil'; Value = '[Replaced Text 3]' } # Omit comma for the last array item.
)
# Folder where we are looking for files.
$inputFolder = 'C:\Users\Michael\PowerShell\Using regex in a key value lookup table in powershell\input'
# Here I've created some sample files using Latin placeholder text from
# https://lipsum.com/
# Folder where we are saving the modified files.
# This can be the same as the input folder.
# I'm creating this so we can test without corrupting the original files.
$outputFolder = 'C:\Users\Michael\PowerShell\Using regex in a key value lookup table in powershell\output'
#$outputFolder = $inputFolder
# We are only interested in files ending with .dns
$filterString = '*.dns'
# Here is an example for text files.
#$filterString = '*.txt'
# For all files.
#$filterString = '*.*'
# More info.
# https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-childitem?view=powershell-6#parameters
# Search on the page for -Filter
# You won't need to update any variables after this line.
# ===================================================================
# Generate a list of files to look at.
$fileList = Get-ChildItem $inputFolder -Filter $filterString
# Simple example.
# get-content .\apple.dns | % { $_ -replace "sit", "michael" } | set-content "C:\output\apple.dns"
# input file substitutions output
# Set up loops.
# For each file.
#{
# For each key-value pair.
#}
# "For each key-value pair."
# Create a function.
# Pipe in a string.
# Specify a list of substitutions.
# Make the substitutions.
# Output a modified string.
filter find_and_replace ([object[]] $substitutions)
{
# The automatic variable $_ will be a line from the file.
# This comes from the pipeline.
# Copy the input string.
# This avoids modifying a pipeline object.
$myString = $_
# Look at each key-value pair passed to the function.
# In practice, these are the ones we defined at the top of the script.
foreach ($pair in $substitutions)
{
# Modify the strings.
# Update the string after each search.
# case-sensitive -creplace instead of -replace
$myString = $myString -creplace $pair.Key, $pair.Value
}
# Output the final, modified string.
$myString
}
# "For each file."
# main
# Do something with each file.
foreach ($file in $fileList)
{
# Where are we saving the output?
$outputFile = Join-Path -Path $outputFolder -ChildPath $file.Name
# Create a pipeline.
# Pipe strings to our function.
# Let the function modify the strings.
# Save the output to the output folder.
# This mirrors our simple example but with dynamic files and substitutions.
# find_and_replace receives strings from the pipeline and we pass $substitutions into it.
Get-Content $file | find_and_replace $substitutions | Set-Content $outputFile
# The problem with piping files into a pipeline is that
# by the time the pipeline gets to Set-Content,
# we only have modified strings
# and we have no information to create the path for an output file.
# ex [System.IO.FileInfo[]] | [String[]] | [String] | Set-Content ?
#
# Instead, we're in a loop that preserves context.
# And we have the opportunity to create and use the variable $outputFile
# ex foreach ($file in [System.IO.FileInfo[]])
# ex $outputFile = ... $file ...
# ex [String[]] | [String] | Set-Content $outputFile
# Quote
# (Get-Content -Path $file -Raw)
# By omitting -Raw, we get: one string for each line.
# This is instead of getting: one string for the whole file.
# This keeps us from having to use
# the .NET regular expression multiline option (and the subexpression \r?$)
# while matching.
#
# What it is.
# Multiline Mode
# https://learn.microsoft.com/en-us/dotnet/standard/base-types/regular-expression-options#Multiline
#
# How you would get started.
# Miscellaneous Constructs in Regular Expressions
# https://learn.microsoft.com/en-us/dotnet/standard/base-types/miscellaneous-constructs-in-regular-expressions
}
I am using PowerShell to read and loop through a CSV file in order to create a new file for each row of the CSV file. I need to use the header names as part of each new file.
For each row of the CSV, how can I loop through each column and output the key and value for each variable in the output of each new file?
For example, if Master.csv contains
a,b,c
1,2,3
4,5,6
I would like to output a file named file1.txt:
a=1
b=2
c=3
and a file named file2.txt:
a=4
b=5
c=6
Is there an advantage to converting the array into a hash table, and using something like $d.Keys?
I am trying the below, but cannot get the key:
Import-Csv "C:\Master.csv" | %{
$CsvObject = $_
Write-Output "Working with $($CsvObject.a)"
$CsvObject | ForEach-Object {
Write-Output "Key = Value`n"
}
}
this will do the job, it seems. [grin] it uses the hidden .PSObject property to iterate thru the properties of each object.
# fake reading in a CSV file
# in real life, use Import-CSV
$Instuff = #'
a,b,c
1,2,3
4,5,6
'# | ConvertFrom-Csv
$Counter = 1
foreach ($IS_Item in $Instuff)
{
$FileName = "$env:TEMP\HamletHub_File$Counter.txt"
$TextLines = foreach ($Prop in $IS_Item.PSObject.Properties.Name)
{
'{0} = {1}' -f $Prop, $IS_Item.$Prop
}
Set-Content -LiteralPath $FileName -Value $TextLines
$Counter ++
}
HamletHub_File1.txt content ...
a = 1
b = 2
c = 3
I used the below steps to retrieve a string from file
$variable = 'abc#yahoo.com'
$test = $variable.split('#')[0];
$file = Get-Content C:\Temp\file1.txt | Where-Object { $_.Contains($test) }
$postPipePortion = $file | Foreach-Object {$_.Substring($_.IndexOf("|") + 1)}
This results in all lines that contain $test as a substring. I just want the result to contain only the lines that exactly matches $test.
For example, If a file contains
abc_def|hf#23$
abc|ohgvtre
I just want the text ohgvtre
If I understand the question correctly you probably want to use Import-Csv instead of Get-Content:
Import-Csv 'C:\Temp\file1.txt' -Delimiter '|' -Header 'foo', 'bar' |
Where-Object { $_.foo -eq $test } |
Select-Object -Expand bar
To address the exact matching, you should be testing for equality (-eq) rather than substring (.Contains()). Also, there is no need to parse the data multiple times. Here is your code, rewritten to to operate in one pass over the data using the -split operator.
$variable = 'abc#yahoo.com'
$test = $variable.split('#')[0];
$postPipePortion = (
# Iterate once over the lines in file1.txt
Get-Content C:\Temp\file1.txt | foreach {
# Split the string, keeping both parts in separate variables.
# Note the backslash - the argument to the -split operator is a regex
$first, $second = ($_ -split '\|')
# When the first half matches, output the second half.
if ($first -eq $test) {
$second
}
}
)