Having trouble using the export-csv - powershell

I have 3 csv files and they are all only 1 column long. I have tried lots of thing to put it all in one csv file but i cant get it to work. When i output it, it all ends up in one column here is what i did so far
#Putting Csv files to Array
$CSV1 = #(gc $EmailPathCsv)
$CSV2 = #(gc $UserLPathCsv)
$CSV3 = #(gc $EmailPathCsv)
#
for ($i=0; $i -lt $CSV1.Count; $i++)
{
$CSV4 += $CSV1[$i] + "," + $CSV2[$i] + "," + $CSV3[$i] + " "
}
$csv4 | out-file -append $MergedCsvExport

Your loop is adding everything into $CSV4, each time through the loop $CSV4 gets longer and longer.
Then you print it once. That's why you get one really long line. Try printing it once every time through the loop, and overwriting $CSV4 every time:
#Putting Csv files to Array
$CSV1 = #(gc $EmailPathCsv)
$CSV2 = #(gc $UserLPathCsv)
$CSV3 = #(gc $EmailPathCsv)
#
for ($i=0; $i -lt $CSV1.Count; $i++)
{
$CSV4 = $CSV1[$i] + "," + $CSV2[$i] + "," + $CSV3[$i] + " "
out-file -InputObject $csv4 -Append -FilePath $MergedCsvExport
}

I'd use a foromat string for that.
$CSV1 = #(gc $EmailPathCsv)
$CSV2 = #(gc $UserLPathCsv)
$CSV3 = #(gc $EmailPathCsv)
for ($i=0; $i -lt $CSV1.Count; $i++)
{
'"{0}","{1}","{2}"' -f $CSV1[$i],$CSV2[$i],$CSV3[$i] |
add-content $MergedCsvExport
}

As a more fun answer:
$CSV1 = 1,2,3,4
$CSV2 = 10,11,12,13
$CSV3 = 'a','b','c','d'
$c = gv -Name CSV* | select -expand name | sort
(gv -Va $c[0])|%{$r=0}{#($c|%{(gv -Va $_)[$r]}) -join ',';$r++}
Sample output:
1, 10, a
2, 11, b
3, 12, c
4, 13, d
You could put |Out-File "merged-data.csv" on the end to save to a file.
It works for more columns too, just add more arrays called $CSV{something}.
Edit: I wonder if Get-Variable's output is in a predictable order, or unspecified? If you don't mind if the column order might change, it collapses to:
$CSV1 = 1,2,3,4
$CSV2 = 10,11,12,13
$CSV3 = 'a','b','c','d'
(gv CSV*)[0].Value|%{$r=0}{#((gv CSV*)|%{(gv -Va $_.Name)[$r]}) -join ',';$r++}
Edit again: Well, in case anyone notices and is curious, and has time on their hands, I've expanded it all out with an explanation of what it's doing:
# Search the local variable scope for variables named CSV*
# This will find $CSV1, $CSV2, etc.
# This means the number of columns
# isn't fixed, you can easily add more.
# Take their names, sort them.
# Result: an array of strings "CSV1", "CSV2", ...
# for however many variables there are
$columnVariables = Get-Variable -Name "CSV*" | Select-Object -Expand Name | Sort-Object
# NB. if you remove $CSV3= from your code, it is
# still in memory from previous run. To remove
# it, use `Remove-Variable "CSV3"
# In pseudo-code, what the next part does is
# for row in range(data):
# #(CSV1[row], CSV2[row], ... CSVn[row]) -join ','
# The outer loop over the number of columns
# is done by piping something of the right length
# into a foreach loop, but ignoring the content.
# So get the first column array content:
$firstColumn = (Get-Variable $columnVariables[0]).Value
# and pipe it into a loop.
# The loop uses ForEach {setup} {loop} pattern
# to setup a row-counter before the loop body
$firstColumn | ForEach { $row=0 } {
# Now we know the row we are on, we can get
# $CSV1[row], $CSV2[row], ...
# Take the column variable array "CSV1", "CSV2", ..
# Loop over them
$rowContent = $columnVariables | ForEach {
# $_ a string of the name, e.g. "CSV1"
# Use Get-Variable to convert it
# into the variable $CSV1
# with -ValueOnly to get the array itself
# rather than details about the variable
$columnVar = Get-Variable -ValueOnly $_
# $columVar is now one of the $CSVn variables
# so it contains, e.g. 1,2,3,4
# Index into that for the current row
# to get one item, e.g. 3
# Output it into the pipeline
($columnVar)[$row]
} # This is the end of the inner loop
# The pipeline now contains the column
# values/content making up a single row
# 1
# 10
# 'a'
# Back in the outer row loop, Take the row
# content and make it a comma separated string
# e.g. "1,10,a"
# Output this into the pipeline
#($rowContent) -join ','
# Increment the row counter
$row++
}

Related

Surrounding a string variable with quotes

I am writing an IIS log parser and having trouble wrapping a variable value in quotes while doing some string processing.
Here is a truncated log file, as an example:
#Fields: date time s-ip cs-method ...
2021-08-09 19:00:16.367 0.0.0.0 GET ...
2021-08-09 19:01:42.184 0.0.0.0 POST ...
Here is how I am executing the code below:
.\Analyse.ps1 cs-method -eq `'POST`'
If the line marked with #PROBLEM is executed as is, the output looks like this:
> .\Analyse.ps1 cs-method -eq `'POST`'
"""""G""E""T""""" ""-""e""q"" ""'""P""O""S""T""'""
"""""P""O""S""T""""" ""-""e""q"" ""'""P""O""S""T""'""
But if I replace $quoted with $value, so that the code reads like this:
$thisInstruction = $thisInstruction -replace $key , $value #PROBLEM
The output looks like this:
> .\Analyse.ps1 cs-method -eq `'POST`'
GET -eq 'POST'
POST -eq 'POST'
The problem is that I want the first value on each line of the output (the GET and the POST before the -eq) to be wrapped in quotes.
How can I achieve this?
Here is my code:
# compile cli args into single line instruction
$instruction = $args -join " "
# define key array
$keys = #('date','time','s-ip','cs-method','cs(Host)','cs-uri-stem','cs-uri-query','s-computername','s-port','cs-username','c-ip','s-sitename','cs(User-Agent)','cs(Referer)','sc-status','sc-substatus','sc-win32-status','TimeTakenMS','x-forwarded-for')
# <#
# get current execution folder
$currentFolder = Get-Location
# define string splitter regex https://www.reddit.com/r/PowerShell/comments/2h5elx/split_string_by_spaces_unless_in_quotes/ckpkydh?utm_source=share&utm_medium=web2x&context=3
$splitter = ' +(?=(?:[^\"]*\"[^\"]*\")*[^\"]*$)'
# process *.log files in folder
Get-Childitem -Path $currentFolder *.log1 | ForEach-Object {
# process each line in the file
Get-Content $_.Name | ForEach-Object {
# duplicate instruction
$thisInstruction = $instruction
# exclude comment lines
if (!$_.StartsWith('#')) {
# split line into array
$logEntryArr = $_ -Split $splitter
# populate dictionary with contents of array
For ($i=0; $i -le $keys.length; $i++) {
# get key
$key = $keys[$i]
# get value
$value = $logEntryArr[$i]
$quoted = "`""+$value+"`""
# replace mention of key in instruction with dictionary reference
$thisInstruction = $thisInstruction -replace $key , $quoted #PROBLEM
}
# process rule from command line against dictionary
echo $thisInstruction
}
}
}
#>
I do know why, thanks to #mathias-r-jessen commenting
I don't know why, but altering the For loop to iterate one fewer fixed the problem and does not appear to leave out any keys. The only significant change is this:
For ($i=0; $i -le $keys.length-1; $i++) {
This PowerShell script can be used to query a folder of log files echo out matching rows, eg:
.\Analyse.ps1 cs-method -eq 'GET'
The above would print out all log entries with a cs-method value of GET.
Here's the code:
# compile cli args into single line instruction
$instruction = $args -join " "
# define key array
$keys = #('date','time','s-ip','cs-method','cs(Host)','cs-uri-stem','cs-uri-query','s-computername','s-port','cs-username','c-ip','s-sitename','cs(User-Agent)','cs(Referer)','sc-status','sc-substatus','sc-win32-status','TimeTakenMS','x-forwarded-for')
# get current execution folder
$currentFolder = Get-Location
# define string splitter regex https://www.reddit.com/r/PowerShell/comments/2h5elx/split_string_by_spaces_unless_in_quotes/ckpkydh?utm_source=share&utm_medium=web2x&context=3
$splitter = ' +(?=(?:[^\"]*\"[^\"]*\")*[^\"]*$)'
# process *.log files in folder
Get-Childitem -Path $currentFolder *.log | ForEach-Object {
# process each line in the file
Get-Content $_.Name | ForEach-Object {
# duplicate instruction
$thisInstruction = $instruction
# exclude comment lines
if (!$_.StartsWith('#')) {
# split line into array
$logEntryArr = $_ -Split $splitter
# populate dictionary with contents of array
For ($i=0; $i -lt $keys.length; $i++) {
# get key
$key = $keys[$i]
# get value
$quoted = "`'"+$logEntryArr[$i]+"`'"
# replace mention of key in instruction with dictionary reference
$thisInstruction = $thisInstruction -replace $key , $quoted
}
# process rule from command line against dictionary
$answer = Invoke-Expression $thisInstruction
if ($answer) {
echo $_
}
}
}
}

How to set as variable csv column using powershell?

I have csv file like this
ID Name
4 James
6 John
1 Cathy
I want to save those file as .cmd with this format
SET NUMBER1=4
SET NUMBER2=6
SET NUMBER3=1
The total of ID in the csv file is not always 3. If the ID more than 3, it means my cmd file be like this
SET NUMBER1=4
SET NUMBER2=6
SET NUMBER3=1
SET NUMBERN=N
Anyone can help please. I really new in powershell, really need help and advice please. Thanks
$ID = Import-Csv .\Data.csv | Select-Object -ExpandProperty ID
$ID.Count
ForEach ( $id in $ID ) {
}
I am stuck here
An alternative approach is below if your headers are always present in the file. It doesn't matter what the delimiter is as long as it isn't a number. Your delimited data in the sample is not consistent. Otherwise, Import-Csv would be a safer option.
$fileData = Get-Content file.csv
$output = for ($i = 1; $i -lt $fileData.count; $i++) {
"SET NUMBER{0}={1}" -f $i,($fileData[$i] -replace "(?<=^\d+).*")
}
$output | Out-File file.cmd
Explanation:
The format operator (-f) is used to help construct the output strings. The ID numbers are selected using regex by replacing everything that comes after the beginning digits on each line.
Try this:
# set current directory to script directory
Set-Location $PSScriptRoot
# import csv-file, delimiter = space
$content = Import-Csv 'test.csv' -Delimiter ' '
$output = ''
# create output lines
for( $i = 1; $i -le $content.Count; $i++ ) {
$output += 'SET NUMBER' + $i.ToString() + '=' + $content[$i-1].ID.ToString() + [environment]::NewLine
}
# output to file
$output | Out-File 'result.bat' -Force

Transpose rows to columns in PowerShell

I have a source file with the below contents:
0
ABC
1
181.12
2
05/07/16
4
Im4thData
5
hello
-1
0
XYZ
1
1333.21
2
02/02/16
3
Im3rdData
5
world
-1
...
The '-1' in above lists is the record separator which indicates the start of the next record. 0,1,2,3,4,5 etc are like column identifiers (or column names).
This is my code below.
$txt = Get-Content 'C:myfile.txt' | Out-String
$txt -split '(?m)^-1\r?\n' | ForEach-Object {
$arr = $_ -split '\r?\n'
$indexes = 1..$($arr.Count - 1) | Where-Object { ($_ % 2) -ne 0 }
$arr[$indexes] -join '|'
}
The above code creates output like below:
ABC|181.12|05/07/16|Im4thData|hello
XYZ|1333.21|02/02/16|Im3rdData|World
...
But I need output like below. When there are no columns in the source file, then their row data should have blank pipe line (||) like below in the output file. Please advise the change needed in the code.
ABC|181.12|05/07/16||Im4thData|hello ← There is no 3rd column in the source file. so blank pipe line (||).
XYZ|1333.21|02/02/16|Im3rdData||World ← There is no 4th column column in the source file. so blank pipe line (||).
...
If you know the maximum number of columns beforehand you could do something like this:
$cols = 6
$txt = Get-Content 'C:myfile.txt' | Out-String
$txt -split '(?m)^-1\r?\n' | ForEach-Object {
# initialize array of required size
$row = ,$null * $cols
$arr = $_ -split '\r?\n'
for ($n = 0; $n -lt $arr.Count; $n += 2) {
$i = [int]$arr[$n]
$row[$i] = $arr[$n+1]
}
$row -join '|'
}
Otherwise you could do something like this:
$txt = Get-Content 'C:myfile.txt' | Out-String
$txt -split '(?m)^-1\r?\n' | ForEach-Object {
# create empty array
$row = #()
$arr = $_ -split '\r?\n'
$k = 0
for ($n = 0; $n -lt $arr.Count; $n += 2) {
$i = [int]$arr[$n]
# if index from record ($i) is greater than current index ($k) append
# required number of empty fields
for ($j = $k; $j -lt $i-1; $j++) { $row += $null }
$row += $arr[$n+1]
$k = $i
}
$row -join '|'
}
Needs quite a bit of processing. There might be a more efficient way to do this, but the below does work.
$c = Get-Content ".\file.txt"
$rdata = #{}
$data = #()
$i = 0
# Parse the file into an array of key-value pairs
while ($i -lt $c.count) {
if($c[$i].trim() -eq '-1') {
$data += ,$rdata
$rdata = #{}
$i++
continue
}
$field = $c[$i].trim()
$value = $c[++$i].trim()
$rdata[$field] = $value
$i++
}
# Check if there are any missing values between 0 and the highest value and set to empty string if so
foreach ($row in $data) {
$top = [int]$($row.GetEnumerator() | Sort-Object Name -descending | select -First 1 -ExpandProperty Name)
for($i = 0; $i -lt $top; $i++) {
if ($row["$i"] -eq $null) {
$row["$i"] = ""
}
}
}
# Sort each hash by field order and join with pipe
$data | ForEach-Object { ($_.GetEnumerator() | Sort-Object -property Name | Select-Object -ExpandProperty Value) -join '|' }
In the while loop, we are just iterating over each line of the file. The field number an value are separated by a value of one, so each iteration we take both values and add them to the hash.
If we encounter -1 then we know we have a record separator, so add the hash to an array, reset it, bump the counter to the next record and continue to the next iteration.
Once we've collected everything we need to check if there are any missing field values, so we grab the highest number from each hash, loop over it from 0 and fill any missing values with an empty string.
Once that is done you can then iterate the array, sort each hash by field number and join the values.

Find replace using PowerShell get-content

I am attempting to mask SSN numbers with Random SSNs in a large text file. The file is 400M or .4 gigs.
There are 17,000 instances of SSNs that i want to find and replace.
Here is an example of the powershell script I am using.
(get-content C:\TrainingFile\TrainingFile.txt) | foreach-object {$_ -replace "123-45-6789", "666-66-6666"} | set-content C:\TrainingFile\TrainingFile.txt
My problem is that that i have 17,000 lines of this code to that I have in a .ps1 file. The ps1 file looks similar to
(get-content C:\TrainingFile\TrainingFile.txt) | foreach-object {$_ -replace "123-45-6789", "666-66-6666"} | set-content C:\TrainingFile\TrainingFile.txt
(get-content C:\TrainingFile\TrainingFile.txt) | foreach-object {$_ -replace "122-45-6789", "666-66-6668"} | set-content C:\TrainingFile\TrainingFile.txt
(get-content C:\TrainingFile\TrainingFile.txt) | foreach-object {$_ -replace "223-45-6789", "666-66-6667"} | set-content C:\TrainingFile\TrainingFile.txt
(get-content C:\TrainingFile\TrainingFile.txt) | foreach-object {$_ -replace "123-44-6789", "666-66-6669"} | set-content C:\TrainingFile\TrainingFile.txt
For 17,000 powershell commands in the .ps1 file. One command per line.
I did a test on just one command and it took about 15 secoonds to execute. Doing the math, 170000 X 15 seconds comes out to about 3 days to run my .ps1 script of 17,000 commands.
Is there a faster way to do this?
The reason for poor performance is that a lot of extra work is being done. Let's look the process as a pseudoalgorithm like so,
select SSN (X) and masked SSN (X') from a list
read all rows from file
look each file row for string X
if found, replace with X'
save all rows to file
loop until all SSNs are processed
So what's the problem? It is that for each SSN replacement, you process all the rows. Not only those that do need masking but those that don't. That's a lot of extra work. If you got, say 100 rows and 10 replacements, you are going to use 1000 steps when only 100 are needed. In addition, reading and saving file creates disk IO. Whlist that's not often an issue for single operation, multiply the IO cost with loop count and you'll find quite large a time wasted for disk waits.
For great performance, tune the algorithm like so,
read all rows from file
loop through rows
for current row, change X -> X'
save the result
Why should this be faster? 1) You read and save the file once. Disk IO is slow. 2) You process each row only once, so extra work is not being done. As how to actually perform the X -> X' transform, you got to define more carefully what the masking rule is.
Edit
Here's more practical an resolution:
Since you already know the f(X) -> X' results, you should have a pre-calculated list saved to disk like so,
ssn, mask
"123-45-6789", "666-66-6666"
...
"223-45-6789", "666-66-6667"
Import the file into a hash table and work forward by stealing all the juicy bits from Ansgar's answer like so,
$ssnMask = #{}
$ssn = import-csv "c:\temp\SSNMasks.csv" -delimiter ","
# Add X -> X' to hashtable
$ssn | % {
if(-not $ssnMask.ContainsKey($_.ssn)) {
# It's an error to add existing key, so check first
$ssnMask.Add($_.ssn, $_.mask)
}
}
$dataToMask = get-content "c:\temp\training.txt"
$dataToMask | % {
if ( $_ -match '(\d{3}-\d{2}-\d{4})' ) {
# Replace SSN look-a-like with value from hashtable
# NB: This simply removes SSNs that don't have a match in hashtable
$_ -replace $matches[1], $ssnMask[$matches[1]]
}
} | set-content "c:\temp\training2.txt"
Avoid reading and writing the file multiple times. I/O is expensive and is what slows your script down. Try something like this:
$filename = 'C:\TrainingFile\TrainingFile.txt'
$ssnMap = #{}
(Get-Content $filename) | % {
if ( $_ -match '(\d{3}-\d{2}-\d{4})' ) {
# If SSN is found, check if a mapping of that SSN to a random SSN exists.
# Otherwise create a new mapping.
if ( -not $ssnMap.ContainsKey($matches[1]) ) {
do {
$rnd = Get-Random -Min 100000 -Max 999999
$newSSN = "666-$($rnd -replace '(..)(....)','$1-$2')"
} while ( $ssnMap.ContainsValue($newSSN) ) # loop to avoid collisions
$ssnMap[$matches[1]] = $newSSN
}
# Replace the SSN with the corresponding randomly generated SSN.
$_ -replace $matches[1], $ssnMap[$matches[1]]
} else {
# If no SSN is found, simply print the line.
$_
}
} | Set-Content $filename
If you already have a list of random SSNs and also have them mapped to specific "real" SSNs, you could read those mappings from a CSV (example column titles: realSSN, randomSSN) into the $ssnMap hashtable:
$ssnMap = #{}
Import-Csv 'C:\mappings.csv' | % { $ssnMap[$_.realSSN] = $_.randomSSN }
If you've already generated a list of random SSNs for replacement, and each SSN in the file just needs to be replaced with one of them (not necessarily mapped to a specific replacement string), thing I think this will be much faster:
$inputfile = 'C:\TrainingFile\TrainingFile.txt'
$outputfile = 'C:\TrainingFile\NewTrainingFile.txt'
$replacements = Get-Content 'C:\TrainingFile\SSN_Replacements.txt'
$i=0
Filter Replace-SSN { $_ -replace '\d{3}-\d{2}-\d{4}',$replacements[$i++] }
Get-Content $inputfile |
Replace-SSN |
Set-Content $outputfile
This will walk through your list of replacement SSNs, selecting the next one in the list for each new replacement.
Edit:
Here's a solution for mapping specific SSNs to specific replacement strings. It assumes you have a CSV file of the original SSNs and their intended replacement strings, as columns 'OldSSN' and 'NewSSN':
$inputfile = 'C:\TrainingFile\TrainingFile.txt'
$outputfile = 'C:\TrainingFile\NewTrainingFile.txt'
$replacementfile = 'C:\TrainingFile\SSN_Replacements.csv'
$SSNmatch = [regex]'\d{3}-\d{2}-\d{4}'
$replacements = #{}
Import-Csv $replacementfile |
ForEach-Object { $replacements[$_.OldSSN] = $_.NewSSN }
Get-Content $inputfile -ReadCount 1000|
ForEach-Object {
foreach ($Line in $_){
if ( $Line -match $SSNmatch ) #Found SSN in line
{ if ( $replacements.ContainsKey($matches[0]) ) #Found replacement string for this SSN
{ $Line -replace $SSNmatch,$replacements[$matches[0]] } #Replace SSN and ouput line
else {Write-Warning "Warning - no replacement string found for $($matches[0])"
}
}
else { $Line } #No SSN in this line - output line as-is
}
} | Set-Content $outputfile
# Fairly fast PowerShell code for masking up to 1000 SSN number per line in a large text file (with unlimited # of lines in the file) where the SSN matches the pattern of " ###-##-#### ", " ##-####### ", or " ######### ".
# This code can handle a 14 MB text file that has SSN numbers in nearly every row within about 4 minutes.
# $inputFilename = 'C:/InputFile.txt'
$inputFileName = "
1
0550 125665 338066
- 02 CR05635 07/06/16
0 SAMPLE CUSTOMER NAME
PO BOX 12345
ROSEVILLE CA 12345-9109
EMPLOYEE DEFERRALS
FREDDIE MAC RO 16 9385456 164-44-9120 XXX
SALLY MAE RO 95 9385356 07-4719130 XXX
FRED FLINTSTONE RO 95 1185456 061741130 XXX
WILMA FLINTSTONE RO 91 9235456 364-74-9130 123456789 123456389 987354321 XXX
PEBBLES RUBBLE RO 10 9235456 06-3749130 064-74-9150 034-74-9130 XXX
BARNEY RUBBLE RO 11 9235456 06-3449130 06-3749140 063-74-9130 XXX
BETTY RUBBLE RO 16 9235456 9-74-9140 123456789 123456789 987654321 XXX
PLEASE ENTER BELOW ANY ADDITIONAL PARTICIPANTS FOR WHOM YOU ARE
REMITTING. FOR GENERAL INFORMATION AND SERVICE CALL
"
$outputFilename = 'D:/OutFile.txt'
#(Get-Content $inputFilename ) | % {
($inputFilename ) | % {
$NewLine=$_
# Write-Host "0 new line value is ($NewLine)."
$ChangeFound='Y'
$WhileCounter=0
While (($ChangeFound -eq 'Y') -and ($WhileCounter -lt 1000))
{
$WhileCounter=$WhileCounter+1
$ChangeFound='N'
$matches = $NewLine | Select-String -pattern "[ ][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9][0-9][0-9][ |\t|\r|\n]" -AllMatches
If ($matches.length -gt 0)
{
$ChangeFound='Y'
$NewLine=''
for($i = 0; $i -lt 1; $i++){
for($k = 0; $k -lt 1; $k++){
# Write-Host "AmHere 1a `$i ($i), `$k ($k), `$NewLine ($NewLine)."
$t = $matches[$i] -replace $matches[$i].matches[$k].value, (" ###-##-" + $matches[$i].matches[$k].value.substring(8) )
$NewLine=$NewLine + $t
# Write-Host "AmHere 1b `$i ($i), `$k ($k), `$NewLine ($NewLine)."
}
}
# Write-Host "1 new line value is ($NewLine)."
}
$matches = $NewLine | Select-String -pattern "[ ][0-9][0-9]-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][ |\t|\r|\n]" -AllMatches
If ($matches.length -gt 0)
{
$ChangeFound='Y'
$NewLine=''
for($i = 0; $i -lt 1; $i++){
for($k = 0; $k -lt 1; $k++){
# Write-Host "AmHere 2a `$i ($i), `$k ($k), `$NewLine ($NewLine)."
$t = $matches[$i] -replace $matches[$i].matches[$k].value, (" ##-###" + $matches[$i].matches[$k].value.substring(7) )
$NewLine=$NewLine + $t
# Write-Host "AmHere 2b `$i ($i), `$k ($k), `$NewLine ($NewLine)."
}
}
# Write-Host "2 new line value is ($NewLine)."
}
$matches = $NewLine | Select-String -pattern "[ ][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][ |\t|\r|\n]" -AllMatches
If ($matches.length -gt 0)
{
$ChangeFound='Y'
$NewLine=''
for($i = 0; $i -lt 1; $i++){
for($k = 0; $k -lt 1; $k++){
# Write-Host "AmHere 3a `$i ($i), `$k ($k), `$NewLine ($NewLine)."
$t = $matches[$i] -replace $matches[$i].matches[$k].value, (" #####" + $matches[$i].matches[$k].value.substring(6) )
$NewLine=$NewLine + $t
# Write-Host "AmHere 3b `$i ($i), `$k ($k), `$NewLine ($NewLine)."
}
}
#print the line
# Write-Host "3 new line value is ($NewLine)."
}
# Write-Host "4 new line value is ($NewLine)."
} # end of DoWhile
Write-Host "5 new line value is ($NewLine)."
$NewLine
# Replace the SSN with the corresponding randomly generated SSN.
# $_ -replace $matches[1], $ssnMap[$matches[1]]
} | Set-Content $outputFilename

Extracting columns from text file using PowerShell

I have to extract columns from a text file explained in this post:
Extracting columns from text file using Perl one-liner: similar to Unix cut
but I have to do this also in a Windows Server 2008 which does not have Perl installed. How could I do this using PowerShell? Any ideas or resources? I'm PowerShell noob...
Try this:
Get-Content test.txt | Foreach {($_ -split '\s+',4)[0..2]}
And if you want the data in those columns printed on the same line:
Get-Content test.txt | Foreach {"$(($_ -split '\s+',4)[0..2])"}
Note that this requires PowerShell 2.0 for the -split operator. Also, the ,4 tells the the split operator the maximum number of split strings you want but keep in mind the last string will always contain all extras concat'd.
For fixed width columns, here's one approach for column width equal to 7 ($w=7):
$res = Get-Content test.txt | Foreach {
$i=0;$w=7;$c=0; `
while($i+$w -lt $_.length -and $c++ -lt 2) {
$_.Substring($i,$w);$i=$i+$w-1}}
$res will contain each column for all rows. To set the max columns change $c++ -lt 2 from 2 to something else. There is probably a more elegant solution but don't have time right now to ponder it. :-)
Assuming it's white space delimited this code should do.
$fileName = "someFilePath.txt"
$columnToGet = 2
$columns = gc $fileName |
%{ $_.Split(" ",[StringSplitOptions]"RemoveEmptyEntries")[$columnToGet] }
To ordinary、
type foo.bar | % { $_.Split(" ") | select -first 3 }
Try this. This will help to skip initial rows if you want, extract/iterate through columns, edit the column data and rebuild the record:
$header3 = #("Field_1","Field_2","Field_3","Field_4","Field_5")
Import-Csv $fileName -Header $header3 -Delimiter "`t" | select -skip 3 | Foreach-Object {
$record = $indexName
foreach ($property in $_.PSObject.Properties){
#doSomething $property.Name, $property.Value
if($property.Name -like '*CUSIP*'){
$record = $record + "," + '"' + $property.Value + '"'
}
else{
$record = $record + "," + $property.Value
}
}
$array.add($record) | out-null
#write-host $record
}