Suppose I have two csv files. One is
id_number,location_code,category,animal,quantity
12212,3,4,cat,2
29889,7,6,dog,2
98900,
33221,1,8,squirrel,1
the second one is:
98900,2,1,gerbil,1
The second file may have a newline or something at the end (maybe or maybe not, I haven't checked), but only the one line of content. There may be three or four or more different varieties of the "second" file, but each one will have a first element (98900 in this example) that corresponds to an incomplete line in the first file similar to what is in this example.
Is there a way using powershell to automatically merge the line in the second (plus any additional similar) csv file into the matching line(s) of the first file, so that the resulting file is:
12212,3,4,cat,2
29889,7,6,dog,2
98900,2,1,gerbil,1
33221,1,8,squirrel,1
main.csv
id_number,location_code,category,animal,quantity
12212,3,4,cat,2
29889,7,6,dog,2
98900,
33221,1,8,squirrel,1
correction_001.csv
98900,2,1,gerbil,1
merge code used at the commandline, or in the .ps1 file of your choice
$myHeader = #('id_number','location_code','category','animal','quantity')
#Stage all the correction files: last correction in the most recent file wins
$ToFix = #{}
filter Plumbing_Import-Csv($Header){import-csv -LiteralPath $_ -Header $Header}
ls correction*.csv | sort -Property LastWriteTime | Plumbing_Import-Csv $myHeader | %{$ToFix[$_.id_number]=$_}
function myObjPipe($Header){
begin{
function TextTo-CsvField([String]$text){
#text fields which contain comma, double quotes, or new-line are a special case for CSV fields and need to be accounted for
if($text -match '"|,|\n'){return '"'+($text -replace '"','""')+'"'}
return $text
}
function myObjTo-CsvRecord($obj){
return ''+
$obj.id_number +','+
$obj.location_code +','+
$obj.category +','+
(TextTo-CsvField $obj.animal)+','+
$obj.quantity
}
$Header -join ','
}
process{
if($ToFix.Contains($_.id_number)){
$out = $ToFix[$_.id_number]
$ToFix.Remove($_.id_number)
}else{$out = $_}
myObjTo-CsvRecord $out
}
end{
#I assume you'd append any leftover fixes that weren't used
foreach($out in $ToFix.Values){
myObjTo-CsvRecord $out
}
}
}
import-csv main.csv | myObjPipe $myHeader | sc combined.csv -encoding ascii
You could also use ConvertTo-Csv, but my preference is to not have all the extra " cruft.
Edit 1: reduced code redundancy, accounted for \n, fixed appends, and used #OwlsSleeping suggestion about the -Header commandlet parameter
also works with these files:
correction_002.csv
98900,2,1,I Win,1
correction_new.csv
98901,2,1,godzilla,1
correction_too.csv
98902,2,1,gamera,1
98903,2,1,mothra,1
Edit 2: convert gc | ConvertTo-Csv over to Import-Csv to fix the front-end \n issues. Now also works with:
correction_003.csv
29889,7,6,"""bad""
monkey",2
This is a simple solution assuming there's always exactly one match, and you don't care about output order. Change the output path to csv1 to overwrite.
I added headers manually in both input files, but you can specify them in Import-Csv instead if you'd rather avoid changing your files.
[array]$MissingLine = Import-Csv -Path "C:\Users\me\Documents\csv2.csv"
[string]$MissingId = $MissingLine[0].id_number
[array]$BigCsv = Import-Csv -Path "C:\Users\me\Documents\csv1.csv" |
Where-Object {$_.id_number -ne $MissingId}
($BigCsv + $MissingLine) |
Export-Csv -Path "C:\Users\me\Documents\Combined.csv"
Trying to edit this line of a file ("VoIP.Enabled "1"). I wanna change the 1 to a zero. When I change it with
$dewprefs = Get-Content .\dewrito_prefs.cfg
$dewprefs | Select-String "VoIP.Enabled" | ForEach-Object {$_ -replace "1","0"} | Set-Content .\dewrito_prefs.cfg}`
However when I use this script, it removes 100 other lines, edits the right line, then deletes everything else, just leaving the line I wanted to edit.
Any help on this matter would be highly appreciated
Select-String acts as a filter: that is, the input it is given is only passed out if it matches a pattern.
Therefore, only the line of interest is written to the output file.
Do not use Select-String if all input lines - though possibly modified - should be passed through; use only ForEach-Object, and conditionally modify each input line:
$dewprefs = Get-Content .\dewrito_prefs.cfg
$dewprefs |
ForEach-Object { if ($_ -match 'VoIP\.Enabled') { $_ -replace '1', '0' } else { $_ } } |
Set-Content .\dewrito_prefs.cfg
$_ -match 'VoIP\.Enabled' now does what Select-String did in your original command: it matches only if the input line at hand contains literal VoIP.Enabled (note how the . is escaped as \. to ensure that is treated as a literal in the context of a regular expression).
Note how both branches of the if statement produce output:
$_ -replace '1', '0' outputs the result of replacing all instances of 1 in the input line with 0
$_ simply passes the input line through as-is.
Most likely you could replace the if statement with a single -replace expression, however, and, assuming that the file is small enough to be read as a whole (quite likely, in the case of a configuration file), you can use a variant of Stu's helpful simplification.
Taking full advantage of the fact that -replace supports regexes (regular expressions), the code can update lines based on a key name such as VoIP.Enabled only, without needing to know that key's current value.
$key = 'VoIP.Enabled'
$newValue = '1'
# Construct a regex that matches the entire target line.
$regex = '^\s*' + [regex]::Escape($key) + '\b.*$'
# Build the replacement line.
$modifiedLine = "$key $newValue"
(Get-Content .\dewrito_prefs.cfg) -replace $regex, $modifiedLine | Set-Content .\dewrito_prefs.cfg
Note that writing the output back to the input file only works because the input file was read into memory as a whole, up front, due to enclosing the Get-Content call in (...).
This will work too, with PowerShell v3+, and is a little more succinct:
(Get-Content .\dewrito_prefs.cfg).replace('"VoIP.Enabled "1"', '"VoIP.Enabled "0"') |
Set-Content .\dewrito_prefs.cfg
Your quotes are a little strange (3 double quotes in total?), I've mimicked what you've asked, however.
I'm trying to get an application version from Version.txt file looping through a bunch of folders. It's not a big deal by itself but the problem is that there are a lot of other stuff in these files.
Examples:
'1.0.0.1'
'Version - 0.11.0.11'
'ApplicationName - 1.0.12.89'
'Definitely some useful information.
ApplicationName - 1.0.13.0'
The file always ends with the version but there are no other correlations. The length of the version is different every time because there can be different number of digits between dots.
It drives me crazy. Any suggestions?
solution 1
((get-content "C:\temp\file1.txt" -Tail 1) -split "-|'")[1].Trim()
#Code decomposed for explain
#get last row of file
$lastrowfile=get-content "C:\temp\file1.txt" -Tail 1
#split last row with - or ' as separator
$arraystr=$lastrowfile -split "-|'"
#take element 1 of split and trim final string
$arraystr[1].Trim()
Because the version is always at the last line, use the Get-Content cmdlet with the -tail parameter to read only the last line. Then select the version using the Select-String cmdlet and a regex:
(Get-Content 'Your_File_Path.txt' -Tail 1 | Select-String "(?<=- ).*(?=')").Matches.Value
Output:
1.0.13.0
Solution 2
((get-content "C:\temp\file1.txt" | where {$_ -like "*ApplicationName*"} | select -Last 1) -split "-|'")[1]
This would search the file for all lines that appear to have a version number on them, take the very last line with a match in that file, and return just the version number.
$content = Get-Content 'path\to\your\version.txt'
$regex = [regex]"\d+(\.\d+)+"
# Grab the last line in the version file that appears to have a version number
$versionLine = $content -match $regex | Select-Object -Last 1
if ($versionLine) {
# Parse and return the version
$regex.Match($versionLine).Value
}
else {
Write-Warning 'No version found.'
}
That works with all of the version numbers you posted, and would work if the version number appears to be at the end of the file, but there's additional whitespace afterward, etc.
you can use get-content and then split :
((get-content "C:\test.txt" | where {$_ -like "*ApplicationName*"} | select -Last 1) -split "-|'")[1]
Using Powershell, I am simply trying to pull 15 characters starting from the 37th position of any record that begins with a 6. I'd like to loop through and generate a record for each instance so it can later be put into an output file. But I seem to not be hitting the correct syntax just to return the 15 characters I know I am missing something obvious. Been at this for a while. Here is my script:
$content = Get-Content -Path .\tmfhsyst*.txt | Where-Object { $_.StartsWith("6") }
foreach ($line in $contents)
{
$val102 = $line.substring(36,15)
}
write-output $val102
Just as Bill_Stewart pointed out, you need to move your Write-Output line inside the ForEach loop. A possibly better way to do it would just be to pipe it:
Get-Content -Path .\tmfhsyst*.txt | Where-Object { $_.StartsWith("6") } | foreach{$_.substring(36,15)}
That should give you the output you desired.
Using Substring() has the disadvantage that it will raise an error if the string is shorter than start index + substring length. You can avoid this with a regular expression match:
(Get-Content -Path .\tmfhsyst*.txt) -match '^6.{35}(.{15})' | % { $matches[1] }
I often want to process a file resulting in a modified output file. I can't seem to find the PowerShell way to do this - it always ends up looking like code in a one-liner, IYSWIM. This is an example.
I have an LMHOSTS file. Don't ask why! I want to get a new LMHOSTS file that only contains servers that respond to pings. I also want to pass through any comment lines.
Comment lines begin with a #.
Data lines are something like this (space-separated):
10.123.177.1 SVRDS1 #PRE #DOM:DOM1
or this:
10.241.177.30 SVRDS30 #PRE
This is what I've got (with thanks to Luke for help with Ping-Host):
gc 'C:\work\lmhosts.txt' | % { if ($_ -like '#*') { $_ | out-file 'C:\work\lmhosts2.txt' -append } else { if ($(Ping-Host $_.Substring(0, $_.indexof(' ')) -count 1 -timeout 10).received -eq 1) { $_ | out-file 'C:\work\lmhosts2.txt' -append } } }
It works but it's not nice. I've taken a programmer's approach but done it as a one-liner. Is there a better way of doing it, that's more 'shell' and less 'code'?
In this particular example, you are filtering the contents of 1 file and outputting it to another. Where-Object is usually a good choice for this. You can then simplify your one-liner a bit:
gc 'C:\work\lmhosts.txt' |
?{ ($_ -like '#*') -or
($(Ping-Host $_.Split(' ')[0]) -count 1 -timeout 10).received -eq 1) } >
'C:\work\lmhosts2.txt'
Some notes:
"?" is an alias for Where-Object
The -or operator will short circuit, that is, if the first operand results in True, the second will not bother executing.
You can use the redirection operator ">" instead of "| Out-File".
I replaced Substring with Split, it seemed slightly simpler, not sure if works in general with your input.
Whitespace is your friend! In Powershell scripts I write that aren't immediately thrown away, almost every '|' is followed by a newline + indent.