Blank result for String concatenation in foreach - powershell
There might not be the best way but what is wrong with this code. Why do i get blank result from $str in the foreach loop, whereas if it try to concatenate individual cells i get the right result.
$csv = import-csv -Path "C:\Users\abc\csv4script\TEST.csv" -Header 'IPs'
$str = ''
$x = 1
foreach ($cell in $csv.){
if ($x -le 4000){
$str = $str + ", " + $cell.IPs
if ($x -eq 4000){
$x = 1}
$str = ''
}
$x = $x + 1
}
$str
# $str = $str + $csv[1].IPs + ", " + $csv[2].IPs
$str
It doesn't have a value because you misplaced a '}' character.
$x = 1}
Move it to after $str = ''
$x = 1
$str = ''}
Also as Thomas pointed out, remove the period after $csv variable in your foreach statement, it should be like: foreach($cell in $csv)
Also if you want to increment a value,
$x++ is quicker and easier to read than $x = $x + 1
Lastly, as you have it, if your list has more than 4000 IPs the $str variable will empty out.
Related
How to cut specific characters in a string and save it to a new file?
I have a string, and I want to cut some characters and store it to a new file. I tried this code, but it still error. $a = ";Code=NB" $b = $a -split "=" $b[1] $Save = "[AGM]", "CR=JP", "LOC= $b[1]"| Out-File "C:\Users\Out.txt"
Try something like this: $a = ";Code=NB" $null, $b, $null = $a -split '=', 3 $b $Save = "[AGM]", "CR=JP", "LOC= $b"| Out-File "C:\Users\Out.txt"
Something that would be easier to maintain would be this:
#Words to remove from string
$wordsToCut = "This","is"
#Phrase to remove words from
$phrase = "This is a test"
#Running through all words in words to remove
foreach ($word in $wordsToCut){
#Replace current word with nothing
$phrase = $phrase.Replace($word,"")
}
#Output end result
Write-Host $phrase
You would also use trim to remove any leading or trailing spaces. The above code outputs:
a test
perl replace characters in a string but retain special character or space
I would like to create a program that replaces characters and retains the special characters. An example input and output is shown below.
Here's what I did so far:
$sentence = userinput;
#words = split(/ /, $sentence);
for ($i = 0; $i < #words.length; $i ++){
$words[$i] =~ s/\W//g;
#characters = split(//, $words[$i]);
#print $words[$i] . "\n";
$wordlength = length($words[$i]);
for ($j = 0; $j < #characters.length; $j ++){
$char = $characters[$j];
for ($x = 0; $x < $wordlength; $x++){
$char++;
if ($char eq "aa"){
$char = "a";
}
elsif ($char eq "AA"){
$char = "A";
}
}
print $char;
if ($x = 0){
$output[$i] = $char;
}
else {
$output[$i] = join ($char);
}
}
print $output[$i];
}
Input:
Hi! how are you doing?
Output:
Jk! krz duh brx itnsl?
A couple of things in your code don't make sense:
Missing use strict; use warnings;.
All variables are global (you should be using my to create variables)
#foo.length is not the number of elements in the array #foo. It's the number of elements in the array #foo concatenated with the number of characters in $_ (because arrays in scalar context return their length, . concatenates strings, and length works on $_ by default).
join ($char) always returns the empty string: You're joining an empty list (no elements) using $char as a separator.
Here's an attempt to fix all of these issues:
use strict;
use warnings;
my $sentence = readline;
$sentence =~ s{([A-Za-z]+)}{
my $word = $1;
join '', map {
my $base = ord(/^[A-Z]/ ? 'A' : 'a');
chr((ord($_) - $base + length($word)) % 26 + $base)
} split //, $word
}eg;
print $sentence;
I think what you are doing is rot3 encoding, but if so then your example is wrong my $sentence = 'Hi! how are you doing?'; $sentence =~ tr/A-Za-z/D-ZA-Cd-za-c/; print $sentence, "\n"; output Kl! krz duh brx grlqj? which is similar, but not identical to Jk! krz duh brx itnsl?
Is there a string concatenation shortcut in PowerShell?
Like with numerics? For example, $string = "Hello" $string =+ " there" In Perl you can do my $string = "hello" $string .= " there" It seems a bit verbose to have to do $string = $string + " there" or $string = "$string there"
You actually have the operator backwards. It should be +=, not =+: $string = "Hello" $string += " there" Below is a demonstration: PS > $string = "Hello" PS > $string Hello PS > $string += " there" PS > $string Hello there PS > However, that is about the quickest/shortest solution you can get to do what you want.
Avoid += for building strings
As for using the increase assignment operator (+=) to create a collection, strings are also mutual, therefore using the increase assignment operator (+=) to build a string might become pretty expensive as it will concatenate the strings and assign (copy!) it back into the variable. Instead I recommend to use the PowerShell pipeline with the -Join operator to build your string.
Apart from the fact it is faster it is actually more DRY as well:
$String = 'Hello', 'there' -Join ' ' # Assigns: 'Hello there'
Or just
-Join #('One', 'Two', 'Three') # Yields: 'OneTwoThree'
For just a few items it might not make much sense but let me give you a a more common example by creating a formatted list of numbers, something like:
[Begin]
000001
000002
000003
[End]
You could do this:
$x = 3
$String = '[Begin]' + [Environment]::NewLine
for ($i = 1; $i -le $x; $i++) { $String += '{0:000000}' -f $i + [Environment]::NewLine }
$String += '[End]' + [Environment]::NewLine
But instead, you might actually do it the PowerShell way:
$x = 3
$String = #(
'[Begin]'
for ($i = 1; $i -le $x; $i++) { '{0:000000}' -f $i }
'[End]'
) -Join [Environment]::NewLine
Performance Testing
To show the performance decrease of using the increase assignment operator (+=), let's increase $x with a factor 1000 up till 20.000:
1..20 | ForEach-Object {
$x = 1000 * $_
$Performance = #{x = $x}
$Performance.Pipeline = (Measure-Command {
$String1 = #(
'[Begin]'
for ($i = 1; $i -le $x; $i++) { '{0:000000}' -f $i }
'[End]'
) -Join [Environment]::NewLine
}).Ticks
$Performance.Increase = (Measure-Command {
$String2 = '[Begin]' + [Environment]::NewLine
for ($i = 1; $i -le $x; $i++) { $String2 += '{0:000000}' -f $i + [Environment]::NewLine }
$String2 += '[End]' + [Environment]::NewLine
}).Ticks
[pscustomobject]$Performance
} | Format-Table x, Increase, Pipeline, #{n='Factor'; e={$_.Increase / $_.Pipeline}; f='0.00'} -AutoSize
Results:
x Increase Pipeline Factor
- -------- -------- ------
1000 261337 252704 1,03
2000 163596 63225 2,59
3000 432524 127788 3,38
4000 365581 137370 2,66
5000 586370 171085 3,43
6000 1219523 248489 4,91
7000 2174218 295355 7,36
8000 3148111 323416 9,73
9000 4490204 373671 12,02
10000 6181179 414298 14,92
11000 7562741 447367 16,91
12000 9721252 486606 19,98
13000 12137321 551236 22,02
14000 14042550 598368 23,47
15000 16390805 603128 27,18
16000 18884729 636184 29,68
17000 21508541 708300 30,37
18000 24157521 893584 27,03
19000 27209069 766923 35,48
20000 29405984 814260 36,11
See also: PowerShell scripting performance considerations - String addition
Powershell - search for string, remove excess whitespace, print second field
I'm having a problem parsing some string data in powershell and need a little help. Basically I have an application command that doesn't output objects, rather string data.
a = is the item I'm searching for
b = is the actual ouput from the command
c = replaces all the excess whitespace with a single space
d = is supposed to take $c "hostOSVersion 8.0.2 7-Mode" and just print "8.0.2 7-Mode"
However, $d is not working it just prints the same value as $c. I'm a UNIX guy and this would be easy in one awk statement. If you know how to do this in one command that would be nice, or just tell me what's wrong with my $d syntax below.
$a = "hostOSVersion"
$b = "hostOSVersion 8.0.2 7-Mode"
$c = ($a -replace "\s+", " ").Split(" ")
$d = ($y -replace "$a ", "")
Well you might have to futz around with the exact pattern, but one way is with a regex:
$b = "hostOSVersion 8.0.2 7-Mode"
$b -match '(\d.*)'
$c = $matches[1]
If you really wanted to oneline it with -replace:
$($($b -replace $a, '') -replace '\s{2}', '').trim()
Your line
$c = ($a -replace "\s+", " ").Split(" ")
should reference the $b variable instead of $a
$c = ($b -replace "\s+", " ").Split(" ")
Then, you will notice the output of $d becomes
hostOSVersion
8.0.2
7-Mode
and a statement like $d[1..2] -join ' ' would produce 8.0.2 7-Mode
Powershell / Perl : Merging multiple CSV files into one?
I have the following CSV files, I want to merge these into a single CSV
01.csv
apples,48,12,7
pear,17,16,2
orange,22,6,1
02.csv
apples,51,8,6
grape,87,42,12
pear,22,3,7
03.csv
apples,11,12,13
grape,81,5,8
pear,11,5,6
04.csv
apples,14,12,8
orange,5,7,9
Desired output:
apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,,,87,42,12,81,5,8,,,
pear,17,16,2,22,3,7,11,5,6,,,
orange,22,6,1,,,,,,5,7,9
Can anyone provide guidance on how to achieve this? Preferably using Powershell but open to alternatives like Perl if that's easier.
Thanks Pantik, your code's output is close to what I want:
apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,87,42,12,81,5,8
orange,22,6,1,5,7,9
pear,17,16,2,22,3,7,11,5,6
Unfortunately I need "placeholder" commas in place for when the entry is NOT present in a CSV file, e.g. orange,22,6,1,,,,,,5,7,9 rather than orange,22,6,1,5,7,9
UPDATE: I would like these parsed in order of the filenames, e.g.:
$myFiles = #(gci *.csv) | sort Name
foreach ($file in $myFiles){
regards
ted
Here is my Perl version:
use strict;
use warnings;
my $filenum = 0;
my ( %fruits, %data );
foreach my $file ( sort glob("*.csv") ) {
$filenum++;
open my $fh, "<", $file or die $!;
while ( my $line = <$fh> ) {
chomp $line;
my ( $fruit, #values ) = split /,/, $line;
$fruits{$fruit} = 1;
$data{$filenum}{$fruit} = \#values;
}
close $fh;
}
foreach my $fruit ( sort keys %fruits ) {
print $fruit, ",", join( ",", map { $data{$_}{$fruit} ? #{ $data{$_}{$fruit} } : ",," } 1 .. $filenum ), "\n";
}
Which gives me:
apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,,,,87,42,12,81,5,8,,,
orange,22,6,1,,,,,,,5,7,9
pear,17,16,2,22,3,7,11,5,6,,,
So do you have a typo for grape or i have misunderstood something?
Ok, gangabass solution works, and is cooler than mine, but I'll add mine anyway. It is slightly stricter, and preserves a data structure that can be used as well. So, enjoy. ;)
use strict;
use warnings;
opendir my $dir, '.' or die $!;
my #csv = grep (/^\d+\.csv$/i, readdir $dir);
closedir $dir;
# sorting numerically based on leading digits in filename
#csv = sort {($a=~/^(\d+)/)[0] <=> ($b=~/^(\d+)/)[0]} #csv;
my %data;
# To print empty records we first need to know all the names
for my $file (#csv) {
open my $fh, '<', $file or die $!;
while (<$fh>) {
if (m/^([^,]+),/) {
#{ $data{$1} } = ();
}
}
close $fh;
}
# Now we can fill in values
for my $file (#csv) {
open my $fh, '<', $file or die $!;
my %tmp;
while (<$fh>) {
chomp;
next if (/^\s*$/);
my ($tag,#values) = split (/,/);
$tmp{$tag} = \#values;
}
for my $key (keys %data) {
unless (defined $tmp{$key}) {
# Fill in empty values
#{$tmp{$key}} = ("","","");
}
push #{ $data{$key} }, #{ $tmp{$key} };
}
}
&myreport;
sub myreport {
for my $key (sort keys %data) {
print "$key," . (join ',', #{$data{$key}}), "\n";
}
}
Powershell:
$produce = "apples","grape","orange","pear"
$produce_hash = #{}
$produce | foreach-object {$produce_hash[$_] = #(,$_)}
$myFiles = #(gci *.csv) | sort Name
foreach ($file in $myFiles){
$file_hash = #{}
$produce | foreach-object {$file_hash[$_] = #($null,$null,$null)}
get-content $file | foreach-object{
$line = $_.split(",")
$file_hash[$line[0]] = $line[1..3]
}
$produce | foreach-object {
$produce_hash[$_] += $file_hash[$_]
}
}
$ofs = ","
$out = #()
$produce | foreach-object {
$out += [string]$produce_hash[$_]
}
$out | out-file "outputfile.csv"
gc outputfile.csv
apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,,,,87,42,12,81,5,8,,,
orange,22,6,1,,,,,,,5,7,9
pear,17,16,2,22,3,7,11,5,6,,,
Should be easy to modify for additional items. Just add them to the $produce array.
Second Powershell solution (as requested)
$produce = #()
$produce_hash = #{}
$file_count = -1
$myFiles = #(gci 0*.csv) | sort Name
foreach ($file in $myFiles){
$file_count ++
$file_hash = #{}
get-content $file | foreach-object{
$line = $_.split(",")
if ($produce -contains $line[0]){
$file_hash[$line[0]] += $line[1..3]
}
else {
$produce += $line[0]
$file_hash[$line[0]] = #(,$line[0]) + (#($null) * 3 * $file_count) + $line[1..3]
}
}
$produce | foreach-object {
if ($file_hash[$_]){$produce_hash[$_] += $file_hash[$_]}
else {$produce_hash[$_] += #(,$null) * 3}
}
}
$ofs = ","
$out = #()
$produce_hash.keys | foreach-object {
$out += [string]$produce_hash[$_]
}
$out | out-file "outputfile.csv"
gc outputfile.csv
apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,,,,87,42,12,81,5,8,,,
orange,22,6,1,,,,,,,5,7,9
pear,17,16,2,22,3,7,11,5,6,,,
you have to parse the files, I don't see easier way hot to do it
solution in powershell:
UPDATE: ok, adjusted a bit - hopefully understandable
$items = #{}
$colCount = 0 # total amount of columns
# loop through all files
foreach ($file in (gci *.csv | sort Name))
{
$content = Get-Content $file
$itemsToAdd = 0; # columns added by this file
foreach ($line in $content)
{
if ($line -match "^(?<group>\w+),(?<value>.*)")
{
$group = $matches["group"]
if (-not $items.ContainsKey($group))
{ # in case the row doesn't exists add and fill with empty columns
$items.Add($group, #())
for($i = 0; $i -lt $colCount; $i++) { $items[$group] += "" }
}
# add new values to correct row
$matches["value"].Split(",") | foreach { $items[$group] += $_ }
$itemsToAdd = ($matches["value"].Split(",") | measure).Count # saves col count
}
}
# in case that file didn't contain some row, add empty cols for those rows
$colCount += $itemsToAdd
$toAddEmpty = #()
$items.Keys | ? { (($items[$_] | measure).Count -lt $colCount) } | foreach { $toAddEmpty += $_ }
foreach ($key in $toAddEmpty)
{
for($i = 0; $i -lt $itemsToAdd; $i++) { $items[$key] += "" }
}
}
# output
Remove-Item "output.csv" -ea 0
foreach ($key in $items.Keys)
{
"$key,{0}" -f [string]::Join(",", $items[$key]) | Add-Content "output.csv"
}
Output:
apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,,,,87,42,12,81,5,8,,,
orange,22,6,1,,,,,,,5,7,9
pear,17,16,2,22,3,7,11,5,6,,,
Here is a more consise way how to do it. However, it still doesn't add the commas when the item is missing.
Get-ChildItem D:\temp\a\ *.csv |
Get-Content |
ForEach-Object -begin { $result=#{} } -process {
$name, $otherCols = $_ -split '(?<=\w+),'
if (!$result[$name]) { $result[$name] = #() }
$result[$name] += $otherCols
} -end {
$result.GetEnumerator() | % {
"{0},{1}" -f $_.Key, ($_.Value -join ",")
}
} | Sort