How can I increment a hex string in Perl 5.8? - perl

I have a large hex number - $num = 0x80000000, as a string.
I want to increment it, but doing hex($num) does not work, due to integer overflow (comes out negative). using bigint is also not an option as hex with bigint is only implemented in perl 5.10 and beyond, I have 5.8. how can I ++ this string?

Don't confuse the literal representation with the number with the actual value. When you make the assignment, no matter how you represented it, Perl ends up storing a number and no longer cares about the original representation. Use the normal numeric operations on it. When you want to look at it again, you can choose any of the representations you like:
$num = 0x8000000;
$num++;
printf "%0x" $num;
You only need hex() if you're getting your numeric representation as a string, like you would from a command line argument. You only need hex to turn the string into a number. After that, it's the same.
$num = hex( '8000000' ); # or hex( '0x8000000' )
# $num = hex( $ARGV[0] );
$num++;
printf "%0x" $num;
For the other part of your question, bignum works just fine on Perl v5.8:
$ perl5.8.9 -le 'my $i = 0xFFFFFFFF_FFFFFFFF; $i++; print $i'
1.84467440737096e+19
$ perl5.8.9 -Mbignum -le 'my $i = 0xFFFFFFFF_FFFFFFFF; $i++; print $i'
18446744073709551616

I have no issues with this using Perl 5.8.9 via Perlbrew:
#! /usr/bin/env perl
use strict;
use warnings;
my $num = 0x80000000;
print "$num\n"; # No quotes make this a number and not a string
This prints out 2147483648 which is the decimal value of 0x8000000.
What platform are you on?
What you probably want to do is to print out this hex number in hex and not decimal. You can still use ++ to increment it, but to print it out in hex, you need printf:
#! /usr/bin/env perl
#
use strict;
use warnings;
use Data::Dumper;
my $num = 0x80000000;
for my $loop (0..100) {
$num++;
printf "%-20x\n", $num;
}
This prints out:
80000001
80000002
80000003
80000004
80000005
80000006
80000007
80000008
80000009
8000000a
...
If indeed 0x80000000 is a string, using hex on my system using Perl 5.8.9 has no problem converting it into a number.
#! /usr/bin/env perl
use strict;
use warnings;
my $num = "0x80000000"; # This is a string
$num = hex $num; # Now it's a number
print "$num\n"; # Prints 2147483648 because `$num` is numeric
As I said. This is Perl 5.8.9 (I can't get 5.8.8 on Perlbrew), and this is on a Mac. Maybe your platform is a wee bit different. Is it an old Solaris system with a 32bit version of SunOS or Solaris?
I've checked the 5.8.8 documentation and notice that the standard distribution does have Bigint support built in. It also comes with the module Math::Bigint too.
Are you sure you don't have Bigint support?

Related

How to convert string to floating point number inside a Perl hash?

My motive is to convert the string number into floating point number while creating a hash.
I have placed my entire code and error below. Please help me to solve this issue.
Sample code
use strict;
use warnings;
use Data::Dumper;
my $price = 8.5;
my $g={};
$g->{'get'}=sprintf('%.02f',$price);
print Dumper($g);
Current output
$VAR1 = {
'get' => '8.50'
};
Expected output
$VAR1 = {
'get' => 8.50
};
Despite the single quotes around 8.50 in the Dumper output, Perl will still treat it as a numeric value when you go to use it:
use strict;
use warnings;
my $price = 8.5;
my $g={};
$g->{'get'}=sprintf('%.02f',$price);
my $x = 5;
printf "%.02f\n", $x + $g->{get};
Outputs:
13.50
use Scalar::Util 'looks_like_number';
.
.
print Dumper($g) =~ s/'(.*?)'/looks_like_number($1)?$1:"'$1'"/ger;
Which changes the output from Dumper before it's printed. It removes both 's of every single quoted string if it looks like a number according to Scalar::Util.
I suspect you're worrying unnecessarily here. Perl treats strings and numbers as largely interchangeable and will generally do the right thing with data of either type. The number of times when you should care if you have a string or a number is tiny.
In fact, even if you explicitly give Perl a number in code like yours, it will be displayed as a string:
$ perl -MData::Dumper -E'say Dumper { get => 8.5 }'
$VAR1 = {
'get' => '8.5'
};

How to force long double in Perl

I lose precision when doing arithmetic or trying to print (debug) numbers this big:
1234567890.123456789
I think my problems are with $d (result of arithmetic) and the formatted print of $e.
How can I force long doubles? My Perl version (5.8.4 on SUN) says it's possible.
sprintf has a size option for long doubles (q or L or ll), but I haven't figured out how to use it, and don't know if it would work with printf.
Edit: I added BigFloat, which works! But I'd still like to force long doubles.
Try to add 1234567890 + 0.123456789
and subtract 1234567890 - 0.123456789.
use Config;
use Math::BigFloat;
$a = 1234567890;
$b = 123456789;
$c = $b/1e9; # 0.123456789
$d = $a + $c; # not enough precision (32-bit or double?)
$e = sprintf("%d.%.9d",$a,$b); # combine as strings
$f = 1234567890.123456789; # for reference (not enough precision)
# Use BigFloat to bypass lack of longdbl
$aBig = Math::BigFloat->new("$a");
$dSum = $aBig->fadd("$c"); # $dSum = $a + $c
$aBig = Math::BigFloat->new("$a"); # <-- Need a new one for every operation?
$dDif = $aBig->fsub(abs("$c")); # $dDif = $a - $c
print "a $a\n"; # 1234567890
print "c $c\n"; # 0.123456789
print "d=a+c $d\n"; # 1234567890.12346 <-- **Problem**
print "dSum=a+c $dSum\n"; # 1234567890.123456789 <-- Solution
print "dDif=a-c $dDif\n"; # 1234567890.876543211 <-- Solution
print "e $e\n"; # 1234567890.123456789
print "f $f\n"; # 1234567890.12346 <-- double, 52-bit, not longdbl?
printf ("printf e 20.9f %20.9f\n",$e); # 1234567890.123456717 <-- **Problem**
printf ("printf dSum 20.9f %20.9f\n",$dSum); # 1234567890.123456717 <-- **Problem**
printf ("printf dSum 20s %20s\n",$dSum); # 1234567890.123456789
printf ("printf dDif 20.9f %20.9f\n",$dDif); # 1234567890.876543283 <-- **Problem**
printf ("printf dDif 20s %20s\n",$dDif); # 1234567890.876543211
print "uselongdouble $Config{uselongdouble}\n"; # empty. No long doubles by default
print "d_longdbl $Config{d_longdbl}\n"; # "define". Supports long doubles
print "size double longdbl $Config{doublesize} $Config{longdblsize}\n"; # Ans 8 16
I also used this code to try to understand the types, but it didn't help much. Has anyone used it to explain problems like this?
use Devel::Peek 'Dump';
Dump ($dSum); # Wow, it's complicated
Dump ($f);
Perl has one size of float, and it's called NV. The size of an NV is decided when Perl is built.
$ perl -V:nvsize
nvsize='8';
This information is also accessible within a Perl program via the Config module.
$ perl -MConfig -E'say $Config{nvsize}'
8
The size of NV cannot be changed after Perl is built.
You can force Perl to be built to use long double floats as follows when when building Perl:
sh Configure -Duselongdouble ...
-or-
perlbrew install -Duselongdouble ...
-or-
perlbrew install --ld ...
If you don't want to rebuild your perl or make a new one, you will need to use a module. I recommend Math::LongDouble as it provides access to native long double floats, and it does so as transparently as possible.
Another option is to use an arbitrary-precision library such as Math::BigFloat, but that will be slower that necessary if all you need is a long double.
bignum will overload all operators in the current scope to use arbitrary precision integers and floating point operations.
use bignum;
my $f = 123456789.123456789;
print "$f\n"; # 123456789.123456789
print $f + $f, "\n"; # 246913578.246913578
Behind the scenes, bignum turns all numeric constants into Math::BigInt and Math::BigNum objects as appropriate.
It's important to note that bignum is lexically scoped and does not effect the whole program. For example...
{
use bignum;
$f = 123456789.123456789; # $f is a Math::BigNum object
}
$g = 123456789.123456789; # $g is a regular NV
print "$f\n"; # 123456789.123456789
print "$g\n"; # 123456789.123457
# This will use Math::BigNum's addition method, but $g has already lost precision.
print $f + $g, "\n"; # 246913578.246913789
You can get a bit better performance out of this by using the Math::BigInt::GMP plugin to use the GNU Multiple Precision Arithmetic Library for some operations. You have to install that module first using the normal CPAN install process (you don't need GMP). Then tell bignum to use GMP.
use bignum lib => "GMP";

Perl Pack Unpack and Reverse Byte Order

I'm trying to write a script to find hex strings in a text file and convert them to their reverse byte order. The trouble I'm having is that some of the hex strings are 16 bit and some are 64 bits. I've used Perl's pack to pack and unpack the 16 bit hex numbers and that works fine, but the 64 bit does not.
print unpack("H*", (pack('I!', 0x20202032))). "\n"; #This works, gives 32202020
#This does not
print unpack("H*", (pack('I!', 0x4f423230313430343239303030636334))). "\n";
I've tried the second with the q and Q (where I get ffffffffffffffff). Am I approaching this all wrong?
As bit of background, I've got a multi-gigabyte pipe-delimited text file that has hex strings in reverse byte order as explained above. Also, the columns of the file are not standard; sometimes the hex strings appear in one column, and sometimes in another. I need to convert the hex strings to their reverse byte order.
Always use warnings;. If you do, you'll get the following message:
Integer overflow in hexadecimal number at scratch.pl line 8.
Hexadecimal number > 0xffffffff non-portable at scratch.pl line 8.
These can be resolved by use bigint; and by changing your second number declaration to hex('0x4f423230313430343239303030636334').
However, that number is still too large for pack 'I' to be able to handle.
Perhaps this can be done using simple string manipulation:
use strict;
use warnings;
my #nums = qw(
0x20202032
0x4f423230313430343239303030636334
);
for (#nums) {
my $rev = join '', reverse m/([[:xdigit:]]{2})/g;
print "$_ -> 0x$rev\n"
}
__END__
Outputs:
0x20202032 -> 0x32202020
0x4f423230313430343239303030636334 -> 0x3463633030303932343034313032424f
Or to handle digits of non-even length:
my $rev = $_;
$rev =~ s{0x\K([[:xdigit:]]*)}{
my $hex = $1;
$hex = "0$hex" if length($hex) % 2;
join '', reverse $hex =~ m/(..)/g;
}e;
print "$_ -> $rev\n"
To be pedantic, the hex numbers in your example are 32-bit and 128-bit long, not 16 and 64. If the longest one was only 64-bit long, you could successfully use the Q pack template as you supposed (provided hat your perl has been compiled to support 64-bit integers).
The pack/unpack solution can be used anyway (if with the addition of a reverse - you also have to remove the leading 0x from the hex strings or trim the last two characters from the results):
print unpack "H*", reverse pack "H*", $hex_string;
Example with your values:
perl -le 'print unpack "H*", reverse pack "H*", "4f423230313430343239303030636334"'
3463633030303932343034313032424f

Hexadecimal Computations in Perl

I have a string of ASCII characters. I convert this to hex string using the unpack function.
#! /usr/bin/perl
use strict;
use warnings;
my $str="hello";
my $value=unpack("H*",$str);
print $value,"\n";
**output:** 68656c6c6f
Now, lets say, I want to use this output as a string of hex bytes, read one byte at a time and perform some computation on it and store the output in another variable.
For instance,
#! /usr/bin/perl
use strict;
use warnings;
my $str="hello";
my $value=unpack("H*",$str);
my $num=0x12;
my $i=0;
while($i<length($value))
{
my $result.=(substr($value,$i,2)^$num);
$i+=2;
}
print $result,"\n";
**output:**
Argument "6c" isn't numeric in bitwise xor (^) at test.pl line 13.
Argument "6c" isn't numeric in bitwise xor (^) at test.pl line 13.
Argument "6f" isn't numeric in bitwise xor (^) at test.pl line 13.
8683202020
The output is incorrect and also there are several warnings.
If we take the first hex byte of the string, "hello" as an example:
68 xor 12 = 7A
However, the output shows it as 86. The output is incorrect and also I am not sure how
it got an output of 86.
What is the right way to do it?
If something is in hex, it is necessarily a string, since hex is a human-readable representation of a number. You don't want a string; you want a series of numbers, where each of those numbers is the numerical value of the char. You could use ord to get the number character by character, but unpack also provides the means:
my #bytes = unpack 'C*', $str;
Do the processing you want:
$_ ^= $num for #bytes;
And reconstitute the string:
$str = pack 'C*', #bytes;
The above three combined:
$str = pack 'C*', map $_ ^ $num, unpack 'C*', $str;
You can also do it as follows:
my $mask = chr($num) x length($str);
$str ^= $mask;

Is 999...9 a real number in Perl?

sub is_integer {
defined $_[0] && $_[0] =~ /^[+-]?\d+$/;
}
sub is_float {
defined $_[0] && $_[0] =~ /^[+-]?\d+(\.\d+)?$/;
}
For the code mentioned above, if we give input as 999999999999999999999999999999999999999999, it is giving output as not real number.
Why it is behaving like that?
I forgot to mention one more thing:
If I am using this code for $x as the above value:
if($x > 0 || $x <= 0 ) {
print "Real";
}
Output is real.
How is this possible?
$ perl -e 'print 999999999999999999999999999999999999999999'
1e+42
i.e. Perl uses scientific representation for this number and that is why your regexp doesn't match.
Use the looks_like_number function from Scalar::Util (which is a core module).
use Scalar::Util qw( looks_like_number );
say "Number" if looks_like_number 999999999999999999999999999999999999999999;
# above prints "Number"
Just to add one more thing. As others have explained, the number you are working with is out of range for a Perl integer (unless you are on a 140 bit machine). Therefore, the variable will be stored as a floating point number. Regular expressions operate on strings. Therefore, the number is converted to its string representation before the regular expression operates on it.
Others have explained what is going on: out of the box, Perl can't handle numbers that large without using scientific notation.
If you need to work with large numbers, take a look at bignum or its components, such as Math::BigInt. For example:
use strict;
use warnings;
use Math::BigInt;
my $big_str = '900000000000000000000000000000000000000';
my $big_num = Math::BigInt->new($big_str);
$big_num ++;
print "Is integer: $big_num\n" if is_integer($big_num);
sub is_integer {
defined $_[0] && $_[0] =~ /^[+-]?\d+$/;
}
Also, you may want to take a look at bignum in the Perl documentation.