Related
I have an array of hex numbers that I'd like to convert to binary numbers, the problem is, in my code it removes the leading 0's for things like 0,1,2,3. I need these leading 0's to process in a future section of my code. Is there an easy way to convert Hex to Binary and keep my leading 0's in perl?
use strict;
use warnings;
my #binary;
my #hex = ('ABCD', '0132', '2211');
foreach my $h(#hex){
my $bin = sprintf( "%b", hex($h));
push #binary, $bin;
}
foreach (#binary){
print "$_\n";
}
running the code gives me
1010101111001101
100110010
10001000010001
Edit: Found a similar answer using pack and unpack, replaced
sprint( "%b", hex($h));
with
unpack( 'B*', pack('H*' ($h))
You can specify the width of the output in sprintf or printf by putting the number between the % and the format character like this.
printf "%16b\n",hex("0132");
and by preceding the number with 0, make it pad the result with 0s like this
printf "%016b\n",hex("0132");
the latter giving the result of
0000000100110010
But this is all covered in the documentation for those functions.
This solution uses the length of the hex repesentation to determine the length of the binary representation:
for my $num_hex (#nums_hex) {
my $num = hex($num_hex);
my $num_bin = sprintf('%0*b', length($num_hex)*4, $num);
...
}
I tried the following in a Perl script:
$b = 19999999999999999 % 10000000000000000;
print "$b\n";
It incorrectly outputted 0.
Then I found an answer saying to use bignum:
use bignum;
$b = 19999999999999999 % 10000000000000000;
print "$b\n";
It correctly outputted 9999999999999999.
But bignum just converts all integer constants into a Math::BigInt. So I tried the following which should be the same as using bignum:
use Math::BigInt;
$b = Math::BigInt->new(19999999999999999) % Math::BigInt->new(10000000000000000);
print "$b\n";
But that incorrectly outputted 0. Am I doing something wrong with Math::BigInt?
You're still using native Perl numbers first and then converting them to Math::BigInt objects. Try this instead:
my $x = Math::BigInt->new('19999999999999999') % Math::BigInt->new('10000000000000000');
Quoting from perldoc Math::BigInt:
Input given as scalar numbers might lose precision. Quote your input to ensure that no digits are lost:
$x = Math::BigInt->new( 56789012345678901234 ); # bad
$x = Math::BigInt->new('56789012345678901234'); # good
(Also, don't use $b outside of sort and similar routines.)
How do I fix this code so that 1.1 + 2.2 == 3.3? What is actually happening here that's causing this behavior? I'm vaguely familiar with rounding problems and floating point math, but I thought that applied to division and multiplication only and would be visible in the output.
[me#unixbox1:~/perltests]> cat testmathsimple.pl
#!/usr/bin/perl
use strict;
use warnings;
check_math(1, 2, 3);
check_math(1.1, 2.2, 3.3);
sub check_math {
my $one = shift;
my $two = shift;
my $three = shift;
if ($one + $two == $three) {
print "$one + $two == $three\n";
} else {
print "$one + $two != $three\n";
}
}
[me#unixbox1:~/perltests]> perl testmathsimple.pl
1 + 2 == 3
1.1 + 2.2 != 3.3
Edit:
Most of the answers thus far are along the lines of "it's a floating point problem, duh" and are providing workarounds for it. I already suspect that to be the problem. How do I demonstrate it? How do I get Perl to output the long form of the variables? Storing the $one + $two computation in a temp variable and printing it doesn't demonstrate the problem.
Edit:
Using the sprintf technique demonstrated by aschepler, I'm now able to "see" the problem. Further, using bignum, as recommended by mscha and rafl, fixes the problem of the comparison not being equal. However, the sprintf output still indicates that the numbers aren't "correct". That's leaving a modicum of doubt about this solution.
Is bignum a good way to resolve this? Are there any possible side effects of bignum that we should look out for when integrating this into a larger, existing, program?
See What Every Computer Scientist Should Know About Floating-Point Arithmetic.
None of this is Perl specific: There are an uncountably infinite number of real numbers and, obviously, all of them cannot be represented using only a finite number of bits.
The specific "solution" to use depends on your specific problem. Are you trying to track monetary amounts? If so, use the arbitrary precision numbers (use more memory and more CPU, get more accurate results) provided by bignum. Are you doing numeric analysis? Then, decide on the precision you want to use, and use sprintf (as shown below) and eq to compare.
You can always use:
use strict; use warnings;
check_summation(1, $_) for [1, 2, 3], [1.1, 2.2, 3.3];
sub check_summation {
my $precision = shift;
my ($x, $y, $expected) = #{ $_[0] };
my $result = $x + $y;
for my $n ( $x, $y, $expected, $result) {
$n = sprintf('%.*f', $precision, $n);
}
if ( $expected eq $result ) {
printf "%s + %s = %s\n", $x, $y, $expected;
}
else {
printf "%s + %s != %s\n", $x, $y, $expected;
}
return;
}
Output:
1.0 + 2.0 = 3.0
1.1 + 2.2 = 3.3
"What Every Computer Scientist Should Know About Floating-Point Arithmetic"
Basically, Perl is dealing with floating-point numbers, while you are probably expecting it to use fixed-point. The simplest way to handle this situation is to modify your code so that you are using whole integers everywhere except, perhaps, in a final display routine. For example, if you're dealing with USD currency, store all dollar amounts in pennies. 123 dollars and 45 cents becomes "12345". That way there is no floating point ambiguity during add and subtract operations.
If that's not an option, consider Matt Kane's comment. Find a good epsilon value and use it whenever you need to compare values.
I'd venture to guess that most tasks don't really need floating point, however, and I'd strongly suggest carefully considering whether or not it is the right tool for your task.
A quick way to fix floating points is to use bignum. Simply add a line
use bignum;
to the top of your script.
There are performance implications, obviously, so this may not be a good solution for you.
A more localized solution is to use Math::BigFloat explicitly where you need better accuracy.
From The Floating-Point Guide:
Why don’t my numbers, like 0.1 + 0.2 add up to a nice round 0.3, and
instead I get a weird result like
0.30000000000000004?
Because internally, computers use a
format (binary floating-point) that
cannot accurately represent a number
like 0.1, 0.2 or 0.3 at all.
When the code is compiled or
interpreted, your “0.1” is already
rounded to the nearest number in that
format, which results in a small
rounding error even before the
calculation happens.
What can I do to avoid this problem?
That depends on what kind of
calculations you’re doing.
If you really need your results to add up exactly, especially when you
work with money: use a special decimal
datatype.
If you just don’t want to see all those extra decimal places: simply
format your result rounded to a fixed
number of decimal places when
displaying it.
If you have no decimal datatype available, an alternative is to work
with integers, e.g. do money
calculations entirely in cents. But
this is more work and has some
drawbacks.
Youz could also use a "fuzzy compare" to determine whether two numbers are close enough to assume they'd be the same using exact math.
To see precise values for your floating-point scalars, give a big precision to sprintf:
print sprintf("%.60f", 1.1), $/;
print sprintf("%.60f", 2.2), $/;
print sprintf("%.60f", 3.3), $/;
I get:
1.100000000000000088817841970012523233890533447265625000000000
2.200000000000000177635683940025046467781066894531250000000000
3.299999999999999822364316059974953532218933105468750000000000
Unfortunately C99's %a conversion doesn't seem to work. perlvar mentions an obsolete variable $# which changes the default format for printing a number, but it breaks if I give it a %f, and %g refuses to print "non-significant" digits.
abs($three - ($one + $two)) < $some_Very_small_number
Use sprintf to convert your variable into a formatted string, and then compare the resulting string.
# equal( $x, $y, $d );
# compare the equality of $x and $y with precision of $d digits below the decimal point.
sub equal {
my ($x, $y, $d) = #_;
return sprintf("%.${d}g", $x) eq sprintf("%.${d}g", $y);
}
This kind of problem occurs because there is no perfect fixed-point representation for your fractions (0.1, 0.2, etc). So the value 1.1 and 2.2 are actually stored as something like 1.10000000000000...1 and 2.2000000....1, respectively (I am not sure if it becomes slightly bigger or slightly smaller. In my example I assume they become slightly bigger). When you add them together, it becomes 3.300000000...3, which is larger than 3.3 which is converted to 3.300000...1.
Number::Fraction lets you work with rational numbers (fractions) instead of decimals, something like this (':constants' is imported to automatically convert strings like '11/10' into Number::Fraction objects):
use strict;
use warnings;
use Number::Fraction ':constants';
check_math(1, 2, 3);
check_math('11/10', '22/10', '33/10');
sub check_math {
my $one = shift;
my $two = shift;
my $three = shift;
if ($one + $two == $three) {
print "$one + $two == $three\n";
} else {
print "$one + $two != $three\n";
}
}
which prints:
1 + 2 == 3
11/10 + 11/5 == 33/10
Read a file that contains an address and a data, like below:
#0, 12345678
#1, 5a5a5a5a
...
My aim is to read the address and the data. Consider the data I read is in hex format, and then I need to unpack them to binary number.
So 12345678 would become 00010010001101000101011001111000
Then, I need to further unpack the transferred binary number to another level.
So it becomes, 00000000000000010000000000010000000000000001000100000001000000000000000100000001000000010001000000000001000100010001000000000000
They way I did is like below
while(<STDIN>) {
if (/\#(\S+)\s+(\S+)/) {
$addr = $1;
$data = $2;
$mem{$addr} = ${data};
}
}
foreach $key (sort {$a <=> $b} (keys %mem)) {
my $str = unpack ('B*', pack ('H*',$mem{$key}));
my $str2 = unpack ('B*', pack ('H*', $str));
printf ("#%x ", $key);
printf ("%s",$str2);
printf ("\n");
}
It works, however, my next step is to do some numeric operation on the transferred bits.
Such as bitwise or and shifting. I tried << and | operator, both are for numbers, not strings. So I don't know how to solve this.
Please leave your comments if you have better ideas. Thanks.
You can employ Bit::Vector module from metaCPAN
use strict;
use warnings;
use Bit::Vector;
my $str = "1111000011011001010101000111001100010000001111001010101000111010001011";
printf "orig str: %72s\n", $str;
#only 72 bits for better view
my $vec = Bit::Vector->new_Bin(72,$str);
printf "vec : %72s\n", $vec->to_Bin();
$vec->Move_Left(2);
printf "left 2 : %72s\n", $vec->to_Bin();
$vec->Move_Right(4);
printf "right 4 : %72s\n", $vec->to_Bin();
prints:
orig str: 1111000011011001010101000111001100010000001111001010101000111010001011
vec : 001111000011011001010101000111001100010000001111001010101000111010001011
left 2 : 111100001101100101010100011100110001000000111100101010100011101000101100
right 4 : 000011110000110110010101010001110011000100000011110010101010001110100010
If you need do some math with arbitrary precision, you can also use Math::BigInt or use bigint (http://perldoc.perl.org/bigint.html)
Hex and binary are text representation of numbers. Shifting and bit manipulations are numerical operations. You want a number, not text.
my $hex = '5a5a5a5a';
$num = hex($hex); # Convert to number.
$num >>= 1; # Manipulate the number.
$hex = sprintf('%08X', $num); # Convert back to hex.
In a comment, you mention you want to deal with 256 bit numbers. The native numbers don't support that, but you can use Math::BigInt.
My final solution of this is forget about treat them as numbers, just treat them as string . I use substring and string concentration instead of shift. Then for the or operation , I just add each bit of the string, if it's 0 the result is 0, else is 1.
It may not be the best way to solve this problem. But that's the way I finally used.
Is there a simple way in Perl that will allow me to determine if a given variable is numeric? Something along the lines of:
if (is_number($x))
{ ... }
would be ideal. A technique that won't throw warnings when the -w switch is being used is certainly preferred.
Use Scalar::Util::looks_like_number() which uses the internal Perl C API's looks_like_number() function, which is probably the most efficient way to do this.
Note that the strings "inf" and "infinity" are treated as numbers.
Example:
#!/usr/bin/perl
use warnings;
use strict;
use Scalar::Util qw(looks_like_number);
my #exprs = qw(1 5.25 0.001 1.3e8 foo bar 1dd inf infinity);
foreach my $expr (#exprs) {
print "$expr is", looks_like_number($expr) ? '' : ' not', " a number\n";
}
Gives this output:
1 is a number
5.25 is a number
0.001 is a number
1.3e8 is a number
foo is not a number
bar is not a number
1dd is not a number
inf is a number
infinity is a number
See also:
perldoc Scalar::Util
perldoc perlapi for looks_like_number
The original question was how to tell if a variable was numeric, not if it "has a numeric value".
There are a few operators that have separate modes of operation for numeric and string operands, where "numeric" means anything that was originally a number or was ever used in a numeric context (e.g. in $x = "123"; 0+$x, before the addition, $x is a string, afterwards it is considered numeric).
One way to tell is this:
if ( length( do { no warnings "numeric"; $x & "" } ) ) {
print "$x is numeric\n";
}
If the bitwise feature is enabled, that makes & only a numeric operator and adds a separate string &. operator, you must disable it:
if ( length( do { no if $] >= 5.022, "feature", "bitwise"; no warnings "numeric"; $x & "" } ) ) {
print "$x is numeric\n";
}
(bitwise is available in perl 5.022 and above, and enabled by default if you use 5.028; or above.)
Check out the CPAN module Regexp::Common. I think it does exactly what you need and handles all the edge cases (e.g. real numbers, scientific notation, etc). e.g.
use Regexp::Common;
if ($var =~ /$RE{num}{real}/) { print q{a number}; }
Usually number validation is done with regular expressions. This code will determine if something is numeric as well as check for undefined variables as to not throw warnings:
sub is_integer {
defined $_[0] && $_[0] =~ /^[+-]?\d+$/;
}
sub is_float {
defined $_[0] && $_[0] =~ /^[+-]?\d+(\.\d+)?$/;
}
Here's some reading material you should look at.
A simple (and maybe simplistic) answer to the question is the content of $x numeric is the following:
if ($x eq $x+0) { .... }
It does a textual comparison of the original $x with the $x converted to a numeric value.
Not perfect, but you can use a regex:
sub isnumber
{
shift =~ /^-?\d+\.?\d*$/;
}
A slightly more robust regex can be found in Regexp::Common.
It sounds like you want to know if Perl thinks a variable is numeric. Here's a function that traps that warning:
sub is_number{
my $n = shift;
my $ret = 1;
$SIG{"__WARN__"} = sub {$ret = 0};
eval { my $x = $n + 1 };
return $ret
}
Another option is to turn off the warning locally:
{
no warnings "numeric"; # Ignore "isn't numeric" warning
... # Use a variable that might not be numeric
}
Note that non-numeric variables will be silently converted to 0, which is probably what you wanted anyway.
rexep not perfect... this is:
use Try::Tiny;
sub is_numeric {
my ($x) = #_;
my $numeric = 1;
try {
use warnings FATAL => qw/numeric/;
0 + $x;
}
catch {
$numeric = 0;
};
return $numeric;
}
Try this:
If (($x !~ /\D/) && ($x ne "")) { ... }
I found this interesting though
if ( $value + 0 eq $value) {
# A number
push #args, $value;
} else {
# A string
push #args, "'$value'";
}
Personally I think that the way to go is to rely on Perl's internal context to make the solution bullet-proof. A good regexp could match all the valid numeric values and none of the non-numeric ones (or vice versa), but as there is a way of employing the same logic the interpreter is using it should be safer to rely on that directly.
As I tend to run my scripts with -w, I had to combine the idea of comparing the result of "value plus zero" to the original value with the no warnings based approach of #ysth:
do {
no warnings "numeric";
if ($x + 0 ne $x) { return "not numeric"; } else { return "numeric"; }
}
You can use Regular Expressions to determine if $foo is a number (or not).
Take a look here:
How do I determine whether a scalar is a number
There is a highly upvoted accepted answer around using a library function, but it includes the caveat that "inf" and "infinity" are accepted as numbers. I see some regex stuff for answers too, but they seem to have issues. I tried my hand at writing some regex that would work better (I'm sorry it's long)...
/^0$|^[+-]?[1-9][0-9]*$|^[+-]?[1-9][0-9]*(\.[0-9]+)?([eE]-?[1-9][0-9]*)?$|^[+-]?[0-9]?\.[0-9]+$|^[+-]?[1-9][0-9]*\.[0-9]+$/
That's really 5 patterns separated by "or"...
Zero: ^0$
It's a kind of special case. It's the only integer that can start with 0.
Integers: ^[+-]?[1-9][0-9]*$
That makes sure the first digit is 1 to 9 and allows 0 to 9 for any of the following digits.
Scientific Numbers: ^[+-]?[1-9][0-9]*(\.[0-9]+)?([eE]-?[1-9][0-9]*)?$
Uses the same idea that the base number can't start with zero since in proper scientific notation you start with the highest significant bit (meaning the first number won't be zero). However, my pattern allows for multiple digits left of the decimal point. That's incorrect, but I've already spent too much time on this... you could replace the [1-9][0-9]* with just [0-9] to force a single digit before the decimal point and allow for zeroes.
Short Float Numbers: ^[+-]?[0-9]?\.[0-9]+$
This is like a zero integer. It's special in that it can start with 0 if there is only one digit left of the decimal point. It does overlap the next pattern though...
Long Float Numbers: ^[+-]?[1-9][0-9]*\.[0-9]+$
This handles most float numbers and allows more than one digit left of the decimal point while still enforcing that the higher number of digits can't start with 0.
The simple function...
sub is_number {
my $testVal = shift;
return $testVal =~ /^0$|^[+-]?[1-9][0-9]*$|^[+-]?[1-9][0-9]*(\.[0-9]+)?([eE]-?[1-9][0-9]*)?$|^[+-]?[0-9]?\.[0-9]+$|^[+-]?[1-9][0-9]*\.[0-9]+$/;
}
if ( defined $x && $x !~ m/\D/ ) {}
or
$x = 0 if ! $x;
if ( $x !~ m/\D/) {}
This is a slight variation on Veekay's answer but let me explain my reasoning for the change.
Performing a regex on an undefined value will cause error spew and will cause the code to exit in many if not most environments. Testing if the value is defined or setting a default case like i did in the alternative example before running the expression will, at a minimum, save your error log.