I'm trying my best to decipher some Perl code and convert it into C# code so I can use it with a larger program. I've been able to get most of it converted, but am having trouble with the following method:
sub dynk {
my ($t, $s, $v, $r) = (unpack("b*", $_[0]), unpack("b*", pack("v",$_[1])));
$v^=$t=substr($t,$r=$_*$_[($_[1]>>$_-1&1)+2]).substr($t,0,$r)^$s for (1..16);
pack("b*", $v);
}
It is called like:
$sid = 0;
$rand = pack("H*", 'feedfacedeadbeef1111222233334444');
$skey = dynk($rand, $sid, 2, 3) ^ dynk(substr($dbuf, 0, 16), $sid, -1, -4);
I understand most of it except for this section:
$_*$_[($_[1]>>$_-1&1)+2]
I'm not sure how $_ is being used in that context? If someone could explain that, I think I can get the rest.
pack and unpack take a pattern, and some data, and transform this data according to the pattern. For example, pack "H*", "466F6F" treats the data as a hex string of arbitrary length, and decodes it to the bytes it represents. Here: Foo. The unpack function does the reverse, and extracts data from a binary representation to a certain format.
The "b*" pattern stands produces a bit string – unpack "b*", "42" is "0010110001001100".
The v represents one little-endian 16-bit integer.
The Perl is rather obfuscated. Here is a rewrite that simplifies some aspects.
sub dynk {
# Extract arguments: A salt, another parameter, and then two ints that determine rotation.
my ($initial, $sid, $rot_a, $rot_b) = #_;
# Unpack the initial value to a bitstring
my $temp = unpack("b*", $initial);
# Unpack the 16-bit number $sid to a bitstring
my $sid_bits = unpack("b*", pack("v", $sid));
my $v; # an accumulator
# Loop through the 16 bits of our $sid
for my $bit_number (1..16) {
# Pick the $bit_number-th bit from the $sid as an index for the data
my $bit_value = substr($sid_bits, $bit_number-1, 1);
# calculate rotation from one data argument
my $rotation = $bit_number * ( $bit_value ? $rot_b : $rot_a );
# Rotate the $temp bitstring by $rotation bits
$temp = substr($temp, $rotation) . substr($temp, 0, $rotation);
# XOR the $temp with $sid_bits
$temp = $temp ^ $sid_bits;
# ... and XOR with the $v accumulator
$v = $v ^ $temp;
}
# Pack the bitstring back to binary data, return.
return pack("b*", $v);
}
This seems to be some sort of encryption or hashing. It mainly jumbles the first argument according to the following ones. The larger $sid is, the more extra parameters are used: at least one, at most 16. Each bit is used in turn as an index, thus only two extra parameters are used. The length of the first argument stays constant in this operation, but the output is at least two bytes long.
If one of the extra arguments is zero, no rotation takes place during that loop iteration. Unititializes arguments are considered to be zero.
Related
I don't understand why join changes the output of JSON::to_string in the following example:
#!/usr/bin/perl
use v5.26;
use Data::Dumper;
use JSON;
my #version = (1, 2, 3, 4);
say "version: ", join ".", #version; # comment this line out
$Data::Dumper::Terse = 1;
$Data::Dumper::Indent = 0;
say Dumper(\#version);
say to_json(\#version);
The output with the line containing join:
version: 1.2.3.4
[1,2,3,4]
["1","2","3","4"]
But commenting out the line with join the output of to_json suddenly shows integers instead of strings although the output of Data::Dumper is still the same:
[1,2,3,4]
[1,2,3,4]
When you stringify a number, the stringification is stored in the scalar along with the origin number. (You can see a demonstration at the bottom of my answer.)
When you numify a string, the numification is stored in the scalar along with the origin number.
This is an optimization since one often stringify or numify a scalar more than once.
This isn't a problem for Perl since Perl has coercing operators rather than polymorphic operators. But it puts the authors of JSON serializers in the difficult positions of either requiring additional information or guessing which of the values a scalar contains should be used.
You can force a number using $x = 0 + $x;.
You can force a string using $x = "$x";.
More detailed answer follows.
Perl is free to change internal format a scalar as it sees fit. This is usually done as part of modifying the scalar.
$x = 123; # $x contains a signed integer
$x += 0.1; # $x contains a float
$x = 2147483647; # $x contains a signed integer
++$x; # $x contains an unsigned integer (on a build with 32-bit ints)
$x = "123"; # $x contains a downgraded string
$x += 0; # $x contains a signed integer
$x = "abc"; # $x contains a downgraded string
$x .= "\x{2660}"; # $x contains an upgraded string
But sometimes, Perl adds a second value to an scalar as an optimization.
$x = 123; # $x contains a signed integer
$x * 0.1; # $x contains a signed integer AND a float
$x = 123; # $x contains a signed integer
"$x"; # $x contains a signed integer AND a downgraded string
$x = "123"; # $x contains a downgraded string
$x+0; # $x contains a signed integer AND a downgraded string
These aren't the only double (or triple) vars you'll encounter.
my $x = !!0; # $x contains a signed integer AND a float AND a downgraded string
"$!"; # $! contains a float (not a signed integer?!) AND a downgraded string
This isn't a problem in Perl because we use type-coercing operators (e.g. == works on numbers, eq works on strings). But many other languages rely on polymorphic operators (e.g. == can be used to compare strings and to compare numbers).[1]
But it does present a problem for JSON serializers which are forced to assign a single type to a scalar. If $x contains both a string a number, which one should be used?
If the scalar is the result of stringification, using the number would be ideal, but if the scalar is the result of numification, the string would be ideal. There's no way to tell which of these origins pertains to a scalar (if any), so the module's author was left with a tough choice.
Ideally, they would have provided a different interface, but that could have added complexity and a performance penalty.
You can view the internals of a scalar using Devel::Peek's Dump. The relevant line is the FLAGS line.
IOK without IsUV: contains a signed integer
IOK with IsUV: contains an unsigned integer
NOK: contains a float
POK without UTF8: contains a downgraded string
POK with UTF8: contains an upgraded string
ROK: contains a reference
$ perl -MDevel::Peek -e'$x=123; Dump($x); "$x"; Dump($x);' 2>&1 |
perl -M5.014 -ne'next if !/FLAGS/; say join ",", /\b([INPR]OK|IsUV|UTF8)/g'
IOK
IOK,POK
$ perl -MDevel::Peek -e'$x="123"; Dump($x); 0+$x; Dump($x);' 2>&1 |
perl -M5.014 -ne'next if !/FLAGS/; say join ",", /\b([INPR]OK|IsUV|UTF8)/g'
POK
IOK,POK
Well, Perl doesn't have separate operators for the different numeric types, which can cause issues (e.g. -0 exists a float, but not as an int), but these problems are seldom encountered.
Another issue is that the stringification of floats often results in a loss of information.
This is one of the very few times where you must maintain data purity in Perl. Once you create a variable of some type, you must never use it in a context of any other type. If you do need to, copy it to a new variable first to preserve the original.
use feature 'say';
use Data::Dumper;
use JSON;
my #version = (1, 2, 3, 4);
{ say "version: ", join ".", my #copy = #version; }
$Data::Dumper::Terse = 1;
$Data::Dumper::Indent = 0;
say Dumper(\#version);
say to_json(\#version);
Prints:
version: 1.2.3.4
[1,2,3,4]
[1,2,3,4]
I would also recommend using Cpanel::JSON::XS because this is one area where pedantism is called for! It tries pretty hard to get the data types right. It also has some discussion of the conversion issue.
HTH
My script generates some very very huge files, and I am trying to print/save the output in a binary format to reduce the file size as much as possible!
Each time that script generates five values, like:
$a1 = 1.64729
$a2 = 4.33329
$a3 = 3.55724
$a4 = 1.45759
$a5 = 7.474700
It prints in the output like:
A:1.64729,4.33329,3.55724,1.45759,7.474700
I am not sure whether this is the best way, but I thought to pack each row when it is printing to the output! I used pack/unpack built-in function in Perl!
I had a look at perldoc, but I did not understand which format specifiers were proper (???)!
#!/usr/bin/perl
...
#A = ($a1,$a2,$a3,$a4,$a5);
print pack ("???", ("A:", join(",", map { sprintf "%.1f", $_ } #A)), "\n";
If you compress the file (instead of trying to write binary bytes) you will get a small file. That's because your entire file will have mostly the ten digit characters, plus a decimal point, and a comma.
You can compress a file as you write it via IO::Zlib. This will use either the Zlib library, or the gzip command.
However, if you want to use pack, go ahead. Get the Camel Book which gives much clearer documentation than the standard Perldoc.
It's not all that difficult:
my $output = "A:1.64729,4.33329,3.55724,1.45759,7.474700";
$output =~ s/^A://; #Remove the 'A:'
my #numbers = split /,/, $output # Make into an array
my $packed = pack "d5", #numbers; # Pack five inputs as floating point numbers
say join ",", "d5", $packed; # Unpacks those five decimal encoded numbers
You'll probably have to use syswrite and sysread since aren't reading and writing strings. This is unbuffered reading and writing, and you have to specify the number of bytes you're reading or writing.
One more thing: If you know where the decimal point is in the number (that is, it's always a number between 1 and up to 10) you can convert the number into an integer which will allow you to pack the number into an even smaller number of bytes:
my $output = "A:1.64729,4.33329,3.55724,1.45759,7.474700";
$output =~ s/^A://; #Remove the 'A:'
$output =~ s/,//g; #Remove all the decimal points
my #numbers = split /,/, $output # Make into an array
my $packed = pack "L5", #numbers; # Pack five inputs as unsigned long numbers
Read a file that contains an address and a data, like below:
#0, 12345678
#1, 5a5a5a5a
...
My aim is to read the address and the data. Consider the data I read is in hex format, and then I need to unpack them to binary number.
So 12345678 would become 00010010001101000101011001111000
Then, I need to further unpack the transferred binary number to another level.
So it becomes, 00000000000000010000000000010000000000000001000100000001000000000000000100000001000000010001000000000001000100010001000000000000
They way I did is like below
while(<STDIN>) {
if (/\#(\S+)\s+(\S+)/) {
$addr = $1;
$data = $2;
$mem{$addr} = ${data};
}
}
foreach $key (sort {$a <=> $b} (keys %mem)) {
my $str = unpack ('B*', pack ('H*',$mem{$key}));
my $str2 = unpack ('B*', pack ('H*', $str));
printf ("#%x ", $key);
printf ("%s",$str2);
printf ("\n");
}
It works, however, my next step is to do some numeric operation on the transferred bits.
Such as bitwise or and shifting. I tried << and | operator, both are for numbers, not strings. So I don't know how to solve this.
Please leave your comments if you have better ideas. Thanks.
You can employ Bit::Vector module from metaCPAN
use strict;
use warnings;
use Bit::Vector;
my $str = "1111000011011001010101000111001100010000001111001010101000111010001011";
printf "orig str: %72s\n", $str;
#only 72 bits for better view
my $vec = Bit::Vector->new_Bin(72,$str);
printf "vec : %72s\n", $vec->to_Bin();
$vec->Move_Left(2);
printf "left 2 : %72s\n", $vec->to_Bin();
$vec->Move_Right(4);
printf "right 4 : %72s\n", $vec->to_Bin();
prints:
orig str: 1111000011011001010101000111001100010000001111001010101000111010001011
vec : 001111000011011001010101000111001100010000001111001010101000111010001011
left 2 : 111100001101100101010100011100110001000000111100101010100011101000101100
right 4 : 000011110000110110010101010001110011000100000011110010101010001110100010
If you need do some math with arbitrary precision, you can also use Math::BigInt or use bigint (http://perldoc.perl.org/bigint.html)
Hex and binary are text representation of numbers. Shifting and bit manipulations are numerical operations. You want a number, not text.
my $hex = '5a5a5a5a';
$num = hex($hex); # Convert to number.
$num >>= 1; # Manipulate the number.
$hex = sprintf('%08X', $num); # Convert back to hex.
In a comment, you mention you want to deal with 256 bit numbers. The native numbers don't support that, but you can use Math::BigInt.
My final solution of this is forget about treat them as numbers, just treat them as string . I use substring and string concentration instead of shift. Then for the or operation , I just add each bit of the string, if it's 0 the result is 0, else is 1.
It may not be the best way to solve this problem. But that's the way I finally used.
I have a problem understanding and using the 'vec' keyword.
I am reading a logpacket in which values are stored in little endian hexadecimal. In my code, I have to unpack the different bytes into scalars using the unpack keyword.
Here's an example of my problem:
my #hexData1 = qw(50 65);
my $data = pack ('C*', #hexData1);
my $x = unpack("H4",$data); # At which point the hexadecimal number became a number
print $x."\n";
#my $foo = sprintf("%x", $foo);
print "$_-> " . vec("\x65\x50", $_, 1) . ", " for (0..15); # This works.
print "\n";
But I want to use the above statement in the way below. I don't want to send a string of hexadecimal in quotes. I want to use the scalar array of hex $x. But it won't work. How do I convert my $x to a hexadecimal string. This is my requirement.
print "$_-> " . vec($x, $_, 1).", " for (0..15); # This doesn't work.
print "\n";
My final objective is to read the third bit from the right of the two byte hexadecimal number.
How do I use the 'vec' command for that?
You are making the mistake of unpacking $data into $x before using it in a call to vec. vec expects a string, so if you supply a number it will be converted to a string before being used. Here's your code
my #hexData1 = qw(50 65);
my $data= pack ('C*', #hexData1);
The C pack format uses each value in the source list as a character code. It is the same as calling chr on each value and concatenating them. Unfortunately your values look like decimal, so you are getting chr(50).chr(65) or "2A". Since your values are little-endian, what you want is chr(0x65).chr(0x50) or "\x65\x50", so you must write
my $data= pack ('(H2)*', reverse #hexData1);
which reverses the list of data (to account for it being little-endian) and packs it as if it was a list of two-digit hex strings (which, fortunately, it is).
Now you have done enough. As I say, vec expects a string so you can write
print join ' ', map vec($data, $_, 1), 0 .. 15;
print "\n";
and it will show you the bits you expect. To extract the the 3rd bit from the right (assuming you mean bit 13, where the last bit is bit 15) you want
print vec $data, 13, 1;
First, get the number the bytes represent.
If you start with "\x50\x65",
my $num = unpack('v', "\x50\x65");
If you start with "5065",
my $num = unpack('v', pack('H*', "5065"));
If you start with "50","65",
my $num = unpack('v', pack('H*', join('', "50","65"));
Then, extract the bit you want.
If you want bit 10,
my $bit = ($num >> 10) & 1;
If you want bit 2,
my $bit = ($num >> 2) & 1;
(I'm listing a few possibilities because it's not clear to me what you want.)
I've been having this problem in Perl for a few days now, and after scouring countless man pages, perldocs and googling too many search terms, hopefully someone here can help me out.
I am given two strings which represent hex values, i.e. "FFFF", not the Perl hex number 0xFFFF. Given two of these strings, I wish to convert them to binary form, perform a bitwise AND of the two, then take the output of this and examine each bit from LSB to MSB.
I have two problems right now; converting the hex string into a hex number, and shifting
the result of the bitwise AND.
For converting the hex string into a hex number, I've tried the following approaches which don't seem to work when I print them out to examine:
$a = unpack("H*", pack("N*", $a));
$a = sprintf("%H", $a);
Using a 'print' to examine each of these does not show a correct value, nor does using 'sprintf' either...
The second problem I have occurs after I perform a bitwise AND, and I want to examine each bit by shifting right by 1. To avoid the previous problem, I used actual Perl hex numbers instead of hex strings (0xffff instead of "ffff"). If I try to perform a shift right as follows:
#Convert from hex number to binary number
$a = sprintf("%B", $a);
$b = sprintf("%B", $b);
$temp = pack("B*", $a) & pack("B*", $b);
$output = unpack("B*", $temp);
At this point everything looks fine, and using a 'print' I can see that the values of the AND operation look right, but when I try to shift as follows:
$output = pack("B*", $output);
$output = $output >> 1;
$output = unpack("B*", $output);
The resulting value I get is in binary form but not correct.
What is the correct way of performing this kind of operation?
There's no such thing as a "hex number". A number is a number, a hexadecimal representation of a number is just that - a representation.
Just turn it into a number and use bitwise and.
my $num = (hex $a) & (hex $b);
print ($num & 1, "\n") while ($num >>= 1)