I want to generate random hex values and those values should not be repetitive
and it should be of 4 bytes (ie: 0x00000000 to 0xffffffff) and the display output
should contain leading zeros.
For example: if I get the value 1 it should not represented as 0x1 but 0x00000001.
I want a minimum of 100 random values. Please tell me: how can I do that in Perl?
To get a random number in the range 0 .. (2<<32)-1:
my $rand = int(rand(0x100000000));
To print it in hex with leading zeroes:
printf "%08x", $rand;
Do please note this from the Perl man page:
Note: If your rand function consistently returns numbers that
are too large or too small, then your version of Perl was probably compiled with the wrong number of RANDBITS
If that's a concern, do this instead:
printf "%04x%04x", int(rand(0x10000)), int(rand(0x10000));
Note, also, that this does nothing to prevent repetition, although to be honest the chance of a repeating 32 bit number in a 100 number sequence is pretty small.
If it's absolutely essential that you don't repeat, do something like this:
my (%a); # create a hash table for remembering values
foreach (0 .. 99) {
my $r;
do {
$r = int(rand(0x100000000));
} until (!exists($a{$r})); # loop until the value is not found
printf "%08x\n", $r; # print the value
$a{$r}++; # remember that we saw it!
}
For what it's worth, this algorithm shouldn't be used if the range of possible values is less than (or even near to) the number of values required. That's because the random number generator loop will just repeatedly pull out numbers that were already seen.
However in this case where the possible range is so high (2^32) and the number of values wanted so low it'll work perfectly. Indeed with a range this high it's about the only practical algorithm.
perl -e 'printf "%08X\n", int rand 0xFFFFFFFF for 1 .. 100'
Alnitak explained it, but here's a much simpler implementation. I'm not sure how everyone starting reaching for do {} while since that's a really odd choice:
my $max = 0xFFFF_FFFF;
my( %Seen, #numbers );
foreach ( 1 .. 100 )
{
my $rand = int rand( $max + 1 );
redo if $Seen{$rand}++;
push #numbers, $rand;
}
print join "\n", map { sprintf "0x%08x", $_ } #numbers;
Also, as Alnitak pointed out, if you are generating a lot of numbers, that redo might cycle many, many times.
These will only be pseudorandom numbers, but you're not really asking for real random number anyway. That would involve possible repetition. :)
use LWP::Simple "get";
use List::MoreUtils "uniq";
print for uniq map { s/\t//, "0x$_" } split /^/, LWP::Simple::get('http://www.random.org/integers/?num=220&min=0&max=65535&col=2&base=16&format=plain&rnd=date.2009-12-14');
Adjust the url (see the form on http://www.random.org/integers/?mode=advanced) to not always return the same list. There is a minuscule chance of not returning at least 100 results.
Note that this answer is intentionally "poor" as a comment on the poor question. It's not a single question, it's a bunch all wrapped up together, all of which I'd bet have existing answers already (how do I generate a random number in range x, how do I format a number as a hex string with 0x and 0-padding, how do I add only unique values into a list, etc.). It's like asking "How do I write a webserver in Perl?" Without guessing what part the questioner really wants an answer to, you either have to write a tome for a response, or say something like:
perl -MIO::All -e 'io(":80")->fork->accept->(sub { $_[0] < io(-x $1 ? "./$1 |" : $1) if /^GET \/(.*) / })'
To get a random integer:
int(rand(0x10000000))
To format it as 8 hexadecimal digits:
printf "%08x", int(rand(0x10000000))
Related
I would like to normalize the variable from ie. 00000000.1, to 0.1 using Perl
my $number = 000000.1;
$number =\~ s/^0+(\.\d+)/0$1/;
Is there any other solution to normalize floats lower than 1 by removing upfront zeros than using regex?
When I try to put those kind of numbers into an example function below
test(00000000.1, 0000000.025);
sub test {
my ($a, $b) = #_;
print $a, "\n";
print $b, "\n";
print $a + $b, "\n";
}
I get
01
021
22
which is not what is expected.
A number with leading zeros is interpreted as octal, e.g. 000000.1 is 01. I presume you have a string as input, e.g. my $number = "000000.1". With this your regex is:
my $number = "000000.1";
$number =~ s/^0+(?=0\.\d+)//;
print $number;
Output:
0.1
Explanation of regex:
^0+ -- 1+ 0 digits
(?=0\.\d+) -- positive lookahead for 0. followed by digits
Learn more about regex: https://twiki.org/cgi-bin/view/Codev/TWikiPresentation2018x10x14Regex
Simplest way, force it to be treated as a number and it will drop the leading zeros since they are meaningless for decimal numbers
my $str = '000.1';
...
my $num = 0 + $str;
An example,† to run from the command-line:
perl -wE'$n = shift; $n = 0 + $n; say $n' 000.1
Prints 0.1
Another, more "proper" way is to format that string ('000.1' and such) using sprintf. Then you do need to make a choice about precision, but that is often a good idea anyway
my $num = sprintf "%f", $str; # default precision
Or, if you know how many decimal places you want to keep
my $num = sprintf "%.3f", $str;
† The example in the question is really invalid. An unquoted string of digits which starts with a zero (077, rather than '077') would be treated as an octal number except that the decimal point (in 000.1) renders that moot as octals can't be fractional; so, Perl being Perl, it is tortured into a number somehow, but possibly yielding unintended values.
I am not sure how one could get an actual input like that. If 000.1 is read from a file or from the command-line or from STDIN ... it will be a string, an equivalent of assigning '000.1'
See Scalar value constructors in perldata, and for far more detail, perlnumber.
As others have noted, in Perl, leading zeros produce octal numbers; 10 is just a decimal number ten but 010 is equal to decimal eight. So yeah, the numbers should be in quotes for the problem to make any sense.
But the other answers don’t explain why the printed results look funny. Contrary to Peter Thoeny’s comment and zdim’s answer, there is nothing ‘invalid’ about the numbers. True, octals can’t be floating point, but Perl does not strip the . to turn 0000000.025 into 025. What happens is this:
Perl reads the run of zeros and recognises it as an octal number.
Perl reads the dot and parses it as the concatenation operator.
Perl reads 025 and again recognises it as an octal number.
Perl coerces the operands to strings, i.e. the decimal value of the numbers in string form; 0000000 is, of course, '0' and 025 is '21'.
Perl concatenates the two strings and returns the result, i.e. '021'.
And without error.
(As an exercise, you can check something like 010.025 which, for the same reason, turns into '821'.)
This is why $a and $b are each printed with a leading zero. Also note that, to evaluate $a + $b, Perl coerces the strings to numbers, but since leading zeros in strings do not produce octals, '01' + '021' is the same as '1' + '21', returning 22.
Is there some easy way (without sprintf and, of course, printf) to get a list of (001, 002, ... 100) in Perl?
In bash it was something like seq -w 1 100. What about Perl?
you mean like this?
for ('001'..'100') {
print "$_\n";
}
.. in a range, returns a list of values counting by ones, from the left value to the right value.
For more details about how to use range, please refer to:
Perldoc range operator and this link
Printf was created for problems like this. Using it will help you get the answers you want faster.
foreach my $number ( 1 .. 100 ) {
printf "%03d\n", $number;
}
The % is the "begin a format sequence"
The 0 is "leading zeros"
The 3 is "three digits minimum"
The d is "treat the parameter as digits (integer)"
I have this one-line Unix shell script
for i in 1 2 3 4; do sed "$(tr -dc '0-9' < /dev/urandom | fold -w 5 |
awk '$0>=35&&$0<=65570' | head -1)q;d" "$0"; done | perl -p00e
's/\n(?!\Z)/ /g'
The script has 65K words in it, one per line, from line 35 to 65570. The code and the data are in the same file.
This script outputs 4 space-separated random words from this list with a newline at the end. For example
first fourth third second
How can I make this one-liner much shorter with Perl, keeping the
tr -dc '0-9' < /dev/urandom
part?
Keeping it is important since it provides Cryptographically Secure Pseudo-Random Numbers (CSPRNs) for all Unix OSs. Of course, if Perl can get numbers from /dev/urandom then the tr can be replaced with Perl too, but the numbers from urandom need to stay.
For convenience, I shared the base script with 65K words
65kwords.txt
or
65kwords.txt
Please use only core modules. It would be used for generating "human memorable passwords".
Later, the (hashing) iteration count, where we would use this to store the passwords would be extremely high, so brute-force would be very slow, even with many many GPUs/FPGAs.
You mention needing a CSPRN, which makes this a non trivial exercise - if you need cryptographic randomness, then using built in stuff (like rand) is not a good choice, as the implementation is highly variable across platforms.
But you've got Rand::Urandom which looks like it does the trick:
By default it uses the getentropy() (only available in > Linux 3.17) and falls back to /dev/arandom then /dev/urandom.
#!/usr/bin/env perl
use strict;
use warnings;
use Rand::Urandom;
chomp ( my #words = <DATA> );
print $words[rand #words], " " for 1..4;
print "\n";
__DATA__
yarn
yard
wound
worst
worry
work
word
wool
wolf
wish
wise
wipe
winter
wing
wind
wife
whole
wheat
water
watch
walk
wake
voice
Failing that though - you can just read bytes from /dev/urandom directly:
#!/usr/bin/env perl
use strict;
use warnings;
my #number_of_words = 4;
chomp ( my #words = <DATA> );
open ( my $urandom, '<:raw', '/dev/urandom' ) or die $!;
my $bytes;
read ( $urandom, $bytes, 2 * $number_of_words ); #2 bytes 0 - 65535
#for testing
#unpack 'n' is n An unsigned short (16-bit)
# unpack 'n*' in a list context returns a list of these.
foreach my $value ( unpack ( "n*", $bytes ) ) {
print $value,"\n";
}
#actually print the words.
#note - this assumes that you have the right number in your list.
# you could add a % #words to the map, e.g. $words[$_ % #words]
#but that will mean wrapping occurs, and will alter the frequency distribution.
#a more robust solution would be to fetch additional bytes if the 'slot' is
#empty.
print join " ", ( map { $words[$_] } unpack ( "n*", $bytes )),"\n";
__DATA__
yarn
yard
wound
worst
#etc.
Note - the above relies on the fact that your wordlist is the same size as two bytes (16 bits) - if this assumption isn't true, you'll need to deal with 'missed' words. A crude approach would be to take a modulo, but that would mean some wrapping and therefore not quite truly even distribution of word picks. Otherwise you can bit-mask and reroll, as indicated below:
On a related point though - have you considered not using a wordlist, and instead using consonant-vowel-consonant groupings?
E.g.:
#!/usr/bin/env perl
use strict;
use warnings;
#uses /dev/urandom to fetch bytes.
#generates consonant-vowel-consonant groupings.
#each are 11.22 bits of entropy, meaning a 4-group is 45 bits.
#( 20 * 6 * 20 = 2400, which is 11.22 bits of entropy log2 2400
#log2(2400 ^ 4) = 44.91
#but because it's generated 'true random' it's a know entropy string.
my $num = 4;
my $format = "CVC";
my %letters = (
V => [qw ( a e i o u y )],
C => [ grep { not /[aeiouy]/ } "a" .. "z" ], );
my %bitmask_for;
foreach my $type ( keys %letters ) {
#find the next power of 2 for the number of 'letters' in the set.
#So - for the '20' letter group, that's 31. (0x1F)
#And for the 6 letter group that's 7. (0x07)
$bitmask_for{$type} = ( 2 << log ( #{$letters{$type}} ) / log 2 ) - 1 ;
}
open( my $urandom, '<:raw', '/dev/urandom' ) or die $!;
for ( 1 .. $num ) {
for my $type ( split //, $format ) {
my $value;
while ( not defined $value or $value >= #{ $letters{$type} } ) {
my $byte;
read( $urandom, $byte, 1 );
#byte is 0-255. Our key space is 20 or 6.
#So rather than modulo, which would lead to an uneven distribution,
#we just bitmask and discard and 'too high'.
$value = (unpack "C", $byte ) & $bitmask_for{$type};
}
print $letters{$type}[$value];
}
print " ";
}
print "\n";
close($urandom);
This generates 3 character CVC symbols, with a known entropy level (11.22 per 'group') for making reasonably robust passwords. (45 bits as opposed to the 64 bits of your original, although obviously you can add extra 'groups' to gain 11.22 bits per time).
This answer is not cryptographically safe!
I would do this completely in Perl. No need for a one-liner. Just grab your word-list and put it into a Perl program.
use strict;
use warnings;
my #words = qw(
first
second
third
fourth
);
print join( q{ }, map { $words[int rand #words] } 1 .. 4 ), "\n";
This grabs four random words from the list and outputs them.
rand #words evaluates #words in scalar context, which gives the number of elements, and creates a random floating point value between 0 and smaller than that number. int cuts off the decimals. This is used as the index to grab an element out of #words. We repeat this four times with the map statement, where the 1 .. 4 is the same as passing a list of (1, 2, 3, 4) into map as an argument. This argument is ignored, but instead our random word is picked. map returns a list, which we join on one space. Finally we print the resulting string, and a newline.
The word list is created with the quoted words qw() operator, which returns a list of quoted words. It's shorthand so you don't need to type all the quotes ' and commas ,.
If you'd want to have the word list at the bottom you could either put the qw() in a sub and call it at the top, or use a __DATA__ section and read from it like a filehandle.
The particular method using tr and fold on /dev/urandom is a lot less efficient than it could be, so let's fix it up a little bit, while keeping the /dev/urandom part.
Assuming that available memory is enough to contain your script (including wordlist):
chomp(#words = <DATA>);
open urandom, "/dev/urandom" or die;
read urandom, $randbytes, 4 * 2 or die;
print join(" ", map $words[$_], unpack "S*", $randbytes), "\n";
__DATA__
word
list
goes
here
This goes for brevity and simplicity without outright obfuscation — of course you could make it shorter by removing whitespace and such, but there's no reason to. It's self-contained and will work with several decades of perls (yes, those bareword filehandles are deliberate :-P)
It still expects exactly 65536 entries in the wordlist, because that way we don't have to worry about introducing bias to the random number choice using a modulus operator. A slightly more ambitious approach might be to read 48 bytes from urandom for each word, turning it into a floating-point value between 0 and 1 (portable to most systems) and multiplying it by the size of the word list, allowing for a word list of any reasonable size.
A lot of nonsense is talked about password strength, and I think you're overestimating the worth of several of your requirements here
I don't understand your preoccupation with making your code "much shorter with perl". (Why did you pick Perl?) Savings here can only really be useful to make the script quicker to read and compile, but they will be dwarfed by the half megabyte of data following the code which must also be read
In this context, the usefulness to a hacker of a poor random number generator depends on prior knowledge of the construction of the password together with the passwords that have been most recently generated. With a sample of only 65,000 words, even the worst random number generator will show insignificant correlation between successive passwords
In general, a password is more secure if it is longer, regardless of its contents. Forming a long password out of a sequence of English words is purely a way of making the sequence more memorable
"Of course later, the (hashing) iteration count ... would be extreme high, so brute-force [hacking?] would be very slow"
This doesn't follow at all. Cracking algorithms won't try to guess the four words you've chosen: they will see only a thirty-character (or so) string consisting only of lower-case letters and spaces, and whose origin is insignificant. It will be no more or less crackable than any other password of the same length with the same character set
I suggest that you should rethink your requirements and so make things easier for yourself. I don't find it hard to think of four English words, and don't need a program to do it for me. Hint: pilchard is a good one: they never guess that!
If you still insist, then I would write something like this in Perl. I've used only the first 18 lines of your data for
use strict;
use warnings 'all';
use List::Util 'shuffle';
my #s = map /\S+/g, ( shuffle( <DATA> ) )[ 0 .. 3 ];
print "#s\n";
__DATA__
yarn
yard
wound
worst
worry
work
word
wool
wolf
wish
wise
wipe
winter
wing
wind
wife
whole
wheat
output
wind wise winter yarn
You could use Data::Random::rand_words()
perl -MData::Random -E 'say join $/, Data::Random::rand_words(size => 4)'
UPDATE
As pointed out in the answer, this question really has to do with Scalar versus List Context in Perl.
## ## ##
I am learning perl via self-taught crash course (primarily with the Llama book and the web). In attempting some byte swap code, I have found a one liner I do not understand completely. A comment in the script explains what I think is happening in the one-liner.
#!/usr/bin/perl --
#
# Script to print byte-swapped hex values
#
use 5.010;
use warnings;
use strict;
# NOTE: I realize I could use a single variable 'data', but the x- y- z- prefixes may help in
# identification (for clarity) in the code for this SO question.
my $xData;
my $yData;
my $zData;
for ( my $ijk = 998; $ijk < 1001; $ijk++ )
{
printf ( "\n%4d is hex " . (sprintf "0x%04X", $ijk) . "\n", $ijk );
# with byte swap
say "These numbers (bytes swapped) should match...";
# do sprintf, match pattern and store to ($1)($2), now reverse them into ($2)($1).
# BindOp leaves $_ alone, match stuffs $_ & is then used as input for reverse, prints.
say reverse ((sprintf "%04X", $ijk) =~ /(..)(..)/) ; # from perlmonks' webpage
$yData = (reverse ((sprintf "%04X", $ijk) =~ /(..)(..)/) );
say $yData; # does NOT match
$xData = sprintf "%04X", $ijk;
$xData =~ s/(..)(..)/$2$1/ ;
say $xData; # does match
$_ = sprintf "%04X", $ijk;
/(..)(..)/;
$zData = $2 . $1 ;
say $zData; # does match
}
exit 0;
OUTPUT:
998 is hex 0x03E6 These numbers (bytes swapped) should match...
E603
6E30
E603
E603
999 is hex 0x03E7 These numbers (bytes swapped) should match...
E703
7E30
E703
E703
1000 is hex 0x03E8 These numbers (bytes swapped) should match...
E803
8E30
E803
E803
Why does the one liner work, and why doesn't the $yData perform the same way? I'm pretty sure I understand why $xData and $zData work, but I would expect $yData to be the closest equivalent non-one-liner. What is the closest equivalent non-one-liner and why? Where is the discrepancy?
The reverse in your print (say) statement comes in the list context, while when assigned to $yData the context is scalar. This function (may) behave considerably differently based on the context.
From perldoc -f reverse
reverse LIST
In list context, returns a list value consisting of the elements of LIST in the opposite order. In scalar context, concatenates the elements of LIST and returns a string value with all characters in the opposite order.
In this case this produces different results.
When called in list context, it swaps the (two) input bytes, keeping each byte intact (represented by two hexadecimal digits matched in a group). When called in scalar context, it joins the input and returns a character string, running in the opposite order. Taken to represent a hex number this would have each byte changed.
the following perl code converts a float number to the wrong integer number
use strict;
my $zahl =297607.22000;
$zahl=$zahl * 100;
print "$zahl\n";
my $text=sprintf ("%017d",$zahl);
print $text;
The output of this is :
29760722
00000000029760721
The thing is, you can change the given number to other numbers and it works.
Any idea what is wrong here or does Perl simply do it wrong?
Thanks for your help!
This is related to a FAQ (Why am I getting long decimals). $zahl is not rounded properly, it is rounded down to the next lower integer.
22/100 is a periodic number in binary just like 1/3 is a periodic number in decimal. It would take infinite storage to store it exactly in a floating point number.
$ perl -e'$_="297607.22000"; $_*=100; printf "%.20f\n", $_'
29760721.99999999627470970154
int and sprintf %d truncate decimals, so you end up with 29760721. print and sprintf %f round, so you can get the desired result.
$ perl -e'$_="297607.22000"; $_*=100; printf "%017.0f\n", $_'
00000000029760722
When you are doing your floating point multiplication by 100 the result will be something like 29760721.9999999963. Then when you do the %d conversion to an integer this is truncated to 29760721.
Try sprintf('%.10f', $zahl) and you should be able to see this.
You have to be really careful with floating point numbers and treating them as fixed point. Due to various conversions that may take place in the builtins, there may be times where one integer conversion is not exactly the same as another. It appears that this happens many times with x.22 numbers:
use strict;
my $n = 0;
for (0 .. 10_000_000) {
my $float = 100 * "$_.22";
my $str = "$float";
my $int = int $float;
if ($str ne $int) {
$n++;
#say "$float, $str, $int";
}
}
say "n = $n";
which prints
n = 76269
on my system.
A careful look at the Perl source would be required to see where the exact conversion difference is.
I would recommend that if you are going to be working with fixed point numbers, to convert them all to integers (using a common conversion function, preferably looking at the source numbers as strings), and then work with them all under the use integer; pragma which will disable floating point numbers.