Perl 5.32.1 GDBM_File is incredibly slow - perl

I just upgraded 2 machines from Fedora 31 to 33, and with that upgrade, Perl went from 5.30.3 to 5.32.1.
The first thing I noticed is that GDBM_File.pm is no longer included in the Perl Core, but that was no problem.
The second thing I noticed is that GDBM write in fc33/perl5.32.1 is incredibly slow. That's a problem.
I noticed something amiss on the first machine, so I ran a little benchmark with fc31/perl5.30.3 on the second machine before doing the upgrade.
gdbm1.pl is rebuilding a db file from an ascii text file, about 33M entries. gdbm0.pl is reading the same ascii text file, and doing everything exactly the same as gdbm1.pl, except not executing the actual hash assignments "$db{...} = ...". That is the only diff. (The ascii file is around 11GB.)
FC31/Perl5.30.3:
[259] time ./gdbm0.pl 16
real 4m51.593s
user 4m49.808s
sys 0m1.306s
[260] time ./gdbm1.pl 16
real 11m39.682s
user 6m30.619s
sys 3m19.260s
FC33/Perl5.32.1:
[287] time ./gdbm0.pl 16
real 5m10.379s
user 5m8.764s
sys 0m1.299s
[288] time ./gdbm1.pl 16
real 554m48.187s
user 7m49.315s
sys 433m42.435s
Obviously it takes longer to write the DB than not: I always expect gdbm0.pl to be faster than gdbm1.pl. But the only diff btwn gdbm0 and gdbm1 is writing the DB, so the time diff is all due to that. On fc31/perl5.30.3, that diff is under 7m. On fc33/perl5.32.1, the time diff is a staggering 550m - over 9 HOURS, vs 7 MINUTES before.
I've done some web searches for anything about GDBM_File being slow in perl5.32.1, I've found nothing. I don't even know if Perl is the problem, it might be fc33, or some combination of both.
Or it could be that some C lib is missing in fc33, and GDBM_File is doing everything in native perl. I don't know where to go from here.
Update:
#davem: Okay, I have 3 machines: a, b, c. "a" is oldest and slowest, "c" is newest and fastest. Machine "a" runs ubuntu, the other two both run fedora:
Linux a 5.8.0-50-generic #56~20.04.1-Ubuntu SMP Mon Apr 12 21:46:35 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Linux b 5.11.18-200.fc33.x86_64 #1 SMP Mon May 3 15:05:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Linux c 5.11.18-200.fc33.x86_64 #1 SMP Mon May 3 15:05:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
I ran two benchmarks on each machine, first with an in-mem hash, then with a gdbm hash. The in-mem hash result gives a very rough idea of the relative single-thread performance of each machine:
[mem] time perl -e'my %h; $h{$_} = 1 for ("a" .. "zzzzz"); print "#{[scalar(keys(%h))]}\n";'
[gdbm] time perl -e'use GDBM_File; my ($h, %h); $h = "gdbm_write_test"; tie(%h, "GDBM_File", $h, GDBM_NEWDB, 0600); $h{$_} = 1 for ("a" .. "zzzzz"); print "#{[scalar(keys(%h))]}\n"; untie(%h);'
machine_a:
[mem] 12356630
real 0m29.051s
user 0m27.975s
sys 0m0.995s
[gdbm] 12356630
real 4m5.431s
user 2m2.033s
sys 1m36.209s
machine_b:
[mem] 12356630
real 0m12.101s
user 0m11.520s
sys 0m0.559s
[gdbm] 12356630
real 106m35.326s
user 1m0.607s
sys 103m48.518s
machine_c:
[mem] 12356630
real 0m9.498s
user 0m9.163s
sys 0m0.317s
[gdbm] 12356630
real 58m46.555s
user 0m39.566s
sys 48m16.447s
Update 2:
I spent a while fiddling with Perl-DB_File and Perl-BerkeleyDB as possible replacements for Perl-GDBM_File. Because I was too lazy to try to figure out how to file a bug.
False laziness, of course. I finally filed a bug just 2 days ago, and there's already a fix checked in and pending release.
#davem was exactly right, the issue was not Perl itself, but the underlying gdbm library. From the fix commit comment:
"Commit 4fb2326a4a introduced pre-reading of memory mapped
regions. While speeding up searches, it has a negative impact
on write operatons, since every remapping effectively re-reads
the entire database."

Related

What is the 'perl-dynamic' executable

Investigating an issue with multiple Perl installations on our (HP UX IA64 system), I came across an executable called perl-dynamic. What is this and what exactly does it do?
More importantly, how do I know which perl executable it actually points to (we have multiple versions mixing version numbers and architectures installed).
bash-4.4$ ll /usr/bin/perl
lrwxr-xr-x 1 root sys 18 Jan 8 16:34 /usr/bin/perl -> /opt/perl/bin/perl
bash-4.4$ ll /opt/perl/bin/perl
lrwxrwxrwx 1 bin bin 14 Jan 8 16:45 /opt/perl/bin/perl -> ./perl-dynamic
Is this a magical perl binary which decides which version and architecture to use?
$ perl -V
Will show you the details.
Both perl-static and perl-dynamic are built with the same configuration (at least for what I can see on my HP-UX 11.31) and both are built with -Dusedl
perl-static is linked with libperl.a (or linked with all .o files), whereas perl-dynamic is linked using libperl.so. You can check the differences with ldd perl-static and ldd perl-dynamic.
There is no functional difference

bash script calling rdiff-backup never ends

I want to run rdiff-backup and then switch of the raspberrypi it was running on.
I use the following script:
#!/bin/sh
date > /home/mik/rdiff-backup.log
echo "rsync start" >> /home/mik/rdiff-backup.log
rdiff-backup -v5 --print-statistics offlinebackup#server::/srv/backup /srv/datenserverBackup/backup >> /home/mik/rdiff-backup.log 2>&1
sync
date >> /home/mik/rdiff-backup.log
echo "rdiff-backup end" >> /home/mik/rdiff-backup.log
df -h >> /home/mik/rdiff-backup.log
sync
halt
The log file looks good (for the rdiff-backup part):
Sat 12 Aug 08:20:59 UTC 2017
rsync start
Unable to import win32security module. Windows ACLs
not supported by filesystem at /srv/backup
escape_dos_devices not required by filesystem at /srv/backup
Warning: name offlinebackup not found on system, dropping ACL entry.
Further ACL entries dropped with this name will not trigger further warnings
Using rdiff-backup version 1.2.8
Executing ssh -C offlinebackup#server rdiff-backup --server
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists On
Extended attributes On
Windows access control lists Off
Case sensitivity On
Escape DOS devices Off
Escape trailing spaces Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Unable to import win32security module. Windows ACLs
not supported by filesystem at /srv/datenserverBackup/backup/rdiff-backup-data/rdiff-backup.tmp.0
escape_dos_devices not required by filesystem at /srv/datenserverBackup/backup/rdiff-backup-data/rdiff-backup.tmp.0
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
Ownership changing On
Hard linking On
fsync() directories On
Directory inc permissions On
High-bit permissions On
Symlink permissions Off
Extended filenames On
Windows reserved filenames Off
Access control lists On
Extended attributes On
Windows access control lists Off
Case sensitivity On
Escape DOS devices Off
Escape trailing spaces Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Backup: must_escape_dos_devices = 0
Starting increment operation /srv/backup to /srv/datenserverBackup/backup
Processing changed file .
Incrementing mirror file /srv/datenserverBackup/backup
Processing changed file abc
Incrementing mirror file /srv/datenserverBackup/backup/abc
Processing changed file abc/def
Incrementing mirror file /srv/datenserverBackup/backup/abc/def
Processing changed file abc/def/testfile.dxf
Incrementing mirror file /srv/datenserverBackup/backup/abc/def/testfile.dxf
--------------[ Session statistics ]--------------
StartTime 1502526061.00 (Sat Aug 12 08:21:01 2017)
EndTime 1502527913.72 (Sat Aug 12 08:51:53 2017)
ElapsedTime 1852.72 (30 minutes 52.72 seconds)
SourceFiles 151099
SourceFileSize 386321558216 (360 GB)
MirrorFiles 151097
MirrorFileSize 386321447731 (360 GB)
NewFiles 2
NewFileSize 110485 (108 KB)
DeletedFiles 0
DeletedFileSize 0 (0 bytes)
ChangedFiles 1
ChangedSourceSize 0 (0 bytes)
ChangedMirrorSize 0 (0 bytes)
IncrementFiles 4
IncrementFileSize 0 (0 bytes)
TotalDestinationSizeChange 110485 (108 KB)
Errors 0
--------------------------------------------------
The backup is working, but then the script ends right there.
rdiff-backup.log contains the full report of rdiff-backup. But neither the line "rdiff-backup end", nor the output of "df -h".
How can I make it ran to the end?
Thanks for your answers
I finally found a workaround, that solves my problem.
My sciprt which is called after booting from /etc/init.d is calling the other script which does the actual work (i.e. backup my data, and write the log file) as a background task.
/etc/init.d/CallAfterBoot.sh
#!/bin/sh
sleep 30
/home/me/DoBackup.sh & # '&' starts the script in background
/home/me/DoBackup.sh is the script I posted above which is now runing correctly.
Same script running as the same user now behaves differently. There's got to be some bug somewhere, however, it works for me now.

Getting CPU cycles from user mode dump

Process Explorer has columns for CPU time (down to milliseconds) and CPU Cycles. For WinDbg I am aware of the !runaway command, also !runaway 7 for more details, but it shows CPU time only.
Are the CPU cycles also available somehow in a user mode crash dump?
What I have tried:
I looked at dt nt!_KTHREAD and I see it has a CycleTime property
ntdll!_KTHREAD
+0x000 Header : _DISPATCHER_HEADER
+0x018 CycleTime : Uint8B
I tried to query that property in a !for_each_thread, but WinDbg responds that it's available in kernel mode only.
Why do I want those CPU cycles?
I am working on a training for JetBrains dotTrace. It has an option to count CPU cycles and I'd like to explain where this cycles come from. Above kernel structure and Process Explorer is probably enough, but it would be awesome to see it live or post mortem in a user mode dump. I explain a lot of basics with WinDbg.
Following the implementation of GetProcessTimes() in ReactOS, you can see that the information is copied from the process' KPROCESS. So, indeed, it's only physically present in a dump that includes kernel memory.
C:\tw>ls -l
total 0
C:\tw>cdb -c ".dump /ma .\tw.dmp;q" calc.exe | grep writ
Dump successfully written
C:\tw>cdb -c "lm;!peb;.dump /ma .\tw1.dmp;q" calc.exe | grep writ
Dump successfully written
C:\tw>cdb -c ".ttime;q" -z tw.dmp | grep -B 3 quit
Created: Wed Apr 5 20:03:55.919 2017 ()
Kernel: 0 days 0:00:00.046
User: 0 days 0:00:00.000
quit:
C:\tw>cdb -c ".ttime;q" -z tw1.dmp | grep -B 3 quit
Created: Wed Apr 5 20:04:28.682 2017 ()
Kernel: 0 days 0:00:00.031
User: 0 days 0:00:00.000
quit:
C:\tw>

Perl created files with future timestamps on FreeBSD 9.3

I just encountered some strange behavior with Perl 5.16.3 on FreeBSD 9.3-RELEASE-p3. We've got a cron job which runs every five minutes and generates some text status files. I just happened to list the contents of the output directory and saw that the timestamps for some of the files were in the future! The files are created like this:
if (open(OUT, "> $status_file_path")) {
print OUT "$status_info\n";
close OUT;
}
Now, the file handle OUT is used in several places, however it is opened and closed within the same block as shown above. And like I said, out of ten files, only a few had future dates when displayed using ls.
For example, files with the current date had timestamps like 04/02/2015 20:29:46, files with future timestamps were out in November, e.g. 11/10/2015 09:38:41.
What might be going on here?
EDIT
I've got two tests running:
1) a perl script running a loop of 1000 iterations, sleeping a random time up to 10 seconds between iterations, using the open/print/close logic to create an output file and abort the script if the file's modification time is in the future.
2) a cron entry to touch a test file every minute, e.g. touch /home/test/test_file_date_with_cron.txt
TEST RESULTS
Neither of the tests generated output files with a timestamp in the future.
This is scary.
EDIT 2
Here is the filesystem info, the files are written in the /usr directory.
# df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/gpt/gprootfs 2G 133M 1.7G 7% /
devfs 1.0k 1.0k 0B 100% /dev
/dev/gpt/gpusrfs 431G 3.8G 392G 1% /usr
procfs 4.0k 4.0k 0B 100% /proc
EDIT 3
Running the script outside of cron for several hundred iterations didn't duplicate the problem. HOWEVER, I just found some other files, which are created by a CGI script which have the future dates:
-rw-r--r-- 1 test test 5783 Nov 10 2015 Config.xml_20150210_104151
-rw-r--r-- 1 test test 34548 Nov 10 2015 Config2.xml_20150210_104151
-rw-r--r-- 1 test test 6105 Nov 10 2015 Config.xml_20151109_232210
-rw-r--r-- 1 test test 34554 Nov 10 2015 Config2.xml_20151109_232210
-rw-rw-r-- 1 root test 2075 Nov 9 2015 Config.xml_20151109_231055
-rw-rw-r-- 1 root test 1232 Nov 9 2015 Config2.xml_20151109_231055
These are archive files, which get moved and renamed with the file's mtime timestamp. Note that BOTH ls and Perl's stat() function report the future date -- stat() is used to generate the file's timestamp portion of the name.
Looking at the first entry, ls reports "Nov 10 2015", whereas when the CGI script processed it, Perl's stat() reported "20150210_104151", i.e. "Feb 02 2015" which is most likely correct.
Further down, we see ls showing "Nov 10 2015" and stat() reported "20151109_232210", i.e. "Nov 09 2015".
Finding those additional archived config files helped me track down the cause, which was as others have suggested, that the system date and timezone changed.
From: 1447147328 and America/Adak
To: 1426637771 and America/New_York
What was throwing me off, was that I thought the cron script wrote ALL of the output files each time it executes, but that's not the case. The files have different "refresh intervals".

IO::Pty perl module cpan solaris installation

OS is SunOS 5.10 Generic_150400-01 sun4v sparc sun4v. Tried to install IO::Pty perl module via cpan. Got an error related to compiler Tried to Set the PATH env variable to /opt/solarisstudio12.3/bin and modified cpan myconfig.pm to have 'make_arg' => q[CC=/opt/solarisstudio12.3/bin/cc] . But still getting the same error.
cpan> install IO::Pty
CPAN: Storable loaded ok
Going to read /home/D99134/cpan/Metadata
Database was generated on Sat, 09 Nov 2013 06:53:04 GMT
Running install for module IO::Pty
Running make for T/TO/TODDR/IO-Tty-1.10.tar.gz
CPAN: Digest::MD5 loaded ok
CPAN: Compress::Zlib loaded ok
Checksum for /home/D99134/cpan/sources/authors/id/T/TO/TODDR/IO-Tty-1.10.tar.gz ok
Scanning cache /home/D99134/cpan/build for sizes
x IO-Tty-1.10, 0 bytes, 0 tape blocks
x IO-Tty-1.10/ChangeLog, 8192 bytes, 16 tape blocks
x IO-Tty-1.10/MANIFEST, 151 bytes, 1 tape blocks
x IO-Tty-1.10/META.yml, 612 bytes, 2 tape blocks
x IO-Tty-1.10/Makefile.PL, 13360 bytes, 27 tape blocks
x IO-Tty-1.10/Pty.pm, 8709 bytes, 18 tape blocks
x IO-Tty-1.10/README, 1686 bytes, 4 tape blocks
x IO-Tty-1.10/Tty.pm, 7526 bytes, 15 tape blocks
x IO-Tty-1.10/Tty.xs, 22244 bytes, 44 tape blocks
x IO-Tty-1.10/t, 0 bytes, 0 tape blocks
x IO-Tty-1.10/t/test.t, 7306 bytes, 15 tape blocks
x IO-Tty-1.10/try, 2880 bytes, 6 tape blocks
Removing previously used /home/D99134/cpan/build/IO-Tty-1.10
CPAN.pm: Going to build T/TO/TODDR/IO-Tty-1.10.tar.gz
Now let's see what we can find out about your system
(logfiles of failing tests are available in the conf/ dir)...
ERROR: cannot run the configured compiler '/opt/SUNWspro/bin/cc'
(see conf/compilerok.log). Suggestions:
1) The complier '/opt/SUNWspro/bin/cc' is not in your PATH. Add it
to the PATH and try again. OR
2) The compiler isn't installed on your system. Install it. OR
3) You only have a different compiler installed (e.g. 'gcc').
Either fix the compiler config in the perl Config.pm
or install a perl that was built with the right compiler
(you could build perl yourself with the available compiler).
Note: this is a system-administration issue, please ask your local
admin for help. Thank you.
make: Fatal error: No arguments to build
/usr/ccs/bin/make CC=/opt/solarisstudio12.3/bin/cc -- NOT OK
Running make test
Can't test without successful make
Running make install
make had returned bad status, install seems impossible
You have to follow the third suggestion (fixing perl's Config.pm). This is dangerous, but there's a CPAN module which allows you to do Config.pm changes on the fly: ExtUtils::FakeConfig. Just follow the instructions in the documentation.
Here's another article about editing Config.pm on perlmonks: http://www.perlmonks.org/?node_id=570684