Perl - Issue installing or connecting to queue manager from MQSeries - 1.34 (Reason = 2059) - perl

I tried to install MQSeries 1.34 from CPAN, and I was getting an error, which I found a fix for on the CPAN bug tracking website. So I applied the fix and still got a few test errors, but the build seemed okay. Installing it with a force install MQSeries worked. Once I had installed it, I can construct objects and reference the libraries in my other perl scripts, but when I try to connect to the queue manager or queue, I get an error back.
Any advice on what is wrong with the setup or how to fix this and get my connection back up and running would be much appreciated.
Here are specifics about my configuration:
$ uname -srvpio
Linux 2.6.32-573.22.1.el6.x86_64 #1 SMP Thu Mar 17 03:23:39 EDT 2016 x86_64 x86_64 GNU/Linux
$ /opt/mqm/bin/dspmqsver
Name: WebSphere MQ Advanced Message Security
Version: 7.5.0.0
Level: p000-L120604
BuildType: IKAP - (Production)
$ rpm -qa |grep 'MQ' |sort
MQSeriesAMS-7.5.0-0.x86_64
MQSeriesClient-7.5.0-0.x86_64
MQSeriesExplorer-7.5.0-0.x86_64
MQSeriesFTAgent-7.5.0-0.x86_64
MQSeriesFTBase-7.5.0-0.x86_64
MQSeriesFTLogger-7.5.0-0.x86_64
MQSeriesFTService-7.5.0-0.x86_64
MQSeriesFTTools-7.5.0-0.x86_64
MQSeriesGSKit-7.5.0-0.x86_64
MQSeriesJava-7.5.0-0.x86_64
MQSeriesJRE-7.5.0-0.x86_64
MQSeriesMan-7.5.0-0.x86_64
MQSeriesMsg_cs-7.5.0-0.x86_64
MQSeriesMsg_de-7.5.0-0.x86_64
MQSeriesMsg_es-7.5.0-0.x86_64
MQSeriesMsg_fr-7.5.0-0.x86_64
MQSeriesMsg_hu-7.5.0-0.x86_64
MQSeriesMsg_it-7.5.0-0.x86_64
MQSeriesMsg_ja-7.5.0-0.x86_64
MQSeriesMsg_ko-7.5.0-0.x86_64
MQSeriesMsg_pl-7.5.0-0.x86_64
MQSeriesMsg_pt-7.5.0-0.x86_64
MQSeriesMsg_ru-7.5.0-0.x86_64
MQSeriesMsg_Zh_CN-7.5.0-0.x86_64
MQSeriesMsg_Zh_TW-7.5.0-0.x86_64
MQSeriesRuntime-7.5.0-0.x86_64
MQSeriesSamples-7.5.0-0.x86_64
MQSeriesSDK-7.5.0-0.x86_64
MQSeriesServer-7.5.0-0.x86_64
MQSeriesXRClients-7.5.0-0.x86_64
MQSeriesXRService-7.5.0-0.x86_64
$ ./dispModVersion.pl MQSeries
Module Installed CPAN
MQSeries 1.34 1.34
$ perl -v
This is perl 5, version 22, subversion 1 (v5.22.1) built for x86_64-linux
Copyright 1987-2015, Larry Wall
Perl may be copied only under the terms of either the Artistic License or the GNU General Public License, which may be found in the Perl 5 source kit.
Complete documentation for Perl, including FAQ lists, should be found on this system using "man perl" or "perldoc perl". If you have access to the Internet, point your browser at http://www.perl.org/, the Perl Home Page.
$ perl -V
Summary of my perl5 (revision 5 version 22 subversion 1) configuration:
Platform:
osname=linux, osvers=2.6.32-573.22.1.el6.x86_64, archname=x86_64-linux
uname='linux ******* 2.6.32-573.22.1.el6.x86_64 #1 smp thu mar 17 03:23:39 edt 2016 x86_64 x86_64 x86_64 gnulinux '
config_args='-des -Dprefix=******'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2',
optimize='-O2',
cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.4.7 20120313 (Red Hat 4.4.7-16)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64 /usr/local/lib64
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.12.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.12'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'
Characteristics of this binary (from libperl):
Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
USE_PERL_ATOF
Built under linux
Compiled at Apr 18 2016 15:46:59
%ENV:
PERL5BIN="*******/bin"
PERL5HOME="*******"
PERL5SITE="*******/lib/site_perl/5.22.1"
#INC:
*******/lib/site_perl/5.22.1/x86_64-linux
*******/lib/site_perl/5.22.1
*******/lib/5.22.1/x86_64-linux
*******/lib/5.22.1
When I first tried to install MQSeries, I got the following error:
t/99-pod.t .. 2/63
# Failed test 'POD test for MQSeries/Message/IIH.pm'
# at Test/Pod.pm line 187.
# MQSeries/Message/IIH.pm (415): Non-ASCII character seen before =encoding in 'Villagâ–’mez'. Assuming CP1252
# Looks like you failed 1 test of 63.
t/99-pod.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/63 subtests
Test Summary Report
-------------------
t/99-pod.t (Wstat: 256 Tests: 63 Failed: 1)
Failed test: 61
Non-zero exit status: 1
Files=1, Tests=63, 1 wallclock secs ( 0.02 usr 0.01 sys + 0.34 cusr 0.00 csys = 0.37 CPU)
Result: FAIL
Failed 1/1 test programs. 1/63 subtests failed.
Then, I modified the file MQSeries/Message/IIH.pm to add =encoding ISO-8859-1 at the start of the file, per bug report on CPAN, I ran the install again and got the following:
t/99-pod.t .. ok
All tests successful.
Files=1, Tests=63, 0 wallclock secs ( 0.03 usr 0.01 sys + 0.34 cusr 0.01 csys = 0.39 CPU)
Result: PASS
make[1]: Entering directory `build/MQSeries-1.34-SLJq6K/MQClient'
PERL_DL_NONLAZY=1 "perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, '../blib/lib', '../blib/arch')" t/*.t
t/10constants.t ........... ok
t/20convert.t ............. Failed 5/6 subtests
t/30basic.t ............... ok
t/31properties.t .......... ok
t/32async_put.t ........... ok
t/40oo-qmgr.t ............. 3/14 MQCONN failed (Reason = 2059) (Queue manager not available for connection.) at t/40oo-qmgr.t line 38.
# Failed test 'MQSeries::QueueManager - constructor'
# at t/40oo-qmgr.t line 39.
t/40oo-qmgr.t ............. 5/14 MQCONN failed (Reason = 2059) (Queue manager not available for connection.) at t/40oo-qmgr.t line 52.
# Failed test 'MQSeries::QueueManager - Connect'
# at t/40oo-qmgr.t line 58.
t/40oo-qmgr.t ............. 7/14 MQCONN failed (Reason = 2059) (Queue manager not available for connection.) at t/40oo-qmgr.t line 73.
# Failed test 'MQSeries::QueueManager - constructor'
# at t/40oo-qmgr.t line 74.
Can't call method "Open" on an undefined value at t/40oo-qmgr.t line 77.
# Looks like you planned 14 tests but ran 7.
# Looks like you failed 3 tests of 7 run.
# Looks like your test exited with 13 just after 7.
t/40oo-qmgr.t ............. Dubious, test returned 13 (wstat 3328, 0xd00)
Failed 10/14 subtests
t/50oo-command.t .......... ok
t/51oo-command-changes.t .. ok
t/52oo-command-filter.t ... ok
t/60-logs.t ............... ok
t/70-config.t ............. ok
Test Summary Report
-------------------
t/20convert.t (Wstat: 0 Tests: 2 Failed: 1)
Failed test: 2
Parse errors: Bad plan. You planned 6 tests but ran 2.
t/40oo-qmgr.t (Wstat: 3328 Tests: 7 Failed: 3)
Failed tests: 3, 5, 7
Non-zero exit status: 13
Parse errors: Bad plan. You planned 14 tests but ran 7.
Files=11, Tests=5334, 1264 wallclock secs ( 0.96 usr 0.02 sys + 1.56 cusr 0.18 csys = 2.72 CPU)
Result: FAIL
Failed 2/11 test programs. 4/5334 subtests failed.
If I force the install, I can get it to finish the install after everything is said and done, but then I get the following error when trying to connect:
MQCONN failed (Reason = 2059) (Queue manager not available for connection.) at testMQConnectivity.pl line 16.
MQCONN failed (Reason = 2059) (Queue manager not available for connection.) at testMQConnectivity.pl line 16.
at MQSeries/QueueManager.pm line 809.
MQSeries::QueueManager::Connect(MQSeries::QueueManager=HASH(0x24a7180)) called at MQSeries/QueueManager.pm line 155
MQSeries::QueueManager::new(undef, "QueueManager", "*******", "Reason", SCALAR(0x151d4f0), "CompCode", SCALAR(0x151d610), "ClientConn", ...) called at testMQConnectivity.pl line 16
I edited the CONFIG file after pulling the data from CPAN to reference my remote MQ Queues that I can connect to using /opt/mqm/samp/bin/amqsputc or /opt/mqm/samp/bin/amqsgetc, after setting export MQSERVER=..., if I run make test TEST_VERBOSE=1 from the working CPAN directory, I see the following output:
t/20convert.t:
t/20convert.t .............
1..6
ok 1
ok 2
ok 3
ok 4
MQGET failed: CompCode => 2, Reason => 2033
not ok 5
Message conversion failed.
Should be: 'This msg is in EBCDIC'
Is: ''
not ok 6
Failed 2/6 subtests
t/31properties.t:
t/31properties.t ..........
1..22
ok 1 - use MQClient::MQSeries;
Connecting to queue manager '*******' (MQCONN)
ok 2 - MQCONN
Opening queue manager for inquire
ok 3 - MQOPEN - queue manager
Querying queue manager attributes (MQINQ)
ok 4 - MQINQ
Closing queue manager object handle (MQCLOSE)
ok 5 - MQCLOSE - queue manager
Create message handle object (MQCRTMH)
ok 6 - MQCRTMH
Set message property (MQSETMP - string)
ok 7 - MQSETMP - string
Set message property (MQSETMP - float)
ok 8 - MQSETMP - float
Set message property (MQSETMP - int)
ok 9 - MQSETMP - int
Inquire message property (MQINQMP - float)
ok 10 - MQINQMP - float
ok 11 - MQINQMP - float value okay (3.14164996147156)
Delete float property
ok 12 - MQDLTMP - float
Opening queue '*******' (MQOPEN)
ok 13 - MQOPEN - queue
Putting message with properties (MQPUT)
ok 14 - MQPUT
Create message handle object (MQCRTMH #2)
ok 15 - MQCRTMH \#2
Getting message (MQGET)
not ok 16 - MQGET
# Failed test 'MQGET'
# at t/31properties.t line 146.
Inquire message property (MQINQMP - int)
not ok 17 - MQINQMP - int
# Failed test 'MQINQMP - int'
# at t/31properties.t line 156.
Use of uninitialized value $prop in concatenation (.) or string at t/31properties.t line 157.
not ok 18 - MQINQMP - int value okay ()
# Failed test 'MQINQMP - int value okay ()'
# at t/31properties.t line 157.
# got: undef
# expected: '42'
Delete message handle (MQDLTMH)
ok 19 - MQDLTMH
Delete message handle (MQDLTMH #2)
ok 20 - MQDLTMH \#2
Closing queue (MQCLOSE)
ok 21 - MQCLOSE
Disconnecting (MQDISC)
ok 22 - MQDISC
# Looks like you failed 3 tests of 22.
Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/22 subtests
t/32async_put.t:
t/32async_put.t ...........
1..25
ok 1 - use MQClient::MQSeries;
Connecting to queue manager '*******' (MQCONN)
ok 2 - MQCONN
Opening queue manager for inquire
ok 3 - MQOPEN - queue manager
Querying queue manager attributes (MQINQ)
MQINQ returned: CompCode => 0, Reason => 0
ok 4 - MQINQ
Closing queue manager object handle (MQCLOSE)
ok 5 - MQCLOSE - queue manager
Opening queue '*******' (MQOPEN)
ok 6 - MQOPEN - queue
Putting message #1 (async MQPUT)
ok 7 - MQPUT - 1
Putting message #2 (async MQPUT)
ok 8 - MQPUT - 2
Putting message #3 (async MQPUT)
ok 9 - MQPUT - 3
Putting message #4 (async MQPUT)
ok 10 - MQPUT - 4
Putting message #5 (async MQPUT)
ok 11 - MQPUT - 5
Getting status info (MQSTAT)
ok 12 - MQSTAT
not ok 13 - MQSTAT result
# Failed test 'MQSTAT result'
# at t/32async_put.t line 120.
# Structures begin differing at:
# $got->{PutSuccessCount} = '5'
# $expected->{PutSuccessCount} = '0'
Getting message #1 (MQGET)
ok 14 - MQGET - 1
ok 15 - MQGET message content - 1
Getting message #2 (MQGET)
ok 16 - MQGET - 2
ok 17 - MQGET message content - 2
Getting message #3 (MQGET)
ok 18 - MQGET - 3
ok 19 - MQGET message content - 3
Getting message #4 (MQGET)
ok 20 - MQGET - 4
ok 21 - MQGET message content - 4
Getting message #5 (MQGET)
ok 22 - MQGET - 5
ok 23 - MQGET message content - 5
Closing queue (MQCLOSE)
ok 24 - MQCLOSE
Disconnecting (MQDISC)
ok 25 - MQDISC
# Looks like you failed 1 test of 25.
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/25 subtests
And t/40oo-qmgr.t, which was previously failing now executes successfully:
t/40oo-qmgr.t:
t/40oo-qmgr.t .............
1..14
ok 1 - use MQClient::MQSeries;
ok 2 - use MQSeries::QueueManager;
ok 3 - MQSeries::QueueManager - constructor
ok 4 - MQSeries::QueueManager - constructor - AutoConnect=0
ok 5 - MQSeries::QueueManager - Connect
ok 6 - MQSeries::QueueManager - Disconnect
ok 7 - MQSeries::QueueManager - constructor
ok 8 - MQSeries::QueueManager - Open
ok 9 - MQSeries::QueueManager - Inquire
ok 10 - MQSeries::QueueManager - Inquire - Platform
ok 11 - MQSeries::QueueManager - Inquire - CodedCharSetId
ok 12 - MQSeries::QueueManager - Inquire - CommandLevel
ok 13 - MQSeries::QueueManager - Inquire - DeadLetterQName
ok 14 - MQSeries::QueueManager - Close
ok
Yielding the final results:
Test Summary Report
-------------------
t/20convert.t (Wstat: 0 Tests: 6 Failed: 2)
Failed tests: 5-6
t/31properties.t (Wstat: 768 Tests: 22 Failed: 3)
Failed tests: 16-18
Non-zero exit status: 3
t/32async_put.t (Wstat: 256 Tests: 25 Failed: 1)
Failed test: 13
Non-zero exit status: 1
Files=11, Tests=5345, 183 wallclock secs ( 1.02 usr 0.08 sys + 1.48 cusr 0.18 csys = 2.76 CPU)
Result: FAIL
Failed 3/11 test programs. 6/5345 subtests failed.
Any ideas on how to get this resolved and get the package up and running?
If I run /opt/mqm/samp/bin/amqsputc or /opt/mqm/samp/bin/amqsgetc, I can connect to the same host, port, channel, queue manager, and queue from this host.
Thanks!

Related

Nominatim: psycopg2.errors.UndefinedTable: missing FROM-clause entry for table "parent" when trying to import

I had created an empty database called nominatim on my DB cluster, pointed the DSN to my database with the dbname, host, port, user and pass and tried to import the map of Portugal with the following params:
nohup <path/to/file>/build/utils/setup.php \
--threads 4 \
--setup-db \
--import-data \
--create-functions \
--enable-diff-updates \
--enable-debug-statements \
--create-tables \
--create-partition-tables \
--create-partition-functions \
--import-wikipedia-articles \
--load-data \
--import-tiger-data \
--calculate-postcodes \
--index \
--index-noanalyse \
--create-search-indices \
--create-country-names \
--osm-file <path/to/file>/portugal-latest.osm.pbf >> <path/to/file>/logImportPortugal.txt 2>&1
But it is giving me the following error:
WARNING: Starting rank 25
Traceback (most recent call last):
File "/srv/nominatim/Nominatim-3.5.1/nominatim/nominatim.py", line 370, in <module>
Indexer(options).run()
File "/srv/nominatim/Nominatim-3.5.1/nominatim/nominatim.py", line 215, in run
self.index(RankRunner(self.maxrank))
File "/srv/nominatim/Nominatim-3.5.1/nominatim/nominatim.py", line 259, in index
thread = next(next_thread)
File "/srv/nominatim/Nominatim-3.5.1/nominatim/nominatim.py", line 300, in find_free_thread
if thread.is_done():
File "/srv/nominatim/Nominatim-3.5.1/nominatim/nominatim.py", line 178, in is_done
if self.conn.poll() == psycopg2.extensions.POLL_OK:
psycopg2.errors.UndefinedTable: missing FROM-clause entry for table "parent"
LINE 1: parent.place_id
^
QUERY: parent.place_id
CONTEXT: PL/pgSQL function find_parent_for_address(text,text,smallint,geometry) line 25 at RAISE
PL/pgSQL function find_parent_for_poi(character,bigint,smallint,geometry,text,text,boolean) line 31 at assignment
PL/pgSQL function placex_update() line 178 at assignment
ERROR: error status 1 running nominatim!
string(33) "error status 1 running nominatim!"
The flag --setup-db shouldn't create all the relations? What am I missing?
Below is the full log:
nohup: ignoring input
2023-01-10 13:00:49 == module path: /srv/nominatim/build/module
2023-01-10 13:00:49 == Setup DB
Postgres version found: 14
Postgis version found: 3.2
set_config
------------
(1 row)
2023-01-10 13:00:54 == Import data
osm2pgsql version 1.2.0 (64 bit id space)
Mid: loading persistent node cache from /srv/nominatim/flatnode.file
Mid: pgsql, cache=0
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels
Parsing gazetteer style file '/srv/nominatim/Nominatim-3.5.1/settings/import-full.style'.
Using projection SRS 4326 (Latlong)
NOTICE: table "place" does not exist, skipping
Reading in file: /home/izadmin/planet_full/portugal-latest.osm.pbf
Using PBF parser.
^MProcessing: Node(50k 50.0k/s) Way(0k 0.00k/s) Relation(0 0.00/s)^MProcessing: Node(150k 75.0k/s) Way(0k 0.00k/s) Relation(0 0.00/s)^MProcessing: Node(310k 103.3k/s) Way(0k 0.00k/s) Relation>Node stats: total(36476942), max(10423206981) in 223s
Way stats: total(3678221), max(1129408209) in 88s
Relation stats: total(75907), max(15101706) in 7s
Stopping table: planet_osm_nodes
Stopped table: planet_osm_nodes in 0s
Stopping table: planet_osm_ways
Stopped table: planet_osm_ways in 0s
Stopping table: planet_osm_rels
Building index on table: planet_osm_rels
Stopped table: planet_osm_rels in 0s
Osm2pgsql took 319s overall
2023-01-10 13:06:14 == Create Functions
2023-01-10 13:06:14 == Create Tables
2023-01-10 13:06:14 == Create Functions
2023-01-10 13:06:14 == Create Tables
2023-01-10 13:06:15 == Create Partition Tables
2023-01-10 13:06:16 == Create Partition Functions
2023-01-10 13:06:16 == Importing wikipedia articles and redirects
2023-01-10 13:08:08 == Drop old Data
...............................................................................................................................................................................................>2023-01-10 13:08:13 == Loading word list
count
--------
318141
(1 row)
count
-------
14556
(1 row)
2023-01-10 13:09:01 == Load Data
2023-01-10 13:43:34 == Reanalysing database
Latest data imported from 2023-01-09T21:15:53Z.
2023-01-10 13:43:37 == Import Tiger data
2023-01-10 13:43:37 == Found 0 SQL files in path /srv/nominatim/Nominatim-3.5.1/data/tiger
2023-01-10 13:43:37 == WARNING: Tiger data import selected but no files found in path /srv/nominatim/Nominatim-3.5.1/data/tiger
2023-01-10 13:43:37 == Calculate Postcodes
2023-01-10 13:44:47 == Index ranks 0 - 4
'/srv/nominatim/Nominatim-3.5.1/nominatim/nominatim.py' --database nominatim --port 5432 --threads 4 -v --host '127.0.0.1' --user postgres --maxrank 4WARNING: Starting indexing rank (0 to 4) >WARNING: Starting rank 0
WARNING: Done 0/0 in 0 # 0.000 per second - FINISHED rank 0
WARNING: Starting rank 1
WARNING: Done 0/0 in 0 # 0.000 per second - FINISHED rank 1
WARNING: Starting rank 2
WARNING: Done 0/0 in 0 # 0.000 per second - FINISHED rank 2
WARNING: Starting rank 3
WARNING: Done 0/0 in 0 # 0.000 per second - FINISHED rank 3
WARNING: Starting rank 4
WARNING: Done 2/2 in 0 # 3.477 per second - FINISHED rank 4
2023-01-10 13:44:47 == Index ranks 5 - 25
WARNING: Starting indexing rank (5 to 25) using 4 threads
WARNING: Starting rank 5
WARNING: Done 0/0 in 0 # 0.000 per second - FINISHED rank 5
WARNING: Starting rank 6
WARNING: Done 0/0 in 0 # 0.000 per second - FINISHED rank 6
WARNING: Starting rank 7
WARNING: Done 0/0 in 0 # 0.000 per second - FINISHED rank 7
WARNING: Starting rank 8
WARNING: Done 2/2 in 0 # 7.341 per second - FINISHED rank 8
WARNING: Starting rank 9
WARNING: Done 0/0 in 0 # 0.000 per second - FINISHED rank 9
WARNING: Starting rank 10
WARNING: Done 0/0 in 0 # 0.000 per second - FINISHED rank 10
WARNING: Starting rank 11
WARNING: Done 0/0 in 0 # 0.000 per second - FINISHED rank 11
WARNING: Starting rank 12
WARNING: Done 37/37 in 0 # 139.460 per second - FINISHED rank 12
WARNING: Starting rank 13
WARNING: Done 0/0 in 0 # 0.000 per second - FINISHED rank 13
WARNING: Starting rank 14
INFO: Done 100 in 0 # 389.785 per second - rank 14 ETA (seconds): 0.53
WARNING: Done 308/308 in 0 # 321.383 per second - FINISHED rank 14
WARNING: Starting rank 15
WARNING: Done 1/1 in 0 # 661.813 per second - FINISHED rank 15
WARNING: Starting rank 16
INFO: Done 100 in 0 # 587.337 per second - rank 16 ETA (seconds): 5.38
INFO: Done 687 in 1 # 516.197 per second - rank 16 ETA (seconds): 4.99
INFO: Done 1203 in 2 # 507.508 per second - rank 16 ETA (seconds): 4.06
INFO: Done 1710 in 3 # 488.050 per second - rank 16 ETA (seconds): 3.18
INFO: Done 2198 in 4 # 455.463 per second - rank 16 ETA (seconds): 2.34
INFO: Done 2653 in 5 # 450.394 per second - rank 16 ETA (seconds): 1.35
INFO: Done 3103 in 7 # 425.337 per second - rank 16 ETA (seconds): 0.37
WARNING: Done 3262/3262 in 8 # 388.872 per second - FINISHED rank 16
WARNING: Starting rank 17
INFO: Done 100 in 0 # 1959.248 per second - rank 17 ETA (seconds): 0.01
WARNING: Done 127/127 in 0 # 1647.831 per second - FINISHED rank 17
WARNING: Starting rank 18
INFO: Done 100 in 0 # 1458.789 per second - rank 18 ETA (seconds): 2.29
INFO: Done 1558 in 0 # 1776.158 per second - rank 18 ETA (seconds): 1.06
INFO: Done 3334 in 1 # 1774.513 per second - rank 18 ETA (seconds): 0.06
WARNING: Done 3447/3447 in 1 # 1777.266 per second - FINISHED rank 18
WARNING: Starting rank 19
INFO: Done 100 in 0 # 3007.609 per second - rank 19 ETA (seconds): 1.98
INFO: Done 3107 in 1 # 2444.868 per second - rank 19 ETA (seconds): 1.20
INFO: Done 5551 in 2 # 2570.094 per second - rank 19 ETA (seconds): 0.19
WARNING: Done 6053/6052 in 2 # 2644.112 per second - FINISHED rank 19
WARNING: Starting rank 20
INFO: Done 100 in 0 # 2110.150 per second - rank 20 ETA (seconds): 13.73
INFO: Done 2210 in 0 # 2969.273 per second - rank 20 ETA (seconds): 9.04
INFO: Done 5179 in 2 # 2321.562 per second - rank 20 ETA (seconds): 10.29
INFO: Done 7500 in 3 # 2142.735 per second - rank 20 ETA (seconds): 10.07
INFO: Done 9642 in 4 # 2055.257 per second - rank 20 ETA (seconds): 9.45
INFO: Done 11697 in 5 # 2019.092 per second - rank 20 ETA (seconds): 8.60
INFO: Done 13716 in 6 # 1998.935 per second - rank 20 ETA (seconds): 7.68
INFO: Done 15714 in 7 # 1993.985 per second - rank 20 ETA (seconds): 6.70
INFO: Done 17707 in 8 # 1990.804 per second - rank 20 ETA (seconds): 5.71
INFO: Done 19697 in 9 # 1978.861 per second - rank 20 ETA (seconds): 4.74
INFO: Done 21675 in 11 # 1960.704 per second - rank 20 ETA (seconds): 3.77
INFO: Done 23635 in 12 # 1936.453 per second - rank 20 ETA (seconds): 2.81
INFO: Done 25571 in 13 # 1929.992 per second - rank 20 ETA (seconds): 1.81
INFO: Done 27500 in 14 # 1939.698 per second - rank 20 ETA (seconds): 0.81
WARNING: Done 29070/29067 in 14 # 1942.903 per second - FINISHED rank 20
WARNING: Starting rank 21
INFO: Done 100 in 0 # 2234.986 per second - rank 21 ETA (seconds): 0.48
WARNING: Done 1167/1167 in 0 # 1538.119 per second - FINISHED rank 21
WARNING: Starting rank 22
INFO: Done 100 in 0 # 1638.646 per second - rank 22 ETA (seconds): 18.97
INFO: Done 1738 in 0 # 1982.608 per second - rank 22 ETA (seconds): 14.85
INFO: Done 3720 in 2 # 1732.995 per second - rank 22 ETA (seconds): 15.85
INFO: Done 5452 in 3 # 1742.091 per second - rank 22 ETA (seconds): 14.77
INFO: Done 7194 in 4 # 1723.857 per second - rank 22 ETA (seconds): 13.92
INFO: Done 8917 in 5 # 1717.254 per second - rank 22 ETA (seconds): 12.97
INFO: Done 10634 in 6 # 1712.904 per second - rank 22 ETA (seconds): 12.00
INFO: Done 12346 in 7 # 1716.312 per second - rank 22 ETA (seconds): 10.98
INFO: Done 14062 in 8 # 1718.356 per second - rank 22 ETA (seconds): 9.96
INFO: Done 15780 in 9 # 1720.232 per second - rank 22 ETA (seconds): 8.95
INFO: Done 17500 in 10 # 1727.692 per second - rank 22 ETA (seconds): 7.92
INFO: Done 19227 in 11 # 1727.651 per second - rank 22 ETA (seconds): 6.92
INFO: Done 20954 in 12 # 1723.779 per second - rank 22 ETA (seconds): 5.93
INFO: Done 22677 in 13 # 1715.023 per second - rank 22 ETA (seconds): 4.96
INFO: Done 24392 in 14 # 1717.552 per second - rank 22 ETA (seconds): 3.95
INFO: Done 26109 in 15 # 1697.694 per second - rank 22 ETA (seconds): 2.99
INFO: Done 27806 in 16 # 1659.334 per second - rank 22 ETA (seconds): 2.04
INFO: Done 29465 in 17 # 1662.434 per second - rank 22 ETA (seconds): 1.03
INFO: Done 31127 in 18 # 1667.951 per second - rank 22 ETA (seconds): 0.03
WARNING: Done 31184/31183 in 18 # 1668.224 per second - FINISHED rank 22
WARNING: Starting rank 23
WARNING: Done 0/0 in 0 # 0.000 per second - FINISHED rank 23
WARNING: Starting rank 24
INFO: Done 100 in 0 # 2041.400 per second - rank 24 ETA (seconds): 2.52
INFO: Done 2141 in 1 # 1956.016 per second - rank 24 ETA (seconds): 1.59
INFO: Done 4097 in 2 # 1911.586 per second - rank 24 ETA (seconds): 0.60
WARNING: Done 5248/5245 in 2 # 1908.391 per second - FINISHED rank 24
WARNING: Starting rank 25
Traceback (most recent call last):
File "/srv/nominatim/Nominatim-3.5.1/nominatim/nominatim.py", line 370, in <module>
Indexer(options).run()
File "/srv/nominatim/Nominatim-3.5.1/nominatim/nominatim.py", line 215, in run
self.index(RankRunner(self.maxrank))
File "/srv/nominatim/Nominatim-3.5.1/nominatim/nominatim.py", line 259, in index
thread = next(next_thread)
File "/srv/nominatim/Nominatim-3.5.1/nominatim/nominatim.py", line 300, in find_free_thread
if thread.is_done():
File "/srv/nominatim/Nominatim-3.5.1/nominatim/nominatim.py", line 178, in is_done
if self.conn.poll() == psycopg2.extensions.POLL_OK:
psycopg2.errors.UndefinedTable: missing FROM-clause entry for table "parent"
LINE 1: parent.place_id
^
QUERY: parent.place_id
CONTEXT: PL/pgSQL function find_parent_for_address(text,text,smallint,geometry) line 25 at RAISE
PL/pgSQL function find_parent_for_poi(character,bigint,smallint,geometry,text,text,boolean) line 31 at assignment
PL/pgSQL function placex_update() line 178 at assignment
ERROR: error status 1 running nominatim!
string(33) "error status 1 running nominatim!"
I am using Nominatim 3.5.1

SSL connect attempt failed because of handshake problems using Net::FTPSSL

I am trying to create a Perl script with Net::FTPSSL that move the files from the FTP server to actual server.
The script will run every X minutes / seconds to check if the customers uploaded something new on the FTP server and move it.
#!/usr/bin/perl -w
use strict;
use warnings;
use DateTime;
use Net::FTPSSL;
my $ftp;
sub connect_ftp {
$ftp = Net::FTPSSL->new('sftp.domain.com',
Encryption => EXP_CRYPT,
Debug => 1, DebugLogFile => "/usr/logs/ftpssl.log");
$ftp->login('username', 'password');
$ftp->binary();
}
sub copy_files {
my #folder_list;
my $folder;
#folder_list = $ftp->nlst();
foreach $folder (#folder_list) {
if ($folder ne "globa_username") {
$ftp->cwd($folder);
my #file_list;
my $video;
#file_list = $ftp->nlst();
foreach $video (#file_list) {
my $file_status;
my $result;
if ($video =~ /\.mp4$/i) {
if (!file_exist($video, $folder)) {
$file_status = $ftp->is_file($video);
if ($file_status > 0) {
$result = $ftp->get($video, "/usr/media/$folder/$video");
if (defined $result) {
#$ftp->delete($video);
}
}
}
}
}
}
$ftp->cdup();
}
#$ftp->quit();
}
sub file_exist {
my ($video, $folder) = #_;
return -f "/usr/media/$folder/$video";
}
connect_ftp();
for (;;) {
copy_files();
#sleep 60;
#sleep 30;
}
The script works fine until returns: **555 TLSv12: SSL connect attempt failed because of handshake problems**.
Net-FTPSSL Version: 0.42
IO-Socket-SSL Version: 1.94
Net-SSLeay Version: 1.55
IO-Socket-INET Version: 1.33
IO-Socket-INET6 might not be installed.
IO-Socket-IP Version: 0.21
IO Version: 1.25_06
Socket Version: 2.030
No IPv6 support available. You must 1st upgrade IO::Socket::SSL to support it!
Perl: 5.016003 [5.16.3], OS: linux
***** IPv6 not yet supported in Net::FTPSSL! *****
Server (port): sftp.domain.com (21)
Keys: (Debug), (Encryption), (DebugLogFile)
Values: (1), (E), (/usr/logs/ftpssl.log)
SKT <<< 220 (vsFTPd 3.0.2)
SKT >>> AUTH TLS
SKT <<< 234 Proceed with negotiation.
Object HASH Details ... (SSL:arguments - E)
SSL_hostname ==> sftp.domain.com
SSL_verify_mode ==> 0
SSL_version ==> TLSv12
Timeout ==> 120
Object Net::FTPSSL Details ... (sftp.domain.com:21 - E)
_FTPSSL_arguments ==> HASH(0x272cee8)
-- Croak ===> (undef)
-- Crypt ===> E
-- FixGetTs ===> 0
-- FixPutTs ===> 0
-- Host ===> sftp.domain.com
-- Pret ===> 0
-- Timeout ===> 120
-- buf_size ===> 10240
-- data_prot ===> P
-- dcsc_mode ===> 1
-- debug ===> 2
-- debug_extra ===> 0
-- debug_no_help ===> 0
-- ftpssl_filehandle ===> GLOB(0x2a6d8c8)
-- last_ftp_msg ===> 234 Proceed with negotiation.
-- myContext ===> HASH(0x2747d18)
-- SSL_ca_file ----> /etc/pki/tls/certs/ca-bundle.crt
-- SSL_reuse_ctx ----> IO::Socket::SSL::SSL_Context=HASH(0x2a8f1f0)
-- context ++++> 44543648
-- mySocketOpts ===> HASH(0x2746ec0)
-- PeerAddr ----> sftp.domain.com
-- PeerPort ----> 21
-- Proto ----> tcp
-- Timeout ----> 120
-- start_SSL_opts ===> HASH(0x272c840)
-- SSL_hostname ----> sftp.domain.com
-- SSL_verify_mode ----> 0
-- SSL_version ----> TLSv12
-- Timeout ----> 120
-- trace ===> 0
-- type ===> A
_SSL_arguments ==> HASH(0x2a8e770)
-- PeerAddr ===> 33.44.55.66
-- PeerPort ===> 21
-- Proto ===> tcp
-- SSL_ca_file ===> /etc/pki/tls/certs/ca-bundle.crt
-- SSL_cert_file ===> certs/client-cert.pem
-- SSL_check_crl ===> 0
-- SSL_honor_cipher_order ===> 0
-- SSL_hostname ===> sftp.domain.com
-- SSL_key_file ===> certs/client-key.pem
-- SSL_server ===> 0
-- SSL_use_cert ===> 0
-- SSL_verify_mode ===> 0
-- SSL_version ===> TLSv12
_SSL_ctx ==> IO::Socket::SSL::SSL_Context=HASH(0x2a8f1f0)
-- context ===> 44543648
_SSL_fileno ==> 4
_SSL_ioclass_upgraded ==> IO::Socket::INET
_SSL_last_err ==> SSL wants a read first
_SSL_object ==> 44657040
_SSL_opened ==> 1
io_socket_domain ==> 2
io_socket_proto ==> 6
io_socket_timeout ==> 120
io_socket_type ==> 1
************************************************************
>>> USER +++++++
<<< 331 Please specify the password.
>>> PASS *******
<<< 230 Login successful.
>>> HELP
<<< 214-The following commands are recognized.
<<< ABOR ACCT ALLO APPE CDUP CWD DELE EPRT EPSV FEAT HELP LIST MDTM MKD
<<< MODE NLST NOOP OPTS PASS PASV PORT PWD QUIT REIN REST RETR RMD RNFR
<<< RNTO SITE SIZE SMNT STAT STOR STOU STRU SYST TYPE USER XCUP XCWD XMKD
<<< XPWD XRMD
<<< 214 Help OK.
>>> FEAT
<<< 211-Features:
<<< AUTH SSL
<<< AUTH TLS
<<< EPRT
<<< EPSV
<<< MDTM
<<< PASV
<<< PBSZ
<<< PROT
<<< REST STREAM
<<< SIZE
<<< TVFS
<<< UTF8
<<< 211 End
<<+ 111 Auto-adding OPTS Command!
>>> HELP SITE
<<< 214-The following commands are recognized.
<<< ABOR ACCT ALLO APPE CDUP CWD DELE EPRT EPSV FEAT HELP LIST MDTM MKD
<<< MODE NLST NOOP OPTS PASS PASV PORT PWD QUIT REIN REST RETR RMD RNFR
<<< RNTO SITE SIZE SMNT STAT STOR STOU STRU SYST TYPE USER XCUP XCWD XMKD
<<< XPWD XRMD
<<< 214 Help OK.
>>> SITE HELP
<<< 214 CHMOD UMASK HELP
<<+ 214 The HELP command is supported.
>>> TYPE I
<<< 200 Switching to Binary mode.
<<< 200 Switching to Binary mode.
>>> PBSZ 0
<<< 200 PBSZ set to 0.
>>> PROT P
<<< 200 PROT now Private.
>>> PASV
<<< 227 Entering Passive Mode (33,44,55,66,20,207).
--- Host (33.44.55.66) Port (5327)
>>> NLST
<<< 150 Here comes the directory listing.
<<< 226 Directory send OK.
>>> CWD demo3
<<< 250 Directory successfully changed.
>>> PBSZ 0
<<< 200 PBSZ set to 0.
>>> PROT P
<<< 200 PROT now Private.
>>> PASV
<<< 227 Entering Passive Mode (33,44,55,66,20,144).
--- Host (33.44.55.66) Port (5264)
>>> NLST
<<< 150 Here comes the directory listing.
<<< 226 Directory send OK.
>>> CDUP
<<< 250 Directory successfully changed.
>>> CDUP
<<< 250 Directory successfully changed.
>>> CWD demo2
<<< 250 Directory successfully changed.
>>> PBSZ 0
<<< 200 PBSZ set to 0.
>>> PROT P
<<< 200 PROT now Private.
>>> PASV
<<< 227 Entering Passive Mode (33,44,55,66,20,179).
--- Host (33.44.55.66) Port (5299)
>>> NLST
<<< 150 Here comes the directory listing.
<<< 226 Directory send OK.
>>> CDUP
<<< 250 Directory successfully changed.
>>> CWD demo
<<< 250 Directory successfully changed.
>>> PBSZ 0
<<< 200 PBSZ set to 0.
>>> PROT P
<<< 200 PROT now Private.
>>> PASV
...
<<< 227 Entering Passive Mode (33,44,55,66,19,196).
--- Host (33.44.55.66) Port (5060)
>>> NLST
<<< 150 Here comes the directory listing.
<<+ 555 TLSv12: SSL connect attempt failed because of handshake problems
>>> PBSZ 0
<<+ 555 Unexpected EOF on Command Channel [0] (0, 1) ()
>>> PROT P
<<+ 555 Can't write command on socket:
UPDATE:
Tried downgrading SSL_version to SSLv3 but it still doesn't work.
$ftp = Net::FTPSSL->new('sftp.domain.com',
Encryption => EXP_CRYPT,
Debug => 1, DebugLogFile => "/usr/logs/ftpssl.log", SSL_version => "SSLv3");
Net::FTP
I created another script using Net::FTP it's works fine until:
Net::FTP>>> Net::FTP(3.11)
Net::FTP>>> Exporter(5.68)
Net::FTP>>> Net::Cmd(3.11)
Net::FTP>>> IO::Socket::SSL(2.072)
Net::FTP>>> IO::Socket::INET(1.33)
Net::FTP>>> IO::Socket(1.34)
Net::FTP>>> IO::Handle(1.33)
Net::FTP=GLOB(0x11d7400)<<< 220 (vsFTPd 3.0.2)
Net::FTP=GLOB(0x11d7400)>>> AUTH TLS
Net::FTP=GLOB(0x11d7400)<<< 234 Proceed with negotiation.
Net::FTP=GLOB(0x11d7400)>>> PBSZ 0
Net::FTP=GLOB(0x11d7400)<<< 200 PBSZ set to 0.
Net::FTP=GLOB(0x11d7400)>>> PROT P
Net::FTP=GLOB(0x11d7400)<<< 200 PROT now Private.
Net::FTP=GLOB(0x11d7400)>>> USER username
Net::FTP=GLOB(0x11d7400)<<< 331 Please specify the password.
Net::FTP=GLOB(0x11d7400)>>> PASS ....
Net::FTP=GLOB(0x11d7400)<<< 230 Login successful.
Net::FTP=GLOB(0x11d7400)>>> TYPE I
Net::FTP=GLOB(0x11d7400)<<< 200 Switching to Binary mode.
Net::FTP=GLOB(0x11d7400)>>> PASV
Net::FTP=GLOB(0x11d7400)<<< 227 Entering Passive Mode (33,44,55,66,21,107).
Net::FTP=GLOB(0x11d7400)>>> NLST
Net::FTP=GLOB(0x11d7400)<<< 150 Here comes the directory listing.
Net::FTP=GLOB(0x11d7400)<<< 226 Directory send OK.
Net::FTP=GLOB(0x11d7400)>>> CWD demo
Net::FTP=GLOB(0x11d7400)<<< 250 Directory successfully changed.
Net::FTP=GLOB(0x11d7400)>>> PASV
...
Net::FTP=GLOB(0x13e2400)<<< 150 Here comes the directory listing.
Net::FTP=GLOB(0x13e2400)<<< 226 Directory send OK.
Net::FTP=GLOB(0x13e2400)>>> CWD demo3
Net::FTP=GLOB(0x13e2400)<<< 250 Directory successfully changed.
Net::FTP=GLOB(0x13e2400)>>> PASV
Net::FTP=GLOB(0x13e2400)<<< 227 Entering Passive Mode (33,44,55,66,19,196).
Net::FTP=GLOB(0x13e2400)>>> NLST
failed to ssl upgrade dataconn: SSL wants a read first at ./upload_ftp.pl line 40.
Net::FTP=GLOB(0x13e2400)<<< 150 Here comes the directory listing.
Net::FTP=GLOB(0x13e2400)>>> CDUP
Net::FTP: Net::Cmd::getline(): unexpected EOF on command channel: at ./upload_ftp.pl line 56.
Net::FTP: Net::Cmd::_is_closed(): unexpected EOF on command channel: at ./upload_ftp.pl line 56.

Hashcat brute-force is not working properly

My hash (MD5): d0bae1f4cb8fc6b2f0d56c8cb7fad99a
In string: 599142142
Command:
hashcat -a3 -m0 d0bae1f4cb8fc6b2f0d56c8cb7fad99a ?d?d?d?d?d?d?d?d?d -O
Result:
Session..........: hashcat
Status...........: Exhausted
Hash.Name........: MD5
Hash.Target......: d0bae1f4cb8fc6b2f0d56c8cb7fad99a
Time.Started.....: Mon Nov 9 10:36:45 2020 (4 secs)
Time.Estimated...: Mon Nov 9 10:36:49 2020 (0 secs)
Guess.Mask.......: ?d?d?d?d?d?d?d?d?d [9]
Guess.Queue......: 1/1 (100.00%)
Speed.#2.........: 259.0 MH/s (0.46ms) # Accel:16 Loops:7 Thr:512 Vec:1
Recovered........: 0/1 (0.00%) Digests
Progress.........: 1000000000/1000000000 (100.00%)
Rejected.........: 0/1000000000 (0.00%)
Restore.Point....: 1000000/1000000 (100.00%)
Restore.Sub.#2...: Salt:0 Amplifier:994-1000 Iteration:0-7
Candidates.#2....: 976488164 -> 688749494
Mask is correct, Hash mode is correct so why it's not cracking?
For example if I specify first 5 digits in my mask, then it's working:
hashcat -a3 -m0 d0bae1f4cb8fc6b2f0d56c8cb7fad99a 59914?d?d?d?d -O
Result:
d0bae1f4cb8fc6b2f0d56c8cb7fad99a:599142142
Session..........: hashcat
Status...........: Cracked
Hash.Name........: MD5
Hash.Target......: d0bae1f4cb8fc6b2f0d56c8cb7fad99a
Time.Started.....: Mon Nov 9 10:40:06 2020 (0 secs)
Time.Estimated...: Mon Nov 9 10:40:06 2020 (0 secs)
Guess.Mask.......: 59914?d?d?d?d [9]
Guess.Queue......: 1/1 (100.00%)
Speed.#2.........: 3346.7 kH/s (0.13ms) # Accel:32 Loops:1 Thr:512 Vec:1
Recovered........: 1/1 (100.00%) Digests
Progress.........: 10000/10000 (100.00%)
Rejected.........: 0/10000 (0.00%)
Restore.Point....: 0/10000 (0.00%)
Restore.Sub.#2...: Salt:0 Amplifier:0-1 Iteration:0-1
Candidates.#2....: 599145234 -> 599149494
What's the problem? Help please.
P.S. hashcat version v6.1.1
This is a bug with version v6.1.1, I was able to reproduce it and after updating to 6.2.1 it works as expected.
I'm assuming you're asking why you don't see the a password, What I do is apply --show at the end of the command.
[hashcat -a3 -m0 d0bae1f4cb8fc6b2f0d56c8cb7fad99a 59914?d?d?d?d --show ]
Applying this will show as:
d0bae1f4cb8fc6b2f0d56c8cb7fad99a:599142142

Why some entries are missing in Perl ENV hash

Perl is giving me an undef value when I access a variable that is supposed to be defined in the %ENV hash. How is this possible?
root#23cd5f45def7:~/bin$ perl -e 'warn $ENV{SHELL}'
Warning: something's wrong at -e line 1.
I would expect perl to output /bin/bash instead.
More info on the environment:
root#23cd5f45def7:~/bin$ echo $SHELL
/bin/bash
root#23cd5f45def7:~/bin$ $SHELL --version
GNU bash, version 4.2.37(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2011 Free Software Foundation, Inc.
...
root#23cd5f45def7:~/bin$ perl -v
This is perl 5, version 14, subversion 2 (v5.14.2) built for x86_64-linux-gnu-thread-multi
(with 91 registered patches, see perl -V for more detail)
...
I am running this on Debian wheezy in a docker container.
The image was created with
sudo debootstrap wheezy ../_build http://ftp.us.debian.org/debian
sudo tar -C ../_build -c . | docker import - wheezy/bootstrap
I get the same beheviour with perl-5.24.1 compiled manually from sources.
The error message
Warning: something's wrong at -e line 1.
indicates that the environment variable $SHELL does not exist or is not exported.
You can list the exported variables using the export command. You can add SHELL to the exported variables using the command:
export SHELL
Something is wrong with your installation. It looks like the string
Warning: something's wrong at -e line 1.
would only ever be produced by the following code on line 461 in pp_sys.c (that is, when the argument to warn is undefined). Therefore, I am going to deduce that something sanitizes the environment before perl is invoked. You might also want to examine root's .profile, .bashrc, .bash_profile and other possibly relevant configuration files.
421 PP(pp_warn)
422 {
423 dSP; dMARK;
424 SV *exsv;
425 STRLEN len;
426 if (SP - MARK > 1) {
427 dTARGET;
428 do_join(TARG, &PL_sv_no, MARK, SP);
429 exsv = TARG;
430 SP = MARK + 1;
431 }
432 else if (SP == MARK) {
433 exsv = &PL_sv_no;
434 EXTEND(SP, 1);
435 SP = MARK + 1;
436 }
437 else {
438 exsv = TOPs;
439 if (SvGMAGICAL(exsv)) exsv = sv_mortalcopy(exsv);
440 }
441
442 if (SvROK(exsv) || (SvPV_const(exsv, len), len)) {
443 /* well-formed exception supplied */
444 }
445 else {
446 SV * const errsv = ERRSV;
447 SvGETMAGIC(errsv);
448 if (SvROK(errsv)) {
449 if (SvGMAGICAL(errsv)) {
450 exsv = sv_newmortal();
451 sv_setsv_nomg(exsv, errsv);
452 }
453 else exsv = errsv;
454 }
455 else if (SvPOKp(errsv) ? SvCUR(errsv) : SvNIOKp(errsv)) {
456 exsv = sv_newmortal();
457 sv_setsv_nomg(exsv, errsv);
458 sv_catpvs(exsv, "\t...caught");
459 }
460 else {
461 exsv = newSVpvs_flags("Warning: something's wrong", SVs_TEMP); ## <-- Here ...
462 }
463 }
464 if (SvROK(exsv) && !PL_warnhook)
465 Perl_warn(aTHX_ "%" SVf, SVfARG(exsv));
466 else warn_sv(exsv);
467 RETSETYES;
468 }

uwsgi long timeouts

I am using ubuntu 12, nginx, uwsgi 1.9 with socket, django 1.5.
Config:
[uwsgi]
base_path = /home/someuser/web/
module = server.manage_uwsgi
uid = www-data
gid = www-data
virtualenv = /home/someuser
master = true
vacuum = true
harakiri = 20
harakiri-verbose = true
log-x-forwarded-for = true
profiler = true
no-orphans = true
max-requests = 10000
cpu-affinity = 1
workers = 4
reload-on-as = 512
listen = 3000
Client tests from Windows7:
C:\Users\user>C:\AppServ\Apache2.2\bin\ab.exe -c 255 -n 5000 http://www.someweb.com/about/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking www.someweb.com (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Finished 5000 requests
Server Software: nginx
Server Hostname: www.someweb.com
Server Port: 80
Document Path: /about/
Document Length: 1881 bytes
Concurrency Level: 255
Time taken for tests: 66.669814 seconds
Complete requests: 5000
Failed requests: 1
(Connect: 1, Length: 0, Exceptions: 0)
Write errors: 0
Total transferred: 10285000 bytes
HTML transferred: 9405000 bytes
Requests per second: 75.00 [#/sec] (mean)
Time per request: 3400.161 [ms] (mean)
Time per request: 13.334 [ms] (mean, across all concurrent requests)
Transfer rate: 150.64 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 8 207.8 1 9007
Processing: 10 3380 11480.5 440 54421
Waiting: 6 1060 3396.5 271 48424
Total: 11 3389 11498.5 441 54423
Percentage of the requests served within a certain time (ms)
50% 441
66% 466
75% 499
80% 519
90% 3415
95% 36440
98% 54407
99% 54413
100% 54423 (longest request)
I have set following options too:
echo 3000 > /proc/sys/net/core/netdev_max_backlog
echo 3000 > /proc/sys/net/core/somaxconn
So,
1) I make first 3000 requests super fast. I see progress in ab and in uwsgi requests logs -
[pid: 5056|app: 0|req: 518/4997] 80.114.157.139 () {30 vars in 378 bytes} [Thu Mar 21 12:37:31 2013] GET /about/ => generated 1881 bytes in 4 msecs (HTTP/1.0 200) 3 headers in 105 bytes (1 switches on core 0)
[pid: 5052|app: 0|req: 512/4998] 80.114.157.139 () {30 vars in 378 bytes} [Thu Mar 21 12:37:31 2013] GET /about/ => generated 1881 bytes in 4 msecs (HTTP/1.0 200) 3 headers in 105 bytes (1 switches on core 0)
[pid: 5054|app: 0|req: 353/4999] 80.114.157.139 () {30 vars in 378 bytes} [Thu Mar 21 12:37:31 2013] GET /about/ => generated 1881 bytes in 4 msecs (HTTP/1.0 200) 3 headers in 105 bytes (1 switches on core 0)
I dont have any broken pipes or worker respawns.
2) Next requests are running very slow or with some timeout. Looks like that some buffer becomes full and I am waiting before it becomes empty.
3) Some buffer becomes empty.
4) ~500 requests are processed super fast.
5) Some timeout.
6) see Nr. 4
7) see Nr. 5
8) see Nr. 4
9) see Nr. 5
....
....
Need your help
check with netstat and dmesg. You have probably exhausted ephemeral ports or filled the conntrack table.