sphinx inconsistent result - sphinx

have some weird problems with sphinx.
Here's the query log:
[Mon Jan 31 05:43:21.362 2011] 0.158 sec [any/0/ext 511 (0,2000)] [_file] superman
[Mon Jan 31 05:43:51.739 2011] 0.143 sec [any/0/ext 952 (0,2000)] [_file] superman
[Mon Jan 31 05:44:22.042 2011] 0.003 sec [any/0/ext 952 (0,2000)] [_file] superman
[Mon Jan 31 05:44:52.313 2011] 0.003 sec [any/0/ext 952 (0,2000)] [_file] superman
[Mon Jan 31 05:45:22.553 2011] 0.003 sec [any/0/ext 952 (0,2000)] [_file] superman
If you see, the result returned is 511 for the first time, then 952 (the correct result) for the rest. I've tried searching with different result and all seems to be the same.
Some observation:
1) If there's less than 511, the result returned is always correct. It's only when the result is > 511 and less than the max that it is wrong.
2) If there are more result than the max, the returned result will be max (correct).
3) The rest of the results are usually correct, up until the sphinx db re-indexed. Then we'll get 511 again.
Tried it on different sphinx installation, getting the same result.
my client code:
$cl->setServer("localhost", 3312);
$cl->setMaxQueryTime(10);
$cl->SetLimits(0, 2000, 2000);
$cl->setMatchMode(SPH_MATCH_ANY);
call_user_func_array(array($cl, 'setSortMode'), array(SPH_SORT_EXTENDED, '#id DESC'));
$result = $cl->query('superman', '_file');
sphinx.conf:
index download_file
{
source = file
path = /disk1/data/sphinx/file
morphology = stem_en
enable_star=1
min_word_len = 3
min_prefix_len = 0
min_infix_len = 3
}
searchd
{
max_matches = 100000
port = 3312
log = /var/log/searchd/searchd.log
query_log = /var/log/searchd/query.log
pid_file = /var/log/searchd/searchd.pid
}
indexer
{
max_iops = 40
mem_limit = 128M
}

Related

Check if file exists on a Cisco switch

I am trying to check if a file exists on the internal flash:/ disk of a Cisco switch.
switch-2950#dir flash:/
Directory of flash:/
2 -rwx 3721946 Jul 24 2009 16:17:10 +00:00 c2950-i6k2l2q4-mz.121-22.EA13.bin
3 -rwx 2035 Mar 01 1993 00:25:01 +00:00 config.text
5 drwx 4416 Jul 24 2009 16:19:50 +00:00 html
6 -rwx 556 Mar 01 1993 00:49:35 +00:00 vlan.dat
335 -rwx 315 Jul 24 2009 17:43:37 +00:00 env_vars
21 -rwx 112 Jul 24 2009 16:10:20 +00:00 info
22 -rwx 112 Jul 24 2009 16:20:56 +00:00 info.ver
23 drwx 64 Mar 01 1993 00:00:11 +00:00 crashinfo
25 -rwx 13495 May 18 2011 19:57:30 +00:00 config.old
336 -rwx 3832 Mar 01 1993 00:25:01 +00:00 private-config.text
7741440 bytes total (2124800 bytes free)
vlan.dat clearly exists. I can perform operations against it (such as copy).
However, I want to test if that particular file exists before performing
operations against it.
I am trying:
if os.path.isfile("flash:/vlan.dat"):
But it always returns False and the commands inside the 'if' statement are skipped over.
I have looked over numerous posts but they all cover Linux or Windows. I can't find anything regarding a Cisco file system.
I solved it another way. I do a 'dir flash:/', then I check for the existence
of the 'vlan.dat' file.
today = time.strftime("%x")
timenow = time.strftime("%X")
filename = (hostname + '-' + '%s' + '_' + '%s') % (today, timenow)
filename = filename.replace("/", "-")
filename = filename.replace(":", "-")
ssh_channel.send("dir flash:/" + "\n")
time.sleep(0.3)
outp = ssh_channel.recv(2000)
output = outp.decode("utf-8")
if 'vlan.dat' in output:
ssh_channel.send("copy flash:/vlan.dat tftp://192.168.1.106/" + filename + ".dat" + "\n")
time.sleep(0.3)

How to install PGlogical from sources on Freebsd?

I have Postgresql 9.5 server on Freebsd. I try to install PGlogical (http://2ndquadrant.com/en/resources/pglogical/) from sources. But when I downloaded tarball and unpacked it, I saw only these files:
pglogical-1.0.1 ls -la
total 480
drwxr-xr-x 6 root wheel 1536 Mar 24 11:51 .
drwxrwxrwt 16 root wheel 1024 Mar 24 15:42 ..
-rw-rw-r-- 1 1000 1000 8 Jan 19 15:39 .distgitrev
-rw-rw-r-- 1 1000 1000 11 Jan 19 15:39 .distgittag
-rw-rw-r-- 1 1000 1000 169 Dec 31 12:33 .gitignore
-rw-rw-r-- 1 1000 1000 216 Dec 27 15:59 .gitmodules
-rw-rw-r-- 1 1000 1000 6240 Jan 18 06:42 Makefile
-rw-rw-r-- 1 1000 1000 23965 Jan 18 06:42 README.md
drwxrwxr-x 4 1000 1000 512 Jan 18 06:42 compat
drwxrwxr-x 2 1000 1000 512 Jan 19 14:39 expected
-rw-rw-r-- 1 1000 1000 893 Jan 19 14:39 pglogical--1.0.0--1.0.1.sql
-rw-rw-r-- 1 1000 1000 9496 Jan 18 06:42 pglogical--1.0.0.sql
-rw-rw-r-- 1 1000 1000 9504 Jan 18 06:42 pglogical--1.0.1.sql
-rw-rw-r-- 1 1000 1000 12670 Jan 18 06:42 pglogical.c
-rw-rw-r-- 1 1000 1000 184 Dec 22 15:27 pglogical.control.in
-rw-rw-r-- 1 1000 1000 1796 Jan 18 06:42 pglogical.h
-rw-rw-r-- 1 1000 1000 43489 Jan 18 06:42 pglogical_apply.c
-rw-rw-r-- 1 1000 1000 13960 Jan 18 06:42 pglogical_conflict.c
-rw-rw-r-- 1 1000 1000 2038 Dec 19 13:27 pglogical_conflict.h
-rw-rw-r-- 1 1000 1000 38525 Jan 18 06:42 pglogical_create_subscriber.c
drwxrwxr-x 2 1000 1000 1024 Jan 18 07:32 pglogical_dump
-rw-rw-r-- 1 1000 1000 1367 Dec 19 13:27 pglogical_fe.c
-rw-rw-r-- 1 1000 1000 533 Dec 1 09:35 pglogical_fe.h
-rw-rw-r-- 1 1000 1000 41913 Jan 18 06:42 pglogical_functions.c
-rw-rw-r-- 1 1000 1000 8748 Dec 19 13:27 pglogical_hooks.c
-rw-rw-r-- 1 1000 1000 4126 Dec 19 13:27 pglogical_manager.c
-rw-rw-r-- 1 1000 1000 23212 Jan 18 06:42 pglogical_node.c
-rw-rw-r-- 1 1000 1000 1992 Dec 19 13:27 pglogical_node.h
-rw-rw-r-- 1 1000 1000 9323 Dec 19 13:27 pglogical_proto.c
-rw-rw-r-- 1 1000 1000 1431 Sep 30 07:51 pglogical_proto.h
-rw-rw-r-- 1 1000 1000 5493 Dec 19 13:27 pglogical_queue.c
-rw-rw-r-- 1 1000 1000 1026 Dec 19 13:27 pglogical_queue.h
-rw-rw-r-- 1 1000 1000 4832 Jan 18 06:42 pglogical_relcache.c
-rw-rw-r-- 1 1000 1000 1147 Dec 19 13:27 pglogical_relcache.h
-rw-rw-r-- 1 1000 1000 24719 Dec 19 13:27 pglogical_repset.c
-rw-rw-r-- 1 1000 1000 2770 Jan 18 06:42 pglogical_repset.h
-rw-rw-r-- 1 1000 1000 4126 Dec 19 13:27 pglogical_rpc.c
-rw-rw-r-- 1 1000 1000 711 Dec 1 09:35 pglogical_rpc.h
-rw-rw-r-- 1 1000 1000 34577 Jan 18 06:42 pglogical_sync.c
-rw-rw-r-- 1 1000 1000 2472 Dec 19 13:27 pglogical_sync.h
-rw-rw-r-- 1 1000 1000 9808 Jan 18 06:42 pglogical_worker.c
-rw-rw-r-- 1 1000 1000 2898 Dec 19 13:27 pglogical_worker.h
-rw-rw-r-- 1 1000 1000 4667 Oct 28 08:21 regress-pg_hba.conf
-rw-rw-r-- 1 1000 1000 602 Jan 18 06:42 regress-postgresql.conf
drwxrwxr-x 2 1000 1000 512 Jan 19 14:39 sql
And I didn't see configure file and other. How to install this tarball? OS - Freebsd 10.2
You can install pglogical from sources only using PostgreSQL sources:
Download sources of PostgreSQL of necessary version.
You also need pglogical_output extension and its sources. You can read about it in the documentation. Download it from GitHub.
Configure PostgreSQL. More about it in the documentation.
Copy downloaded sources of pglogical and pglogical_output to directories contrib/pglogical and contrib/pglogical_output.
Execute commands:
cd <path-to-postgres-sources>/contrib/pglogical
make install
cd <path-to-postgres-sources>/contrib/pglogical_output
make install
Read the documentation to setup pglogical.
Did you read the README file? It will probably tell you to use make, as there is a Makefile

uwsgi deployment using Emperor: Repeated "announcing my loyalty to the emperor"

Using uwsgi running two psgi Perl Dancer apps.
Is it normal for uwsgi vassals to repeatedly announce their loyalty to the Emperor, upon nearly every request?
Here's a small portion of my uwsgi.log file:
announcing my loyalty to the Emperor...
Mon Aug 17 20:51:59 2015 - [emperor] vassal www.ini is now loyal
[pid: 1713|app: 0|req: 4/11] 0.0.0.0 () {44 vars in 873 bytes} [Mon Aug 17 20:52:12 2015] GET /sitemap-index.xml => generated 284 bytes in 7 msecs (HTTP/1.1 200) 4 headers in 146 bytes (0 switches on core 0)
[pid: 1706|app: 0|req: 2/12] 0.0.0.0 () {42 vars in 808 bytes} [Mon Aug 17 20:52:22 2015] GET / => generated 113840 bytes in 207 msecs (HTTP/1.1 200) 4 headers in 143 bytes (0 switches on core 0)
[pid: 1709|app: 0|req: 1/13] 0.0.0.0 () {42 vars in 844 bytes} [Mon Aug 17 20:52:35 2015] GET /about => generated 124031 bytes in 1325 msecs (HTTP/1.1 200) 4 headers in 143 bytes (0 switches on core 0)
announcing my loyalty to the Emperor...
Mon Aug 17 20:52:36 2015 - [emperor] vassal www.ini is now loyal
[pid: 1713|app: 0|req: 5/14] 0.0.0.0 () {44 vars in 865 bytes} [Mon Aug 17 20:52:38 2015] GET / => generated 113840 bytes in 129 msecs (HTTP/1.1 200) 4 headers in 143 bytes (0 switches on core 0)
These announcements of loyalty to the Emperor appear to occur with nearly every request.
Are these poor vassals trying to kiss up to the Emperor for special favors, or (more likely) is there something wrong with my configuration?

copy specific file in command line

I want to copy specific file done last changes in Oct 16-17,file type is java.
shia#ubuntu:~/code$ ls -alxo
total 96
drwx------ 2 shia 4096 Oct 20 18:54 .
drwxr-xr-x 61 shia 12288 Oct 20 19:24 ..
-rw------- 1 shia 12288 Oct 16 21:52 .Reuse.java.swp
-rw-rw-r-- 1 shia 746 Oct 20 11:16 Argus.class
-rw-rw-r-- 1 shia 302 Oct 20 11:16 Argus.java
-rw------- 1 shia 310 Oct 16 21:30 Call.java
-rw-rw-r-- 1 shia 417 Oct 17 15:20 Ordinary.class
-rw-rw-r-- 1 shia 298 Oct 17 14:57 Overriding.java
-rw-rw-r-- 1 shia 562 Oct 19 21:27 Package.class
-rw-rw-r-- 1 shia 430 Oct 19 21:27 Package.java
-rw------- 1 shia 729 Oct 17 13:50 Reuse.java
-rw------- 1 shia 424 Oct 17 13:47 Room.java
-rw------- 1 shia 321 Oct 16 21:22 Simpleobject.java
-rw-rw-r-- 1 shia 1187 Oct 17 00:04 Static.java
-rw-rw-r-- 1 shia 686 Oct 17 15:20 Super.class
-rw-rw-r-- 1 shia 1010 Oct 17 15:20 Super.java
-rw------- 1 shia 843 Oct 17 14:20 This.java
-rw-rw-r-- 1 shia 521 Oct 17 14:51 b.java
-rw-rw-r-- 1 shia 90 Oct 20 18:54 cp.awk
-rw-rw-r-- 1 shia 105 Oct 20 17:19 file.txt
I try to specific them but i don't know how to copy them.
shia#ubuntu:~/code$ ls -alxo|grep 'Oct 1[67].*java$'|awk '{print $8}'
Call.java
Overriding.java
Reuse.java
Room.java
Simpleobject.java
Static.java
Super.java
This.java
b.java
Any help,thanks a lot!
One way using find:
find . -maxdepth 1 -type f -name "*.java" -newermt 2012-10-16 ! -newermt 2012-10-18 -exec cp '{}' /home/user/dstFolder/ \;
You can use xargs to copy the files found:
...| xargs -i cp '{}' /home/user/dstFolder/
This will copy all the files found to the folder /home/user/dstFolder/.

Move Values In Perl Array

I am having difficulty placing array columns in a format that is consistent. I have the following output:
Mon,Jun,25,14:39:29,2012,971,29,0,25,0,0,0,4,Mon,Jun,25,14:39:29,2012,25,mod_was_ap22_http.c
Mon,Jun,25,14:40:29,2012,972,28,0,25,0,0,0,3,Mon,Jun,25,14:40:29,2012,3,mod_sm22.cpp,22,mod_was_ap22_http.c
Mon,Jun,25,14:41:29,2012,973,27,0,24,0,0,0,3,Mon,Jun,25,14:41:29,2012,24,mod_was_ap22_http.c
Mon,Jun,25,14:42:29,2012,974,26,0,20,0,0,0,6,Mon,Jun,25,14:42:29,2012,1,mod_sm22.cpp,19,mod_was_ap22_http.c
Mon,Jun,25,14:43:29,2012,971,29,0,26,0,0,0,3,Mon,Jun,25,14:43:29,2012,2,mod_sm22.cpp,24,mod_was_ap22_http.c
Mon,Jun,25,14:44:30,2012,957,43,0,41,0,0,0,2,Mon,Jun,25,14:44:30,2012,1,mod_sm22.cpp,40,mod_was_ap22_http.c
Mon,Jun,25,14:45:30,2012,963,37,0,35,0,0,0,2,Mon,Jun,25,14:45:30,2012,2,mod_sm22.cpp,32,mod_was_ap22_http.c
Mon,Jun,25,14:46:30,2012,972,28,0,24,1,1,0,2,Mon,Jun,25,14:46:30,2012,24,mod_was_ap22_http.c,1,ApacheModule.cpp
Mon,Jun,25,14:47:30,2012,961,39,1,37,0,0,0,1,Mon,Jun,25,14:47:30,2012,37,mod_was_ap22_http.c,1,ApacheModule.cpp
Mon,Jun,25,14:48:30,2012,968,32,0,30,0,0,0,2,Mon,Jun,25,14:48:30,2012,30,mod_was_ap22_http.c
Mon,Jun,25,14:49:30,2012,972,28,0,25,0,0,0,3,Mon,Jun,25,14:49:30,2012,1,mod_sm22.cpp,24,mod_was_ap22_http.c
I would like the columns to display:
DayOfWeek,Month,Day,Time,Year,Rdy,Bsy,Rd,Wr,Ka,Log,Dns,Cls,AP22,SM22,ApacheModule
Currently the columns in bold are not in that order (the rest are correct). Each line isn't consistent with that format. The line sometimes has ap22 first, sometimes has sm22 first, and sometimes has none or all three modules. The number before the module relates to the module. How can I move the data into a consistent format?
Note that the 2nd date, mod_was_http.c, mod_sm22.cpp, and ApacheModule.cpp in each line will be removed in the final array.
Here is my code so far:
# This program parses a error log for necessary information and outputs in CSV format.
# chunks of your input to ignore, see below...
my %ignorables = map { $_ => 1 } qw([notice mpmstats: rdy bsy rd wr ka log dns cls bsy: in);
# 3-arg open is safer than 2, lexical my $fh better than a global FH glob
open my $error_fh, '<', 'iset_error_log';
sub findLines {
my($item,#result)=("");
# Iterates over the lines in the file, putting each into $_
while (<$error_fh>) {
# Select only those fields that have the word 'notice'
if (/\[notice/) {
# Place those lines with the word 'rdy' on the next line
if (/\brdy\b/){
push #result,"$item\n";
$item="";
}
else {
$item.=",";
}
# Split the line into fields, separated by spaces, skip the %ignorables
my #line = grep { not defined $ignorables{$_} } split /\s+/;
# More cleanup
s/|^\[|notice|[]]//g for #line; # remove unnecessary elements from the array
# Output the line.
#line = join(",", #line);
s/,,/,/g for #line;
map $item.=$_, #line;
}
}
#result
}
my #array = &findLines;
foreach $line (#array){
print $line; #This is where I would like to organize the lines if possible.
}
My input file looks like this:
[Mon Jun 25 07:51:17 2012] [notice] mpmstats: rdy 990 bsy 10 rd 0 wr 7 ka 0 log 0 dns 0 cls 3
[Mon Jun 25 07:51:17 2012] [notice] mpmstats: bsy: 2 in mod_sm22.cpp, 5 in mod_was_ap22_http.c
[Mon Jun 25 08:08:17 2012] [notice] mpmstats: rdy 974 bsy 26 rd 1 wr 24 ka 0 log 0 dns 0 cls 1
[Mon Jun 25 08:08:17 2012] [notice] mpmstats: bsy: 1 in mod_sm22.cpp, 23 in mod_was_ap22_http.c, 1 in ApacheModule.cpp Mon,Jun,25,14:38:29,2012,962,38,0,36,0,0,0,2,Mon,Jun,25,14:38:29,2012,3,mod_sm22.cpp,33,mod_was_ap22_http.c
[Mon Jun 25 21:54:41 2012] [notice] mpmstats: rdy 999 bsy 1 rd 0 wr 0 ka 0 log 0 dns 0 cls 1
[Mon Jun 25 21:55:41 2012] [notice] mpmstats: rdy 999 bsy 1 rd 0 wr 0 ka 0 log 0 dns 0 cls 1
[Mon Jun 25 21:59:41 2012] [notice] mpmstats: rdy 999 bsy 1 rd 0 wr 1 ka 0 log 0 dns 0 cls 0
[Mon Jun 25 21:59:41 2012] [notice] mpmstats: bsy: 1 in mod_was_ap22_http.c
[Mon Jun 25 22:00:41 2012] [notice] mpmstats: rdy 999 bsy 1 rd 0 wr 1 ka 0 log 0 dns 0 cls 0
[Mon Jun 25 22:00:41 2012] [notice] mpmstats: bsy: 1 in mod_was_ap22_http.c
[Mon Jun 25 22:03:41 2012] [notice] mpmstats: rdy 998 bsy 2 rd 0 wr 2 ka 0 log 0 dns 0 cls 0
[Mon Jun 25 22:03:41 2012] [notice] mpmstats: bsy: 2 in mod_was_ap22_http.c
[Mon Jun 25 22:08:42 2012] [notice] mpmstats: rdy 998 bsy 2 rd 0 wr 2 ka 0 log 0 dns 0 cls 0
[Mon Jun 25 22:08:42 2012] [notice] mpmstats: bsy: 2 in mod_was_ap22_http.c
[Mon Jun 25 22:21:42 2012] [notice] mpmstats: rdy 999 bsy 1 rd 0 wr 1 ka 0 log 0 dns 0 cls 0
[Mon Jun 25 22:21:42 2012] [notice] mpmstats: bsy: 1 in mod_was_ap22_http.c
[Mon Jun 25 22:22:42 2012] [notice] mpmstats: rdy 999 bsy 1 rd 0 wr 1 ka 0 log 0 dns 0 cls 0
[Mon Jun 25 22:22:42 2012] [notice] mpmstats: bsy: 1 in mod_was_ap22_http.c
[Mon Jun 25 22:31:42 2012] [notice] mpmstats: rdy 999 bsy 1 rd 0 wr 0 ka 0 log 0 dns 0 cls 1
[Mon Jun 25 22:32:42 2012] [notice] mpmstats: rdy 999 bsy 1 rd 0 wr 1 ka 0 log 0 dns 0 cls 0
[Mon Jun 25 22:32:42 2012] [notice] mpmstats: bsy: 1 in mod_was_ap22_http.c
[Mon Jun 25 23:06:43 2012] [notice] mpmstats: rdy 999 bsy 1 rd 0 wr 1 ka 0 log 0 dns 0 cls 0
[Mon Jun 25 23:06:43 2012] [notice] mpmstats: bsy: 1 in mod_was_ap22_http.c
You probably want to reorder the columns while you still have the split, before you use join to turn them back into a line of text.
You just need to do a swap.
# 0 ,1 ,2 ,3 ,4 ,5 ,6 ,7 ,8 ,9 ,10 ,11 ,12 ,13 ,14 ,15
# DayOfWeek,Month,Day,Time,Year,Rdy,Bsy,Rd,Wr,Ka,Log,Dns,Cls,AP22,SM22,ApacheModule
#
# Sometimes the last 2 fields are missing and 13 comes before 14 and 15 in the
# input, so fix that.
if (#line < 16) {
push #line, '', ''; # or whatever you want for blanks
}
#line = #line[0..12,14,15,13]; # rearrange the array
Also, your regular expression s/,,/,/g is going to break this if you use blank strings ('') as your empty fields. The short lines missing the last fields will go back to being short missing the correct 13 and 14 fields.
Based upon the kinds of questions being asked here and previously, I highly recommend you get a copy of Modern Perl (available for download or purchase) or Learning Perl to get a better grasp of the language as a whole. I have recently read much of the former and enjoyed it and gained most of my initial Perl knowledge from an earlier edition of the latter.