I'm trying to build a 3-monitor Ceph cluster with 5 OSDs using the manual documentation : http://docs.ceph.com/docs/master/install/manual-deployment/#monitor-bootstrapping.
Since I'm using Debian Buster on arm64, there no pre-built package for debian-luminous, but I built my own (apt-get source ceph-mgr;cd ceph-12*;dpkg-buildpackage). The end result is available here : https://github.com/sebt3/packages.
Deploying the 1st monitor works rather fine, but when I'm adding a second monitor (using http://docs.ceph.com/docs/master/rados/operations/add-or-rm-mons/) I encounter a problem. The second monitor doesnt connect. Starting it manually (/usr/bin/ceph-mon -f --cluster ceph --id nano13 --setuser ceph --setgroup ceph -d --debug_ms 5), I'm guetting these errors :
2018-09-04 18:18:41.077937 7f89c86790 0 mon.nano13#-1(synchronizing) e1 ms_verify_authorizer bad authorizer from mon 192.168.10.12:6789/0
2018-09-04 18:18:41.078176 7f89c86790 0 -- 192.168.10.13:6789/0 >> 192.168.10.12:6789/0 conn(0x55a4947800 :-1 s=STATE_ACCEPTING_WAIT_CONNECT_MSG_AUTH pgs=300 cs=73 l=0).handle_connect_msg: challenging authorizer
2018-09-04 18:18:41.082448 7f89c86790 2 -- 192.168.10.13:6789/0 >> 192.168.10.12:6789/0 conn(0x55a4947800 :-1 s=STATE_ACCEPTING_WAIT_SEQ pgs=302 cs=75 l=0).handle_connect_msg accept write reply msg done
2018-09-04 18:18:41.085526 7f89c86790 2 -- 192.168.10.13:6789/0 >> 192.168.10.12:6789/0 conn(0x55a4947800 :-1 s=STATE_ACCEPTING_WAIT_SEQ pgs=302 cs=75 l=0)._process_connection accept get newly_acked_seq 3
2018-09-04 18:18:41.097364 7f89c86790 0 bad crc in front 4144944612 != exp 3525444667
2018-09-04 18:18:41.097557 7f89c86790 1 -- 192.168.10.13:6789/0 >> 192.168.10.12:6789/0 conn(0x55a4947800 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=302 cs=75 l=0).process decode message failed
2018-09-04 18:18:41.102803 7f8ac88790 1 -- 192.168.10.13:6789/0 >> - conn(0x55a4981000 :6789 s=STATE_ACCEPTING pgs=0 cs=0 l=0)._process_connection sd=29 -
2018-09-04 18:18:41.145781 7f8ac88790 0 mon.nano13#-1(synchronizing) e1 ms_verify_authorizer bad authorizer from mon 192.168.10.12:6789/0
2018-09-04 18:18:41.146050 7f8ac88790 0 -- 192.168.10.13:6789/0 >> 192.168.10.12:6789/0 conn(0x55a4981000 :6789 s=STATE_ACCEPTING_WAIT_CONNECT_MSG_AUTH pgs=0 cs=0 l=0).handle_connect_msg: challenging authorizer
2018-09-04 18:18:41.149062 7f8ac88790 0 -- 192.168.10.13:6789/0 >> 192.168.10.12:6789/0 conn(0x55a4981000 :6789 s=STATE_ACCEPTING_WAIT_CONNECT_MSG_AUTH pgs=0 cs=0 l=0).handle_connect_msg accept connect_seq 76 vs existing csq=75 existing_state=STATE_STANDBY
2018-09-04 18:18:41.149336 7f8ac88790 2 -- 192.168.10.13:6789/0 >> 192.168.10.12:6789/0 conn(0x55a4981000 :6789 s=STATE_ACCEPTING_WAIT_CONNECT_MSG_AUTH pgs=0 cs=0 l=0)._stop
2018-09-04 18:18:41.149481 7f8ac88790 1 -- 192.168.10.13:6789/0 >> 192.168.10.12:6789/0 conn(0x55a4981000 :6789 s=STATE_CLOSED pgs=0 cs=0 l=0).handle_connect_msg stop myself to swap existing
2018-09-04 18:18:41.153284 7f8ac88790 0 mon.nano13#-1(synchronizing) e1 ms_verify_authorizer bad authorizer from mon 192.168.10.12:6789/0
2018-09-04 18:18:41.153537 7f8ac88790 0 -- 192.168.10.13:6789/0 >> 192.168.10.12:6789/0 conn(0x55a4947800 :-1 s=STATE_ACCEPTING_WAIT_CONNECT_MSG_AUTH pgs=302 cs=75 l=0).handle_connect_msg: challenging authorizer
2018-09-04 18:18:41.157265 7f8ac88790 2 -- 192.168.10.13:6789/0 >> 192.168.10.12:6789/0 conn(0x55a4947800 :-1 s=STATE_ACCEPTING_WAIT_SEQ pgs=304 cs=77 l=0).handle_connect_msg accept write reply msg done
2018-09-04 18:18:41.159938 7f8ac88790 2 -- 192.168.10.13:6789/0 >> 192.168.10.12:6789/0 conn(0x55a4947800 :-1 s=STATE_ACCEPTING_WAIT_SEQ pgs=304 cs=77 l=0)._process_connection accept get newly_acked_seq 3
2018-09-04 18:18:41.171493 7f8ac88790 0 bad crc in front 4144944612 != exp 3525444667
Any pointer would be more than welcome :)
Related
I am following the tutorial in the documentation (https://snakemake.readthedocs.io/en/stable/tutorial/advanced.html) and have been stuck on the "Step 4: Rule parameter" exercise. I would like to access a float from my config file using a wildcard in my params directive.
I seem to be getting the same error whenever I run snakemake -np in the command line:
InputFunctionException in line 46 of /mnt/c/Users/Matt/Desktop/snakemake-tutorial/Snakefile:
Error:
AttributeError: 'Wildcards' object has no attribute 'sample'
Wildcards:
Traceback:
File "/mnt/c/Users/Matt/Desktop/snakemake-tutorial/Snakefile", line 14, in get_bcftools_call_priors
This is my code so far
import time
configfile: "config.yaml"
rule all:
input:
"plots/quals.svg"
def get_bwa_map_input_fastqs(wildcards):
print(wildcards.__dict__, 1, time.time()) #I have this print as a check
return config["samples"][wildcards.sample]
def get_bcftools_call_priors(wildcards):
print(wildcards.__dict__, 2, time.time()) #I have this print as a check
return config["prior_mutation_rates"][wildcards.sample]
rule bwa_map:
input:
"data/genome.fa",
get_bwa_map_input_fastqs
#lambda wildcards: config["samples"][wildcards.sample]
output:
"mapped_reads/{sample}.bam"
params:
rg=r"#RG\tID:{sample}\tSM:{sample}"
threads: 2
shell:
"bwa mem -R '{params.rg}' -t {threads} {input} | samtools view -Sb - > {output}"
rule samtools_sort:
input:
"mapped_reads/{sample}.bam"
output:
"sorted_reads/{sample}.bam"
shell:
"samtools sort -T sorted_reads/{wildcards.sample} "
"-O bam {input} > {output}"
rule samtools_index:
input:
"sorted_reads/{sample}.bam"
output:
"sorted_reads/{sample}.bam.bai"
shell:
"samtools index {input}"
rule bcftools_call:
input:
fa="data/genome.fa",
bam=expand("sorted_reads/{sample}.bam", sample=config["samples"]),
bai=expand("sorted_reads/{sample}.bam.bai", sample=config["samples"])
#prior=get_bcftools_call_priors
params:
prior=get_bcftools_call_priors
output:
"calls/all.vcf"
shell:
"samtools mpileup -g -f {input.fa} {input.bam} | "
"bcftools call -P {params.prior} -mv - > {output}"
rule plot_quals:
input:
"calls/all.vcf"
output:
"plots/quals.svg"
script:
"scripts/plot-quals.py"
and here is my config.yaml
samples:
A: data/samples/A.fastq
#B: data/samples/B.fastq
#C: data/samples/C.fastq
prior_mutation_rates:
A: 1.0e-4
#B: 1.0e-6
I don't understand why my input function call in bcftools_call says that the wildcards object is empty of attributes, yet an almost identical function call in bwa_map has the attribute sample that I want. From the documentation it seems like the wildcards would be propogated before anything is run, so why is it missing?
This is the full output of the commandline call snakemake -np:
{'_names': {'sample': (0, None)}, '_allowed_overrides': ['index', 'sort'], 'index': functools.partial(<function Namedlist._used_attribute at 0x7f91b1a58f70>, _name='index'), 'sort': functools.partial(<function Namedlist._used_attribute at 0x7f91b1a58f70>, _name='sort'), 'sample': 'A'} 1 1628877061.8831172
Job stats:
job count min threads max threads
-------------- ------- ------------- -------------
all 1 1 1
bcftools_call 1 1 1
bwa_map 1 1 1
plot_quals 1 1 1
samtools_index 1 1 1
samtools_sort 1 1 1
total 6 1 1
[Fri Aug 13 10:51:01 2021]
rule bwa_map:
input: data/genome.fa, data/samples/A.fastq
output: mapped_reads/A.bam
jobid: 4
wildcards: sample=A
resources: tmpdir=/tmp
bwa mem -R '#RG\tID:A\tSM:A' -t 1 data/genome.fa data/samples/A.fastq | samtools view -Sb - > mapped_reads/A.bam
[Fri Aug 13 10:51:01 2021]
rule samtools_sort:
input: mapped_reads/A.bam
output: sorted_reads/A.bam
jobid: 3
wildcards: sample=A
resources: tmpdir=/tmp
samtools sort -T sorted_reads/A -O bam mapped_reads/A.bam > sorted_reads/A.bam
[Fri Aug 13 10:51:01 2021]
rule samtools_index:
input: sorted_reads/A.bam
output: sorted_reads/A.bam.bai
jobid: 5
wildcards: sample=A
resources: tmpdir=/tmp
samtools index sorted_reads/A.bam
[Fri Aug 13 10:51:01 2021]
rule bcftools_call:
input: data/genome.fa, sorted_reads/A.bam, sorted_reads/A.bam.bai
output: calls/all.vcf
jobid: 2
resources: tmpdir=/tmp
{'_names': {}, '_allowed_overrides': ['index', 'sort'], 'index': functools.partial(<function Namedlist._used_attribute at 0x7f91b1a58f70>, _name='index'), 'sort': functools.partial(<function Namedlist._used_attribute at 0x7f91b1a58f70>, _name='sort')} 2 1628877061.927639
InputFunctionException in line 46 of /mnt/c/Users/Matt/Desktop/snakemake-tutorial/Snakefile:
Error:
AttributeError: 'Wildcards' object has no attribute 'sample'
Wildcards:
Traceback:
File "/mnt/c/Users/Matt/Desktop/snakemake-tutorial/Snakefile", line 14, in get_bcftools_call_priors
If anyone knows what is going wrong I would really appreciate an explaination. Also if there is a better way of getting information out of the config.yaml into the different directives, I would gladly appreciate those tips.
Edit:
I have searched around the internet quite a bit, but have yet to understand this issue.
Wildcards for each rule are based on that rule's output file(s). The rule bcftools_call has one output file (calls/all.vcf), which has no wildcards. Because of this, when get_bcftools_call_priors is called, it throws an exception when it tries to access the unset wildcards.sample attribute.
You should probably set a global prior_mutation_rate in your config file and then access that in the bcftools_call rule:
rule bcftools_call:
...
params:
prior=config["prior_mutation_rate"],
I installed on CentOs successfully ever. However, here is another CentOs I used, and it failed to stared rabbitMq.
My erlang from here.
[rabbitmq-erlang]
name=rabbitmq-erlang
baseurl=https://dl.bintray.com/rabbitmq/rpm/erlang/20/el/7
gpgcheck=1
gpgkey=https://dl.bintray.com/rabbitmq/Keys/rabbitmq-release-signing-key.asc
repo_gpgcheck=0
enabled=1
this is my erl_crash.dump.
erl_crash_dump:0.5
Sat Jun 23 09:17:30 2018
Slogan: init terminating in do_boot ({error,{no such file or directory,asn1.app}})
System version: Erlang/OTP 20 [erts-9.3.3] [source] [64-bit] [smp:24:24] [ds:24:24:10] [async-threads:384] [hipe] [kernel-poll:true]
Compiled: Tue Jun 19 22:25:03 2018
Taints: erl_tracer,zlib
Atoms: 14794
Calling Thread: scheduler:2
=scheduler:1
Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING
Scheduler Sleep Info Aux Work:
Current Port:
Run Queue Max Length: 0
Run Queue High Length: 0
Run Queue Normal Length: 0
Run Queue Low Length: 0
Run Queue Port Length: 0
Run Queue Flags: OUT_OF_WORK | HALFTIME_OUT_OF_WORK
Current Process:
=scheduler:2
Scheduler Sleep Info Flags:
Scheduler Sleep Info Aux Work: THR_PRGR_LATER_OP
Current Port:
Run Queue Max Length: 0
Run Queue High Length: 0
Run Queue Normal Length: 0
Run Queue Low Length: 0
Run Queue Port Length: 0
Run Queue Flags: OUT_OF_WORK | HALFTIME_OUT_OF_WORK | NONEMPTY | EXEC
Current Process: <0.0.0>
Current Process State: Running
Current Process Internal State: ACT_PRIO_NORMAL | USR_PRIO_NORMAL | PRQ_PRIO_NORMAL | ACTIVE | RUNNING | TRAP_EXIT | ON_HEAP_MSGQ
Current Process Program counter: 0x00007fbd81fa59c0 (init:boot_loop/2 + 64)
Current Process CP: 0x0000000000000000 (invalid)
how to identify this problem ? Thank you.
I have TORQUE installed on Ubuntu 16.04, and I am having trouble because my jobs hang. I have a test script test.pbs:
#PBS -N test
#PBS -l nodes=1:ppn=1
#PBS -l walltime=0:01:00
cd $PBS_O_WORKDIR
touch done.txt
echo "done"
And I run it with
qsub test.pbs
The job writes done.txt and echoes "done" just fine, but the job hangs in the C state.
Job id Name User Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
46.localhost test wlandau 00:00:00 C batch
Edit: some diagnostic info on another job from qstat -f 55
qstat -f 55
Job Id: 55.localhost
Job_Name = test
Job_Owner = wlandau#localhost
resources_used.cput = 00:00:00
resources_used.mem = 0kb
resources_used.vmem = 0kb
resources_used.walltime = 00:00:00
job_state = C
queue = batch
server = haggunenon
Checkpoint = u
ctime = Mon Oct 30 07:35:00 2017
Error_Path = localhost:/home/wlandau/Desktop/test.e55
exec_host = localhost/2
Hold_Types = n
Join_Path = n
Keep_Files = n
Mail_Points = a
mtime = Mon Oct 30 07:35:00 2017
Output_Path = localhost:/home/wlandau/Desktop/test.o55
Priority = 0
qtime = Mon Oct 30 07:35:00 2017
Rerunable = True
Resource_List.ncpus = 1
Resource_List.nodect = 1
Resource_List.nodes = 1:ppn=1
Resource_List.walltime = 00:01:00
session_id = 5115
Variable_List = PBS_O_QUEUE=batch,PBS_O_HOST=localhost,
PBS_O_HOME=/home/wlandau,PBS_O_LANG=en_US.UTF-8,PBS_O_LOGNAME=wlandau,
PBS_O_PATH=/home/wlandau/bin:/home/wlandau/.local/bin:/usr/local/sbin
:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/ga
mes:/snap/bin,PBS_O_SHELL=/bin/bash,PBS_SERVER=localhost,
PBS_O_WORKDIR=/home/wlandau/Desktop
comment = Job started on Mon Oct 30 at 07:35
etime = Mon Oct 30 07:35:00 2017
exit_status = 0
submit_args = test.pbs
start_time = Mon Oct 30 07:35:00 2017
Walltime.Remaining = 60
start_count = 1
fault_tolerant = False
comp_time = Mon Oct 30 07:35:00 2017
And a similar tracejob -n2 62:
/var/spool/torque/server_priv/accounting/20171029: No matching job records located
/var/spool/torque/server_logs/20171029: No matching job records located
/var/spool/torque/mom_logs/20171029: No matching job records located
/var/spool/torque/sched_logs/20171029: No matching job records located
Job: 62.localhost
10/30/2017 17:20:25 S enqueuing into batch, state 1 hop 1
10/30/2017 17:20:25 S Job Queued at request of wlandau#localhost, owner =
wlandau#localhost, job name = jobe945093c2e029c5de5619d6bf7922071,
queue = batch
10/30/2017 17:20:25 S Job Modified at request of Scheduler#Haggunenon
10/30/2017 17:20:25 S Exit_status=0 resources_used.cput=00:00:00 resources_used.mem=0kb
resources_used.vmem=0kb resources_used.walltime=00:00:00
10/30/2017 17:20:25 L Job Run
10/30/2017 17:20:25 S Job Run at request of Scheduler#Haggunenon
10/30/2017 17:20:25 S Not sending email: User does not want mail of this type.
10/30/2017 17:20:25 S Not sending email: User does not want mail of this type.
10/30/2017 17:20:25 M job was terminated
10/30/2017 17:20:25 M obit sent to server
10/30/2017 17:20:25 A queue=batch
10/30/2017 17:20:25 M scan_for_terminated: job 62.localhost task 1 terminated, sid=17917
10/30/2017 17:20:25 A user=wlandau group=wlandau
jobname=jobe945093c2e029c5de5619d6bf7922071 queue=batch
ctime=1509398425 qtime=1509398425 etime=1509398425 start=1509398425
owner=wlandau#localhost exec_host=localhost/0 Resource_List.ncpus=1
Resource_List.neednodes=1 Resource_List.nodect=1
Resource_List.nodes=1 Resource_List.walltime=01:00:00
10/30/2017 17:20:25 A user=wlandau group=wlandau
jobname=jobe945093c2e029c5de5619d6bf7922071 queue=batch
ctime=1509398425 qtime=1509398425 etime=1509398425 start=1509398425
owner=wlandau#localhost exec_host=localhost/0 Resource_List.ncpus=1
Resource_List.neednodes=1 Resource_List.nodect=1
Resource_List.nodes=1 Resource_List.walltime=01:00:00 session=17917
end=1509398425 Exit_status=0 resources_used.cput=00:00:00
resources_used.mem=0kb resources_used.vmem=0kb
resources_used.walltime=00:00:00
EDIT: jobs now hanging in E
After some tinkering, I am now using these settings. I have moved on to this tiny pipeline workflow, where some TORQUE jobs wait for other TORQUE jobs to finish. Unfortunately, all the jobs hang in the E state, and any number of jobs more than 4 will just stay queued. To keep things from hanging indefinitely, I have to sudo qdel -p each one, which I think is causing legitimate problems with the project's filesystem as well as an inconvenience.
Job id Name User Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
113.localhost ...b73ec2cda6dca wlandau 00:00:00 E batch
114.localhost ...b6c8e6da05983 wlandau 00:00:00 E batch
115.localhost ...9123b8e20850b wlandau 00:00:00 E batch
116.localhost ...e6d49a3d7d822 wlandau 00:00:00 E batch
117.localhost ...8c3f6cb68927b wlandau 0 Q batch
118.localhost ...40b1d0cab6400 wlandau 0 Q batch
qmgr -c "list server" shows
Server haggunenon
server_state = Active
scheduling = True
max_running = 300
total_jobs = 5
state_count = Transit:0 Queued:1 Held:0 Waiting:0 Running:1 Exiting:3
acl_hosts = localhost
managers = root#localhost
operators = root#localhost
default_queue = batch
log_events = 511
mail_from = adm
query_other_jobs = True
resources_assigned.ncpus = 4
resources_assigned.nodect = 4
scheduler_iteration = 600
node_check_rate = 150
tcp_timeout = 6
mom_job_sync = True
pbs_version = 2.4.16
keep_completed = 0
submit_hosts = SERVER
allow_node_submit = True
next_job_number = 119
net_counter = 118 94 93
And qmgr -c "list queue batch"
Queue batch
queue_type = Execution
total_jobs = 5
state_count = Transit:0 Queued:1 Held:0 Waiting:0 Running:0 Exiting:4
max_running = 300
resources_max.ncpus = 4
resources_max.nodes = 2
resources_min.ncpus = 1
resources_default.ncpus = 1
resources_default.nodect = 1
resources_default.nodes = 1
resources_default.walltime = 01:00:00
mtime = Wed Nov 1 07:40:45 2017
resources_assigned.ncpus = 4
resources_assigned.nodect = 4
keep_completed = 0
enabled = True
started = True
C state means the job has completed and its status is kept in the system. Usually the status is kept after job completion for a period of time specified by the keep_completed parameter. However certain types of failure may result in the job being kept in this state to provide the information necessary to examine the cause of failure.
Check the output of qstat -f 46 to see if there is anything indicating an error.
To tune the keep_completed parameter you can execute the following command to check the value of this parameter on your system.
qmgr -c "print queue batch keep_completed"
If you have administrative privileges on the Torque server you could also change this value with
qmgr -c "set queue batch keep_completed=120"
To keep jobs in completed state for 2 minutes after completion.
In general having keep_completed set is a useful feature. Advanced schedulers use the information on completed jobs to schedule around failures.
I want to make changes to some EDBs and then find out how the IDBs changed as a result.
The docs say that the query stage comes after the final stage and "has access to the effects of stage FINAL". But if I run
query '_(id) <- ^level(id; _).'
(where level is an IDB) I get
block block_1Z7PZ61E: line 2: error: predicate level is an IDB, therefore deltas for it will not be available until stage final. (code: STAGE_INITIAL_IDB_DELTA)
^level(id; _).
^^^^^^^^^^^^^
1 ERROR
BloxCompiler reported 1 error in block 'block_1Z7PZ61E'
I also tried following the diff predicate example. But this
query '_(id) <- (level\level#prev)(id; _).'
causes a syntax error:
block block_1Z80EFSZ: line 2: error: illegal character 'U+005C' (code: ILLEGAL_CHARACTER)
(level\level#prev)(id; _).
^
block block_1Z80EFSZ: line 2: error: unexpected token 'level' (code: UNEXPECTED_TOKEN)
(level\level#prev)(id; _).
^^^^^
block block_1Z80EFSZ: line 2: error: unexpected token ')' (code: UNEXPECTED_TOKEN)
(level\level#prev)(id; _).
^
block block_1Z80EFSZ: line 2: error: unexpected token ';' (code: UNEXPECTED_TOKEN)
(level\level#prev)(id; _).
^
4 ERRORS
BloxCompiler reported 4 errors in block 'block_1Z80EFSZ'
Unfortunately the diff predicate functionality is not yet released (as of Dec 2016). We've noted that as a documentation bug to fix. The feature is expected in LB 4.4 in Q1 2017.
Here's a simple example for printing or querying the values in the IDB predicate after changes to the EDB. You don't need to use delta logic in this case.
[dan#glycerine Fri Dec 30 09:55:54 ~/Temp]
$ cat foo.logic
foo(x), foo_id(x: y) -> int(y).
edb[x] = y -> foo(x), int(y).
idb[x] = y -> foo(x), int(y).
idb[x] = y <- edb[x] + 10 = y.
[dan#glycerine Fri Dec 30 09:56:33 ~/Temp]
$ lb create --overwrite foo && lb addblock foo --name foo --file foo.logic && lb exec foo '+foo(x), +foo_id[x] = 1, +edb[x] = 5.'
created workspace 'foo'
added block 'foo'
[dan#glycerine Fri Dec 30 09:57:23 ~/Temp]
$ lb print foo edb
[10000000005] 1 5
[dan#glycerine Fri Dec 30 09:57:28 ~/Temp]
$ lb print foo idb
[10000000005] 1 15
[dan#glycerine Fri Dec 30 09:57:32 ~/Temp]
$ lb exec foo '^edb[x] = 15 <- foo_id[x] = 1.'
[dan#glycerine Fri Dec 30 09:57:54 ~/Temp]
$ lb print foo idb
[10000000005] 1 25
[dan#glycerine Fri Dec 30 09:57:55 ~/Temp]
$ lb query foo '_[x] = edb[x].'
/--------------- _ ---------------\
[10000000005] 1 15
\--------------- _ ---------------/
I can think of a couple options for watching changes more as they happen. In once case you could create a watcher predicate and add values to it like this.
watcher(x, y) -> int(x), int(y).
^watcher(x, y) <- foo_id[x] = id, ^idb[id] = y.
That could be enhanced to have its own entity and timestamp...
Another way, if you are using web services to make the EDB updates, is to use
lb web-server monitor -w foo idb
to watch a list of predicates.
I'm running apache/php/memcached on ubuntu 12.04
The keys I am setting are persisting beyond their expiration. We are using the same code that worked with the memcached PAAS we are migrating from.
How do I confirm that the expiration is actually being set on a key?
If I telnet and get [mykey] it simply shows the value, not the expiration.
You can verify the expiration time of an item in memcached by running stats cachedump command. Cachedump require two numeric arguments; item index, and the number of items to display. Memcached will return a list of items with size (in bytes), and a unix timestamp.
Example telnet session
// Add something to cache
>> set apples 0 130 3
>> Foo
<< STORED
// List ITEM stats
>> stats items
<< STAT items:1:number 1
<< STAT items:1:age 7
<< STAT items:1:evicted 0
<< STAT items:1:evicted_nonzero 0
<< STAT items:1:evicted_time 0
<< STAT items:1:outofmemory 0
<< STAT items:1:tailrepairs 0
<< STAT items:1:reclaimed 0
<< STAT items:1:expired_unfetched 0
<< STAT items:1:evicted_unfetched 0
<< END
// Verify expiration time
>> stats cachedump 1 1
<< ITEM apples [3 b; 1393944062 s]
<< END
And 1393944062 => Tue, 04 Mar 2014 08:41:02 -0600