Celery with eventlet or gevent doesn't work properly - celery

I'm running Celery using code, like this:
if __name__ == '__main__':
worker = celery.Worker()
worker.setup_defaults(
loglevel=logging.INFO,
pool='eventlet',
concurrency=500
)
worker.start()
When running Celery like this, I get the following output :
-------------- celery#some.server.com v5.2.7 (dawn-chorus)
--- ***** -----
-- ******* ---- Linux-5.10.0-19-cloud-amd64-x86_64-with-glibc2.31 2022-12-14 15:23:55
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app: __main__:0x7fdda296baf0
- ** ---------- .> transport: redis://localhost:6379/6
- ** ---------- .> results: redis://localhost:6379/6
- *** --- * --- .> concurrency: 500 (eventlet)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> celery exchange=celery(direct) key=celery
[tasks]
. task1
Received task: task1[b248771c-6dd5-469d-bc53-eaf63c4f6b60]
Received task: task1[b248771c-6dd5-469d-bc53-eaf63c4f6b62]
Received task: task1[b248771c-6dd5-469d-bc53-eaf63c4f6b64]
Received task: task1[b248771c-6dd5-469d-bc53-eaf63c4f6b67]
Received task: task1[b248771c-6dd5-469d-bc53-eaf63c4f6b68]
Received task: task1[b248771c-6dd5-469d-bc53-eaf63c4f6b70]
But no task are executed.
If I call CTRL+C, a warm shutdown is done, and NOW, the tasks are executed
> CTRL+C
[INFO/MainProcess] Task task1[b248771c-6dd5-469d-bc53-eaf63c4f6b60] succeeded in 1.0062870910001s: None
[INFO/MainProcess] Task task1[b248771c-6dd5-469d-bc53-eaf63c4f6b62] succeeded in 1.0062870910001s: None
[INFO/MainProcess] Task task1[b248771c-6dd5-469d-bc53-eaf63c4f6b64] succeeded in 1.0062870910001s: None
[INFO/MainProcess] Task task1[b248771c-6dd5-469d-bc53-eaf63c4f6b66] succeeded in 1.0062870910001s: None
[INFO/MainProcess] Task task1[b248771c-6dd5-469d-bc53-eaf63c4f6b67] succeeded in 1.0062870910001s: None
[INFO/MainProcess] Task task1[b248771c-6dd5-469d-bc53-eaf63c4f6b68] succeeded in 1.0062870910001s: None
One thing odd I noticed here, is that the number of tasks loaded is related to the concurrency parameter. If I set it to 2, I'll have three tasks loaded, and on warm shutdown, 2 will be executed and the last one is put back in the queue.
(uh?)
Now, if I change the pool to gevent, it loads identically, but NO tasks are executed when I stop the script, instead, they are all added back to the queue.
Finally - AND THIS IS IMPORTANT - If I set the pool to prefork, it works ... perfectly fine ...
So the issue is not related to my code. Do you have any idea what is going on?
I tried to disable mingle, heartbeat and gossip with no luck.
The versions of eventlet and gevent are the latest ones of today:
eventlet : 0.33.2
gevent : 22.10.2
What is going on? Is Celery compatible with eventlet/gevent or is it just a myth?

Related

/opt/gitlab/embedded/bin/ruby: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory

I'm trying to install gitlab-ce on my raspberry pi 4B, 4GB model. My operating system is Raspberry Pi OS Lite 64bit.
Installer taken from here
There was an error running gitlab-ctl reconfigure:
Multiple failures occurred:
* Mixlib::ShellOut::ShellCommandFailed occurred in Chef Infra Client run: runit_service[gitlab-kas] (gitlab-kas::enable line 121) had an error: Mixlib::ShellOut::ShellCommandFailed: ruby_block[restart_log_service] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/provider_runit_service.rb line 65) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/gitlab-kas/log ----
STDOUT: timeout: run: /opt/gitlab/service/gitlab-kas/log: (pid 21560) 34s, got TERM
STDERR:
---- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/gitlab-kas/log ----
Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/gitlab-kas/log returned 1
* Mixlib::ShellOut::ShellCommandFailed occurred in delayed notification: execute[clear the gitlab-rails cache] (gitlab::gitlab-rails line 477) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '127'
---- Begin output of /opt/gitlab/bin/gitlab-rake cache:clear ----
STDOUT:
STDERR: /opt/gitlab/embedded/bin/ruby: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory
---- End output of /opt/gitlab/bin/gitlab-rake cache:clear ----
Ran /opt/gitlab/bin/gitlab-rake cache:clear returned 127
* Mixlib::ShellOut::ShellCommandFailed occurred in delayed notification: runit_service[gitlab-kas] (gitlab-kas::enable line 121) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/gitlab-kas ----
STDOUT: timeout: run: /opt/gitlab/service/gitlab-kas: (pid 21561) 65s, got TERM
STDERR:
---- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/gitlab-kas ----
Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/gitlab-kas returned 1
*Update: Distro info:
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
The installation scripts for the Raspberry Pi only work on debian buster. Notice the distro/version specified for the package is raspbian/buster:
However, you have installed the newer bullseye version of raspbian:
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
In order to use this install script, you'll need to use the legacy debian buster release for Raspberry PI OS.

How to use "-b" with celery

everyone.
I have a problem with celery's parameter "-b", i found it in the celery document:
-b, --broker
celery command line option
but it seems doesn't take effect when I use like this, example:
celery -A tasks worker -b redis://yuhui:mypassword#192.168.1.100/0 --loglevel=INFO
tasks.py
from celery import Celery
app = Celery('tasks')
#app.task
def add(x, y):
return x + y
The command line logs like blow:
-------------- celery#yuhui v4.4.2 (cliffs)
--- ***** -----
-- ******* ---- Linux-5.3.0-46-generic-x86_64-with-debian-buster-sid 2020-04-19 11:45:00
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app: tasks:0x7f903f18cdd0
- ** ---------- .> transport: redis://yuhui:**#192.168.1.100:6379/0
- ** ---------- .> results: disabled://
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> celery exchange=celery(direct) key=celery
But, it will stuck without any return when I execute celery -A tasks inspect active.
If I change this line
app = Celery('tasks', broker='redis://yuhui:mypassword#192.168.1.100/0')
It will be fine.
BTW, I don't have a redis on my current machine.
So, how to use this parameter?
You need to pass broker param to inspect command
celery -A tasks inspect -b redis://yuhui:mypassword#192.168.1.100/0 active

TAP results do not show up during execution time, if a TestClassSetup is present

I have a problem. We use the Matlab testing framework to analyze our codebase. To track the results in our CI system TeamCity we use the TAP-format. Here we have the following problem:
If a test includes a TestClassSetup section, the TAP results show up only at the end, and not during the exection. This results in a few issues for us:
Timestamps created by the CI system might not be correct
If informative output is given within a test-case, it is not shown together with the assertion statment.
We use the following (simplified) snippet to identify out TestSuite and execute it:
testSuite = matlab.unittest.TestSuite.fromFolder('.');
runner = matlab.unittest.TestRunner.withNoPlugins();
runner.addPlugin(matlab.unittest.plugins.TAPPlugin.producingOriginalFormat());
results = runner.run(testSuite);
With the following two classes the issue is reproducible (the content is of course made up & meaningless...):
classdef SomeTest < matlab.unittest.TestCase
properties (TestParameter)
param = {1, 2};
param2 = {1, 2};
end
methods (TestClassSetup)
function someSetup(testCase)
pause(0.1);
end
end
methods (Test)
function testMethod(self, param, param2)
fprintf('I''m here, with the params: %f/%f\n', param, param2);
pause(0.1);
self.assertGreaterThan(param, param2);
end
end
end
classdef SomeOtherTest < matlab.unittest.TestCase
properties (TestParameter)
param = {1, 2};
param2 = {1, 2};
end
methods (Test)
function testMethod(self, param, param2)
fprintf('I''m here, with the params: %f/%f\n', param, param2);
pause(0.1);
self.assertGreaterThan(param, param2);
end
end
end
If you copy all three files into one folder, and execute the runner, you'll see the output (assertions are simplified):
1..8
I'm here, with the params: 1.000000/1.000000
not ok 1 - SomeOtherTest/testMethod(param=1,param2=1)
# ================================================================================
# Assertion failed in SomeOtherTest/testMethod(param=1,param2=1) and it did not run to completion.
# ================================================================================
#
I'm here, with the params: 1.000000/2.000000
not ok 2 - SomeOtherTest/testMethod(param=1,param2=2)
# ================================================================================
# Assertion failed in SomeOtherTest/testMethod(param=1,param2=2) and it did not run to completion.
# ================================================================================
#
I'm here, with the params: 2.000000/1.000000
ok 3 - SomeOtherTest/testMethod(param=2,param2=1)
I'm here, with the params: 2.000000/2.000000
not ok 4 - SomeOtherTest/testMethod(param=2,param2=2)
# ================================================================================
# Assertion failed in SomeOtherTest/testMethod(param=2,param2=2) and it did not run to completion.
# ================================================================================
#
I'm here, with the params: 1.000000/1.000000
I'm here, with the params: 1.000000/2.000000
I'm here, with the params: 2.000000/1.000000
I'm here, with the params: 2.000000/2.000000
not ok 5 - SomeTest/testMethod(param=1,param2=1)
# ================================================================================
# Assertion failed in SomeTest/testMethod(param=1,param2=1) and it did not run to completion.
# ================================================================================
#
not ok 6 - SomeTest/testMethod(param=1,param2=2)
# ================================================================================
# Assertion failed in SomeTest/testMethod(param=1,param2=2) and it did not run to completion.
# ================================================================================
#
ok 7 - SomeTest/testMethod(param=2,param2=1)
not ok 8 - SomeTest/testMethod(param=2,param2=2)
# ================================================================================
# Assertion failed in SomeTest/testMethod(param=2,param2=2) and it did not run to completion.
# ================================================================================
What I would expect is that also in the second case the Assertion statements (and the ok / not ok TAP flags) are aligned with the fprintf-statements.
Has anyone an idea?
The reason the presence of TestClassSetup "defers" the printing of the TAP output is because the TAP output is a streaming format and if there is any TestClassSetup code the frame work actually does not yet know whether the tests will pass or not. For example, if you have a failure in TestClassTeardown (or through an addTeardown function call in TestClassSetup), the end result is that all the tests that shared the TestClassSetup code will fail.
However, given its a streaming format the TAPPLugin wants to print out the result as soon as it knows the result. There is actually a TestRunnerPlugin method specifically designed for this case, the reportFinalizedResult method.
The fundamental issue here is that I would recommend you avoid printing to the log using disp or fprintf. This is less ideal because the plugins don't have any insight into any of the information printed using fprintf. Also, you can't redirect this information anywhere other than the matlab command line.
However, if you instead using the testCase.log method you will get the diagnostics in the right place and it will be more flexible. You will be able to log it at different levels so you can turn it on or off as you please and control whether you want to see it. It will also not only go to the command line but will go much more nicely into the TAP stream as well as the junit xml and the pdf/html test reports and so on. For your case it looks like the following:
runner = matlab.unittest.TestRunner.withNoPlugins();
runner.addPlugin(matlab.unittest.plugins.TAPPlugin.producingOriginalFormat());
results = runner.run(testSuite);
First you run and you don't see any of the log calls because it was logged at verbosity "3" and the default is lower (level 1 I believe)
1..8
not ok 1 - SomeOtherTest/testMethod(param=value1,param2=value1)
# ================================================================================
# Assertion failed in SomeOtherTest/testMethod(param=value1,param2=value1) and it did not run to completion.
# ================================================================================
not ok 2 - SomeOtherTest/testMethod(param=value1,param2=value2)
# ================================================================================
# Assertion failed in SomeOtherTest/testMethod(param=value1,param2=value2) and it did not run to completion.
# ================================================================================
ok 3 - SomeOtherTest/testMethod(param=value2,param2=value1)
not ok 4 - SomeOtherTest/testMethod(param=value2,param2=value2)
# ================================================================================
# Assertion failed in SomeOtherTest/testMethod(param=value2,param2=value2) and it did not run to completion.
# ================================================================================
not ok 5 - SomeTest/testMethod(param=value1,param2=value1)
# ================================================================================
# Assertion failed in SomeTest/testMethod(param=value1,param2=value1) and it did not run to completion.
# ================================================================================
not ok 6 - SomeTest/testMethod(param=value1,param2=value2)
# ================================================================================
# Assertion failed in SomeTest/testMethod(param=value1,param2=value2) and it did not run to completion.
# ================================================================================
ok 7 - SomeTest/testMethod(param=value2,param2=value1)
not ok 8 - SomeTest/testMethod(param=value2,param2=value2)
# ================================================================================
# Assertion failed in SomeTest/testMethod(param=value2,param2=value2) and it did not run to completion.
# ================================================================================
However, if you configure the tap plugin (or the version 13 tap plugin or the report plugin etc) to log at level threee then you see these diagnostics and they are at the expected location as well:
runner = matlab.unittest.TestRunner.withNoPlugins();
runner.addPlugin(matlab.unittest.plugins.TAPPlugin.producingOriginalFormat('Verbosity', 3));
results = runner.run(testSuite);
You see the output. Also try the TAPVersion 13, the structured output that provides might provide an even better result.
1..8
not ok 1 - SomeOtherTest/testMethod(param=value1,param2=value1)
# ================================================================================
# [Detailed] Diagnostic logged (2018-08-09 16:47:18): I'm here, with the params: 1.000000/1.000000
# ================================================================================
# ================================================================================
# Assertion failed in SomeOtherTest/testMethod(param=value1,param2=value1) and it did not run to completion.
# ================================================================================
not ok 2 - SomeOtherTest/testMethod(param=value1,param2=value2)
# ================================================================================
# [Detailed] Diagnostic logged (2018-08-09 16:47:19): I'm here, with the params: 1.000000/2.000000
# ================================================================================
# ================================================================================
# Assertion failed in SomeOtherTest/testMethod(param=value1,param2=value2) and it did not run to completion.
# ================================================================================
ok 3 - SomeOtherTest/testMethod(param=value2,param2=value1)
# ================================================================================
# [Detailed] Diagnostic logged (2018-08-09 16:47:19): I'm here, with the params: 2.000000/1.000000
# ================================================================================
not ok 4 - SomeOtherTest/testMethod(param=value2,param2=value2)
# ================================================================================
# [Detailed] Diagnostic logged (2018-08-09 16:47:19): I'm here, with the params: 2.000000/2.000000
# ================================================================================
# ================================================================================
# Assertion failed in SomeOtherTest/testMethod(param=value2,param2=value2) and it did not run to completion.
# ================================================================================
not ok 5 - SomeTest/testMethod(param=value1,param2=value1)
# ================================================================================
# [Detailed] Diagnostic logged (2018-08-09 16:47:19): I'm here, with the params: 1.000000/1.000000
# ================================================================================
# ================================================================================
# Assertion failed in SomeTest/testMethod(param=value1,param2=value1) and it did not run to completion.
# ================================================================================
not ok 6 - SomeTest/testMethod(param=value1,param2=value2)
# ================================================================================
# [Detailed] Diagnostic logged (2018-08-09 16:47:19): I'm here, with the params: 1.000000/2.000000
# ================================================================================
# ================================================================================
# Assertion failed in SomeTest/testMethod(param=value1,param2=value2) and it did not run to completion.
# ================================================================================
ok 7 - SomeTest/testMethod(param=value2,param2=value1)
# ================================================================================
# [Detailed] Diagnostic logged (2018-08-09 16:47:20): I'm here, with the params: 2.000000/1.000000
# ================================================================================
not ok 8 - SomeTest/testMethod(param=value2,param2=value2)
# ================================================================================
# [Detailed] Diagnostic logged (2018-08-09 16:47:20): I'm here, with the params: 2.000000/2.000000
# ================================================================================
# ================================================================================
# Assertion failed in SomeTest/testMethod(param=value2,param2=value2) and it did not run to completion.
# ================================================================================
Hope that helps!

using celery in airflow

I am new to airflow, for now I find out airflow is using celery to schedule its tasks. To run airflow, I need to run command 'airflow worker' which will start celery. However, there is always a bug here. Since I have searched in Internet, most problem happen to celery.py which write by user themselves. I use celery just by start airflow. So it is a little bit different.
Anyone could help me? Below is the screenshot of the bug.
airflow#linux-test:~$ airflow worker
[2018-06-22 07:29:04,068] {__init__.py:57} INFO - Using executor CeleryExecutor
[2018-06-22 07:29:04,125] {driver.py:124} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2018-06-22 07:29:04,146] {driver.py:124} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
-------------- celery#linux-test v4.2.0 (windowlicker)
---- **** -----
--- * *** * -- Linux-4.15.0-22-generic-x86_64-with-Ubuntu-18.04-bionic 2018-06-22 07:29:04
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app: airflow.executors.celery_executor:0x7f2267122310
- ** ---------- .> transport: amqp://airflow:**#localhost:5672/airflow
- ** ---------- .> results: postgresql://airflow:**#localhost:5432/airflow
- *** --- * --- .> concurrency: 16 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> default exchange=default(direct) key=default
[2018-06-22 07:29:04,630] {__init__.py:57} INFO - Using executor CeleryExecutor
[2018-06-22 07:29:04,689] {driver.py:124} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2018-06-22 07:29:04,715] {driver.py:124} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
Starting flask
[2018-06-22 07:29:04,858] {_internal.py:88} INFO - * Running on http://0.0.0.0:8793/ (Press CTRL+C to quit)
[2018-06-22 07:29:06,122: ERROR/ForkPoolWorker-1] Pool process <celery.concurrency.asynpool.Worker object at 0x7f22648c8e10> error: TypeError("Required argument 'object' (pos 1) not found",)
Traceback (most recent call last):
File "/home/airflow/.local/lib/python2.7/site-packages/billiard/pool.py", line 289, in __call__
sys.exit(self.workloop(pid=pid))
File "/home/airflow/.local/lib/python2.7/site-packages/billiard/pool.py", line 347, in workloop
req = wait_for_job()
File "/home/airflow/.local/lib/python2.7/site-packages/billiard/pool.py", line 447, in receive
ready, req = _receive(1.0)
File "/home/airflow/.local/lib/python2.7/site-packages/billiard/pool.py", line 419, in _recv
return True, loads(get_payload())
File "/home/airflow/.local/lib/python2.7/site-packages/billiard/common.py", line 107, in pickle_loads
return load(BytesIO(s))
TypeError: Required argument 'object' (pos 1) not found
[2018-06-22 07:29:06,127: ERROR/MainProcess] Process 'ForkPoolWorker-1' pid:18839 exited with 'exitcode 1'
Uninstalling librabbitmq worked for me : pip uninstall librabbitmq. I didn't understand very well why, but apparently, there's some optimization on that library that made the thing fail. Here's the answer I found on some website (I had to translate the page, thus my inability to understand well the solution)
Hope it helps

Chef MongoDB Replication with sc-mongodb

I am new to chef and I'm using sc-mongodb, and I can't get this to work. Is there a better way of doing replication for MongoDB with chef?
I was able to get the default recipe working
include_recipe "sc-mongodb::default"
But when I tried to do replication for mongo, I started getting some weird errors.
include_recipe "sc-mongodb::replicaset"
Error:
================================================================================
Recipe Compile Error in /tmp/kitchen/cache/cookbooks/c_mongo/recipes/default.rb
================================================================================
Net::HTTPServerException
------------------------
400 "Bad Request"
Cookbook Trace:
---------------
/tmp/kitchen/cache/cookbooks/sc-mongodb/definitions/mongodb.rb:236:in `block in from_file'
/tmp/kitchen/cache/cookbooks/sc-mongodb/recipes/replicaset.rb:36:in `from_file'
/tmp/kitchen/cache/cookbooks/c_mongo/recipes/default.rb:54:in `from_file'
Relevant File Content:
----------------------
/tmp/kitchen/cache/cookbooks/sc-mongodb/definitions/mongodb.rb:
229: notifies :run, 'ruby_block[config_sharding]', :immediately if new_resource.is_mongos && new_resource.auto_configure_sharding
230: # we don't care about a running mongodb service in these cases, all we need is stopping it
231: ignore_failure true if new_resource.name == 'mongodb'
232: end
233:
234: # replicaset
235: if new_resource.is_replicaset && new_resource.auto_configure_replicaset
236>> rs_nodes = search(
237: :node,
238: "mongodb_cluster_name:#{new_resource.cluster_name} AND "\
239: 'mongodb_is_replicaset:true AND '\
240: "mongodb_config_mongod_replication_replSetName:#{new_resource.replicaset_name} AND "\
241: "chef_environment:#{node.chef_environment}"
242: )
243:
244: ruby_block 'config_replicaset' do
245: block do
System Info:
------------
chef_version=13.8.5
platform=centos
platform_version=7.4.1708
ruby=ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-linux]
program_name=chef-client worker: ppid=28997;start=00:31:33;
executable=/opt/chef/bin/chef-client
Running handlers:
[2018-03-27T00:31:35+00:00] ERROR: Running exception handlers
Running handlers complete
[2018-03-27T00:31:35+00:00] ERROR: Exception handlers complete
Chef Client failed. 0 resources updated in 01 seconds
[2018-03-27T00:31:35+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
[2018-03-27T00:31:35+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2018-03-27T00:31:35+00:00] ERROR: 400 "Bad Request"
[2018-03-27T00:31:35+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
I have tried so many ways to resolve this problem, looking at the issues from the github repository. From the errors, it looks like the attributes aren't getting set, so people are setting them manually. :
# attempt1------------
#node.default['mongodb']['config']['replSet'] = true
#node.default[:mongodb][:cluster_name] = "repl-name"
#include_recipe "sc-mongodb::replicaset"
# attempt2----------
#node.normal['mongodb']['install_method'] = 'mongodb-org'
#node.normal['mongodb']['config']['bind_ip'] = '0.0.0.0'
#node.normal['mongodb']['dbconfig_file'] = '/etc/mongod.conf'
#node.normal['mongodb']['config']['replSet'] = true
#node.normal['mongodb']['is_replicaset'] = true
#node.normal['mongodb']['cluster_name'] = 'scribe'
#node.normal['mongodb']['replSet'] = 'scribe'
#node.normal['mongodb']['is_shard'] = false
#include_recipe "sc-mongodb::replicaset"
#attempt3------------
#node.default[:mongodb][:cluster_name] = "cluster_name"
#include_recipe "sc-mongodb::replicaset"
#attempt4------------
#if node['mongodb']['config']['replSet'].nil?
# node.default['mongodb']['config']['replSet'] = "repl-name"
#end
#include_recipe "sc-mongodb::replicaset"
#attempt5-------------
#https://github.com/sous-chefs/mongodb/issues/167
#node.default['mongodb']['config']['mongod']['replication']['replSetName'] = "rs-name"
#include_recipe "sc-mongodb::replicaset"
This one gives me a different error:
#attempt6-----------
node.default['mongodb']['config']['mongod']['replication']['replSetName']= 'rs_default'
node.default['mongodb']['cluster_name'] = 'cluster'
node.default['mongodb']['auto_configure']['replicaset'] = true
include_recipe "sc-mongodb::replicaset"
Stacktrace:
================================================================================
Error executing action `run` on resource 'ruby_block[config_replicaset]'
================================================================================
NoMethodError
-------------
undefined method `[]' for nil:NilClass
Cookbook Trace:
---------------
/tmp/kitchen/cache/cookbooks/sc-mongodb/libraries/mongodb.rb:74:in `configure_replicaset'
/tmp/kitchen/cache/cookbooks/sc-mongodb/definitions/mongodb.rb:246:in `block (3 levels) in from_file'
Resource Declaration:
---------------------
# In /tmp/kitchen/cache/cookbooks/sc-mongodb/definitions/mongodb.rb
244: ruby_block 'config_replicaset' do
245: block do
246: MongoDB.configure_replicaset(node, replicaset_name, rs_nodes) unless new_resource.replicaset.nil?
247: end
248: action :nothing
249: end
250:
Compiled Resource:
------------------
# Declared in /tmp/kitchen/cache/cookbooks/sc-mongodb/definitions/mongodb.rb:244:in `block in from_file'
ruby_block("config_replicaset") do
params {:mongodb_type=>"mongod", :action=>[:enable, :start], :logpath=>"/var/log/mongodb/mongod.log", :configservers=>[], :replicaset=>true, :notifies=>[], :not_if=>[], :name=>"mongod"}
action [:nothing]
retries 0
retry_delay 2
default_guard_interpreter :default
block_name "config_replicaset"
declared_type :ruby_block
cookbook_name "sc-mongodb"
recipe_name "replicaset"
block #<Proc:0x00000003ebdec8#/tmp/kitchen/cache/cookbooks/sc-mongodb/definitions/mongodb.rb:245>
end
Platform:
---------
x86_64-linux
I've had a lot of trouble with this cookbook, you're not alone.
From what I've gathered, you need to run this cookbook multiple times and/or in different configurations depending on what you are trying to achieve or what state your node is in. For example, I believe the auto_configure attribute should only be set for the last node in the set after the others have been cheffed with that set to false. Similarly for their user recipe, mongodb only allows admin coll operations on the primary and so you should ensure this recipe is executed on the designated primary node.
Unfortunately the documentation is not clear and for someone like me new to Chef and Ruby, the src and errors are tricky to interpret. I am still in the process of figuring out this cookbook and can report back if I have something concrete to give you, have you been able to get this working since your post? Sorry I can't be of much more help, you will have to try configurations out with test-kitchen VMs.