Django/Celery: max_retries works for re-try, but printed value is always the Celery default (3) - celery

Question

#app.task(bind=True, autoretry_for=(Exception,), retry_kwargs={'max_retries': 10, 'countdown': 5})
def job_deliver_message(self, message_id):
print('Try {0}/{1}'.format(self.request.retries, self.max_retries))
...
...
...
To test, I'm causing this task to fail on purpose. it DOES re-try 10 times!, however, the printout is like:
[2020-06-03 19:14:06,700: WARNING/ForkPoolWorker-15] Try 0/3
[2020-06-03 19:14:12,080: WARNING/ForkPoolWorker-1] Try 1/3
[2020-06-03 19:14:17,553: WARNING/ForkPoolWorker-3] Try 2/3
[2020-06-03 19:14:23,000: WARNING/ForkPoolWorker-5] Try 3/3
[2020-06-03 19:14:28,489: WARNING/ForkPoolWorker-7] Try 4/3
[2020-06-03 19:14:33,603: WARNING/ForkPoolWorker-9] Try 5/3
[2020-06-03 19:14:39,038: WARNING/ForkPoolWorker-11] Try 6/3
[2020-06-03 19:14:44,525: WARNING/ForkPoolWorker-13] Try 7/3
[2020-06-03 19:14:49,688: WARNING/ForkPoolWorker-15] Try 8/3
[2020-06-03 19:14:54,985: WARNING/ForkPoolWorker-1] Try 9/3
[2020-06-03 19:14:54,985: WARNING/ForkPoolWorker-1] Try 10/3
What am I missing? am I printing the value for the max_retries from the wrong place?
and Yes: I do know i can just show the "10"... as I explicitly set it... but just wanted to know why it's giving me 3 when it's given 10 and actually acting upon it for the number of re-tries...

Answer 1

You are using self.max_retries which is set to 3 by default.
The retry method has another variable max_retries which is set to 10 in this case.
Code reference: task.py

Answer 2

I' ve implemented retry slighthly differnt i.e. i have following celery task in one of my projects:
#app.task(bind=True, max_retries=4, default_retry_delay=3)
def get_external_sentence_translation(self, translation_request):
"""
Passing translation data to translator
:param self
:param translation_request:
:return:
"""
url = settings.TRANSLATOR_BULK_TRANSLATION_RESOURCE
try:
session = requests.Session()
result = session.post(url=url, data=translation_request)
if result.status_code == 500:
self.retry()
result_json = result.json(encoding='utf-8')
# print (u'json: ' + json.dumps(result_json))
if 'status' not in result_json:
message = 'Auto translation in get_external_sentence_translation json status is undefined'
logger.warning(message)
raise RuntimeError(message)
if 'result' not in result_json and result_json['status'] is False:
message = 'Auto translation in get_external_sentence_translation json status is False'
logger.warning(message)
raise RuntimeError(message)
session.close()
if 'result' in result_json:
validation_result = JsonEncoder.from_json(result_json['result'])
return validation_result # result_json[u'result']
return result_json
except Exception as e:
logger.error('Raised exception in get_external_sentence_translation')
logger.error(e)
try:
raise self.retry()
except MaxRetriesExceededError:
return BulkTranslationDto(status='False').to_json()
here i'm manually call retry() and catch MaxRetriesExceededError to stop retrying

Answer 3

am I printing the value for the max_retries from the wrong place?
I think this could be the source of the issue. Can you delve into the Celery source and / or set breakpoints?
You could try setting max_retries in a class-based task and see if you get a different result:
class BaseTaskWithRetry(Task):
autoretry_for = (Exception,)
retry_kwargs = {'max_retries': 5}