How can I pass data from python_callable to on_success_callback? - callback

Question

I have a DAG with a PythonOperator calling a function. I also have a callback function. I want to pass some information from the callable to the callback. How can I do this?
I tried using the context dictionary:
from datetime import datetime
from pprint import pprint
from airflow import DAG
from airflow.operators.python_operator import PythonOperator
args = {"owner": "airflow",
"start_date": datetime(2019, 11, 18)}
dag = DAG(dag_id="context_demo",
default_args=args)
def test_callable(**context):
print('Starting callable...')
context['z_my_var'] = 'my_val'
pprint(context)
print('Callable finished.')
def test_callback(context):
print('Starting callback...')
pprint(context)
print('Callback finished.')
print_exec_date = PythonOperator(
task_id="print_exec_date",
python_callable=test_callable,
on_success_callback=test_callback,
provide_context=True,
dag=dag,
)
But the value I set in test_callable does not persist into test_callback.

Answer 1

In my case, I use xCom as a work around. Here is a complete example from Official airflow Docs. You can read more about xcome here.
# inside a PythonOperator called 'pushing_task'
def push_function():
return value
# inside another PythonOperator where provide_context=True
def pull_function(**context):
value = context['task_instance'].xcom_pull(task_ids='pushing_task')