Airflow initdb slot_pool does not exists - postgresql

Question

I'm facing an issue with airflow initialization on postgres backend
Ubuntu : 18.04.1
Airflow : v1.10.6
Postgres : 10.10
Python 3.6
And when I run
airflow initdb
I get
[2019-11-22 10:17:23,564] {db.py:368} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.UndefinedTable: relation "airflow.slot_pool" does not exist
LINE 2: FROM airflow.slot_pool
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 37, in <module>
args.func(args)
File "/usr/local/lib/python3.6/dist-packages/airflow/bin/cli.py", line 1131, in initdb
db.initdb(settings.RBAC)
File "/usr/local/lib/python3.6/dist-packages/airflow/utils/db.py", line 106, in initdb
upgradedb()
File "/usr/local/lib/python3.6/dist-packages/airflow/utils/db.py", line 377, in upgradedb
add_default_pool_if_not_exists()
File "/usr/local/lib/python3.6/dist-packages/airflow/utils/db.py", line 74, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/airflow/utils/db.py", line 90, in add_default_pool_if_not_exists
if not Pool.get_pool(Pool.DEFAULT_POOL_NAME, session=session):
File "/usr/local/lib/python3.6/dist-packages/airflow/utils/db.py", line 70, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/airflow/models/pool.py", line 44, in get_pool
return session.query(Pool).filter(Pool.pool == pool_name).first()
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 3265, in first
ret = list(self[0:1])
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 3043, in __getitem__
return list(res)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 3367, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 3392, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 982, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1101, in _execute_clauseelement
distilled_params,
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1250, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "airflow.slot_pool" does not exist
LINE 2: FROM airflow.slot_pool
^
[SQL: SELECT airflow.slot_pool.id AS airflow_slot_pool_id, airflow.slot_pool.pool AS airflow_slot_pool_pool, airflow.slot_pool.slots AS airflow_slot_pool_slots, airflow.slot_pool.description AS airflow_slot_pool_description
FROM airflow.slot_pool
WHERE airflow.slot_pool.pool = %(pool_1)s
LIMIT %(param_1)s]
[parameters: {'pool_1': 'default_pool', 'param_1': 1}]
(Background on this error at: http://sqlalche.me/e/f405)
I've tried deleting / recreating database and user rights (with search_path as said in doc). My postgres is accessible and well configured as tables have been previously created by airflow before the crash ;)
Any ideas?
I've made a try with with Airflow 1.10.2 and it work smoothly with postgres backend.

Answer 1

This is might be because of examples. Try load_examples = False in airflow.cfg
andrun airflow upgradedb or airflow resetdb

Answer 2

Have you tried
airflow resetdb
airflow initdb: Initialize the metadata database
airflow resetdb: Burn down and rebuild the metadata database
airflow upgradedb: Apply missing migrations - this is also idempotent and safe but it tracks migrations so if your tables aren't in the state that alembic thinks that they are in you would need to find that "state" and edit that
Most importantly, if you have any connections or variables set they will be deleted if you run this.

Answer 3

Airflow uses default backend SQLite, and have default tables(like slot_pool) there. So if you consider to use another backend like postgres you should at least add these default tables there.
$ airflow initdb

Answer 4

I know it's kinda late, but I experienced the same trying $ airflow db init against a MySQL 8 backend, and got a similar error.
Turned out that there was a mismatch in my configuration:
The database was named airflow_db, so I had
sql_alchemy_conn = mysql+mysqlconnector://airflow:******#localhost:3306/airflow_db, but a few lines down in the airflow.cfg, I also had the line
sql_alchemy_schema = airflow