No module named 'airflow' when initializing Apache airflow docker - docker-compose

Question

I am trying to run apache airflow as a docker on a Centos 7 machine.
I followed all the instructions here:https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html
when i am trying to initialize the docker by running docker-compose up airflow-init
i am getting this error
[root#centos7 centos]# docker-compose up airflow-init
Creating network "centos_default" with the default driver
Creating volume "centos_postgres-db-volume" with default driver
Creating centos_redis_1 ... done
Creating centos_postgres_1 ... done
Creating centos_airflow-init_1 ... done
Attaching to centos_airflow-init_1
airflow-init_1 | BACKEND=postgresql+psycopg2
airflow-init_1 | DB_HOST=postgres
airflow-init_1 | DB_PORT=5432
airflow-init_1 |
airflow-init_1 | Traceback (most recent call last):
airflow-init_1 | File "/home/airflow/.local/bin/airflow", line 5, in <module>
airflow-init_1 | from airflow.__main__ import main
airflow-init_1 | ModuleNotFoundError: No module named 'airflow'
airflow-init_1 | Traceback (most recent call last):
airflow-init_1 | File "/home/airflow/.local/bin/airflow", line 5, in <module>
airflow-init_1 | from airflow.__main__ import main
airflow-init_1 | ModuleNotFoundError: No module named 'airflow'
airflow-init_1 | Traceback (most recent call last):
airflow-init_1 | File "/home/airflow/.local/bin/airflow", line 5, in <module>
airflow-init_1 | from airflow.__main__ import main
airflow-init_1 | ModuleNotFoundError: No module named 'airflow'
airflow-init_1 | Traceback (most recent call last):
airflow-init_1 | File "/home/airflow/.local/bin/airflow", line 5, in <module>
airflow-init_1 | from airflow.__main__ import main
airflow-init_1 | ModuleNotFoundError: No module named 'airflow'
airflow-init_1 | Traceback (most recent call last):
airflow-init_1 | File "/home/airflow/.local/bin/airflow", line 5, in <module>
airflow-init_1 | from airflow.__main__ import main
airflow-init_1 | ModuleNotFoundError: No module named 'airflow'
centos_airflow-init_1 exited with code 1
i used the standard YAML file from here:https://airflow.apache.org/docs/apache-airflow/2.0.1/docker-compose.yaml
i found that it's a known issue here:https://github.com/apache/airflow/issues/14520
but i could not understand how to solve this problem.
any advice?

Answer 1

I solved this problem this way.
login with a non-root user.
find your user id :
echo $UID
create .env file and put these lines inside it . replace 4003 with your user id:
AIRFLOW_UID=4003
AIRFLOW_GID=0
If you have not created these directories, first create these and run docker-compose:
sudo mkdir ./dags ./logs ./plugins
sudo chmod 777 -R logs
sudo docker-compose up airflow-init
sudo docker-compose up

Answer 2

I found the problem.
There is a bug on version 2.0.1 that doesn’t let you run the airflow containers using root.
You have to run the installation under another user name (with sudo).

Answer 3

This can happen if AIRFLOW_GID is not set properly in the .env file.
The instructions include running the command echo -e "AIRFLOW_UID=$(id -u)\nAIRFLOW_GID=0" > .env.
To check this worked as expected, look at the contents of the .env file by running cat .env.
You should see something that looks like this:
AIRFLOW_UID=1000
AIRFLOW_GID=0
If you do not you might need to manually edit the .env file to set the airflow uid and gid.