Why does starting daskdev/dask into a Pod fail? - kubernetes

Question

Why does kubectl run dask --image daskdev/dask fail?
# starting the container with docker to make sure it basically works
➜ ~ docker run --rm -it --entrypoint bash daskdev/dask:latest
(base) root#5b34ce038eb3:/# python
Python 3.8.0 (default, Nov 6 2019, 21:49:08)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dask
>>>
>>> exit()
(base) root#5b34ce038eb3:/# exit
exit
# now trying to fire up the container on a minikube cluster
➜ ~ kubectl run dask --image daskdev/dask
pod/dask created
# let's see what's going on with the Pod
➜ ~ kubectl get pods -w
NAME READY STATUS RESTARTS AGE
dask 0/1 CrashLoopBackOff 1 13s
dask 0/1 Completed 2 24s
dask 0/1 CrashLoopBackOff 2 38s
# not sure why the logs look like something is missing
➜ ~ kubectl logs dask --tail=100
+ '[' '' ']'
+ '[' -e /opt/app/environment.yml ']'
+ echo 'no environment.yml'
+ '[' '' ']'
+ '[' '' ']'
+ exec
no environment.yml

Answer 1

So basically, if you will check result of kubectl describe pod dask, you will see that last state was Terminated with Exit Code 0, that literally means you container was launched successfully, did it job and finished also successfully. What else you expect to happen with pod?
IN addition, when you create pod using kubectl run dask --image daskdev/dask- it creates with the restartPolicy: Always by default!!!!
Always means that the container will be restarted even if it exited with a zero exit code (i.e. successfully).
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Completed
Exit Code: 0
Started: Fri, 02 Apr 2021 15:06:00 +0000
Finished: Fri, 02 Apr 2021 15:06:00 +0000
Ready: False
Restart Count: 3
Environment: <none>
There is no /opt/app/environment.yml in your container. If im not mistake, you should first configure it with prepare.sh. PLease check more here - DASK
section
#docker run --rm -it --entrypoint bash daskdev/dask:latest
(base) root#431d69bb9a80:/# ls -la /opt/app/
total 12
drwxr-xr-x 2 root root 4096 Mar 27 15:43 .
drwxr-xr-x 1 root root 4096 Mar 27 15:43 ..
not sure why the logs look like something is missing ➜ ~ kubectl logs dask --tail=100
...
exec no environment.yml
There is already prepared helm DASK chart. Use it. It works fine:
helm repo add dask https://helm.dask.org/
helm repo update
helm install raffael-dask-release dask/dask
NAME: raffael-dask-release
LAST DEPLOYED: Fri Apr 2 15:43:38 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing DASK, released at name: raffael-dask-release.
This release includes a Dask scheduler, 3 Dask workers, and 1 Jupyter servers.
The Jupyter notebook server and Dask scheduler expose external services to
which you can connect to manage notebooks, or connect directly to the Dask
cluster. You can get these addresses by running the following:
export DASK_SCHEDULER="127.0.0.1"
export DASK_SCHEDULER_UI_IP="127.0.0.1"
export DASK_SCHEDULER_PORT=8080
export DASK_SCHEDULER_UI_PORT=8081
kubectl port-forward --namespace default svc/raffael-dask-release-scheduler $DASK_SCHEDULER_PORT:8786 &
kubectl port-forward --namespace default svc/raffael-dask-release-scheduler $DASK_SCHEDULER_UI_PORT:80 &
export JUPYTER_NOTEBOOK_IP="127.0.0.1"
export JUPYTER_NOTEBOOK_PORT=8082
kubectl port-forward --namespace default svc/raffael-dask-release-jupyter $JUPYTER_NOTEBOOK_PORT:80 &
echo tcp://$DASK_SCHEDULER:$DASK_SCHEDULER_PORT -- Dask Client connection
echo http://$DASK_SCHEDULER_UI_IP:$DASK_SCHEDULER_UI_PORT -- Dask dashboard
echo http://$JUPYTER_NOTEBOOK_IP:$JUPYTER_NOTEBOOK_PORT -- Jupyter notebook
NOTE: It may take a few minutes for the LoadBalancer IP to be available. Until then, the commands above will not work for the LoadBalancer service type.
You can watch the status by running 'kubectl get svc --namespace default -w raffael-dask-release-scheduler'
NOTE: It may take a few minutes for the URLs above to be available if any EXTRA_PIP_PACKAGES or EXTRA_CONDA_PACKAGES were specified,
because they are installed before their respective services start.
NOTE: The default password to login to the notebook server is `dask`. To change this password, refer to the Jupyter password section in values.yaml, or in the README.md.
If you still want create manually pod, use below... Main idea is set restartPolicy: Never.
apiVersion: v1
kind: Pod
metadata:
name: dask-tesssssst
labels:
foo: bar
spec:
restartPolicy: Never
containers:
- image: daskdev/dask:latest
imagePullPolicy: Always
name: dask-tesssssst
Please check DASK KubeCluster official documentation for more examples. Last one I took exactly from there.