Kubernetes delete deployment after script execution - kubernetes

Question

I'm working on creating a distributed locust service for benchmarking and REST API testing in the platform. Architecture is as follows:
First pod running a docker image with master flag for controlling the whole process
A collection of pods running a docker image with worker flag which will make the work (which can vary depending on the requeriments)
Deployment and Service files are:
01-locust-master.yaml
apiVersion: v1
kind: Service
metadata:
name: locust-master
labels:
name: locust
spec:
type: LoadBalancer
selector:
name: locust
role: master
ports:
- port: 8089
protocol: TCP
name: master-web
- port: 5557
protocol: TCP
name: master-port1
- port: 5558
protocol: TCP
name: master-port2
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: locust-master
spec:
replicas: 1
template:
selector:
matchLabels:
name: locust
role: master
template:
metadata:
labels:
name: locust
role: master
spec:
containers:
- name: locust
image: locust-image:latest
imagePullPolicy: Always
env:
- name: LOCUST_MODE
value: master
- name: LOCUST_LOCUSTFILE_PATH
value: "/locust-tasks/locustfiles/the_file.py"
- name: LOCUST_TARGET_HOST
value: "the_endpoint"
- name: LOCUST_USERS
value: !!integerEnv 300
- name: LOCUST_SPAWN_RATE
value: !!integerEnv 100
- name: LOCUST_TEST_TIME
value: "5m"
- name: LOCUST_OUTPUT_DIR
value: "/locust-tasks/locust-output"
- name: LOCUST_TEST_API_TOKEN
value: "some_api_topken"
- name: LOCUST_S3_OUTPUT_BUCKET
value: "s3-bucket"
ports:
- containerPort: 8089
- containerPort: 5557
- containerPort: 5558
resources:
limits:
cpu: 2000m
memory: 2048Mi
02-locust-worker.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: locust-worker
spec:
replicas: 3
selector:
matchLabels:
name: locust
template:
metadata:
labels:
name: locust
role: worker
spec:
containers:
- name: locust
image: locust:latest
imagePullPolicy: Always
env:
- name: LOCUST_MODE
value: worker
- name: LOCUST_MASTER_NODE_HOST
value: locust-master
- name: LOCUST_LOCUSTFILE_PATH
value: "/locust-tasks/locustfiles/the_file.py"
- name: LOCUST_TARGET_HOST
value: "the_endpoint"
- name: LOCUST_TEST_API_TOKEN
value: "the_api_token"
- name: LOCUST_S3_OUTPUT_BUCKET
value: "s3_bucket"
resources:
limits:
cpu: 1500m
memory: 850Mi
requests:
cpu: 1200m
memory: 768Mi
DockerFile
FROM python:3.7.3
# Install packages
COPY requirements.txt /tmp/
RUN pip install --upgrade pip
RUN pip install --requirement /tmp/requirements.txt
RUN pip install awscli
# Add locustfiles
COPY common/ /locust-tasks/common/
COPY templates/ /locust-tasks/templates/
COPY locustfiles/ /locust-tasks/locustfiles/
# Set the entrypoint
COPY docker-entrypoint.sh /
RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
EXPOSE 5557 5558 8089
docker-entrypoint.sh
#!/bin/bash -x
LOCUST_MODE=${LOCUST_MODE:="standalone"}
LOCUST_MASTER=${LOCUST_MASTER:=""}
LOCUST_LOCUSTFILE_PATH=${LOCUST_LOCUSTFILE_PATH:="/locust-tasks/locustfiles/the_file.py"}
LOCUST_TARGET_HOST=${LOCUST_TARGET_HOST:="the_endpoint"}
LOCUST_OUTPUT_DIR=${LOCUST_OUTPUT_DIR:="/locust-tasks/locust-output"}
LOCUST_TEST_API_TOKEN=${LOCUST_TEST_API_TOKEN:="the_token"}
LOCUST_S3_OUTPUT_BUCKET=${LOCUST_S3_OUTPUT_BUCKET:="s3_bucket"}
cd /locust-tasks
if [[ ! -e $LOCUST_OUTPUT_DIR ]]; then
mkdir $LOCUST_OUTPUT_DIR
elif [[ ! -d $LOCUST_OUTPUT_DIR ]]; then
echo "$LOCUST_OUTPUT_DIR already exists but is not a directory" 1>&2
fi
LOCUST_PATH="/usr/local/bin/locust"
LOCUST_FLAGS="-f $LOCUST_LOCUSTFILE_PATH --host=$LOCUST_TARGET_HOST --csv=$LOCUST_OUTPUT_DIR/locust-${LOCUST_MODE}"
if [[ "$LOCUST_MODE" = "master" ]]; then
LOCUST_FLAGS="$LOCUST_FLAGS --master --headless -u $LOCUST_USERS -r $LOCUST_SPAWN_RATE -t $LOCUST_TEST_TIME"
elif [[ "$LOCUST_MODE" = "worker" ]]; then
LOCUST_FLAGS="$LOCUST_FLAGS --worker --master-host=$LOCUST_MASTER_NODE_HOST"
fi
auth_token=$LOCUST_TEST_API_TOKEN $LOCUST_PATH $LOCUST_FLAGS
# Copy test output files to S3
today=$(date +"%Y/%m/%d")
S3_OUTPUT_DIR="s3://${LOCUST_S3_OUTPUT_BUCKET}/${today}/${HOSTNAME}"
echo "Copying locust output files from [$LOCUST_OUTPUT_DIR] to S3 [$S3_OUTPUT_DIR]"
aws s3 cp --recursive $LOCUST_OUTPUT_DIR $S3_OUTPUT_DIR
retVal=$?
if [ $retVal -ne 0 ]; then
echo "Something went wrong, exit code is ${retVal}"
fi
exit $retVal
So my requeriment / idea is to run the script above and after that delete the whole thing. But instead of that, I'm getting and endless pods restarting:
NAME READY STATUS RESTARTS AGE
locust-master-69b4547ddf-7fl4d 1/1 Running 4 23m
locust-worker-59b9689857-l5jhw 1/1 Running 4 23m
locust-worker-59b9689857-l5nd2 1/1 Running 4 23m
locust-worker-59b9689857-lwqbb 1/1 Running 4 23m
How can I delete both deployments after the shellscript ends?

Answer 1

I think you are looking for Jobs.
As pods successfully complete, the Job tracks the successful
completions. When a specified number of successful completions is
reached, the task (ie, Job) is complete. Deleting a Job will clean up
the Pods it created.
You can use ttl mechanism for cleaning up the finished job by specifying the .spec.ttlSecondsAfterFinished field of the Job
https://kubernetes.io/docs/concepts/workloads/controllers/job/#ttl-mechanism-for-finished-jobs