The doc https://docs.openshift.com/container-platform/3.9/dev_guide/cron_jobs.html provides details of creating a cron job.
To start a scheduled task that executes a build every 10 mins I use the command:
oc run run-build 161/my-app --image=myimage --restart=OnFailure --schedule='*/10 * * * *'
Which returns:
cronjob.batch/run-build created
But the job fails to start:
The log of pod displays:
Error: unknown command "161/my-app" for "openshift-deploy"
Run 'openshift-deploy --help' for usage.
Have I configured the command ( oc run run-build 161/my-app --image=myimage --restart=OnFailure --schedule='*/10 * * * *' ) to start the cron job incorrectly ?
You are trying to override the image CMD/ARG with the 161/my-app command (which seems not to be valid).
You should use:
oc run run-build --image=myimage --schedule='*/10 * * * *' \
--restart=OnFailure \
--command -- <YOUR COMMAND HERE>
Where run-build is the name of your created cronjob.
If you want to use the default CMD/ARG built in the container image, just omit the --command flag and its value.
First of all, It is not easy to find a full docs for the oc run, so let's discuss with the source code
As the cronjob.batch/run-build has been created, the build is scheduled by kubernetes, so there may be no problem for the schedule part.
The prolem is now why the image run failed.
we can find it from the logs, 161/my-app is recognized as an args for the command openshift-deploy which should be the CMD defined in --image=myimage
Error: unknown command "161/my-app" for "openshift-deploy"
Run 'openshift-deploy --help' for usage.
You have to expain the 161/my-app and update the command based on it.
There is always a CMD defined in a Docker image, so we have to decide whether to use the default CMD:
If the default CMD would be used and want to modify the args: Check this example
oc run nginx --image=nginx -- <arg1> <arg2> ... <argN>
If a new CMD and args would be used: Check this Example
oc run nginx --image=nginx --command -- <cmd> <arg1> ... <argN>
I noticed two more infos in your questions, you can check here and update the question if necessary:
for the openshift-deploy part, you may reference here
for the openshift-build part, you may reference here
Related
I've built a Quarkus 2.7.1 console application using picocli that includes several subcommands. I'd like to be able to run this application within a Kubernetes cluster and decide its arguments at run-time. This is so that I can use the same container image to run the application in different modes within the cluster.
To get things started I added the JIB extension and tried setting the arguments using a configuration value quarkus.jib.jvm-arguments. Unfortunately it seems like this configuration value is locked at build-time so I'm unable to update this at run-time.
Next I tried setting quarkus.args while using default settings for JIB. The configuration value documentation makes it sound general enough for the job but it doesn't seem to have an affect when the application is run in the container. Since most references to this configuration value in documentation are in the context of Dev Mode I'm wondering if this may be disabled outside of that.
How can I get this application running in a container image with its arguments decided at run-time?
You can set quarkus.jib.jvm-entrypoint to any container entrypoint command you want, including scripts. An example in the doc is quarkus.jib.jvm-entrypoint=/deployments/run-java.sh. You could make use of $CLI_ARGUMENTS in such a script. Even something like quarkus.jib.jvm-entrypoint=/bin/sh,-c,'/deployments/run-java.sh $CLI_ARGUMENTS' should work too, as long as you place the script run-java.sh at /deployments in the image. The possibility is limitless.
Also see this SO answer if there's an issue. (The OP in the link put a customer script at src/main/jib/docker/run-java.sh (src/main/jib is Jib's default "extra files directory") so that Jib places the script in the image at /docker/run-java.sh.
I was able to find a solution to the problem with a bit of experimenting this morning.
With the quarkus-container-image-docker extension (instead of quarkus.jib.jvm-arguments) I was able to take the template Dockerfile.jvm and extend it to pass through arguments to the CLI. The only line that needed changing was the ENTRYPOINT (details included in the snippet below). I changed the ENTRYPOINT form (from exec to shell) and added an environment variable as an argument to pass-through program arguments.
FROM registry.access.redhat.com/ubi8/ubi-minimal:8.3
ARG JAVA_PACKAGE=java-11-openjdk-headless
ARG RUN_JAVA_VERSION=1.3.8
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en'
# Install java and the run-java script
# Also set up permissions for user `1001`
RUN microdnf install curl ca-certificates ${JAVA_PACKAGE} \
&& microdnf update \
&& microdnf clean all \
&& mkdir /deployments \
&& chown 1001 /deployments \
&& chmod "g+rwX" /deployments \
&& chown 1001:root /deployments \
&& curl https://repo1.maven.org/maven2/io/fabric8/run-java-sh/${RUN_JAVA_VERSION}/run-java-sh-${RUN_JAVA_VERSION}-sh.sh -o /deployments/run-java.sh \
&& chown 1001 /deployments/run-java.sh \
&& chmod 540 /deployments/run-java.sh \
&& echo "securerandom.source=file:/dev/urandom" >> /etc/alternatives/jre/lib/security/java.security
# Configure the JAVA_OPTIONS, you can add -XshowSettings:vm to also display the heap size.
ENV JAVA_OPTIONS="-Dquarkus.http.host=0.0.0.0 -Djava.util.logging.manager=org.jboss.logmanager.LogManager"
# We make four distinct layers so if there are application changes the library layers can be re-used
COPY --chown=1001 target/quarkus-app/lib/ /deployments/lib/
COPY --chown=1001 target/quarkus-app/*.jar /deployments/
COPY --chown=1001 target/quarkus-app/app/ /deployments/app/
COPY --chown=1001 target/quarkus-app/quarkus/ /deployments/quarkus/
EXPOSE 8080
USER 1001
# [== BEFORE ==]
# ENTRYPOINT [ "/deployments/run-java.sh" ]
# [== AFTER ==]
ENTRYPOINT "/deployments/run-java.sh" $CLI_ARGUMENTS
I have tried the above approaches but they didn't work with the default quarkus JIB's ubi8/openjdk-17-runtime image. This is because this base image doesn't use /work as the WORKIR, but instead the /home/jboss.
Therefore, I created a custom start-up script and referenced it on the properties file as following. This approach works better if there's a need to set application params using environment variables:
File: application.properties
quarkus.jib.jvm-entrypoint=/bin/sh,run-java.sh
File: src/main/jib/home/jboss/run-java.sh
java \
-Djavax.net.ssl.trustStore=/deployments/truststore \
-Djavax.net.ssl.trustStorePassword="$TRUST_STORE_PASSWORD" \
-jar quarkus-run.jar
I am running a Celery Executor and I'm trying to run some python script in the KubernetesPodOperator. Below are examples of what I have tried that didn't work. What am I doing wrong?
Running sctipt
org_node = KubernetesPodOperator(
namespace='default',
image="python",
cmds=["python", "somescript.py" "-c"],
arguments=["print('HELLO')"],
labels={"foo": "bar"},
image_pull_policy="Always",
name=task,
task_id=task,
is_delete_operator_pod=False,
get_logs=True,
dag=dag
)
Running function load_users_into_table()
def load_users_into_table(postgres_hook, schema, path):
gdf = read_csv(path)
gdf.to_sql('users', con=postgres_hook.get_sqlalchemy_engine(), schema=schema)
org_node = KubernetesPodOperator(
namespace='default',
image="python",
cmds=["python", "somescript.py" "-c"],
arguments=[load_users_into_table],
labels={"foo": "bar"},
image_pull_policy="Always",
name=task,
task_id=task,
is_delete_operator_pod=False,
get_logs=True,
dag=dag
)
The script somescript.py must be in Docker image.
Step-1: let's create a image https://docs.docker.com/develop/develop-images/dockerfile_best-practices/.
FROM python:3.8
# copy requirement.txt from local to container
COPY requirements.txt requirements.txt
# install dependencies into container (geopandas, sqlalchemy)
RUN pip install -r requirements.txt
# copy the python script from local to container
COPY somescript.py somescript.py
ENTRYPOINT [ "python", "somescript.py"]
Step-2: Build and push the image into public Docker repository https://hub.docker.com.
NB: kubernetes_pod_operator looks for image from public docker repo
# build image
docker build -t my-python-img:latest .
# test if your image works perfectly
docker run my-python-img:latest
# push image.
docker tag my-python-img username/my-python-img
docker push username/my-python-img
docker pull username/my-python-img
step-3: Lest's create k8s task.
p = KubernetesPodOperator(
namespace='default',
image='username/my-python-img:latest',
labels={'dag-id': dag.dag_id},
name='airflow-my-image-pod',
task_id='load-users',
in_cluster=False, #False: local, True: cluster
cluster_context='microk8s',
config_file='/usr/local/airflow/include/.kube/config',
is_delete_operator_pod=True,
get_logs=True,
dag=dag
)
If you don't understand where configuration file comes from, look here: https://www.astronomer.io/docs/cloud/stable/develop/kubepodoperator-local.
Finally: I want to mention something important when working with databases (credentials). Kubernetes offers the use secret to secure sensitive information. https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/operators.html
KubernetesPodOperator launches a Kubernetes pod that runs a container as specified in the operator's arguments.
First Example
In the first example, the following happens:
KubernetesPodOperator instructs K8s to lunch a pod and prepare to run a container in it using the python image (the image parameter) from hub.docker.com (the default image registry)
ENTRYPOINT of the python image is replaced by ["python", "somescript.py" "-c"] (the cmd parameter)
CMD of the python image is replaced by ["print('HELLO')"] (the arguments parameter)
...
The container is run
So, the complete command that is run in the container is
python somescript.py -c print('HELLO')
Obviously, the official Python image from Docker Hub does not have somescript.py in its working directory. Even if did, it probably would have been not the one that you wrote. That is why the command fails with something like:
python: can't open file 'somescrit.py': [Errno 2] No such file or directory
Second Example
In the second example, pretty much the same happens as in the first example, but the command that is run in the container (again based on the cmd and arguments parameters) is
python somescript.py -c None
(None is the string representation of the load_users_into_table()'s return value)
This command fails, because of the same reasons as in the first example.
How It Could be Done (a Sketch)
You could build a Docker image with somescript.py and all its dependencies. Push the image to an image registry. Specify the image, ENTRYPOINT, and CMD in the corresponding parameters of KubernetesPodOperator.
I am trying to create a TPM-based unlock script using tpm2-tools with instructions from Tevora Secure boot tpm2. I have set up the key, loaded it with cryptsetup luksAddKey secret.bin, then tested it using tpm2_unlock -c 0x81000000 --auth pci:sha1:0,2,3,7 and returns the value of secret.bin. For extra measures, to make sure it works, I loaded secret.bin into "/etc/crypttab", ran # update-initramfs -u -k all, and rebooted. Upon reboot, the system unlocked.
I copied over the following code into "/etc/initramfs-tools/hooks/tpm2"
#!/bin/sh -e
if [ "$1" = "prereqs" ]; then exit 0; fi
. /usr/share/initramfs-tools/hook-functions
copy_exec /usr/local/bin/tpm2_unseal
copy_exec /usr/local/lib/libtss2-tcti-device.so
I appended my etc/crypttab from cryptname UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx none luks to cryptname UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx none luks,keyscript=/usr/local/bin/passphrase-from-tpm
I rewrote the following script because the tpm2-tools command was outdated, edited in the new command, and stored it in /usr/local/bin/passphrase-from-tpm:
#!/bin/sh
set -e
echo "Unlocking via TPM" >&2
export TPM2TOOLS_TCTI="device:/dev/tpm0"
/usr/local/bin/tpm2_unseal -c 0x81000000 --auth pcr:sha1:0,2,3,7
if [ $? -eq 0 ]; then
exit
fi
/lib/cryptsetup/askpass "Unlocking the disk fallback $CRYPTTAB_SOURCE ($CRYPTTAB_NAME)\nEnter passphrase: "
I ran # update-initramfs -u -k all then rebooted. In reboot, I get the following error: /lib/cryptsetup/scripts/passphrase-from-tpm: line 5: /usr/local/bin/tpm2_unseal: not found
I have tried many times to edit passphrase-from-tpm unsuccessfully, including:
Moving both passphrase-from-tpm into "/boot/efi/EFI/BOOT/" and referencing crypttab to that file
Modifying passphrase-from-tpm to use a relative file path to tpm_unseal
Before I figured out how to create a backup linux boot using:
objcopy \
--add-section .osrel=/etc/os-release --change-section-vma .osrel=0x20000 \
--add-section .cmdline=cmdline.txt --change-section-vma .cmdline=0x30000 \
--add-section .linux="/boot/vmlinuz" --change-section-vma .linux=0x40000 \
--add-section .initrd="/boot/initrd.img" --change-section-vma .initrd=0x3000000 \
/usr/lib/systemd/boot/efi/linuxx64.efi.stub /boot/EFI/BOOT/BOOT_RECX64.EFI
I would be locked out of the system completely because of the error and had to reinstall Ubuntu about 40 times. I have suffered a lot and want to quit but I am too stubborn to throw in the flag.
just copy tpm2_unseal to /usr/local/bin/
I'm trying to make a working setup following basically those instructions, plus a few others I have found. While not working 100% yet, check that both /etc/initramfs-tools/hooks/tpm2 and /usr/local/bin/passphrase-from-tpm are marked executable (sudo chmod ug+x $filename).
After making the initramfs, you can run the following to ensure that the TPM related files are actually in the image. Replace the path in the filename by whatever update-initramfs said it was generating:
$ lsinitramfs /boot/initrd.img-5.0.0-37-generic | egrep "(tpm|libtss)"
lib/cryptsetup/scripts/passphrase-from-tpm
lib/modules/5.0.0-37-generic/kernel/crypto/asymmetric_keys/tpm_key_parser.ko
lib/modules/5.0.0-37-generic/kernel/crypto/asymmetric_keys/asym_tpm.ko
lib/udev/rules.d/tpm-udev.rules
usr/local/lib/libtss2-sys.so.0
usr/local/lib/libtss2-mu.so.0
usr/local/lib/libtss2-sys.so.0.0.0
usr/local/lib/libtss2-tcti-device.so
usr/local/lib/libtss2-tcti-device.so.0
usr/local/lib/libtss2-tcti-device.so.0.0.0
usr/local/lib/libtss2-mu.so.0.0.0
usr/local/bin/tpm2_unseal
Additionally, I have modified /usr/local/bin/passphrase-from-tpm to the following:
#!/bin/sh
TPM_DEVICE=/dev/tpm0
TPM_REGISTER=0x81000001
TPM_SEAL_POLICY=sha256:0,2,4,7
export TPM2TOOLS_TCTI="device:$TPM_DEVICE"
if [ "$CRYPTTAB_TRIED" -eq 0 ]; then
echo "Unlocking via TPM" >&2
/usr/local/bin/tpm2_unseal -H $TPM_REGISTER -L $TPM_SEAL_POLICY
UNSEAL_STATUS=$?
echo "Unseal status $UNSEAL_STATUS" >&2
if [ $UNSEAL_STATUS -eq 0 ]; then
exit
fi
else
echo "TPM unlocking previously failed for $CRYPTTAB_SOURCE ($CRYPTTAB_NAME)" >&2
/lib/cryptsetup/askpass "Enter passphrase for $CRYPTTAB_SOURCE ($CRYPTTAB_NAME): "
fi
Note that the command line options to tpm2_unseal are for the 3.x versions of tpm2-tools. If you're using another version, you might need to update the options.
I pulled out various bits into variables at the top of the file. Modify TPM_REGISTER and TPM_SEAL_POLICY to match how you created the TPM object. set -e was removed since if any command failed, the whole script would exit, preventing the askpass fallback from ever running if tpm2_unseal failed.
Additionally, I noticed that if the script fails for some reason, systemd will attempt to run it again. If the secret in the TPM doesn't match the LUKS key, this will render the system unbootable, since the unseal succeeds, but unlocking fails, and systemd will run the script again.
Looking at the man page for crypttab, I discovered that one of the environment variables provided to the keyscript is CRYPTTAB_TRIED which is the number of tries it has attempted to unlock the volume. If CRYPTTAB_TRIED is 0, it'll attempt to use the TPM, as shown by this test (Running as non-root, so accessing the TPM device fails):
$ export CRYPTTAB_SOURCE=some_device
$ export CRYPTTAB_NAME=some_device_name
$ export CRYPTTAB_TRIED=0
$ ./passphrase-from-tpm
Unlocking via TPM
ERROR:tcti:src/tss2-tcti/tcti-device.c:440:Tss2_Tcti_Device_Init() Failed to open device file /dev/tpm0: Permission denied
ERROR: tcti init allocation routine failed for library: "device" options: "/dev/tpm0"
ERROR: Could not load tcti, got: "device"
Unseal status 1
When it tries running the script again, CRYPTTAB_TRIED will be greater than 0, making it display the password prompt instead:
$ export CRYPTTAB_TRIED=1
$ ./passphrase-from-tpm
TPM unlocking previously failed for some_device (some_device_name)
Enter passphrase for some_device (some_device_name):
Hopefully this is still of use to you, and helpful to anyone else trying to get the house of cards that is disk encryption with a TPM on Linux working.
I was new to concourse, and set up the environment in my centos7.6 like below.
$ wget https://concourse-ci.org/docker-compose.yml
$ docker-compose up -d
Then login by `fly --target example login --team-name main --concourse-url http://192.168.77.140:8080/ -u test -p test`
I can see below.
[root#centostest ~]# fly targets
name url team expiry
example http://192.168.77.140:8080 main Sun, 16 Jun 2019 02:23:48 UTC
I used below yaml.xml named with 2.yaml
---
resources:
- name: my-git-repo
type: git
source:
uri: https://github.com/ruanbekker/concourse-test
branch: basic-helloworld
jobs:
- name: hello-world-job
public: true
plan:
- get: my-git-repo
- task: task_print-hello-world
file: my-git-repo/ci/task-hello-world.yml
Then I run below commands step by step.
fly -t example sp -c 2.yaml -p pipeline-01
fly -t example up -p pipeline-01
fly -t example tj -j pipeline-01/hello-world-job --watch
But i just hang on there , no useful response like below.
[root#centostest ~]# fly -t example tj -j pipeline-01/hello-world-job --watch
started pipeline-01/hello-world-job #3
Theoretically， it should print something like below.
Cloning into '/tmp/build/get'...
Fetching HEAD
292c84b change task name
initializing
running echo hello world
hello world
succeeded
Where I did wrong? thanks.
welcome to Concourse!
One thing that can be confusing when starting with Concourse is understanding when Concourse detects that the pipeline has changed and what happens if the pipeline is one file or multiple files.
Your pipeline (as the majority of real-world pipelines) is "nested": main pipeline file 2.yaml refers to a task file named my-git-repo/ci/task-hello-world.yml
What sets Concourse apart from other CI systems is that:
the main pipeline file (2.yaml) can reside everywhere, also in a different repository.
Due to 1, Concourse is unable to detect a change to the main pipeline file, you have to tell Concourse that the file has changed, either with fly set-pipeline or with automatic means such as the concourse-pipeline-resource.
So the following errors happen often:
Changing the main pipeline file, committing and pushing, and expecting Concourse to pick up the change. Missing: you have to do fly set-pipeline
Once doing fly set-pipeline becomes second nature, you can stumble upon the opposite error: Change both the main pipeline file and the nested task file, not pushing, doing set-pipeline. In this case, the only changes picked up by Concourse will be the ones to the main pipeline file, not to the task file. Missing: commit and push.
From the description of your problem, I have the feeling that it is a mixture of the gotchas I mentioned.
I'm trying to setup Kube's cluster with 2 master node 10.0.11.108 and 10.0.11.97 (Ubuntu) with config "ai ai" in cluster/ubuntu/config-default.sh file.
When I run
KUBERNETES_PROVIDER=ubuntu ./kube-up.sh,
it run deployscript in node 10.0.11.97 twice. And become error:
[sudo] password to copy files and start node: cp: cannot create
regular file ‘/opt/bin/etcd’: Text file busy cp: cannot create regular
file ‘/opt/bin/kube-apiserver’: Text file busy cp: cannot create
regular file ‘/opt/bin/kube-controller-manager’: Text file busy cp:
cannot create regular file ‘/opt/bin/kube-scheduler’: Text file busy
start: Job is already running: etcd
I setup Kube's cluster with 1 master, run twice, encounter the same error message,
I modified utuntu/util.sh, and get 'Cluster validation succeeded'.
First add -f for all cp commands, like this: cp ~/kube/default/* /etc/default/ ===> cp -f ~/kube/default/* /etc/default/
Then rerun ./kube-up.sh, you will encounter "start: Job is already running: etcd"
Then modified utuntu/util.sh, modify all 'service XXX start' to 'service XXX restart' , and run kube-up.sh, You'll get the success message.