Scaleway scw init Inside Docker Container - azure-devops

I am trying to make the Scaleway CLI installed as part of a docker image I'm building to run Azure Pipelines.
My Dockerfile looks like this:
FROM ubuntu:18.04
# To make it easier for build and release pipelines to run apt-get,
# configure apt to not require confirmation (assume the -y argument by default)
ENV DEBIAN_FRONTEND=noninteractive
RUN echo "APT::Get::Assume-Yes \"true\";" > /etc/apt/apt.conf.d/90assumeyes
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates \
curl \
jq \
git \
iputils-ping \
libcurl4 \
libicu60 \
libunwind8 \
netcat\
docker.io \
s3cmd
# Install Scaleway CLI
RUN curl -o /usr/local/bin/scw -L "https://github.com/scaleway/scaleway-cli/releases/download/v2.1.0/scw-2-1-0-linux-x86_64"
RUN chmod +x /usr/local/bin/scw
# Add config for Scaleway CLI
RUN mkdir -p ./config
RUN mkdir -p ./config/scw
COPY ./config/config.yaml $HOME/.config/scw/config.yaml
RUN scw init
# Add private key for SSH connections
COPY ./config/id_rsa $HOME/.ssh/id_rsa
# Config s3cmd
COPY ./config/.s3cfg $HOME/.s3cfg
WORKDIR /azp
COPY ./start.sh .
RUN chmod +x start.sh
CMD ["./start.sh"]
The key section being:
# Install Scaleway CLI
RUN curl -o /usr/local/bin/scw -L "https://github.com/scaleway/scaleway-cli/releases/download/v2.1.0/scw-2-1-0-linux-x86_64"
RUN chmod +x /usr/local/bin/scw
# Add config for Scaleway CLI
RUN mkdir -p ./config
RUN mkdir -p ./config/scw
COPY ./config/config.yaml $HOME/.config/scw/config.yaml
RUN scw init
The config.yaml file referenced above looks like the following (minus the real values of course):
access_key: <key>
secret_key: <secret>
default_organization_id: <orgId>
default_project_id: <projectId>
default_region: nl-ams
default_zone: nl-ams-1
However, when it executes RUN scw init, the output is Invalid email or secret-key: ''
I have tried without running scw init at all, but then calls to scw fail, saying
Access key is required
Details: Access_key can be initialised using the command "scw init".
After initialisation, there are three ways to provide access_key:
with the Scaleway config file, in the access_key key: /root/.config/scw/config.yaml;
with the SCW_ACCESS_KEY environement variable;
Note that the last method has the highest priority.
More info:
https://github.com/scaleway/scaleway-sdk-go/tree/master/scw#scaleway-config
Hint: You can get your credentials here:
https://console.scaleway.com/account/credentials
Which admittedly is one of the better error messages I've seen, but nonetheless has not helped me. I am going to try the Environment Variable approach, which I suspect may do the trick, but I'd still like to know what I'm doing wrong with this config.yaml file.
Lastly... someone with more rep than me needs to create the tag "scaleway". Hard to reference the actual technology in question when the tag doesn't exist.

Related

Deploying Jenkins using skaffold via GitHub Action Runner

I am deploying Jenkins Using GitHub Action Runner using Skaffold.
While the Skaffold is installed over the default image of GitHub Runner
The pod is restarting due to crash loop back off error and causing it to restart.
I am not sure why it is happening.
When I am deploying runner over Google Kubernetes Engine my runner is failing because of following error:
'''A runner exists with the same name
√ Successfully replaced the runner
√ Runner connection is good
# Runner settings
√ Settings Saved.
√ Connected to GitHub
Current runner version: '2.294.0'
2022-12-01 06:03:57Z: Listening for Jobs
Runner update in progress, do not shutdown runner.
Downloading 2.299.1 runner
Waiting for current job finish running.
Generate and execute update script.
Runner will exit shortly for update, should be back online within 10 seconds.
Runner update process finished.
Runner listener exit because of updating, re-launch runner in 5 seconds
Restarting runner...
/home/docker/actions-runner/run-helper.sh: line 20: /home/docker/actions-runner/bin/Runner.Listener: No such file or directory
Exiting with unknown error code: 127
Exiting runner...
'''
Following is the Dockerfile used for runner :
'''FROM ubuntu:22.04
#instalIng skaffold
RUN apt-get update -y && apt-get upgrade -y sudo
RUN apt-get install -y curl
RUN curl -LO https://storage.googleapis.com/skaffold/releases/v2.0.2/skaffold-linux-amd64 \
&& sudo chmod +x skaffold-linux-amd64 \
&& sudo mv skaffold-linux-amd64 /usr/local/bin/skaffold
#install
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update -y && apt-get upgrade -y && useradd -m docker
RUN apt-get install -y curl jq build-essential libssl-dev libffi-dev python3 python3-venv python3-dev ca-certificates gnupg2 iputils-ping software-properties-common apt-transport-https lsb-release git zip unzip postgresql-client python3-pip npm
RUN ln -sf /usr/bin/python3 /usr/bin/python
# set the github runner version
ARG RUNNER_VERSION="2.294.0"
# cd into the user directory, download and unzip the github actions runner
RUN cd /home/docker && mkdir actions-runner && cd actions-runner \
&& curl -O -L https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz \
&& tar xzf ./actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz
# install some additional dependencies
RUN chown -R docker ~docker && /home/docker/actions-runner/bin/installdependencies.sh
#Docker
RUN curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - && \
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable" && \
apt-get update && \
apt-get -y install docker-ce
# Downloading gcloud package
RUN curl https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.tar.gz > /tmp/google-cloud-sdk.tar.gz
# Installing the package
RUN mkdir -p /usr/local/gcloud \
&& tar -C /usr/local/gcloud -xvf /tmp/google-cloud-sdk.tar.gz \
&& /usr/local/gcloud/google-cloud-sdk/install.sh
# Adding the package path to local
ENV PATH $PATH:/usr/local/gcloud/google-cloud-sdk/bin
#Install Kubectl
RUN curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl" \
&& chmod +x ./kubectl \
&& mv ./kubectl /usr/local/bin/kubectl
# copy over the start.sh script
COPY start.sh start.sh
# make the script executable
RUN chmod +x start.sh && mv start.sh /home/docker
# since the config and run script for actions are not allowed to be run by root,
# set the user to "docker" so all subsequent commands are run as the docker user
USER docker
# set the entrypoint to the start.sh script
ENTRYPOINT ["/home/docker/start.sh"] '''
Below is the startup script :
#!/bin/bash
SNAPTIME=`date '+%Y%m%d%H%M%S'`
echo "Started $SNAPTIME"
ORGANIZATION=$ORGANIZATION
ACCESS_TOKEN=`cat /etc/pat/pat`
GH_PROJECT=$GH_PROJECT
RUNNER_NAME="${RUNNER_NAME:-RUN$SNAPTIME}"
RUNNER_LABELS="${RUNNER_LABELS:-simple}"
REG_TOKEN=$(curl -sX POST -H "Authorization: token ${ACCESS_TOKEN}" https://api.github.com/repos/${ORGANIZATION}/$GH_PROJECT/actions/runners/registration-token | jq .token --raw-output)
# gcloud auth activate-service-account --key-file=${GOOGLE_APPLICATION_CREDENTIALS}
cd /home/docker/actions-runner
./config.sh --name $RUNNER_NAME --labels ${RUNNER_LABELS} --url https://github.com/${ORGANIZATION}/${GH_PROJECT} --unattended --replace --token ${REG_TOKEN}
cleanup() {
echo "Removing runner..."
./config.sh remove --unattended --token ${REG_TOKEN}
}
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM
./run.sh & wait $!
The pod is restarting whenever the load is coming into it:
runner-automation-dev-docker-595f48c7dc-k2wbz 1/2 CrashLoopBackOff 7 (67s ago) 18m
I am not sure what exactly is causing this issue.

ECS container exit code 2

i am creating an ECS cluster with docker image library/wordpress:latest and i get the desired task in running state but when i build this image using following Dockerfile and push it to my dockerhub repo and then try to create this cluster using my new image the containers fails by giving Exit code 2
Could you please suggest me what am i doing wrong here?
#Base image
FROM wordpress:latest
LABEL version="latest" maintainer="xxxxxxx <xxxxxx>"
# Update apt
RUN apt-get update
# Add a user for running applications.
RUN useradd apps
RUN mkdir -p /home/apps && chown apps:apps /home/apps
## for apt to be noninteractive
ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
# Install all necessary packages
RUN apt-get -y install build-essential libpoppler-cpp-dev pkg-config x11vnc xvfb fluxbox wget wmctrl gnupg2 unzip zip
# Set the Chrome repo.
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list
# Install Chrome.
RUN apt-get update && apt-get -y install google-chrome-stable
# Install Chrome driver
RUN wget https://chromedriver.storage.googleapis.com/94.0.4606.61/chromedriver_linux64.zip \
&& unzip chromedriver_linux64.zip \
&& mv chromedriver /usr/bin/chromedriver \
&& chown root:root /usr/bin/chromedriver \
&& chmod +x /usr/bin/chromedriver
# create folder to store requirements.txt file
RUN mkdir /home/automation
RUN mkdir /home/automation/FrontEnd
WORKDIR /home/automation
# Copy config and scripts
COPY requirements.txt ./requirements.txt
COPY TestSuites /home/automation/FrontEnd/TestSuites
COPY Resources /home/automation/FrontEnd/Resources
COPY TestRunner.py /home/automation/FrontEnd
COPY TestRail/ /home/automation/TestRail
COPY run-frontend-tests.sh /home/automation/run-tests.sh
COPY FrontEndResultParser.py /home/automation/FrontEnd/FrontEndResultParser.py
# Install python 3.9 and pip3
RUN apt-get -y install python3-dev python3.9 python3-pip
# Install dependencies
RUN pip install "setuptools==58.0.0"
RUN pip install -r requirements.txt
CMD ["sh", "run-tests.sh"]
i am basically just trying to run a script into the container
I used a worpress image and built my own image out of it, thought it would keep the container up and my script will be executed but that didnt happen. My ECS cluster didnt have any running task, all i saw iin the events was service stage-fe-auto has started 1 tasks: task e83587e734c94f77. and when i opened the task details, it had Exit Code 2 and Working directory /home/app but in my Dockerfile my work directory is differen. Not sure what i did wrong

Use local mirror of Alpine when build images of docker-compose instead legacy repositories

Is it possible to use a local repository (mirror) that is used by docker when building a container with docker-compose?
I have a local mirror, and my interest is that when I build a container with docker-compose build, the apk is downloaded from the local mirror, instead of the Internet
Example.
If a local deploy with Dockerfile
FROM alpine:3.13
# Install packages and remove default server definition
RUN apk --no-cache add php8=8.0.2-r0 php8-fpm php8-opcache php8-mysqli php8-json \
php8-openssl php8-curl php8-soap php8-zlib php8-xml php8-phar php8-intl php8-dom php8-xmlreader php8-ctype \
php8-session php8-simplexml php8-mbstring php8-gd nginx supervisor curl php8-exif php8-zip php8-fileinfo \
php8-iconv php8-soap tzdata htop mysql-client php8-pecl-imagick php8-pecl-redis php8-tokenizer php8-xmlwriter \
nano && rm /etc/nginx/conf.d/default.conf
When deploy container with docker-compose build I like that use my local repository of alpine instead any internet mirrors of Alpine such:
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
(1/124) Installing ca-certificates (20191127-r5)
(2/124) Installing brotli-libs (1.0.9-r3)
(3/124) Installing nghttp2-libs (1.42.0-r1)
...
After read alpine docs Alpine Linux package management I get solution using static IP of local mirror. I tried to use the name of the hostname (repoalpine.test) but I couldn't find how to make the hostname public on the docker network.
RUN apk --no-cache -X http://172.20.0.254/v3.13/main -X http://172.20.0.254/v3.13/community \
add php8=8.0.2-r0 php8-fpm php8-opcache php8-mysqli php8-json \
php8-openssl php8-curl php8-soap php8-zlib php8-xml php8-phar php8-intl php8-dom php8-xmlreader php8-ctype \
php8-session php8-simplexml php8-mbstring php8-gd nginx supervisor curl php8-exif php8-zip php8-fileinfo \
php8-iconv php8-soap tzdata htop mysql-client php8-pecl-imagick php8-pecl-redis php8-tokenizer php8-xmlwriter \
nano && rm /etc/nginx/conf.d/default.conf
Now work
Step 7/22 : RUN apk --no-cache -X http://172.20.0.254/v3.13/main -X http://172.20.0.254/v3.13/community add php8=8.0.2-r0 php8-fpm php8-opcache php8-mysqli php8-json php8-openssl php8-curl php8-soap php8-zlib php8-xml php8-phar php8-intl php8-dom php8-xmlreader php8-ctype php8-session php8-simplexml php8-mbstring php8-gd nginx supervisor curl php8-exif php8-zip php8-fileinfo php8-iconv php8-soap tzdata htop mysql-client php8-pecl-imagick php8-pecl-redis php8-tokenizer php8-xmlwriter nano && rm /etc/nginx/conf.d/default.conf
---> Running in 4f2c6521e6e6
fetch http://172.20.0.254/v3.13/community/x86_64/APKINDEX.tar.gz
...
You need a local Docker Registry

VS Code Remote-Containers: cannot create directory ‘/home/appuser’:

I'm trying to use the Remote - Containers extension for Visual Studio Code, but when I "Open Folder in Container", I get this error:
Run: docker exec 0d0c1eac6f38b81566757786f853d6f6a4f3a836c15ca7ed3a3aaf29b9faab14 /bin/sh -c set -o noclobber ; mkdir -p '/home/appuser/.vscode-server/data/Machine' && { > '/home/appuser/.vscode-server/data/Machine/.writeMachineSettingsMarker' ; } 2> /dev/null
mkdir: cannot create directory ‘/home/appuser’: Permission denied
My Dockerfile uses:
FROM python:3.7-slim
...
RUN useradd -ms /bin/bash appuser
USER appuser
I've also tried:
RUN adduser -D appuser
RUN groupadd -g 999 appuser && \
useradd -r -u 999 -g appuser appuser
USER appuser
Both of these work if I build them directly. How do I get this to work?
What works for me is to create a non-root user in my Dockerfile and then configure the VS Code dev container to use that user.
Step 1. Create the non-root user in your Docker image
ARG USER_ID=1000
ARG GROUP_ID=1000
RUN groupadd --system --gid ${GROUP_ID} MY_GROUP && \
useradd --system --uid ${USER_ID} --gid MY_GROUP --home /home/MY_USER --shell /sbin/nologin MY_USER
Step 2. Configure .devcontainer/devcontainer.json file in the root of your project (should be created when you start remote dev)
"remoteUser": "MY_USER" <-- this is the setting you want to update
If you use docker compose, it's possible to configure VS Code to run the entire container as the non-root user by configuring .devcontainer/docker-compose.yml, but I've been happy with the process described above so I haven't experimented further.
You might get some additional insight by reading through the VS Code docs on this topic.
go into your WSL2 and check what is your local uid (non-root) using command id.
in my case it is UID=1000(ubuntu).
Change your dockerfile, to something like this:
# For more information, please refer to https://aka.ms/vscode-docker-python
FROM python:3.8-slim-buster
# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE=1
# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED=1
# Install pip requirements
COPY requirements.txt .
RUN python -m pip install -r requirements.txt
WORKDIR /home/ubuntu
COPY . /home/ubuntu
# Creates a non-root user and adds permission to access the /app folder
# For more info, please refer to https://aka.ms/vscode-docker-python-configure-containers
RUN useradd -u 1000 ubuntu && chown -R ubuntu /home/ubuntu
USER ubuntu
# During debugging, this entry point will be overridden. For more information, please refer to https://aka.ms/vscode-docker-python-debug
CMD ["python", "app.py"]

How to upgrade docker-compose to latest version

I have installed docker-compose using the command
sudo apt install docker-compose
It installed docker-compose version 1.8.0 and build unknown
I need the latest version of docker-compose or at least a version of 1.9.0
Can anyone please let me know what approach I should take to upgrade it or uninstall and re-install the latest version.
I have checked the docker website and can see that they are recommending this to install the latest version'
sudo curl -L https://github.com/docker/compose/releases/download/1.21.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
But before that, I have to uninstall the present version, which can be done using the command
sudo rm /usr/local/bin/docker-compose
but this can be used only when the installation was done using curl. I am not sure if the installation was done by curl as I have used
sudo apt install docker-compose
Please let me know what should I do now to uninstall and re-install the docker-compose.
First, remove the old version:
If installed via apt-get
sudo apt-get remove docker-compose
If installed via curl
sudo rm /usr/local/bin/docker-compose
If installed via pip
pip uninstall docker-compose
Then find the newest version on the release page at GitHub or by curling the API and extracting the version from the response using grep or jq (thanks to dragon788, frbl, and Saber Hayati for these improvements):
# curl + grep
VERSION=$(curl --silent https://api.github.com/repos/docker/compose/releases/latest | grep -Po '"tag_name": "\K.*\d')
# curl + jq
VERSION=$(curl --silent https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)
Finally, download to your favorite $PATH-accessible location and set permissions:
DESTINATION=/usr/local/bin/docker-compose
sudo curl -L https://github.com/docker/compose/releases/download/${VERSION}/docker-compose-$(uname -s)-$(uname -m) -o $DESTINATION
sudo chmod 755 $DESTINATION
The easiest way to have a permanent and sustainable solution for the Docker Compose installation and the way to upgrade it, is to just use the package manager pip with:
pip install docker-compose
I was searching for a good solution for the ugly "how to upgrade to the latest version number"-problem, which appeared after you´ve read the official docs - and just found it occasionally - just have a look at the docker-compose pip package - it should reflect (mostly) the current number of the latest released Docker Compose version.
A package manager is always the best solution if it comes to managing software installations! So you just abstract from handling the versions on your own.
If you tried sudo apt-get remove docker-compose and get E: Unable to locate package docker-compose, try this method :
This command must return a result, in order to check it is installed here :
ls -l /usr/local/bin/docker-compose
Remove the old version :
sudo rm -rf docker-compose
Download the last version (check official repo : docker/compose/releases) :
sudo curl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
(replace 1.24.0 if needed)
Finally, apply executable permissions to the binary:
sudo chmod +x /usr/local/bin/docker-compose
Check version :
docker-compose -v
If the above methods aren't working for you, then refer to this answer: https://stackoverflow.com/a/40554985
curl -L "https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m)" > ./docker-compose
sudo mv ./docker-compose /usr/bin/docker-compose
sudo chmod +x /usr/bin/docker-compose
Based on #eric-johnson's answer, I'm currently using this in a script:
#!/bin/bash
compose_version=$(curl https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)
output='/usr/local/bin/docker-compose'
curl -L https://github.com/docker/compose/releases/download/$compose_version/docker-compose-$(uname -s)-$(uname -m) -o $output
chmod +x $output
echo $(docker-compose --version)
it grabs the latest version from the GitHub api.
Here is another oneliner to install the latest version of docker-compose using curl and sed.
curl -L "https://github.com/docker/compose/releases/download/`curl -fsSLI -o /dev/null -w %{url_effective} https://github.com/docker/compose/releases/latest | sed 's#.*tag/##g' && echo`/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && chmod +x /usr/local/bin/docker-compose
Do it in three steps. (showing for apt-get installs)
Uninstall the last one. e.g. for apt-get installs
sudo apt-get remove docker-compose
Install the new one (https://docs.docker.com/compose/install/)
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
and then
sudo chmod +x /usr/local/bin/docker-compose
Check your version
docker-compose --version
Simple Solution to update docker-compose
This will remove the existing binary of docker-compose and install a new version.
sudo cd /usr/local/bin && sudo rm -rf docker-compose
sudo sudo curl -SL https://github.com/docker/compose/releases/download/v2.2.3/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose
sudo chmod +x docker-compose
for the latest version visit https://github.com/docker/compose/releases and replace the latest one with v2.1.1
I was trying to install docker-compose on "Ubuntu 16.04.5 LTS" but after installing it like this:
sudo curl -L "https://github.com/docker/compose/releases/download/1.26.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
I was getting:
-bash: /usr/local/bin/docker-compose: Permission denied
and while I was using it with sudo I was getting:
sudo: docker-compose: command not found
So here's the steps that I took and solved my problem:
sudo curl -L "https://github.com/docker/compose/releases/download/1.26.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo ln -sf /usr/local/bin/docker-compose /usr/bin/docker-compose
sudo chmod +x /usr/bin/docker-compose
use this from command line: sudo curl -L "https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
Write down the latest release version
Apply executable permissions to the binary:
sudo chmod +x /usr/local/bin/docker-compose
Then test version:
$ docker-compose --version
If you installed with pip, to upgrade you can just use:
pip install --upgrade docker-compose
or as Mariyo states with pip3 explicitly:
pip3 install --upgrade docker-compose
Using latest flag in url will redirect you to the latest release of the repo
As OS name is lower case in github's filename, you should convert uname -s to lower case using sed -e 's/\(.*\)/\L\1/'.
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s|sed -e 's/\(.*\)/\L\1/')-$(uname -m)" -o /usr/local/bin/docker-compose && sudo chmod +x /usr/local/bin/docker-compose
On mac (also working on ubuntu):
sudo curl -L "https://github.com/docker/compose/releases/download/<release-version>/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
NOTE: write the as here:
https://github.com/docker/compose/releases
Use,
$ sudo curl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && sudo chmod +x /usr/local/bin/docker-compose
$ docker-compose -v
Docker Engine and Docker Compose Plugin
Since Microsoft took over Docker they worked on porting docker-compose to their Docker Engine CLI plugins. For future support and updates I would recommend using docker compose plugin (Notice the missing dash) which can be install via the docker-compose-plugin package. The following instructions assume that you are using Ubuntu as Distro or any Distro thats using apt as package manager.
Installation Preparations
Update your mirrors:
sudo apt-get update
Make sure the following packages are installed:
sudo apt-get install \
ca-certificates \
curl \
gnupg \
lsb-release
After that add the official Docker GPG Key:
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
And finally add the the stable repository:
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
Also make sure Docker Engine and other needed dependencies are installed:
sudo apt-get install docker-ce docker-ce-cli containerd.io
Installation of docker compose plugin
sudo apt-get install docker-compose-plugin
Any future updates of the plugin are easily applied via apt.
For further reference take a look at the official installation instructions of Docker Engine and Docker Compose.
After a lot of looking at ways to perform this I ended up using jq, and hopefully I can expand it to handle other repos beyond Docker-Compose without too much work.
# If you have jq installed this will automatically find the latest release binary for your architecture and download it
curl --silent "https://api.github.com/repos/docker/compose/releases/latest" | jq --arg PLATFORM_ARCH "$(echo `uname -s`-`uname -m`)" -r '.assets[] | select(.name | endswith($PLATFORM_ARCH)).browser_download_url' | xargs sudo curl -L -o /usr/local/bin/docker-compose --url
On ubuntu desktop 18.04.2, I have the 'local' removed from the path when using the curl command to install the package and it works for me. See above answer by Kshitij.
In my case, using Windows + WSL2 with Ubuntu 20.04, was necessary only this:
sudo apt update
and then:
sudo apt upgrade
Centos/RHEL
Follow my answer below if you're using Centos7 with an x86-64 architecture. This answer is also available in my github.
Stop Your Docker Containers
I noticed other answers did not talk about stopping your docker containers/images instances before attempting to upgrade gracefully. Assumptions are inevitable but can be costly. Onward we go!
Options to update Docker-Compose
There are 2 options to upgrade docker-compose if you first downloaded and installed docker-compose using the Curl command.
Using Curl, jq package, and Github's direct URL to the docker-compose repository.
Using Curl, Sed, and Github's direct URL to the docker-compose repository.
Note: some of the commands below require "sudo" privileges.
Demonstration
The script below was saved to a file called "update_docker_compose.sh". You need to give this file executable permissions.
Like so:
chmod +x update_docker_compose.sh
"docker_docker_compose.sh" file content:
#!/bin/bash
# author: fullarray (stackoverflow user)
# Contribution shared on: stackoverflow.com
# Contribution also available on: github.com
# date: 06112022
# Stop current docker container running
docker stop containerID
# Remove current docker network running
docker rm containerID
# Remove image of target application(s)
docker image rm imageID
# Delete either dangling (unatagged images) docker containers or images or network
docker system prune -f
# This step depends on the jq package.
# Uncomment jq package installation command below if using Centos7 x86-64.
# sudo yum install jq
# Declare variable to get latest version of docker-compose from github repository
compose_version=$(curl https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)
# Declare variable to target installation directory
target_install_dir='/usr/local/bin/docker-compose'
# Get OS and build (assumes Linux Centos7 and x86_64)
get_local_os_build=$(uname -s)-$(uname -m)
# Execute curl command to carry download and installation operation
curl -L https://github.com/docker/compose/releases/download/$compose_version/docker-compose-$get_local_os_build -o $target_install_dir
# Use chmod to modify permissions to target installation directory (to make it executable)
chmod +x $target_install_dir
# Print docker-compose version to terminal to verify upgrade
$(docker-compose --version)
Edit the script with variables specific to your environment
The script above has a few variables you need to edit with values specific to your docker environment. For instance, you need to replace container ID and image ID with the values that the following commands output.
docker ps
and
docker images output
Once you finalize creating the file (including the edits). Switch to the directory that contains the file. For example, if you created the file in /home/username/script/update_docker_compose.sh
cd /home/username/script
Last, run the script by executing the following
./update_docker_compose.sh
Option 2
Create a script file name "update_docker_compose.sh"
Edit the file and add the following content:
#!/bin/bash
# author: fullarray (stackoverflow user)
# Contribution shared on: stackoverflow.com
# Contribution also available on: github.com
# date: 06112022
# Stop current docker container running
docker stop containerID
# Remove current docker network running
docker rm containerID
# Remove image of target application(s)
docker image rm imageID
# Delete either dangling (unatagged images) docker containers or images or network
docker system prune -f
# Declare variable to target installation directory
target_install_dir='/usr/local/bin/docker-compose'
# Get OS and build (assumes Linux Centos7 and x86_64)
get_local_os_build=$(uname -s)-$(uname -m)
# Execute curl and sed command to carry out download and installation operation
# compose_latest_version=$(curl -L "https://github.com/docker/compose/releases/download/`curl -fsSLI -o /dev/null -w %{url_effective} https://github.com/docker/compose/releases/latest | sed 's#.*tag/##g' && echo`/docker-compose-$get_local_os_build") -o $target_install_dir
# Use chmod to modify permissions to target installation directory (to make it executable)
chmod +x $target_install_dir
# Print docker-compose version to terminal to verify upgrade
$(docker-compose --version)
Edit the script with variables specific to your environment
The script above also has a few variables you need to edit with values specific to your docker environment. For instance, you need to replace container ID and image ID with the values that the following commands output.
docker ps
and
docker images output
Once you finalize creating the file (including the edits). Switch to the directory that contains the file. For example, if you created the file in /home/username/script/update_docker_compose.sh
cd /home/username/script
Last, run the script by executing the following
./update_docker_compose.sh
This is the method of installing docker compose version 2.12.x
Update debian package manager
# apt-get update
# apt-get install docker-compose-plugin
Then install the plugin manualy
DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}
mkdir -p $DOCKER_CONFIG/cli-plugins
curl -SL https://github.com/docker/compose/releases/download/v2.12.2/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
Give permisson of execution of file
chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
Last test the installation
docker compose version
// Docker Composer Version v2.12.2
If you have homebrew you can also install via brew
$ brew install docker-compose
This is a good way to install on a Mac OS system
Most of these solutions are outdated or make you install old version.
To install the latest
sudo apt install jq
DOCKER_COMPOSE_VERSION=$(curl --silent https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)
sudo curl -L "https://github.com/docker/compose/releases/download/$DOCKER_COMPOSE_VERSION/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
Well, my case was pretty weird. I am using wsl2, and Docker Desktop (Windows 11). I stop getting this error after rename the folder "docker" to "config-dev-server" and update de Dockerfile like this this:
COPY ./docker/apache/apache2.conf /etc/apache2/apache2.conf
to
COPY ./config-dev-server/apache/apache2.conf /etc/apache2/apache2.conf
With a newer Docker Desktop for Mac 3.3.0, you don't need to install Docker Compose as a seperate package. Docker Compose comes as a first class citizen installed with Docker by default. Check out the below CLI:
docker compose version
Docker Compose version 2.0.0-beta.1%