I am trying to compile Swift code via AWS Lambda.
Therefore I am using an Ubuntu 18.04 Image as base.
The Swift Version is 5.0.1.
When the image is executed locally, it works fine.
When I try to execute it in AWS Lambda, I get the following error:
/usr/bin/ld.gold: fatal error: /tmp/project/src/a.out: Operation not
permitted\nclang-7: error: linker command failed with exit code 1 (use
-v to see invocation)
I think that the problem is caused by the read-only aws lambda container, that only allows to write into the /tmp/ folder.
Do you know how to fix this error? It seems that swift needs permissions for folders, it doesnt have permission for?
Dockerfile
FROM ubuntu:18.04
# install clang
RUN apt-get update
RUN apt-get install -y clang
# install wget
RUN apt-get install -y wget
# install swift dependencies
RUN apt-get install -y libcurl3 libpython2.7 libpython2.7-dev
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get install -y --no-install-recommends \
binutils \
git \
libc6-dev \
libcurl4 \
libedit2 \
libgcc-5-dev \
libpython2.7 \
libsqlite3-0 \
libstdc++-5-dev \
libxml2 \
pkg-config \
tzdata \
zlib1g-dev \
libbsd-dev
RUN apt-get install -y libicu-dev
# install swift 5.0.1
RUN wget https://swift.org/builds/swift-5.0.1-release/ubuntu1804/swift-5.0.1-RELEASE/swift-5.0.1-RELEASE-ubuntu18.04.tar.gz RUN tar xzf swift-5.0.1-RELEASE-ubuntu18.04.tar.gz RUN mv swift-5.0.1-RELEASE-ubuntu18.04 /usr/lib/swift RUN echo "export PATH=/usr/lib/swift/usr/bin:$PATH" >> ~/.bashrc
RUN . ~/.bashrc
RUN chmod -R o+r /usr/lib/swift
This is the command executed in the AWS-Lambda handler function:
swiftc hello_world.swift -o a.out
hello_world.swift
print("Hello World!")
Your output must be set in tmp folder
swiftc hello_world.swift -o /tmp/a.out
A pyenv Python version (e.g. 3.10.4) has the "normal" expected Python executables associated with it (e.g., pip, 2to3, pydoc)
$ ls "${PYENV_ROOT}/versions/3.10.4/bin"
2to3 idle idle3.10 pip3 pydoc pydoc3.10 python-config python3-config python3.10-config
2to3-3.10 idle3 pip pip3.10 pydoc3 python python3 python3.10 python3.10-gdb.py
and a pyenv-virtualenv virtual environment has only the executables that one would a get inside the virtual environment directory structure
$ pyenv virtualenv 3.10.4 venv
$ ls "${PYENV_ROOT}/versions/venv"
bin include lib lib64 pyvenv.cfg
$ ls "${PYENV_ROOT}/versions/venv/bin/"
Activate.ps1 activate activate.csh activate.fish pip pip3 pip3.10 pydoc python python3 python3.10
by default after creation, the venv virtual environment doesn't know about executables of the Python version that it is associated to, like 2to3
$ pyenv activate venv
(venv) $ 2to3 --help
pyenv: 2to3: command not found
The `2to3' command exists in these Python versions:
3.10.4
Note: See 'pyenv help global' for tips on allowing both
python2 and python3 to be found.
so to allow for a virtual environment like venv to have access to these executables you add both it and the Python that created it to pyenv global so that pyenv will "fall back" to the Python version when the executable isn't found
(venv) $ pyenv deactivate
$ pyenv global venv 3.10.4
(venv) $ pyenv global
venv
3.10.4
(venv) $ 2to3 --help | head -n 3
Usage: 2to3 [options] file|dir ...
Options:
This pattern works for one virtual environment, but how do you maintain access to executables like 2to3 (or pipx as seen below`) when you have multiple virtual environments in play?
(venv) $ pyenv virtualenv 3.10.4 example && pyenv activate example
(example) $ 2to3
pyenv: 2to3: command not found
The `2to3' command exists in these Python versions:
3.10.4
Note: See 'pyenv help global' for tips on allowing both
python2 and python3 to be found.
Reproducible example
Using the following Dockerfile
FROM debian:bullseye
SHELL ["/bin/bash", "-c"]
USER root
RUN apt-get update -y && \
apt-get install --no-install-recommends -y \
make \
build-essential \
libssl-dev \
zlib1g-dev \
libbz2-dev \
libreadline-dev \
libsqlite3-dev \
wget \
curl \
llvm \
libncurses5-dev \
xz-utils \
tk-dev \
libxml2-dev \
libxmlsec1-dev \
libffi-dev \
liblzma-dev \
g++ && \
apt-get install -y \
git && \
apt-get -y clean && \
apt-get -y autoremove && \
rm -rf /var/lib/apt/lists/*
# Install pyenv and pyenv-virtualenv
ENV PYENV_RELEASE_VERSION=2.3.0
RUN git clone --depth 1 https://github.com/pyenv/pyenv.git \
--branch "v${PYENV_RELEASE_VERSION}" \
--single-branch \
~/.pyenv && \
pushd ~/.pyenv && \
src/configure && \
make -C src && \
echo 'export PYENV_ROOT="${HOME}/.pyenv"' >> ~/.bashrc && \
echo 'export PATH="${PYENV_ROOT}/bin:${PATH}"' >> ~/.bashrc && \
echo 'eval "$(pyenv init -)"' >> ~/.bashrc && \
. ~/.bashrc && \
git clone --depth 1 https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv && \
echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bashrc
# Install CPython
ENV PYTHON_VERSION=3.10.4
RUN . ~/.bashrc && \
echo "Install Python ${PYTHON_VERSION}" && \
PYTHON_MAKE_OPTS="-j8" pyenv install "${PYTHON_VERSION}"
# Make 'base' virtual envirionment, add it and its Python version to global for
# executables like 2to3 or pipx to be findable
# c.f. https://github.com/pyenv/pyenv-virtualenv/issues/16#issuecomment-37640961
# and then install pipx into the 'base' virtual environment and use pipx to install
# pepotron
RUN . ~/.bashrc && \
pyenv virtualenv "${PYTHON_VERSION}" base && \
echo "" && echo "Python ${PYTHON_VERSION} has additional executables..." && \
ls -lh "${PYENV_ROOT}/versions/${PYTHON_VERSION}/bin" && \
echo "" && echo "...compared to 'base' virtualenv made with Python ${PYTHON_VERSION}" && \
ls -lh "${PYENV_ROOT}/versions/base/bin" && \
echo "" && echo "...because 'base' is actually a symlink" && \
ls -lh "${PYENV_ROOT}/versions/" && \
pyenv global base "${PYTHON_VERSION}" && \
python -m pip --quiet install --upgrade pip setuptools wheel && \
python -m pip --quiet install pipx && \
python -m pipx ensurepath && \
eval "$(register-python-argcomplete pipx)" && \
pipx install pepotron
WORKDIR /home/data
built with
docker build . --file Dockerfile --tag pyenv/multiple-virtualenvs:debug
it can be run with the following to demonstrate the problem
$ docker run --rm -ti pyenv/multiple-virtualenvs:debug
(base) root#26dfa530cd82:/home/data# pyenv global
base
3.10.4
(base) root#26dfa530cd82:/home/data# 2to3 --help | head -n 3
Usage: 2to3 [options] file|dir ...
Options:
(base) root#26dfa530cd82:/home/data# pipx list
venvs are in /root/.local/pipx/venvs
apps are exposed on your $PATH at /root/.local/bin
package pepotron 0.6.0, installed using Python 3.10.4
- bpo
- pep
(base) root#26dfa530cd82:/home/data# pep 3.11
https://peps.python.org/pep-0664/
(base) root#26dfa530cd82:/home/data# pyenv virtualenv 3.10.4 example
(base) root#26dfa530cd82:/home/data# pyenv activate example
pyenv-virtualenv: prompt changing will be removed from future release. configure `export PYENV_VIRTUALENV_DISABLE_PROMPT=1' to simulate the behavior.
(example) root#26dfa530cd82:/home/data# 2to3
pyenv: 2to3: command not found
The `2to3' command exists in these Python versions:
3.10.4
Note: See 'pyenv help global' for tips on allowing both
python2 and python3 to be found.
(example) root#26dfa530cd82:/home/data# pipx
pyenv: pipx: command not found
The `pipx' command exists in these Python versions:
3.10.4/envs/base
base
Note: See 'pyenv help global' for tips on allowing both
python2 and python3 to be found.
So how can one have something like pipx, that is designed to be installed globally, work globally if it is installed in a pyenv-virtualenv virtual environment so that you don't have to have anything installed with pip in the system Python?
It would seem that instead of ever using pyenv activate to activate a virtual environment you would need to deactivate any virtual environments and then only use pyenv global <virtual environment name> <virtual environment Python version> to effectively switch environments. I assume that can't be the only way to use Python version executables inside of a virtual environment, as that seems like that would remove the point of having a separate pyenv activate CLI API for pyenv-virtualenv.
You can directly execute the pipx binary in the pyenv prefix, it should work correctly.
Pyenv's shims mechanism isn't really designed for global binaries like this. I naively expected that the global environment would act as fallback when a local environment doesn't have an installed binary, but I think pyenv only looks at the system Python before falling back to $PATH.
So, if you don't install pipx into system (which, if you're not installing a system pip, I doubt you're doing), then the naive fallback doesn't work.
An alternative is to run pyenv with a temporary environment, i.e.
PYENV_VERSION=my-pipx-env pyenv exec pipx.
I'd want to make this an executable, so I'd suggest adding something like this into a special PATH directory that takes precedence from the pyenv paths:
#!/usr/bin/env bash
set -eu
export PYENV_VERSION="pipx"
exec "${PYENV_ROOT}/libexec/pyenv" exec pipx "$#"
Although, I'd be tempted to forgo the whole activation logic and just directly exec the pipx binary from the environment /bin to avoid running into any shell configuration errors.
I have a working UBUNTU image with this Dockerfile:
FROM perl:5.14
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update -y
RUN apt-get upgrade -y
RUN apt-get install -y libgd-dev
RUN perl -MCPAN -e 'install (Try::Tiny)'
RUN perl -MCPAN -e 'install (Kafka::Connection)'
RUN perl -MCPAN -e 'install (YAML)'
RUN perl -MCPAN -e 'install (GD::Simple)'
RUN perl -MCPAN -e 'install (GD::Graph)'
RUN perl -MCPAN -e 'install (JSON)'
RUN perl -MCPAN -e 'install (JSON::MaybeXS)'
RUN perl -MCPAN -e 'install (HTTP::Request)'
RUN perl -MCPAN -e 'install (HTTP::Response)'
RUN perl -MCPAN -e 'install (HTTP::Daemon)'
COPY run.sh /run.sh
RUN chmod +x "/run.sh"
RUN mkdir -p /code_path
WORKDIR /code_path
CMD ["/run.sh"]
I'n trying to get a slimmer alpine version like this:
FROM alpine:3.10.3
## alpine curl and wget aren't fully compatible, so we install them
## here. gnupg is needed for Module::Signature.
RUN apk update && apk upgrade
RUN apk add curl tar make gcc build-base wget gnupg
RUN mkdir -p /usr/src/perl
WORKDIR /usr/src/perl
## from perl; `true make test_harness` because 3 tests fail
## some flags from http://git.alpinelinux.org/cgit/aports/tree/main/perl/APKBUILD?id=19b23f225d6e4f25330e13144c7bf6c01e624656
RUN curl -SLO https://www.cpan.org/src/5.0/perl-5.30.0.tar.gz \
&& echo 'aa5620fb5a4ca125257ae3f8a7e5d05269388856 *perl-5.30.0.tar.gz' | sha1sum -c - \
&& tar --strip-components=1 -xzf perl-5.30.0.tar.gz -C /usr/src/perl \
&& rm perl-5.30.0.tar.gz \
&& ./Configure -des \
-Duse64bitall \
-Dcccdlflags='-fPIC' \
-Dcccdlflags='-fPIC' \
-Dccdlflags='-rdynamic' \
-Dlocincpth=' ' \
-Duselargefiles \
-Dusethreads \
-Duseshrplib \
-Dd_semctl_semun \
-Dusenm \
&& make libperl.so \
&& make -j$(nproc) \
&& true TEST_JOBS=$(nproc) make test_harness \
&& make install \
&& curl -LO https://raw.githubusercontent.com/miyagawa/cpanminus/master/cpanm \
&& chmod +x cpanm \
&& ./cpanm App::cpanminus \
&& rm -fr ./cpanm /root/.cpanm /usr/src/perl
## from tianon/perl
ENV PERL_CPANM_OPT --verbose --mirror https://cpan.metacpan.org --mirror-only
RUN cpanm Digest::SHA Module::Signature && rm -rf ~/.cpanm
ENV PERL_CPANM_OPT $PERL_CPANM_OPT --verify
RUN perl -MCPAN -e 'install (Try::Tiny)'
RUN perl -MCPAN -e 'install (Kafka::Connection)'
RUN perl -MCPAN -e 'install (YAML)'
RUN perl -MCPAN -e 'install (GD::Simple)'
RUN perl -MCPAN -e 'install (GD::Graph)'
RUN perl -MCPAN -e 'install (JSON)'
RUN perl -MCPAN -e 'install (JSON::MaybeXS)'
RUN perl -MCPAN -e 'install (HTTP::Request)'
RUN perl -MCPAN -e 'install (HTTP::Response)'
RUN perl -MCPAN -e 'install (HTTP::Daemon)'
COPY run.sh /run.sh
RUN chmod +x "/run.sh"
RUN mkdir -p /code_path
WORKDIR /code_path
CMD ["/run.sh"]
I keep getting this error
OS.c:18:10: fatal error: obstack.h: No such file or directory
#include <obstack.h> /* glibc's handy obstacks */
how do I get all the dependencies for Kafka on the image?
Alpine does not ship glibc, and obstack.h is not part of musl-dev.
Try alpine-pkg-glibc.
You could try grabbing obstack.h on Alpine by installing the newly available musl-obstack-dev package of the edge/testing repository:
apk add musl-obstack-dev --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing
Note that edge/testing packages are cutting edge and are considered experimental (as opposed to main stable package).
While Alpine (musl libc) is generally not glibc compatible, it does provide lightweight glibc compatibility in terms of convenience headers and libraries, particularly via the libc6-compat package, and packages as the aforementioned.
Tip: use Alpine's excellent package search for locating missing files. Alpine is vivid and active and sees new packages regularly. Most of the time you'll find that missing files are available in apk packages. Results for obstack.h package contents search:
https://pkgs.alpinelinux.org/contents?file=obstack.h&path=&name=&branch=edge
I eventually searched Github to find a newer alpine perl image and was greatly helped by valino's answer in arriving at this Dockerfile:
FROM alpine:3.10.3
## alpine curl and wget aren't fully compatible, so we install them
## here. gnupg is needed for Module::Signature.
RUN apk update && apk upgrade
RUN apk add --no-cache curl tar make gcc build-base wget gnupg ca-certificates g++ git gd-dev
RUN apk add --no-cache zlib zlib-dev
RUN apk add --no-cache perl perl-dev
RUN curl -L <check the link above "newer alpine perl image" for this line it was rejected> > /bin/cpanm && chmod +x /bin/cpanm
RUN cpanm App::cpm
WORKDIR /usr
RUN cpm install Try::Tiny
RUN cpm install YAML
RUN cpm install JSON
RUN cpm install JSON::MaybeXS
RUN cpm install HTTP::Request
RUN cpm install HTTP::Response
RUN cpm install HTTP::Daemon
RUN cpm install GD::Simple
RUN cpm install GD::Graph
RUN cpm install Data::HexDump::Range
RUN cpm install Proc::Daemon
RUN cpm install Test::Block
RUN cpm install Text::Colorizer
RUN cpm install Gzip::Faster
ENV PERL5LIB=/usr/local/lib/perl5
ENV PATH=/usr/local/bin:$PATH
RUN apk add --no-cache musl-obstack-dev --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing
RUN cpm install Proc::ProcessTable
RUN cpm install Kafka::Connection
COPY run.sh /run.sh
RUN chmod +x "/run.sh"
RUN mkdir -p /code_path
WORKDIR /code_path
CMD ["/run.sh"]
I have installed docker-compose using the command
sudo apt install docker-compose
It installed docker-compose version 1.8.0 and build unknown
I need the latest version of docker-compose or at least a version of 1.9.0
Can anyone please let me know what approach I should take to upgrade it or uninstall and re-install the latest version.
I have checked the docker website and can see that they are recommending this to install the latest version'
sudo curl -L https://github.com/docker/compose/releases/download/1.21.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
But before that, I have to uninstall the present version, which can be done using the command
sudo rm /usr/local/bin/docker-compose
but this can be used only when the installation was done using curl. I am not sure if the installation was done by curl as I have used
sudo apt install docker-compose
Please let me know what should I do now to uninstall and re-install the docker-compose.
First, remove the old version:
If installed via apt-get
sudo apt-get remove docker-compose
If installed via curl
sudo rm /usr/local/bin/docker-compose
If installed via pip
pip uninstall docker-compose
Then find the newest version on the release page at GitHub or by curling the API and extracting the version from the response using grep or jq (thanks to dragon788, frbl, and Saber Hayati for these improvements):
# curl + grep
VERSION=$(curl --silent https://api.github.com/repos/docker/compose/releases/latest | grep -Po '"tag_name": "\K.*\d')
# curl + jq
VERSION=$(curl --silent https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)
Finally, download to your favorite $PATH-accessible location and set permissions:
DESTINATION=/usr/local/bin/docker-compose
sudo curl -L https://github.com/docker/compose/releases/download/${VERSION}/docker-compose-$(uname -s)-$(uname -m) -o $DESTINATION
sudo chmod 755 $DESTINATION
The easiest way to have a permanent and sustainable solution for the Docker Compose installation and the way to upgrade it, is to just use the package manager pip with:
pip install docker-compose
I was searching for a good solution for the ugly "how to upgrade to the latest version number"-problem, which appeared after you´ve read the official docs - and just found it occasionally - just have a look at the docker-compose pip package - it should reflect (mostly) the current number of the latest released Docker Compose version.
A package manager is always the best solution if it comes to managing software installations! So you just abstract from handling the versions on your own.
If you tried sudo apt-get remove docker-compose and get E: Unable to locate package docker-compose, try this method :
This command must return a result, in order to check it is installed here :
ls -l /usr/local/bin/docker-compose
Remove the old version :
sudo rm -rf docker-compose
Download the last version (check official repo : docker/compose/releases) :
sudo curl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
(replace 1.24.0 if needed)
Finally, apply executable permissions to the binary:
sudo chmod +x /usr/local/bin/docker-compose
Check version :
docker-compose -v
If the above methods aren't working for you, then refer to this answer: https://stackoverflow.com/a/40554985
curl -L "https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m)" > ./docker-compose
sudo mv ./docker-compose /usr/bin/docker-compose
sudo chmod +x /usr/bin/docker-compose
Based on #eric-johnson's answer, I'm currently using this in a script:
#!/bin/bash
compose_version=$(curl https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)
output='/usr/local/bin/docker-compose'
curl -L https://github.com/docker/compose/releases/download/$compose_version/docker-compose-$(uname -s)-$(uname -m) -o $output
chmod +x $output
echo $(docker-compose --version)
it grabs the latest version from the GitHub api.
Here is another oneliner to install the latest version of docker-compose using curl and sed.
curl -L "https://github.com/docker/compose/releases/download/`curl -fsSLI -o /dev/null -w %{url_effective} https://github.com/docker/compose/releases/latest | sed 's#.*tag/##g' && echo`/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && chmod +x /usr/local/bin/docker-compose
Do it in three steps. (showing for apt-get installs)
Uninstall the last one. e.g. for apt-get installs
sudo apt-get remove docker-compose
Install the new one (https://docs.docker.com/compose/install/)
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
and then
sudo chmod +x /usr/local/bin/docker-compose
Check your version
docker-compose --version
Simple Solution to update docker-compose
This will remove the existing binary of docker-compose and install a new version.
sudo cd /usr/local/bin && sudo rm -rf docker-compose
sudo sudo curl -SL https://github.com/docker/compose/releases/download/v2.2.3/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose
sudo chmod +x docker-compose
for the latest version visit https://github.com/docker/compose/releases and replace the latest one with v2.1.1
I was trying to install docker-compose on "Ubuntu 16.04.5 LTS" but after installing it like this:
sudo curl -L "https://github.com/docker/compose/releases/download/1.26.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
I was getting:
-bash: /usr/local/bin/docker-compose: Permission denied
and while I was using it with sudo I was getting:
sudo: docker-compose: command not found
So here's the steps that I took and solved my problem:
sudo curl -L "https://github.com/docker/compose/releases/download/1.26.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo ln -sf /usr/local/bin/docker-compose /usr/bin/docker-compose
sudo chmod +x /usr/bin/docker-compose
use this from command line: sudo curl -L "https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
Write down the latest release version
Apply executable permissions to the binary:
sudo chmod +x /usr/local/bin/docker-compose
Then test version:
$ docker-compose --version
If you installed with pip, to upgrade you can just use:
pip install --upgrade docker-compose
or as Mariyo states with pip3 explicitly:
pip3 install --upgrade docker-compose
Using latest flag in url will redirect you to the latest release of the repo
As OS name is lower case in github's filename, you should convert uname -s to lower case using sed -e 's/\(.*\)/\L\1/'.
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s|sed -e 's/\(.*\)/\L\1/')-$(uname -m)" -o /usr/local/bin/docker-compose && sudo chmod +x /usr/local/bin/docker-compose
On mac (also working on ubuntu):
sudo curl -L "https://github.com/docker/compose/releases/download/<release-version>/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
NOTE: write the as here:
https://github.com/docker/compose/releases
Use,
$ sudo curl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && sudo chmod +x /usr/local/bin/docker-compose
$ docker-compose -v
Docker Engine and Docker Compose Plugin
Since Microsoft took over Docker they worked on porting docker-compose to their Docker Engine CLI plugins. For future support and updates I would recommend using docker compose plugin (Notice the missing dash) which can be install via the docker-compose-plugin package. The following instructions assume that you are using Ubuntu as Distro or any Distro thats using apt as package manager.
Installation Preparations
Update your mirrors:
sudo apt-get update
Make sure the following packages are installed:
sudo apt-get install \
ca-certificates \
curl \
gnupg \
lsb-release
After that add the official Docker GPG Key:
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
And finally add the the stable repository:
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
Also make sure Docker Engine and other needed dependencies are installed:
sudo apt-get install docker-ce docker-ce-cli containerd.io
Installation of docker compose plugin
sudo apt-get install docker-compose-plugin
Any future updates of the plugin are easily applied via apt.
For further reference take a look at the official installation instructions of Docker Engine and Docker Compose.
After a lot of looking at ways to perform this I ended up using jq, and hopefully I can expand it to handle other repos beyond Docker-Compose without too much work.
# If you have jq installed this will automatically find the latest release binary for your architecture and download it
curl --silent "https://api.github.com/repos/docker/compose/releases/latest" | jq --arg PLATFORM_ARCH "$(echo `uname -s`-`uname -m`)" -r '.assets[] | select(.name | endswith($PLATFORM_ARCH)).browser_download_url' | xargs sudo curl -L -o /usr/local/bin/docker-compose --url
On ubuntu desktop 18.04.2, I have the 'local' removed from the path when using the curl command to install the package and it works for me. See above answer by Kshitij.
In my case, using Windows + WSL2 with Ubuntu 20.04, was necessary only this:
sudo apt update
and then:
sudo apt upgrade
Centos/RHEL
Follow my answer below if you're using Centos7 with an x86-64 architecture. This answer is also available in my github.
Stop Your Docker Containers
I noticed other answers did not talk about stopping your docker containers/images instances before attempting to upgrade gracefully. Assumptions are inevitable but can be costly. Onward we go!
Options to update Docker-Compose
There are 2 options to upgrade docker-compose if you first downloaded and installed docker-compose using the Curl command.
Using Curl, jq package, and Github's direct URL to the docker-compose repository.
Using Curl, Sed, and Github's direct URL to the docker-compose repository.
Note: some of the commands below require "sudo" privileges.
Demonstration
The script below was saved to a file called "update_docker_compose.sh". You need to give this file executable permissions.
Like so:
chmod +x update_docker_compose.sh
"docker_docker_compose.sh" file content:
#!/bin/bash
# author: fullarray (stackoverflow user)
# Contribution shared on: stackoverflow.com
# Contribution also available on: github.com
# date: 06112022
# Stop current docker container running
docker stop containerID
# Remove current docker network running
docker rm containerID
# Remove image of target application(s)
docker image rm imageID
# Delete either dangling (unatagged images) docker containers or images or network
docker system prune -f
# This step depends on the jq package.
# Uncomment jq package installation command below if using Centos7 x86-64.
# sudo yum install jq
# Declare variable to get latest version of docker-compose from github repository
compose_version=$(curl https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)
# Declare variable to target installation directory
target_install_dir='/usr/local/bin/docker-compose'
# Get OS and build (assumes Linux Centos7 and x86_64)
get_local_os_build=$(uname -s)-$(uname -m)
# Execute curl command to carry download and installation operation
curl -L https://github.com/docker/compose/releases/download/$compose_version/docker-compose-$get_local_os_build -o $target_install_dir
# Use chmod to modify permissions to target installation directory (to make it executable)
chmod +x $target_install_dir
# Print docker-compose version to terminal to verify upgrade
$(docker-compose --version)
Edit the script with variables specific to your environment
The script above has a few variables you need to edit with values specific to your docker environment. For instance, you need to replace container ID and image ID with the values that the following commands output.
docker ps
and
docker images output
Once you finalize creating the file (including the edits). Switch to the directory that contains the file. For example, if you created the file in /home/username/script/update_docker_compose.sh
cd /home/username/script
Last, run the script by executing the following
./update_docker_compose.sh
Option 2
Create a script file name "update_docker_compose.sh"
Edit the file and add the following content:
#!/bin/bash
# author: fullarray (stackoverflow user)
# Contribution shared on: stackoverflow.com
# Contribution also available on: github.com
# date: 06112022
# Stop current docker container running
docker stop containerID
# Remove current docker network running
docker rm containerID
# Remove image of target application(s)
docker image rm imageID
# Delete either dangling (unatagged images) docker containers or images or network
docker system prune -f
# Declare variable to target installation directory
target_install_dir='/usr/local/bin/docker-compose'
# Get OS and build (assumes Linux Centos7 and x86_64)
get_local_os_build=$(uname -s)-$(uname -m)
# Execute curl and sed command to carry out download and installation operation
# compose_latest_version=$(curl -L "https://github.com/docker/compose/releases/download/`curl -fsSLI -o /dev/null -w %{url_effective} https://github.com/docker/compose/releases/latest | sed 's#.*tag/##g' && echo`/docker-compose-$get_local_os_build") -o $target_install_dir
# Use chmod to modify permissions to target installation directory (to make it executable)
chmod +x $target_install_dir
# Print docker-compose version to terminal to verify upgrade
$(docker-compose --version)
Edit the script with variables specific to your environment
The script above also has a few variables you need to edit with values specific to your docker environment. For instance, you need to replace container ID and image ID with the values that the following commands output.
docker ps
and
docker images output
Once you finalize creating the file (including the edits). Switch to the directory that contains the file. For example, if you created the file in /home/username/script/update_docker_compose.sh
cd /home/username/script
Last, run the script by executing the following
./update_docker_compose.sh
This is the method of installing docker compose version 2.12.x
Update debian package manager
# apt-get update
# apt-get install docker-compose-plugin
Then install the plugin manualy
DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}
mkdir -p $DOCKER_CONFIG/cli-plugins
curl -SL https://github.com/docker/compose/releases/download/v2.12.2/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
Give permisson of execution of file
chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
Last test the installation
docker compose version
// Docker Composer Version v2.12.2
If you have homebrew you can also install via brew
$ brew install docker-compose
This is a good way to install on a Mac OS system
Most of these solutions are outdated or make you install old version.
To install the latest
sudo apt install jq
DOCKER_COMPOSE_VERSION=$(curl --silent https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)
sudo curl -L "https://github.com/docker/compose/releases/download/$DOCKER_COMPOSE_VERSION/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
Well, my case was pretty weird. I am using wsl2, and Docker Desktop (Windows 11). I stop getting this error after rename the folder "docker" to "config-dev-server" and update de Dockerfile like this this:
COPY ./docker/apache/apache2.conf /etc/apache2/apache2.conf
to
COPY ./config-dev-server/apache/apache2.conf /etc/apache2/apache2.conf
With a newer Docker Desktop for Mac 3.3.0, you don't need to install Docker Compose as a seperate package. Docker Compose comes as a first class citizen installed with Docker by default. Check out the below CLI:
docker compose version
Docker Compose version 2.0.0-beta.1%
I'm new to Docker and am trying to create a Docker image with Raspbian base and PowerShell Core installed.
EDIT: Updated Dockerfile to include libicu52 package, which resolved the main error: lack of libpsl-native or dependencies not available. Changed CMD parameters and now have a different error.
Here is my Dockerfile:
# Download the latest RPi3 Debian image
FROM resin/raspberrypi3-debian:latest
# Update the image and install prerequisites
RUN apt-get update && apt-get install -y \
wget \
libicu52 \
libunwind8 \
&& apt-get clean
# Grab the latest tar.gz
RUN wget https://github.com/PowerShell/PowerShell/releases/download/v6.0.0-rc.2/powershell-6.0.0-rc.2-linux-arm32.tar.gz
# Make folder to put PowerShell
RUN mkdir ~/powershell
# Unpack the tar.gz file
RUN tar -xvf ./powershell-6.0.0-rc.2-linux-arm32.tar.gz -C ~/powershell
# Run PowerShell
CMD pwsh -v
New error:
hostname: you must be root to change the host name
/bin/sh: 1: pwsh: not found
How do I resolve these errors?
Thanks in advance!
Instead of downloading from source and extracting it in your container, I'd recommend using the official apt installer packages for your Dockerfile from Microsoft's official Debian repository as described at:
https://learn.microsoft.com/en-us/powershell/scripting/setup/installing-powershell-core-on-macos-and-linux?view=powershell-6#debian-9
So transforming that to Dockerfile format:
# Install powershell related system components
RUN apt-get install -y \
gnupg curl apt-transport-https \
&& apt-get clean
# Import the public repository GPG keys
RUN curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
# Register the Microsoft's Debian repository
RUN sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch main" > /etc/apt/sources.list.d/microsoft.list'
# Install PowerShell
RUN apt-get update \
&& apt-get install -y \
powershell
# Start PowerShell
CMD pwsh
Alternatively you can also try to start from one of the original Microsoft docker Linux images, but of course then you need to solve then the raspberry installation for yourself:
https://hub.docker.com/r/microsoft/powershell/tags/