In the following Dockerfile I'm trying to copy a jar file from a location on the host into the container, but seems Docker does not like it as I guess I'm missing something. Here is my Dockerfile:
FROM anapsix/alpine-java:jdk8
MAINTAINER joesan
ENV SBT_VERSION 0.13.15
ENV CHECKSUM 18b106d09b2874f2a538c6e1f6b20c565885b2a8051428bd6d630fb92c1c0f96
ENV APP_NAME my-app
ENV PROJECT_HOME /opt/apps
RUN mkdir -p $PROJECT_HOME/$APP_NAME
# Copy the jar file
COPY ./target/scala-*/my-app-*.jar $PROJECT_HOME/$APP_NAME
# Copy the database file
COPY .my-db.mv.db $PROJECT_HOME/$APP_NAME
# Run the application
CMD ["$PROJECT_HOME/$APP_NAME java -Denv=dev -jar my-app-*.jar"]
In my build pipeline, I could see the following error message:
Step 8/10 : COPY ./target/scala-*/my-app-*.jar $PROJECT_HOME/$APP_NAME
COPY failed: no source files were specified
REPOSITORY TAG IMAGE ID CREATED SIZE
<none> <none> 4a240742a379 Less than a second ago 171MB
anapsix/alpine-java jdk8 ed55c27d366d 3 years ago 171MB
Error response from daemon: No such image: [secure]
Pushing image [secure] to repository hub.docker.com
The push refers to repository [docker.io/[secure]/my-app]
An image does not exist locally with the tag: [secure]/my-app
What is that I'm missing and how could I debug this? I mean I could add some echo statements to print out the path, but I'm not sure why I face this error!
This is probably because the target folder is not in "./" folder. which can be because it's ignored by .dockerignore file or the build context is not pointing to the parent folder of the target folder.
In case you are not familiar with build context, it's explained here
Related
I generated a dart project with dart create -t server-shelf . --force.
On the top folder I created a json file (my_data.json) with some mock data.
In my code I am using the data from the json file like:
final _data = json.decode(File('my_data.json').readAsStringSync()) as List<dynamic>;
But if I try to start my server with docker run -it -p 8080:8080 myserver I am getting:
FileSystemException: Cannot open file, path = 'my_data.json' (OS
Error: No such file or directory, errno = 2)
My Dockerfile:
# Use latest stable channel SDK.
FROM dart:stable AS build
# Resolve app dependencies.
WORKDIR /app
COPY pubspec.* ./
RUN dart pub get
# Copy app source code (except anything in .dockerignore) and AOT compile app.
COPY . .
RUN dart compile exe bin/server.dart -o bin/server
# Build minimal serving image from AOT-compiled `/server`
# and the pre-built AOT-runtime in the `/runtime/` directory of the base image.
FROM scratch
COPY --from=build /runtime/ /
COPY --from=build /app/bin/server /app/bin/
COPY my_data.json /app/my_data.json
# Start server.
EXPOSE 8080
CMD ["/app/bin/server"]
I think since you didn't set the WORKDIR for the new image that you started building FROM scratch. You can fix this simply by adding WORKDIR /app again, to the specification of the new image you're building, which is being used to run your application. It will look like this:
...
# Start server.
WORKDIR /app
EXPOSE 8080
CMD ["/app/bin/server"]
Replace
COPY my_data.json /app/my_data.json
with
COPY --from=build app/my_data.json app/
let say i have a new yocto image call stargazer-cmd
what file should i edit so that every time i source poky/oe-init-env
it display as a build option to the user?
kj#kj-Aspire-V3-471G:~/stm32Yoctominimal$ source poky/oe-init-build-env build-mp1/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-sato
meta-toolchain
meta-ide-support
I wish to add stargazer-cmd on top of core-image-minimal, i am not sure what to google and what is the file i need to change.
Let me explain how to add a custom configuration to the OpenEmbedded build process.
First of all, here is the process that is done when running:
source poky/oe-init-build-env
The oe-init-build-env script initializes OEROOT variable to point to the location of the script itself.
The oe-init-build-env script sources another file $OEROOT/scripts/oe-buildenv-internal which will:
Check if OEROOT is set
Set BUILDDIR to your custom build directory name $1, or just build if you do not provide one
Set BBPATH to the poky/bitbake folder
Adds $BBPATH/bin and OEROOT/scripts to PATH (This will enable commands like bitbake and bitbake-layers ...)
Export BUILDDIR and PATH to the next file
The oe-init-build-env script continues by running the final build script with:
TEMPLATECONF="$TEMPLATECONF" $OEROOT/scripts/oe-setup-builddir
The oe-setup-builddir script will:
Check if BUILDDIR is set
Create the conf directory under $BUILDDIR
Sources a template file that will check if there is a TEMPLATECONF variable is set:
. $OEROOT/.templateconf
That file contains:
# Template settings
TEMPLATECONF=${TEMPLATECONF:-meta-poky/conf}
it means that if TEMPLATECONF variable is not set, set it to meta-poky/conf, and that is where the default local.conf and bblayers.conf are coming from.
Copy $TEMPLATECONF to $BUILDDIR/conf/templateconf.cfg
Set some variables pointing to custom local.conf and bblayers.conf:
OECORELAYERCONF="$TEMPLATECONF/bblayers.conf.sample"
OECORELOCALCONF="$TEMPLATECONF/local.conf.sample"
OECORENOTESCONF="$TEMPLATECONF/conf-notes.txt"
In the oe-setup-builddir there is a comment saying that TEMPLATECONF can point to a directory:
#
# $TEMPLATECONF can point to a directory for the template local.conf & bblayers.conf
#
Copy local.conf.sample and bblayers.conf.sample from TEMPLATECONF directory into BUIDDIR/conf:
cp -f $OECORELOCALCONF "$BUILDDIR/conf/local.conf"
sed -e "s|##OEROOT##|$OEROOT|g" \
-e "s|##COREBASE##|$OEROOT|g" \
$OECORELAYERCONF > "$BUILDDIR/conf/bblayers.conf"
Finally it will print what is inside OECORENOTESCONF which points to TEMPLATECONF/conf-notes.txt:
[ ! -r "$OECORENOTESCONF" ] || cat $OECORENOTESCONF
and by default that is located under meta-poky/conf/conf-notes.txt:
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-sato
meta-toolchain
meta-ide-support
You can also run generated qemu images with a command like 'runqemu qemux86'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks
So, now, after understanding all of that, here is what you can do:
Create a custom template directory for your project, containing:
local.conf.sample
bblayers.conf.sample
conf-notes.txt
Do not forget to set the path to poky in bblayers.conf to ##OEROOT## as it will be set automatically by the build script.
Set your custom message in conf-notes.txt
Before any new build, just set TEMPLATECONF:
TEMPLATECONF="<path/to/template-directory>" source poky/oe-init-build-env <build_name>
Then, you will find a build with your custom local.conf and bblayers.conf with additional file conf/templateconf.cfg containing the path of TEMPLATECONF
conf/conf-notes.txt in your layer.
OECORENOTESCONF should point to the file.
For test purposes I create and run an Azurite docker image, in a test pipeline.
I would like to have the blob container automatically created though after Azurite is started, as it would simplify things.
Is there any good way to achieve this?
For the Postgres image we use, we can specify an init.sql which is run on startup. If something similar is available for Azurite, that would be awesome.
You can use the following Dockerfile to install the azure-storage-blob Python package on the Alpine based azurite image. The resulting image size is ~400MB compared to the ~1.2GB azure-cli image.
ARG AZURITE_VERSION="3.17.0"
FROM mcr.microsoft.com/azure-storage/azurite:${AZURITE_VERSION}
# Install azure-storage-blob python package
RUN apk update && \
apk --no-cache add py3-pip && \
apk add --virtual=build gcc libffi-dev musl-dev python3-dev && \
pip3 install --upgrade pip && \
pip3 install azure-storage-blob==12.12.0
# Copy init_azurite.py script
COPY ./init_azurite.py init_azurite.py
# Copy local blobs to azurite
COPY ./init_containers init_containers
# Run the blob emulator and initialize the blob containers
CMD python3 init_azurite.py --directory=init_containers & \
azurite-blob --blobHost 0.0.0.0 --blobPort 10000
The init_azurite.py script is a local Python script that uses the azure-storage-blob package to batch upload files and directories to the azurite blob storage emulator.
import argparse
import os
from time import sleep
from azure.core.exceptions import ResourceExistsError
from azure.storage.blob import BlobServiceClient, ContainerClient
def upload_file(container_client: ContainerClient, source: str, dest: str) -> None:
"""
Upload a single file to a path inside the container.
"""
print(f"Uploading {source} to {dest}")
with open(source, "rb") as data:
try:
container_client.upload_blob(name=dest, data=data)
except ResourceExistsError:
pass
def upload_dir(container_client: ContainerClient, source: str, dest: str) -> None:
"""
Upload a directory to a path inside the container.
"""
prefix = "" if dest == "" else dest + "/"
prefix += os.path.basename(source) + "/"
for root, dirs, files in os.walk(source):
for name in files:
dir_part = os.path.relpath(root, source)
dir_part = "" if dir_part == "." else dir_part + "/"
file_path = os.path.join(root, name)
blob_path = prefix + dir_part + name
upload_file(container_client, file_path, blob_path)
def init_containers(
service_client: BlobServiceClient, containers_directory: str
) -> None:
"""
Iterate on the containers directory and do the following:
1- create the container.
2- upload all folders and files to the container.
"""
for container_name in os.listdir(containers_directory):
container_path = os.path.join(containers_directory, container_name)
if os.path.isdir(container_path):
container_client = service_client.get_container_client(container_name)
try:
container_client.create_container()
except ResourceExistsError:
pass
for blob in os.listdir(container_path):
blob_path = os.path.join(container_path, blob)
if os.path.isdir(blob_path):
upload_dir(container_client, blob_path, "")
else:
upload_file(container_client, blob_path, blob)
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Initialize azurite emulator containers."
)
parser.add_argument(
"--directory",
required=True,
help="""
Directory that contains subdirectories named after the
containers that we should create. Each subdirectory will contain the files
and directories of its container.
"""
)
args = parser.parse_args()
# Connect to the localhost emulator (after 5 secs to make sure it's up).
sleep(5)
blob_service_client = BlobServiceClient(
account_url="http://localhost:10000/devstoreaccount1",
credential={
"account_name": "devstoreaccount1",
"account_key": (
"Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq"
"/K1SZFPTOtr/KBHBeksoGMGw=="
)
}
)
# Only initialize if not already initialized.
if next(blob_service_client.list_containers(), None):
print("Emulator already has containers, will skip initialization.")
else:
init_containers(blob_service_client, args.directory)
This script will be copied to the azurite container and will populate the initial blob containers every time the azurite container is started unless some containers were already persisted using docker volumes. In that case, nothing will happen.
Following is an example docker-compose.yml file:
azurite:
build:
context: ./
dockerfile: Dockerfile
args:
AZURITE_VERSION: 3.17.0
restart: on-failure
ports:
- 10000:10000
volumes:
- azurite-data:/opt/azurite
volumes:
azurite-data:
Using such volumes will persist the emulator data until you destroy them (e.g. by using docker-compose down -v).
Finally, init_containers is a local directory that contains the containers and their folders/files. It will be copied to the azurite container when the image is built.
For example:
init_containers:
container-name-1:
dir-1:
file.txt
img.png
dir-2:
file.txt
container-name-2:
dir-1:
file.txt
img.png
I've solved the issue by creating a custom docker image and executing azure-cli tools from a health check. There could certainly be better solutions, and I will update the accepted answer if someone posts a better solution.
In more details
A solution to create the required data on startup is to run my own script. I chose to trigger the script from a health check I defined in docker-compose. What it does is use azure cli tools to create a container and then verify that it exists.
The script:
AZURE_STORAGE_CONNECTION_STRING="UseDevelopmentStorage=true"
export AZURE_STORAGE_CONNECTION_STRING
az storage container create -n images
az storage container show -n images
exit $?
However, the azurite image is based on alpine, which doesn't have apt, so installing azure cli was a bit tricky. So I did it the other way around, and based my image on mcr.microsoft.com/azure-cli:latest. With that done I installed Azurite like this:
RUN apk add npm
RUN npm install -g azurite --silent
All that's left is to actually run azurite, see the official azurite dockerfile for details.
It is possible to do this without azure-cli and use curl instead (and with that, not having to use the azure-cli docker image). However this was a bit complicated to get the authentication header working properly, so using azure-cli was easier.
I'm trying to run cypress tests inside a docker container. I've simplified my setup so I can just try to get a simple container instance running and a few tests executed.
I'm using docker-compose
version: '2.1'
services:
e2e:
image: test/e2e:v1
command: ["./node_modules/.bin/cypress", "run", "--spec", "integration/mobile/all.js"]
build:
context: .
dockerfile: Dockerfile-cypress
container_name: cypress
network_mode: host
and my Dokerfile-cypress
FROM cypress/browsers:chrome69
RUN mkdir /usr/src/app
WORKDIR /usr/src/app
RUN npm install cypress#3.1.0
COPY cypress /usr/src/app/cypress
COPY cypress.json /usr/src/app/cypress
RUN ./node_modules/.bin/cypress verify
when I run docker-compose up after I build my image I see
cypress | name_to_handle_at on /dev: Operation not permitted
cypress | Can't run because no spec files were found.
cypress |
cypress | We searched for any files matching this glob pattern:
cypress |
cypress | integration/mobile/all-control.js
cypress exited with code 1
I've verified that my cypress files and folders have been copied over and I can verify that my test files exist. I've been stuck on this for a while and i'm unsure what to do besides giving up.
Any guidance appreciated.
Turns out cypress automatically checks for /cypress/integration folder. Moving all my cypress files inside this folder got it working.
The problem: No cypress specs files were found on your automation suite.
Solution: Cypress Test files are located in cypress/integration by default, but can be configured to another directory.
In this folder insert your suites per section:
for example:
- cypress/integration/billing-scenarios-suite
- cypress/integration/user-management-suite
- cypress/integration/proccesses-and-handlers-suite
I assume that these suite directories contains sub directories (which represents some micro logic) ,therefore you need to run it recursively to gather all files:
cypress run --spec \
cypress/integration/<your-suite>/**/* , \
cypress/integration/<your-suite>/**/**/*
If you run in cypress on docker verify that cypress volume contains the tests and mapped and mounted in container on volume section (on Dockerfile / docker-compose.yml file) and run it properly:
docker exec -it <container-id> cypress run --spec \
cypress/integration/<your-suite>/**/* , \
cypress/integration/<your-suite>/**/**/*
I noticed that if you CLICK AND DRAG file method to get file path in VSC, then it generates path with SMALL c-drive letter and this causes error: "Can't run because no spec files were found. + We searched for specs matching this glob pattern:"
e.g. by click and drag I get:
cypress run --spec c:\Users\dmitr\Desktop\cno-dma-replica-for-cy-test\cypress\integration\dma-playground.spec.js
Notice SMALL c, in above.
BUT if I use right click 'get path', I get BIG C, and it works for some reason:
cypress run --spec C:\Users\dmitr\Desktop\cno-dma-replica-for-cy-test\cypress\integration\dma-playground.spec.js
and this causes it to work.
Its strange I know, but there you go.
but if you just use:
I have followed instructions to create an .ipk file, the Packages.gz and host them on a web server as a repo. I have set the opkg.conf in my other VM to point to this repo. The other VM is able to update and list the contents of repositories successfully.
But, when I try to install, I get this message. Can you please describe why I am getting this and what needs to be changed?
Collected errors:
* wfopen: /etc/repo/d1/something.py: No such file or directory
* wfopen: /etc/repo/d1/something-else.py: No such file or directory
While creating the .ipk, I had created a folder named data that had a file structure as /etc/repo/d1/ with the file something.py stored at d1 location. I zipped that folder to data.tar.gz. And, then together with control.tar.gz and 'debian-binary`, I created the .ipk.
I followed instructions from here:
http://bitsum.com/creating_ipk_packages.htm
http://www.jumpnowtek.com/yocto/Managing-a-private-opkg-repository.html
http://www.jumpnowtek.com/yocto/Using-your-build-workstation-as-a-remote-package-repository.html
It is very likely that the directory called /etc/repo/d1/ does not exist on the target system. If you create the folder manually, and try installing again, it probably will not fail. I'm not sure how to force opkg to create the empty directory by itself :/
Update:
You can solve this problem using a preinst script. Just create the missing directories on it, like this:
#!/bin/sh
mkdir -p /etc/repo/d1/
# always return 0 if success
exit 0