How can I change the installation path of an autotools-based Bitbake recipe? - yocto

I have an autotools-based BitBake recipe which I would like to have binaries installed in /usr/local/bin and libraries installed in /usr/local/lib (instead of /usr/bin and /usr/lib, which are the default target directories).
Here's a part of the autotools.bbclass file which I found important.
CONFIGUREOPTS = " --build=${BUILD_SYS} \
--host=${HOST_SYS} \
--target=${TARGET_SYS} \
--prefix=${prefix} \
--exec_prefix=${exec_prefix} \
--bindir=${bindir} \
--sbindir=${sbindir} \
--libexecdir=${libexecdir} \
--datadir=${datadir} \
--sysconfdir=${sysconfdir} \
--sharedstatedir=${sharedstatedir} \
--localstatedir=${localstatedir} \
--libdir=${libdir} \
...
I thought that the easiest way to accomplish what I wanted to do would be to simply change ${bindir} and ${libdir}, or perhaps change ${prefix} to /usr/local, but I haven't had any success in this area. Is there a way to change these installation variables, or am I thinking about this in the wrong way?
Update:
Strategy 1
As per Ross Burton's suggestion, I've tried adding the following to my recipe:
prefix="/usr/local"
exec_prefix="/usr/local"
but this causes the build to fail during that recipe's do_configure() task, and returns the following:
| checking for GLIB... no
| configure: error: Package requirements (glib-2.0 >= 2.12.3) were not met:
|
| No package 'glib-2.0' found
This package can be found during a normal build without these modified variables. I thought that adding the following line might allow the system to find the package metadata for glib:
PKG_CONFIG_PATH = " ${STAGING_DIR_HOST}/usr/lib/pkgconfig "
but this seems to have made no difference.
Strategy 2
I've also tried Ross Burton's other suggestion to add these variable assignments into my distribution's configuration file, but this causes it to fail during meta/recipes-extended/tzdata's do_install() task. It returns that DEFAULT_TIMEZONE is set to an invalid value. Here's the source of the error from tzdata_2015g.bb
# Install default timezone
if [ -e ${D}${datadir}/zoneinfo/${DEFAULT_TIMEZONE} ]; then
install -d ${D}${sysconfdir}
echo ${DEFAULT_TIMEZONE} > ${D}${sysconfdir}/timezone
ln -s ${datadir}/zoneinfo/${DEFAULT_TIMEZONE} ${D}${sysconfdir}/localtime
else
bberror "DEFAULT_TIMEZONE is set to an invalid value."
exit 1
fi
I'm assuming that I've got a problem with ${datadir}, which references ${prefix}.

Do you want to change paths for everything or just one recipe? Not sure why you'd want to change just one recipe to /usr/local, but whatever.
If you want to change all of them, then the simple way is to set prefix in your local.conf or distro configuration (prefix = "/usr/local").
If you want to do it in a particular recipe, then just assigning prefix="/usr/local" and exec_prefix="/usr/local" in the recipe will work.
These variables are defined in meta/conf/bitbake.conf, where you can see that bindir is $exec_prefix/bin, which is probably why assigning prefix didn't work for you.

Your first strategy was on the right track, but you were clobbering more than you wanted by changing only "prefix". If you look in sources/poky/meta/conf/bitbake.conf you'll find everything you are clobbering when you set the variable "prefix" to something other than "/usr" (like it was in my case). In order to modify only the install path with what would manually be the "--prefix" option to configure, I needed to set all the variables listed here in that recipe:
prefix="/your/install/path/here"
datadir="/usr/share"
sharedstatedir="/usr/com"
exec_prefix="/usr"

Related

How to decide Quarkus application arguments in Kubernetes at run-time?

I've built a Quarkus 2.7.1 console application using picocli that includes several subcommands. I'd like to be able to run this application within a Kubernetes cluster and decide its arguments at run-time. This is so that I can use the same container image to run the application in different modes within the cluster.
To get things started I added the JIB extension and tried setting the arguments using a configuration value quarkus.jib.jvm-arguments. Unfortunately it seems like this configuration value is locked at build-time so I'm unable to update this at run-time.
Next I tried setting quarkus.args while using default settings for JIB. The configuration value documentation makes it sound general enough for the job but it doesn't seem to have an affect when the application is run in the container. Since most references to this configuration value in documentation are in the context of Dev Mode I'm wondering if this may be disabled outside of that.
How can I get this application running in a container image with its arguments decided at run-time?
You can set quarkus.jib.jvm-entrypoint to any container entrypoint command you want, including scripts. An example in the doc is quarkus.jib.jvm-entrypoint=/deployments/run-java.sh. You could make use of $CLI_ARGUMENTS in such a script. Even something like quarkus.jib.jvm-entrypoint=/bin/sh,-c,'/deployments/run-java.sh $CLI_ARGUMENTS' should work too, as long as you place the script run-java.sh at /deployments in the image. The possibility is limitless.
Also see this SO answer if there's an issue. (The OP in the link put a customer script at src/main/jib/docker/run-java.sh (src/main/jib is Jib's default "extra files directory") so that Jib places the script in the image at /docker/run-java.sh.
I was able to find a solution to the problem with a bit of experimenting this morning.
With the quarkus-container-image-docker extension (instead of quarkus.jib.jvm-arguments) I was able to take the template Dockerfile.jvm and extend it to pass through arguments to the CLI. The only line that needed changing was the ENTRYPOINT (details included in the snippet below). I changed the ENTRYPOINT form (from exec to shell) and added an environment variable as an argument to pass-through program arguments.
FROM registry.access.redhat.com/ubi8/ubi-minimal:8.3
ARG JAVA_PACKAGE=java-11-openjdk-headless
ARG RUN_JAVA_VERSION=1.3.8
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en'
# Install java and the run-java script
# Also set up permissions for user `1001`
RUN microdnf install curl ca-certificates ${JAVA_PACKAGE} \
&& microdnf update \
&& microdnf clean all \
&& mkdir /deployments \
&& chown 1001 /deployments \
&& chmod "g+rwX" /deployments \
&& chown 1001:root /deployments \
&& curl https://repo1.maven.org/maven2/io/fabric8/run-java-sh/${RUN_JAVA_VERSION}/run-java-sh-${RUN_JAVA_VERSION}-sh.sh -o /deployments/run-java.sh \
&& chown 1001 /deployments/run-java.sh \
&& chmod 540 /deployments/run-java.sh \
&& echo "securerandom.source=file:/dev/urandom" >> /etc/alternatives/jre/lib/security/java.security
# Configure the JAVA_OPTIONS, you can add -XshowSettings:vm to also display the heap size.
ENV JAVA_OPTIONS="-Dquarkus.http.host=0.0.0.0 -Djava.util.logging.manager=org.jboss.logmanager.LogManager"
# We make four distinct layers so if there are application changes the library layers can be re-used
COPY --chown=1001 target/quarkus-app/lib/ /deployments/lib/
COPY --chown=1001 target/quarkus-app/*.jar /deployments/
COPY --chown=1001 target/quarkus-app/app/ /deployments/app/
COPY --chown=1001 target/quarkus-app/quarkus/ /deployments/quarkus/
EXPOSE 8080
USER 1001
# [== BEFORE ==]
# ENTRYPOINT [ "/deployments/run-java.sh" ]
# [== AFTER ==]
ENTRYPOINT "/deployments/run-java.sh" $CLI_ARGUMENTS
I have tried the above approaches but they didn't work with the default quarkus JIB's ubi8/openjdk-17-runtime image. This is because this base image doesn't use /work as the WORKIR, but instead the /home/jboss.
Therefore, I created a custom start-up script and referenced it on the properties file as following. This approach works better if there's a need to set application params using environment variables:
File: application.properties
quarkus.jib.jvm-entrypoint=/bin/sh,run-java.sh
File: src/main/jib/home/jboss/run-java.sh
java \
-Djavax.net.ssl.trustStore=/deployments/truststore \
-Djavax.net.ssl.trustStorePassword="$TRUST_STORE_PASSWORD" \
-jar quarkus-run.jar

How bitbake searches for recipe in build process?

I am trying to find out that how bitbake search for recipe in build process ?
For example,
I have a recipe something like below:
DESCRIPTION = "PetaLinux GSTREAMER supported packages"
inherit packagegroup
GSTREAMER_PACKAGES = " \
gstreamer1.0 \
gstreamer1.0-python \
gstreamer1.0-meta-base \
gstreamer1.0-plugins-base \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-rtsp-server \
gst-shark \
gstd \
gst-perf \
gst-interpipes \
"
GSTREAMER_PACKAGES_append_zynqmp = " gstreamer1.0-omx"
RDEPENDS_${PN} = "${GSTREAMER_PACKAGES}"
When I searched gstreamer1.0 related recipe in yocto layers, I found two recipe, one of them is gstreamer1.0_1.16.1.bb in meta layer, and the other is gstreamer1.0_%.bbappend in meta-petalinux layer.
Both of these layers was added to the BBLAYERS in bblayers.conf file and the priorities that spesified with BBFILE_PRIORITY_* in related layer's layer.conf file is same.
So,
Which recipe will be used in build process in that case ?
What is the recipe lookup rules in yocto ?
I changed somethings to understand the behaviour:
For example,
I entered the invalid github URL that spesified in gstreamer1.0_%.bbappend recipe. When I tried to build the linux system, I encountered with an error. Thats fine.
Then I corrected the github URL in this recipe and entered invalid source code address that spesified in gstreamer1.0_1.16.1.bb recipe. When I tried to build linux system, process finished successfully.
Then I increased the priority of meta layer. I supposed to encounter with an error in this case but again build process finished successfully.
Could you please help me to understand this behaviour ?
Thanks.
You have two different files: a .bb and a .bbappend.
A .bb is the base recipe of one (or multiple) packages. It generally describe how to fetch, configure, compile, install files in a package for your target.
A .bbappend file is an 'append' file. It allows a meta (here meta-petalinux) to modify an existing recipe in another meta without copying it. A .bbappend can modify any steps of the bb file: source fetch, configure, compile, install...
You can for example create your own bbappend of Gstreamer, to enable pango (disbaled by default on my Yocto). The bbappend filename is gstreamer1.0-plugins-base_%.bbappend and only contains PACKAGECONFIG_append = "pango"
The Yocto Manual can give you more information on bbappend files here.

Yocto: Nothing provides python-re-native

I'm running into an issue including python pyparted as a native dependency in one of my image creation bbclasses.
There is a python scrip that runs to create a partitioned image file, normally I run sudo apt install python-pyparted to have pyparted in the environment in ubuntu. But I'm not sure what I did (update??), the ubuntu environment is completely ignored now. I tried figuring out how to make sure the dependencies are correct in my sdimage bbclass.
do_image_sdimage[depends] = "parted-native:do_populate_sysroot \
dosfstools-native:do_populate_sysroot \
mtools-native:do_populate_sysroot \
virtual/kernel:do_deploy \
splash-images:do_deploy \
python3-native:do_populate_sysroot \
python3-pyparted-native:do_populate_sysroot \
${#d.getVar('IMAGE_BOOTLOADER', True) and d.getVar('IMAGE_BOOTLOADER', True) + ':do_deploy' or ''}"
I get an error showing
ERROR: Nothing PROVIDES 'python3-re-native' (but virtual:native:/home/dev/app/OS/sources/meta-openembedded/meta-python/recipes-extended/python-pyparted/python3-pyparted_3.10.7.bb DEPENDS on or otherwise requires it). Close matches:
python3-rpm-native
python3-native
python3-nose-native
python3-native RPROVIDES python3-re-native
ERROR: Required build target 'my-image-default' has no buildable providers.
Missing or unbuildable dependency chain was: ['my-image-default', 'python3-pyparted-native', 'python3-re-native']
based on this it looks like I should be able to do this, but the depency chain ignores python3-native's RPROVIDES?

Using PREMIRRORS in Bitbake configuration

How do I use PREMIRRORS in Bitbake local configurations or recipes?
I want to provide my own download locations for some slow or inaccessible third-party URLs, but the official PREMIRRORS documentation is vague and lacks examples.
Note: These results are based on experimentation with Yocto 2.3, but probably apply to 2.5 as well.
A simple example
Suppose that your recipe file contains this target URL:
SRC_URI = "http://download.example.com:8080/foo/bar/baz-1.0.tar.gz"
Then in your local.conf, you can define your custom download location as:
PREMIRRORS_prepend = "http://download\.example\.com:8080/.* http://my-mirror.example.com/copies/\n"
In this default case (with no special placeholders) Bitbake does not include the additional /foo/bar path elements, and instead tries to download just the filename from http://my-mirror.example.com/copies/baz-1.0.tar.gz
Advanced examples
These samples use special predefined placeholders, which are detailed in the next section.
HTTP/HTTPS with same file structure
Recipe: SRC_URI = "https://example.com:1234/foo/bar.zip"
Setting: PREMIRRORS_prepend = "http(s)?://example\.com(:\d+)?/.* http://mirror.local/PATH\n"
Attempts: http://mirror.local/foo/bar.zip
HTTP/HTTPS with flat structure
Recipe: SRC_URI = "https://example.com:1234/foo/bar.zip"
Setting: PREMIRRORS_prepend = "http(s)?://example\.com(:\d+)?/.* http://mirror.local/MIRRORNAME\n"
Attempts: http://mirror.local/example.com.1234.foo.bar.zip
Just switch the hostname
Recipe: SRC_URI = "ftp://example.com:1234/foo/bar.zip"
Setting: PREMIRRORS_prepend = "(\w+)://example\.com(:\d+)?/.* TYPE://mirror.local/PATH\n"
Attempts: ftp://mirrors.local/foo/bar.zip
Placeholders in replacement URI
PREMIRRORS parses all matched URIs and provides five special placeholder values in the target URI. Supposing the matched URI is http://host.example.com:1234/foo/bar/baz.txt:
TYPE https
HOST host.example.com%3A1234
PATH foo/bar/baz.txt
BASENAME baz.txt
MIRRORNAME host.example.com.1234.foo.bar.baz.txt
Altering the PREMIRRORS variable
The PREMIRRORS variable consists of series of lines (separated by \n) each with a regular expression to match a URI, and then a replacement string, with both portions separated by a space.
Bitbake tries them in order of appearance, and you generally want your private mirrors to take priority, so prepend onto PREMIRRORS, ex:
PREMIRRORS_prepend = "http://original/location/.* http://alternate/location/\n"
What file should I edit?
You can add entries to PREMIRRORS inside your bitbake recipes, but it is not recommended, since a major use of PREMIRRORS is for people reusing your recipe in some other context or location.
Instead, you can put it inside your local.conf file in an existing build directory. Alternately, edit the source template which Poky script use when creating a new local.conf in a fresh build-directory.
Other questions
What about SOURCE_MIRROR_URL?
The SOURCE_MIRROR_URL is a quick way to add a series of PREMIRROR entries for all supported protocols. For example, this setting:
INHERIT += "own-mirrors"
SOURCE_MIRROR_URL = "TYPE://mirror.local/PATH"
is the same as writing:
PREMIRRORS_prepend = "\
cvs://.*/.* TYPE://mirror.local/PATH \
svn://.*/.* TYPE://mirror.local/PATH \
git://.*/.* TYPE://mirror.local/PATH \
gitsm://.*/.* TYPE://mirror.local/PATH \
hg://.*/.* TYPE://mirror.local/PATH \
bzr://.*/.* TYPE://mirror.local/PATH \
p4://.*/.* TYPE://mirror.local/PATH \
osc://.*/.* TYPE://mirror.local/PATH \
https?$://.*/.* TYPE://mirror.local/PATH \
ftp://.*/.* TYPE://mirror.local/PATH \
npm://.*/?.* TYPE://mirror.local/PATH \
"
It seems the INHERIT+SOURCE_MIRROR_URL directives will still work if used in your local.conf (as opposed to a particular recipe.) However, Bitbake will emit warnings, so it may not be the intended use-case. Ex:
WARNING: Invalid protocol in PREMIRRORS: ('cvs://.*/.*', 'TYPE://mirror.local/PATH')
How can I check and debug my settings?
The -D debug flag will cause bitbake to emit information about what URLs it attempts to download from. You can also use -C do_fetch, which will force it to try the fetch step and re-download anything needed for the given recipe.
bitbake -D -C do_fetch software-recipe-name-here
Here's some example debug output, showing the PREMIRROR URL it attempts to access:
DEBUG: some-software-1.0 do_fetch: Trying PREMIRRORS
DEBUG: some-software-1.0 do_fetch: Fetcher accessed the network with the command /usr/bin/env wget -t 2 -T 30 -nv --passive-ftp --no-check-certificate -P /home/user/build_foo/DL_DIR 'http://mirror.local/path/to/the/filename.ext
If you need to experiment and run bitbake many times, it will be faster to temporarily put your new PREMIRRORS_prepend directive into a particular test-recipe, as opposed to modifying the local.conf. This is because Bitbake won't need to re-parse all the other recipes whenever you change it.
What if I want to isolate a port-number, e.g. http://host:123/foo?
Apparently there's no easy way to get the 123 on its own. While PREMIRRORS allows you to match with regular expressions, it does not seem to support using captured text from the match inside the replacement URI.
The port number is present inside HOST and MIRRORNAME, but there's no standard mechanism to split those values apart.
Fllow up. What if I want to isolate a port-number, e.g. http://host:123/foo?
Maybe you can use captured.
like:
org: "http://somewhere.org:1234/somedir1/somedir2/somefile_1.2.3.tar.gz"
reg: "http://somewhere.org(:\d+)?/.*"
sub: "http://somewhere2.org\1/somedir3"
result: "http://somewhere2.org:1234/somedir3/somefile_1.2.3.tar.gz"

Buildroot Config Option for applying custom patch

I am new to buildroot and working to build Linaro with buildroot ..I have multiple fragment kernel config files and specified that in buildroot defconfig.
I have specified a custom kernel patches directory with BR2_LINUX_PATCH_DIR .
I dont have some of the config flags not set which are supposed to be there in the .config files..so i suspect that the Patches are applied successfully..so i tried giving a non existing location as Linux Patch dir and it does not give any error..
Is there anything other than giving value to BR2_LINUX_PATCH_DIR and what should be the format of the dir structure...in buildroot manual it says it should be
Package_name/patch name..For linux what should be the package name? It should be the same with which linux dir is created.for example for me it is linux-custom
Plz suggest and guide me in this.
Thanks in Advance
The option is named BR2_LINUX_KERNEL_PATCH, there is nothing named BR2_LINUX_PATCH_DIR. It applies all patches listed in this option (if those are files), or all files named *.patch if what's given in this option is a directory. See the code in linux/linux.mk:
define LINUX_APPLY_LOCAL_PATCHES
for p in $(filter-out ftp://% http://% https://%,$(LINUX_PATCHES)) ; do \
if test -d $$p ; then \
$(APPLY_PATCHES) $(#D) $$p \*.patch || exit 1 ; \
else \
$(APPLY_PATCHES) $(#D) `dirname $$p` `basename $$p` || exit 1; \
fi \
done
endef
Also, I would recommend that you watch the output of Buildroot: it shows everything it is doing, especially it lists the patches it applied. Look at the line >>> linux .... Patching, which is the marker for the beginning of the patching step of the linux package.