Wrong file installed when building a machine - yocto

I have 3 layers:
meta-a:
meta-a
└── recipes-my
└── mypgk
├── mypgk_1.0.bb
└── mypkg_rel
├── config.conf
└── mypkg.service
mypgk_1.0.bb:
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}_rel:"
SRC_URI = "file://config.conf mypkg.service"
meta-b:
meta-b
└── recipes-my
└── mypgk
├── mypgk_1.0.bbappend
└── mypkg
└── config.conf
mypgk_1.0.bbappend:
FILESEXTRAPATHS_prepend_${MACHINE} := "${YOCTOROOT}/meta-a/recipes-my/${PN}/${PN}_rel:"
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
SRC_URI += "file://config.conf "
I also have a meta-c layer.
meta-c depends on meta-b,
meta-b depends on meta-a.
Layers priority:
meta-a = 14
meta-b = 15
meta-c = 16
Every layer defines a machine (machinea, machineb, machinec)
When I build machinea, the config.conf file from meta-a is installed.
When I build machineb, the config.conf file from meta-b is installed.
When I build machinec, the config.conf file from meta-a is installed instead the one in meta-b which have a higher priority.
When I build machinec I want mypkg from meta-b as-is, why bitbake use the other config.conf file?
If I rename config.conf to config_b.conf (changing .bbappend accordingly) in meta-b everything works as axpected.
EDIT:
meta-c content regarding mypgk is empty, i want it to inherit from meta-b.
in meta-c layer.conf i have this:
LAYERDEPENDS_c = "b"
mypgk_1.0.bb defined in meta-a is:
FILESEXTRAPATHS_prepend_$(MACHINE) := "${THISDIR}/${PN}_rel:"
SRC_URI = "file://config.conf "
SYSTEMD_PACKAGES = "${PN}"
SYSTEMD_SERVICE_${PN} = " mypkg.service"
do_install() {
install -m 755 -d ${D}${bindir}
install -m 755 -d ${D}${sysconfdir}
install -m 644 ${WORKDIR}/config.conf ${D}${sysconfdir}/config.conf
install -d ${D}${systemd_system_unitdir}
install -m 644 ${WORKDIR}/mypkg.service ${D}${systemd_system_unitdir}/mypkg.service
}
EDIT 2:
also copying recipes-my/* from meta-b to meta-c/ doesn't work and config.conf from meta-a is installed

the issue was the order of direictories added to FILESEXTRAPATHS.
modifing mypgk_1.0.bbappend in meta-b to:
FILESEXTRAPATHS_append_${MACHINE} := "${YOCTOROOT}/meta-a/recipes-my/${PN}/${PN}_rel:"
FILESEXTRAPATHS_prepend_${MACHINE} := "${THISDIR}/${PN}:"
solved the issue

Related

tileserver-gl: custom config file docker-compose

I am trying to include a Tileserver GL container to my existing docker-compose, using a personalized config.json file. Here is the relevant part of the docker-compose.yml
osm_tile_server:
image: maptiler/tileserver-gl
container_name: open_tile_server
volumes:
- ./Tile_server/data:/data
ports:
- '8081:8080'
- '5431:5432'
command:
- '-c my_config.json'
the data folder structure
.Tile_server/data/
├── malta.bbox
├── malta.osm.pbf
├── my_config.json
├── quickstart_checklist.chk
├── styles
│ └── my_style.json
└── tiles.mbtiles
when running docker-compose up the -c my_config.json file is ignored.
However it works if I simply run docker run -it -v $(pwd)/Tile_server/data:/data -p 8081:80 maptiler/tileserver-gl -c my_config.json and even weirdly, if I use --verbose as the command instance instead of -c my_config.json, the option is executed.

Why does autostart with systemd not work?

Yocto Version is warrior.
I did a yocto project with a GO/golang user app (https-server) that works just fine on a raspi3.
Now I'm trying to autostart it at the yocto image and doesnt't got it working.
I know there are plenty outher questions regarding this, but I didnt found something that helped.
e.g. I tried to follow all steps in this post
Enable systemd services using yocto
but it doesn't autostart at the raspi.
These files at the raspi for the service I found:
/lib/systemd/system/https-server.service
/etc/systemd/system/multi-user.target.wants/https-server.service
The application itself is running great if I start it manually,
it is at the raspi at /usr/bin/https-server
my build/conf/local.conf:
IMAGE_INSTALL_append = " kernel-image kernel-devicetree sudo apt dnsmasq nano dhcpcd git glibc-utils localedef curl go https-server"
meta-https-server/
├── conf
│   └── layer.conf
└── recipes-https-server
└── https-server
├── files
│   ├── https-server.go
│   ├── https-server.service
│   ├── mytest
│   ├── server.crt
│   ├── server.key
│   └── testvideo.mp4
├── go-sw.inc
└── https-server.bb
https-server.bb
require go-sw.inc
inherit go systemd
#inherit go update-rc.d systemd
SRC_URI += "file://https-server.service"
SRC_URI += "file://https-server.go"
SYSTEMD_PACKAGES = "${PN}"
INITSCRIPT_PACKAGES = "${PN}"
SYSTEMD_SERVICE_${PN} = "https-server.service"
# Path
MY_KEY = "/data/yocto/2020-04-21-poky-warrior/poky-warrior/meta-https-server/recipes-https-server/https-server/files/server.key"
MY_CERT = "/data/yocto/2020-04-21-poky-warrior/poky-warrior/meta-https-server/recipes-https-server/https-server/files/server.crt"
TESTVIDEO = "/data/yocto/2020-04-21-poky-warrior/poky-warrior/meta-https-server/recipes-https-server/https-server/files/testvideo.mp4"
MY_TEST = "/data/yocto/2020-04-21-poky-warrior/poky-warrior/meta-https-server/recipes-https-server/https-server/files/mytest"
# COMPILER
do_compile() {
go build /data/yocto/2020-04-21-poky-warrior/poky-warrior/meta-https-server/recipes-https-server/https-server/files/https-server.go
}
# INSTALL
do_install() {
# install -d to create directories, "${D}/${bindir}" is /usr/bin
# systemd
install -d ${D}${systemd_unitdir}/system
install -m 0644 ${WORKDIR}/https-server.service ${D}${systemd_unitdir}/system
# HTTPS certificate and key
install -d "${D}/${bindir}"
install -m 0755 "${MY_KEY}" "${D}/${bindir}"
install -m 0755 "${MY_CERT}" "${D}/${bindir}"
install -m 0777 "${TESTVIDEO}" "${D}/${bindir}"
install -m 0777 "${MY_TEST}" "${D}/${bindir}"
# HTTPS Server Software
install -m 0755 "${S}/build/https-server" "${D}/${bindir}"
}
FILES_${PN} += "${bindir}"
FILES_${PN} += "${libexecdir}"
FILES_${PN} += "${systemd_system_unitdir}"
REQUIRED_DISTRO_FEATURES= "systemd"
the service https-server.service
[Unit]
Description=HTTPS Server sw startup script
[Service]
ExecStart=/usr/bin/https-server
[Install]
WantedBy=multi-user.target
I found out what was causing the problem:
At my local configuration at build/conf/local.conf
I had only this:
DISTRO_FEATURES_append = " systemd "
after I added the following:
DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
VIRTUAL-RUNTIME_init_manager = "systemd"
VIRTUAL-RUNTIME_initscripts = ""
It was working fine. The process https-server started at startup.
I checked the running processes after startup with ps, as
systemctl was not working on my core-image-minimal image:
root#raspberrypi3:~# ps
[...]
152 root 861m S /usr/bin/https-server
[...]
root#raspberrypi3:~#
So this made the difference. Don't know if the missing space at my
DISTRO_FEATURES_append = " systemd" was also wrong... ??
#DISTRO_FEATURES_append = " systemd"
DISTRO_FEATURES_append = " systemd "
DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
VIRTUAL-RUNTIME_init_manager = "systemd"
VIRTUAL-RUNTIME_initscripts = ""
The oe-pkgdata-util that #Nayfe suggested is a very helpful tool here:
user#machine:[...]/poky-warrior/build$
oe-pkgdata-util list-pkg-files -p https-server
https-server:
/lib/systemd/system/https-server.service
/usr/bin/https-server
/usr/bin/mytest
/usr/bin/server.crt
/usr/bin/server.key
/usr/bin/testvideo.mp4
https-server-dbg:
/usr/bin/.debug/https-server
https-server-dev:
https-server-ptest:
I also worked over the recipe https-server.bb from above
to avoid absolute paths as #Nayfe suggested. This was not causing
the problem but it was bad style.
Don't use /data/yocto/2020-04-21-poky-warrior/poky-warrior/meta-https-server/recipes-https-server/https-server/files/ prefix, and add every files in SRC_URI instead.
require go-sw.inc
inherit go systemd
# "${D}/${bindir}" is /usr/bin
# ${WORKDIR} is path at local directory,
# this can be used instead of absolute paths
SRC_URI += "file://https-server.service"
SRC_URI += "file://https-server.go"
SRC_URI += "file://server.key"
SRC_URI += "file://server.crt"
SRC_URI += "file://testvideo.mp4"
SRC_URI += "file://mytest"
SYSTEMD_PACKAGES = "${PN}"
INITSCRIPT_PACKAGES = "${PN}"
SYSTEMD_SERVICE_${PN} = "https-server.service"
# COMPILER
do_compile() {
go build ${WORKDIR}/https-server.go
}
# INSTALL
do_install() {
# install -d to create directories, "${D}/${bindir}" is /usr/bin
# systemd
install -d ${D}${systemd_unitdir}/system
install -m 0644 ${WORKDIR}/https-server.service ${D}${systemd_unitdir}/system
# HTTPS certificate, key and testdata for https-server
install -d "${D}/${bindir}"
install -m 0755 ${WORKDIR}/server.key "${D}/${bindir}"
install -m 0755 ${WORKDIR}/server.crt "${D}/${bindir}"
install -m 0777 ${WORKDIR}/testvideo.mp4 "${D}/${bindir}"
install -m 0777 ${WORKDIR}/mytest "${D}/${bindir}"
# HTTPS Server Software
install -m 0755 "${WORKDIR}/build/https-server" "${D}/${bindir}"
}
# FILES_${PN} is Yocto’s way of specifying
# which files are expected to be installed along with which package
# (${PN} is a variable holding the main package’s name).
FILES_${PN} += "${bindir}"
FILES_${PN} += "${libexecdir}"
FILES_${PN} += "${systemd_system_unitdir}"
REQUIRED_DISTRO_FEATURES= "systemd"
Thanks to #kostix and #nayfe for their suggestions.

Postgres Docker: "postgres: could not access the server configuration file "/var/lib/postgresql/data/postgresql.conf": No such file or directory"

I am having weird issues with official postgres docker image. Most of the time it works fine, if I shut down the container and launch it again, I sometimes get this error but it's not every time:
PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres: could not access the server configuration file "/var/lib/postgresql/data/postgresql.conf": No such file or directory
I am launching postgres image using this command:
export $(grep -v '^#' .env | xargs) && docker run --rm --name postgres \
-e POSTGRES_USER=$POSTGRES_USER \
-e POSTGRES_DB=$POSTGRES_DB \
-e POSTGRES_PASSWORD=$POSTGRES_PASSWORD \
-p $POSTGRES_PORT:$POSTGRES_PORT \
-v $POSTGRES_DEVELOPMENT_DATA:/var/lib/postgresql/data \
postgres
I keep variables in .env file, they look like this:
POSTGRES_USER=custom-db
POSTGRES_DB=custom-db
POSTGRES_PASSWORD=12345678
POSTGRES_PORT=5432
POSTGRES_DEVELOPMENT_DATA=/tmp/custom-db-pgdata
When I try to echo variables the values are there so I don't think I'm passing null values to docker env variables.
The directory on my host machine looks something like this:
/tmp/custom-db-pgdata
├── base
│   ├── 1
│   ├── 13407
│   ├── 13408
│   └── 16384
├── global
├── pg_logical
├── pg_multixact
│   ├── members
│   └── offsets
├── pg_notify
├── pg_stat
├── pg_stat_tmp
├── pg_subtrans
├── pg_wal
│   └── archive_status
└── pg_xact
If it's inconsistent in how it works between executions on the same machine and same session (aka without rebooting) then something isn't mapping your directories properly. Finding what it is that's breaking will be difficult, more so since you're on a Mac. Docker on a Mac you has the extra bonus of running through a VM, so docker is mapping your local drive/path through to the VM and then mapping that into the container image, so there are two different layers where things can go wrong.
Dario has the right idea in his clarifying comments, you shouldn't rely on /tmp since that also has Mac Magic to it. It's actually /var/private/somegarbagestring and is different every bootup. Try switching to a /Users/$USER/dbpath folder and move your data to that, so at least you're debugging with one less layer of magic between data and database.

AlpineLinux PXE boot specify startup script as kernel parameter

Is there a way to specify a script as kernel parameter during pxe boot? I want run a bunch of computers as workers. I want them to use PXE to boot AlpineLinux and then run a bash script that will load my app and join my cluster.
Change dir:
cd /tmp
Create directory structure:
.
└── etc
├── init.d
│   └── local.stop
└── runlevels
└── default
└── local.stop -> /etc/init.d/local.stop
mkdir ./etc/{init.d,runlevels/default}/
Create file ./etc/init.d/local.stop:
#!/sbin/openrc-run
start () {
wget http://172.16.11.8/share/video.mp4 -O /root/video.mp4
}
chmod +x ./etc/init.d/local.stop
cd /tmp/etc/runlevels/default
Make symlink:
ln -s /etc/init.d/local.stop local.stop
Go back:
cd /tmp
Create archive:
tar -czvf alpine-test-01.tar.gz ./etc/
Make pxelinux (on your tftp server) menu:
label insatll-alpine
menu label Install Alpine Linux [test]
kernel alpine-installer/boot/vmlinuz-lts
initrd alpine-installer/boot/initramfs-lts
append ip=dhcp alpine_repo=https://dl-cdn.alpinelinux.org/alpine/latest-stable/main modloop=https://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/x86_64/netboot/modloop-lts modules=loop,squashfs,sd-mod,usb-storage apkovl=http://{YOUR_WEBSERVER}/{YOUR_DIR}/alpine-test-01.tar.gz
And run:
My webserver log:
10.10.15.43 172.16.11.8 - [27/Aug/2021:01:15:22 +0300] "GET /share/video.mp4 HTTP/1.1" 200 5853379 "-" "Wget"

What kind of files or directory is expected by mongorestore when using the -d flag?

I want to use the mongorestore command in a script, but I am having troubles understanding exactly what kind of input it is looking for.
After using the mongodump command, I end up with this tree:
mydirectory
└── dump
├── mydb1
│   ├── schemas.bson
│   └── schemas.metadata.json
├── mydb2
│   ├── schemas.bson
│   ├── schemas.metadata.json
│   ├── status.bson
│   └── status.metadata.json
└── mydb3
├── schemas.bson
└── schemas.metadata.json
I understood that I can use the mongorestore command like this:
mydirectory$ mongorestore
since it is looking by default for the dump directory.
However, I do not understand why using the following command:
mydirectory/dump$ mongorestore mydb1
give the following results:
2018-01-02T14:35:59.823+0100 building a list of dbs and collections to restore from mydb1 dir
2018-01-02T14:35:59.823+0100 don't know what to do with file "mydb1/schemas.bson", skipping...
2018-01-02T14:35:59.823+0100 don't know what to do with file "mydb1/schemas.metadata.json", skipping...
2018-01-02T14:35:59.823+0100 done
Moreover, when I use the -d flag to specify a database to restore, it only works when I specify the directory in which this database is located, for example:
mydirectory/dump$ mongorestore mydb1 -d mydb1
(I would have expected this command to work without the -d flag)
What kind of files or directory is mongorestore expecting when using (or not) the -d flag?
mongorestore expects the dump folder to contain sub-folders with the database name, which in turn contain the BSON dump and the metadata. The error you're seeing is because it didn't find any subdirectory with BSON/metadata files in it.
Rather than restoring by going into the dump directory, it's better to use the --nsInclude option instead (new in MongoDB 3.4). See the nsInclude documentation for more details.
The option --nsInclude requires you to supply the namespace in the form of <database>.<collection>. For example, to restore the test database:
mongorestore --nsInclude "test.*"
To restore the test collection inside the test database:
mongorestore --nsInclude "test.test"
Make sure that you execute the restore from the dump directory's parent, and not from inside it.