Bitbake build of simple source file not generating object file unless I set the -f (forced) flag option - yocto

Let me just say that I am new to Yocto. I have been able to create recipes, packages, images etc. however I am encountering the following issue. Using online references of which there are many, I have tried to create and build a simple Yocto 'helloworld' recipe using bitbake, which has the following structure:
├── conf
│ └── layer.conf
├── COPYING.MIT
├── README
└── recipes-example
├── example
│ └── example_0.1.bb
└── helloworld
├── files
│ └── hellopeterworld.c
└── helloworld_0.1.bb
The content of the hellopeterworld.c source file is as follows:
#include <stdio.h>
int main() {
printf("Hello, C Programming World! This is Peter!");
return 0;
}
The content of the helloworld_0.1.bb recipe is as follows:
SUMMARY = "bitbake-layers recipe"
DESCRIPTION = "A friendly program that prints Hello World!"
PRIORITY = "optional"
SECTION = "examples"
LICENSE = "MIT"
LIC_FILES_CHKSUM = file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302
SRC_URI = file://hellopeterworld.c
S = "${WORKDIR}"
do_compile() {
${CC} ${CFLAGS} ${LDFLAGS} hellopeterworld.c -o hellopeterworld
}
do_install() {
install -d ${D}${bindir}
install -m 0755 hellopeterworld ${D}${bindir}
}
When attempting to build the recipe, say, the build output in the folder:
/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-ostl-linux-gnueabi/helloworld/0.1-r0/
is as follows:
-rw-r—r—1 pgroves pgroves 65 Feb 23 10:45 configure.sstate
drwxr-xr-x 3 pgroves pgroves 4096 Feb 23 10:20 deploy-debs
-rwxr-xr-x 1 pgroves pgroves 13512 Feb 23 10:45 hellopeterworld
-rw-rw-r—1 pgroves pgroves 101 Feb 23 10:15 hellopeterworld.c
drwxr-xr-x 3 pgroves pgroves 4096 Feb 23 10:45 image
drwxr-xr-x 2 pgroves pgroves 4096 Feb 23 10:45 patches
drwxrwxr-x 2 pgroves pgroves 4096 Feb 23 10:45 pseudo
drwxrwxr-x 5 pgroves pgroves 4096 Feb 23 10:45 recipe-sysroot
drwxrwxr-x 7 pgroves pgroves 4096 Feb 23 10:45 recipe-sysroot-native
drwxr-xr-x 2 pgroves pgroves 4096 Feb 23 10:45 source-date-epoch
drwxrwxr-x 2 pgroves pgroves 4096 Feb 23 10:55 sstate-install-package_write_deb
drwxrwxr-x 2 pgroves pgroves 20480 Feb 23 10:55 temp
The problem is that the ‘hellopeterworld’ object is not always being generated on subsequent 'bitbake helloworld' builds. Running a bitbake -c clean helloworld removes the contents of '0.1-r0' directory. The contents is restored only if I force the build using bitbake -f -c compile helloworld but this has a side effect with the following WARNING being raised:
'helloworld_0.1.bb:do_compile is tainted from a forced run'.
The funny thing is that if I edit the source file and add a deliberate bad syntax change, the rebuild detects the bad syntax and generates an error, as expected:
NOTE: Executing Tasks
ERROR: helloworld-0.1-r0 do_compile: Execution of
'<my_workspace>/build-
openstlinuxweston-stm32mp13-disco/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-ostl-linux-
gnueabi/helloworld/0.1-r0/temp/run.do_compile.83597' failed with exit code 1:
hellopeterworld.c: In function 'main':
hellopeterworld.c:4:1: error: expected expression before '?' token
4 | ? printf("Hello, C Programming World! This is Peter!");
| ^
But if I then re-edit and fix the file and rebuild, the build completes but the the object file still isn’t being regenerated. I'm seeing the same thing with more complex examples.
Does anyone know the reason for this? Am I missing something here?

First of all, it is weird why it didn't fail because of a syntax error in variables SRC_URI and LIC_FILES_CHKSUM, they need to be in ".
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
SRC_URI = "file://hellopeterworld.c"
I have tested your recipe and the build restarts automatically when I change the hellopeterworld.c file in the recipe folder.
However, when you run do_clean it does not remove the sstate object of the recipe, it only removes the build output, so when you rerun again it will do nothing.
In order to clean the sstate object you need to run do_cleansstate.
Other than that, I do not see any issue with your recipe.

Related

Pyenv-installed Python creates virtualenv with empty include dir, no python-config in bin

Note: There are several similar questions, which I have seen and read. None of them are the precise problem I'm having, and none of their answers work for me.
I have installed several Python versions (Python 2.7, 3.8, 3.9, and 3.10) on my macOS Ventura system using Pyenv. The installed directories include a python-config file in /bin and an /include directory with many header files, so I do not need to install python-devel or similar:
$ ls -al /Users/williamsn/.pyenv/versions/3.10.9/bin/*-config
lrwxr-xr-x 1 williamsn staff 17 Dec 19 17:24
/Users/williamsn/.pyenv/versions/3.10.9/bin/python-config -> python3.10-config
lrwxr-xr-x 1 williamsn staff 17 Dec 19 17:24
/Users/williamsn/.pyenv/versions/3.10.9/bin/python3-config -> python3.10-config
-rwxr-xr-x 1 williamsn staff 2073 Dec 19 17:24
/Users/williamsn/.pyenv/versions/3.10.9/bin/python3.10-config
$ ls -al /Users/williamsn/.pyenv/versions/3.10.9/include/python3.10/
total 1192
drwxr-xr-x 86 williamsn staff 2752 Dec 19 17:24 .
drwxr-xr-x 3 williamsn staff 96 Dec 19 17:24 ..
-rw-r--r-- 1 williamsn staff 3224 Dec 19 17:24 Python.h
...
-rw-r--r-- 1 williamsn staff 3026 Dec 19 17:24 import.h
...
-rw-r--r-- 1 williamsn staff 48878 Dec 19 17:24 pyconfig.h
...
-rw-r--r-- 1 williamsn staff 2863 Dec 19 17:24 weakrefobject.h
As a best practice, I work exclusively in virtualenvs to keep the base Python installation clean and pristine. I create my virtualenvs using the venv module built in to Python 3:
$ python3.10 -m venv my_venv
After doing so, there are two problems:
The my_venv/include directory is empty (does not contain the python3.10 directory or any of the header files from the parent Python and does not symlink to the parent Python)
There is no python-config / python3-config / python3.10-config in my_venv/bin.
So, for things requiring the Python headers:
If it tries to find the headers using python-config (which many projects do), it will fail.
If it tries to find the headers using the Python prefix and adding /include to it (which many projects do, notably Boost), it will fail.
If it tries to find the headers by importing sysconfig and calling sysconfig.get_paths(), it will succeed, albeit with paths outside the virtualenv.
Now, on a case-by-case basis, I can work around this, and already have. I can manually copy over python_config and then either export CPATH to add /Users/williamsn/.pyenv/versions/3.10.9/include/python3.10 or modify the virtualenv to symlink to /Users/williamsn/.pyenv/versions/3.10.9/include. These work, but they don't seem right to me. I can't make a global workaround anywhere (such as exporting CPATH in my Bash profile), because I'm working with multiple Python versions, and I'd end up with the wrong headers half the time. It seems to me like a virtualenv that has an empty include directory and is missing python-config when its parent has a full include directory and contains python-config is a broken virtualenv.
Is there an option I'm missing to include these pieces? Is this a bug I need to file against Python/venv?

How to really see the contents of the different perl scripts linking to one file?

I wanted to view the contents of a perl script in our environment which is called dfv_run.pl and specifically check line 245. Line 245 from that script printed the message "Finished checking test result" in my simulation log file. After executing % which dfv_run.pl I am pointed to this location:
drwxr-xr-x 2 dfvmgr dfvadmin 4.0K Jun 22 2017 .SYNC
-r--r--r-- 1 dfvmgr dfvadmin 3.2K Jun 22 2017 loadenv.csh
-r-xr-xr-x 1 dfvmgr dfvadmin 5.2K Jun 22 2017 loadenv
drwxr-xr-x 4 dfvmgr dfvadmin 4.0K Jun 22 2017 ..
drwxr-xr-x 3 dfvmgr dfvadmin 4.0K Jun 22 2017 .
lrwxrwxrwx 1 dfvmgr dfvadmin 7 Jun 22 2017 dfv_comp.pl -> loadenv
lrwxrwxrwx 1 dfvmgr dfvadmin 7 Jun 22 2017 dfv_run.pl -> loadenv
lrwxrwxrwx 1 dfvmgr dfvadmin 7 Jun 22 2017 dfv_sim.pl -> loadenv
/tools/dfv/scripts/v11/bin
However the script is only less than 200 lines and I can see the same content for dfv_comp.pl, dfv_run.pl, and dfv_sim.pl (also diff did not show any difference among the 3 perl scripts). loadenv of course also showed me the same contents.
Any help as to how I can view the real content of each perl script is much appreciated. Additional info which might help:
SHELL=/bin/tcsh
KONSOLE_DBUS_SERVICE=:1.46
KONSOLE_DBUS_WINDOW=/Windows/17
KONSOLE_DBUS_SESSION=/Sessions/30
Kindly let me know if additional info is needed. Thank you in advance!
What you're seeing is the real content. All three *.pl filenames are aliases to the loadenv script - that one script handles all four commands.
If you look at the contents of loadenv (or any of the other three names), you will most likely see that it checks to see which name was used to invoke it and then sets some flags which will cause it to behave differently depending on which name was used.

babel . --copy-files incorrectly copy node_modules

I have a project structure like the following:
-rw-r--r-- 1 chung2014 staff 2774 Nov 7 19:13 README.md
-rw-r--r-- 1 chung2014 staff 75 Nov 26 23:27 babel.config.js
drwxr-xr-x 588 chung2014 staff 18816 Nov 26 23:01 node_modules
-rw-r--r-- 1 chung2014 staff 781 Nov 26 22:25 nodemon.json
-rw-r--r-- 1 chung2014 staff 377691 Nov 26 22:08 package-lock.json
-rw-r--r-- 1 chung2014 staff 1551 Nov 26 23:27 package.json
-rw-r--r-- 1 chung2014 staff 2941 Nov 26 23:29 server.js
drwxr-xr-x 11 chung2014 staff 352 Nov 26 23:03 src
drwxr-xr-x 5 chung2014 staff 160 Nov 26 21:55 test
if I have all the source code inside the src directory, (e.g put server.js into src as well), I can have a script babel src --out-dir dist/ --copy-files in my package.json to compile the all the source code in src to dist/ directory.
However, due to some restriction, I cannot put my server.js inside src directory. So when I try to have a script babel . --out-dir dist/ --copy-files in my package.json, I let babel incorrectly copy files in node_modules to dist, which is not what I want.
So my question is how I can just only compile and copy files from both server.js and src/ to the destination directory dist/ without copying files in node_modules/?
$ cat babel.config.js
const presets = [
"#babel/preset-env",
];
module.exports = { presets };
New solution available
--no-copy-ignored is a new argument which allows the value of --ignore to be respected when copying files.
Usage
Example:
babel src -d dist --ignore 'src/**/*.spec.js' --copy-files --no-copy-ignored
The spec files are not going to be present on the output directory.
Source: https://github.com/babel/babel/issues/6226#issuecomment-590283042
The only way to do this would be to drop --copy-files and do --ignore node_modules, e.g.
babel . --out-dir dist/ --ignore node_modules
and you'll also want to ignore dist/ and babel.config.js and anything else in the root that might contain JS files.
babel . --out-dir dist/ --ignore node_modules,dist,babel.config.js
Realistically, the better option would be for server.js to just proxy through to dist instead through, so you could do
babel src --out-dir dist/
and move server.js to src/server.js. If having a server.js is 100% necessary, then have it do require("./dist/server");.
create a script like:
require('fs-extra').copy(
process.argv.slice(-2).shift(),
process.argv.slice(-2).pop(),
{ filter: (src,dist)=>{ return (src.match(/\.js|\.jsx|stories|test/)===null)} },
err => { if (err) return console.error (err); console.log ('Copy success!');
});
and append this to your build command
&& ./scripts/--copy-files src dist/commonjs
the gist:
https://gist.github.com/kmrk/bbc52a4d54b407398aff1695e5b710b7

Postgresql could not load library

after days of regular fight with postgis I stuck on this:
postgres=# CREATE EXTENSION postgis; ERROR: could not load library
"/usr/pgpro-9.6/lib/rtpostgis-2.4.so": libgdal.so.20: cannot open
shared object file: No such file or directory
and have no idea how to pass it, because library path is correct...
[combat#urpordfinal ~]$ ls -alt /usr/pgpro-9.6/lib/ total 13080
-rwxr-xr-x 1 root root 1440056 Apr 23 11:52 postgis_topology-2.4.so
drwxr-xr-x 5 root root 8192 Apr 23 11:52 .
-rwxr-xr-x 1 root root 1878728 Apr 23 11:52 rtpostgis-2.4.so
I'm running pgsql 9.6.8 and psotgis build from source
You have to make sure that libgdal.so.20 is on the shared library path.
Find out where the library is and add that directory to the shared library path.
On Linux, you would normally do that by adding the directory to /etc/ld.so.conf (or, better, to a PostGIS configuration file in /etc/ld.so.conf.d) and running ldconfig.

Pydev tags import as “unresolved import” all compiled extensions

PyDev with Python3.5 seems unable to recognize imports from c-compiled extensions, including packages compiled via Cython.
I am working on an up-to-date debian/stretch machine with a stripped-down self-installed (in home dir) Eclipse/Neon with PyDev added via update site, if it matters.
I have both "Python 2.7.13" and "Python 3.5.2+" installed.
One of the problematic packages is lxml. I installed the debian packages and also tried installing manually via pip (and re-created the interpreter in eclipse afterwards to ensure full finding). In all cases packages do work.
In Python2 everything works as advertised.
Under Python3 PyDev flags from lxml import etree as error (but resulting program works):
Unresolved import: etree richiedi_certificato_dispositivo.py /trasmissione-telematica/Serializzazione line 8 PyDev Problem
Note: I can import lxml with no error, but then any access to lxml.etree... will be flagged in error. Data completion is consistent (i.e.: etree won't be in the offered list).
lxml is installed in the usual location:
mcon#vocore:~$ ls -l /usr/lib/python3/dist-packages/lxml
total 2156
-rw-r--r-- 1 root root 8152 Sep 5 2014 builder.py
-rw-r--r-- 1 root root 3366 May 5 2016 cssselect.py
-rw-r--r-- 1 root root 18387 May 5 2016 doctestcompare.py
-rw-r--r-- 1 root root 7641 Sep 25 2011 ElementInclude.py
-rw-r--r-- 1 root root 9490 Aug 20 06:48 _elementpath.py
-rw-r--r-- 1 root root 1710088 Aug 24 10:14 etree.cpython-35m-x86_64-linux-gnu.so
drwxr-xr-x 3 root root 4096 Jan 3 08:58 html
drwxr-xr-x 3 root root 4096 Jan 3 08:58 includes
-rw-r--r-- 1 root root 551 Oct 7 2012 __init__.py
drwxr-xr-x 4 root root 4096 Jan 3 08:58 isoschematron
-rw-r--r-- 1 root root 17450 Aug 20 06:48 lxml.etree_api.h
-rw-r--r-- 1 root root 8902 Aug 20 06:48 lxml.etree.h
-rw-r--r-- 1 root root 366440 Aug 24 10:14 objectify.cpython-35m-x86_64-linux-gnu.so
drwxr-xr-x 2 root root 4096 Jan 3 08:58 __pycache__
-rw-r--r-- 1 root root 92 Sep 5 2014 pyclasslookup.py
-rw-r--r-- 1 root root 8531 Nov 20 2014 sax.py
-rw-r--r-- 1 root root 230 Sep 25 2011 usedoctest.py
mcon#vocore:~/trasmissione-telematica$
As You can see etree is in a shared lib, as is objectify; a quick check shows also objectify is not handled by PyDev. I checked a few other "c-extension" packages (e.g.: import pycurl and from Crypto.Util import strxor) with the same result, so it seems a problem with "C" extensions.
Have you tried to put the offending packages in the forced builtins (as described in http://www.pydev.org/manual_101_interpreter.html)?
If that doesn't work, do you have some error in your error log? (see http://www.pydev.org/faq.html#PyDevFAQ-HowdoIReportaBUG%3F for details on getting it)