Yocto - exclude files from -dev package - yocto

I have a great problem moving some header files from FILES_${PN}-dev to a custom dev-internal package.
In OpenEmbedded documentation there is explicitly stated that there is "no actual support for explicitly excluding files from packaging".
I tried this:
FILES_${PN}-dev = ""
PACKAGES += "${PN}-dev-internal"
FILES_${PN}-dev-internal = "${includedir}/<my-pattern>.h"
FILES_${PN}-dev = "<original-content>"
but it seems that the first defined package captures first anyway.
Is there any known workaround for this? Except of naming everything explicitly in both packages content which is highly, really undesirable.

but it seems that the first defined package captures first anyway.
Yes. So put PN-dev-internal before PN-dev in PACKAGES. This might work:
PACKAGES =+ "${PN}-dev-internal"
If not, PACKAGE_BEFORE_PN is useful.

You can probably change the component installation config to better separate the headers (e.g. place them in different directories).

Related

Clarification of replace and extra sections in composer.json for TYPO3 extensions

Update (Aug 17, 2020): the section "my_ext_key": "self.version" is no longer found in newer examples or in helhum/composer.json. See clarifications below.
There are several examples for composer.json for TYPO3 around. Some include a replace section with something like the following:
replace:
"replace": {
"my_ext_key": "self.version",
"typo3-ter/my-ext-key": "self.version"
},
or
"replace": {
"vendorname/my-extension": "self.version",
"typo3-ter/my-extension": "self.version"
},
https://wiki.typo3.org/Composer#Extension_key_.2F_composer_name
And / or an extra section
"extra": {
"typo3/cms": {
"extension-key": "extension_key"
}
}
When are these sections necessary, what do they do and what is recommended (for TYPO3 extensions)? Are there differences between TYPO3 versions specific to these sections?
Specifically:
The second line in the replace example
"typo3-ter/my-ext-key": "self.version"
for example, makes sense if extension was published in TER and was
available from composer.typo3.org, I guess. If extension was only
available via packagist, it does not really make sense to include it.
About the first line of replace, when is this necessary and what does it do?
"my_ext_key": "self.version",
Usually, replace it used to replace some other package, e.g. as a drop-in-replacement which is explained for example here. This section was available in a previous example composer.json but has now been removed (see helhum/composer.json). Also, it would now cause a warning on composer validate, see also comment. Is it still necessary for older TYPO3 versions?
The extra part maps the extension key (with underscore) to package name (with dash). However, this is already the default. So, when is this required or can it be removed entirely?
As the official composer documentation for replace states:
Lists packages that are replaced by this package. This allows you to fork a package, publish it under a different name with its own version numbers, while packages requiring the original package continue to work with your fork because it replaces the original package.
This is also useful for packages that contain sub-packages, for example the main symfony/symfony package contains all the Symfony Components which are also available as individual packages. If you require the main package it will automatically fulfill any requirement of one of the individual components, since it replaces them.
Caution is advised when using replace for the sub-package purpose explained above. You should then typically only replace using self.version as a version constraint, to make sure the main package only replaces the sub-packages of that exact version, and not any other version, which would be incorrect.
For a more TYPO3 related explanation of the composer.json replace property there is an explanation available provided by Helhum:
The replace property MUST be provided if the composer package name part does not match the extension name and is highly recommended thus SHOULD be provided in any case. If it is provided, the first replace value MUST match exactly the extension key and the second replace value SHOULD match the TER package key (extension key prefixed with “typo3-ter/” and with underscores (_) replaced with dashes (-). The version constraint MUST exactly be self.version
For additional information please follow the links as they contain further information.

How does the key "file" of the structure "provides" work with "META.*" for "CPAN::Meta::Spec"?

I'm trying to better understand what I could use CPAN::Meta::Spec for and came across the following sentence in the spec for the key file:
[...]to a file that contains or generates the package. It may be given as META.yml or META.json to claim a package for indexing without needing a *.pm.
This sentence reads to me like one was able to directly specify some META.* within the config using a path to a file instead of a *.pm. Hence using the wording it, that clearly associates to the formerly mentioned path. Pretty much like in the following example:
provides => {
'Foo::Bar' => {
file => 'lib/Foo/Bar.pm',
version => '0.27_02'
},
'Foo::Bar2' => {
file => 'lib/Foo/Bar2.yml', <-- META.yml?
},
'Foo::Bar3' => {
file => 'lib/Foo/Bar3.json', <-- META.json?
version => '0.3'
}
So, while Foo/Bar2.pm and Foo/Bar3.pm might exist in the distribution, they are not defined explicitly, but implicitly using the META.* files.
How does such a META.* look like, what does it contain? Only things like name and version, which is what a native Perl package might provide as well? Or additional things like license and keyword, maybe everything except dependencies?
How do CPAN-clients handle such cases? META.* obviously isn't the Perl package itself and I don't see how it's used to generate it. So what gets actually installed in a system in the end? Is there some additional mechanism generating the package somehow?
How is providing META.* instead of *.pm compatible with the key version and the following restriction:
[...]If the package does not have a $VERSION, this field must be omitted.
Does META.* count as a package containing $VERSION in this case? Or is it expected that somehow the package gets generated in the end and that simply must have $VERSION as well and as long as the package isn't generated, the version of META.* can simply be used?
Thanks for your clarification!
The provides metadata is a list of the packages provided by the distribution primarily for use by the PAUSE indexer, but also may be used by analysis tools. If it is present, PAUSE will not inspect your files for packages and their versions but will trust provides. For each package in the distribution it must list the file in which the package resides, and the version of the package if it has one. Since this is an "override", it is not required to match reality, but unless you're doing something very strange it should. The ability to set the file to META.yml or META.json is simply a fallback if you have a package which has no associated file; it is extremely uncommon that you will need to do this, and it places no additional requirement on the META.json or META.yml except they should exist. As always, in implementation this metadata is always set in META.json and META.yml included in the distribution.

Yocto: how to remove/blacklist some dependency from RDEPENDS of a package?

I have a custom machine layer based on https://github.com/jumpnow/meta-wandboard.
I've upgraded the kernel to 4.8.6 and want to add X11 to the image.
I'm modifying to image recipe (console-image.bb).
Since wandboard is based on i.MX6, I want to include the xf86-video-imxfb-vivante package from meta-fsl-arm.
However, it fails complaining about inability to build kernel-module-imx-gpu-viv. I believe that happens because xf86-video-imxfb-vivante DEPENDS on imx-gpu-viv which in turn RDEPENDS on kernel-module-imx-gpu-viv.
I realize that those dependencies have been created with meta-fsl-arm BSP and vanilla Poky distribution. But those things are way outdated for wandboard, hence I am using the custom machine layer with modern kernel.
The kernel is configured to include the Vivante DRM module and I really don't want the kernel-module-imx-gpu-viv package to be built.
Is there a way to exclude it from RDEPENDS? Can I somehow swear my health to the build system that I will take care of this specific run-time dependency myself?
I have tried blacklisting 'kernel-module-imx-gpu-viv' setting PNBLACKLIST[kernel-module-imx-gpu-viv] in my local.conf, but that's just a part of a solution. It helps avoid build failures, but the packaging process still fails.
IIUC you problem comes from these lines in img-gpu-viv recipe:
FILES_libgal-mx6 = "${libdir}/libGAL${SOLIBS} ${libdir}/libGAL_egl${SOLIBS}"
FILES_libgal-mx6-dev = "${libdir}/libGAL${SOLIBSDEV} ${includedir}/HAL"
RDEPENDS_libgal-mx6 += "kernel-module-imx-gpu-viv"
INSANE_SKIP_libgal-mx6 += "build-deps"
I would actually qualify this RDEPENDS as a bug, usually kernel module dependencies are specified as RRECOMMENDS because most modules can be compiled into the kernel thus making no separate package at all while still providing the functionality. But that's another issue.
There are several ways to fix this problem, the first general route is to tweak RDEPENDS for the package. It's just a bitbake variable, so you can either assign it some other value or remove some portion of it. In the first case it's going to look somewhat like this:
RDEPENDS_libgal-mx6 = ""
In the second one:
RDEPENDS_libgal-mx6_remove = "kernel-module-imx-gpu-viv"
Obviously, these two options have different implications for your present and future work. In general I would opt for the softer one which is the second, because it has less potential for breakage when you're to update meta-fsl-arm layer, which can change imx-gpu-viv recipe in any kind of way. But when you're overriding some more complex recipe with big lists in variables and you're modifying it heavily (not just removing a thing or two) it might be easier to maintain it with full hard assignment of variables.
Now there is also a question of where to do this variable mangling. The main option is .bbappend in your layer, that's what appends are made for, but you can also do this from your distro configuration (if you're maintaining your own distro it might be easier to have all these tweaks in one place, rather than sprayed into numerous appends) or from your local.conf (which is a nice place to quickly try it out, but probably not something to use in longer term). I usually use .bbappend.
But there is also a completely different approach to this problem, rather than fixing package dependencies you can also fix what some other package provides. If for example you have a kernel configured to have imx-gpu-viv module built right into the main zimage you can do
RPROVIDES_kernel-image += "kernel-module-imx-gpu-viv"
(also in .bbappend, distro configuration or local.conf) and that's it.
In any case your approach to fixing this problem should reflect the difference between your setup and recipe assumptions. If you do have the module, but in a different package, then go for RPROVIDES, if you have some other module providing the same functionality to libgal-mx6 package then fix libgal-mx6 dependencies (and it's better to fix them, meaning not only drop something you don't need, but also add things that are relevant for your setup).

How to specify package versions in a Bitbake recipe with multiple packages?

I have a single Bitbake recipe that creates several packages, declared by using the PACKAGES variable and specifying the contents of each package using FILE_package1, FILE_package2, etc.
I would like to specify a different version string for each package, but they all share the version string stored in ${PV}. How can I provide a different version for each package?
Although I could be wrong, I don't think it's possible to do what you want. The ${PV} variable is typically specified in the title of the recipe (e.g. busybox_1.22.1.bb) and Bitbake sets the ${PV} variable based on that.
In order to specify different versions for each package I think you're going to have to split the recipe into multiple recipes, each one with its own version.

Using OpenWrap with Scope

I may not fully understand the wiki article on scoping, so forgive me if this sounds dumb.
Intro:
I have a solution (ABC.sln) with over 40 projects and am trying to implement OpenWrap for package management.
So I did the following in the solution's root folder:
o init-wrap -all
That worked fine: I now have a file called SLN.wrapdesc in the solution's root folder. All of the .csproj files in the subfolders contain the OpenWrap targets line.
I then proceded to add the different wraps to the solution with:
o add-wrap -Name xxx
Again, this worked fine: I have some wraps in the wraps folder, and the build doesn't break after removing the old references from the projects.
Problem:
All of the contents of the wraps are going to all of the projects, even for those that don't need it. I would like to be able to specify which wraps go where, eg AjaxControlToolkit only goes into web projects.
What I tried
First, I removed the AjaxControlToolkit from the wrapdesc:
o remove-wrap AjaxControlToolkit
This causes the build to break (as expected). Then I tried the following:
1. Try to add the wrap back with a scope:
o add-wrap -Name AjaxControlToolkit -scope webproject
This simply puts the wrap back in the wraps folder. I then added <OpenWrap-Scope>customscope</OpenWrap-Scope> to the project file, but the build still broke.
2. Try and manually add a file called ABC.webproject.wrapdesc to the root folder. This causes the following error when I try to open the solution:
The "exists" function only accepts a scalar value, but its argument "#(_WrapFile->'%(FullPath)')" evaluates to "D:\Projects\ABC.webproject.wrapdesc;D:\Projects\ABC.wrapdesc" which is not a scalar value.
I guess it doesn't like 2 wrapdesc files. That is strange because the wiki says "...you can add a second descriptor alongside your default descriptor..."
So now I'm stuck. Anyone have any ideas?
The per-msbuild file is really not a recommended approach to managing dependencies. Doing it per project is not quite the design philosophy behind OpenWrap, so the system is not quite optimized for those scenarios.
If you don't need something from those assemblies then the easiest way to solve it is to not use the references by not using any code from those packages. This solves the problem very easily as nothing will get loaded (or even need to be on disk) if no code has been added to it.
That said, add-wrap -scope newscope will create an additional .wrapdesc file that will add the new dependency to the new scope, by creating a myProject.newscope.wrapdesc file independently of the original myProject.wrapdesc.
If you do want to do this per-project, have you tried using the convention-based scoping? Something like:
directory-structure: src\*{scope: Web=WebProjects}*
Would take any project in a folder child of src containing Web in the name and assign those to the WebProjects scope.
I know that one has worked fine for my projects so far, although you do have to restart VS as it aggressively caches certain files and will not see the change.
Customizing the msbuild file itself is not fully tested (and the wiki entry was very much a design spec rather than final documentation, not all of it has been built that way) so it may or may not work. Happy to take a look if you can open a bug ticket on http://github.com/openrasta/openwrap/issues