I have a BitBake/Yocto question. I am building a project which uses Poky and some custom layers added on top of it. While the project is built it outputs "Build Configuration" that includes details like BB_VERSION, BUILD_SYSTEM, DISTRO_NAME, etc. It also has git branch name and version for all included layers like meta-<layer_name> : "<branch_name>:<revision_hash>". I am trying to access branch name and revision for custom layers added from the recipe but could not find a way to get access to it. Also these branch detail variables are lowercase vs other example variables that I listed above are uppercase. I am not sure if that makes any difference as I can access those uppercase variables during build but can't access the ones with lower case.
Would anyone have any clue how can I get git branch name and version via these variables without having to use git commands. Currently I am executing git commands to get those details. I have looked in BitBake documentation and the closest I could find was METADATA_BRANCH variable which gives me meta-yocto-bsp layer branch details.
The part where layers and their versions are printed are done by a function, not from plain variables. Check BUILDCFG_FUNCS for a list of functions that print build configuration data and specifically the get_layers_branch_rev function for the layer data
Related
Updated
I want to build and run many versions of my source code on Git which has many tags. The names of the tags have a rule. I want to run every versions of my source code if the tag of the version are fit to a specific policy. For example, I will merely run sources with tag staring with 'runnable-'.
Once I bring tag lists that satisfies with above, I want Jenkins to iterate a tag among tags list and trigger each one to run another pipeline which would clone source code into its working directory and build and deploy its binary to a remote server and send information for letting the remote server whether or not it is updated.
1) How can I get every tags from a repository of git?
2) How can I filter unmatched tags from the list above?
I could get the answer of 1) and 2) through below command
git tag –list ‘runnable-*
3) How can I iterate the source version of those tags? My initial thought is I will declare a global parameter to point out a tag name and trigger another pipeline for each one. But once I do like this, Is it possible to check each pipeline's result after all processes are done?
As I mention here, a git tag -l can accept a pattern
git tag -l <pattern>
Then for each tag, you can git checkout atag in order to make your working tree reflect the content of that tag.
Each checkout will result in a detached HEAD, but that won't matter since you don't develop (ie make new commits).
Is it good practice to edit source code in poky/build/tmp/work directory ? because if we accidentally cleansstate ,the changes will be erased.
Alternatively we can edit source code in "files" directory along with recipe file but since mostly code here is in zipped form due to large number of files , so we will need to unzip and zip again just to change one line of code.
So what is best way to edit source code in yocto ?
If your question is about permanent changes, then Dan's answer is the one to follow. I.e. add a <recipe name>.bbappend to the recipe in your own layer, in which you add
SRC_URI += "file://mypatch1.patch \
file://mypatch2.patch \
"
enumerating all the patches you need.
If there's a large number of patches, it might make sense to fork the upstream repository, and maintain your own branch in your fork. In that case, you'll likely want to reference your own repository, instead of either the upstream repository or tarball.
OTOH, if your question was more about work-in-progress; then sure, doing it in oky/build/tmp/workoky/build/tmp/work/xxxx will work. (And quite likely, it's what most people have been doing for a long time).
However, there's a much better way in recent releases (from 1.8, fido). The new tool is called devtool. You can use it as follows:
devtool modify -x <recipe-name> <path-to-unpack-source>
unpacks the source and creates a new bbappend to build from the unpacked source. It also creates a git repo in the source directory.
Now you can modify the source. You can test-build your modified source by running devtool build <recipe-name>. Once you're satisfied, use git add ... and git commit to commit your changes to the local repo. Once you've commited the changes to the local repo, you can run:
devtool update-recipe <recipe-name>
to update the recipe in question. When you're satisfied, you can run devtool reset <recipe-name> to remove the temporary bbappend.
See also: Yocto manual on modifying source code
If you are continuously "patching" a given package manually, I would recommend you to look at implementing a .bbappend file in a separate layer which applies your patch using the do_patch function (http://www.yoctoproject.org/docs/2.0/mega-manual/mega-manual.html#patching-dev-environment).
In my development team we use requirements management to control the expected behavior and functions of our products and a bug report tool to track 'problem reports' (PR). Any change in the requirements is done by a 'change proposal' (CP) which acts much like a commit on a code repository.
In order to make any commit that changes the final product one must supply on the CVS commit commentary a trace which can be either a CP (this means that the change on your code reflects a change in the product) or a PR (which means the changes in the code are being made to correct a problem). CPs and PRs are numbered so that one can link changes in code to the causing item (CP or PR).
Sample commentaries
Error correction commit commentary:
Kind: Error Correction
Trace: PR-015 Crashing upon startup
Description: Edited file foo.c in order to verify uninitialized variables.
Product change commit commentary:
Kind: Development
Trace: CP-053 New login mechanism
Description: Added login mechanism with library X and blablabla.
My problem here is that I don't have any easy way of retrieving all the files that were changed for a specific CP or PR neither can I retrieve all the changes made to the code.
I have tried to use regexp (RE from python) in order to parse the log but it has been a little tough to cover all possible logs. My regular expressions failed to retrieve the list here and there even after adapting it some times.
So, I've been thinking if there isn't any easier way or any project or product or even a CVS built-in feature that might help me here.
The objective question: How do I retrieve the list of modified files in a commit which is identifiable by a well-formed tag (the CP or PR)? Is there an easier way or should I stick to log parsing?
Environment details:
OS: Windows XP
CVS server: cvsnt
CVS client: tortoise / cvsnt
Didn't want to answer my own question but I think it may be helpful for future reference for people with the same problem.
Well, I managed to perform a search within the 'log messages' (cvs term for the commit commentaries) filtering by the content of the text in the log message and group the files changed on that commit.
As pointed out by 'Joakim Elofsson' here, cvsps is a good tool for grouping commit information in 'patch sets' which are individual commits with references to all the files changed in those commits.
I used the version of cvsps packed for cygwin as at this moment there's no port for windows.
Just install the cygwin with the internet setup available here and, when choosing packages, search for cvsps and cvs, install both.
I couldn't manage to use the checked out files on my system (through the /cygdrive/c folder) so I checked them out from the cygwin shell.
BEGIN Obs for tortoise users
You'll need to setup CVSROOT environment var to the current CVSROOT of your repository by the command:
export CVSROOT="Your CVSROOT string here"
Usually, if check the properties page of any file of your checked out code base there will be a tab named CVS. There'll be your CVSROOT string. (if you're using :sspi: to connect to your remote repository as I do, you may try to switch it to :pserver:, it did work for me but I don't know exactly why).
END Obs for tortoise users
Well, after checking out your repository use command:
cvsps
This will create the cvsps patchset base for your requests. Then use:
cvsps -l "Some regexp code"
It will search the patch sets for log messages matching the input regexp.
This is a sample from using cvsps -lP100-PR-FEX` on my database (changed some filenames and paths for being able to make it public...):
PatchSet 71
Date: 2012/10/25 11:30:44
Author: GUARITA
Branch: HEAD
Tag: (none)
Branches:
Log:
Kind: Error correction
Trace: P100-PR-FEX145
Description:
Corrections of the TRUE and FALSE conventions used by the C++ (true:everything but 0, false:0) P100 interface to the VB6 (false:0, true:-1 or 'all bits set to 1 which is -1 in 2's complement') P100Interface ActiveX object.
P100 Panel Version increment.
Members:
SidePanels/P100.wimp:1.2->1.3
SidePanels/Calcs/P100Interface/private/P100Interface.cpp:1.2->1.3
You may also use it to compare changes between tags (which I use to control releases) with cvsps -r <tag1> -r <tag2>.
I have already used some VCS like CVS, SVN and Git. One feature that I am missing cannot be found anywhere.
There are files which I would like to have in the repository but every user should have its own. So when you checkout you get a default of that file and that commit your changes only for yourself.
Why do I want this? There are some files like configuration where I would like to have a default version in the repository (e.g. for building releases or a starting base for new team members) but the changes to that file are only relevant to a certain developer (or working copy) because it will contain paths only valid for that developer/working copy.
Currently when I do not add this files:
- I miss them when creating a new working copy or exporting for a release build
- Have no history which changes I might have done for myself for experimenting
Currently when I add this files to the repository:
- I might never commit them so I have a default in the repository but my file is always flagged "changed". In SVN I can add it to the "ignore-on-commit" changelist to improve a bit.
- I might loose my very own changes of a difficult configuration file (data crash, laptop theft, etc.)
Is there a VCS capable of this? Do SNV or git support something regarding this I might have overseen?
If I understood and decomposed your task correctly
"Have a set of default templates of something, which are starting point of per-user customization and these customized versions must be stored separately and be accessible only by responsible person"
you can use this workflow (draft, subject of modifications and corrections), Subversion based for simplicity and transparent management (strong point of any CVCS really)
Subversion repository
Each user of repo have own predefined path inside repository-tree (with common path-pattern for manageability and easy automation of processes)
One special admin-only managed path also exist, not accessible by ordinary users
Our tree may seems like this (where Repository dir is a root of repository)
z:\>dir /s /B
z:\Repository
z:\Repository\Users
z:\Repository\Template
z:\Repository\Users\Alpha
z:\Repository\Users\Bravo
For every user-path we use Path-based Authentication, which provides access for every and each user only to own subtree in repository,
Template contains (as name assumes) templates stub for all user's documents
Adding new users to repo, obviously, becomes simple and easy automated task:
svn copy Template into new user's dir
add rw permissions for created location for user in authz-file
tell user URL of his personal tree in repo
I don't think the VCS is the problem here. It looks like if you have a file whose contents are dependent on the local environment, you should auto-generate it with a script. This way, you ignore the generated file, but version the script and each developer still gets a perfectly valid copy of the config files at run time. This is the same approach that is used, for example, with user specific IDE settings: .suo files on Visual Studio for example.
Update:
If you specifically need a set of defaults, then the solution is this:
Add the defaults to the repository.
Each dev works in their own branch. This way, they can version the
changes to the config files.
When re-basing onto master and/or merging, the devs simply never
merge their customized configs.
You can always set up a hook to check if the default config has been modified, and if so, maybe email the dev. You simply view such a commit the same as you would view a commit that does not compile.
Devs are smart. Sure, they make mistakes. But never under-estimate the power of some simple communication.
Of course, when the default configs do get overwritten with the customized ones of Dev X, then you use the powers of git to fix that commit immediately.
I am working on a project that depends on external programs, and needs to know the paths to them. I develop and use the project on several machines, using mercurial for version control. The paths are machine-dependent, so I keep them in a machine-specific config file. I would like the config file for each host to be version-controlled, but I need to ensure that the config file from one host would never overwrite the config file for another host when pushing or pulling between hosts. Is there any way to accomplish this?
In principle, Wim is right: machine specific configurations shouldn't be part of the project's source control. As long as you walk alone, this isn't a real problem, but once you want to provide generic releases of your project, you have to get rid of them. In that case you might not be happy about the fact, that the change history contains files with machine specific data.
Nevertheless, it may make sense to have machine specific data in version controlled files (personally I do this for my dot-rc files and shell scripts). In that case I would suggest to separate generic and specific configurations into different files and include/utilize the specific one at build- or runtime, depending on the currently used machine.
If it is not possible to detect the current machine automatically, you could still create an unversioned symbolic link on each machine, pointing to the appropriate specific configuration file. For instance, on the machine foo the file layout could look like this:
generic.conf version-controlled
specific-foo.conf version-controlled
specific-bar.conf version-controlled
specific.conf → specific-foo.conf unversioned symbolic link
An alternative to symbolic links is to use a hook which automatically creates specific.conf, e.g. on each invocation of hg update. As hooks are set in a repository's hgrc file, it can be defined individually on each machine. Here's an example of a corresponding hooks section in the .hg/hgrc file of a repository clone on the machine foo:
[hooks]
update = cp specific-foo.conf specific.conf
Machine specific configuration settings should not be version controlled in the same repository as the project code.
However, it is still a good idea to put an inactive sample configuration file in your code repository. And this sample could show a bunch of typical locations for the external program paths you mentioned as lines that are commented out. That way you make it easier to get your project running on new machines.