Yocto has a set of independent repositories containing the base system (Poky) and various software components (all the meta-* repositories here, and also openembedded layer index). So when you want to build an image for specific device and purpose, you need a handful of repositories checked out.
These are all tied together by the conf/bblayers.conf and conf/local.conf files in the build directory. But that is a build directory—it is supposed to be disposable, containing only information that can be easily regenerated on request. And it does—except for the list of layers in conf/bblayers.conf and a couple of definitions like the MACHINE in the conf/local.conf that define the target system to build for.
How should I version this information?
Currently we have a rather hacky script that assembles the build directory and writes the config files, but it does not know how to properly update them when it changes.
So is there a better option? Preferably one that would avoid any additional steps between checkout/update (with submodules or repo), oe-init-build-env init script (which creates the build directory if it does not exist) and running bitbake with appropriate target image?
Actually, repo is a convenient tool for managing manifest files with all the needed repositories.
Then you can use TEMPLATECONF to version local.conf and bblayers.conf. Here is how we do it: https://pelux.io/software-factory/master/chapters/baseplatform/building-PELUX-sources.html
The Poky distribution itself uses the Combo Layer tool, which seems to be designed to address this particular problem. However, it's not very clear what the workflow is supposed to look like, when using this tool.
Regarding the default bblayers.conf and local.conf files, you can either version them anywhere in your project and have a script copy them in your build folder after calling oe-init-build-env, or simmply use meta-poky/conf/bblayers.conf.sample and meta-poky/conf/local.conf.sample, which are automatically installed by oe-init-build-env when first creating the build directory.
Now, when you make changes or add layers, you will have to clear the build directory for the changes in the .sample files to take effect.
Related
I have a repository/program that requires Electron. GitHub doesn't allow adding files larger than 100MB, however, the electron.exe file in the Electron node module is 105MB so I can't figure out how to add it into my repo. Any ideas?
I think that is not required. Because Git is needed for management source code. Electron.exe and node modules can be processed by package.json.
So if you add package.json includes electron and other information, you can use easily later.
We are using VSTS for CI and CD in my team, we got over 40 repositories which are separated projects. but all of them have to run the same PowerShell script in one of their Build steps.
the PowerShell file is bigger too big to be kept as the inline script, so we need to save it inside a file. obviously, I got a copy of the PowerShell file in each repository.
Problem:
Now whenever I need to update the script, then I end up to update it in every repository, which is over 40 at the moment.
I think there should be a better approach. Is there any way that I can put my script in one single repo (a repo dedicated to holding the script) then I use it within each build, therefore we I need to update it I only need to update it once.
There are a few options.
My general recommendation is to publish the script as a package (NuGet or otherwise) and restore it during your application builds. This allows consumers to stay "pinned" to a known-good, known-working version, and update on a schedule that works for them.
Another option is to add a submodule to each repository that requires the script dependency, then initialize the submodule during the build process.
A third option is to turn the shared script into a VSTS build task or extension. This is extensively documented and easily located so I won't belabor the point by including instructions for doing that here.
You can add a git repository to store your powershell file.
Then add a build step to get you file from that repository during build and use it.
I got two solution setting at same location. This two solution are sharing some of the projects along with some dedicated ones.
I have created two separate build definition with gated check in trigger but issue is that when I make any change in one solution it triggers both the build definition.
Can I somehow control the triggering of the build definition based on the solution that I am checking in?
You need to configure your workspace correctly for this to work. Any change in a Build definition's mapped workspace will cause a build to trigger. Due to this, it completely depends on your source control layout, whether it's possible to setup a build that only triggers when something changes that belongs to either solution.
This setup will become very hard to manage quite quickly, as such I recommend you put each set of projects in their own subfolder, that makes it a lot easier.
So ensure that you build definitions won't both trigger, open the Source Settings panel of your build definition and apply a cloak rule to each file or folder by changing "active" in the first column to "cloaked".
To cloak a file you need to enter its full path in TFS, the UI will only offer you a folder picker, but entering a path to a file will work.
These files should:
Not be needed to build the solution
and changes to should not trigger the build.
Do note that the cloak will cause Team Build to not get these files on the Build agent, so it's not possible to have files your build depends on, but not trigger the build when these files change.
You should create gated check-in build definitions per project not per solution.
I am using TeamCity and I am new to it. I have added a Build Configuration to the TeamCity and I created one VCS root to attach to it.
However, my project have a special requirement to detect a particular file that was changed in the VCS root location and use that file in build step. I am sure this could be done in TeamCity, I am not able to figure out how.
Any help? Thanks,
To get the names of the files changed this is what I did. Thanks to Sam Jones.
I used System.TeamCity.build.changedFiles.file variable as follows.
Add a command line build step
Select Run as Custom Script
Add the script copy "%system.teamcity.build.changedFiles.file%" changelog.txt in script box.
You will get the changes in changelog.txt file in the format specified on this link.
NOTE: teamcity.build.changedFiles.file does not work. You need to use system.teamcity.build.changedFiles.file
It sounds like you want a VCS Trigger that specifies VCS Trigger Rules, so that a build configuration will run when someone makes a change to a particular file. The documentation has some nice examples of how to do this. If you're trying to trigger a build on one particular file, try this:
+:foo/bar.txt
This excludes all files from the trigger rule and then includes bar.txt in the foo directory. Paths are relative to the root of the repository (do not include a preceding slash). If someone modifies foo/bar.txt, the build configuration will be triggered to run.
VCS Trigger Rules also support pattern matching and all sorts of other options. Check out the documentation.
I'm trying to get continuous integration working using Hudson, Maven & Perforce (with an scm poll used to trigger each build)
One of the requirements is to automatically update POM snapshot version numbers following each successful build.
This is trivial using the Maven release plugin, but the act of checking in the newly versioned POMs triggers another build - which continues ad infinitum.
Is there any way of checking in version updates without triggering another build? There doesn't seem to be any way of excluding specific user checkins, specific changelists, or specific files when the scm poll makes the decision to rebuild.
This seems like a basic requirement - been doing it for years using Cruise Control / Ant / Perforce,
Cheers,
Mark
I have been contemplating a move to Hudson, but the inability to exclude specific files from the scm poll would be a deal breaker. Did you also ask this question on the Hudson mailing list?
users <at> hudson <dot> dev <dot> java <dot> net
The only workaround I can see from reading through the docs seems to be triggering builds based on p4 triggers, where you could then programmatically do whatever you want, but that doesn't seem to be a very simple approach.
I don't believe there is any way to configure Hudson to omit part of a Perforce workspace when polling. You may have better luck by setting up two different workspaces - one for build test using a default POM and one for your releases with the auto-updated release POM. The two workspaces could differ on just this one file and modifying the release POM would not trigger a new build.
Check out how to do individual file mapping in a workspace in the P4Guide.
Its possible to update the description of a changelist after it has been submitted, so if you build to a certain changelist number you can then update the description of that changelist to include your build number. This shouldnt trigger a new build.
What is "POM"?
You can have Hudson "Create or Update Label in Perforce" (see near the bottom of your project's configuration.) You could use that along with the BUILD_NUMBER environment variable that Hudson provides, giving you unique build numbers and stamped P4 labels of said build.
I have all tagging, ChangeLog updates, and packaging done by our checked in build code. This allows us to:
Create RC and appropriate tags without starting anther build
Allows us to debug all aspects of our build on any dev machine, not just those with Hudson.
Allows us to exactly recreate how a package, tag, etc was done in at any point in the past.
I had a similar problem checking in compiled src. I was able to find an alternative but i did consider the following.
In hudson specify a child folder of your project to poll. I.e. don't actually have hudson check out your project, just an incidental folder.
Add a shell script to check out your project proper.
Run you normal scripts.
check in your pom. The trick here is that the folder being polled won[t have changed at this point.
Not ideal, but works.
The perforce plugin has had polling masks for a while now. It's in the Advanced section under "View Mask".
Basically, you list a subset of files that you want polling to check for changes. Unfortunately there's no easy way to exclude files, so you'll have to create a list of file specs that specify everything BUT the file you want to exclude. So if the version is contained in a pom.xml, I would need to specify the files and directories that don't contain it.
For example, if your directory structure looked like this:
./pom.xml
./lib/
./src/
You would specify,
//depot/project/lib/...
//depot/project/src/...
as the view mask, and check the checkbox to use it only for polling. There are a few limitations with this feature, such as not being able to use '-' to exclude file specs, but it's still a workable solution for now.