I am trying to specify rather complicated labeling rule in VCS of Teamcity. Not sure if what I am trying to do is possible or not.
This is my directory structure I have inherited.
mysvn/abc/repos
|
-TestDomain
-TestSystem
|
-MyFrameWork
-MySoftware
|
-MySoftwareDevices
-MySoftwareFiles
|
-branches
-tags
-trunk1
-MySoftwareDriver
|
- branches
- tags
- trunk2
I want to specify such a rule that in the working directory of the checkout directory of Teamcity has structure like this:
Teamcity checkout directory
|
-FolderA
-FolderB
Where FolderA has contents of trunk1 and FolderB has contents of trunk2.
Is it possible to be done?
SVN URL like:
mysvn/abc/repos/TestSystem/MySoftware/MySoftwareFiles/trunk1
does give me trunk1. But I need contents of trunk1 and trunk2 under two different folders in the same build checkout directory.
Labeling I have been using: trunk=>tags
Ok got the thing working. Its kind of hit and trial. Hoping someone comes up with proper explanations.
Labeling rules still has:
trunk=>tags
The SVN URL : mysvn/abc/repos/TestSystem
It is the checkout rules which I added:
-:.
+:MySoftware/MySoftwareFiles/trunk1=>MyDir/FolderA
+:MySoftware/MySoftwareDriver/trunk2=>MyDir/FolderB
Related
My team <team-1> is sharing a github repo with <team-2>. The repo looks something like this (Simplified example):
infrastructure/
|
|-- .github/CODEOWNERS
|
|-- directory1/
|
|-- subdirectory1.1/
|
|-- subdirectory1.2/
|
|-- directory2/
|
|-- subdirectory2.1/
|
|-- subdirectory2.2/
|
|-- directory3/
|
|-- subdirectory3.1/
|
|-- subdirectory3.2/
<team-2> is the CODEOWNER of every directory in the repo, and my team <team-1> owns only subdirectory1.1 and subdirectory2.1.
In otherwords the CODEOWNERS file looks something like this:
github/CODEOWNERS
* #mycompany/team2
/infrastructure/directory1/subdirectory1.1 #mycompany/team1
/infrastructure/directory2/subdirectory2.1 #mycompany/team1
Given the aforementioned, what I would like to do is exclude team2 from every folder that team1 owns, ideally without removing the wildcard in the codeowners file.
* #mycompany/team2
/infrastructure/directory1/subdirectory1.1 #mycompany/team1 AND EXCLUDE TEAM2
/infrastructure/directory2/subdirectory2.1 #mycompany/team1 AND EXCLUDE TEAM2
What is the best way to do that without rewriting the whole codeowners logic?
According to the documentation, your example should work just fine.
Quote from the example:
# Order is important; the last matching pattern takes the most
# precedence. When someone opens a pull request that only
# modifies JS files, only #js-owner and not the global
# owner(s) will be requested for a review.
*.js #js-owner
So, in this case, it seems that one of the following is the case:
Github has fixed the issue, their docs are up to date, and your example now works just as you'd like it to.
OR: Their docs are incorrect and Github needs to fix either the feature or their docs to accurately describe behavior.
GitHub native option: GitHub Action
You can try and configure the GitHub Action mszostok/codeowners-validator which can validate the GitHub CODEOWNERS file based on specified checks.
For instance:
notowned: Reports if a given repository contain files that do not have specified owners in CODEOWNERS file.
Non-GitHUb native options
Gitolite
If by "code owner" you means push/write access to a folder, this is not supported natively by Git, or GitHub: if you can push to part of a repository, you can push to all of the repository.
What you might consider is pushing to an intermediate gateway repository, on a server you control, and where you can set-up an authorization layer, like gitolite
With Gitolite, you can with VREF restrict pushes by the names of dirs and files changed.
In your case:
repo foo
RW+ = #team2
R = #team1
RW VREF/NAME/infrastructure/directory1/subdirectory1.1 #team1
RW VREF/NAME/infrastructure/directory1/subdirectory2.1 #team1
Once the push is validated by Gitolite, that same server could in turn push to GitHub.
Split repository and git filter-repo
But a more natural way would be to split the repository in two, referencing the team1 content in the main parent teeam2 repository, as a submodule.
Not easy though, considering it would involve history rewriting, and folder reorganization.
In theory, prefixing a line with ! should work, since
the syntax for CODEOWNERS should be the same as for .gitignore (see https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners#codeowners-syntax)
.gitignore allows excluding (i.e. re-including) patterns by prefixing ! (see https://git-scm.com/docs/gitignore#_pattern_format)
In practice, though, I tried this and the CODEOWNERS lines starting with ! are invalid according to mszostok/codeowners-validator checks, so, no luck.
I have this structure of projects (folders) in git repository:
/src
/src/Sample.Backend.Common
/src/Sample.Backend.Common.Tests
/src/Sample.Backend.Common.Domain
/src/Sample.Backend.Common.Domain.Tests
/src/Sample.Backend.Pricing.Abstractions
/src/Sample.Backend.Pricing.Domain
/src/Sample.Backend.Pricing.Domain.Tests
/src/Sample.Backend.Pricing.Persistence
/src/Sample.Backend.Pricing.Persistence.Tests
/src/Sample.Backend.Accounting.Abstractions
/src/Sample.Backend.Accounting.Domain
/src/Sample.Backend.Accounting.Domain.Tests
/src/Sample.Backend.Accounting.Persistence
/src/Sample.Backend.Accounting.Persistence.Tests
/src/Sample.Backend.Api
/src/Sample.Common
/src/Sample.Frontend.Common
/src/Sample.Frontend.Web
/src/Sample.Tests.Common
(The sample is simplified, in real there are much more projects/folders.)
I want different pipelines for different parts. For example a pipeline to be triggered whenever any file is commited in master branch in any Backend project. Something like this:
trigger:
branches:
include:
- master
paths:
include:
- src/Sample.Backend.*
- src/Sample.Common
- src/Sample.Tests.Common
The problem is, that filter src/Sample.Backend.* is not working. I have to add exact name of each Backend folder to get it working. I could use exclude but I have the same problem - there are many other projects and I would have to name them all.
I found that wildcards are not supported: https://github.com/MicrosoftDocs/azure-devops-docs/issues/397#issuecomment-422958966
Is there any other way to achieve the same result?
Does Azure YAML pipelne support wildcards in path filter in trigger?
This is a known request on our main forum for product:
Support wildcards (*) in Trigger > Path Filters
This feature has not yet been implemented; you could add your comment and vote this on user voice.
As a workaround for us, we add an inline PowerShell task as the first task to execute the git command line git diff HEAD HEAD~ --name-only then get the modified file names and filter the files name in the latest submit, and use Logging Command to sets variables which are then referenced in custom conditions in the next steps in the build pipeline:
and(succeeded(), eq(variables['CustomVar'], 'True'))
Our inline PowerShell script:
cd $(System.DefaultWorkingDirectory)
$editedFiles = git diff HEAD HEAD~ --name-only
echo "$($editedFiles.Length) files modified:"
$editedFiles | ForEach-Object {
echo $_
Switch -Wildcard ($_ ) {
'XXXX/Src/Sample.Backend.*' {
Write-Host ("##vso[task.setvariable variable=CustomVar]True")
}
'XXXX/Src/Sample.Common*' {
Write-Host ("##vso[task.setvariable variable=CustomVar]True")}
'XXXX/Src/Sample.Tests.Common' {
Write-Host ("##vso[task.setvariable variable=CustomVar]True")}
}
}
Then add the condition for all remaining tasks:
In this case, if the changed files do not meet our filters, then all remaining tasks will be skipped.
UPDATE: 09/09/2021
This is possible now as it is written here
Wild cards can be used when specifying inclusion and exclusion branches for CI or PR triggers in a pipeline YAML file. However, they cannot be used when specifying path filters. For instance, you cannot include all paths that match src/app//myapp*. This has been pointed out as an inconvenience by several customers. This update fills this gap. Now, you can use wild card characters (, *, or ?) when specifying path filters.
Note: documentation seems to be not updated yet.
Old answer:
No this is not possible at the moment. You have even feature request here and I would recommend to upvote it. (I already did this) Rick in above mentioned topic shared his idea how to overcome the issue:
I currently achieve this by having 3 files:
azure-pipelines.yml ( This calls some python on each commit )
azure-pipelines.py (This checks for changed folders and has some parameters to ignore certain folders, then calls the API directly)
azure-pipelines-trigger.yml ( This is called by the python based on the changed folders )
It works well enough, but it is unfortunate for the need to go through these loops.
But it needs an extra work.
This feature will roll out over the next two to three weeks according to the latest release notes
Update on this.
It took a few weeks but the change mentioned by pavlo in the comments above finally got rolled out and path triggers are now supported in YAML.
I am working on a project which has multiple modules with their own doxyfiles. My idea is to have a single master doxyfile which can include other private doxyfiles to create a one big documentation of the project. The directory structure looks like following:
MyProject+
|-Private_Prj1+
| |-Doc+
| |-doxyfile_privateprj1
|-Private_Prj2+
|-Doc+
|-doxyfile_privateprj2
|-Doc+
|-doxyfile_myproject(AKA Master Doxyfile)
How can I configure the doxyfile_myproject to include doxyfile_privateprj1 and doxyfile_privateprj2 in such a way that when I run Doxygen on the doxyfile_myproject, it then sequentially runs other doxyfiles?
I have a module, built with Dist::Zilla. I have Dist::Zilla set up to automatically push changes out to my GitHub repo. Works great when the repo is private.
However, as soon as I make the repo public, I start getting errors during the build process. Specifically, these lines in the dist.ini
[Bugtracker]
web = http://github.com/myaccount/%s/issues
If I comment out these lines, it works. With these lines left in, I get an error:
Duplication of element resources.bugtracker.web at /Users/me/perl5/perlbrew/perls/perl-5.24.1/lib/site_perl/5.24.4/Dist/Zilla.pm line 595.
OK, so fine, I comment out the lines. However, another problem crops up. The version number of my builds no longer autoincrements and is stuck at the same number every time I try to release a build.
Is there some configuration setting I need to change with Dist::Zilla so it will play nice with public github repos? Here is the full dist.ini file:
name = Module-Test
author = me
license = Perl_5
copyright_holder = Me
copyright_year = 2018
[Repository]
;[Bugtracker]
;web = http://github.com/sdondley/%s/issues
[Git::NextVersion]
[GitHub::Meta]
[PodVersion]
[PkgVersion]
[NextRelease]
[Run::AfterRelease]
run = mv Changes tmp && cp %n-%v/Changes Changes
[InstallGuide]
[PodWeaver]
[ReadmeAnyFromPod]
type = markdown
location = root
phase = release
[Git::Check]
[Git::Commit]
allow_dirty = README.mkdn
allow_dirty = Changes
allow_dirty = INSTALL
[Git::Tag]
[Git::Push]
[Run::AfterRelease / MyAppAfter]
run = mv tmp/Changes Changes
[GatherDir]
[AutoPrereqs]
[PruneCruft]
[PruneFiles]
filename = weaver.ini
filename = README.mkdn
filename = dist.ini
filename = .gitignore
[ManifestSkip]
[MetaYAML]
[License]
[Readme]
[ExtraTests]
[ExecDir]
[ShareDir]
[MakeMaker]
[Manifest]
[TestRelease]
[FakeRelease]
Your [Bugtracker] entry leads to duplication because you are also setting the bugtracker through [GitHub::Meta]. Choose one or the other.
As for version number management, note that [Git::NextVersion] is based on your git tags. Make sure that these tags are present in your local repository and have the correct format. That plugin uses a command line invocation similar to this to obtain all tags:
git rev-list --simplify-by-decoration --pretty=%d HEAD | grep -oE 'tag: [^,)\s]+'
Public GitHub repos should not be a problem for Dist::Zilla – this is exactly the setup most dzil distros use anyway. But interactions between multiple plugins can lead to hard to track down bugs, especially since the order of plugins is important. It can help to organize your plugins by the phase in which they run, and to test whether the problem persists after removing optional plugins. It also tends to be better to start with a simple dist.ini and add plugins as pain points in your development process become apparent.
I'm using Terraform in a modular fashion in order to build out my infrastructure. I do this by having a configuration file that calls in the different modules. I want to pass an infrastructure variable which picks up what tagged version of the Github repository the application should be building out. Most importantly I'm trying to figure out how to make a concatenation of a string happen in the "source" variable of the configuration file.
module "athenaelb" {
source = "${concat("git::https://github.com/ORG/REPONAME.git?ref=",var.infra_version)}"
aws_access_key = "${var.aws_access_key}"
aws_secret_key = "${var.aws_secret_key}"
aws_region = "${var.aws_region}"
availability_zones = "${var.availability_zones}"
subnet_id = "${var.subnet_id}"
security_group = "${var.athenaelb_security_group}"
branch_name = "${var.branch_name}"
env = "${var.env}"
sns_topic = "${var.sns_topic}"
s3_bucket = "${var.elb_s3_bucket}"
athena_elb_sns_topic = "${var.athena_elb_sns_topic}"
infra_version = "${var.infra_version}"
}
I want it to compile and for the source to look like this (for example): git::https://github.com/ORG/REPONAME.git?ref=v1
Anyone have any thoughts on how to make this work?
Thanks,
Keren
This is not possible currently in Terraform itself.
The only way to achieve something like this is to use a separate script to interact with the git repository that Terraform clones into a subdirectory of the .terraform/modules directory and switch it to a different tag depending on which version you need. This is non-ideal since Terraform organizes these into directories based on a hash of the module path, but if you can identify the module in question it is safe to run git checkout within these repositories as long as you do not run terraform get again afterwards.
For more details and discussion on this issue, see issue #1439 in Terraform's issue tracker, where this feature was requested.
You could use envsubst or python jinja and use these wrapper scripts in your pipeline deploy script to actually build the scripts from .envsubst and .jinja files before your terraform plan/apply
https://github.com/uvoo/process-templates/tree/main/scripts
I wish terraform would support this but my guess is they never will so just add some simple functions/files into deploy scripts which is usually the best way to deploy.