I need as a part of the build process to download contents from external github repository. I set up repository under "services" but I can not find a task which will download artifacts from that repo.
I use TFS 2017 on prem. My repository is already set to Git repo and I need to have one of build steps to pull data from yet another Git repo. How do i do that?
Build for the same github repo
If you want to download artifacts to your local path, you only need to use copy files task in your build definition.
Get source: select Github and use github token to authoize. If you want CI build, set in Triggers Tab.
Copy Files: set $(Build.SourcesDirectory) as Source Folder, specify the file you want to download in Contents, set a local path as Target Folder.
If you want to download/publish artifacts to VSTS server or share folder, you can use copy files task and publish build artifacts task in you build defnition.
Get source: select from github.
Copy Files: set $(Build.SourcesDirectory) as Source Folder, specify the file you want to download in Contents, set $(build.artifactstagingdirectory) as Target Folder.
Publish Build Artifacts: set $(build.artifactstagingdirectory) as Path to Publish, select the type you want to publish.
The way to connect github repo for TFS build:
In TFS build definition -> Repository Tab -> select External Git -> click Manage to add an External Git Service Endpoint -> input your github repo URL, username and password -> OK -> Then select the endpoint as connection.
Build for a git repo, and also need to download code from another github repo
You can use Command Line task to clone the github repo to your $(Build.SourcesDirectory) folder.
Settings of command Line task:
Tool: git
Arguments: clone https://github.com/username/repo
Now the code of the github repo is cloned in $(Build.SourcesDirectory)\repo.
Related
I have created a pipeline in azuredevops for .NET project.
My .sln file and docker file are present in different git repos.
How can I specify the dockerfile path in my azurepipeline.yml file.
Since your .sln file and docker file are in different git repos, you need to check out two git repos in Yaml file.
If your two git repos are in the same organization, you could directly add check out step in Steps.
For example:
steps:
- checkout: self
- checkout: git://Test Project/Docker Repo#master
Explaination:
The first checkout step is used to check out the repo where the yaml file is located (e.g. sln repo).
The second checkout step is used to checkout another repo(e.g. dockerfile repo).
Then two Repos will be downloaded to the $(Build.SourcesDirectory) (C:\agent_work\1\s)
So the dockerfile path could be $(Build.SourcesDirectory)\RepoName\...\Dockerfile.
If two git repos are in different resource (e.g. github , other organizaitons) , you need to add repo resource and checkout the repo.
Here is the doc about Use multiple repositories in your pipeline.
I have set up a multi-repo in Azure DevOps using Git. Assume I have 2 repos (Repo A and B) for simplicity.
Repo A's outcome is a common library DLL. Repo B references Repo A's generated DLL (via a NuGet package).
The CD pipeline of Repo A pushes the NuGet package (which was packed during the CI pipeline of Repo A) to my Azure Artifacts feed.
Is there a way to add a task in the CD pipeline of Repo A to automatically change the NuGet package reference (to a newer version) in the project of Repo B? This way, when Repo B gets pulled on the local repository, the reference would be updated instead of going through the NuGet Package Manager and updating manually in Visual Studio?
Update:
After applying your suggestions, everything is working perfectly except the fact that when I pull the changes of RepoB to my local repo, the new NuGet package of Repo A is not available in the packages folder [of Repo B] and I have to manually get it from the Package Manager.
Pre-Build event helps here but I want Repo B be fully up-to-date upon pulling from Azure Repos.
Below is the CD pipeline of Repo A.
Is there a way to add a task in the CD pipeline of Repo A to automatically change the NuGet package reference (to a newer version) in the project of Repo B?
There is no such out-of-the-box approach to doing this.
The simple way is:
Add a Pre-build event to the project in the Repo B with following command line:
nuget.exe update $(ProjectDir)packages.config
In case, when Repo B gets pulled on the local repository, the reference would be updated when you build the project in the repo B. But the limitation for this way is that this method will only modify our local files and will not directly modify the files in the repo. We still need to submit the changes to the repo manually.
The Complex way is:
Add a command line task in the CD pipeline of Repo A to use git command line to clone repo B:
git config --global user.email "xxx#xyz.com"
git config --global user.name "Admin"
git clone <repo> <directory>
Then add powershell or any other task to update the Reference, HintPath info in the project file and the package version in the packages.config file.
After modifying the files, add another command line task to submit the changes to the repo:
git commit -m "Update package version"
git push -u origin master
Update:
When you use git clone <repo> <directory> to clone the repo, you need to provide your certificate in your source link, usually using PAT:
The link looks like:
https://<OrganizationName>#dev.azure.com/<OrganizationName>/MyTestProject/_git/TestSample
Then we need to replace the first OrganizationName with PAT. So, it will be:
https://<PAT>#dev.azure.com/<OrganizationName>/MyTestProject/_git/TestSample
For your case, it should be:
https://<PAT>#xxxxx.Visualstudio.com/....
Then I could clone it successfully.
Hope this helps.
This can be done a bit differently than asked.
Using Project References (Newer projects):
Change your Repo B project's package reference version to get the latest using the wilcard symbol * in the Version attribute.
<PackageReference Include="MyPackageName" Version="*" />
Using packages.config
packages.config does not support the wildcard symbol and takes a few more steps.
You will need to add the following to your nuget.config file inside the <configuration> section:
<config>
<add key="dependencyversion" value="Highest" />
</config>
Be sure that the nuget.config is a part of the repo.
Then In the pipeline's Nuget restore task under the Feeds to use section select Feed in my NuGet.config and then specify the path to the repo's nuget.config.
In either case, when Repo B compiles in the pipeline it will grab the latest. When the repo is pulled down clean or for the first time it will grab the latest on the initial compile as well and on any new version that has been published.
I am very new to VSTS platform. In one of my project, I am trying to integrate the bitbucket source control to VSTS. By this way I should be able to see the updates made on bitbucket onto the VSTS account.
I have tried creating build on VSTS, but that only shows the commits history of the selected repository of bitbucket.
Is there a way to manage all the bitbucket changes on VSTS as source control?
To sync changes from bitbucket repo to VSTS git repo automatically, you can achieve it by using a VSTS build definition. Detail steps as below:
1. Create a build definition with Bitbucket repo
When creating a VSTS build definition -> Select the Bitbucket repo you want to sync -> create.
2. Enable continuous integration
In the build definition -> Triggers Tab -> Enable continuous integration -> Include all branches with *.
3. Add PowerShell task with the script to sync bitbucket repo with VSTS git repo
Add a PowerShell task with below script:
if ( $(git remote) -contains 'vsts' )
{git remote rm vsts
echo 'remove remote vsts'
}
$branch="$(Build.SourceBranch)".replace("refs/heads/","")
git remote add vsts https://Personal%20Access%20Token:PAT#account.visualstudio.com/project/_git/repo
git checkout $branch
git push vsts $branch -f
For the detail steps to add and config the PowerShell task as below:
Edit your build definition -> Click + to add a task for your agent phase -> Search powershell task -> click Add -> click the PowerShell task you added -> select Inline type -> then add your powershell script in the Script option -> Save build definition.
Now no matter which branch is updated in your bitbucket repo, VSTS git repo will be synced automatically.
Yo sync changes from VSTS git repo to bitbucket repo, you can create another CI build to achieve it. Detail steps as below:
1. Create a CI build with VSTS git repo
2. Enable continuous integration
3. Add a PowerShell task with below aspects
if ( $(git remote) -contains 'bitbucket' )
{git remote rm bitbucket
echo 'remove remote bitbucket'
}
git remote add bitbucket https://username:password#bitbucket.org/username/repo.git
$branch="$(Build.SourceBranch)".replace("refs/heads/","")
git checkout $branch
git push bitbucket $branch -f
When you connect your Bitbucket account to VSTS, you are setting up build triggers to run automated builds on pull requests or merges. This is what is called "continuous integration" in the DevOps world.
Consider reading the documentation for more information on this topic.
You will continue to "manage" your Bitbucket repos on Bitbucket. It's totally separate. If you want to manage everything through VSTS, you should import your Bitbucket repo to your VSTS account.
I maintain a public repository on GitHub where changes are only made to a single YAML file. I'm looking for a solution to process that file on every push and generate files based on it. Essentially, a pipeline or CI should parse the file and create many different markdown files. These files (or more specifically, the changes to these files) should then be pushed back to the repository.
Requirements:
Manual changes to the YAML file and automatic changes to the markdown files should both be pushed to the master branch.
The version history should be kept (e.g. forced push might not work).
There is an arbitrary number of files that are generated.
There are Travis providers for GitHub Pages and GitHub Releases. But both have limitations that make them unsuitable for my requirements.
Using what tool/CI/pipeline can I achieve that on GitHub? I would prefer a service over a self-hosted CI.
Assuming that you already have the program/script to parse the YAML file and to generate the Markdown files, I can give you some insights on how I would do this from Jenkins CI. While I draw my experience from running my own instance, there are also hosted options such as CloudBees that you can explore.
Create a new Jenkins Freestyle project.
Under the Source Code Management section, configure your GitHub project coordinates.
Under Build Triggers section, activate the 'Build when a change is pushed to GitHub' option. That would launch the CI job at the moment you push a new version of the YAML file into the repository.
Under the build section, add an Execute shell build step.
In the shell step, launch the program or script that processes the YAML file/generates the .md files. End the script by adding the git add ., git commit -m "message", git pull and git push commands (assumes git is in the path).
Enable the new job to make it active in Jenkins.
You can do this now with the free GitHub Actions option for the repositories.
You need to put this step into your YAML file.
- name: Commit back to GitHub
run: |
git config --global user.name "github-actiuons[bot]"
git config --global user.email "41898282+github-actions[bot]#users.noreply.github.com"
git add -A
git commit -m "Updating some file"
git push
There are some items in the marketplace, but they didn't work for me.
The email of the bot is based on this thread:
https://github.community/t/github-actions-bot-email-address/17204
Update the commit message.
Be careful with the folder paths if you decide to push a specific file in a folder.
I'm trying to write a job in Jenkins that pulls my latest code from a Github repo, zips it all up, and then pushes it to another builds Github repo. The Publisher event in post-build seems to only push back to the repo you specify in source code management. Is there a way I can push to a different/separate repo?
You can create the zip and than use a bash command with git push ...
Another option is to keep the zip in artifacts repository - Nexus or artifactory ...