Is there a way to download all files with a specific name from the master branches of all the projects in an Azure DevOps installation?
I have been tasked with documenting all of the entries in all of the appsettings.json files in our entire codebase and I would prefer to not have to go through all 300 repositories to manually download these files if I don't absolutely have to.
Is there a way to download all files with a specific name from the
master branches of all the projects in an Azure DevOps installation?
This is not supported. Please check this document, we recommend different project(with one or more repos) for different products/sub-modules of big product. So in Azure Devops Service there's no such out-of-box feature to find/download files across projects.
A possible direction:
If those appsettings.json files are in root directory of your repos. You may save some time by using these two Rest APIs:
List all repos in current organization:
GET https://dev.azure.com/{OrganizationName}/_apis/git/repositories?api-version=5.1
Get File(Download):
GET https://dev.azure.com/{OrganizationName}/_apis/git/repositories/{Repositoryid}/items?scopePath=/appsettings.json&download=true&api-version=5.1
You can use PowerShell script to combine these two apis. The first one will list all Repositoryids in your organization, and the second one can download the appsettings.json from different repos via different Repositoryids. So the possible way could be run the first api once to get list of reposID(You can check this similar one) and then add a loop to get the files one by one.
Related
We have an Azure DevOps Pipeline defined as a template in a .yml file, and would like to use this to build 100+ projects.
There is zero parameterization to do, and the template essentially just runs against the project repo as-is.
Normally, we would just go into each of those project repos and create an azure-pipelines.yml file and reference the template pipeline, easy.
However, I would like to know if we can avoid creating these azure-pipelines.yml in each of the 100+ project repos. Is there some way to just run the template against 100+ repos, but as separate pipelines?
No.
YAML pipelines are based upon the YAML being stored in the application's repo, as code, alongside the application code. You can pull in templates from other repos, but you can't have one pipeline that is automatically valid for any repo.
Trying to use one template to build 100+ applications without the possibility of pinning the template to a specific version on an application by application basis is setting yourself up for a nightmare at some point in the future, when the template changes in such a way that it's no longer valid for a subset of your 100 applications.
I would like to schedule (for my company) a backup of our most important data in Azure DevOps, and that, for different reasons : security, urgent recovery required, virus, migration, etc...
I can execute a backup of the repositories and the Wikis (because it's under GIT, so easy to download), but how can do a backup of the "Board" section (Backlogs, Work items, etc...), and the build pipelines definitions?
How to backup the data on Azure Devops?
In current Azure DevOps, there is no out of the box solution to this. You could manually save the project data through below ways:
Source code and custom build templates: You can download your files
as a zip file. Open ... Repository actions actions for the
repository, file, or folder and choose Download as Zip. You can
also Download from the right side of the screen to download
either all of the files in the currently selected folder, or the
currently selected file.
This process doesn't save any change history or links to other
artifacts.
If you use Git, clone your repositories to retain the full project
history and all the branches.
Build data: To save logs and data in your drop build folders, see
View build results.
Work item tracking data: Create a work item query and open it using
Excel. Save the Excel spreadsheet.
This process doesn't save any attachments, change history, or links
to other artifacts.
build/release defintions: you could export the json file for them and then import them when restoring them.
There has been a related user voice, you could monitor and vote up it: https://developercommunity.visualstudio.com/content/idea/365441/provide-a-backup-service-for-visual-studio-team-se.html.
Here are some tickets(ticket1 ,ticket2) with the same issue you can refer to.
If you want to create scheduled tasks, you can write a script by using the Azure CLI with the Azure Devops Extension
As you said, for the repositories, it's quiet easy as they are Git repositories.
I wrote such a script that we could improve to also backup the Workitems, Backlog, etc...
It's open source, let me know what you would like to backup first and I'll improve it.
Github : azure-devops-repository-backup
Simply speaking, does Azure Devops have something that works very close to GitHub releases?
I would like to publish artifacts that are created during an Azure DevOps pipeline so that they be easily viewed and downloaded afterwards in a central location. The closest equivalent to what I'm looking for is how GitHub releases work, where there is a web page listing out all the versions of the repository and the assets that can be downloaded for each version.
It seems to me that published artifacts within Azure DevOps pipelines are always tied to the run of the pipeline, and there isn't an easy way to see one list of artifacts that have been created in a historical view like GitHub releases provides, but maybe I'm missing something.
Azure Artifacts does not meet my needs because it is tied to particular packaging formats and is meant to be used for developer tooling.
I would like to publish artifacts that are created during an Azure
DevOps pipeline so that they be easily viewed and downloaded
afterwards in a central location.
As a workaround ,you can switch Artifact publish location to A file share in Publish Pipeline Artifacts task then specify your network drive folder path.
Specifying the path to the file share where you want to copy the files. The path must be a fully-qualified path or a valid path relative to the root directory of your repository. Publishing artifacts from a Linux or macOS agent to a file share is not supported.
Besides, there should be no other built-in hosting, I am afraid that Azure Artifact is the closest to your needs. because it is tied to particular packaging formats you can consider using Universal Packages.
Anybody know if it is possible to pass in a repo name / base the build on a dynamic repo name? This would allow us to share the same build definition across different branches, cutting down on definitions when creating a feature branch, etc.
When using a TFVC repo we would store the different releases in the same repo but different paths. We could reuse the same build definition across different releases/FB's by altering the source path such as $/product/$(release)/......
It appears Git likes to have the repo hard-coded into the build (hence the dropdown - no way to plug in a variable.
While the question is targeted to On-prem Azure DevOps, if it is possible in the hosted environment it would be helpful to know.
I recommend using YAML build templates. By default these check out "self" and are stored in the repo. That way they work on forks, branches etc. Each branch can contain tweaks to the build process as well.
With the 'old' UI based builds this isn't possible.
What you are looking for is actually two things:
templates - this allows you reuse definition accross different pipelines
triggers - this allows you to trigger pipeline when commit happens on different branches
Looks like Task Groups solved the need (mostly). I was hoping to have one build definition that could be shared across multiple branches; while this appears to be possible on the hosted model, on prem is different.
I am able to clone a build (or use templates) to have an entry point into the repo/branch to get the sources, then pass off the work to a common task group. If I need to modify the build process for multiple branches, just modify the task group.
I have got a project in an old org (from VSTS), that I want to move to my new one.
I can't see any options in Azure DevOps on migrating projects, or any information on the interwebs.
Anyone know how to do it?
If you just need to move repos, you can use the built in clone functions:
Go to the Azure Devops source repo -> Files
Click "Clone"
Choose "Generate Git Credentials"
Create the target repo in the target Azure DevOps
Choose "Import a repository"
Use the URL and credentials from Step 3
Done
This is not supported today. But this feature was planned to develop: make it possible to move a Team Project between Team Project Collections
If your Azure Devops project only tracks code versions using a single Git repo, hence no boards, user stories, tasks, pipelines, etc. then you can do the following:
Clone your project repo.
For example with Visual Studio.
You don't need to clone if you already have a local repo.
Destroy the association with the remote.
For this typically, you need to open a command line prompt in the folder that contains the .git database folder, most likely the solution folder of Visual Studio and type git remote rm origin.
Here is an example using git bash showing the content of the solution folder, including the .git database and the *.sln Visual Studio solution file:
Open the solution with Visual Studio if not already done.
It should now show that you have many commits waiting to be pushed to a remote. For illustration purpose, my toy project only have 8 commits in total.
Click the up arrow and choose your new remote, say a brand new Azure DevOps project, in the organization of your choice, then push.
You are now done cloning the project in another organization. If needed, then destroy the project in the old organization to complete the "move" operation.
There are 3 projects that I know of to achieve this.
A paid for option by Ops Hub -
OpsHub Visual Studio Migration Utility
An open source tool that requires making changes to the work item process template - Azure DevOps Migration tools
An lastly an Unofficial but still written by Microsoft tool to create Azure DevOps project templates - Azure DevOps Demo Generator & extractor tool
With the last one (the Demo Generator) you extract the project as a template, then apply it to the new organisation. As it is a tool for demo's there is no support provided and in my experience it works for simple projects but falls over on anything complex.
Expanding on others' answers, this post regards Pipelines.
Azure DevOps API
Migrating nearly all aspects of a project across organizations is doable, but it is a lot of manual work using the Azure DevOps API. The link below shows you all the end points, variables, etc. From there you'll probably want to write a Power Shell script and do a couple test runs to a dummy Organization.
https://learn.microsoft.com/en-us/rest/api/azure/devops/?view=azure-devops-rest-6.1&viewFallbackFrom=azure-devops-rest-6.0
In App options
If you avoid the API, there is no way to migrate pipelines that preserve build or release history, but you can preserve your configurations and processes by going into your Pipelines and selecting View YAML. From here you can either take this away as notes to recreate the GUI steps in your new org/project location, or actually adopt the YAML standard in your git repository.
I do not believe there is a way to migrate pipeline variables outside of the API. However, you can move the variables to Azure Key Vault and change your pipeline settings (YAML) to reference values from key vault. This is not a large amount of effort and is a nice process improvement.
Lastly, if you have any locally installed pipeline agents for releases, you will need to run the Power Shell script for your new organization on the boxes. Very simple 5m step, but right now the Agent Pools are not sharable across organization.
As #Frederic mentioned in his answer, we can actually easily do it with Visual Studio. I have done this without Visual Studio. The steps involved are below.
Add a User to Both Organization
Configure SSH Key
Update the SSH Key in Source DevOps and Clone the Repository
Check out all the Branches and Tags
Update the SSH Key in the Destination DevOps
Remove Old and Add New Origin
Push all the branches
The commands and detailed explanations can be found here.
BTW, if you need to change the entire Devops Organization tied to your personal Tenant (E.g. VS Enterprise Subscription) and move it to new Tenant, you can change the AAD and point it to the new one e.g. your EA Tenant on Azure commercial cloud.
Before you switch your organization directory, make sure the following statements are true:
You're in the Project Collection Administrator group for the
organization.
You're a member or a guest in the source Azure AD and a
member in the destination Azure AD
You have 100 or fewer users in
your source organization. Otherwise you will have to open a support ticket.
You may have to add the users back in destination org if they do not exist becuase they will loose access the moment you switch the AAD.
you could just download as a zip file and then download it to the destination repo