I am working on an Android project. I have a few files under version control that git-crypt encrypt them for me. The content of some of them is important to be decrypted before running the build process.
I have no problem at all with building the project on my laptop. The problem is on the the CI side. When I push the code to the Origin, the pipeline (I am using Azure) triggers a pipeline and after a few seconds, the build fails. That's because the build system is looking for the information I have in keys.properties file and it's encrypted.
Based on my understanding, git-crypt uses public/private keys in order to encrypt/decrypt files. The public key is already under the version control, root/.git-crypt/keys/default/0/7F64F6C566153F29E4D9D44070C2A1QW029CE338.gpg. Therefore, I think I need to move the private key somewhere. Am I right? If I am right then my question is where to put it? Is it still under the root of the computer (macOS Image/VM on CI) or I can put it under the root of the project?
Note: Azure lets me upload secure files under its Libraries and then I am able to reference it from the azure-piplines.yaml file.
where to put it?
Based on my understanding on git-crypt and azure devops, compare with storing the private key in root of PC, I'd suggest you put it under the root of project.
Because if you store it at root of PC, it would has no trouble while you run the pipeline with Private agent. BUT, what the issue is if someone using Hosted agent but the private key is in local. As you know, Hosted agent is the one which located at cloud, and it is loaded very dynamically. So, it's hard to get the private key from local computer.
In one word, keeping it in the repository remove the hassle of storing it somewhere else and using it between developers.
However, above is just the suggest choice between store at root of PC and store at root of project.
Although, it would much convenient if put the private key into the root of project.
But on personal suggestion and info security, I strongly stand by the method that you mentioned at last: Store the private key in Library, and reference it during the build/release. Azure DevOps library store the secrets in a safe way and use them as and when needed without making it visible to anyone.
Related
I am using Azure CI/CD pipelines for Flutter build. In my Pubspec yaml file, I have dependencies that are private to my project and the code is hosted in same azure devops project but in different repository. During Build (i.e. Flutter Packages get) it gives me error saying Authentication failed?. I tried with PAT token where in prior to flutter build task i used git command to set that token, but it didn't solve the issue. Can anyone help me out?
I am open to being shown a better way but these are the steps I took to solve this issue a little while ago.
Assuming you are referencing the package in your pubspec.yaml using git over ssh on azure devops like:
repo_name:
git:
ref: 'tag or other identifier'
url: you#vs-ssh.visualstudio.com:v3/you/project/repo_name
Generate a new ssh key pair on your machine.
Upload the private key to the library secure files section on azure devops.
Add the install ssh key task to your azure pipelines build, using the key pair generated in the previous steps and referencing the private key uploaded to the secure files library. link
Upload the public key to your list of public keys. (This step I'm not 100% sure is necessary but I did it initially and things have worked so I haven't changed removed it)
So in my azure-pipelines.yaml the install ssh key step looks kinda like this where id_rsa is the name of the private key in my secure files.
- task: InstallSSHKey#0
inputs:
knownHostsEntry: 'vs-ssh.visualstudio.com, ...etc'
sshPublicKey: 'ssh-rsa ...etc'
sshKeySecureFile: id_rsa
A private feed is created with permissions such that only you have access.
The build agent run with user(build service account), give to this user permissions in the feed. From Feed settings->Permissions, assign your build service account owner permission.
Also verify the token is working, make sure you have selected sufficient scopes for this token to authorize for your specific tasks.
Besides try adding a variable system.debug with a value of true you’ll get more information in the failure. That might help pinpoint the problem.
I got a .net framework solution that has secret settings (API keys, connectionstrings etc) in a web.config.
The solution code lives at github.com.
I want to avoid having those secret settings at github at all.
I got a teamcity build server, that has those settings and merges them into the web.config at build time for the different enviroments, just before deploy time. This works great.
But I need ideas of how to handle those settings for developing locally, where the teamcity server (of course) does not build and merges settings.
Could it maybe be possible to inject some code i global.asax on Application_Start, to fetch the settings from a secure place and set the settings there?
Or maybe have a different config file just for development that is only loaded when developing and is not committed in github?
Or is there some other smart way that I can't think of?
I found a way through powershell and the 'file' reference in web.config and made a Blog post about it: https://codebuildplay.wordpress.com/2018/11/30/how-to-store-settings-in-teamcity-and-merging-into-a-net-config/
The company I work for currently has several projects in TFS 2015, each with their own build definitions.
We are in the process of transitioning from TFS to VSTS.
One of the features of VSTS that we were trying to utilize in the builds is the Secure Files in the projects' Library, to store the certificate issued by our company used for digitally signing assets in different installers.
It is currently duplicated in each project.
We were planning on uploading the certificate to one project and then use the 'Download Secure File' task in all of the projects' builds (that need it), to eliminate the maintenance of having the same certificate in multiple projects.
Not surprisingly, after uploading to one project, the file is not listed in any other project's Library or available for download as part of the build, even if I try to assign the project(s)/team(s) as a security role to the secure file (even giving it Administrator role).
Is there a way to have a secure file in one project's Library be shared across other projects, so that it can be downloaded as part of a build task?
It's not a big secret that the VSTS team has been working towards making individual projects portable. To allow you to take a project, with all that belongs to it, and then move it to another account. No clue when or even if this will ever be released, but it serves as a basis behind some of the separation between separate team projects.
In order to make this seamless, direct links and dependencies between projects are actively being discouraged and old features that are cross-project are slowly disappearing from the UI (even if the API supports it in many cases).
I suspect that if you configure the builds scope to be "Collection" that the REST API will be able to access the secret file from the other project, but it would require a custom task.
The guidance would be to replicate the secret file to each team project that needs access to it.
Does anyone know of a solution (web hosted or otherwise) for a source code control system that would work well in a university environment where information technology is the focus? We'd like to offer it as a campus-wide "version-control service", much like universities do with an email service. Specifically, I'm talking about the following peculiarities:
There are a large number of new repositories created/managed each semester. Any programming course or research project could require students to use source code control, in various source code environments (including .NET, Java, C++, LaTeX).
Students should be able to create and manage themselves the repositories. Involving an administrator/instructor/etc. is not scalable otherwise.
Repository storage should be secure (private), and archivable for respecting intellectual property (preventing plagiarism, protecting research IP).
Any or all of the flavors of source code control (e.g., CVS/SVN/GIT) would be acceptable.
Remote access to repositories is essential. Student/researchers have freedom to work either in designated lab spaces or remotely. Marking of assignments can be done by instructors who've "checked out" the code anywhere.
If an academic license exists, it must scale for >500 students.
Many commercial/free products (web-based or otherwise) don't satisfy conditions #1 and #2, as they require superusers to administer accounts/repositories/accesses. Solutions such as Google Code, sourceforge.net, GitHub, etc. don't satisfy condition #3, as the repositories are always public.
Here's free one: http://gitlabhq.com/
You can add repositories over this tool.
For security you use RSA Keys.
And I would suggest to use Git. SVN and CVS are outdated.
GitHub would appear to satisfy your requirements. You can set up your own instance in your intranet; https://enterprise.github.com/
You could use git in the students private file storage if they have such a thing, git doesn't require hosting other than a place to store files.
Redmine (SVN, CVS, Git, Mercurial, Bazaar and Darcs)
UberSVN (SVN)
Private Assembla (?) (SVN, Git, Mercurial)
One solution I use is to create a master GIT repository in a Truecrypt variable size encrypted container. The container is placed in a Dropbox folder. The repository is cloned to the local hard drive which becomes the working directory. All the work is done and checked in on the local repository. I wrote scripts that mount the encrypted container, pushes/pulls the local repo to the master repo and dismounts the encrypted container. Dropbox detects the changes in the encrypted container and syncs it to the Dropbox server. Security is maintained as an encrypted file is the only thing sent to the server. Only real way to ensure security is doing the encryption yourself.
All you need to set this up is a few scripts, truecrypt installed and a Dropbox account. Could probably write some basic software to automate some of the steps. To make it scaleable and low cost, the basic steps are still valid. Create a master and local repository, encrypt the master repo, work on the local repo and sync changes to the master, back up the encrypted master repo online or on a server.
I develop on my local machine with VS2010 and SQL Server. Naturally, my web.config points to my local SQL Server and I can debug/development and all is well. Unfortunately, I am not entirely sure on how to go about deploying my code to a live server.
Currently, my live server consists of a virtual machine (my site is accessible from the internet). When I'm ready to put my changes on the live server I publish my app (right click on solution explorer -> publish). Then I go to the directory it publishes to and dump all the files into a network share that goes to my site on the live server. On the initial copy over, I have to manually edit the web.config so that the connection string points to the SQL Server on the live server instead of my local machine. So this is my first stumbling block. How can I easily manage development settings and "live" settings in the web.config?
Now, I also use version control (Kiln). Can I possibly tag a changeset and have it automatically deployed to my live server somehow? Let's say someone submits a bug and I fix it. I push my changeset and now Kiln has the latest version of my code with the bug fix. What's the best way to get these changes on to a live server?
I'm unable to find any documentation that covers the entire workflow but I feel like there has go to be a better way. Surely, something like this can be accomplished without having to manually edit the web.config everytime I publish and pray to the computer Gods that I didn't miss something in the connection string.
It's just me so I have complete control over all of my environments, including the server and what's accessible via the internet, and anything is possible if only I knew what to do.
How can I easily manage development settings and "live" settings in the web.config?
Re: With VS 2010 web.config transformations, it is quite easy. Please take a look at this blog:
http://blogs.msdn.com/b/webdevtools/archive/2009/05/04/web-deployment-web-config-transformation.aspx
For VS 2008 or older, we used to have multiple config file based on environment and we used to create Debug/Release/DevTest/UAT/PROD release configuration and then in the post build event we used to replace the web.config with the release configuration based config. For example - if you build the project using "Prod" release configuration then we copy the PROD web.config to the publishing folder.
Now, I also use version control (Kiln).
Can I possibly tag a changeset and have it automatically deployed to my live server somehow? Let's say someone submits a bug and I fix it. I push my changeset and now Kiln has the latest version of my code with the bug fix. What's the best way to get these changes on to a live server?
Re: Source control and publishing to live server are two different things. The first question you are asking here related to how you manage multiple releases and have control over bug fixes for each release. The way I would do it is I will have PROD branch in my source control which will be the first release and for every major release I will sub branch it to have more control over e-fixes.
For the other question about how to get it to live server, it depends on your environment. We do it differently based on how customer environment is setup. If they have given us the FTP, we use that or otherwise we package the application into an MSI and then deploy it to UAT.. Until UAT signoff is done, we keep on updating the MSI. Once signoff received, the MSI goes to PROD.
Hope this helps.