I'm trying to use codeship to automate building docs from a repository.
After the Executing the command mkdocs build --clean I get a path to where my site folder is supposed to be.
INFO - Cleaning site directory
INFO - Building documentation to directory: /home/rof/src/bitbucket.org/josephkobti/test/site
The thing is that I can't find that folder using the ssh console for debugging.
The reason for the folder not existing was a misunderstanding of Codeship's SSH Debug Build feature, documented here https://documentation.codeship.com/basic/builds-and-configuration/ssh-access/
The VMs started for the debug feature are not the actual VMs that run the automated builds. They are new VMs running the same initialization steps as the automated builds (i.e. cloning the repository, configuring project specific environment variables, ...) but none of the actual setup or test commands.
Because of this the mkdocs build --clean command wasn't run either when Joseph connected via to the debug VM, and as such the generated site wasn't available.
Related
I have a .NET Core 2.0 console app. I can successfully build or publish this app and run it locally. I can also successfully build and publish this app in Azure DevOps. However, if I build the app in Azure DevOps, I cannot run the result.
In Azure DevOps, I tried building using:
dotnet build -c Release -r win-x64 -o app
This generates a small number of files with just the project related files. It does not include all of the System.*.dll files, etc that seem excessive for most of my cases. This command works fine when I run it on my local machine and I can successfully click the MyApp.exe file and run my console app. However, if I run the same command on Azure DevOps, the MyApp.exe file that gets generated does not run as expected. Instead, it starts then immediately quits. Nothing is printed in the console app. I see no errors. The app is very basic, includes a "try-catch" around everything and has a Console.ReadLine at the end. So, I thought it would stay open.
When I run:
dotnet publish -c Release -r win-x64 -o app
I get the same files, but with all of the System.*.dll files, etc. included. This time, I've noticed that I can successfully run MyApp.exe and it behaves as expected.
Why does dotnet build ... work locally, but I don't seem to get the same behavior when I run dotnet build ... in Azure DevOps. It seems I'm forced to use dotnet publish. My issue is, the resulting .zip file goes from ~500kb to 30MB. This is big difference.
An answer from the horse's mouth:
The dotnet build command builds the project and its dependencies into
a set of binaries. The binaries include the project's code in
Intermediate Language (IL) files with a .dll extension and symbol
files used for debugging with a .pdb extension. A dependencies JSON
file (*.deps.json) is produced that lists the dependencies of the
application. A *.runtimeconfig.json file is produced, which specifies
the shared runtime and its version for the application.
If the project has third-party dependencies, such as libraries from
NuGet, they're resolved from the NuGet cache and aren't available with
the project's built output. With that in mind, the product of dotnet
build isn't ready to be transferred to another machine to run.
dotnet build - Builds a project and all of its dependencies.
dotnet publish - Packs the application and its dependencies into a folder for deployment to a hosting system. (PS - this also builds the application before packing)
The description is actually very good considering it's coming directly from Microsoft so I will not duplicate the words here.
As an exercise create a solution with multiple projects. For one of the projects add a reference to another project. Add some static files which your code references and a few NuGet packages. And run these commands at the solution root level and at the project level and observe the output in the bin folder.
Commands to run:
dotnet build
dotnet publish
dotnet clean to clean the bin folder
Also, run this at the root level and observe the output with the self-contained flag enabled:
dotnet publish -o ./output --runtime win10-x64 --self-contained
More info on self-contained builds
The different between them is that:
For publish, the necessary assembly files (packages) will be included in build folder and the app uses these assemblies.
But for build, the app references packages that are in the user’s folder. That’s why the zip file just 500 kb.
Since it references packages that are in the user’s folder, so the app needs to be built under the same user’s account, then you can run the app without publishing. So, you need to change build agent’s service account to your account (log on as), then restart the service and queue a new build.
Otherwise, you need to publish the app.
I have an ubuntu staging server where I have installed apache, php, mysql, git, composer installed. I have a private git repository setup on the bitbucket, the project is already cloned to the staging server and to my local development machine. The Laravel setup is working perfectly fine on both machine.
What I am currently doing is whenever there is an update to the git repository, I do login to the staging server, pull the latest code from the git repository and do composer install, npm install, bower install.
I want to automate this process via capistrano tool. I checked the tutorials online, but all of them do the clone of repository whenever, I issue a deploy command and creates a fresh installation every time. Can't capistrano helps me to work on the existing folder that is already setup?
The basic premise of Capistrano is the idea that a new installation is created each time, such that there is not much to be done initially in terms of setup. If you'd rather use a different mechanism, a different tool would work better for you! For such cases, you could try to write a script using SSHKit directly (fairly advanced), or write a makefile or some other tool to automate your process.
If you do want to try to make Capistrano work on its terms, look into how linked_dirs and linked_files work in it. They allow you to have some files (e.g. config files, log dirs, etc) which are outside of the deployment directory and as such are shared between deploys.
I'm trying to use TFS 2015.1 on premise to build a CI pipeline for our dev & uat. I've created a vNext CI build, which builds fine. But when I want to add a deploy step for on prem IIS server, I only then see Azure Web Deployment options.
Ideally I wanted to add a step which uses the existing deploy (MS Deploy) profiles, which I'm able to use from VS2015 directly, using 'Publish'. However I see no option to do so.
How can I deploy the latest build to internal dev servers (not Azure)? I would like to use the MS Deploy option, unless there's a better way of doing it?
The fact that their is no option to starts to make me think there's probably a different way to accomplish it!
Thanks.
If you're able to upgrade to TFS 2015.2, web-based Release Management came out with it that works similarly to Build vNext with flexible and open-source tasks. You can also customize tasks.
Here's a link for IIS Web App Deployment from the vso-agent-task's GitHub repo where Microsoft stores updated versions of their tasks that you can download for web-based Build and Release Management.
I'll be publishing a blog about web-based RM with TFS 2015 Update 2 or VSTS on my website in the next few weeks. To give you an idea though, the starting point (for a web application) is a folder in your web project called WebDeploy (no significance - any name will do) that contains a PowerShell DSC script that configures the server, deploys the web files and then replaces any tokenised configs. To give you an idea see this post about how to use DSC to configure servers. (Only covers part of the final script though!) The next steps are:
In the build hub create a Website artifact - containing your web files and DSC script.
In the release hub for an environment use a Windows Machine File Copy task to deploy the artifact to a temp folder on the target node.
Then use a PowerShell on Target Machines task to execute the DSC script. After configuring the server the script copies the web files to their proper location, sorts out config using xReleaseManagement and cleans up the WebDeploy folder.
See this article for general details of the route I'm taking, but watch out as it has some errors eg the firewall instructions are incomplete (file and print sharing through the firewall needs to be enabled).
I can thoroughly recommend the PowerShell DSC route - I've had a few glitches but on the whole it feels very productive and the right way to be going.
I'm new to Capistrano and struggling a little to get started. A brief description of what I need to do:
git pull the latest code from our git repo, on a central build server. This build server's environment matches the deployment environment exactly. I need the code to be built here. I don't want to deploy a binary that was built on a Mac laptop, for example.
compile the binary on this machine.
deploy it from this machine to all the target machines.
There is a shared user we can all SSH into on the build machine to do the builds.
The build machine is behind a gateway machine, not directly accessible.
All of the deployment target machines also have this shared user and are also behind the gateway.
The deployed binary is a single executable, and there is an init script on the target machines. After deploying the binary and changing the symlink to it, restart the service via the init script.
Everyone has appropriate SSH keys and agent forwarding for all necessary tasks.
So in principle it seems rather simple, but Capistrano seems opinionated and a bit magical. As a result I'm not sure how to accomplish all of this. It seems like it wants to check out my code and copy it to the remote machines, for example without building it first.
I think I need to ignore all of Capistrano's default smarts and just make it run some shell commands on the appropriate servers. In pseudo-code:
ssh buildmachine via gateway "cd repo && git pull && make"
ssh targetmachine(s) via gateway "scp buildmachine:repo/binary .; <mv && symlink>; service foo restart"
Am I even using the right tool for the job? It seems a lot like a round peg in a square hole.
Can someone explain to me what the contents of the Capistrano configuration files should be, and what cap commands I'd run to accomplish this?
BTW, I've searched around and looked at questions like deploying with capistrano with remote git repo but without git running on production server and From manual pull on server to Capistrano
The question is rather old, but you never know when someone steps onto it in need of information...
First and formost, consider that Capistrano might just not be the right tool for the job you want to do.
That said, it is not impossible to accomplish what you expect. While in projects that deploy large amount of files and modify them (like css/js minify, js builds etc.) I would avoid it, in your case, you can consider runing a "deployment repository" and configure it in capistrano as the source. Your process would look like this :
run the local build with whatever tools you need
upload resulting binary to a deployment repository
run capistrano that will connect to application servers, fetch fresh binary from repository, perform any server side tasks required and symlink to "current"
As a side effect you end up with full history of deployed binaries
I have an MVC4 + EF4.0 .NET 4.5 project (say, MyProject) I'm able to run the project locally just fine. When I FTP deploy it to Azure Websites (not cloud service) it runs fine too. However, if I do a GIT deploy, the site 'runs' for the most part until it does some EF5.0 database operations. I get an exception Unable to load the specified metadata resource.
Upon debugging I noticed that if I:
GIT deploy the entire MVC4 project (as before)
FTP in and then replace bin\MyProject.dll with the bin\MyProject.dll file that I just built locally (Windows 8 x64, VS2012, Oct'12 Azure tools) after the GIT push (i.e. same source)
then the Azure hosted website runs just fine (even the EF5.0 database functionality portion).
The locally built .dll is about 5KB larger than the Azure GIT publish built one and both are 'Release' mode. It's obvious that the project as built after the GIT push (inside Azure) is being built differently than as on my own PC. I checked the portal and it's set to .NET 4.5. I'm also GIT pushing the entire solution folder (with just one project) and not just small bits and pieces.
When I load the locally built as well as the remotely built MyProject.dll files, I noticed the following difference(FrameworkDisplayName)
local: System.Runtime.Versioning.TargetFrameworkAttribute(".NETFramework,Version=v4.5", FrameworkDisplayName = ".NET Framework 4.5"),
remote: System.Runtime.Versioning.TargetFrameworkAttribute(".NETFramework,Version=v4.5", FrameworkDisplayName = ""),
Anyone knows why this is happening and what the fix might be?
Yes, this is a bug that will be fixed in the next release. The good news is that it's possible to work around it today:
First, you need to use a custom deployment script, per this post.
Then you need to change the MSBuild command line in the custom script per this issue.
Credit goes to David above for the pointers and hints. I voted him up but I'll also post the exact solution to the issue here. I've edited my original post because I found there was a major bug that I didn't notice until I started from scratch (moved GIT servers). So here is the entire process, worked for me.
Download Node.JS (it's needed even for .NET projects because the GIT deploy tools use it)
Install the azure-cli tool (open regular command prompt => npm install azure-cli -g)
In the command prompt, cd to the root of your repository (cd \projects\MyRepoRoot)
In there, type azure site deploymentscript --aspWAP PathToMyProject\MyProject.csproj -s PathToMySolution.sln (obviously adjust the paths as needed)
This will create the .deployment and deploy.cmd files
Now edit the deploy.cmd file, find the line starting with %MSBUILD_PATH% (will be just one)
Insert the /t:Build parameter. For example:
[Before] %MSBUILD_PATH% <blah blah> /verbosity:m /t:pipelinePreDeployCopyAllFilesToOneFolder
[After] %MSBUILD_PATH% <blah blah> /verbosity:m /t:Build /t:pipelinePreDeployCopyAllFilesToOneFolder)
Push to GIT (check the GIT output if everything went ok)
Browse to your website and confirm it works!
I'll be glad when it's fixed in the next revision so we won't maintain the build script