I have an Azure web app. This web app has a QA deployment slot for pre-production testing. When I check in my code from VS, I have it setup to build and deploy to the QA deployment slot. This works great. However, a few configurations need to be updated in the QA web app so the application points to the correct service endpoints (i.e. not dev). To do this, my initial approach was to add a PS task to the Release that unzips my deployment zip, updates the configuration files, rezips them and then allows the Release flow to deploy the updated zip. This works locally, but running into filename length issues on the server when unzipping, which I can't change.
Now I'm trying to just include my update PS scripts in my deployment package, and then run the scripts AFTER the deployment has occurred. So, I'm looking at this Powershell on Target Machines task to run a PS on the QA slot server to update configurations. However, it's asking for Machines, which would be the server name of the slot server. I don't have that. I also don't know where to get it. I also don't have the path to the PS scripts once I have the server name. I dumped out the server variables and none of them help me, unless there is a cmdlet to look up environments that I'm not aware of.
System.DefaultWorkingDirectory: 'C:\a\2ed23b64d'
System.TeamFoundationServerUri: 'https://REDACTED.vsrm.visualstudio.com/DefaultCollection/'
System.TeamFoundationCollectionUri: 'https://REDACTEDvisualstudio.com/DefaultCollection/'
System.TeamProject: 'REDACTED'
System.TeamProjectId: 'REDACTED'
Release.DefinitionName: 'REDACTED'
Release.EnvironmentUri: 'vstfs:///ReleaseManagement/Environment/46'
Release.EnvironmentName: 'QA'
Release.ReleaseDescription: 'Triggered by REDACTED Build Definition 20160425.4.'
Release.ReleaseId: '31'
Release.ReleaseName: 'Release-31'
Release.ReleaseUri: 'vstfs:///ReleaseManagement/Release/31'
Release.RequestedFor: 'Matthew Mulhearn'
Release.RequestedForId: ''
Agent.HomeDirectory: 'C:\LR\MMS\Services\Mms\TaskAgentProvisioner\Tools\agents\1.98.1'
Agent.JobName: 'Release'
Agent.MachineName: 'TASKAGENT5-0020'
Agent.Name: 'Hosted Agent'
Agent.RootDirectory: 'C:\a'
Agent.WorkingDirectory: 'C:\a\SourceRootMapping\REDACTED'
Agent.ReleaseDirectory: 'C:\a\2ed23b64d'
Anyone have any idea, or a better approach, to accomplish what I'm attempting?
Related
I have a pipeline deploying to my Azure web app, that most of the times errors out because it couldn't deploy to my web app. The task take around 25 mins :
...
Copying file: 'frontend/.gitignore'
Copying file: 'frontend/README.md'
Copying file: 'frontend/package.json'
Copying file: 'frontend/tsconfig.json'
Copying file: 'frontend/yarn.lock'
Omitting next output lines...
An error has occurred during web site deployment.
Kudu Sync failed
\n/opt/Kudu/Scripts/starter.sh "/home/site/deployments/tools/deploy.sh"
##[error]Failed to deploy web package to App Service.
##[error]To debug further please check Kudu stack trace URL : https://$someapp:***#someapp.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace
##[error]Error: Package deployment using ZIP Deploy failed. Refer logs for more details.
...
When i enable : system.debug = true , i see these logs repeated many time , before start copying the artifact files :
POLL URL RESULT: {"statusCode":202,"statusMessage":"Accepted","headers":{"transfer-encoding":"chunked","content-type":"application/json; charset=utf-8","location":"http://XXXXXXXXX.scm.azurewebsites.net:80/api/deployments/latest?deployer=VSTS_ZIP_DEPLOY&time=2021-07-09_09-01-41Z","server":"Kestrel","date":"Fri, 09 Jul 2021 09:23:37 GMT","connection":"close"},"body":{"id":"68a7a8811796416b993924437493ff87","status":0,"status_text":"Building and Deploying '68a7a8811796416b993924437493ff87'.","author_email":"N/A","author":"N/A","deployer":"VSTS_ZIP_DEPLOY","message":"Created via a push deployment","progress":"Running deployment command...","received_time":"2021-07-09T09:01:50.4159225Z","start_time":"2021-07-09T09:01:51.775357Z","end_time":null,"last_success_end_time":null,"complete":false,"active":false,"is_temp":false,"is_readonly":true,"url":null,"log_url":null,"site_name":"XXXXXXXXXXXXe"}}
Deployment status: 0 'Building and Deploying '68a7a8811796416b993924437493ff87'.'. retry after 5 seconds
setting affinity cookie ["ARRAffinity=c06e9bb74f52245b3695b3079a52f6acbc70c3ee812f67e4fa3f5f65088ff4f7;Path=/;HttpOnly;Secure;Domain=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX.scm.azurewebsites.net","ARRAffinitySameSite=c06e9bb74f52245b3695b3079a52f6acbc70c3ee812f67e4fa3f5f65088ff4f7;Path=/;HttpOnly;SameSite=None;Secure;Domain=XXXXXXXXXXXXXXX.scm.azurewebsites.net"]
[GET]https://XXXXXXXXXXX-test.scm.azurewebsites.net:443/api/deployments/latest?deployer=VSTS_ZIP_DEPLOY&time=2021-07-09_09-01-41Z
POLL URL RESULT: {"statusCode":202,"statusMessage":"Accepted","headers":{"transfer-encoding":"chunked","content-type":"application/json; charset=utf-8","location":"http://XXXXXXXXXXXXXXXXXXX.scm.azurewebsites.net:80/api/deployments/latest?deployer=VSTS_ZIP_DEPLOY&time=2021-07-09_09-01-41Z","server":"Kestrel","date":"Fri, 09 Jul 2021 09:23:45 GMT","connection":"close"},"body":{"id":"68a7a8811796416b993924437493ff87","status":0,"status_text":"Building and Deploying '68a7a8811796416b993924437493ff87'.","author_email":"N/A","author":"N/A","deployer":"VSTS_ZIP_DEPLOY","message":"Created via a push deployment","progress":"Running deployment command...","received_time":"2021-07-09T09:01:50.4159225Z","start_time":"2021-07-09T09:01:51.775357Z","end_time":null,"last_success_end_time":null,"complete":false,"active":false,"is_temp":false,"is_readonly":true,"url":null,"log_url":null,"site_name":"XXXXXXXXXXXX"}}
Deployment status: 0 'Building and Deploying '68a7a8811796416b993924437493ff87'.'. retry after 5 seconds
setting affinity cookie ["ARRAffinity=c06e9bb74f52245b3695b3079a52f6acbc70c3ee812f67e4fa3f5f65088ff4f7;Path=/;HttpOnly;Secure;Domain=XXXXXXXXXXXXXXXXXXXXXX.scm.azurewebsites.net","ARRAffinitySameSite=c06e9bb74f52245b3695b3079a52f6acbc70c3ee812f67e4fa3f5f65088ff4f7;Path=/;HttpOnly;SameSite=None;Secure;Domain=XXXXXXXXXXXXXXXXXX"]
This task fails only in specific slot in myweb app , authors slots and production slot works fine and the job take around 6 mins
Any ideas what could be wrong?
As per the discussion and troubleshooting performed here, I tried to setup a Linux App Service on Standard S1 pricing tier enabling 5 (max) slots with CI/CD configured via Azure Pipelines. Unfortunately, I wasn't able to reproduce the same error as yours despite multiple different trials.
I'd suggest you to try the following:
Kudu Sync failed in the deployment log resembles this open issue from about a year ago: ZipDelpoy on azure web app linux fails during kudu sync #2972. Please check the trace/deployment log files on kudu at https://<appname>.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace or /deployment or from Kudu's DebugConsole (/LogFiles/kudu/\*) and check if this is caused by deployment lock failures. In that case, check this wiki out for dealing with locked files during deployment.
Try a different deployment method like run from package (to avoid resource locking), using FTP/S, or local git deployment.
This should help you narrow down the issue further, whether it is caused in the App service/deployment method, or the ADO pipeline/task.
Scale up to the next higher tier and re-trigger your pipeline. If it succeeds, you may scale back down to the original tier. This would indirectly restart your SCM sites as well.
If the above workarounds don't help, you could check on the following:
Customize your deploy task with options like TakeAppOfflineFlag, DeploymentType or RenameFilesFlag to streamline your deployment.
Try restarting the app/slot just before the deployment in order to recycle the app pool.
Check if your app is running into any of the prescribed limits (ex: file system storage) for your tier.
Drill down into available metrics for your app to identify any CPU/Memory anomalies.
Try the Diagnose and solve problems tool for any additional insights about your app.
If your environment permits, try setting up and deploying to a new slot within your App Service, or try verifying if this happens to another app in a different region.
I'm unable to run the Docker Compose task on Azure DevOps and every solution I've looked up online, either makes no sense, or does not work for my scenario.
The job output for the failure is:
This is a very simple process, artifacts are copied to a folder during build, and the docker-compose.yml and .dockerfile is added to this directory, which then needs to be run.
One article explained that if you add your docker-compose.yml to the same folder as the files the image will be hosting and the .dockerfile, that it might cause the daemon to fall over and generate this generic error, so I've added a .dockerignore file, but this issues persists.
I'm using a Hosted Agent - Ubuntu-18.04.
My task looks like this:
steps:
- task: DockerCompose#0
displayName: 'Run a Docker Compose command'
inputs:
azureSubscription: 'Test Dev Ops'
azureContainerRegistry: '{"loginServer":"testdevops.azurecr.io", "id" : "/subscriptions/{subscription_key}/resourceGroups/Test.Devops/providers/Microsoft.ContainerRegistry/registries/testdevops"}'
dockerComposeFile: '$(System.DefaultWorkingDirectory)/$(Release.PrimaryArtifactSourceAlias)/test.ng.$(Build.BuildNumber)/dist/testweb/docker-compose-build.yml'
dockerComposeCommand: build
arguments: '--build-arg azure_pat=$(System.AccessToken) --build-arg azure_username=Azure'
The idea here is that this container is composed and delivered straight to Azure's Container Registry.
I have ensured that the user that's running this process, as been granted permissions in that ACR, as well as added the user to the Administrative group in Azure DevOps.
A lot of responses talks about adding the user to the Docker group, but this is a Hosted Agent, not a private agent, so there is no such option.
I have even tried installing Docker CLI before this task, but nothings working.
Am I being daft to think that I can compose in Azure DevOps?
Edit
The contents of my artifacts folder looks something like this:
This error message is extremely misleading. If anyone from Microsoft is looking at this question, please consider making the error more specific, if possible.
It turned out, I missed a semi-colon in a build task that replaced tokens before the build artifacts was pushed from the build output, and because of that, the yaml file had a #{..} token inside of it, which caused the docker-compose to fail.
It had nothing to do with permissions, nor a .dockerignore file, very misleading.
If I understand correctly the standard git deploy implementation with capistrano v3 deploys the same repository on all roles. I have a more difficult app that has several types of servers and each type has its own code base with its own repository. My database server for example does not need to deploy any code.
How do I tackle such a problem in capistrano v3?
Should I write my own deployment tasks for each of the roles?
How do I tackle such a problem in capistrano v3?
All servers get the code, as in certain environments the code is needed to perform some actions. For example in a typical setup the web server needs your static assets, the app server needs your code to serve the app, and the db server needs your code to run migrations.
If that's not true in your environment and you don't want the code on the servers in some roles, you could easily send a pull request to add the no_release feature back from Cap2 in to Cap3.
You can of course take the .rake files out of the Gem, and load those in your Capfile, which is a perfectly valid way to use the tool, and modify them for your own needs.
The general approach is that if you don't need code on your DB server, for example, why is it listed in your deployment file?
I can confirm you can use no_release: true to disable a server from deploying the repository code.
I needed to do this so I could specifically run a restart task for a different server.
Be sure to give your server a role so that you can target it. There is a handy function called release_roles() you can use to target servers that have your repository code.
Then you can separate any tasks (like my restart) to be independent from the deploy procedure.
For Example:
server '10.10.10.10', port: 22, user: 'deploy', roles: %w{web app db assets}
server '10.10.10.20', port: 22, user: 'deploy', roles: %w{frontend}, no_release: true
namespace :nginx do
desc 'Reloading PHP will clear OpCache. Remove Nginx Cache files to force regeneration.'
task :reload do
on roles(:frontend) do
execute "sudo /usr/sbin/service php7.1-fpm reload"
execute "sudo /usr/bin/find /var/run/nginx-cache -type f -delete"
end
end
end
after 'deploy:finished', 'nginx:reload'
after 'deploy:rollback', 'nginx:reload'
# Example of a task for release_roles() only
desc 'Update composer'
task :update do
on release_roles(:all) do
execute "cd #{release_path} && composer update"
end
end
before 'deploy:publishing', 'composer:update'
I can think of many scenarios where this would come in handy.
FYI, this link has more useful examples:
https://capistranorb.com/documentation/advanced-features/property-filtering/
I consider to use Capistrano to deploy my rails app on my server. Currently I'm using a script, which does all the work for me. But Capistrano looks pretty nice and I want to give it a try.
My first problem/question now is: How to use Capistrano properly in open source projects? I don't want to publish my deploy.rb for several reasons:
It contains sensible informations about my server. I don't want to publish them :)
It contains the config for MY server. For other people, which deploy that open source project to their own server, the configuration may differ. So it's pretty senseless to publish my configuration, because it's useless for other people.
Second problem/question: How do I manage different environments?
Background: On my server I provide two different environments for my application: The stable system using the current stable release branch and located under www.domain.com. And a integration environment for the develop team under dev.domain.com running the master branch.
How do I tell Capistrano to deploy the stable system or the dev system?
The way I handle sensitive information (passwords etc.) in Capistrano is the same way I handle them in general: I use an APP_CONFIG hash that comes from a YAML file that isn't checked into version control. This is a classic technique that's covered e.g. in RailsCast #226, or see this StackOverflow question.
There are a few things you have to do a little differently when using this approach with Capistrano:
Normally APP_CONFIG is loaded from your config/application.rb (so it happens early enough to be usable everywhere else); but Capistrano cap tasks won't load that file. But you can just load it from config/deploy.rb too; here's the top of a contrived config/deploy.rb file using an HTTP repository that requires a username/password.
require 'bundler/capistrano'
APP_CONFIG = YAML.load_file("config/app_config.yml")
set :repo_user, APP_CONFIG['repo_user']
set :repo_password, APP_CONFIG['repo_password']
set :repository, "http://#{repo_user}:#{repo_password}#hostname/repositoryname.git/"
set :scm, :git
# ...
The config/app_config.yml file is not checked into version control (put that path in your .gitignore or similar); I normally check in a config/app_config.yml.sample that shows the parameters that need to be configured:
repo_user: 'usernamehere'
repo_password: 'passwordhere'
If you're using the APP_CONFIG for your application, it probably needs to have different values on your different deploy hosts. So have your Capistrano setup make a symlink from the shared/ directory to each release after it's checked out. You want to do this early in the deploy process, because applying migrations might need a database password. So in your config/deploy.rb put this:
after 'deploy:update_code', 'deploy:symlink_app_config'
namespace :deploy do
desc "Symlinks the app_config.yml"
task :symlink_app_config, :roles => [:web, :app, :db] do
run "ln -nfs #{deploy_to}/shared/config/app_config.yml #{release_path}/config/app_config.yml"
end
end
Now, for the second part of your question (about deploying to multiple hosts), you should configure separate Capistrano "stages" for each host. You put everything that's common across all stages in your config/deploy.rb file, and then you put everything that's unique to each stage into config/deploy/[stagename].rb files. You'll have a section in config/deploy.rb that defines the stages:
# Capistrano settings
require 'bundler/capistrano'
require 'capistrano/ext/multistage'
set :stages, %w(preproduction production)
set :default_stage, 'preproduction'
(You can call the stages whatever you want; the Capistrano stage name is separate from the Rails environment name, so the stage doesn't have to be called "production".) Now when you use the cap command, insert the stage name between cap and the target name, e.g.:
$ cap preproduction deploy #deploys to the 'preproduction' environment
$ cap production deploy #deploys to the 'production' environment
$ cap deploy #deploys to whatever you defined as the default
Are there any simple ways to see what wars have been deployed in Jboss AS and undeploy some of them? I want to do this from the command line.
I tried using jmx but I keep getting exception saying "org.jboss.util.NestedRuntimeException: jmx not bound;" I'd prefer to do it in a way where I don't have to stop and start the application server.
You can approach this in three ways in JBoss Application Server 7:
Management Console
Management CLI
Deployment folder
Management Console
Being a GUI, the Management Console is the most visual of the three, and you can see the list of deployed applications under the Deployment window. You have the option to Disable and Remove the deployed applications listed here. This screenshot is from AS7.0.2, and some windows have and will change with the addition of tabs, but the general functionality remains the same.
Management CLI
The Management Command Line Interface is a new addition to AS7. The CLI exposes a lot of low-level functionality, and is a powerful tool once you get familiar with the commands and operations. As you might expect, you can run help to show the commands, or run <commandname> --help for more information on a specific command. Two useful commands are deploy and undeploy, so let's look at their help information. I'll give Linux examples, but you can insert your flavour of OS as required.
Here's deploy:
[standalone#localhost:9999 /] deploy --help
SYNOPSIS
deploy (file_path [--name=deployment_name] [--runtime_name=deployment_runtime_name] [--force] | --name=deployment_name) [--server-groups=group_name (,group_name)* | --all-server-groups]
DESCRIPTION
Deploys the application designated by the file_path or enables an already existing
but disabled in the repository deployment designated by the name argument.
If executed w/o arguments, will list all the existing deployments.
ARGUMENTS
file_path - the path to the application to deploy. Required in case the deployment
doesn't exist in the repository.
The path can be either absolute or relative to the current directory.
--name - the unique name of the deployment. If the file path argument is specified
the name argument is optional with the file name been the default value.
If the file path argument isn't specified then the command is supposed to
enable an already existing but disabled deployment, and in this case the
name argument is required.
--runtime_name - optional, the runtime name for the deployment.
--force - if the deployment with the specified name already exists, by default,
deploy will be aborted and the corresponding message will printed.
Switch --force (or -f) will force the replacement of the existing deployment
with the one specified in the command arguments.
--server-groups - comma separated list of server group names the deploy command should apply to.
Either server-groups or all-server-groups is required in the domain mode.
This argument is not applicable in the standalone mode.
--all-server-groups - indicates that deploy should apply to all the available server groups.
Either server-groups or all-server-groups is required in domain mode.
This argument is not applicable in the standalone mode.
-l - in case none of the required arguments is specified the command will
print all of the existing deployments in the repository. The presence of the -l switch
will make the existing deployments printed one deployment per line, instead of
in columns (the default).
And here's undeploy:
[standalone#localhost:9999 /] undeploy --help
SYNOPSIS
undeploy name [--server-groups=group_name (,group_name)* | --all-relevant-server-groups] [--keep-content]
DESCRIPTION
Undeploys the deployment with the given name and, depending on the arguments, removes
its content from the repository.
If the deployment name isn't specified, prints the list of all the existing deployments.
ARGUMENTS
name - the name of the deployment to undeploy.
--server-groups - comma separated list of server group names the undeploy command should apply to.
Either server-groups or all-relevant-server-groups is required in the domain mode.
This argument is not applicable in the standalone mode.
--all-relevant-server-groups - indicates that undeploy should apply to all the server groups
in which the deployment is enabled.
Either server-groups or all-relevant-server-groups is required in domain mode.
This argument is not applicable in the standalone mode.
--keep-content - by default undeploy, besides disabling the deployment, also removes its
content from the repository. The presence of --keep-content will only disable
the deployment w/o removing its content from the repository.
This argument can be used in both standalone and domain modes.
-l - in case the deployment name isn't specified, the presence of the -l switch
will make the existing deployments printed one deployment per line, instead of
in columns (the default).
The CLI In Action
Running the deploy or undeploy command without any arguments will list all the applications available. So your workflow to log in to the CLI and undeploy an application would be like this (simplified):
Change directory from EAP_HOME to the bin folder:
[user#home EAP_HOME]$ cd bin
Run the CLI logon script:
[user#host bin]$ ./jboss-admin.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
Instruct the API to connect (this can also be passed at runtime as ./jboss-admin.sh --connect).
[disconnected /] connect
Connected to standalone controller at localhost:9999
Run the undeploy command to display available applications.
[standalone#localhost:9999 /] undeploy
test.ear
Run the undeploy command to undeploy the application. In this case, the test.ear.
[standalone#localhost:9999 /] undeploy test.ear
Successfully undeployed test.ear.
Deployment Folder
If you are running an instance of AS7 as a development tool, and are using the deployment folder, you can simply delete the application. You will notice that a marker file is created, such as test.ear.failed, to signify the status of the application deployment.
With JBoss Application Server it's possible handle deployment with plain file operations on wars/ears in deploy dir.
Just ls to enumerate, delete to undeploy, copy to deploy and touch to redeploy. No server restart required.
Also for the newest version of JBoss Application Server 7 you can use CLI.
You can also try CLI GUI. The Deployments-->Undeploy menu option provides a pick list of deployments. A couple of clicks and you are done.
See https://community.jboss.org/wiki/AGUIForTheCommandLineInterface
You dont have to restart the JBOSS server if you remove the WAR and EAR the application gets undeployed .
The article (http://docs.jboss.org/jbossas/docs/Clustering_Guide/4/html/clustering-intro-farm.html ) gives the perfect explanation of how it works.
Let all know if you have any questions about it .
We have been doing the development on Jboss the same way in our DEV , QA and Prod environment and have not faced any issues with it so-far.