GitLab CI - Project Build In Neverending Pending-State - deployment

I'm in some trouble with GitLab CI.
I followed offical guide on:
https://github.com/gitlabhq/gitlab-ci/blob/master/doc/installation.md
Everything was ok, no errors nowhere. I followed Runner-Setup, too.
Anything alright.
But...
When I add a runner to a project and then try to build nothing happens.
It could be that I have not fully understood something or some of my configs are wrong.
I'm absolutely new to GitLab CI, but I like it and I want to learn new stuff.
I would be very very glad if someone could help me in some way.
Thanks!
BIG UPDATE:
Just figured out that:
~/gitlab-runners/gitlab-ci-runner$ bin/runner
Starting a runner process manually solves the problem but if I look at the gitlab-ci-runner in /etc/init.d -> it is running !?!
~/gitlab-runners/gitlab-ci-runner$ sudo /etc/init.d/gitlab-ci-runner start
Number of registered runners in PID file=1
Number of running runners=0
Error! GitLab CI runner(s) (gitlab-ci-runner) appear to be running already! Try stopping them first. Exiting.
~/gitlab-runners/gitlab-ci-runner$ sudo /etc/init.d/gitlab-ci-runner stop
Number of registered runners in PID file=1
Number of running runners=0
WARNING: Numbers of registered runners don't match number of running runners. Will try to stop them all
Registered runners=1
Running runners=0
Trying to stop registered runners...kill: No such process
OK
Trying to kill ghost runners...OK
What's wrong here? I'm out of my power or not seeing the problem?!

Problem solved!
You need to edit some values in /etc/init.d/gitlab-ci-runner script!
APP_ROOT="**PATH_TO**/gitlab-runners/gitlab-ci-runner"
APP_USER="**USER_WITH_DIRRIGHTS!**"
PID_PATH="$APP_ROOT/tmp/pids"
PROCESS_NAME="ruby ./bin/runner"
RUNNERS_PID="$PID_PATH/runners.pid"
RUNNERS_NUM=1 # number of runners to spawn
START_RUNNER="nohup bundle exec ./bin/runner"
Now it works!

In my case tags in the runner were different from tags in the .gitlab-ci.yml. Once I changed them so runner tags include all of the config file tests, tasks began to run.

Related

Azure Pipelines - "Label sources" fails without any useful diagnostics

Please, observe:
[Error]Failed to create ref refs/tags/dryrun-master-CI_64.0.0.25914-noat-test at 330dd52a89ed97f5dcd216bcf89e04b864247053.
[Error]Failed to create ref refs/tags/dryrun-master-CI_64.0.0.25914-noat-test at 330dd52a89ed97f5dcd216bcf89e04b864247053.
Created ref refs/tags/dryrun-master-CI_64.0.0.25914-noat-test at 330dd52a89ed97f5dcd216bcf89e04b864247053.
Running the build with the diagnostics does not actually produce any more output in this step.
The build url is https://dev.azure.com/Ceridian-dryrun/SharpTop/_build/results?buildId=1672629&view=logs&j=ca395085-040a-526b-2ce8-bdc85f692774&t=9ff468ea-e6fc-49e0-b3ce-f8332e9d6e3d, but I doubt it can be viewed by anyone.
I tried to reproduce it on a small repo, but apparently only this particular build is vulnerable.
How is one supposed to troubleshoot it? I am more than willing to inspect the source code of that task, but it is not amongst the tasks found in https://github.com/microsoft/azure-pipelines-tasks/tree/master/Tasks. So, what can we do here?
Another weird thing - the duration of the step. It took 5 minutes to fail.

Azure DevOps Pipelines "Waiting for console output from an agent..."

I require something from the output of a running release task in order for it to complete (an authenticate code). But the console is now not updating. All I get is "Waiting for console output from an agent..."
This happens on both our self-hosted agents (Linux or Windows) and on the Hosted Ubuntu 1604 agent.
The step in question is the standard Kubernetes task: https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks/KubernetesV1
This was not always happening.
To rule out the possibility of kubectl awaiting console input (as has been discussed above), you could try
kubectl apply --dry-run=client [other args]
or
kubectl apply --dry-run=server [other args]
This could give you guidance as to how to proceed, perhaps with --force or --overwrite flags if needed.
I have the same issue. After troubleshooting and canceling the task, I noticed that the agent was waiting for a response from the user.
In my case, I was trying to unzip a file where the destination folder already exists with content. So the system was asking the user to replace the destination folder content that's why the agent was waiting.
2020-03-23T04:14:57.8941954Z unzip /home/azure-deploy-test/AutoEcole.zip -d /home/test-deployment/
2020-03-23T04:14:57.9086229Z Archive: /home/azure-deploy-test/AutoEcole.zip
2020-03-23T04:14:57.9087639Z
2020-03-23T04:14:57.9136932Z ##[error]replace /home/test-deployment/AutoEcole? [y]es, [n]o, [A]ll, [N]one, [r]ename:
2020-03-23T04:53:12.1979529Z ##[error]The operation was canceled.
This was an issue with Microsoft's Azure DevOps Services that has been acknowledged and rectified by Microsoft.
This issue was reported as an issue with the "Liveness in Release Management UI".
All you have to do is access your project using the below URL:-
https://dev.azure.com/{your organization}/{your project}.
This is an official solution provided by Microsoft. This resolved the issue for me.
Please share more details in the comments section if you still face the issue.

Running Powershell scripts on Web App machine

I have an Azure web app. This web app has a QA deployment slot for pre-production testing. When I check in my code from VS, I have it setup to build and deploy to the QA deployment slot. This works great. However, a few configurations need to be updated in the QA web app so the application points to the correct service endpoints (i.e. not dev). To do this, my initial approach was to add a PS task to the Release that unzips my deployment zip, updates the configuration files, rezips them and then allows the Release flow to deploy the updated zip. This works locally, but running into filename length issues on the server when unzipping, which I can't change.
Now I'm trying to just include my update PS scripts in my deployment package, and then run the scripts AFTER the deployment has occurred. So, I'm looking at this Powershell on Target Machines task to run a PS on the QA slot server to update configurations. However, it's asking for Machines, which would be the server name of the slot server. I don't have that. I also don't know where to get it. I also don't have the path to the PS scripts once I have the server name. I dumped out the server variables and none of them help me, unless there is a cmdlet to look up environments that I'm not aware of.
System.DefaultWorkingDirectory: 'C:\a\2ed23b64d'
System.TeamFoundationServerUri: 'https://REDACTED.vsrm.visualstudio.com/DefaultCollection/'
System.TeamFoundationCollectionUri: 'https://REDACTEDvisualstudio.com/DefaultCollection/'
System.TeamProject: 'REDACTED'
System.TeamProjectId: 'REDACTED'
Release.DefinitionName: 'REDACTED'
Release.EnvironmentUri: 'vstfs:///ReleaseManagement/Environment/46'
Release.EnvironmentName: 'QA'
Release.ReleaseDescription: 'Triggered by REDACTED Build Definition 20160425.4.'
Release.ReleaseId: '31'
Release.ReleaseName: 'Release-31'
Release.ReleaseUri: 'vstfs:///ReleaseManagement/Release/31'
Release.RequestedFor: 'Matthew Mulhearn'
Release.RequestedForId: ''
Agent.HomeDirectory: 'C:\LR\MMS\Services\Mms\TaskAgentProvisioner\Tools\agents\1.98.1'
Agent.JobName: 'Release'
Agent.MachineName: 'TASKAGENT5-0020'
Agent.Name: 'Hosted Agent'
Agent.RootDirectory: 'C:\a'
Agent.WorkingDirectory: 'C:\a\SourceRootMapping\REDACTED'
Agent.ReleaseDirectory: 'C:\a\2ed23b64d'
Anyone have any idea, or a better approach, to accomplish what I'm attempting?

Webistrano - how to clear global HTML cache after deployment

I am new to webistrano so apologies if this is a trivial matter...
I am using webistrano to deploy php code to several production servers, this is all working great. My problem is that I need to clear HTML cache on my cache servers (varnish cache) after the code update. I can't figure out how to build a recipe that will be executed on the webistrano machine (and will run the relevant shell script that will clear the cache) and not on each of the deployment target machines.
Thanks for the help,
Yariv
Simpliest method is to execute varnishadm tool with proper parameters inside deploy:restart
set :varnish_ban_pattern, "req.url ~ ^/"
set :varnish_terminal_address_port, "127.0.0.1:6082"
set :varnish_varnishadm, "/usr/bin/varnishadm"
task :restart, :roles => :web do
run "#{varnish_varnishadm} -T #{varnish_terminal_address_port} ban \"#{varnish_ban_pattern}\""
end
Thanks for the answer. I actually need to do some more stuf than to only clear the the cache so I will execute a bash script locally as described in below:
How do I execute a Capistrano task locally?

Unicorn restart issue with capistrano

We're deploying with cap and using a script that send USR2 to the unicorn process to reload and it usually works but every once in a while it will fail. When that happens looking in the unicorn log reveals that it's looking for a Gemfile in an old release directory that no longer exists.
Exception :
/usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.0.21/lib/bundler/definition.rb:14:in `build': /var/www/railsapps/inventory/releases/20111128233407/Gemfile not found (Bundler::GemfileNotFound)
To clarify that's not the current release but an older one that's since been removed.
When it works it does seem to work correctly - ie it does pickup the new code - so I don't think it's somehow stuck referring to the old release.
Any ideas?
In your unicorn.rb add the before_exec block
current_path = "/var/www/html/my project/current"
before_exec do |server|
ENV['BUNDLE_GEMFILE'] = "#{current_path}/Gemfile"
end
Read more about it here http://blog.willj.net/2011/08/02/fixing-the-gemfile-not-found-bundlergemfilenotfound-error/
You should set the BUNDLE_GEMFILE environment variable before you start the server, point it at current/Gemfile.