I require something from the output of a running release task in order for it to complete (an authenticate code). But the console is now not updating. All I get is "Waiting for console output from an agent..."
This happens on both our self-hosted agents (Linux or Windows) and on the Hosted Ubuntu 1604 agent.
The step in question is the standard Kubernetes task: https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks/KubernetesV1
This was not always happening.
To rule out the possibility of kubectl awaiting console input (as has been discussed above), you could try
kubectl apply --dry-run=client [other args]
or
kubectl apply --dry-run=server [other args]
This could give you guidance as to how to proceed, perhaps with --force or --overwrite flags if needed.
I have the same issue. After troubleshooting and canceling the task, I noticed that the agent was waiting for a response from the user.
In my case, I was trying to unzip a file where the destination folder already exists with content. So the system was asking the user to replace the destination folder content that's why the agent was waiting.
2020-03-23T04:14:57.8941954Z unzip /home/azure-deploy-test/AutoEcole.zip -d /home/test-deployment/
2020-03-23T04:14:57.9086229Z Archive: /home/azure-deploy-test/AutoEcole.zip
2020-03-23T04:14:57.9087639Z
2020-03-23T04:14:57.9136932Z ##[error]replace /home/test-deployment/AutoEcole? [y]es, [n]o, [A]ll, [N]one, [r]ename:
2020-03-23T04:53:12.1979529Z ##[error]The operation was canceled.
This was an issue with Microsoft's Azure DevOps Services that has been acknowledged and rectified by Microsoft.
This issue was reported as an issue with the "Liveness in Release Management UI".
All you have to do is access your project using the below URL:-
https://dev.azure.com/{your organization}/{your project}.
This is an official solution provided by Microsoft. This resolved the issue for me.
Please share more details in the comments section if you still face the issue.
Related
I have been working on an installation of OKD on an air-gapped environment. The first major step has been mirroring the OKD images so that they can be moved over to the new environment and pulled locally. I've been following a combination of the OpenShift documentation and this article, as well as this resource for getting my certificates set up. I have been making slow but consistent progress.
However, I am now having trouble when attempting to actually mirror the files using
oc adm -a ${LOCAL_SECRET_JSON} release mirror \
--from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE}-${ARCHITECTURE} \
--to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} \
--to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE}
I get the following, encouraging response:
info: Mirroring 120 images to host.okd-registry.dns:5000/ocp4/openshift4 ...
followed by blobs: and manifests: lines, and finally the line
stats: shared=0 unique=7 size=105.3MiB ratio=1.00
I then get about 50 lines stating
error: unable to retrieve source image quay.io/openshift-release-dev/ocp-v4.0-art-dev manifest
sha256:{some value}: unauthorized: access to the requested resource is not authorized
I have a quay account but I am not sure if that is required even after my research, and if it is, where or how I would log into it. I have attempted doing so using oc login followed by various addresses within the release structure, but if this is the solution, I may be using the wrong arguments as I have not been able to find any instructions on doing this.
I have also tried the command with sudo. I doubt that is an issue but I tried it anyway.
I suppose the issue could be with my certificates, but I am not sure how to determine if this is the case.
Any guidance or suggestions would be much appreciated.
It has been determined that the OKD documentation is inaccurate at the time that I am posting this answer, and was instructing readers to pull from the OCP image repository rather than the OKD repository, which apparently requires additional credentials. A bug has been logged and the documentation will hopefully be updated soon.
The correct environment variables and full command to mirror the images are as follows:
LOCAL_REGISTRY=localhost:5000 (or your local domain name and port for the registry)
LOCAL_REPOSITORY=okd
LOCAL_SECRET_JSON=<full path to your pull secret>
OCP_RELEASE=4.5.0-0.okd-2020-10-15-235428
PRODUCT_REPO=openshift
RELEASE_NAME=okd
ARCHITECTURE=not-used-in-okd
oc adm -a ${LOCAL_SECRET_JSON} release mirror \
--from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE} \
--to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} \
--to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE} --dry-run
I'm unable to run the Docker Compose task on Azure DevOps and every solution I've looked up online, either makes no sense, or does not work for my scenario.
The job output for the failure is:
This is a very simple process, artifacts are copied to a folder during build, and the docker-compose.yml and .dockerfile is added to this directory, which then needs to be run.
One article explained that if you add your docker-compose.yml to the same folder as the files the image will be hosting and the .dockerfile, that it might cause the daemon to fall over and generate this generic error, so I've added a .dockerignore file, but this issues persists.
I'm using a Hosted Agent - Ubuntu-18.04.
My task looks like this:
steps:
- task: DockerCompose#0
displayName: 'Run a Docker Compose command'
inputs:
azureSubscription: 'Test Dev Ops'
azureContainerRegistry: '{"loginServer":"testdevops.azurecr.io", "id" : "/subscriptions/{subscription_key}/resourceGroups/Test.Devops/providers/Microsoft.ContainerRegistry/registries/testdevops"}'
dockerComposeFile: '$(System.DefaultWorkingDirectory)/$(Release.PrimaryArtifactSourceAlias)/test.ng.$(Build.BuildNumber)/dist/testweb/docker-compose-build.yml'
dockerComposeCommand: build
arguments: '--build-arg azure_pat=$(System.AccessToken) --build-arg azure_username=Azure'
The idea here is that this container is composed and delivered straight to Azure's Container Registry.
I have ensured that the user that's running this process, as been granted permissions in that ACR, as well as added the user to the Administrative group in Azure DevOps.
A lot of responses talks about adding the user to the Docker group, but this is a Hosted Agent, not a private agent, so there is no such option.
I have even tried installing Docker CLI before this task, but nothings working.
Am I being daft to think that I can compose in Azure DevOps?
Edit
The contents of my artifacts folder looks something like this:
This error message is extremely misleading. If anyone from Microsoft is looking at this question, please consider making the error more specific, if possible.
It turned out, I missed a semi-colon in a build task that replaced tokens before the build artifacts was pushed from the build output, and because of that, the yaml file had a #{..} token inside of it, which caused the docker-compose to fail.
It had nothing to do with permissions, nor a .dockerignore file, very misleading.
I installed a fresh gcloud sdk (in Azure Cloud Shell if it matters...)
chad#Azure:~$ gcloud --version
Google Cloud SDK 279.0.0
bq 2.0.53
core 2020.01.31
gsutil 4.47
I'm trying to run 'gcloud init' -- but the URL it renders does not work...
chad#Azure:~$ gcloud init
Welcome! This command will take you through the configuration of gcloud.
Your current configuration has been set to: [default]
You can skip diagnostics next time by using the following flag:
gcloud init --skip-diagnostics
Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
Reachability Check passed.
Network diagnostic passed (1/1 checks passed).
You must log in to continue. Would you like to log in (Y/n)?
Go to the following link in your browser:
https://accounts.google.com/o/oauth2/auth?code_challenge= I REMOVED
A BUNCH OF CODES HERE &scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth
Enter verification code:
Am I missing something?
#John Hanley -- for the win!
Can't believe I didn't think of that -- thanks much.
Chad
p.s. looks like a bug in Azure Console or Windows Terminal truncating wrapped text...
What setting up a new Kubernetes endpoint and clicking "Verify Connection" the error message:
"The Kubconfig does not contain user field. Please check the kubeconfig. " - is always displayed.
Have tried multiple ways of outputting the config file to no avail. I've also copy and pasted many sample config files from the web and all end up with the same issue. Anyone been successful in creating a new endpoint?
This is followed by TsuyoshiUshio/KubernetesTask issue 35
I try to reproduce, however, I can't do it.
I'm not sure, however, I can guess it might the mismatch of the version of the cluster/kubectl which you download by the download task/kubeconfig.
Workaround might be like this:
kubectl version in your local machine and check the current server/client version
specify the same version as the server on the download task. (by default it is 1.5.2)
See the log of your release pipeline which is fail, you can see which kubectl command has been executed, do the same thing on your local machine with fitting your local pc's environment.
The point is, before go to the VSTS, download the kubectl by yourself.
Then, put the kubeconfg on the default folder like ~/.kube/config or set environment variables KUBECONFIG to the binary.
Then execute kubectl get nodes and make sure if it works.
My kubeconfig is different format with yours. If you use AKS, az aks install-cli command and az aks get-credentials command.
Please refer https://learn.microsoft.com/en-us/azure/aks/kubernetes-walkthrough .
If it works locally, the config file must work on the VSTS task environment. (or this task or VSTS has a bug)
I had the same problem on VSTS.
Here is my workaround to get a Service Connection working (in my case to GCloud):
Switched Authentication to "Service Account"
Run the two commands told by the info icon next to the fields Token and Certificate: "Token to authenticate against Kubernetes.
Use the ‘kubectl get serviceaccounts -o yaml’ and ‘kubectl get secret
-o yaml’ commands to get the token."
kubectl get secret -o yaml > kubectl-secret.yaml
Search inside the the file kubectl-secret.yaml the values ca.crt and token
Enter the values inside VSTS to the required fields
The generated config I was using had a duplicate line, removing this corrected the issue for me.
users:
- name: cluster_stuff_here
- name: cluster_stuff_here
I'm in some trouble with GitLab CI.
I followed offical guide on:
https://github.com/gitlabhq/gitlab-ci/blob/master/doc/installation.md
Everything was ok, no errors nowhere. I followed Runner-Setup, too.
Anything alright.
But...
When I add a runner to a project and then try to build nothing happens.
It could be that I have not fully understood something or some of my configs are wrong.
I'm absolutely new to GitLab CI, but I like it and I want to learn new stuff.
I would be very very glad if someone could help me in some way.
Thanks!
BIG UPDATE:
Just figured out that:
~/gitlab-runners/gitlab-ci-runner$ bin/runner
Starting a runner process manually solves the problem but if I look at the gitlab-ci-runner in /etc/init.d -> it is running !?!
~/gitlab-runners/gitlab-ci-runner$ sudo /etc/init.d/gitlab-ci-runner start
Number of registered runners in PID file=1
Number of running runners=0
Error! GitLab CI runner(s) (gitlab-ci-runner) appear to be running already! Try stopping them first. Exiting.
~/gitlab-runners/gitlab-ci-runner$ sudo /etc/init.d/gitlab-ci-runner stop
Number of registered runners in PID file=1
Number of running runners=0
WARNING: Numbers of registered runners don't match number of running runners. Will try to stop them all
Registered runners=1
Running runners=0
Trying to stop registered runners...kill: No such process
OK
Trying to kill ghost runners...OK
What's wrong here? I'm out of my power or not seeing the problem?!
Problem solved!
You need to edit some values in /etc/init.d/gitlab-ci-runner script!
APP_ROOT="**PATH_TO**/gitlab-runners/gitlab-ci-runner"
APP_USER="**USER_WITH_DIRRIGHTS!**"
PID_PATH="$APP_ROOT/tmp/pids"
PROCESS_NAME="ruby ./bin/runner"
RUNNERS_PID="$PID_PATH/runners.pid"
RUNNERS_NUM=1 # number of runners to spawn
START_RUNNER="nohup bundle exec ./bin/runner"
Now it works!
In my case tags in the runner were different from tags in the .gitlab-ci.yml. Once I changed them so runner tags include all of the config file tests, tasks began to run.