Exception calling IAM: There were concurrent policy changes - gcloud

I get the above error after running mvn -DskipTests package appengine:deploy. Here's the whole error:
Exception calling IAM: There were concurrent policy changes. Please retry the whole read-modify-write with exponential backoff. The request's ETag '\007\005\364\274\033.\262\307' did not match the current policy's ETag '\007\005\364\274\033\177\220\315'.
From my reading, this occurs when there multiple policy changes at the same time. The other questions I've seen throw this error in regard to a POST request but this is to do with an appengine:deploy
What IAM policy does the service account need to work?
This is the IAM account that I believe was created by the appengine:deploy process and it's permissions:
Note - I don't think it's applicable but I added this deploy to appengine step to a Spring Boot app speakiong to a Postgres db.
EDIT
As mentioned in the comment, #Robina's answer has moved me along but not the logs show this error:
UnsupportedClassVersionError: org/springframework/boot/loader/JarLauncher has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtome only recognizes class file version up to 55.0
But setting the java version to 17 throws a build error; here's my appengine.yaml:
runtime: java11
instance_class: F1
EDIT 2
I came across this while reading through the Google guide here which said:
Deploying an executable JAR
Use any build framework to build an executable JAR locally, then do one of the following depending on whether you created an app.yaml file for your app:
If you created an app.yaml file:
Copy the file into the same directory as the executable JAR file you created.
From the directory that contains the app.yaml and your JAR, enter the following command:
gcloud app deploy
gcloud app deploy will create an app.yaml file that contains the
minimum settings, using all default values.
I thought, why not? So I deleted the appengine folder and its app.yaml and then ran the command from above, minus the jar:
gcloud app deploy
After, probably 10 minutes of the command running the app was deployed and points to the Postgres database without any problems. Thanks #Robina for the help!

The service account must have the App Engine Deployer role in order to deploy an application to App Engine. The IAM policy needed for the service account to function depends on what it is used for. Your application is probably being deployed to Google App Engine using the service account if you are using appengine:deploy.
A gcloud command example for granting the role:
gcloud projects add-iam-policy-binding [PROJECT_ID] --member=serviceAccount:[SERVICE_ACCOUNT_EMAIL] --role=roles/appengine.deployer
[PROJECT ID] and [SERVICE ACCOUNT EMAIL] should be changed to reflect your project ID and service account email, respectively.

Related

Cloud Functions Deployment Issue (.Net 6)

This is the first time I'm working on Google Cloud Functions. I've created a http trigger cloud function using .Net6 framework and trying to deploy it using gcloud Cli. The deployment was failing, and I know why but don't know how to solve it.
I've couple of cs-projects, Cloud Functions project (CFP), Application Services project (ASP) and DBRepositories project (DBP). CFP has a reference of ASP which then has a reference of DBP. I've initiated the deployment from the directory of where Function1.cs(CFP) located. The deployment was unable to locate the other two projects rereferred to CFP.
Project reference visualization:
CloudFunctions.csproj <- ApplicationServices.csproj <- DBRepositories.csproj
I even initiated the deployment from the directory where solution file is located but failed with error "Too many projects found."
Cli command:
gcloud functions deploy demo-function --entry-point HelloFunction.Function1 --runtime dotnet6 --trigger-http --allow-unauthenticated --gen2 --region us-central1
I'm stuck here, any help greatly appreciated. TIA

ERROR: (gcloud.app.deploy) Error Response: [9] Flex operation projects/.../regions/us-central1/operations/... error [FAILED_PRECONDITION]

I'm pretty new on Google Cloud, and I just wanted to deploy my first streamlit webapp. I'm on Windows in command line. I already did the Google Cloud "Hello World" Example, which worked without any error.
When I deploy the streamlit webapp, I got after 3-4 minutes waiting "Updating Server" the following error:
ERROR: (gcloud.app.deploy) Error Response: [9] Flex operation projects/XXXX/regions/us-central1/operations/f0c89d22-2d09-410d-bf99-fc49ad337800 error [FAILED_PRECONDITION]: An internal error occurred while processing task /app-engine-flex/flex_await_healthy/flex_await_healthy>2021-05-27T06:13:50.278Z10796.jc.0: 2021-05-27 06:15:32.787 An update to the [server] config option section was detected. To have these changes be reflected, please restart streamlit.
That's my app.yaml file:
service: default
runtime: custom
env: flex
manual_scaling:
instances: 1
resources:
cpu: 1
memory_gb: 0.5
disk_size_gb: 10
Posting my comment as an answer for better visibility and to summarize.
In this particular case, the error was caused by a mistake in the Dockerfile.
Here are some steps you can follow to fix or narrow down the error:
Try to deploy a test app to see the differences in configuration. Example.
Try deploying your app after updating the gcloud with gcloud components update command.
Make sure you run the SDK as an Admin.
If the error recurs, run the gcloud app deploy app.yaml --verbosity=debug to try getting more specified error.
It's good practice to include references in questions for folks who aren't familiar with e.g. Streamlit. I assume it's this: https://streamlit.io/
I suspect (!) that Streamlit does not (by default) satisfy App Engine's requirements:
A web app on port 8080
No additional (apt get) dependencies
No C-based dependencies
The Streamlit wiki references various deployment alternatives and includes Google Kubernetes Engine (aka GKE) (see below) but not App Engine.
This doesn't mean that it won't work on App Engine (standard) just that it may not be trivial.
The GKE instructions reference installing Cython an optimizing c-compiler and that gives me pause about using App Engine standard. Unless you're familiar with Kubernetes, I'd discourage you from trying GKE as there's more complexity.
So, it would be helpful if others with experience with Streamlit weigh in but, until then, you may wish to consider using Streamlit sharing.
It would be helpful if someone who has deployed Streamlit to App Engine (flexible?) or perhaps Cloud Run can provide an overview.

ERROR: (gcloud.app.deploy) [/home/sarthis7/app.yaml] does not exist

when use gcloud app deploy i am getting following error
ERROR: (gcloud.app.deploy) [/home/sarthis7/app.yaml] does not exist.
You need to have an app.yaml file in the root directory if you want to deploy your app in App Engine
Quoting from the docs:
An app.yaml file specifies settings for your App Engine service's runtime environment. Your service will not deploy without this file.
I suggest checking the app.yaml reference to understand which values you should put in the file.
Also, take a look at the App Engine quickstart, it may help you understand the whole process of Deploying to App Engine.

Error copying pip.conf from bucket to Cloud Composer Airflow environment

Similar to this lonely questioner I'm trying to install a Python package from a private PyPI repo such that it's available to our Google Cloud Composer Airflow instance.
I've followed these instructions but Airflow continues not to know about my package:
No module named 'foopackage'
I can't find any reference to my pip.conf in any logs anywhere so I'm not sure whether the file is in the right place, or has the right contents.
How can I proceed with debugging this problem?
The Cloud Composer environment logs show that there was a problem with copying pip.conf from the bucket, but don't give any other details:
{
insertId: "16qa4c8g540zxs3"
logName: "projects/{my-env}/logs/composer-agent"
receiveTimestamp: "2020-02-06T15:59:03.164564368Z"
resource: {…}
severity: "ERROR"
textPayload: "Copying gs://{my-bucket}/config/pip/pip.conf...
"
timestamp: "2020-02-06T15:59:00.857642186Z"
}
I first thought this might be a permissions issue, but the file seems to have the same set of permissions as other files in this bucket.
Where can I get more detailed information on what went wrong when copying that file?
update
I'm on composer-1.7.2-airflow-1.10.2.
update
The service account for my Composer environment already has the project.editor role.
This is an indicator that the Docker image used for the web server failed to build. To find the root cause, please view Cloud Build logs in project.
The reason for this, is a failed or taking long time operation, it timed out on the Composer’s backend. In some cases these errors persist in the backend, blocking future attempts. You can try re-enabling the API:
First solution that comes to my mind is running following commands in cloud shell:
gcloud services disable composer.googleapis.com
gcloud services enable composer.googleapis.com
After enabling the API, please update your Composer environment as usual.
When you install packages, the Composer environment re-creates Docker containers for the Airflow workers and scheduler, then performs a rolling update within the GKE cluster to update the workers to keep workers available. You can check Kubernetes Engine > Workloads to see if your environment timed out because of waiting for the scheduler and workers to come back online.
When Composer environment is using a custom service account and does not have IAM access to use Cloud Build, builds will fail immediately, so please check it. You can diagnose these by going to Cloud Build > History, and when you see builds without a log, it means that builds failed even before trying to build a container.
When your package implement bindings, it will fail at runtime if the libraries don't exist on the system. This means it is incompatible with Cloud Composer, because getting shared libraries into the build environment is not currently supported.
Another thing, make sure if your project is packed in correct way.
I hope you find the above pieces of information useful.

Failing while trying to deploy Talend Agent in Pivtal Cloud FOundry

I have been trying to deploy Talend Agent as app in PCF, I literally have no idea about Talend. However for PCF guy, its an java jar file for me what i got from DATA team.
I am getting no buildpack supported error. I also tried passing java buildpack by command but failed again with incompatible buildpack.
Error: No container can run this application. Please ensure that you've pushed a valid JVM artifact or artifacts using the -p command line argument or path manifest entry. Information about valid JVM artifacts can be found at https://github.com/cloudfoundry/java-buildpack#additional-documentation.
Failed to compile droplet: Failed to run finalize script: exit status 1
I was expecting this to be deployed as an App which i can access.
Do we have any one who can help me with this?
The CF Java buildpack expects a Java jar file to have certain characteristics in order for it to know how to execute the code in the jar file. The most common characteristics are a self-executable Spring Boot app, an app containing a Main class, and an app containing Tomcat.
I don't know anything about the Talend Agent, but a typical Java agent jar file is not meant to be executed as a stand-alone app. An agent is meant to be installed in the JVM used to run an app, in order to instrument the JVM and/or the app. An typical agent jar file won't have any of the execution entry points recognized by the CF Java buildpack, and therefore the buildpack will reject it with an error message similar to the one you show.
The CF Java buildpack does understand how to install several specific agents (listed under Standard Frameworks in the buildpack docs) into the JVM when an app is deployed. The Talend Agent is not currently in this list. If it is in fact a typical Java agent jar file, you would have to modify the Java buildpack to add support for it.