I'm trying to deploy the mongo db quick start as seen here.
Unfortunately, it quickly fails with status CREATE_FAILED with the following error which I can't understand a lot from:
Embedded stack arn:aws:cloudformation:us-west-****** was not successfully created:
The following resource(s) failed to create:
[NAT1EIP, NAT2EIP, PublicSubnet1RouteTableAssociation, PrivateSubnet2ARouteTableAssociation, PublicSubnetRoute, PrivateSubnet1ARouteTableAssociation, PublicSubnet2RouteTableAssociation].
I tried using both my own user's role and a new role I created where the trusted entity was CF and permissions of power user. It failed in both cases.
I'm surely missing something very basic, any thoughts?
Thank you
If you are unable to create these networking resources, it could be very well due to IAM user restrictions.
Related
I deleted my KF cluster last night to create a new one (using kubectl cluster command not Kfctl delete), and then when I tied to create a new one, it fails, it does not work with CLI not Console. I found other people have run into this issue before, for example (here and here)
"However, as I said even with CLI my deployment fails, the error from console is:
ailed to apply: (kubeflow.error): Code 500 with message: coordinator Apply failed for gcp: (kubeflow.error): Code 500 with message: gcp apply could not update deployment manager Error could not update storage-kubeflow.yaml; Insert deployment error: googleapi: Error 403: Request had insufficient authentication scopes.
More details:
Reason: insufficientPermissions, Message: Insufficient Permission"
and the error I get from Console is:
"Please enable APIs for your project and try again
Please enable cloud resource manager API: https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/ and iam API: https://console.developers.google.com/apis/api/iam.googleapis.com/"
Note that this error is wrong, all the apis are active already. I'm quite sure this is a bug of KF but not sure how to find a workaround, any thoughts?
With CLI, I'm using my own account which has "owner" privileges.
Thanks
It seems you have an issue with IAM and the installation of Kubeflow, a 3rd party product that itself is not supported by us; nevertheless I went ahead and dig some information about this Machine Learning product.
The main issues (and although it seems you already cover permissions) are permissions, number of projects and some fine grained points.
I was checking and found out the following things that may help
a) Troubleshooting Kubeflow 1
b) Deploying Kubeflow in GKE[2]
c) Kubleflow auto deployer for GKE[3]
There are also some discussion about a mismatch permissions setting in Kubeflow that may be worth reading [4]
Finally there is a group that, also on a best-effort basis due the nature of Kubeflow:"google-kubeflow-support#google.com" that may come in handy.
I trust this information will be useful for you to solve your issue
I have fabric composer 0.72 installed on my mac, and I was able to follow this thread to get it connected to my Blockchain (v.61 of Fabric) on Bluemix.
fabric-composer-integration-with-bluemix-blockchain-service
Now I am trying to build an ubuntu (16.04) docker container and run composer-rest-server there. When I try to connect to my blockchain service from my docker container (using the same id, WebAppAdmin, that I used on my mac) I get an error:
Discovering types from business network definition ...
Connection fails: Error: Identity or token does not match.
It will be retried for the next request.
{ Error: Identity or token does not match.
at /home/composer/.nvm/versions/node/v6.10.3/lib/node_modules /composer-rest-server/node_modules/grpc/src/node/src/client.js:417:17 code: 2, metadata: Metadata { _internal_repr: {} } }
I tried copying the cert from my mac to my docker container:
/home/composer/.composer-credentials/member.WebAppAdmin
but when I did that I got a different error that says "signature does not verify". I did some additional testing, and I discovered that if I used an id that I had not previously used with composer (i.e. user_type1_0) then I could connect, and I could see a new cert in my .composer-credentials directory.
I tried deleting that container and building a new one (I dorked something else up) I could not use that same userid again.
Does anybody know how security and these certs are supposed to work? It would seem as though something to do with certificate generation/validation is tied to the client (i.e. hardware address), such that if I try to re-use an id on a different machine, the certs or keys or something don't match. I have a way to make things work, but it doesn't seem like it's the right way if I can't use the same id from different machines.
Thanks!
Hi i tried to recreate this by having blockchain running on a unix machine and then i copied my connection profile and certificate to my mac and then edited my connection profile to update the ip address and key store. I then did a composer network ping and it worked fine.
I am using composer v0.7.4 so you could try that?
I have also faced this issue, and concluded that
There is inconsistent behavior while deploying network using composer on Cloud environment includeing Bluemix. Problem is not with composer, but with fabric 0.6.
I am assuming that this issue is also indirectly related to following known bugs into fabric 0.6, which will not be fixed in fabric 0.6.
ERROR:
"
throw er; // Unhandled 'error' event
^
Error
at ClientDuplexStream._emitStatusIfDone (/home/ubuntu/.nvm/versions/node/v6.9.5/lib/node_modules/composer-cli/node_modules/grpc/src/node/src/client.js:189:19)
at ClientDuplexStream._readsDone (/home/ubuntu/.nvm/versions/node/v6.9.5/lib/node_modules/composer-cli/node_modules/grpc/src/node/src/client.js:158:8)
at readCallback (/home/ubuntu/.nvm/versions/node/v6.9.5/lib/node_modules/composer-cli/node_modules/grpc/src/node/src/client.js:217:12)
"
So far, We have understood that following three JIRA are root cause , where essentially the cloud networking layer ends up killing the idle event hub connection after a period of inactivity and the fabric SDK cannot handle this.
https://jira.hyperledger.org/browse/FAB-4002 FAB-3310
https://jira.hyperledger.org/browse/FAB-3310
or FAB-2787
Conclusion:
There is no alternative way of fixing this issue with Bluemix or any cloud environment with fabric 0.6
You may not experience this issue with Fabric 1.0, but there is still possibilities as all above mentioned defects are not fixed yet.
i just signed up for Google Could Engine and started the most basic container engine quickstart on a brand new project:
https://cloud.google.com/container-engine/docs/quickstart
a few steps in it has me run this command
gcloud container clusters create example-cluster
which errors out:
$ gcloud container clusters create example-cluster
ERROR: (gcloud.container.clusters.create) ResponseError: code=503, message=Project hello-world-161713 is not fully initialized with the default service accounts. Please try again later.
so far, "trying again later" has not helped: it's been doing this every time for the past few hours.
as usual, Google has no obvious way of getting help in any timely manner, and Googling the error turns up nothing useful.
kind of a long shot but i found a link sending me over here on one of their pages (great support guys) so figured i'd give it a shot. thanks in advance.
The Container Engine API needs to be enabled, and unfortunately that error message is not correct (trying again later won't help).
If you visit the Google Container Engine page in the wb console https://console.cloud.google.com/kubernetes/list it should enable Google Container Engine. Make sure you select the project you're using with the quickstart. You can create your cluster from that page too if you'd prefer.
You can also enable the Container Engine API manually here: https://console.cloud.google.com/apis/api/container.googleapis.com/overview
I've stopped my Google Cloud SQL 2nd generation instance on 02 Jan this year.
Today I'm trying to start it again but just receives an error:
"Could not complete the operation"
This is the only info in the logs:
{
protoPayload: {…}
insertId: "54775E151DAA9.A2E1542.960A7970"
resource: {…}
timestamp: "2017-02-01T10:55:00.523Z"
severity: "ERROR"
logName: "projects/hti-info-center/logs/cloudaudit.googleapis.com%2Factivity"
}
All functions including restoring of backups are disabled as the instance is stopped.
Is there anywhere I can get more information about the instance's current state in order to try and resolve this issue, without having to sign up for a Google Support package?
This was suggested by Google support, and worked for me:
try to start your service via gcloud by running the following command:
gcloud sql instances patch [INSTANCE_NAME] --activation-policy=[ACTIVATION_POLICY]
and set the activation policy to "ALWAYS" or "ON_DEMAND".
Similar situation it's happening to me, post a comment here: https://code.google.com/p/googlecloudsql/issues/detail?can=2&start=0&num=100&q=&colspec=ID%20Type%20Status%20Priority%20Milestone%20Owner%20Summary%20log&groupby=&sort=&id=216
It seems that they have a bug with the cloud SQL instances since Jan 25th (although I have experimented a similar error on January 13th).
Very very frustrating...
In your case, if the instance is stopped, have you tried to create a clone or export the data and create a new instance?
This is a different scenario from other question on this topic. My deployment almost succeeded and I can see the following lines at the end of my log
[datalab].../#015Updating module [datalab]...done.
Jul 25 16:22:36 datalab-deploy-main-20160725-16-19-55 startupscript: Deployed module [datalab] to [https://main-dot-datalab-dot-.appspot.com]
Jul 25 16:22:36 datalab-deploy-main-20160725-16-19-55 startupscript: Step deploy datalab module succeeded.
Jul 25 16:22:36 datalab-deploy-main-20160725-16-19-55 startupscript: Deleting VM instance...
The landing page keeps showing a wait bar indicating the deployment is still in progress. I have tried deploying several times in last couple of days.
About additions described on the landing page -
An App Engine "datalab" module is added. - when I click on the pop-out url "https://datalab-dot-.appspot.com/" it throws an error page with "404 page not found"
A "datalab" Compute Engine network is added. - Under "Compute Engine > Operations" I can see a create instance for datalab deployment with my id and a delete instance operation with *******-ompute#developer.gserviceaccount.com id. not sure what it means.
Datalab branch is added to the git repo- Yes and with all the components.
I think the deployment is partially successful. When I visit the landing page again, the only option I see is to deploy the datalab again and not to start it. Can someone spot the problem ? Appreciate the help.
I read the other posts on this topic and tried to verify my deployment using - "https://console.developers.google.com/apis/api/source/overview?project=" I get the following message-
The API doesn't exist or you don't have permission to access it
You can try looking at the App Engine dashboard here, to verify that there is a "datalab" service deployed.
If that is missing, then you need to redeploy again (or switch to the new locally-run version).
If that is present, then you should also be able to see a "datalab" network here, and a VM instance named something like "gae-datalab-main-..." here. If either of those are missing, then try going back to the App Engine console, deleting the "datalab" service, and redeploying.