PermissionDeniedError: From /job:worker/replica:0/task:0 - google-cloud-storage

Question

I am using google colaboratory to pretrain bert.My train_data is on google cloud storage,my code is on colab
IS_COLAB_BACKEND = 'COLAB_GPU' in os.environ
if IS_COLAB_BACKEND:
from google.colab import auth
auth.authenticate_user()
model_saved_path = 'gs://zhongxing_train/model/roberta_pretrain_model2.ckpt'
corpus_paths = ['gs://zhongxing_train/corpus.0.tfrecord' ]
tpu_address = 'grpc://' + os.environ['COLAB_TPU_ADDR']
The flowing is error message:
PermissionDeniedError Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1355 try:
-> 1356 return fn(*args)
1357 except errors.OpError as e:
10 frames
PermissionDeniedError: From /job:worker/replica:0/task:0:
Error executing an HTTP request: HTTP response code 403 with body '{
"error": {
"code": 403,
"message": "service-555555555555#cloud-tpu.iam.gserviceaccount.com does not have storage.objects.get access to zhongxing_train/corpus.0.tfrecord.",
"errors": [
{
"message": "service-555555555555#cloud-tpu.iam.gserviceaccount.com does not have storage.objects.get access to zhongxing_train/corpus.0.tfrecord.",
"domain": "global",
"reason": "forbidden"
}
]
}
}
'
when reading metadata of gs://zhongxing_train/corpus.0.tfrecord
[[{{node MultiDeviceIteratorGetNextFromShard}}]]
[[RemoteCall]]
[[IteratorGetNextAsOptional]]
During handling of the above exception, another exception occurred:
PermissionDeniedError Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1368 pass
1369 message = error_interpolation.interpolate(message, self._graph)
-> 1370 raise type(e)(node_def, op, message)
1371
1372 def _extend_graph(self):
PermissionDeniedError: From /job:worker/replica:0/task:0:
Error executing an HTTP request: HTTP response code 403 with body '{
"error": {
"code": 403,
"message": "service-555555555555#cloud-tpu.iam.gserviceaccount.com does not have storage.objects.get access to zhongxing_train/corpus.0.tfrecord.",
"errors": [
{
"message": "service-555555555555#cloud-tpu.iam.gserviceaccount.com does not have storage.objects.get access to zhongxing_train/corpus.0.tfrecord.",
"domain": "global",
"reason": "forbidden"
}
]
}
}
'
when reading metadata of gs://zhongxing_train/corpus.0.tfrecord
[[{{node MultiDeviceIteratorGetNextFromShard}}]]
[[RemoteCall]]
[[IteratorGetNextAsOptional]]
This is my colab cord
I think maybe I need to set permission for my train_data on cloud storage
How can I get permission from google cloud storage

Answer 1

You need to grant your service account with storage.objects.get permission, which you can do by granting the service account with at least the Storage Object Viewer role.
You can do it using the Cloud Console or the Cloud SDK.