CEPH S3 Exception while Listing Blobs - kubernetes

Question

I have created an S3 bucket backed by CEPH and through java S3 client and via S3 object gateway am listing the directory in a paginated fashion and randomly the listing is failing some times after listing 1100 blobs in batches, some times after listing 2000 blobs in batches and am not able to figure out how to debug this issue, this is the exception am getting and if you notice there is a requestId in the exception I think basis this I can filter the logs but where can i find the logs is the question, I have checked the s3 gateway pod logs but I couldn't find any such logs over there, please let me know where should I look for the same
com.amazonaws.services.s3.model.AmazonS3Exception: null (Service: Amazon S3; Status Code: 500; Error Code: UnknownError; Request ID: tx00000000000000000e7df-005e626049-1146-rook-ceph-store; S3 Extended Request ID: 1146-rook-ceph-store-rook-ceph-store), S3 Extended Request ID: 1146-rook-ceph-store-rook-ceph-store
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1799)
and this is my code to iterate through the blobs, this is non paginated, the paginated version, both the versions are throwing the same exception after listing few hundred blobs
ObjectListing objects = conn.listObjects(bucket.getName());
do {
for (S3ObjectSummary objectSummary : objects.getObjectSummaries()) {
System.out.println(objectSummary.getKey() + "\t" +
objectSummary.getSize() + "\t" +
StringUtils.fromDate(objectSummary.getLastModified()));
}
objects = conn.listNextBatchOfObjects(objects);
} while (objects.isTruncated());
So, any pointers on how to debug this would be helpful.. Thanks

Answer 1

Try ListObjectV2.
Returns some or all (up to 1,000) of the objects in a bucket.
https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html