403 forbidden error on S3 REST API HEAD request - rest

Question

Im trying do do a HEAD Object request to the S3 REST API but I keep getting a 403 Forbidden error, even though I have the policy setup with the necessary permissions on S3. The response body is empty, so I don't think its a signature problem. I've tried several changes to the policy, nothing seems to make it work. I'm able to PUT objects and DELETE objects normally, just HEAD doesn't work.
Here's my bucket policy:
{
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam:: 999999999999:user/User"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::my-bucket"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-bucket/*"
},
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::999999999999:user/User"
},
"Action": [
"s3:GetObject",
"s3:GetObjectVersion",
"s3:DeleteObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::my-bucket/*"
}
]
}
Any ideas?
Update:
As Michael pointed out it seems to be a problem with my signature, though Im failing to see what.
def generate_url options={}
options[:action] = options[:action].to_s.upcase
options[:expires] ||= Time.now.to_i + 100
file_path = "/" + #bucket_name + "/" + options[:file_name]
string_to_sign = ""
string_to_sign += options[:action]
string_to_sign += "\n\n#{options[:mime_type]}\n"
string_to_sign += options[:expires].to_s
string_to_sign += "\n"
string_to_sign += file_path
signature = CGI::escape(
Base64.strict_encode64(
OpenSSL::HMAC.digest('sha1', SECRET_KEY, string_to_sign)
)
)
url = "https://s3.amazonaws.com"
url += file_path
url += "?AWSAccessKeyId=#{ACCESS_KEY}"
url += "&Expires=#{options[:expires]}"
url += "&Signature=#{signature}"
url
end
The generated string to sign looks like this:
HEAD\n\n\n1418590715\n/video-thumbnails/1234.jpg"
Solution:
It seems at some point while developing the file PUT part I actually have broken GET and HEAD. I was passing an empty string as the body of the request, instead of passing nothing, making the mime type required on the signature and breaking it because I was providing no mime type. I simply removed the empty request body and it worked perfectly. Thanks Michael for pointing me out of the wrong direction I was(I wasted so much time changing the bucket policy).

Answer 1

It still could be your signature, and I suspect that it is, for the following reasons:
Your observation that the message body is a good observation; however, it doesn't mean what you have concluded it means.
The lack of a response body does not give you any information at all about the nature of the error, in this case, because a web server is not supposed to return a body along with a HEAD response, no matter what:
The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response
— http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html (RFC-2616)
Testing this on my side, I've confirmed that S3's response to an unsigned HEAD request and to an incorrectly-signed HEAD request is no different: it's always HTTP/1.1 403 Forbidden with no message body.
Note, also, that a signed URL for GET is not valid for HEAD, and vice versa.
In both S3 Signature Version 2 and S3 Signature Version 4, the "String to Sign" includes the "HTTP Verb," which would be GET or HEAD, meaning that a signature that's valid for GET would not be valid for HEAD, and vice versa... the request method must be known at the time of signing, because it's an element that's used in the signing process.
The s3:GetObject permission is the only documented permission required for using HEAD, which seems to eliminate permissions as the problem, if GET is working, which points back to the signature as the potential issue.

Answer 2

Confirmed that HEAD a presigned-URL will get 403 Forbidden.
If set custom headers such as content-type of the object.
The 403 response will not contain the custom header and still get application/xml.

Answer 3

Additional comment on #Michael-sqlbot 's answer above ...
I faced the identical symptoms but I had a different root cause.
If you are trying to HEAD a file which does not exist, then this will also return a 403-forbidden error, UNLESS you have the s3:ListBucket permission.
In my case, I had the s3.GetObject, s3.PutObject, and s3.HeadBucket permissions, but it wasn't until I added s3.ListBucket that I got the correct 404 - not found error.
This is also explained here: https://aws.amazon.com/premiumsupport/knowledge-center/s3-rest-api-cloudfront-error-403/

Answer 4

Had the same issue but with a different root cause - was trying to create a bucket, and instead of getting a 404, got 403. As S3 is globally namespaced, someone else had created the bucket, so while I had the correct permissions and setup for my account, I still would get 403 from a HEAD request. Solution was to check if the bucket exists globally first, and if so, try a different bucket name.

Answer 5

I was also getting this error as a red herring; during pytest using freezegun. I had frozen time to a time in the past, and was getting a 403 error. So clock skew could cause this.
I found this by trying another API call, where I received:
E botocore.exceptions.ClientError: An error occurred (RequestTimeTooSkewed) when calling the ListObjects operation: The difference between the request time and the current time is too large.