Cloudfront + S3 + HTTPS template also serving unwanted HTTP - aws-cloudformation

I have a Cloudformation template for the serving of S3 content over HTTPS, via Cloudfront (see stack at end).
It works fine over HTTPS -
(env) justin#justin-XPS-13-9360:~/work/gists/ae46e471eb0e10e877731d787a9fbc2a$ curl -iv https://cloudfront-https-demo.s3-eu-west-1.amazonaws.com/index.json
* Trying 52.218.90.64...
* TCP_NODELAY set
* Connected to cloudfront-https-demo.s3-eu-west-1.amazonaws.com (52.218.90.64) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=US; ST=Washington; L=Seattle; O=Amazon.com, Inc.; CN=*.s3-eu-west-1.amazonaws.com
* start date: Aug 4 00:00:00 2020 GMT
* expire date: Aug 9 12:00:00 2021 GMT
* subjectAltName: host "cloudfront-https-demo.s3-eu-west-1.amazonaws.com" matched cert's "*.s3-eu-west-1.amazonaws.com"
* issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert Baltimore CA-2 G2
* SSL certificate verify ok.
> GET /index.json HTTP/1.1
> Host: cloudfront-https-demo.s3-eu-west-1.amazonaws.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< x-amz-id-2: zVJ4idiIpuyIJIDvXIXbykQe9Po1YebyXEnYMLPkQydhwPLBpkypQfas3vCKuMIfd2cv+yISoqo=
x-amz-id-2: zVJ4idiIpuyIJIDvXIXbykQe9Po1YebyXEnYMLPkQydhwPLBpkypQfas3vCKuMIfd2cv+yISoqo=
< x-amz-request-id: 3E4B1B22DB6EE25D
x-amz-request-id: 3E4B1B22DB6EE25D
< Date: Tue, 25 Aug 2020 19:32:49 GMT
Date: Tue, 25 Aug 2020 19:32:49 GMT
< Last-Modified: Tue, 25 Aug 2020 15:42:18 GMT
Last-Modified: Tue, 25 Aug 2020 15:42:18 GMT
< ETag: "50521abab7a013be83ee00ded3f9f424"
ETag: "50521abab7a013be83ee00ded3f9f424"
< Accept-Ranges: bytes
Accept-Ranges: bytes
< Content-Type: application/json
Content-Type: application/json
< Content-Length: 19
Content-Length: 19
< Server: AmazonS3
Server: AmazonS3
<
{"hello": "world"}
* Connection #0 to host cloudfront-https-demo.s3-eu-west-1.amazonaws.com left intact
(I pushed an index.json file to the bucket)
but a little too well, as it also still serves plain HTTP, despite me specifying https-only in the template -
(env) justin#justin-XPS-13-9360:~/work/gists/ae46e471eb0e10e877731d787a9fbc2a$ curl -iv http://cloudfront-https-demo.s3-eu-west-1.amazonaws.com/index.json
* Trying 52.218.88.232...
* TCP_NODELAY set
* Connected to cloudfront-https-demo.s3-eu-west-1.amazonaws.com (52.218.88.232) port 80 (#0)
> GET /index.json HTTP/1.1
> Host: cloudfront-https-demo.s3-eu-west-1.amazonaws.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< x-amz-id-2: 9MKgLpvr1BmjF49D98VGThFi0JN4wUzAtIaocNh/8DTss54Zlo+/WFMymbLTdKpUoN0fvvLpMLg=
x-amz-id-2: 9MKgLpvr1BmjF49D98VGThFi0JN4wUzAtIaocNh/8DTss54Zlo+/WFMymbLTdKpUoN0fvvLpMLg=
< x-amz-request-id: 5H1V5SBZAJAY2QAW
x-amz-request-id: 5H1V5SBZAJAY2QAW
< Date: Tue, 25 Aug 2020 19:33:27 GMT
Date: Tue, 25 Aug 2020 19:33:27 GMT
< Last-Modified: Tue, 25 Aug 2020 15:42:18 GMT
Last-Modified: Tue, 25 Aug 2020 15:42:18 GMT
< ETag: "50521abab7a013be83ee00ded3f9f424"
ETag: "50521abab7a013be83ee00ded3f9f424"
< Accept-Ranges: bytes
Accept-Ranges: bytes
< Content-Type: application/json
Content-Type: application/json
< Content-Length: 19
Content-Length: 19
< Server: AmazonS3
Server: AmazonS3
<
{"hello": "world"}
* Connection #0 to host cloudfront-https-demo.s3-eu-west-1.amazonaws.com left intact
What do I need to do to the template to turn off plain HTTP / leave me with HTTPS only ?
---
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
AppName:
Type: String
Resources:
CloudFrontOAI:
Type: AWS::CloudFront::CloudFrontOriginAccessIdentity
Properties:
CloudFrontOriginAccessIdentityConfig:
Comment: access-identity-passport-front.s3.amazonaws.com
SiteCloudFrontDist:
Type: AWS::CloudFront::Distribution
Properties:
DistributionConfig:
Enabled: true
DefaultRootObject: index.json
Origins:
- DomainName:
Fn::Sub:
- ${app_name}.s3.amazonaws.com
- app_name:
Ref: AppName
Id:
Fn::Sub:
- ${app_name}-site-cf-origin
- app_name:
Ref: AppName
S3OriginConfig:
OriginAccessIdentity:
Fn::Sub:
- origin-access-identity/cloudfront/${oai}
- oai:
Ref: CloudFrontOAI
DefaultCacheBehavior:
AllowedMethods:
- HEAD
- GET
- OPTIONS
ForwardedValues:
QueryString: false
TargetOriginId:
Fn::Sub:
- ${app_name}-site-cf-origin
- app_name:
Ref: AppName
ViewerProtocolPolicy: https-only
ViewerCertificate:
CloudFrontDefaultCertificate: true
SiteBucket:
Type: AWS::S3::Bucket
Properties:
BucketName:
Ref: AppName # NB same as app for URL convenience
SiteBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket:
Ref: SiteBucket
PolicyDocument:
Statement:
- Action: s3:GetObject
Effect: Allow
Principal: '*'
Resource:
Fn::Sub:
- arn:aws:s3:::${bucket_name}/*
- bucket_name:
Ref: SiteBucket
Version: '2012-10-17'

In both cases you are using domain:
cloudfront-https-demo.s3-eu-west-1.amazonaws.com
This is not a CloudFront domain, which has format of, e.g.:
d111111abcdef8.cloudfront.net
It looks to me that you are using the url of the form:
https://<bucket name>.s3-<AWS Region>.amazonaws.com/<object name>
However, this is not CloudFront URL as indicated in the docs:
Use your Amazon S3 URL to verify that your content is publicly accessible, but remember that this is not the URL you'll use when you're ready to distribute your content with CloudFront.

Need to modify the bucket policy to disable non- secure connections -
{
"Version": "2012-10-17",
"Id": "ExamplePolicy",
"Statement": [
{
"Sid": "AllowSSLRequestsOnly",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::bucket_name/*",
"Condition": {
"Bool": {
"aws:SecureTransport": "true"
}
}
}
]
}

Related

MongoDB HTTPS Endpoints - access querystring in function

I have an HTTPS endpoint in MongoDB Realm. How can I get the values of individual query string parameters inside a function? I'm trying to use the Request object, but I can only get back the entire querystring.
Sample URL: https:///endpoint/search?param1=apple&param2=banana
The endpoint calls the function:
exports = function(){
query = context.request.rawQueryString;
return {query};
};
A GET request to the endpoint returns:
{
"query": "param1=apple&param2=banana"
}
How can I modify my function to get values of a named query string parameter, e.g.?
const queryParam1 = request.query.param1 would evaluate to "apple".
Thank you.
Grab the specific fields from the query object:
const {value1, value2} = query;
I believe the variable names need to match the query string parameters. For example, assume the following CURL command (notice the query string parameters):
curl --include \
--verbose \
--header "Accept: application/json" \
--header "Content-Type: application/json" \
--request POST "https://eastus2.azure.data.mongodb-api.com/app/simplewebservice-ryzao/endpoint/test?queryStringParam1=abc&queryStringParam2=123" \
--data '{ "fieldA": { "$int": "1" }, "fieldB": {"$date": "2022-09-26T22:20:17.123Z" } }'
Because I have two query string parameters I need to catch them using their specific names - queryStringParam1 and queryStringParam2. See the code listing below for a complete working function:
exports = function({ query, headers, body}, response) {
try {
if(body === undefined) {
throw new Error(`Request body was not defined.`)
}
const {queryStringParam1, queryStringParam2} = query;
console.log(queryStringParam1);
console.log(queryStringParam2);
console.log("contentTypes = " + headers["Content-Type"])
var bodyJson = EJSON.parse(body.text());
console.log(bodyJson);
var calculatedObject = new Object();
calculatedObject.queryStringParam1 = queryStringParam1;
calculatedObject.queryStringParam2 = queryStringParam2;
calculatedObject.requestPayloadFieldA = bodyJson.fieldA;
calculatedObject.requestPayloadFieldB = bodyJson.fieldB;
response.setStatusCode(200);
response.setBody(JSON.stringify(calculatedObject));
} catch (error) {
response.setStatusCode(400);
response.setBody(error.message);
console.log(error.message);
}
};
In this example, I simply repackage and reply with the same data as what was sent. The response payload combines query string variables and request payload variables into a single response payload.
Example of output:
barry#barry-laptop:~$ curl --include --verbose --header "Accept: application/json" --header "Content-Type: application/json" --request POST "https://eastus2.azure.data.mongodb-api.com/app/simplewebservice-ryzao/endpoint/test?queryStringParam1=abc&queryStringParam2=123" --data '{ "fieldA": { "$int": "1" }, "fieldB": {"$date": "2022-09-26T22:20:17.123Z" } }'
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 20.71.94.254:443...
* Connected to eastus2.azure.data.mongodb-api.com (20.71.94.254) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=data.mongodb-api.com
* start date: Aug 8 12:45:54 2022 GMT
* expire date: Nov 6 12:45:53 2022 GMT
* subjectAltName: host "eastus2.azure.data.mongodb-api.com" matched cert's "eastus2.azure.data.mongodb-api.com"
* issuer: C=US; O=Let's Encrypt; CN=R3
* SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x56145e5b6e80)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> POST /app/simplewebservice-ryzao/endpoint/test?queryStringParam1=abc&queryStringParam2=123 HTTP/2
> Host: eastus2.azure.data.mongodb-api.com
> user-agent: curl/7.81.0
> accept: application/json
> content-type: application/json
> content-length: 78
>
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* We are completely uploaded and fine
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection state changed (MAX_CONCURRENT_STREAMS == 2147483647)!
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
< HTTP/2 200
HTTP/2 200
< vary: Origin
vary: Origin
< x-appservices-request-id: 633227f5d92f1e1d9c61bb7b
x-appservices-request-id: 633227f5d92f1e1d9c61bb7b
< x-frame-options: DENY
x-frame-options: DENY
< date: Mon, 26 Sep 2022 22:30:13 GMT
date: Mon, 26 Sep 2022 22:30:13 GMT
< content-length: 148
content-length: 148
< content-type: text/plain; charset=utf-8
content-type: text/plain; charset=utf-8
< server: envoy
server: envoy
<
* Connection #0 to host eastus2.azure.data.mongodb-api.com left intact
{"queryStringParam1":"abc","queryStringParam2":"123","requestPayloadFieldA":{"$int":"1"},"requestPayloadFieldB":{"$date":"2022-09-26T22:20:17.123Z"}}

ACCESS_TOKEN_SCOPE_INSUFFICIENT error with Google Analytics Data GA4 runReport request

We are receiving an error ACCESS_TOKEN_SCOPE_INSUFFICIENT when trying to run the runReport method with the Google Analytics Data GA4 API.
As you can see in the debugging dump, have all necessary scopes in the authorization to make the request:
Request:
curl -X POST -H "User-Agent: python-requests/2.27.1" -H "Accept-Encoding: gzip, deflate" -H "Accept: */*" -H "Connection: keep-alive" -H "Content-Length: 399" -H "Content-Type: application/json" -d '{"dateRanges": [{"startDate": "2022-04-25", "endDate": "2022-04-25"}], "dimensions": [{"name": "date"}, {"name": "eventName"}, {"name": "sessionDefaultChannelGrouping"}, {"name": "sessionSource"}, {"name": "source"}], "metrics": [{"name": "eventCount"}, {"name": "eventValue"}, {"name": "newUsers"}, {"name": "sessions"}], "metricFilter": null, "dimensionFilter": null, "limit": 100000, "offset": 0}' 'https://analyticsdata.googleapis.com/v1beta/properties/264786259:runReport?access_token=REMOVED'
Response
{
"error": {
"code": 403,
"message": "Request had insufficient authentication scopes.",
"status": "PERMISSION_DENIED",
"details": [
{
"#type": "type.googleapis.com/google.rpc.ErrorInfo",
"reason": "ACCESS_TOKEN_SCOPE_INSUFFICIENT",
"domain": "googleapis.com",
"metadata": {
"service": "analyticsdata.googleapis.com",
"method": "google.analytics.data.v1beta.BetaAnalyticsData.RunReport"
}
}
]
}
}
further debugging dump
% curl -X POST -H 'Accept: */*' -H 'Connection: keep-alive' -H 'Content-Length: 399' -H 'Content-Type: application/json' -H 'User-Agent: python-requests/2.27.1' -d '{"dateRanges": [{"startDate": "2022-04-13", "endDate": "2022-04-13"}], "dimensions": [{"name": "date"}, {"name": "eventName"}, {"name": "sessionDefaultChannelGrouping"}, {"name": "sessionSource"}, {"name": "source"}], "metrics": [{"name": "eventCount"}, {"name": "eventValue"}, {"name": "newUsers"}, {"name": "sessions"}], "metricFilter": null, "dimensionFilter": null, "limit": 100000, "offset": 0}' 'https://analyticsdata.googleapis.com/v1beta/properties/264786259:runReport?access_token=<REMOVED>' -vv
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 2a00:1450:400d:805::200a...
* TCP_NODELAY set
* Connected to analyticsdata.googleapis.com (2a00:1450:400d:805::200a) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=upload.video.google.com
* start date: Mar 28 02:05:27 2022 GMT
* expire date: Jun 20 02:05:26 2022 GMT
* subjectAltName: host "analyticsdata.googleapis.com" matched cert's "*.googleapis.com"
* issuer: C=US; O=Google Trust Services LLC; CN=GTS CA 1C3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7f8cda80f400)
> POST /v1beta/properties/264786259:runReport?access_token=<REMOVED> HTTP/2
> Host: analyticsdata.googleapis.com
> Accept: */*
> Connection: keep-alive
> Content-Length: 399
> Content-Type: application/json
> User-Agent: python-requests/2.27.1
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
* We are completely uploaded and fine
< HTTP/2 403
< www-authenticate: Bearer realm="https://accounts.google.com/", error="insufficient_scope", scope="https://www.googleapis.com/auth/analytics.readonly https://www.googleapis.com/auth/analytics https://www.google.com/analytics/feeds https://www.google.com/analytics/feeds/data https://www.google.com/analytics/feeds/"
< vary: X-Origin
< vary: Referer
< vary: Origin,Accept-Encoding
< content-type: application/json; charset=UTF-8
< date: Thu, 21 Apr 2022 08:10:43 GMT
< server: ESF
< cache-control: private
< x-xss-protection: 0
< x-frame-options: SAMEORIGIN
< x-content-type-options: nosniff
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
< accept-ranges: none
<
{
"error": {
"code": 403,
"message": "Request had insufficient authentication scopes.",
"status": "PERMISSION_DENIED",
"details": [
{
"#type": "type.googleapis.com/google.rpc.ErrorInfo",
"reason": "ACCESS_TOKEN_SCOPE_INSUFFICIENT",
"domain": "googleapis.com",
"metadata": {
"method": "google.analytics.data.v1beta.BetaAnalyticsData.RunReport",
"service": "analyticsdata.googleapis.com"
}
}
]
}
}
* Connection #0 to host analyticsdata.googleapis.com left intact
* Closing connection 0
could you please let us know why this request fails although we have authorized all the required scopes?
You are sending an access token with your request.
When you requested consent of the user in order to get that access token. You must have requested consent with one of the following scopes
https://www.googleapis.com/auth/analytics.readonly
https://www.googleapis.com/auth/analytics
With out it your access token has insufficient authentication scopes. Request access of the user again and request the proper scopes.

Curl doesn't redirect properly for some facebook profile IDs

I'm trying to hack together a bit of code (c++) that returns a current facebook display name for a given user ID. The code works perfectly fine for most IDs but for some IDs curl returns a 404 even though the same URL opens fine in a browser. Here's the bit of code related to curl.
int main(int argc, char const *argv[]) {
CURL* curl;
char userAgent[] = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13";
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
string fbID = argv[1];
string searchTerm = "https://www.facebook.com/profile.php?id=" + fbID;
cout << "searching for: " << searchTerm << endl;
if(!curl)
return 0;
else{
curl_easy_setopt(curl, CURLOPT_USERAGENT, userAgent);
curl_easy_setopt(curl, CURLOPT_URL, searchTerm.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &writeCallback);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, true);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); //tell curl to output
its progress
}
//scan the retrieved data for the name...
}
size_t writeCallback(char* buf, size_t size, size_t nmemb, void* up)
{ //i copied this function
for (int c = 0; c<size*nmemb; c++)
{
data.push_back(buf[c]);
}
return size*nmemb;
}
I don't exactly know where to look because it works for most ids but for example "1094145063" returns a 404 even though https://www.facebook.com/profile.php?id=1094145063
opens fine in a browser. (1331601579 for example works fine with my code)
I can't spot the difference between the sites.
It does however make a redirect to https://www.facebook.com/"username" which works fine once CURLOPT_FOLLOWLOCATION is set, but only on some sites. Here's the message curl outputs.
* Hostname was NOT found in DNS cache
* Trying 31.13.84.36...
* Connected to www.facebook.com (31.13.84.36) port 443 (#0)
* found 174 certificates in /etc/ssl/certs/ca-certificates.crt
* server certificate verification OK
* common name: *.facebook.com (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: EC
* certificate version: #3
* subject: C=US,ST=California,L=Menlo Park,O=Facebook\, Inc.,CN=*.facebook.com
* start date: Fri, 09 Dec 2016 00:00:00 GMT
* expire date: Thu, 25 Jan 2018 12:00:00 GMT
* issuer: C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert SHA2 High Assurance Server CA
* compression: NULL
* cipher: AES-128-GCM
* MAC: AEAD
> GET /profile.php?id=1094145063 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13
Host: www.facebook.com
Accept: */*
HTTP/1.1 404 Not Found
X-XSS-Protection: 0
public-key-pins-report-only: max-age=500; pin-sha256="WoiWRyIOVNa9ihaBciRSC7XHjliYS9VwUGOIud4PB18="; pin-sha256="r/mIkG3eEpVdm+u/ko/cwxzOMo1bk4TyHIlByibiA5E="; pin-sha256="q4PO2G2cbkZhZ82+JgmRUyGMoAeozA+BSXVXQWB8XWQ="; report-uri="http://reports.fb.com/hpkp/"
Pragma: no-cache
content-security-policy: default-src * data: blob:;script-src *.facebook.com *.fbcdn.net *.facebook.net *.google-analytics.com *.virtualearth.net *.google.com 127.0.0.1:* *.spotilocal.com:* 'unsafe-inline' 'unsafe-eval' fbstatic-a.akamaihd.net fbcdn-static-b-a.akamaihd.net *.atlassolutions.com blob: data: 'self';style-src data: blob: 'unsafe-inline' *;connect-src *.facebook.com *.fbcdn.net *.facebook.net *.spotilocal.com:* *.akamaihd.net wss://*.facebook.com:* https://fb.scanandcleanlocal.com:* *.atlassolutions.com attachment.fbsbx.com ws://localhost:* blob: *.cdninstagram.com 'self';
Cache-Control: private, no-cache, no-store, must-revalidate
Strict-Transport-Security: max-age=15552000; preload
X-Content-Type-Options: nosniff
Expires: Sat, 01 Jan 2000 00:00:00 GMT
Vary: Accept-Encoding
Content-Type: text/html; charset=UTF-8
X-FB-Debug: pA9SSTdWWx2a66QeP7Je/4ik/2a+/ZL/m/nckHKf+KoEZLloClzu+qMDzyE/B8M1PRDl4SdS19C9vIIl7f43mA==
Date: Tue, 06 Jun 2017 18:25:15 GMT
Transfer-Encoding: chunked
Connection: keep-alive
* Connection #0 to host www.facebook.com left intact
Any help would be appreciated.
(my first SO post so please don't be too harsh if i did something dumb)

New-WebServiceProxy failing to authenticate with NTLM

I'm dealing with a rather peculiar issue. We have a need to hit the Lists service on our SharePoint farm. Web authentication federated through an Oracle SSO, but we do have accounts configured for automation that can perform web requests. Using AAM, we have an "internal" URL configured for server side automation that bypasses directly to AD, and everything else gets pushed to the SSO.
Here's the code (sanitized) that I'm using to try to get the list collection.
$username = "DOMAIN\username"
$password = "somepassword"
$site = "https://sp.biz.com/sites/SiteCollection"
$credentials = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $username, (ConvertTo-SecureString $password -AsPlainText -Force)
$proxy = New-WebServiceProxy -Uri "$site/_vti_bin/Lists.asmx" -Credentials $credentials
$proxy.GetListCollection()
I'm hit with a 403 when I use that code.
Exception calling "GetListCollection" with "0" argument(s): "Server was unable to process request. ---> Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))"
If I change $site to use the internal URL (set via AAM) and run that on one of the front ends, I receive the list collection successfully. Now, at first I thought there was an issue with the account and permissions, but after running a Fiddler capture I see it not authenticating at all.
When I run the following cURL command, it authenticates and returns the list collection. Soap.xml is just the basic GetListCollection packet copied straight from the WDSL.
curl -v -u 'username':'pass' --ntlm -X POST -H "Content-Type: text/xml" --data-binary #soap.xml https://sp.biz.com/sites/SiteCollection/_vti_bin/Lists.asmx
Here's the sanitized verbose output from cURL.
* STATE: INIT => CONNECT handle 0x600056190; line 1029 (connection #-5000)
* Hostname was NOT found in DNS cache
* Trying <IPv6>...
* STATE: CONNECT => WAITCONNECT handle 0x600056190; line 1082 (connection #0)
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to sp.biz.com (<IPv6>) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: /usr/ssl/certs/ca-bundle.crt
CApath: none
* SSLv3, TLS handshake, Client hello (1):
} [data not shown]
* STATE: WAITCONNECT => PROTOCONNECT handle 0x600056190; line 1222 (connection #0)
* SSLv3, TLS handshake, Server hello (2):
{ [data not shown]
* SSLv3, TLS handshake, CERT (11):
{ [data not shown]
* SSLv3, TLS handshake, Server finished (14):
{ [data not shown]
* SSLv3, TLS handshake, Client key exchange (16):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Finished (20):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
{ [data not shown]
* SSLv3, TLS handshake, Finished (20):
{ [data not shown]
* SSL connection using TLSv1.2 / DES-CBC3-SHA
* SSL certificate verify ok.
* STATE: PROTOCONNECT => DO handle 0x600056190; line 1241 (connection #0)
* Server auth using NTLM with user 'DOMAIN\username'
> POST /sites/SiteCollection/_vti_bin/Lists.asmx HTTP/1.1
> Authorization: NTLM <snip>
> User-Agent: curl/7.39.0
> Host: sp.biz.com
> Accept: */*
> Content-Type: text/xml
> Content-Length: 0
>
* STATE: DO => DO_DONE handle 0x600056190; line 1314 (connection #0)
* STATE: DO_DONE => WAITPERFORM handle 0x600056190; line 1441 (connection #0)
* STATE: WAITPERFORM => PERFORM handle 0x600056190; line 1454 (connection #0)
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 401 Unauthorized
* Server Microsoft-IIS/7.5 is not blacklisted
< Server: Microsoft-IIS/7.5
< SPRequestGuid: <snip>
< WWW-Authenticate: NTLM <snip>
< X-Powered-By: ASP.NET
< MicrosoftSharePointTeamServices: 14.0.0.7006
< X-MS-InvokeApp: 1; RequireReadOnly
< Date: Fri, 16 Jan 2015 01:02:56 GMT
< Content-Length: 0
< Set-Cookie: BIGipServerserver_pool=<snip>; expires=Sat, 17-Jan-2015 01:02:56 GMT; path=/
<
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
* Connection #0 to host sp.biz.com left intact
* Issue another request to this URL: 'https://sp.biz.com/sites/SiteCollection/_vti_bin/Lists.asmx'
* STATE: PERFORM => CONNECT handle 0x600056190; line 1601 (connection #-5000)
* Found bundle for host sp.biz.com: 0x60006aef0
* Re-using existing connection! (#0) with host sp.biz.com
* Connected to sp.biz.com (<IPv6>) port 443 (#0)
* STATE: CONNECT => DO handle 0x600056190; line 1075 (connection #0)
* Server auth using NTLM with user 'DOMAIN\username'
> POST /sites/SiteCollection/_vti_bin/Lists.asmx HTTP/1.1
> Authorization: NTLM <snip>
> User-Agent: curl/7.39.0
> Host: sp.biz.com
> Accept: */*
> Content-Type: text/xml
> Content-Length: 353
>
} [data not shown]
* upload completely sent off: 353 out of 353 bytes
* STATE: DO => DO_DONE handle 0x600056190; line 1314 (connection #0)
* STATE: DO_DONE => WAITPERFORM handle 0x600056190; line 1441 (connection #0)
* STATE: WAITPERFORM => PERFORM handle 0x600056190; line 1454 (connection #0)
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 200 OK
< Cache-Control: private, max-age=0
< Content-Type: text/xml; charset=utf-8
* Server Microsoft-IIS/7.5 is not blacklisted
< Server: Microsoft-IIS/7.5
< SPRequestGuid: <snip>
< Set-Cookie: FedAuth=<snip>; expires=Fri, 16-Jan-2015 08:36:07 GMT; path=/; secure; HttpOnly
< X-SharePointHealthScore: 0
< X-AspNet-Version: 2.0.50727
< Persistent-Auth: true
< X-Powered-By: ASP.NET
< MicrosoftSharePointTeamServices: 14.0.0.7006
< X-MS-InvokeApp: 1; RequireReadOnly
< Date: Fri, 16 Jan 2015 01:02:56 GMT
< Content-Length: 104088
< Vary: Accept-Encoding
<
{ [data not shown]
* STATE: PERFORM => DONE handle 0x600056190; line 1626 (connection #0)
100 101k 100 101k 100 353 219k 762 --:--:-- --:--:-- --:--:-- 219k
* Connection #0 to host sp.biz.com left intact
Any assistance is greatly appreciated. I'm not opposed to a C# solution over PowerShell if the cmdlets are lacking.
01-16-2015 12:13PM EST Update - I updated the question to reflect HighlyUnavailable's suggestion and included headers from the Fiddler capture.
Here are the sanitized headers from the PowerShell script:
CONNECT sp.biz.com:443 HTTP/1.1
Host: sp.biz.com
Connection: Keep-Alive
HTTP/1.1 200 Connection Established
FiddlerGateway: Direct
StartTime: 12:14:46.372
Connection: close
------------------------------------------------------------------
GET https://sp.biz.com/sites/SiteCollection/_vti_bin/Lists.asmx HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50727.5485)
Host: sp.biz.com
Connection: Keep-Alive
HTTP/1.1 200 OK
Cache-Control: private, max-age=0
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/7.5
SPRequestGuid: <snip>
X-SharePointHealthScore: 0
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
MicrosoftSharePointTeamServices: 14.0.0.7006
X-MS-InvokeApp: 1; RequireReadOnly
Date: Fri, 16 Jan 2015 17:14:46 GMT
Connection: keep-alive
Content-Length: 9066
Set-Cookie: BIGipServerserver_pool=<snip>; expires=Sat, 17-Jan-2015 17:14:46 GMT; path=/
Vary: Accept-Encoding
------------------------------------------------------------------
GET https://sp.biz.com/_vti_bin/Lists.asmx?disco HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50727.5485)
Host: sp.biz.com
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/xml; charset=utf-8
Server: Microsoft-IIS/7.5
SPRequestGuid: <snip>
X-SharePointHealthScore: 0
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
MicrosoftSharePointTeamServices: 14.0.0.7006
X-MS-InvokeApp: 1; RequireReadOnly
Date: Fri, 16 Jan 2015 17:14:46 GMT
Connection: close
Content-Length: 747
------------------------------------------------------------------
CONNECT sp.biz.com:443 HTTP/1.1
Host: sp.biz.com
Connection: Keep-Alive
HTTP/1.1 200 Connection Established
FiddlerGateway: Direct
StartTime: 12:14:47.505
Connection: close
------------------------------------------------------------------
GET https://sp.biz.com/_vti_bin/Lists.asmx?wsdl HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50727.5485)
Host: sp.biz.com
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/xml; charset=utf-8
Server: Microsoft-IIS/7.5
SPRequestGuid: <snip>
X-SharePointHealthScore: 0
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
MicrosoftSharePointTeamServices: 14.0.0.7006
X-MS-InvokeApp: 1; RequireReadOnly
Date: Fri, 16 Jan 2015 17:14:46 GMT
Connection: close
Content-Length: 72672
Set-Cookie: BIGipServerserver_pool=<snip>; expires=Sat, 17-Jan-2015 17:14:47 GMT; path=/
Vary: Accept-Encoding
------------------------------------------------------------------
CONNECT sp.biz.com:443 HTTP/1.1
Host: sp.biz.com
Connection: Keep-Alive
HTTP/1.1 200 Connection Established
FiddlerGateway: Direct
StartTime: 12:14:48.727
Connection: close
------------------------------------------------------------------
POST https://sp.biz.com/_vti_bin/Lists.asmx HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50727.5485)
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://schemas.microsoft.com/sharepoint/soap/GetListCollection"
Host: sp.biz.com
Content-Length: 321
Expect: 100-continue
HTTP/1.1 500 Internal Server Error
Cache-Control: private
Content-Type: text/xml; charset=utf-8
Server: Microsoft-IIS/7.5
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
MicrosoftSharePointTeamServices: 14.0.0.7006
X-MS-InvokeApp: 1; RequireReadOnly
Date: Fri, 16 Jan 2015 17:14:48 GMT
Content-Length: 459
Set-Cookie: BIGipServerserver_pool=686493706.47873.0000; expires=Sat, 17-Jan-2015 17:14:48 GMT; path=/
------------------------------------------------------------------
Here are the headers for the cURL command.
CONNECT sp.biz.com:443 HTTP/1.1
Host: sp.biz.com:443
User-Agent: curl/7.39.0
Connection: Keep-Alive
Content-Type: text/xml
HTTP/1.1 200 Connection Established
FiddlerGateway: Direct
StartTime: 12:21:07.928
Connection: close
------------------------------------------------------------------
POST https://sp.biz.com/sites/SiteCollection/_vti_bin/Lists.asmx HTTP/1.1
Authorization: NTLM <snip>=
User-Agent: curl/7.39.0
Host: sp.biz.com
Accept: */*
Content-Type: text/xml
Content-Length: 0
HTTP/1.1 401 Unauthorized
Server: Microsoft-IIS/7.5
SPRequestGuid: <snip>
WWW-Authenticate: NTLM <snip>
X-Powered-By: ASP.NET
MicrosoftSharePointTeamServices: 14.0.0.7006
X-MS-InvokeApp: 1; RequireReadOnly
Date: Fri, 16 Jan 2015 17:21:07 GMT
Content-Length: 0
Set-Cookie: BIGipServerserver_pool=<snip>; expires=Sat, 17-Jan-2015 17:21:07 GMT; path=/
Proxy-Support: Session-Based-Authentication
------------------------------------------------------------------
POST https://sp.biz.com/sites/SiteCollection/_vti_bin/Lists.asmx HTTP/1.1
Authorization: NTLM <snip>
User-Agent: curl/7.39.0
Host: sp.biz.com
Accept: */*
Content-Type: text/xml
Content-Length: 417
HTTP/1.1 200 OK
Cache-Control: private, max-age=0
Content-Type: text/xml; charset=utf-8
Server: Microsoft-IIS/7.5
SPRequestGuid: <snip>
Set-Cookie: FedAuth=<snip>; expires=Sat, 17-Jan-2015 03:20:50 GMT; path=/; secure; HttpOnly
X-SharePointHealthScore: 0
X-AspNet-Version: 2.0.50727
Persistent-Auth: true
X-Powered-By: ASP.NET
MicrosoftSharePointTeamServices: 14.0.0.7006
X-MS-InvokeApp: 1; RequireReadOnly
Date: Fri, 16 Jan 2015 17:21:07 GMT
Content-Length: 66628
Vary: Accept-Encoding
------------------------------------------------------------------
You're mixing two fundamentally different techniques here.
$proxy = New-WebServiceProxy -Uri "$site/_vti_bin/Lists.asmx" -UseDefaultCredential
$proxy.PreAuthenticate = $TRUE
$proxy.Credentials = $credentials
UseDefaultCredential will attempt to pass your currently logged in Windows domain user to the site. However, you're setting Credentials as well. Normally, you would use -Credential $credentials (see http://technet.microsoft.com/en-us/library/hh849841.aspx )
The curl command you're running is more akin to using -Credential: -u is equivalent.
Try using something like $proxy = New-WebServiceProxy -Uri "$site/_vti_bin/Lists.asmx" -Credential $credentials instead.
If that doesn't work, please edit your question to include the headers being returned from the Oracle SSO connection - it could be that it simply isn't even asking for credentials.
I never ended up coming up with a solution for this, but I can explain why. In our environment we use Forms Based Authentication against our Oracle Identity Foundation SSO with SAML v1.1.
When you attempt to authenticate, it redirects you to the SSO, but the client is attempting to use NTLM against the actual Web Front Ends instead of the SSO. To make this work, you need to include the X-FORMS_BASED_AUTH_ACCEPTED: f header in your request for it to actually authenticate using NTLM against the WFE (and not the SSO).
Here's the issue: You can't add headers to New-WebServiceProxy in PowerShell (up to 4.0 -- I haven't rolled out 5 yet). The only recommendation I can make for others having issues is to follow HighlyUnavailable's suggestions, or use Invoke-WebRequest and build your SOAP calls by hand.
The only issue is that Invoke-WebRequest can chew up your encoding, so here's how I've worked around it. If anybody has a suggestion for working around the encoding issue, I'm all ears.
# Set your credentials here.
$UserName = 'BartSimpson'
$Password = '3atmMySh0rtz!'
$Domain = 'SF'
$SecurePassword = ConvertTo-SecureString -String $Password -AsPlainText -Force
$Credentials = New-Object System.Management.Automation.PSCredential (($Domain + "\" + $UserName), $SecurePassword)
# SOAP request headers and body
$BaseHeaders = #{"X-FORMS_BASED_AUTH_ACCEPTED" = 'f';
"SOAPAction" = "`"http://schemas.microsoft.com/sharepoint/soap/GetListCollection`"";
"Content-Type" = "text/xml; charset=utf-8"}
$SOAP = #"
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<GetListCollection xmlns="http://schemas.microsoft.com/sharepoint/soap/" />
</soap:Body>
</soap:Envelope>
"#
# Gives us a random temp file to pipe output to
$TmpFile = [System.IO.Path]::GetTempFileName()
Invoke-WebRequest -Uri $URL -Headers $BaseHeaders -Credential $Credentials -Method POST -Body $SOAP -OutFile $TmpFile
# Get the outfile with UTF8 encoding
[xml]$Result = Get-Content -Raw -Path $TmpFile -Encoding UTF8
# Remove the temporary file
Remove-Item $TmpFile
Seems like a long way to go, which it is, but it works if you insist on using PowerShell.
I switched to python-suds and was able to do what I needed to.
In the above code, there is extra 'S' in -Credentials.
So replace the code:
$proxy = New-WebServiceProxy -Uri "$site/_vti_bin/Lists.asmx" -Credentials $credentials
with below code:
$proxy = New-WebServiceProxy -Uri "$site/_vti_bin/Lists.asmx" -Credential $credentials

401 Authorization Required: Failed to validate oauth signature and token

I've tried using Net::Twitter::Role::OAuth to add Sign in with Twitter to my application.
I've used this successfully in the past, but not with SSL enabled, which apparently is now required by the Twitter API. I have a controller action very similar to the examples in the synopsis:
sub twitter_authorize : Local {
my($self, $c) = #_;
my $nt = Net::Twitter->new(traits => [qw/API::RESTv1_1 OAuth/], %param);
my $url = $nt->get_authorization_url(callback => $callbackurl);
$c->response->cookies->{oauth} = {
value => {
token => $nt->request_token,
token_secret => $nt->request_token_secret,
},
};
$c->response->redirect($url);
}
However, this fails at the $nt->get_authorization_url() call with a 401 Unauthorized error.
Looking at the oauth/request_token docs, I tried running the request through cURL, as follows:
curl --request 'POST' 'https://api.twitter.com/oauth/request_token' --header 'Authorization: OAuth oauth_consumer_key="xxxx", oauth_nonce="xxxx", oauth_signature="xxxx", oauth_signature_method="HMAC-SHA1", oauth_timestamp="xxxx", oauth_callback="oob", oauth_version="1.0"' --verbose
And the response is as follows:
* About to connect() to api.twitter.com port 443 (#0)
* Trying 199.16.156.104...
connected
* Connected to api.twitter.com (199.16.156.104) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using AES128-SHA
* Server certificate:
* subject: C=US; ST=California; L=San Francisco; O=Twitter, Inc.; OU=Twitter Security; CN=api.twitter.com
* start date: 2014-08-03 00:00:00 GMT
* expire date: 2016-12-31 23:59:59 GMT
* subjectAltName: api.twitter.com matched
* issuer: C=US; O=VeriSign, Inc.; OU=VeriSign Trust Network; OU=Terms of use at https://www.verisign.com/rpa (c)10; CN=VeriSign Class 3 Secure Server CA - G3
* SSL certificate verify ok.
> POST /oauth/request_token HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
> Host: api.twitter.com
> Accept: */*
> Authorization: OAuth oauth_consumer_key="xxxx", oauth_nonce="xxxx", oauth_signature="xxxx", oauth_signature_method="HMAC-SHA1", oauth_timestamp="xxxx", oauth_callback="oob", oauth_version="1.0"
>
< HTTP/1.1 401 Authorization Required
< cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0
< content-length: 44
< content-security-policy-report-only: default-src https:; connect-src https:; font-src https: data:; frame-src https: http://*.twimg.com http://itunes.apple.com about: javascript:; img-src https: data:; media-src https:; object-src https:; script-src 'unsafe-inline' 'unsafe-eval' about: https:; style-src 'unsafe-inline' https:; report-uri https://twitter.com/i/csp_report?a=NVXW433SMFUWY%3D%3D%3D&ro=true;
< content-type: text/html; charset=utf-8
< date: Tue, 21 Oct 2014 10:29:57 UTC
< expires: Tue, 31 Mar 1981 05:00:00 GMT
< last-modified: Tue, 21 Oct 2014 10:29:57 GMT
< pragma: no-cache
< server: tsa_b
< set-cookie: _twitter_sess=BAh7CDoPY3JlYXRlZF9hdGwrCD2PQTJJAToHaWQiJTE3M2Q4OWIyZWE1Nzc1%250AZmYxMjRkYmUyZDVjOTBlYjQxIgpmbGFzaElDOidBY3Rpb25Db250cm9sbGVy%250AOjpGbGFzaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--b807e4ebb8d45756e9686971b951a549d0d83b61; domain=.twitter.com; path=/; secure; HttpOnly
< set-cookie: guest_id=v1%3A141388739758201626; Domain=.twitter.com; Path=/; Expires=Thu, 20-Oct-2016 10:29:57 UTC
< status: 401 Unauthorized
< strict-transport-security: max-age=631138519
< vary: Accept-Encoding
< x-connection-hash: 54a185631d5f0b3a3a9dc46fe1f40a57
< x-content-type-options: nosniff
< x-frame-options: SAMEORIGIN
< x-mid: 0258025664ce095129d0cc294100d71a2e6e66ac
< x-runtime: 0.01294
< x-transaction: 6fad295009a89877
< x-ua-compatible: IE=edge,chrome=1
< x-xss-protection: 1; mode=block
<
* Connection #0 to host api.twitter.com left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
Failed to validate oauth signature and token
Weirdly, if I remove the oauth_callback key from the Authorization header, it works fine and I get the tokens. However, the API docs suggest that this parameter is required. Is there something wrong with how I'm passing in the oauth_callback item?
I've tried setting it to oob (which is supposed to work for "out of band" access). And I've copied the encoded URL from the API docs. It doesn't work with either.
As it works without oauth_callback, it's not a time issue on my machine, as is a commonly reported problem. I haven't tried sending the Net::Twitter request without the callback (I haven't checked if that's possible) but I assume that would fix it there too. However, I do need the request to have a valid callback URL or the user won't be redirected back to the application for the rest of the sign in flow.
Adam,
I was having a similar issue and after a thorough investigation of what other Twitter API wrapper libraries were doing discovered that the oauth_callback needed to be encoded twice.
On doing a retrospective? search to get a little more explanation/clarity I found the following SO answer Twitter Oauth URL encoding inconsistencies? which extremely succinctly explained the issue I was having.
Is this perhaps the issue you have encountered?