Using IE11 I am making a get request to SITE A:
GET http://www.test.com/?documentId=ef746317-7711-4458-8873-a73700fc1b85 HTTP/1.1
Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.2; WOW64; Trident/7.0; .NET4.0E; .NET4.0C; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 3.0.30729)
Connection: Keep-Alive
Host: www.test.com
I receive a redirect with 2 cookies:
HTTP/1.1 302 Found
Date: Wed, 15 Mar 2017 23:48:00 GMT
Content-Type: text/html; charset=UTF-8
Location: https://www.newSite.com/test/Edit/ef746317-7711-4458-8873-a73700fc1b85
Set-Cookie: Auth=EAAAAIQfMoK32BNjBypXapcJppWc==; path=/; secure
Set-Cookie: Auth=EAAAAN+xPT6eioV8LESTR6CViGIvc834gP==; path=/; secure
Cache-Control: private, s-maxage=0
Server: Microsoft-IIS/10.0
X-AspNet-Version: 4.0.30319
X-AspNetMvc-Version: 4.0
X-Powered-By: ASP.NET
P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTR STP IND DEM"
P3P: policyref="/w3c/p3p.xml", CP="IDC DSP COR IVAi IVDi OUR TST"
Content-Length: 0
IE would appear to be following the redirect and does a GET, but as you can see is not sending back the cookies:
GET https://www.newSite.com/test/Edit/ef746317-7711-4458-8873-a73700fc1b85 HTTP/1.1
Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.2; WOW64; Trident/7.0; .NET4.0E; .NET4.0C; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 3.0.30729)
Connection: Keep-Alive
Host: www.newSite.com
And then of course is the 401:
HTTP/1.1 401 Unauthorized
Content-Type: text/html
Server: Microsoft-IIS/10.0
X-Powered-By: ASP.NET
Date: Wed, 15 Mar 2017 23:48:00 GMT
Content-Length: 1293
<HTML>Blah blah blah access denied error</HTML>
I tried adding the P3P headers to force IE to send the cookies in the redirect but no dice. I have read there may be an issue with IE sending cookies in redirect when going from HTTP to HTTPS, or because the "secure" cookies are being sent back to the browser on HTTP which when redirects to HTTP sees the different domain and chokes. I cannot change the web sites as they are vendors, but I can alter the 302 being sent back to IE11 with the interface middle-ware I am working on. Any thoughts on how I can trick/force IE to send back these cookies on the redirect?
Update 1: I have tried Firefox 52, IE11, and Chrome. No browser is accepting that 302 and sending the get back with the cookies. Someone out there must understand how redirects with cookies work. No responses makes me wonder if this site is reaching the right people.
I got around my issue with sort of a hack. Since the browser would not forward cookies back in a 302 redirect, I just send back a little page that does the posting for me, instead of my interface software.
<!DOCTYPE html>
<html>
<head>
<title>Redirect</title>
</head>
<body>
<form action="https://testAPI.test.com/" method="POST">
<input name="UserName" value="Test APIUser"/>
<input name="UserEmail" value="test#test.com"/>
<input name="PatientId" value="1d11eb2e-2606-485e-ad5d-a70c00daa37a"/>
<input name="Timestamp" value="Mon, 20 Mar 2017 19:11:24 GMT"/>
84c6-a7210111648b"/>
<input name="Token" value="MRVp/pBRBJ08F8cYMavfL8 ="/>
</form>
<script language="javascript"> window.setTimeout('document.forms[0].submit()', 0);</script>
</body>
</html>
You can disable Protected Mode. Protected Mode is designed to prevent malicious software from exploiting vulnerabilities in Internet Explorer 11. This mode may also block cookies depending on the current setup.
Open Internet Explorer 11.
Click Tools and then select Internet options.
Go to Security tab.
Under Security level for this zone, clear the check box for Enable Protected Mode (requires restarting Internet Explorer).
Click OK.
Close Internet Explorer 11 and then launch it again.
Related
I am facing an issue in downloading the txt file from a website. The script below downloads the http code instead of the actual txt file and its contents.
$WebClient = New-Object System.Net.WebClient
$WebClient.DownloadFile("https://thegivebackproject.org/CheckStatus.txt", "D:\CheckStatus.txt")
Short Answer
The server is doing browser sniffing to send different responses based on the User-Agent header in your request. You can get the response you want by sending a canned user agent string:
$useragent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36"
Invoke-WebRequest -URI https://thegivebackproject.org/CheckStatus.txt -OutFile c:\temp\CheckStatus.txt -UserAgent $useragent
Long Answer
The server responding to the url you're hitting is doing browser sniffing to decide what content to return. If you give it a User-Agent header that it recognises it will return the response you're expecting (i.e. the literal text "Azeemkhan-WaseemRaza").
If you don't include a User-Agent header (and $WebClient.DownloadFile doesn't include one), the server is responding with a html page instead.
You can see this behaviour yourself if you install a HTTP trace tool like Fiddler. When you hit the page in a browser you see this HTTP request and response pair:
request
GET https://thegivebackproject.org/CheckStatus.txt HTTP/1.1
Host: thegivebackproject.org
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36
Sec-Fetch-User: ?1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: SPSI=ee952ba44e33e958f963807ede78624b
response
HTTP/1.1 200 OK
Server: nginx
Date: Tue, 12 Nov 2019 08:13:57 GMT
Content-Type: text/plain
Content-Length: 20
Connection: keep-alive
Last-Modified: Thu, 07 Nov 2019 16:15:48 GMT
Accept-Ranges: bytes
X-Cache: MISS
Azeemkhan-WaseemRaza
but when you use $WebClient.DownloadFile you see this instead:
request
GET https://thegivebackproject.org/CheckStatus.txt HTTP/1.1
Host: thegivebackproject.org
response
HTTP/1.1 200 OK
Server: nginx
Date: Tue, 12 Nov 2019 08:14:21 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: SPSI=9c24f8993046ef610e25cc727c4a4ae2; Path=/
Set-Cookie: adOtr=obsvl; Expires=Thu, 2 Aug 2001 20:47:11 UTC; Path=/
Set-Cookie: UTGv2=D-h4d40f620bfdd6c3b77b035ee99f96621134; Expires=Wed, 11-Nov-20 08:14:21 GMT; Path=/
cache-control: no-store, no-cache, max-age=0, must-revalidate, private, max-stale=0, post-check=0, pre-check=0
Vary: Accept-Encoding
X-Cache: MISS
Accept-Ranges: bytes
5908
<!doctype html>
<head>
<meta charset="utf-8">
<meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<title>StackPath</title>
<style>
* {
box-sizing: border-box;
}
... etc...
The workaround is to include a recognised User-Agent header in your request, which is easier to to if you use Invoke-WebRequest like #BiNZGi suggested, rather than the WebClient class - see the "short answer" above for the code.
Also, note that this sniffing behaviour with the User-Agent is specific to "thegivebackproject.org" website and isn't necessarily true for other websites - you don't always need to include a User-Agent header as a rule of thumb.
You can use the easier Invoke-WebRequest:
Invoke-WebRequest -URI https://thegivebackproject.org/CheckStatus.txt -OutFile D:\CheckStatus.txt
I have a small site where I have a mailing list contact form in an iFrame, and once its submitted, a callback page I registered with the mailing list service is called, displaying in the iFrame and asking the user to check their email. The page I registered is http://mydomain.com/verify.html. In vertify.html I use "window.parent.document.getElementById('lightbox4').style.display='none';" to close the lightbox div that contains the I frame. This all works well, as long as the user initially visits http://mydomain.com, but if they visit http://www.mydomain.com, then calling "window.parent.document.getElementById('lightbox4').style.display='none';" doesn't work, because its a cross domain request.
So, no problem I thought, I'll just create a redirect rule to convert calls from www.mydomain.com, to mydomain.com. But now I'm getting the error "This webpage has a redirect loop" when I try to go to either www.mydomain.com or mydomain.com. In IIS7, I have two bindings, one for mydomain.com and one for www.mydomain.com. My DNS zone has an A record for mydomain.com, and a CNAME for www.mydomain.com.
Am I doing something stupid here? Is there ome way to debug this? I can see in Firefox, using the Live HTTP headers plugin, the URL is redirected properly from www.mydomain.com to mydomain.com , but then tries to keep trying to redirect mydomain.com to mydomain.com, creating the endless loop:
http://www.mydomain.com/
GET / HTTP/1.1
Host: www.mydomain.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0) Gecko/20100101 Firefox/20.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
HTTP/1.1 302 Redirect
Content-Type: text/html; charset=UTF-8
Location: http://mydomain.com/
Server: Microsoft-IIS/7.0
X-Powered-By: ASP.NET
Date: Sun, 21 Apr 2013 15:20:12 GMT
Content-Length: 150
----------------------------------------------------------
http://mydomain.com/
GET / HTTP/1.1
Host: mydomain.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0) Gecko/20100101 Firefox/20.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
HTTP/1.1 302 Redirect
Content-Type: text/html; charset=UTF-8
Location: http://mydomain.com/
Server: Microsoft-IIS/7.0
X-Powered-By: ASP.NET
Date: Sun, 21 Apr 2013 15:20:12 GMT
Content-Length: 150
----------------------------------------------------------
http://mydomain.com/
GET / HTTP/1.1
Host: mydomain.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0) Gecko/20100101 Firefox/20.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
HTTP/1.1 302 Redirect
Content-Type: text/html; charset=UTF-8
Location: http://mydomain.com/
Server: Microsoft-IIS/7.0
X-Powered-By: ASP.NET
Date: Sun, 21 Apr 2013 15:20:12 GMT
Content-Length: 150
----------------------------------------------------------
and it keeps going until "This webpage has a redirect loop" is displayed
I expect I have to create a new virtual directory for www.mydomain.com and then redirect that to mydomain.com, but that seems awkward.
I have three websites hosted (example1.com, example2.com, example3.com) on a server. There is a page (test.php) on example1.com with just code below inside it:
<?php
header('Location:http://example2.com/a.php');
?>
When I browse test.php it goes to http://example1.com/a.php . it doesn't understand it is another domain url, it tried to find the page on itself.
but when I put http://google.com instead of example2.com/a.php it works correct. I really get confused.
What is the problem ? Should I set some configuration on the server?
( I am administrator of the hosting server ).
Ps. The server is behind a pound server.
Edited:
Here's the Firebug Net output for example1.com/test.php
Response Headers:
HTTP/1.1 302 Found
Date: Tue, 09 Oct 2012 09:03:34 GMT
Server: Apache/2.2.16 (Debian)
Location: http://example1.com/a.php
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 21
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8
Request Headers:
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding gzip, deflate
Accept-Language en-us,en;q=0.5
Connection keep-alive
Cookie mycookie
Host example1.com
User-Agent Mozilla/5.0 (X11; Linux i686; rv:14.0) Gecko/20100101 Firefox/14.0.1
the problem is solved. it was because of pound server configuration. 'RewriteLocation' entry in pound server configuration must be set to 2 to this server doesn't change the redirect location.
anyway, thank you for answering.
This code works great for all browsers except Internet Explorer.
Basically when the redirect is sent to IE it just request the exact same URL again from my server, it just ignores the redirect.
Using the 3.1.1 code from Naitik Shah
Here's the code:
// $g_facebook is declared earlier and given app id and secret
$par[ 'scope' ] = array( 'publish_stream' , // publish to the user's stream
'offline_access' , // access these functions when the user is offline
// 'user_status' , // get the user's latest status
// 'read_stream' , // read the user's stream
'email' , // provides the user's email address
'user_groups' , // provides the user's groups
// 'sms' , // send and receive txt w/ user
'publish_actions', // publish scores and achievements
);
header( 'Location: ' . $g_facebook->getLoginUrl( $par ) );
exit( );
Here's what happens on the wire (picked it up with tcpdump):
GET /fork HTTP/1.1
Accept: image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/x-shockwave-flash, application/x-ms-application, application/x-ms-xbap, application/vnd.ms-xpsdocument, application/xaml+xml, */*
Accept-Language: en-us
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET4.0C)
Accept-Encoding: gzip, deflate
Host: fbar.toolsteam.com
Connection: Keep-Alive
Cookie: PHPSESSID=7f32d7e4acd63696bd8d0998913f608c; PHPSESSID=e30076106b21e40142397219283fd55f
HTTP/1.0 302 Moved Temporarily
Date: Mon, 07 May 2012 07:36:12 GMT
Server: Apache
X-Powered-By: PHP/5.3.10
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=a9f17a1119dc262bef693d2d39a15317; expires=Tue, 07-May-2013 07:36:12 GMT; path=/
Location: http://www.facebook.com/dialog/oauth?client_id=336243633108439&redirect_uri=http%3A%2F%2Ffbar.toolsteam.com%2Ffork&state=b52dd5dd08e0058e28ae8734f269cd77&scope=publish_stream%2Coffline_access%2Cemail%2Cuser_groups%2Cpublish_actions
Content-Length: 0
Content-Type: text/html
X-Cache: MISS from base
X-Cache-Lookup: MISS from base:3128
Via: 1.1 base:3128 (squid/2.7.STABLE9)
Connection: keep-alive
When IE sees the 302 it just sends the original request again and again. It never follows the redirect to facebook.
As said before, Chrome and Firefox have no problems.
Ideas?
The answer was in the request headers:
Cookie: PHPSESSID=7f32d7e4acd63696bd8d0998913f608c; PHPSESSID=e30076106b21e40142397219283fd55f
There are two servers involved in this facebook auth, one is the originating website, the second is an intermediate server that negotiates the facebook permissions. The facebook server is a sub-domain of the primary site.
Turns out both of them were starting php sessions. The facebook server's cookie was at the sub-domain scope, the primary site's cookie was at the top-domain scope.
For whatever reason IE couldn't handle sending the same cookie twice on a request to the facebook server - it handled the transaction just fine but for whatever reason would just re-request the same URL and ignore the 302 redirect. IE is like that.
I switched the session variable name on the facebook server and the problem disappeared.
I have two possible flows:
ConsoleClient -(1)-> ApplicationServer
or
SilverlightClient -(2)-> WebServer -(3)-> ApplicationServer
Fiddler successfully captures the HTTP traffic on the (1) and the (2), but not on the (3). Here is a sample capture on (1):
POST /WcfDemo/ws HTTP/1.1
Content-Type: application/soap+xml; charset=utf-8
Host: il-mark-lt
Content-Length: 521
Expect: 100-continue
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing"><s:Header><a:Action s:mustUnderstand="1">http://tempuri.org/IWcfDemoService/Add</a:Action><a:MessageID>urn:uuid:d7fde351-12fd-4872-bc26-52ff97f126e9</a:MessageID><a:ReplyTo><a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address></a:ReplyTo><a:To s:mustUnderstand="1">http://il-mark-lt/WcfDemo/ws</a:To></s:Header><s:Body><Add xmlns="http://tempuri.org/"><x>4</x><y>5</y></Add></s:Body></s:Envelope>
HTTP/1.1 200 OK
Content-Length: 399
Content-Type: application/soap+xml; charset=utf-8
Server: Microsoft-HTTPAPI/2.0
Date: Sat, 17 Sep 2011 20:57:16 GMT
<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing"><s:Header><a:Action s:mustUnderstand="1">http://tempuri.org/IWcfDemoService/AddResponse</a:Action><a:RelatesTo>urn:uuid:d7fde351-12fd-4872-bc26-52ff97f126e9</a:RelatesTo></s:Header><s:Body><AddResponse xmlns="http://tempuri.org/"><AddResult>9</AddResult></AddResponse></s:Body></s:Envelope>
And here is an example of (2):
POST /WcfDemoService.svc/ws HTTP/1.1
Host: localhost:56970
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8,application/json
Accept-Language: fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.6,he-IL;q=0.5,he;q=0.4,ru-RU;q=0.3,ru;q=0.1
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Referer: http://localhost:56970/ClientBin/SilverlightClient.xap
Content-Length: 581
Content-Type: application/soap+xml; charset=utf-8
<s:Envelope xmlns:a="http://www.w3.org/2005/08/addressing" xmlns:s="http://www.w3.org/2003/05/soap-envelope"><s:Header><a:Action s:mustUnderstand="1">http://tempuri.org/IWcfDemoService2/Add</a:Action><a:MessageID>urn:uuid:e8420d3e-f568-49ce-bfc7-5631d5bf3fd0</a:MessageID><a:ReplyTo><a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address></a:ReplyTo><a:To s:mustUnderstand="1">http://localhost:56970/WcfDemoService.svc/ws</a:To></s:Header><s:Body><Add xmlns="http://tempuri.org/"><x>11</x><y>22</y><serverChannelKind>ws</serverChannelKind></Add></s:Body></s:Envelope>
HTTP/1.1 200 OK
Server: ASP.NET Development Server/10.0.0.0
Date: Sat, 17 Sep 2011 20:59:23 GMT
X-AspNet-Version: 4.0.30319
Content-Length: 401
Cache-Control: private
Content-Type: application/soap+xml; charset=utf-8
Connection: Close
<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing"><s:Header><a:Action s:mustUnderstand="1">http://tempuri.org/IWcfDemoService2/AddResponse</a:Action><a:RelatesTo>urn:uuid:e8420d3e-f568-49ce-bfc7-5631d5bf3fd0</a:RelatesTo></s:Header><s:Body><AddResponse xmlns="http://tempuri.org/"><AddResult>33</AddResult></AddResponse></s:Body></s:Envelope>
Now, I am absolutely sure the (3) does get through. So, it all boils down to some misconfiguration on the WebServer, but I cannot nail it. The Web server is just a trivial ASP.NET application hosted within IIS. It even has the following lines in the web.config:
<system.net>
<defaultProxy>
<proxy bypassonlocal="false" usesystemdefault="true" />
</defaultProxy>
</system.net>
Still, this does not work.
To further strengthen my suspicion on the web server configuration, I have checked the SilverlightClient --> ApplicationServer flow and it is captured just fine.
I am using the Asp.Net development server.
Edit
Running procmon reveals that the following suspicious registry key is consulted (amongst others):
HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\ProxyBypass
And on my machine it was set to 1. I have changed it to 0 and seems like it solved my issue. The only problem is that when I change it back to 1 Fiddler continues to capture the problematic leg! Very interesting.
Anyway, I am satisfied, for now.
You are calling "localhost" right?
Fiddler is not able to capture the local traffic if you are using "localhost" as hostname.
Solutions:
Use servername (e.g. myserver)
Use ip4.fiddler (e.g. http://ipv4.fiddler:8787)
Not sure if these are causing it ... but,
A few things to check:
In IIS7 the appPool has a loadUserProfile setting. It causes the session to load a user profile which means it can get system proxy settings.
Check the code making the request from the webServer - even if you configure to use the system proxy and bypass onLocal (which only applies to names without dots in it), code making the request can still explicitly set to use or not to use a proxy.
Far fetched but you may want to play with the account the appPool runs as - local account with profile vs. Network Service.
Hope that helps - these network things have a lot of variables between two points :)