I'm trying to scrape a website that first asks for a certificate. After you are authenticated it then will ask you to provide a username and password. Anyway I'm trying to use fiddler to see what i need to replicate and when i turn on it to "Capture traffic" and then I go to internet explorer and try to access the site it says " Internet Explorer cannot display the webpage " then when i turn off fiddler and try it again it brings me to the login page.
I've tried this link and i tried everything (Turning it into a .cer file and dragging into the folder also tried copy and pasting the code into the rule file to accept the certificate) and neither way worked. Does anyone have any other ideas.
Am I suppose to replace the fiddler root certificate somehow? or is there any solution that someone can suggest?
I figured out this problem. Just incase this trips someone else up. The oSession bit of code needed to be the first line after the OnBeforeRequest line. I don't remember where it was before but it didn't work. Example:
static function OnBeforeRequest(oSession: Session) {
oSession["https-Client-Certificate"]= "C:\\Yourcerfilelocation.cer";
//I omitted the standard code
}
Edit: OnBeforeRequest is a better method to place it instead of OnPeekAtRequestHeaders.
Related
I'm new to Fiddler and have run across something that seems strange to me. If I select an entry and then click Replay, I get different behavior from when I drag an entry into the Composer window and click Execute.
Should the different behavior between these two methods of re-making a request be different?
Note: I called the second method "cloning" a request because the Composer window says "You can clone a prior request by dragging and dropping a session from the Web Sessions list)."
What is the "different behavior" specifically?
The two operations you describe should behave the same way unless the server returns a redirect or an authentication challenge, in which case preferences will control whether Fiddler automatically authenticates and/or follows the redirect.
Please feel free to email me (Help > Send Feedback) details and/or screenshots of the difference you see.
I am trying to access Soundcloud from a local HTML page on my laptop. I am stuck at the part of hosting "callback.html" as a redirect_uri. The script I am trying to run is the basic Authenication JavaScript from the Soundcloud documentation page:
<script src="http://connect.soundcloud.com/sdk.js"></script>
<script>
// initialize client with app credentials
SC.initialize({
client_id: 'my_client_id',
redirect_uri: 'http://127.0.0.1/Users/Maria/Documents/SoundcloudClient/callback.html'
});
// initiate auth popup
SC.connect(function() {
SC.get('/me', function(me) {
alert('Hello, ' + me.username);
});
});
</script>
This script gets me to the connect pop-up when I launch the page in Chrome and Firefox.
But, once I have logged in as a Soundcloud user, I get the following error:
Oops! Google Chrome could not connect to 127.0.0.1
If I change my redirect_uri to localhost I get the same error.
If I try:
files:///C:/Users/Maria/Documents/SoundcloudThinClient/callback.html
I get a similar error.
I also tried:
ocalhost:3000
and:
localhost:8080
even though I'm not sure what would be listening on those ports.
So, basically, I'm asking what path do I put for callback.html in order for this to work?
I confess I don't know how the redirct_uri actually functions. I looked at the Oauth pages for it, but I don't understand them. I am beginning to think that I can't simply create an HTML page, paste the JavaScript, create a callback.html file and have this work, even though the SC documentation seems to say that this is possible. If so, what steps am I missing?
I am beginning to attempt this. I believe you have to go to the developer site and sign up as having an app. The redirect uri is asked for and the form gives you an API key you can use in your app.
I'm using drupal so, perhaps adding the oath module and using Php to add the api key might work well.
I had the same problem and I think I solved it.
Morning-after-edit: I posted this dead-tired after working towards a solution through the night. Now, the day after, I realize that you were speaking about the general problem, whereof I face a very particular instance. The following only applies directly to registering soundcloudlabs' soundcloud-group-recorder: https://github.com/soundcloudlabs/soundcloud-group-recorder. There is probably a more general principle lurking behind there, though:
First: yes, you do have to register the app as your own at Soundcloud. At least I presumed so. And doing that, you must register correctly where on your server you will place the callback.html file. Take the ClientID assigned to your app and use that in the API intialize procedure.
Now, I'm a novice and know very little coding. But I started looking around in the main file, application.js.
At the top of the file there are two instances of client_id and redirect_uri each. I'm not sure if that serves a purpose or if one is technically superfluous. Through trial and error I found out that replacing the second instance of each with my own data worked.
Then there is groupId and groupUrl, both of which should contain your info, within quotation marks.
After a lot of trial and error, still having trouble getting the thing to run, I looked around and saw that, whereas early in the file, client_id was hooked within SC.initialize, redirect_uri was not. Under the line:
client_id: CLIENT_ID
I added:
redirect_uri: REDIRECT_URI
– with a customary comma in between. And that's it. It runs.
I'd like to create an application that can integrate with Twitter.
So I went to the website: https://dev.twitter.com/apps/new to create an application.
But the WebSite: field always failed. It said that "Website: Not a valid URL format".
I tried different "valid" URLs, but failed.
Do you have any ideas on the twitter application?
Thanks in advance.
Michael
I was trying to set localhost URL and got the same error message.
After replacing
http://localhost
http://localhost/twitter/callback
with
http://127.0.0.1
http://127.0.0.1/twitter/callback
it let me saved my test application
Mine is working. Save it in this format http://url.com. Even this domain.com works.
You can try doing it in a different browser, logout and login.
In my case the problem was in trailing whitespace from copy/paste.
One easy mistake to make is both the callback url and website need to start with http://
This worked for me:
WebSite: http://example.com
CallbackURL: http://example.com/auth/twitter/callback/
I'm able to make applications if I include "http://" at the beginning. Also it might be something browser related (I'm using Chrome) so check that.
I was having the same Issue, then after googling, someone said that:
"I think your website and callback should match. Some examples:
website: http://www.quoteicon.com
callback: http://www.quoteicon.com/twitter/callback "
summarizing the field callback has to be prefixed with the website, otherwise it gets error that the URL is invalid!
and they dont accept local IPs to the webSite fields.
It worked for me, hope this help you.
Old question but I ran into this today. Twitter does not allow localhost or IP addresses in the URL (plus it does require http:// or https:// as a prefix).
However this doesn't help with local debugging.
So an easy workaround is to add something like myapp.here.com in your /etc/hosts file
127.0.0.1 myapp.here.com
and then enter http://myapp.here.com and http://myapp.here.com/auth/twitter in the fields.
That should allow local debugging. It is odd as Facebook makes this a bit easier and it seems like a common task for debugging these types of integrations.
You could even override your actual domain locally if you wanted to use the same Twitter App ID.
Be aware of localhost, if you are developing on local machine. Twitter doesn't allow url with http://localhost:3000 or without http.
I'm wonder, why ;] Hope, They will add it.
I just found that I didn't have a problem with my callback url at all, but the error message was for the Website url!
Tried everything now, eventually only https://example.com/auth worked.
I simply added www and the error was gone.
Previously
http://twitter.com/...
After
http://www.twitter.com/...
This issue is resolved when I give my Website & Callback URL same htt://URL.Hope it will help to resolve your solution.
I have a toy facebook app I'm playing with so I can understand how it all works. It's fine if you go the the app like this: http://apps.facebook.com/pushup-challenge/ (and connect it). But if you then go to it from your facebook page, FB uses the URL http://apps.facebook.com/pushup-challenge/?ref=bookmarks.
In my log file, I see that FB is POSTing the data and including the /?ref=bookmarks to it's call to my codeigniter system. This is causing it to either say "invalid URI parameters" or give me a 404, depending on if I've edited the system/core/URI.php file to add rawurlencode() to a particular call.
I've tried using mod_rewrite to get rid of the query_string, too, but since it's POSTing, it doesn't appear to be working (though I'm not exactly sure why).
Has anyone else run into this? How did you fix it?
Thanks in advance,
Hans
try $config['uri_protocol'] = “PATH_INFO”; and set enable_query_strings = TRUE
or
set
$config['permitted_uri_chars'] = 'a-z 0-9~%.:_\-?=';
in config.php
Because it isn't calling your file by name (just ?ref=bookmarks) the server runs thru the standard default files: index.htm, index.html, index.asp. Because you need to accept a POST, you need a server that allows POSTs to htm & html if you choose to use those. Index.asp will accept POSTs on most servers, and that works for me.
SOLUTION: Add a file (index.asp), that calls the real app that you named in the App settings.
I'm getting this error... What is puzzling me is the fact that i do not have any type of security enabled for the page i'm invoking...
WebClient myWebClient = new WebClient();
myWebClient.UploadData(myUrl,"POST", myByteArray);
When i access directly to the page ( on ie ) it works fine, with the WebClient it's not working.
Anyone has a clue on what can be wrong?
IE supports integrated security, which will use your current login credentials as the username and password, so you don't have to manually enter it.
If you are using IE, use Firefox and see if you are prompted for a username and password. Or, in IE, you can go to Tools, Options, Advanced, and uncheck Integrated Windows Authentication.
Most likely, you'll have to configure the web application to support anonymous access, or you'll have to implement security in your WebClient.
If you are struggling to get to the bottom of it Fiddler might shed some light. You will have to manually edit your code to go through the Fiddler proxy, but once that's done you can compare the IE result to the code one.