So I want to make my own Internet Filter. Don't worry, i'm not asking for a tutorial. I'm just wondering if Fiddler would make a good backbone for it. I'm a little worried because it seemed that there's a few things Fiddler can't always pick up - or that there are workarounds. So, my question:
Would Fiddler grab all web data? i.e, chats, emails, websites, etc.
Are there any known workarounds?
Any other reasons not to use it?
Thanks!
I think you mean FiddlerCore rather than Fiddler. Fiddler(Core) is a web proxy meaning it captures HTTP/HTTPS traffic; it won't capture traffic that uses other protocols (e.g. IRC, etc). To capture traffic from other protocols, you'll need a lower-level interception point (e.g. a Windows Firewall filter) which will capture everything, but it will not be able to decrypt HTTPS traffic, and parsing / modifying the traffic will prove MUCH harder.
Related
If I have a static site on AWS S3 (and maybe using CloudFront) that's pretty cool, because it scales easily, and has zero-downtime deployments, because you're just updating static assets, and gets distributed to edge locations, woohoo!
But, if I wanted to have a little live chat support feature using Socket.io on the contact page, how would I tell Amazon to deal with Websockets? Could I use Route53 to do something different with Websocket requests to a particular domain, like redirect them to Lambda? (Lambda can't run socket.io can it?)
Similar to your other question, the answer here involves the fact that DNS is not involved in path resolution, so Route 53 is not a factor in this question.
Socket.io is almost certainly going to require a server and connecting through CloudFront seems unlikely.
Although I am not versed in socket.io's underlying transport protocol(s?), I don't see a way around this. CloudFront is a reverse proxy that only supports proper, standard HTTP request/response behavior, which is not well-suited to real-time event-oriented operations. CloudFront does not support websockets, though Socket.io may not need them and may have the flexibility to fall back to a compatible behavior but it will -- at best -- be suboptimal if possible at all, because even with long-polling (inefficient), you're limited to under 30 seconds for a single response, because CloudFront has a fixed timeout timer of 30 seconds, that cannot be modified.
Similarly, Lambda functions accessed through API Gateway are suited only to handling a single HTTP request/response cycle, not anything persistent and no intrinsic mechanism for handling "state" across requests.
My assumption going in would be that you'd need one or more servers behind an ELB Classic Load Balancer with SSL and operating in TCP mode, using a subdomain of your site's domain, connecting browsers to the back-end for persistent connections.
Even if this answer is helpful, I'm honestly not certain that it is sufficiently helpful... so you may wish to hold off on accepting it, since someone may come along and offer an answer that delves more deeply into the internals of socket.io and how that is going to interoperate with CloudFront, if such interoperation is possible.
I want to setup a personal videoconferencing service for my family, friends and myself. The main problem I have with current options is that they are either closed-source and centralized (GG hangouts, skype) or open-source but not working in corporate environment or in hotels (due to strict firewalling rules and the "Skype is going through, if you want VOIP use that" kind of netadmin reaction).
I have two solutions then. Either setup a STUN/TURN relay server and use XMPP and SIP as I used to, but that would require my friends to setup that too. Or setup a whole VOIP server. 2 solutions come to mind: SIP and XMPP. Though to my knowledge, each of them ultimately uses the (S)RTP/RTCP protocol.
And that's the problem. Out of the specific signaling part used by the two of them, I really can't figure out the difference between them, their typical use case.
I think you're right in that as far as setting up a video conferencing system XMPP and SIP are equivalent. They both are signalling only protocols and the media sessions they set up typically use RTP (although they can both be used to set up any kind of session you want but RTP is the norm).
The biggest problem is also going to be the one you mention about getting video streams out of a corporate firewall. Skype overcomes this obstacle by sending it's media over an SSL connection and is thus able to get through firewalls. Theoretically you could do the same with RTP and in the past I once used openvpn connections with a SIP client to test some audio calls. My experience wasn't great as the audio was very choppy, assumedly as a result of all the extra packaging that is required to get the high volume of small audio packets from one end to the other. That was nearly a decade ago though so perhaps with the better CPU and bandwidth resources available now it would work better.
Personally I think I'd stick with Skype as it's going to be a big hassle to set up your own system. If you were to go ahead with your own the first option I would try would be Asterisk combined with openvpn so that if the clients were behind a firewall or had NAT issues they could connect over it.
So, I have read that it is possible to trace the IP of a Facebook friend while talking to him on chat by using Fiddler and Firebug. Now, as far as I am know, Facebook uses HTTPS and all in all, I cannot seem to get anything precise from Fiddler.
May anyone be kind enough to explain if this is really possible and if so, how the process goes?
I dont have a direct answer to your question, but i can give you some guidelines:
If the chat is working as a peer-to-peer network (which i highly doubt) you can trace the incoming tcp/udp connection and search it for the message using a sniffing program (like wireshark) and from there fetch their IP.
If the chat is based on that every message goes through a server (which is probably based on SOMM. not that it matters. (Server Oriented Messaging Model)). now if it is that way there is virtually no way to figure out the IP because the servers are acting as a proxy and masking the original IP, however. if FB includes some sort of meta data (which they do for location for phones etc.) that might contain the sender IP and some other stuff like the mac address etc etc. now i'm not sure of that but its a good place to look.
If you want help with firebug and all those other html/http/browser development tools there are plenty of tutorials out there. If you already know how to use it you might want to check the resources that's loaded when a message is sent. From experience i can tell that when a comment is added to something a whole bunch of crap happens that append stuff to the current html document. but i have never ever seen some meta data through that. anyway it's a good place to start.
We want to use a web service in our app which obviously requires to call a URL. It's not HTTPS, just plain old HTTP, using NSURLConnection.
The problem is: This web service is VERY expensive and every thousand calls costs us real money. The fear is that someone could figure out which URL we call and then misuse that, letting the costs explode. There is no way for us to track if a call to that web service was legitimate.
We're calculating based on how many apps we sell, multiplied by an assumption of how often that app will be used per user in average. We have some good statistics on which we base our assumptions.
Are there known ways of figuring out which URL an app is calling on the Internet to retrieve information?
You could easily use a network sniffer while the phone is on WiFi to figure out this information. It sounds like it is actually critical that you use SSL with some sort of secure token in the URL.
If this is not an option perhaps you can provide your own proxy service that would use SSL and security tokens? Proxy also grants the ability to throttle requests and block users known to be malicious. Throttling puts an upper bound on the expense each user can incur within a given time interval. Another benefit of a proxy is that it allows one to gather statistics and measure the costs incurred by different users facilitating malicious user detection and business planning. Proxy could also save you some money if the service behind it is stateless by adding a cache that would remove a lot of expensive calls.
If the Web service is not encrypted, it would be trivial to use a proxy to intercept the Web requests made by the phone. If the expensive Web service does not offer at least some form of basic authentication, I would seriously reconsider including its URL in a public app.
Using plain URLs is a sure way of letting script kiddies run you out of business. If there is no way for you to track if a call to the expensive web service was legitimate, set up your own web service that fronts the real web service to make sure that your own web service can verify the legitimacy of the call before forwarding the request to the real web service.
Yes, there's plenty of ways to do this. For one example, hook up the iPhone to a wifi network, in which the router has a transparent proxy. Examine the proxy's logs. You'll see all URLs. Depends how determined your users are, but this is rather easy.
Ignoring the fact that people who jailbreak their devices could possibly look at your application, I believe it is possible to examine traffic like any other device (laptop, tablet, etc.) if someone was sniffing traffic over a WiFi hotspot using applications such as WireShark. However, I doubt there would be much risk of this over a cellular 3G network.
Good question.
As many have said, yes, it's easy to figure out the urls your app requests.
Note about HTTPS:
But since you are using HTTPS you are okay because over HTTPs the domain will be obscured to the IP address, and people cannot see the URL query string parameters. For example, if your URL was https://somewebsite.com?uid=mylogin&pass=mypass, they definitely won't be able to see "uid=mylogin&pass=mypass", and they probably can only see the IP address, not the domain name itself. (see https://serverfault.com/questions/186445/can-an-attacker-sniff-data-in-a-url-over-https)
Sidenote:
Might be safe to assume that Apple performs some sort of HTTP request diagnostics when they review your app -- which would make sense because it's in their best interest to try and figure out what your app does from many angles.
I am trying to build an application that can inform a user about website specific information whenever they are visiting a website that is present in my database. This must be done in a browser independent way so the user will always see the information when visiting a website (no matter what browser or other tool he or she is using to visit the website).
My first (partially successful) approach was by looking at the data packets using the System.Net.Sockets.Socket class etc. Unfortunately I discoverd that this approach only works when the user has administrator rights. And of course, that is not what I want. My goal is that the user can install one relatively simple program that can be used right away.
After this I went looking for alternatives and found a lot about WinPcap and some of it's .NET wrappers (did I tell you I am programming c# .NET already?). But with WinPcap I found out that this must be installed on the user's pc and there is nog way to just reference some dll files and code away. I already looked at including WinPcap as a prerequisite in my installer but that is also to cumbersome.
Well, long story short. I want to know in my application what website my user is visiting at the moment it is happening. I think it must be done by looking at the data packets of the network but can't find a good solution for this. My application is build in C# .NET (4.0).
You could use Fiddler to monitor Internet traffic.
It is
a Web Debugging Proxy which logs all HTTP(S) traffic between your computer and the Internet. Fiddler allows you to inspect traffic, set breakpoints, and "fiddle" with incoming or outgoing data. Fiddler includes a powerful event-based scripting subsystem, and can be extended using any .NET language.
It's scriptable and can be readily used from .NET.
One simple idea: Instead of monitoring the traffic directly, what about installing a browser extension that sends you the current url of the page. Then you can check if that url is in your database and optionally show the user a message using the browser extension.
This is how extensions like Invisible Hand work... It scans the current page and sends relevant data back to the server for processing. If it finds anything, it uses the browser extension framework to communicate those results back to the user. (Using an alert, or a bar across the top of the window, etc.)
for a good start, wireshark will do what you want.
you can specify a filter to isolate and view http streams.
best part is wireshark is open source, and built opon another program api, winpcap which is open source.
I'm guessing this is what you want.
capture network data off the wire
view the tcp traffic of a computer, isolate and save(in part or in hole) http data.
store information about the http connections
number 1 there is easy, you can google for a winpcap tutorial, or just use some of their sample programs to capture the data.
I recomend you study up on the pcap file format, everything with winpcap uses this basic format and its structers.
now you have to learn how to take a tcp stream and turn it into a solid data stream without curoption, or disorginized parts. (sorry for the spelling)
again, a very good example can be found in the wireshark source code.
then with your data stream, you can simple read the http format, and html data, or what ever your dealing with.
Hope that helps
If the user is cooperating, you could have them set their browser(s) to use a proxy service you provide. This would intercept all web traffic, do whatever you want with it (look up in your database, notify the user, etc), and then pass it on to the original location. Run the proxy on the local system, or on a remote system if that fits your case better.
If the user is not cooperating, or you don't want to make them change their browser settings, you could use one of the packet sniffing solutions, such as fiddler.
A simple stright forward way is to change the comupter DNS to point to your application.
this will cause all DNS traffic to pass though your app which can be sniffed and then redirected to the real DNS server.
it will also save you the hussel of filtering out emule/torrent traffic as it normally work with pure IP address (which also might be a problem as it can be circumvented by using IP address to browse).
-How to change windows DNS Servers
-DNS resolver
Another simple way is to configure (programmaticly) the browsers proxy to pass through your server this will make your life easier but will be more obvious to users.
How to create a simple proxy in C#?