I am trying to build an application that can inform a user about website specific information whenever they are visiting a website that is present in my database. This must be done in a browser independent way so the user will always see the information when visiting a website (no matter what browser or other tool he or she is using to visit the website).
My first (partially successful) approach was by looking at the data packets using the System.Net.Sockets.Socket class etc. Unfortunately I discoverd that this approach only works when the user has administrator rights. And of course, that is not what I want. My goal is that the user can install one relatively simple program that can be used right away.
After this I went looking for alternatives and found a lot about WinPcap and some of it's .NET wrappers (did I tell you I am programming c# .NET already?). But with WinPcap I found out that this must be installed on the user's pc and there is nog way to just reference some dll files and code away. I already looked at including WinPcap as a prerequisite in my installer but that is also to cumbersome.
Well, long story short. I want to know in my application what website my user is visiting at the moment it is happening. I think it must be done by looking at the data packets of the network but can't find a good solution for this. My application is build in C# .NET (4.0).
You could use Fiddler to monitor Internet traffic.
It is
a Web Debugging Proxy which logs all HTTP(S) traffic between your computer and the Internet. Fiddler allows you to inspect traffic, set breakpoints, and "fiddle" with incoming or outgoing data. Fiddler includes a powerful event-based scripting subsystem, and can be extended using any .NET language.
It's scriptable and can be readily used from .NET.
One simple idea: Instead of monitoring the traffic directly, what about installing a browser extension that sends you the current url of the page. Then you can check if that url is in your database and optionally show the user a message using the browser extension.
This is how extensions like Invisible Hand work... It scans the current page and sends relevant data back to the server for processing. If it finds anything, it uses the browser extension framework to communicate those results back to the user. (Using an alert, or a bar across the top of the window, etc.)
for a good start, wireshark will do what you want.
you can specify a filter to isolate and view http streams.
best part is wireshark is open source, and built opon another program api, winpcap which is open source.
I'm guessing this is what you want.
capture network data off the wire
view the tcp traffic of a computer, isolate and save(in part or in hole) http data.
store information about the http connections
number 1 there is easy, you can google for a winpcap tutorial, or just use some of their sample programs to capture the data.
I recomend you study up on the pcap file format, everything with winpcap uses this basic format and its structers.
now you have to learn how to take a tcp stream and turn it into a solid data stream without curoption, or disorginized parts. (sorry for the spelling)
again, a very good example can be found in the wireshark source code.
then with your data stream, you can simple read the http format, and html data, or what ever your dealing with.
Hope that helps
If the user is cooperating, you could have them set their browser(s) to use a proxy service you provide. This would intercept all web traffic, do whatever you want with it (look up in your database, notify the user, etc), and then pass it on to the original location. Run the proxy on the local system, or on a remote system if that fits your case better.
If the user is not cooperating, or you don't want to make them change their browser settings, you could use one of the packet sniffing solutions, such as fiddler.
A simple stright forward way is to change the comupter DNS to point to your application.
this will cause all DNS traffic to pass though your app which can be sniffed and then redirected to the real DNS server.
it will also save you the hussel of filtering out emule/torrent traffic as it normally work with pure IP address (which also might be a problem as it can be circumvented by using IP address to browse).
-How to change windows DNS Servers
-DNS resolver
Another simple way is to configure (programmaticly) the browsers proxy to pass through your server this will make your life easier but will be more obvious to users.
How to create a simple proxy in C#?
Related
Ok lets say I want to create a connection between my iPhone app and my server (i'd like to try and use GoDaddy servers for this) to server real time location data to users.
I've seen plenty of good stuff online about using sockets, streams, ASIHttpmessage, CFHTTPMessageRef, etc., but what I'm unclear about is how to set up a server that continuously servers real time data to users (I believe you'd need a stream of data going to the user for this, not just a single http request and response). How does one take a host like GoDaddy and run server code on it. I know you can set up a server like this using terminal, but I don't have access to command line or the ability to run this "server program" from my web host as far as I know. Is there software I can download on my cpanel for this? Do I need a virtual private server and different hosting via GoDaddy maybe?
Does anyone know how I can do this or if my understanding of this whole thing is wrong. Please keep in mind I need this real time (or close to). Please, educate me. I really just need a better understanding of how this works.
So, I have read that it is possible to trace the IP of a Facebook friend while talking to him on chat by using Fiddler and Firebug. Now, as far as I am know, Facebook uses HTTPS and all in all, I cannot seem to get anything precise from Fiddler.
May anyone be kind enough to explain if this is really possible and if so, how the process goes?
I dont have a direct answer to your question, but i can give you some guidelines:
If the chat is working as a peer-to-peer network (which i highly doubt) you can trace the incoming tcp/udp connection and search it for the message using a sniffing program (like wireshark) and from there fetch their IP.
If the chat is based on that every message goes through a server (which is probably based on SOMM. not that it matters. (Server Oriented Messaging Model)). now if it is that way there is virtually no way to figure out the IP because the servers are acting as a proxy and masking the original IP, however. if FB includes some sort of meta data (which they do for location for phones etc.) that might contain the sender IP and some other stuff like the mac address etc etc. now i'm not sure of that but its a good place to look.
If you want help with firebug and all those other html/http/browser development tools there are plenty of tutorials out there. If you already know how to use it you might want to check the resources that's loaded when a message is sent. From experience i can tell that when a comment is added to something a whole bunch of crap happens that append stuff to the current html document. but i have never ever seen some meta data through that. anyway it's a good place to start.
I need to stream data from a web server to clients. The data is location data that is collected and stored on the server. The clients will click a button on an html page to 'opt in' to start receiving the data. This data is never ending and there is at least one of the clients that needs to receive the data 24-7, with as few breaks as possible. The data being streamed will be client specific, as each client wont receive the exact same data.
I've done several multi-threaded tcp servers over sockets, and websockets are the way I would like to attack this, but the requirements are that this has to work in ie9.
The initial requirement was that this be a vb.net cgi executable - but during testing, I havent been able to 'use' the stream from the vb.net executable until the app finishes - like it wasn't able to flush the stdout even though I was specificly using the console.out.flush(). So If this isn't a viable option, and I can support this with facts, then I can get this requirement changed.
I've also read quite a bit about using a third party server to stream the data like Orbit and APE I think was a couple of them, but requirements are for 1 server - the web server. No other hardware can be required.
I'm pretty sure the vb.net CGI isn't the ideal solution based on what i've found, but is it doable or do I need to abandon that solution and move on to a newer technology , ISAPI? Any ideas or suggestions, even if they just point me in the right direction, are greatly appreciated.
You might go few ways.
If you would go C# .Net, then you might look into Silverlight solution. But it requires plugin in browser to be installed (like Flash). Good thing here, is that you are able to send data through normal sockets, in pure realtime from server. In same time Silverlight uses .Net so it makes some code to be shared. That helps development process. As well the way it will work in different browsers will be same.
You might have a look in similar solution using Java Applet with Java backend (can be even .Net, but again, easier to develop when both in same language).
Another option is to have fron-end using WebSockets, but as you know its not supported in IE9 and below (IE10 promises to be), and Opera is not supporting it as well.
Backend can be done in what you prefer. But bear in mind that WebSockets uses framing, and for constant but little packets its not efficient, because if you send 10 bytes, then it will create frame 2-12 bytes, and TCP packet header that is 40 bytes in average.
To support older browsers you might have a look in long-polling, but it is not as reliable as websockets.
As well it is important to calculate the amount of data and approximate amount of users that will use your system. Based on calculations you will have approximate information about how real it is, and what server will be required to handle.
I have a VB6 app which is used by a large number of clients.
I need to allow the clients to be able to send emails to me. In the past I have done this using Microsoft MAPI controls. However, not all of them have an email client installed, since they use webmail instead.
Is there any other method anyone can recommend which would allow them to do this?
SMTP
You can use CDO for Windows to do this if we make a few assumptions:
Your users are all on Win2K or later.
The users will never be behind a firewall blocking SMTP or proxying all SMTP port use to a corporate server.
You have an SMTP server that you have an account you can let the user-mails be sent with.
You embed the server's address and account credentials in your program.
Sometimes using an SMTP server listening on an alternate port will address the second issue, but often such an alternate port is even more likely to be blocked.
SMTP is Dying
Abuse over time has made SMTP less and less viable for automated/assisted user contact. There are just too many variables involved in trying to open some sort of "clear channel" for SMTP communication as people work harder to fight spammers and such.
Today I would be much more likely to use either WebDAV or a Web Service for this. Both use HTTP/HTTPS which is more likely to get past firewalls and usually get by most proxy servers as well. WebDAV is often more "slippery" at this than Web Services, which more and more proxies are bocking. You can also use something more RESTful than SOAPy since the traffic "smells more like" user browsing to proxy servers.
WebDAV is a Clean Option
There are even free WebDAV providers offering 2GB of storage with a main and a guest user. The guest account can be given limited rights to various folders so some folders they might post your messages to, other folders they might get data from (read only folders), etc. For a paid account you can get more storage, additional users, etc.
This works well. You can even use the same hosting for program version files, new version code to be downloaded and installed, etc. All you need on your end is an aggregator program that scoops up user posted messages and deletes them using the main user/pw.
You still need to embed user credentials in your program, but it can be a simpler matter to change passwords over time. Just have the program fetch an info file with a new password and an effective date and have the program flip the "new" password to "current" once run on that date or after.
WebDAV support in Windows varies. From WinXP SP3 forward you can simply programmatically map a drive letter to a WebDAV share and then use regular file I/O statements against it, and unmap the letter when done. For more general use across even Win9x you can build a simple WebDAV client on top of XMLHTTPRequest or use a 3rd party library.
Web Services Have Higher Costs
Just to start with you have server-side code to write and maintain, and you have to use a specific kind of hosting. For example if you built it using PHP you need a PHP host, ASP an ASP host, ASP.Net an ASP.Net host, etc.
Web Services can also be more problematic in terms of versioning. If you later update your program to provide different information in these user contact posts you have to make another Web Service as well as changing both the application and the aggregator. Using WebDAV you can just make a "new format" folder on the server and have the new program post the data there in the new format. Your aggregator can simply pull from both folders and do any necessary reformatting into your new local database/message repository format.
This is merely an incremental additional effort though and a Web Service might be the way to go, even if it is just something written like an HTML Form GET/POST acceptor.
Although this question is for VBA you may find it of interest. Sending Emails using VBA without MAPI
I need to implement a process where users punch in a few details into a web page, and have this information fired as some
sort of an event to a Java rich client application (SWING) on the same host.
One idea was perhaps implementing an applet that would initiate socket communication with a listener implemented by the SWING
application, but not sure whether this is possible at all.
This sort of puzzling piece of integration is basically a given fact.
Essentially both the web application and the SWING one are already active and in use.
The only missing bit is sharing info between the two, in a way that would be easy to implement. no matter how dirty.
Any ideas?
Thanks!
Sounds a little confusing to the user if nothing else.
I would go one of two ways.
Have your rich client communicate over the network. And put whatever form you were going to have in the browser there.
Put your rich client into an applet.
Have both connect to a server somewhere (even locally), which your rich client can poll to see if the form has been filled in.