How can I check if a proxy is online and properly working with perl? I was considering running a get operation and comparing output but i'll be running this check so frequently this overhead would be huge, any other more lightweight alternative?
No, this is exactly how you do it. If you use a light-weight method such as HEAD, TRACE or OPTIONS instead, you cannot know whether the proxy is actually useful or censoring or even subtly subverting the unencrypted data.
You can keep the overhead small by testing against a minimal useful HTML document.
Like daxim said, I think that testing against a very small HTML document is going to be lightweight enough for most scenarios.
The insuperable lightweight solution will be to use a web service that responds you with minimal data about your proxy IP address, if it is online and working fast enough, etc. This of course is going to include a third party(who's going to be doing the not so light work, doing the requests to all the proxies) and this, like everything, has their pros and cons.
I use this proxy checker from Google code to do exactly what you need and I also obtain some more info of each IP address, like country and a couple of the proxy speed measurements. It's a very simple code that consumes a web service from http://proxyipchecker.com/ .
PS: The example is in PHP but is trivial to do the same in Perl.
Related
I was wondering if I can implement bi-directional communication channel between 2 kext modules using sockets under the domain PF_SYSTEM. this method mostly used to communicate between driver and user-space agent..
In my particular case I've got one module based on IOKit and the other which is simple kernel module with start and stop callback functions and I'd like to pass some small messages between them..
Do you think this approach is suitable for my needs or there's other preferable way (shared memory ? mach ports ? )
EDIT, after digging a little deeper, maybe there's an option to export an API from one driver to the other by modifying the client driver plist file as follows.. is it possible ?
<key>OSBundleLibraries</key>
<dict>
<key>com.driver.server_driver</key>
<string>1</string>
This however, doesn't work because when i try to manually load the client driver after the server driver already loaded (visible from kextstat), I get the No kexts found for these libraries error.
Using messaging techniques normally used for IPC for communicationg between kernel extensions is unusual, as it's a lot more complex than taking advantage of the fact that they're running in the same address space anyway. I covered some of the details of this latter approach in my answer to your other question which you've obviously already seen, but I'm linking to for the benefit of others in a similar situation.
To answer your question: I suspect both ends of a system socket being in the kernel is probably not very well tested, and you could run into bugs in the kernel. The in-kernel public socket KPI is also quite fiddly: getting the buffering right is tricky, so I'd only use sockets if I absolutely had to, and it clearly isn't here.
My gut instinct is that Mach messaging would work more reliably and require less code, but again I think it would be quite unusual to use it in this way.
It's hard to give useful advice on exactly what you should do, as we don't know the reasons for the separation into 2 kexts, what their relationship is, what kind of communication is required, etc. There are many possible ways on how to exchange information, but whether they are a good idea will depend on the details of the project. (This sort of question isn't really suitable to Stack Overflow's format - this is the sort of problem for which a company will bring in an expert to consult. For a private project, you might have more luck on the Software Engineering Stack Exchange Site, where this sort of question is on-topic, although I'm not sure you'll get a good/useful answer. For a private project it's probably best you keep it simple and maybe combine the 2 kexts into one?)
I would like to add a REST interface to an existing TCL codebase (so that the programms in other language can use the existing TCL code).
I found a list of Webserver with TCL support but I have no idea which one would be a good solution to quickly map our TCL functions to HTTP/REST calls without tons of boilerplate code.
Has anyone here already done something like this and can tell me which of these servers would be a good (or bad/difficult) solution?
Is there maybe another server/framework that is even better for this use case?
Consider Naviserver. Tcl is its embedded interpreter language. It has a low profile memory overhead, and is regularly maintained and tested for performance and low latency.
For what you’re describing, you might consider Wapp. It’ll do exactly the boilerplate elimination you want, and it’s easy to dive into. You’d probably want to use it as a library, rather than an app, given that you’ve got an existing codebase, but its operation past the initial setup is the same for that use case.
I have been trying to decide if my web project is a candidate for implementation using PSGI, but I don't really see what good it would do for my application at this stage.
I don't really understand all the fuss. To me PSGI seems like a framework that provides a common interface between different Apache modules which lets you move your application between them. e.g Easily move your application from running on mod_perl to fastcgi, and provide the application support for running on both options.
Is that right, or have I missed something?
As I and the team I am a part of not only develop the application, but also pretty much do maintenance and setup of servers I don't see the value for us of being able to run on fastcgi, cgi, and mod_perl, we do just fine with just mod_perl.
Have I misunderstood the PSGI functionality, or is it just not suitable for my project?
Forget the Apache bit. It's a way of writing your application so that the choice of webserver becomes less relevant. At $work we switched to Plack/PSGI after finding our app running with very high CPU load after upgrading to Apache2 - benchmarking various Apache configs and NYTProf'ing were unable to determine the reason, and using PSGI and the Starman webserver worked out much better for us.
Now everything is handled in one place by our PSGI app (URL re-writes, static content, expiry headers, etc) rather than Apache configuration, so it's a) Perl, and b) easily tested via our standard /t/ scripts. Also our tests are now testing exactly what a user sees, rather than just the basic app itself.
It may well not be relevant to you if you're happy with Apache and mod_perl, and I'm sure others will be able to give much better answers, but for us not having to deal with anything Apache-related again is such a relief in itself. The ease of testing, and the ability to just stick in a Data::Dumper and see what's going on rather than wrestling with ModRewrite and friends, is a great boon.
Borrowing from a recent blog post by chromatic, Why PSGI/Plack Matters (Testing), here's what it is:
It's a good idea borrowed from Python's WSGI and Ruby's Rack but made Perlish; it's a simple formalizing of a pattern of web application development, where the entry point into the application is a function reference and the exit point is a tuple of header information and a response body.
That's it. That's as simple as it can be, and that simplicity deceives a lot of people who want to learn it.
An important benefit is, ibid.,
Given a Plack application, you don't have to deploy to a web server—even locally—to test your application as if it were deployed … Plack and TWMP (and Plack::Test) use the well-defined Plack pattern to make something which was previously difficult into something amazingly easy. They're not the first and they won't be the last, but they do demonstrate the value of Plack.
Started wrote an answer and after 50 lines I deleted it. Simply because it is impossible tell (in short) why is PSGI extremely cool. I'm new in PSGI too, but zilion things now are much easier as before in my apache/mod_perl era.
I can give you next advices:
read the Plack advent calendar - all days, step-by-step. You must understand the basic philosophy, what is good on onions and so on... :)
search CPAN for "Plack::Middleware::" - and read the first few lines in each. Here are MANY. (Really should be somewhere some short overview for each one, unfortunately don't know any faster way. Simply it is good to know, what middlewares are already developed. (For example, you sure will need the Plack::Middleware::Session, or Plack::Middleware::Static and so on...)
read about Plack::Builder (already done, when you done with the advent calendar) :)
try write some apps with it and will find than Plack is like the first sex - now you didn't understand that you could live without it.
ps:
If was here something like "Perl Oscar", will sure nominating MyiagavaSan. :)
I'm currently using a modified HTTP::Daemon::Threaded server in
combination with SOAP::WSDL and Pod::WSDL to provide web services
used by a variety of client types and roles.
---- that's not the question, the following is -----
I'd like to arrive at an optimal solution (as far as is possible) with respect to the following topics:
Request/Dispatch/Response speed
Protocol security (proper use of client-authenticated SSLv3/TLS)
Resource security (security roles/traits on per-resource & per-method bases)
Declarative specification of types, method signatures, and required security roles & traits.
Questions:
I'd like to be using an IO::Select or IO::Async::Loop::IO_Ppoll -based server, but I understand that this is not compatible with in-server client authenticated SSL. Is my understanding correct?
I'd like to move away from verifying the client certificate on each request, to something like CA SiteMinder, where I give out a time-limited session cookie after successful client certificate verification, which can be used on subsequent requests to avoid this time penalty (and to lessen server load). Is this going to be as secure? (or even good enough?)
Is there some module/framework I can build on to provided Trait and Role -based Authorisation for the exposed object and methods. Pod::WSDL really only deals with types (and not even complex ones). I'd like to use/implement some declarative annotation (or external YAML) -based scheme to handle complex WSDL typing as well as Trait & Role Authorisation. Has anyone done this? (even separately?) Are there any other modules which might be a good starting point?
Finally - Am I just crazy for doing this in Perl5 ? ;)
Ok, everyone's answering everything but the real questions.
I'll break this out into specific questions in separate posts, and won't make any mention at all of the server make-up - a topic which everyone in this thread seems to want to discuss, and which is completely irrelevant.
I know this is an old question, but FYI IO::Async will work just fine with SSL, ever since the IO::Async::SSL module.
You're crazy for doing this in Perl :-)
Seriously though, more power to you. My question is, presuming you have some reason to reinvent this wheel, is why not consider Python? Perl is alive and well but so much of this kind of thing (low level scripting) is being done in Python now.
Finally, presuming you don't have an actual reason to be doing this (aside from fun), you should really consider a Web Framework (Django of course) and something like nginx to handle the HTTP interaction.
I've been trying to find a good form of RPC to standardize on, but so far I've ran into a ton of walls and was wondering what the stackoverflow communities view was.
My ideal RPC would provide the following:
Somewhat wide support in other languages, in that people shouldn't have to write a custom stack to use our server
Input validation
Ideally, some way to turn the above input validation into some sort of automated documentation to distribute
Clean and maintainable code
I am a fan of the catalyst framework and would prefer to stick to it, but if there is a clearly better alternative for RPC servers I'd be open to that as well.
So far I have looked at the following:
Catalyst::Controller::SOAP
Doesn't appear to support returning of complex data structures, only string('literals'). I could probably serialize data on top of that, but that seems very hacky. It also lets you return a pre-formed XML object, but I couldn't get that to work and it looks like you'd need to re-create a lot of SOAP data structure for it to work.
I do like the idea of WSDLs, but the complexity of the overall spec also makes me wonder how well support for communicating with other languages will be.
Custom POSTing XML based controller
We tried to roll our own by hand in a similar way to how we've seen two other projects do it recently where there is a dispatch url that you post XML to. This lets you have XSD validation/documentation, but required creating a lot more code than we want to maintain at this point.
Catalyst::Plugin::Server::XMLRPC
Gave a warning about using a deprecated method that will be removed in a future version of Catalyst.
No input validation or doc creation, but otherwise the best I've found
JSONRPC
Looks pretty similar to XMLRPC only the module is actually updated. I'll probably go with this next unless someone suggests something better
There also appears to be two different modules for catalyst that do JSONRPC
I realize that REST isn't pure RPC (only a subset), but...
I would recommend the Catalyst::Controller::REST and Catalyst::Action::REST modules. They're frequently updated and the documentation is fairly good. There is also a good (but rather dated) example of using Catalyst::Controller::Rest in the 2006 Catalyst Advent calendar titled Day 9 - Web Services with Catalyst::Action::REST.
FWIW, Catalyst::Controller::SOAP does support returning complex data. Take a look at the documentation http://search.cpan.org/~druoso/Catalyst-Controller-SOAP-1.23/lib/Catalyst/Controller/SOAP.pm, which will show you that you can use a WSDL to describe your service. Also, see http://daniel.ruoso.com/categoria/perl/soap-today.html.en for a more detailed step-by-step process.