looking into connecting to a secure ftp site (using perl), and downloading all the .log files, saving in new directories named after the day I downloaded the files. I want to do this without modules, as a learning experience, but before I start I wanted to know if you guys thought it was doing, or is way too much for a relatively new programmer and I should just learn the modules?
If it's production work, no, use the modules. Your implementation will be buggy, missing features and unknown to the next person maintaining that code.
Otherwise, yes. It's good to learn the principles of a network protocol. I do have a reservation about FTP as it is a bit baroque, insecure, inefficient and on its way out. scp, HTTP or rsync would be more useful to put your energy into.
I'd start with reading the RFC and putting together your own FTP module using just network sockets. Document and test it as if you were going to release to CPAN as a full learning exercise in making a network module. Run it against some various FTP server implementations as they often interpret the spec differently (or not at all). Don't be afraid to cheat and look at what the existing modules do. Who knows, you might write something better than what's already there.
Learning the principals, just like we did at school for long multiplication and division, means we know how things work when we use a short hand.
However, when new to the world,just like when you learn to speak, you did "A is for Apple" etc, you didnt get explained about the finesse of grammar and all that, you learnt to express yourself enough to be understood.
Programming is a little like the same. While in an ideal world you can easily argue a prewritten generic library is often way less efficient than a specifically targeted set of routines. If the wheel you are using was already invented, it seems a lot of work to make a new one.
So, use the wheels and cogs afailable, once you really have the hang of it, NOW look at inventing your own more efficient ones.
Ad cpan modules:
Modules are an great learning source. Here is zilion modules and you can really learn much studying some of them.
And when/while you mastering your perl, you will start writing you own modules. When your program will use modules anyway (yours one), you can ask - why don't use modules already developed and debugged?
So, learn perl basics, study some modules (for example Net::SFTP) and if you still want write your own solution - it is up to you. :)
'
Related
I would like to add a REST interface to an existing TCL codebase (so that the programms in other language can use the existing TCL code).
I found a list of Webserver with TCL support but I have no idea which one would be a good solution to quickly map our TCL functions to HTTP/REST calls without tons of boilerplate code.
Has anyone here already done something like this and can tell me which of these servers would be a good (or bad/difficult) solution?
Is there maybe another server/framework that is even better for this use case?
Consider Naviserver. Tcl is its embedded interpreter language. It has a low profile memory overhead, and is regularly maintained and tested for performance and low latency.
For what you’re describing, you might consider Wapp. It’ll do exactly the boilerplate elimination you want, and it’s easy to dive into. You’d probably want to use it as a library, rather than an app, given that you’ve got an existing codebase, but its operation past the initial setup is the same for that use case.
Many standard modules are all using straight up perl -- problem is these guys arent using Moosey stuff, so I catch myself wrapping them with Moose or reinventing some simple functions in bigger libraries for convenience.
I wondered if there was any general approach to how developers using Moose incorporate other libraries that are non-Moose.
Being new to Perl and Moose I'd like to have a better understanding of how Moose is used in situations like this, or when it is generally preferred to use Moose vs Perl or even MooseX, or some other package, or whether its arbitrary.
Seems like there are different schools of thought, but Perl being as old as it is -- there are too many conflicting sources, so it's hard to navigate to a consistent truth. I'm not sure what to believe!
Anyone have a definitive source they turn to for "modern" usage of perl? Understand I've only been using perl for a month so I'm green to this community.
Updated
I don't want to hurt anybody's feelings by talking about libraries they love in a way they may not appreciate, so I've removed my side commentary about certain libraries Ive used to refocus on the question at hand.
Thanks for your guidance!
While I do not know what others do, I would be very reluctant to create myself extra work. I do not see any general need to Moosify a bunch of modules that already work.
If you want to inherit from non-Moose modules, take a look at MooseX::NonMoose.
If the HTML generation cruft in CGI.pm bothers you, you can use CGI::Simple.
About Reinventing CGI
CGI is a library which has been thoroughly tried and tested and if it needs improvement, you could build an extension, or contribute/contact the maintainers. You have to remember that modules are only as good as their track record (reliability) and their upkeep. Many people created decent modules, but didn't continue maintaining it, so they sort of fell to obscurity.
CGI is a boat all of its own, which if you think there's a lot of overhead, you could use CGI::Simple or CGI::Minimal. CGI.pm does more than parse querystrings, it also has cookie management (sessions), HTML generation, and other useful functions.
Others have had some criticisms of the overhead with CGI.pm, but that's why they developed FastCGI, which is modifying the server to use a persistent state of the script, thus loading the overhead once, rather than on every page load.
It is possible for you to create another (even better) version, but why bother? Many people may probably tell you, you shouldn't reinvent the wheel, with good reason. CGI has been around for almost 2 decades, with so many users testing it, finding holes, and having patched the holes; however, I'm never a big fan of saying "you shouldn't do something." If you think something could be better, make it better. There are many OSes that exist today just because of that reason, why settle for something that does 95% of what you need, if you need that other 5% too? But I also say, weigh your costs vs. benefits and determine if you want to devote your time to this, or if maybe there's another problem out there that has yet to be solved, that could use a little more manpower. To have something successful, you're going to need to test it thoroughly, and will most likely need to create something that other people would want and (at this point) there isn't much of a reason for CGI-users to be motivated to switch.
About Modern Perl
I think "modern Perl" is an oxymoron. I would jokingly call modern Perl; Ruby or Python.
That isn't to say that Perl isn't useful, because it is, but it's been around a long time. While it has had its significant share of changes from version to version, the most popular, Perl5, hasn't changed all that much; mind you, my definition of change is not adding to the language (new operators and functionality), but deprecating/replacing old features or changing the behavior of existing ones (like for/foreach loops).
Note: Perl6 could be considered modern perl (and does have many significant changes), but it's not widely adopted and was supposed to be released many many years ago (it's the Duke Nukem 4 Ever of programming languages).
About XS
I haven't done much module programming, but if memory serves correct, XS is the interface between Perl and C, which I think allows you to compile your perl modules for faster execution. Consider the PostgreSQL DBI module. There's a DBD::PgPP, which is a pure perl module to interface with Postgres, but there's also DBD::Pg, which I think compiles some of the code using C and takes advantage of some other OS utilities. Compiled modules have the benefit of faster load and execution (there may be some better resource management in there too).
I've looked into the source code of DBIx::Class recently and found that I don't understand a thing (though I mastered a couple tricks while trying to).
So my question is: which CPAN modules are a must read for someone who wants to learn, and in what order?
If I were doing the same I’d probably start with the ::Tiny space. I’d expect it to be less distracting—fewer edge cases cluttering things—and more idiomatic—terseness lends itself to Perl idiom—in general.
Then I’d attack the medium–large nodes from this excellent document—Map of the CPAN’s authors (large PDF). Update: Web version. Zoom into the bigger nodes then search on search.cpan.org for them. The largest nodes sometimes represent old-school and while exceptional code exists in the old school, not a lot of good teaching examples do (so I say). Authors like Miyagawa, Kennedy, and Kogman come to mind immediately as worth reviewing. There are plenty of others. Basically any module you see recommended here often, look up the author and poke around his or her other packages, as it were.
I learned quite a bit (tie-ing, platform independent filesystem access, etc) by reading the code for File::chdir. It is also a very handy module to use in your scripts, I use it all the time.
I would also add to bvr's list: read the source for modules that you use frequently, since you are already familiar with their expected behavior, you can more clearly see what is being done to achieve that result.
The question is what you want to learn, but it is certainly good idea to study various modules, because you learn to read other people code and learn various tricks. Some random recommendations I can think of
start with smaller modules with clear interface you know and are interested in
once you feel familiar with organization of modules and basics, try something larger
try rather newer modules
look into test suite and into examples
if you don't understand specific piece, try to make reduced example a play with it
It is hard to recommend something specific, but I liked my recent look into Web::Scraper module.
If you're fluent in perl - if don't perldoc ;) - , sugest learn the packages Task::Kensho or Modern::Perl.
These packages do cover comprehensive in culture Perl, since tests until hacks, passing by crawling, modules to developers, e-mail, dates, modern oriented obejct in Perl.
Participe of discussion lists, read the history of list, irc.
Perl have many tricks, the community always responds with enthusiasm =)
This is more of a use-case type of question... but also generic enough to be more broadly applicable:
In short, I'm working on a module that's more or less a command-line wrapper; OO naturally. Without going into too many details (unless someone wants them), there isn't a crazy amount of complexity to the system, but it did feel natural to have three or four objects in this framework. Finally, it's an open source thing I'll put out there, rather than a module with a few developers in the same firm working on it.
First I implemented the OO using Class::Std, because Perl Best Practices (Conway, 2005) made a good argument for why to use inside-out objects. Full control over what attributes get accessed and so on, proper encapsulation, etc. Also his design is surprisingly simple and clever.
I liked it, but then noticed that no one really uses this; in fact it seems Conway himself doesn't really recommend this anymore?
So I moved to everyone's favorite, Moose. It's easy to use, although way way overkill feature-wise for what I want to do. The big, major downside is: it's got a slew of module dependencies that force users of my module to download them all. A minor downside is it's got way more functionality than I really need.
What are recommendations? Inconvenience fellow developers by forcing them to use a possibly-obsolete module, or force every user of the module to download Moose and all its dependencies?
Is there a third option for a proper Perl OO framework that's popular but neither of these two?
To be perfectly fair, seeing virtually everything interesting these days in Perl world has Moose somewhere as a dependency, I don't see it being much a debt for other "fellow Perl developers".
Chances are they already have it installed as we speak!
Edit: Some statistics:
Moose is currently rated at 65th place on the "Most Depended on" modules list, Aliases top 100, with over 1637 packages depending on it. Thats almost as much as stuff like Time::HiRes , and more than DBI, and I don't think you're as likely to question depending on those would you?
The currently accepted "modern Perl OO framework" is Moose. I'd say make your users download it, or you can bundle it up with your modules in the installation using PAR::Packer.
Quoting from "But I can't use CPAN" (...because my users won't want to install things):
Assuming you're just handling your users a tarball, then Module::Install provides a solution - if you put your script into script/ and then do
install_script(glob 'script/*');
auto_install;
in your Makefile.PL, then not only will 'make install' put your script somewhere useful for you but 'make installdeps' will invoke cpan (or if present, cpanplus) to install all missing dependencies for you.
Well, there's Mouse, which is like Moose but without all the dependencies (and some of the features). It also starts up a bit faster. I haven't tried it myself, but it's generally well thought of.
To add to the existing fine answers...
Some of what was recommended in PBP isn't bad advice, but Perl marches on. When it was written, inside-out objects were the new hotness. Now the Moose has absorbed all. There is MooseX::InsideOut which gives you the power of Moose with the total encapsulation of Class::Std, but unless you work with undisciplined programmers its really not necessary.
Those features of Moose you don't need now, you'll need them eventually. Even if you don't need all of them, with Moose you won't have to use and learn Yet Another OO System every time you need an interesting feature. And god forbid you need TWO features at the SAME TIME!
There is also a Perl Module Object::InsideOut , actively maintained as of 2010.
Kind of a predecessor to Moose, or to be clear: development started independently at the same time as Moose started,
I know it exists but I haven't used it.
I have inherited a broad, ill-designed web portfolio at my job. Most pages are written in Perl as most of the data ingested, processed, and displayed on the site comes in the form of flat-files which then have to be meticulously regexed and databased in our MySQL and Oracle databases.
As the first IT-trained manager of this environment, I am taking it upon myself to scrub the websites and lay down some structure to the development process. One of the choices I have been given is to choose whether or not to continue in Perl. There is substantial in-house talent for Java and PHP is rather easy to learn. I have considered taking the reins off the developers and letting them choose whatever language they want to use for their pages but that sounds like it might be trouble if the guy who chose PHP gets hit by a bus and no one else can fix it.
As the years go by hiring Perl programmers becomes more and more difficult and the complexity of maintaining legacy Perl code from previous developers whose main focus may have been just getting a page up and running is becoming very resource-consuming. Another, previous (non-IT) manager was more focused in on quick turnaround and immediate gratification of pages rather than ensuring that it was done right the first time (he has since been promoted outside our branch).
The production server is solaris. MySQL has most of our data but new projects have begun using Oracle more and more (for GIS data). Web servers are universally Apache. We live in an intranet disconnected from the regular internet. Our development is conducted in an Agile, iterative manner.
Whatever language is chosen to push forward into, there are resources to have the existing development staff re-trained. No matter what, the data that comes into our environment will have to be regexed to death so perl isn't going away anytime soon. My question to the community is, what are the pros and cons of the following languages for the above-defined web dev environment: Perl, PHP, Java, Python, and --insert your favorite language here--. If you had it to do all over again, which language set-up would you have chosen?
Edits and Clarifications:
Let me clarify a bit on my original post. I'm not throwing everything away. I've been given the opportunity to adjust the course of the ship to what I believe is a better heading. Even if I chose a new language, the perl code will be around for some time to come.
Hypothetically speaking, if I chose Assembly as my new language (haha) I would have to get the old developers up to speed probably by sending them to some basic assembly classes. New pages/projects would be in the new language and the old pages/projects would have to play nice with the new pages/projects. Some might one day be rewritten into the new language, some may never be changed.
What will likely always be in Perl will be the parsing scripts we wrote years ago to sift through and database information from the flat files. But that's okay because they don't interface with the webpages, they interface with the database.
Thank you all for your input, it has been very very helpful thus far.
It seems that your problem is more legacy code and informal development methodology than the language per se. So if you already have Perl developers on staff, why not start modernizing your methods and your code base, instead of switching to a new language, and creating an heterogeneous code base.
Modern Perl offers a lot in terms of good practices and powerful tools: testing is emphasized, with the Test::* modules and WWW::Mechanize, data base interaction can be done through plain DBI, but also using ORM modules like the excellent DBIx::Class, OO with Moose is now on par with more modern languages, mod_perl gives you access to a lot of power within Apache. There are also quite a few MVC frameworks for Perl. One that's getting a lot of attention is Catalyst.
Invest in a few copies of Perl Best Practices, bring in a proper trainer for a few classes on modern development methods, and start changing the culture of your group.
And if you have trouble finding developers that are already proficient in Perl, you can always hire good PHP people and train them, that shouldn't be too difficult. At least their willingness to learn a new language would be a good sign of their flexibility and will to improve.
It is always tempting to blame the state of your code on the language its written in, but in your case I am not sure that is the case. Lots of big companies seem to have no problem managing huge code bases in Perl, the list is long but the main Web companies are all there, along with a number of financial institutions.
I would bring in someone who is very good at Perl, to at least look at the current design. They would be able to tell you how bad the Perl code really is, and what needs to be done to get it into good shape.
At that point I would start considering my options. If the Perl code is salvageable, well than great, hire someone proficient in Perl. Also train some of your existing staff to help on the existing code-base. If you don't have someone proficient in Perl in charge of the Perl code, your code-base may become even worse than it already is.
Only if it was in terrible shape, would I consider abandoning it for another language. What that language is, that is something your going to have to think about that yourself.
p.s. I'm a bit biased, I prefer Perl
If regex is important I would choose a language with good support.
If you would use java, you will not be able to just copy paste your regex code from the perl code because the slashes have to be escaped. So I would vote against java.
I'm not familiar with php enough to know its regex features, but given your choices I would go for python. You can create cleaner code in python.
Would ruby also be an option? It also has good perl like regex support and rails supports agile web development out of the box.
first off let me point out that MySQL's spatial extensions work with GIS.
Second, if you have a bunch of Perl programmers that will need tow ork on the new sites then your best bet is to choose something they won't have too much trouble understanding. The obvious "something" there is PHP. When I learnt PHP I'd done some Perl years ago and picked up PHP in no time at all.
Switching to something like Java or .Net (or even Ruby on Rails) would be a far more dramatic shift in design.
Plus with Apache servers you already have your environment set up and you can probably stage any development as a mix of Perl and PHP reasonably easily.
As to the last part of your question: what would you do it in if you could do it over again? To me that's a seprate and basically irrelevant question. The fact is you're not rewinding and starting over, you're just... starting over. So legacy support, transition, skills development and all those other issues are far more important than any hypothetical question of what you'd choose in a perfect world if all other things were equal.
Love it or hate it, PHP is popular and is going nowhere anytime soon. Finding skiled people to do it is not too difficult (well, the dificulty is filtering them out from the self-taught cowboy script jockeys who think they can code but can't) and it's not a far step from Web-based Perl.
If your developers are any good, they'll be able to handle anything thrown at them. Deciding what language to use is quite a tricky strategic position, but I recommend you think very carefully before introducing any MORE (i.e. don't).
Unless of course, there is something that you absolutely cannot do (or cannot do sanely) with what you've got.
I think choosing a single language is key and if your database is primarily MySQL, then PHP seems like the obvious choice. It naturally works with your database, it's open source and there is tons of documentation, source code, doesn't require compiling, etc.
People come and go through positions and any website will evolve over time. If you have the ability to set some guidelines and rules, I would choose something that is forgiving, common-place, and easy(er) to learn.
I'd also suggest writing it down so people in the future don't re-invent the wheel.