Are there any libraries out there that do this? Playing around with Common Lisp it seems like this would be one of the most useful things to lower barrier of entry for newcomers. ASDF seems mostly designed for deployment, not for rapid prototyping and development. Following threads on comp.lang.lisp it seems like people agree that CL's package system is powerful, but lacks the ease of something like Python's dead simple module system. It is FAIL in the sense that it's designed for power not usability.
Glad to know if I'm wrong. If I'm right, I'm stunned that noone has tried to build a Python module-like system on top of ASDF.
Zach Beane wrote how he nowadays starts new Common Lisp projects by using Quicklisp and Quickproject. This might be along the lines you want.
Not sure if it's ready for prime time or whether it fits your requirements at all, but here's a link to XCVB.
I don't know. I mostly use ASDF for my in-development compilation needs. Once you notice that you'd benefiot from more than one source file, open <projectname>.asd, slap in a basic ASDF system definition template and start slapping filenames in. As and when you notice a cross-file dependency, update the dependency list.
But, then, I use the exact same method dealing with Makefiles (yes, I know there are automatic dependency checkers that can do it for you, but since I mostly code on my own, it's easier to just amend the Makefile/ASDF definition as I go).
In SBCL, there's a hook on REQUIRE that checks for ASDF systems, so you end up with something that is about as convenient as Python's import, but somehow I suspect that is not what you meant.
This may not be the answer you want, but clearly you have some idea of what you want in a module system. Have you considered creating one yourself? That is, taking your limited domain, your limited requirements, your environment and simply pounding out whatever abstractions will quickly make your life easier?
That's one of the key benefits of Lisp I'm sure you know, is that these simple abstractions and little tools are typically very easy to craft in Lisp.
I'm not suggesting solving everyone who has a problem with the package system or ASDF, I'm simply suggesting solving your own problem as you understand it, which is likely simpler and smaller than some more powerful larger scope.
There is Mudballs now, too.
If you're looking for a piece of software to add this functionality to then it's a good bet.
If you want a command line tool that just uses bash to generate new common lisp project directory and file layouts, you may find one that I created for myself useful: lispproject. If it doesn't match your needs, go ahead and fork it or the repo it gets it templates from to suit your needs: lisp-project-template. Look at the sh file in lispproject repo to see how the templates are used. Also, please note that you may need to adjust the calls to sed to fit your platform as I am using this on macOS. Alternative sed calls are in the main script but just commented out if you need them.
it's designed for power not usability
that's how most Lisp gurus like it.
Related
I have developed some Common Lisp functions in a couple of Lisp source files that I'd like easily available to other functions I write, or make available on github if I think they'd be useful for someone else. For now, I've just been putting them in some pre-defined folder and using (require "/path/to/my/modules/module.lisp").
I'm wanting to understand what is the correct (canonical Lisp) way of defining Lisp library of modules. And the second part of the question is how to use them (whether I've defined them, or whether I've obtained one from someone else).
I've been reading a lot about defpackage and defsystem and asdf. But everything I've read seems to focus on some specific corner of the universe of this task. I'm having a lot of trouble seeing the big picture of custom module creation, deployment, and use. So assuming I have the Lisp environment in front of me (CLISP or SBCL) and one or more .lisp files I'd like to make a package or library out of, is there a document somewhere that explains what steps are required to do that? It's probably something I've already read, but didn't track due to not understanding the context. What I've read about ASDF seems functionally to be what I'm after, but I'm left not understanding whether ASDF is my only option, or whether it just happens to be a de facto standard and what most other people use, or whatever. I played with it a bit in SBCL and wasn't sure I was using it right, and didn't see info on how to set it up in CLISP. So I'm wanting to understand what is the up-the-middle, vanilla approach to this task.
I know this is a big, sloppy set of sub-questions. Again, if there are some good references to look at, I can read. I'm just having some trouble getting a big picture view of how this is supposed to work, and whether there is any "best" approach, or whether, in Lisp, it's a bit of a "Wild West" choose-the-library-manager-you like approach. I did the Google thing and read anything that looked relevant, but my brain is spinning from all of it.
Thanks.
A system is a collection of files and sub-systems. One can compile or load such a system. There are also other operations possible. It keeps track of dependencies and tries to do a minimal amount of work.
If you are using SBCL and CLISP, then ASDF is the tool to choose. See http://www.cliki.net/asdf
ASDF provides, amongst other things, a DEFSYSTEM macro to describe such systems.
Don't use PROVIDE/ REQUIRE- unless you know what you are doing. ASDF is the way to go.
To publish your code and make it easily loadable by others then use QUICKLISP. See: http://www.quicklisp.org/beta/
A package in Common Lisp is not like a package in most other senses. It's not an archive of list files, but more like what most other languages would call a module or namespace that lets you select which symbols (names) from your code you want to show to the outside world of code.
If you just have one file for your little library, you can just distribute that. If you have multiple files, that where a tool like ASDF comes in to make sure, for example, that files defining macros are loaded before the files that use those macros.
Here are some good resources for you to look at, both chapters from Practical Common Lisp:
21. Programming in the Large: Packages and Symbols: This will give you a much better sense of what packages are and how and why to use them.
32. Conclusion: What's Next?: The "Delivering Applications" section has some good resources on this stuff in general, and has some mention of ASDF.
ASDF and Quicklisp are useful tools, that is an established fact. However, I would like to give an alternative point of view on the concept of "library" as it is discussed on previous answers
ASDF is designed to automate the compilation and loading of a set of source files. It is to CL what make is to Unix. It is perfectly valid to write and distribute a program without make just as it is fine to write a program in CL that does not use ASDF.
If your project is simple enough, it is sufficient to provide a file (e.g. load.lisp) that contains the statements to load the dependent files of your project in the right order. Therefore ASDF is not involved in the concept of a library.
In a provocative way, I would say that the canonical way of defining a module in CL is to use the defpackage declaration because it is the language unit that allows a programmer to isolate his/her declarations from those of someone else.
Then is the question of how to make it available to others. If you write portable CL code, then ASDF is the most popular system definition facility and you should use it. If you want to make it easier for others to obtain, then Quicklisp is the tool that changed the face of CL in the last few years.
Finally I would like to add that neither ASDF nor Quicklisp are standards, they are tools (which does not remove to their usefulness). ANSI Common Lisp is a standard and I would love to see a system definition standardized in CL.
Yes, ASDF is a de facto standard, and Quicklisp is another standard.
From Lisp's viewpoint what you want is to define one or several namespaces (packages). This is regulated by the ANSI standard. From your code's viewpoint you want to arrange a bunch of files so that they became a whole and somehow provide that packages. This is where ASDF plugs in. And Quicklisp allows you to manage ASDF systems in the easiest way concievable. You can both download a lot of libraries from Quicklisp's repository and manage your local systems creating symlinks in the quicklisp/local-project folder.
If you have Quicklisp installed, you can type
(ql:quickload :cl-fad)
at the REPL and thus load the CL-FAD library (possibly downloading it); then the packages CL-FAD and PATH become available, you forget about ASDF-systems and stick with the logic of packages.
A good idea would be to take a look at asd-files of several projects downloaded with Quicklisp.
I setup my own libraries, that I use often, using ASDF system definitions and linking them into the local-projects (or similar named) folder of Quicklisp. That allows me to load them using quicklisp like any published package.
If you want to learn how to setup such an ASDF system, I would suggest installing Quicklisp (which for once is really easy, when it comes to installing 'unfinished' software from the internet), quickload a well known package or two and look at its .asd file, consulting the documentation when in doubt.
This way, you have your libraries already setup to be published like most of the well known CommonLisp packages out there.
I have come accross vi-style keybindings for Emacs, but my past experience in mixing Lisp based config in .emacs file for enabling various modes etc., hasn't been very pleasant so far.
Read several articles on Emacs + Erlang, but is there something that might be slightly easier for folks familiar mostly with vi (and who are unfamiliar with Lisp)?
The entire set of possibilities (keybindings) is quite overwhelming. Is there a condensed key-map/shortcuts reference specially relevant for Erlang development?
Just use vi and a command line. There are numerous people doing that and it seems to work just fine for them. Even though I use Emacs, it is not the Emacs-erlang interaction I use. Rather I usually just have a separate Erlang shell and then I load modules in that shell with the l() command for interaction.
Answering your questions one by one:
Debugging in Erlang is funny. Firstly, if your program is kept functional you will find that you need much less debugging to figure out what the program does incorrectly because you can simply write small tests for each of the numerous functions to test its correctness. And you can add those into unit-test frameworks.
As soon as you add multiple processes to the mix you will find that traditional debugging doesn't work anymore anyway. The trick is then to trace and assert, and you will need to learn a way to read programs without running them and see where they go with a debugger.
That said, try to execute debugger:start() in an Erlang shell :) There are also the tracing systems and redbug, a 3rd party tool built on top of them (part of the eper suite).
Profiling can be done with one of the 3-4 profilers: cprof, eprof and fprof are all slightly different in scope and in how much impact they have on your program and if they can be run on a production system or not. I tend to use eprof and I have a knob in my program which will spawn an eprof and then attach it at program start.
Try appmon:start() in a shell.
The standard vi-way of browsing large code bases is to create tags-files so you can jump to the definition point of the thing-under-the-cursor. Emacs can do the same. I have a make tags target to create these files so I can easily make jumps around in the source code. When jumping you have a stack of the former jumps so you can return to the point you jumped from later on.
Finally there are tools like xref which can be worked to create call graphs and find odd things in the code. It will require some coding on your part, but it does provide you with the necessary tooling.
Viper mode provides Vi-style keybindings. That's probably the easiest way to start. Enabling a new mode in your .emacs is a pretty fundamental part of using Emacs, so it's a learning hurdle best cleared early!
The Emacs manual contains more detailed information about making individual changes to your keybindings.
You may find Xah's keybinding tutorial helpful, as it provides many examples.
Emacs has this seemingly very nice facility for building autoload files based on magic source code comments ("autoload cookies") of the form ;;;###autoload, which are to be placed on lines by themselves immediately above each definition to be autoloaded; see (elisp)Autoload.
This would seem to be an ideal tool for maintaining autoloads for those little one-file packages that Emacs users inevitably end up installing in their profiles. There's just one small problem: this facility (in GNU Emacs, anyway) appears to be almost entirely focused on generating the loaddefs.el file for Emacs itself, with very little (if any) concession to other uses.
This doesn't stop largish packages from using the autoloads.el machinery to build their own autoload files, but those that I've looked at have a fair bit of rather hairy code devoted to making it do what is needed, though some of the hairiness might be due to GNU Emacs/XEmacs divergence.
(I think XEmacs is a bit better on this front, probably due at least in part to the fact that its official package system uses this machinery to make separate autoload files for each package. Hopefully GNU Emacs' inclusion of the ELPA package system, which also uses this machinery, leads to similar improvements on their side.)
So, my question for you is:
How should I maintain an autoload file for all .el files in a directory, assuming they have all the necessary ;;;###autoload comments (autoload cookies) in place already?
[Hmm. block quotes look a lot coolor on tex.SE...]
I'm currently using GNU Emacs 23.2.1, though the farther back an answer works, the better. (For that matter, it'd be nice if it would also work with XEmacs.)
I'm on Windows, but have MSYS installed alongside Emacs so sh/bash scripts will probably be fine as long as they don't call anything terribly exotic.
[I'm not entirely certain this doesn't belong on superuser rather than on SO. If there is already a package that can take care of this with only a small amount of configuration, it probably does; on the other hand if (as I suspect) there are only fairly rough snippets of code that might require a good deal of direct changes, I think it probably belongs here on SO.]
Check out this answer to "emacs23 / elisp: how to properly autoload this library?".
In summary, you dump all the packages in a particular directory, and create a package of your own update-auto-loads.el, which builds the loaddefs.el and provides a function you can regularly run to rebuild the loaddefs.el file whenever you want.
In 2017, you don't have to maintain it yourself. Instead, you can use a modern package manager such as straight.el which is specifically designed to take care of this (as well as many other things) for you.
You could use package.el instead of straight.el, but package.el has a number of serious problems, and a particularly relevant one is that it's very awkward to get package.el to load a local package you've written yourself, and in particular package.el has absolutely no support for making modifications to a package once it's been installed.
You could use other package managers, as well. I wrote an extensive section comparing straight.el to other package managers.
I started using Dist::Zilla several months ago. However, at YAPC::NA someone mentioned that they use ShipIt instead. Then today I noticed a .shipit file in miyagawa's cpanminus directory on github, so I decided to look into it some more...
My initial impression is that ShipIt has a subset of what is available with Dist::Zilla, but I don't want to jump to conclusions. So, for those who have had experience with both, what are the strengths/weaknesses of ShipIt vs Dist::Zilla?
crossposted at perlmonks
I'm the author of Dist::Zilla.
I evaluated ShipIt pretty extensively before choosing to go ahead and write Dist::Zilla, and initially they covered almost exactly the same problem space: doing all the boring grunt work of building and uploading a CPAN distribution. All of the features that Dist::Zilla now has beyond ShipIt are later additions, more or less.
If you only need the features of ShipIt, I still advise you to strongly consider Dist::Zilla, for one very simple reason: hackability. If I had been able to not write something new, I would've used ShipIt, but I found it to be underdocumented and difficult to extend. Its plugins were not generic enough and the core behavior made too many assumptions about how you'd like to work.
Dist::Zilla was inspired specifically by this problem: it turned everything into a plugin, and every plugin was given a very, very small interface so that its assumptions would be forcibly limited.
One benefit of ShipIt over Dist::Zilla is that ShipIt has (to the best of my knowledge) no plugins that will alter the way you actually write your code. This means your documentation will still look the same, you will still have a Makefile.PL, and so on. Some hackers don't like that so many DZ-based dists fundamentally change the assumptions of how to test and build CPAN code from its source repository. ShipIt will never change that.
It's possible to avoid using any such plugins with Dist::Zilla, but in general my experience is that people do use them, almost always, in one form or another.
As far as I can tell, my initial impressions were correct.
ShipIt provides functionality for releasing distributions:
keeping track of version numbers
integrating with version control
uploading to CPAN
displaying the changelog file in an editor so that you can edit it before release.
Dist::Zilla, by default, provides the ability to upload distributions to CPAN with a single command (i.e. dzil release). Dist::Zilla also has functionality for creating new distributions (i.e. dzil new My::New::Module). It also automatically generates so many of the files that I used to have to maintain by hand.
Using plugins, Dist::Zilla seems able to provide most, if not all, of the functionality available with ShipIt. It is also relatively easy to add brand new features using plugins.
I am relatively new in a software development. I have noticed that in some cases a text editor with extended text processing capabilities (I use Notepad++) gives me a better productivity than an IDE (I use the Eclipse and the Netbeans). In the era of IDEs, does it makes sense to learn emacs (or some other tool that you suggest?)
Yes, and no.
Yes for the exact same reason why a doctor should be able to get an approximate diagnose from your symptoms by using his experience, and not putting the list of symptoms in a google query and find the answer.
Yes for the exact same reason why airliner pilots are taught to fly without fly-by-wire even if all airplanes are today fly-by-wire, so almost everybody is able to keep them flying.
No, because if you need specific tools to make your life easier, such as GUI designers, Intellisense, access to documentation, then clearly Emacs is not enough.
Still, I remember that many developers at Microsoft organized a fund raising for uganda vim children.
Summing up, you need to use what makes you more productive. In many cases, emacs (or vim) is more productive than a huge IDE that makes coffee.
Even if you were using an IDE, it's still useful to know Emacs/VIM. You don't have your IDE around all the time, and while doing something via SSH, you don't really have any other option (yeah you can use nano, but thats not very effective).
When you do software development, you often deal with a lot of text besides code. I may use an IDE for most of my coding, but often I'll use Vim for plain ol' text viewing and manipulation.
Sometimes I need to view code, SQL scripts, XML, CSV, or TXT files. Other times I may want to perform bulk replacements on those files, or extract out certain chunks of text from it.
IDEs are good for writing and refactoring code, but aren't meant to be used for generic text viewing and manipulation. For that, I'd recommend having the full power of something like Emacs or Vim. Notepad++ can be good too.
In short, use the right tool for the job.
Everyone here seems to think that Emacs/Vim are light-weight compared to IDE's. This couldn't be further from the truth. Even the best IDE's do not have the features that Emacs does. In what IDE's can you program completely without moving your hands from the keyboard, read your email, chat with Jabber, run an integrated debugger, view your calendar, program your own functions, and send dbus commands? That's only the surface of what Emacs allows; I'm sure Vim has similar capabilities.
Ignoring productivity completely; remember why you started programming in the first place. You like to create stuff, you like to know how stuff works, you like creating clever solutions to obscure problems, you like tinkering, you like learning new things, you like creating tools that help you do things.
With this in mind the answer is yes, learn it if you think it will add to the delight to your days. Maybe you will also get some work done along the way. Fiddling with eamcs will not make you melt. You might even make a life-long friend of it. Happy Hacking.
Emacs is an IDE. In fact, you could argue it's a whole operating system.
vi, on the other hand, is an editor.
Yes, it makes sense to learn vi, since it's about the only editor you can use on anything vaguely posix, even if the GUI isn't running or the network is incredibly slow (vi is usable at 300 baud). Basically, it's the unix administrator's safety net of an editor. I've used it to rescue myself from broken device drivers on an OS X server that would only come up so far as single user, so even the must GUIfied Unix out there can still be saved by humble vi.
It makes some sense to learn emacs too, but perhaps not quite so much these days.
I would say it helps to learn emacs (or say vim).
Personally I'm comfortable using IDE (eclipse) for Java developement, when I code in perl or python I prefer to use emacs. Also if you are in a resource constriant environment (say) starting an IDE like Eclipse (which would crawl if there is anything less than 1GB RAM) to write a perl script might not be feasible. In such cases a vim would be neat tool of choice.