When I use Eclipse or Visual Studio or Xcode, structure of the project gets created automatically and one can get started immediately.
Now I am trying to create a project on Allegro CL express edition and if I collect all lisp files and run it, it shows package errors which I believe is because of files not getting executed in a specific order as packages must be created in some file that must be executed first.
I am not able to understand how to approach this problem. There is no or little information on creating large projects in lisp and how to incrementally deal with its size and complexity.
I want to port Maxima onto Allegro CL. Can I get some help here and also if one can explain this break up of code in multiple files and packages and basically how to load the whole system.
I know basics of lisp but I don't understand lisp project's structure.
Good news: Maxima already has an Allegro port. You should be able to build it using ./configure --with-acl then type make. I haven't used this recently, but I would expect this to work.
If you want to know more about how stuff is loaded, look at src/maxima.system. It's a bit archaic because it is written for defsystem, which has now been replaced pretty much everywhere else by asdf.
Well, Maxima can be compiled via defsystem or asdf by several Lisps, including Allegro. See INSTALL.lisp for details.
The only limitation that I know of is that the Allegro Express version cannot compile the SLATEC-derived code (the functions translated from Fortran are too big or something like that). So you will have to comment out the SLATEC stuff in maxima.system or maxima.asd.
Related
I have developed some Common Lisp functions in a couple of Lisp source files that I'd like easily available to other functions I write, or make available on github if I think they'd be useful for someone else. For now, I've just been putting them in some pre-defined folder and using (require "/path/to/my/modules/module.lisp").
I'm wanting to understand what is the correct (canonical Lisp) way of defining Lisp library of modules. And the second part of the question is how to use them (whether I've defined them, or whether I've obtained one from someone else).
I've been reading a lot about defpackage and defsystem and asdf. But everything I've read seems to focus on some specific corner of the universe of this task. I'm having a lot of trouble seeing the big picture of custom module creation, deployment, and use. So assuming I have the Lisp environment in front of me (CLISP or SBCL) and one or more .lisp files I'd like to make a package or library out of, is there a document somewhere that explains what steps are required to do that? It's probably something I've already read, but didn't track due to not understanding the context. What I've read about ASDF seems functionally to be what I'm after, but I'm left not understanding whether ASDF is my only option, or whether it just happens to be a de facto standard and what most other people use, or whatever. I played with it a bit in SBCL and wasn't sure I was using it right, and didn't see info on how to set it up in CLISP. So I'm wanting to understand what is the up-the-middle, vanilla approach to this task.
I know this is a big, sloppy set of sub-questions. Again, if there are some good references to look at, I can read. I'm just having some trouble getting a big picture view of how this is supposed to work, and whether there is any "best" approach, or whether, in Lisp, it's a bit of a "Wild West" choose-the-library-manager-you like approach. I did the Google thing and read anything that looked relevant, but my brain is spinning from all of it.
Thanks.
A system is a collection of files and sub-systems. One can compile or load such a system. There are also other operations possible. It keeps track of dependencies and tries to do a minimal amount of work.
If you are using SBCL and CLISP, then ASDF is the tool to choose. See http://www.cliki.net/asdf
ASDF provides, amongst other things, a DEFSYSTEM macro to describe such systems.
Don't use PROVIDE/ REQUIRE- unless you know what you are doing. ASDF is the way to go.
To publish your code and make it easily loadable by others then use QUICKLISP. See: http://www.quicklisp.org/beta/
A package in Common Lisp is not like a package in most other senses. It's not an archive of list files, but more like what most other languages would call a module or namespace that lets you select which symbols (names) from your code you want to show to the outside world of code.
If you just have one file for your little library, you can just distribute that. If you have multiple files, that where a tool like ASDF comes in to make sure, for example, that files defining macros are loaded before the files that use those macros.
Here are some good resources for you to look at, both chapters from Practical Common Lisp:
21. Programming in the Large: Packages and Symbols: This will give you a much better sense of what packages are and how and why to use them.
32. Conclusion: What's Next?: The "Delivering Applications" section has some good resources on this stuff in general, and has some mention of ASDF.
ASDF and Quicklisp are useful tools, that is an established fact. However, I would like to give an alternative point of view on the concept of "library" as it is discussed on previous answers
ASDF is designed to automate the compilation and loading of a set of source files. It is to CL what make is to Unix. It is perfectly valid to write and distribute a program without make just as it is fine to write a program in CL that does not use ASDF.
If your project is simple enough, it is sufficient to provide a file (e.g. load.lisp) that contains the statements to load the dependent files of your project in the right order. Therefore ASDF is not involved in the concept of a library.
In a provocative way, I would say that the canonical way of defining a module in CL is to use the defpackage declaration because it is the language unit that allows a programmer to isolate his/her declarations from those of someone else.
Then is the question of how to make it available to others. If you write portable CL code, then ASDF is the most popular system definition facility and you should use it. If you want to make it easier for others to obtain, then Quicklisp is the tool that changed the face of CL in the last few years.
Finally I would like to add that neither ASDF nor Quicklisp are standards, they are tools (which does not remove to their usefulness). ANSI Common Lisp is a standard and I would love to see a system definition standardized in CL.
Yes, ASDF is a de facto standard, and Quicklisp is another standard.
From Lisp's viewpoint what you want is to define one or several namespaces (packages). This is regulated by the ANSI standard. From your code's viewpoint you want to arrange a bunch of files so that they became a whole and somehow provide that packages. This is where ASDF plugs in. And Quicklisp allows you to manage ASDF systems in the easiest way concievable. You can both download a lot of libraries from Quicklisp's repository and manage your local systems creating symlinks in the quicklisp/local-project folder.
If you have Quicklisp installed, you can type
(ql:quickload :cl-fad)
at the REPL and thus load the CL-FAD library (possibly downloading it); then the packages CL-FAD and PATH become available, you forget about ASDF-systems and stick with the logic of packages.
A good idea would be to take a look at asd-files of several projects downloaded with Quicklisp.
I setup my own libraries, that I use often, using ASDF system definitions and linking them into the local-projects (or similar named) folder of Quicklisp. That allows me to load them using quicklisp like any published package.
If you want to learn how to setup such an ASDF system, I would suggest installing Quicklisp (which for once is really easy, when it comes to installing 'unfinished' software from the internet), quickload a well known package or two and look at its .asd file, consulting the documentation when in doubt.
This way, you have your libraries already setup to be published like most of the well known CommonLisp packages out there.
Can go run dynamically in order to be used for a plugin based application ?
In eclipse, we can create some plugins that Eclipse can run dynamically.
Would the same thing be possible in Go ?
I'll argue that those are two separate problems :
having dynamic load
having plugins
The first one is simply no : A Go program is statically linked, which means you can't add code to a running program. And which also means you must compile the program to let it integrate plugins.
Fortunately, you can define a program accepting plugins in Go as in most languages, and Go, with interfaces and fast compilation doesn't make that task hard.
Here are two possible approaches :
Solution 1 : Plugin integrated in the main program
Similarly to Eclipse plugins, we can integrate the "plugins" in the main program memory, by simply recompiling the program. In this sense we can for example say that database drivers are plugins.
This may not feel as simple as in Java, as you must have a recompilation and you must in some point of your code import the "plugin" (see how it's done for database drivers) but, given the standardization of Go regarding directories and imports, it seems easy to handle that with a simple makefile importing the plugin and recompiling the application.
Given the ease and speed of compilation in Go, and the standardization of package structure, this seems to me to be a very viable solution.
Solution 2 : Separate process
It's especially easy in Go to communicate and to handle asynchronous calls. Which means you could define a solution based on many process communicating by named pipes (or any networking solution). Note that there is a rpc package in Go. This would probably be efficient enough for most programs and the main program would be able to start and stop the plugin processes. This could very well feel similar to what you have in Eclipse with the added benefits of memory space protection.
A last note from somebody who wrote several Eclipse plugins : you don't want that mess; keep it simple.
Go 1.8 supports plugins (to be released soon Feb 2017.)
https://tip.golang.org/pkg/plugin/
As dystroy already said, it's not possible to load packages at runtime.
In the future (or today with limitations) it may be possible to have this feature with projects like go-eval, which is "the beginning of an interpreter for Go".
A few packages I found to do this:
https://golang.org/pkg/net/rpc/
https://github.com/hashicorp/go-plugin
https://github.com/natefinch/pie
Emacs has this seemingly very nice facility for building autoload files based on magic source code comments ("autoload cookies") of the form ;;;###autoload, which are to be placed on lines by themselves immediately above each definition to be autoloaded; see (elisp)Autoload.
This would seem to be an ideal tool for maintaining autoloads for those little one-file packages that Emacs users inevitably end up installing in their profiles. There's just one small problem: this facility (in GNU Emacs, anyway) appears to be almost entirely focused on generating the loaddefs.el file for Emacs itself, with very little (if any) concession to other uses.
This doesn't stop largish packages from using the autoloads.el machinery to build their own autoload files, but those that I've looked at have a fair bit of rather hairy code devoted to making it do what is needed, though some of the hairiness might be due to GNU Emacs/XEmacs divergence.
(I think XEmacs is a bit better on this front, probably due at least in part to the fact that its official package system uses this machinery to make separate autoload files for each package. Hopefully GNU Emacs' inclusion of the ELPA package system, which also uses this machinery, leads to similar improvements on their side.)
So, my question for you is:
How should I maintain an autoload file for all .el files in a directory, assuming they have all the necessary ;;;###autoload comments (autoload cookies) in place already?
[Hmm. block quotes look a lot coolor on tex.SE...]
I'm currently using GNU Emacs 23.2.1, though the farther back an answer works, the better. (For that matter, it'd be nice if it would also work with XEmacs.)
I'm on Windows, but have MSYS installed alongside Emacs so sh/bash scripts will probably be fine as long as they don't call anything terribly exotic.
[I'm not entirely certain this doesn't belong on superuser rather than on SO. If there is already a package that can take care of this with only a small amount of configuration, it probably does; on the other hand if (as I suspect) there are only fairly rough snippets of code that might require a good deal of direct changes, I think it probably belongs here on SO.]
Check out this answer to "emacs23 / elisp: how to properly autoload this library?".
In summary, you dump all the packages in a particular directory, and create a package of your own update-auto-loads.el, which builds the loaddefs.el and provides a function you can regularly run to rebuild the loaddefs.el file whenever you want.
In 2017, you don't have to maintain it yourself. Instead, you can use a modern package manager such as straight.el which is specifically designed to take care of this (as well as many other things) for you.
You could use package.el instead of straight.el, but package.el has a number of serious problems, and a particularly relevant one is that it's very awkward to get package.el to load a local package you've written yourself, and in particular package.el has absolutely no support for making modifications to a package once it's been installed.
You could use other package managers, as well. I wrote an extensive section comparing straight.el to other package managers.
Here is the back story skip to the bottom if you do not care and only want to see the question.
So I have been playing around in LISP for a little while. Some basic functions, some classes ,and file IO. When I run across this article:
http://www.adampetersen.se/articles/lispweb.htm
And I am excited to try and use lisp for a web application. I go and download the packages, but for the life of me do not know how to load them into my Allegro IDE.
Hmm... ok, well the hunchentoot site says a lot of the basic packages are in LispWorks. So I download that. Still not sure how to get the source for the packages that I downloaded into these IDEs. They seem to have binaries of the packages?
Oh well maybe ill switch to my ubuntu server and apt-get all the packages and setup slime (i have not used it before because I just wanted to learn lisp. Learning emacs and lisp at the same time seemed real daunting). I apt get all the packages needed and load up slime and again same problem there aren't available.
I dig around some more and see this program called ASDF. It looks like ASDF is some kind of package builder for lisp? I don't know it seems confusing. I'm about to give up at this point.
If you are still reading this here is my question.
1. How do I load the source for these packages into my lisp environment. trying to learn lisp hasn't been too hard but the information about the environments has been sparse. Do I need to build the packages I download with ASDF.
2. Is there a simple way for someone to just get up and running in lisp without having to speed a large amount of time upfront learning all the tools?
Hmm... ok, well the hunchentoot site
says a lot of the basic packages are
in LispWorks. So I download that.
This just means that the author has written a lot of Lispworks-specific code in Hunchentoot. It does not mean that Hunchentoot only works on Lispworks.
Still not sure how to get the source for the packages that I downloaded into these IDEs.
You need to use ASDF.
They seem to have binaries of the packages?
That's unlikely.
Oh well maybe ill switch to my ubuntu server and apt-get all the packages and setup slime > (i have not used it before because I just wanted to learn lisp. Learning emacs and lisp
at the same time seemed real daunting).
Don't do it then. You don't need to use Emacs or Slime.
I apt get all the packages needed and load up slime and again same problem there aren't
available.
For quick results try clbuild: http://common-lisp.net/project/clbuild/
I dig around some more and see this program called ASDF. It looks like ASDF is some
kind of package builder for lisp? I don't know it seems confusing.
ASDF is a bit like a Makefile for Common Lisp applications.
I'm about to give up at this point.
That's about the worst thing you could so (at this or any other point). I'm glad you have decided to post your problems here instead.
How do I load the source for these packages into my lisp environment.
trying to learn lisp hasn't been too hard but the information about the
environments has been sparse. Do I need to build the packages I download with ASDF.
clbuild should give you all you need, but here are some hints if you don't want to use it:
CLISP, SBCL: ASDF is part of your Lisp. Run (require :asdf). Lispworks, Allegro: you need to download and load ASDF. Save asdf.lisp somewhere then run (load "/path/to/asdf.lisp").
For every library/application ("system" in ASDF speak) you need to download und unpack it to some place. Repeat until all dependencies are satisfied. Note down these places (directories).
For every place from step #2 add the place to the ASDF registry: (push "/path/to/dir/" asdf:*central-registry*). Don't forget the trailing slash.
Load the system using (asdf:oos 'asdf:load-op :system-name).
Is there a simple way for someone to just get up and running in lisp without having to speed a large amount of time upfront learning all the tools?
See above -- use clbuild.
The quickest way in Ubuntu is to use the packages included in that distribution. It is "ok" if you just want to try some things, but these versions are often comparatively old. I would recommend the packages sbcl and slime. If you don't know emacs yet, you can get into that quite fast through its built-in tutorial (C-h t (press Control-h, release, then press t)).
You can then start emacs, start slime (through M-x slime), open a lisp file (C-x C-f ~/lisp/first-try.lisp), and you're ready to go. As a tutorial for Lisp, I think that Practical Common Lisp is a very nice book, and it's freely available.
Now, when you have come to like Lisp, you might want more up to date packages. I would recommend to use clbuild for that (see the link for further information, including FAQ). You can then also build a new sbcl (bootstrapped by the distribution's version).
ASDF, by the way, is only a system definition facility. It doesn't know how to download packages, it only knows how to load systems into a running Lisp image. In other words, it just solves the problem of automatically loading the multiple files that some "system" (library) consists of in the right order. The newest versions allow loading a package (after it is installed, e.g. through clbuild) with a simple
(asdf:load-sys 'foo)
Older versions show ASDF's internal concept of operations:
(asdf:operate 'asdf:load-op 'foo)
The above load-sys is a shorthand for this common use case. Further information (one could say, all you need to know about it) is at the ASDF Getting Started guide. ASDF is also included in SBCL.
When you load a source file, it is automatically compiled (producing .fasl files (fast-load)) so that loading is much faster next time.
Probably one of the fastest ways to get started is to use Lisp in a Box (or a spinoff like LispBox). These are full sets of everything you need.
You could also try the Lisp Resource Kit, which is a bootable CDROM with Lisp tools and documentation, all already set up for you. Just put it into your CDROM drive and boot!
All of these answers are good, however they've become a little outdated with the new popularity of Quicklisp. Very loosely speaking, quicklisp is the package manager to asdf's make. Once Quicklisp is installed on a system, you can use (ql:quickload "name of lisp library") to load that library into your lisp environment, including downloading it and any of its dependencies if required. For example, to download, install, and load Hunchentoot and all of its dependencies, use (ql:quickload "hunchentoot"). In later lisp sessions, calling (ql:quickload "hunchentoot") again will simply load the version already downloaded and installed, making ql:quickload a simple way to load any library available locally or remotely. To install Quicklisp, I refer you to quicklisp.org.
More detailed explanation
Lisp works a little differently from other languages when it comes to libraries. The first thing to know is that the language itself provides almost no library functionality- it has load (which goes through and runs each line in a file as if you'd typed them at the REPL) and compile-file (which creates a "fast load" file, a precompiled version of the file which loads and can perform much faster). Using just what the core language provides, in order to load a library you have to go to each of its files and load it (or (load (compile-file "filename")) it, for better load speed/performance). This got tedious, so a variety of libraries were created for managing the loading of libraries, and at the moment asdf is king (so much so that many lisp implementations actually bundle it). In asdf terminology, libraries are called "systems" and .asdf files describe all of the metadata involved in loading them- the systems they depend on and what order to load files in, mostly, but they still can get quite complex. Quicklisp, then, sits on top of asdf. Basically, when asdf discovers that it cannot locate a system, Quicklisp steps in and checks to see if that system is available from one of Quicklisp's online repositories, and if so downloads it and has asdf continue on its way. ql:quickload is basically just a wrapper around the asdf machinery for loading a system that enables quicklisp to help out as needed.
Are there any libraries out there that do this? Playing around with Common Lisp it seems like this would be one of the most useful things to lower barrier of entry for newcomers. ASDF seems mostly designed for deployment, not for rapid prototyping and development. Following threads on comp.lang.lisp it seems like people agree that CL's package system is powerful, but lacks the ease of something like Python's dead simple module system. It is FAIL in the sense that it's designed for power not usability.
Glad to know if I'm wrong. If I'm right, I'm stunned that noone has tried to build a Python module-like system on top of ASDF.
Zach Beane wrote how he nowadays starts new Common Lisp projects by using Quicklisp and Quickproject. This might be along the lines you want.
Not sure if it's ready for prime time or whether it fits your requirements at all, but here's a link to XCVB.
I don't know. I mostly use ASDF for my in-development compilation needs. Once you notice that you'd benefiot from more than one source file, open <projectname>.asd, slap in a basic ASDF system definition template and start slapping filenames in. As and when you notice a cross-file dependency, update the dependency list.
But, then, I use the exact same method dealing with Makefiles (yes, I know there are automatic dependency checkers that can do it for you, but since I mostly code on my own, it's easier to just amend the Makefile/ASDF definition as I go).
In SBCL, there's a hook on REQUIRE that checks for ASDF systems, so you end up with something that is about as convenient as Python's import, but somehow I suspect that is not what you meant.
This may not be the answer you want, but clearly you have some idea of what you want in a module system. Have you considered creating one yourself? That is, taking your limited domain, your limited requirements, your environment and simply pounding out whatever abstractions will quickly make your life easier?
That's one of the key benefits of Lisp I'm sure you know, is that these simple abstractions and little tools are typically very easy to craft in Lisp.
I'm not suggesting solving everyone who has a problem with the package system or ASDF, I'm simply suggesting solving your own problem as you understand it, which is likely simpler and smaller than some more powerful larger scope.
There is Mudballs now, too.
If you're looking for a piece of software to add this functionality to then it's a good bet.
If you want a command line tool that just uses bash to generate new common lisp project directory and file layouts, you may find one that I created for myself useful: lispproject. If it doesn't match your needs, go ahead and fork it or the repo it gets it templates from to suit your needs: lisp-project-template. Look at the sh file in lispproject repo to see how the templates are used. Also, please note that you may need to adjust the calls to sed to fit your platform as I am using this on macOS. Alternative sed calls are in the main script but just commented out if you need them.
it's designed for power not usability
that's how most Lisp gurus like it.