What are my options for developing a piece of software for the OLPC project. From looking at the various sites and wikis, I can honestly say that I am still totally confused. Is it to be sugar, c++, smalltalk or python? HELP!
Thanks,
First, the short answer: You want to use Python, and you want to make your python programs "sugarized".
Sugar is not a programming language or development toolkit. It's a GUI environment and "activity" framework. The applications that kids use on OLPC laptops are called "activities", and Sugar provides a bunch of tools for them to use so that they can store their information in the versioned object database (the "journal" from the user's perspective), can show up in the list of available activities, etc.
To make sugarized applications, you write them in python, using the pygtk and/or pygames APIs for graphical work and the Sugar APIs for other Sugar services (like storage, access to the camera, mic, the very cool networking stuff, etc.). You also use a Sugar API to make the program available as a Sugar activity.
You can write C++ code for the OLPC, but Python is the preferred language.
As for Smalltalk, the OLPC project does provide a Squeak environment, but it's intended primarily for kids to play with Smalltalk programming, rather than as a tool for building activities intended to be distributed for use on the OLPC. Not that you couldn't use it that way (I think sugarization of Smalltalk apps is possible -- not sure), but it would be a memory hog. Smalltalk isn't inherently more memory-hungry than Python, but the OLPC devs have done some clever things to optimize Python memory usage. Basically, nearly all of the memory consumed by the Python interpreter is shared among all Python-based activities. Adding a Smalltalk activity to the mix would mean running another interpreter.
You can find lots of activities to look at (with source code) at http://activities.sugarlabs.org. If you develop anything for the OLPC, it's highly recommended that you get a Sugarlabs account and use their git infrastructure (http://git.sugarlabs.org).
I'd also highly recommend that you join the Sugar development mailing list. It's intended primarily for discussion of development of the Sugar platform, but there are very helpful and very knowledgeable people there who can answer questions and point you to the right resources. http://lists.sugarlabs.org/listinfo/sugar-devel
EDIT: Another useful resource for getting started is:
http://wiki.sugarlabs.org/go/Development_Team/Quickstart
Faisal Anwar and JediErik developed the excellent Sugar Almanac, which covers a lot of what you need to know to develop for Sugar. It has library descriptions and sample code for topics such as
creating a valid Sugar activity bundle
handling presence, threading and internationalization
interfacing with the Journal and other Sugar-specific system-wide features.
handling mouse, video, and other inputs
Swillden's post is excellent. I will just add a few more points:
People have successfully created Sugarized Squeak activities (see several games from Potsdam University as well as the work by OLE Nepal). OLE Nepal credits the rapid-prototyping aspects of the Squeak environment for their ability to quickly create a curriculum satisfying the desires of the Nepal teachers. I believe that these activities take longer to load, though; as Swillden points out, they lack the "home field" advantage of Python.
I recommend using Python, unless you are significantly more comfortable with the Squeak environment. The approach I used when I developed a Sugar activity (Implode) was to first develop the activity as an application using just Python/pygtk on a standard desktop (Windows or Ubuntu), then port it to Sugar. The code/debug cycle is faster on the desktop since you don't have to switch to an emulator or check error messages in the "Log" activity. If you architect the code right, you can isolate most of the desktop/Sugar differences to a couple modules, so that you can continue to develop and test in both environments. I wrote a pygtk activity, but I expect a pygame-based activity could be created in a similar manner. Of course, if your activity relies on access to certain Sugar-specific features -- like speech synthesis or mesh networking -- this approach may not work as well.
If you want to code in C or C++, whether for performance-critical or legacy code, I recommend writing it as a Python extension module called from a Python-based activity. I believe this is how the Write activity (wrapping Abiword) and Browse activity (wrapping Firefox) are implemented. If you have an existing X application in C/C++, it is possible to make it run under Sugar (see the SimCity, Etoys, and XaoS activities), but it will lack the look and feel of the other Sugar activities.
Finally, I've found the easiest way to add some particular functionality to a Sugar activity is to first find an existing activity that already does it, then read the code to find out how they did it. The Sugar system is not particularly well-documented yet; in some cases the only documentation is the code itself. As Swillden points out, the code for most activities -- as well as for Sugar itself -- is available in the SugarLabs git repository.
Related
I have integrated Lua with my ObjC code (iphone game). The setup was pretty easy, but now, I have a little problem with the bridging. I have googled for results, etc... and it seems there isn't anything that could work without modifications. I mean, I have checked luaobjc bridge (it seems pretty old and dicontinued), I heard about LuaCocoa but it seems not to work on iphone, and wax is too thick.
My needs are pretty spare, I just need to be able to call objc methods from lua and don't mind having to do extra work to make it work (I don't need a totally authomatic bridging system).
So, I have decided to build a little bridge myself based on this page http://anti-alias.me/?p=36. It has key information about how to accomplish what I need, but the tutorial is not completed and I have some doubts about how to deal with method overloading when called from lua, etc...
Do anybody know if there exist any working bridge between objc and lua on the iphone or if it could be so hard to complete the bridge the above site offers?
Any information will be welcomed.
Don't reinvent the wheel!
First, you are correct that luaobjc and some other variants are outdated. A good overview can be found on the LuaCocoa page. LuaCocoa is fine but apparently doesn't support iPhone development, so the only other choice is Wax. Both LuaCocoa and Wax are runtime bridges, which means that you can (in theory) access every Objective-C class and method in Lua at the expense of runtime performance.
For games and from my experience the runtime performance overhead is so significant that it doesn't warrant the use of any runtime binding library. From a perspective of why one would use a scripting language, both libraries defy the purpose of favoring a scripting language over a lower-level language: they don't provide a DSL solution - which means you're still going to write what is essentially Objective-C code but with a slightly different syntax, no runtime debugging support, and no code editing support in Xcode. In other words: runtime Lua binding is a questionable solution at best, and has lots of cons going against it. Runtime Lua bindings are particularly unsuited for fast-paced action games aiming at a constantly high framerate.
What you want is a static binding. Static bindings at a minimum require you to declare what kind of methods will be available in Lua code. Some binding libraries scan your header files, others require you to provide a special declaration file similar to a header file. Most binding libraries can use both approaches. The benefit is optimal runtime performance, and being able to actually design what classes, methods and variables Lua scripts have access to.
There are but 3 candidates to bind Lua code to an iPhone app. To be fair, there are a lot more but most have one or more crucial flaws or are simply not stable or for special purposes only, or simply don't work for iPhone apps. The candidates are:
tolua and tolua++
luabind
SWIG
Big disadvantage shared by all Lua static binding libraries: none of them can bind directly to Objective-C code. All require to have an additional C or C++ layer available that ultimately interfaces with your Objective-C code. This has to do with how Objective-C works as a language and how small a role it has played (so far) when it comes to embedding Lua in Objective-C apps.
I recently evaluated all three binding libraries and came to enjoy SWIG. It is very well documented but has a bit of a learning curve. But I believe that learning curve is warranted because SWIG can be used to combine nearly any programming and scripting language, it can be advantageous to know how to use SWIG for future projects. Plus, once you understand their definition file implementation it turns out to be very easy (especially when compared to luabind) and considerably more flexible than tolua.
OK, bit late to the party but in case others come late also to this post here's another approach to add to the choices available: hand-code your LUA APIs.
I did a lecture on this topic where I live coded some basic LUA bindings in an hour. Its not hard. From the lecture I made a set of video tutorials that shows how to get started.
The approach of using a bindings generation tool like SWIG is a good one if you already have exactly the APIs that you need to call written in Objective-C and it makes sense to bring all those same API's over into LUA.
The pros of the hand-coding approach:
your project just compiles with one standard Xcode target
your project is all C & Obj-C
the LUA is just data shipped along with your images
no fiddling with "do I check in generated code" to Git
you create LUA functions for just the things you want
you can easily have hosted scripts that live inside an object
the API is under your control and is well known
dont expose engine APIs to level building team/tools
The last point is just that if you have detail functions that only make sense at the engine level and you don't want to see those when coding the game play you'll need to tell SWIG not to bind those.
Steffens answer is perfect and this approach is just another option, that may suit some folks better depending on the project.
I'm moving away from strict Android development and wanting to create iPhone applications. My understanding is that I can code the backend of iOS applications in C/C++ and also that I can use the NDK to include C/C++ code in Android apps. My question however is how? I've googled quite a bit and I can't find any clear and concise answers.
When looking at sample code for the NDK, it seems that all the function names etc. are Android (or at least Java) specific and so I would not be able to use this C/C++ backend to develop an iPhone frontend?
I'd appreciate some clarification on this issue and if at all available some code to help me out? (even just a simple Hello World that reads a string from a C/C++ file and displays it in an iOS and Android app).
Thanks guys
Chris
Note that I almost exclusively work on "business/utility/productivity" applications; things that rely heavily on fairly standard UI elements and expect to integrate well with their platform. This answer reflects that. See Mitch Lindgren's comment to Shaggy Frog's answer for good comments for game developers, who have a completely different situation.
I believe #Shaggy Frog is incorrect here. If you have effective, tested code in C++, there is no reason not to share it between Android and iPhone, and I've worked on projects that do just that and it can be very successful. There are dangers that should be avoided, however.
Most critically, be careful of "lowest common denominator." Self-contained, algorithmic code, shares very well. Complex frameworks that manage threads, talk on the network, or otherwise interact with the OS are more challenging to do in a way that doesn't force you to break the paradigms of the platform and shoot for the LCD that works equally badly on all platforms. In particular, I recommend writing your networking code using the platform's frameworks. This often requires a "sandwich" approach where the top layer is platform-specific and the very bottom layer is platform-specific, and the middle is portable. This is a very good thing if designed carefully.
Thread management and timers should also be done using the platform's frameworks. In particular, Java uses threads heavily, while iOS typically relies on its runloop to avoid threads. When iOS does use threads, GCD is strongly preferred. Again, the solution here is to isolate the truly portable algorithms, and let platform-specific code manage how it gets called.
If you have a complex, existing framework that is heavily threaded and has a lot of network or UI code spread throughout it, then sharing it may be difficult, but my recommendation still would be to look for ways to refactor it rather than rewrite it.
As an iOS and Mac developer who works extensively with cross-platform code shared on Linux, Windows and Android, I can say that Android is by far the most annoying of the platforms to share with (Windows used to hold this distinction, but Android blew it away). Android has had the most cases where it is not wise to share code. But there are still many opportunities for code reuse and they should be pursued.
While the sentiment is sound (you are following the policy of Don't Repeat Yourself), it's only pragmatic if what you can share that code in an efficient manner. In this case, it's not really possible to have a "write once" approach to cross-platform development where the code for two platforms needs to be written in different languages (C/C++/Obj-C on iPhone, Java for Android).
You'll be better off writing two different codebases in this case (in two different languages). Word of advice: don't write your Java code like it's C++, or your C++ code like it's Java. I worked at a company a number of years ago who had a product they "ported" from Java to C++, and they didn't write the C++ code like it was C++, and it caused all sorts of problems, not to mention being hard to read.
Writing a shared code base is really practical in this situation. There is some overhead to setting up and keeping it organized, but the major benefits are these 1) reduce the amount of code by sharing common functionality 2) Sharing bug fixes to the common code base. I'm currently aware of two routes that I'm considering for a project - use the native c/c++ (gains in speed at the expense of losing garbage collection and setting targets per processor) or use monodroid/monotouch which provide c# bindings for each os's platform functionality (I'm uncertain of how mature this is.)
If I was writing a game using 3d I'd definitely use approach #1.
I posted this same answer to a similar question but I think it's relevant so...
I use BatteryTech for my platform-abstraction stuff and my project structure looks like this:
On my PC:
gamename - contains just the common code
gamename-android - holds mostly BatteryTech's android-specific code and Android config, builders point to gamename project for common code
gamename-win32 - Just for building out to Windows, uses code from gamename project
On my Mac:
gamename - contains just the common code
gamename-ios - The iPhone/iPad build, imports common code
gamename-osx - The OSX native build. imports common code.
And I use SVN to share between my PC and Mac. My only real problems are when I add classes to the common codebase in Windows and then update on the mac to pull them down from SVN. XCode doesn't have a way to automatically add them to the project without scripts, so I have to pull them in manually each time, which is a pain but isn't the end of the world.
All of this stuff comes with BatteryTech so it's easy to figure out once you get it.
Besides using C/C++ share so lib.
If to develop cross-platform apps like game, suggest use mono-based framework like Unity3D.
Else if to develop business apps which require native UI and want to share business logic code cross mobile platforms, I suggest use Lua embedded engine as client business logic center.
The client UI is still native and get best UI performance. i.e Java on Android and ObjectC on iOS etc.
The logic is shared with same Lua scripts for all platform.
So the Lua layer is similar as client services (compare to server side services).
-- Anderson Mao, 2013-03-28
Though I don't use these myself as most of the stuff I write won't port well, I would recommend using something like Appcelerator or Red Foundry to build basic applications that can then be created natively on either platform. In these cases, you're not writing objective-c or java, you use some kind of intermediary. Note that if you move outside the box they've confined you to, you'll need to write your own code closer to the metal.
A repeating theme in my development work has been the use of or creation of an in-house plug-in architecture. I've seen it approached many ways - configuration files (XML, .conf, and so on), inheritance frameworks, database information, libraries, and others. In my experience:
A database isn't a great place to store your configuration information, especially co-mingled with data
Attempting this with an inheritance hierarchy requires knowledge about the plug-ins to be coded in, meaning the plug-in architecture isn't all that dynamic
Configuration files work well for providing simple information, but can't handle more complex behaviors
Libraries seem to work well, but the one-way dependencies have to be carefully created.
As I seek to learn from the various architectures I've worked with, I'm also looking to the community for suggestions. How have you implemented a SOLID plug-in architecture? What was your worst failure (or the worst failure you've seen)? What would you do if you were going to implement a new plug-in architecture? What SDK or open source project that you've worked with has the best example of a good architecture?
A few examples I've been finding on my own:
Perl's Module::Plugable and IOC for dependency injection in Perl
The various Spring frameworks (Java, .NET, Python) for dependency injection.
An SO question with a list for Java (including Service Provider Interfaces)
An SO question for C++ pointing to a Dr. Dobbs article
An SO question regarding a specific plugin idea for ASP.NET MVC
These examples seem to play to various language strengths. Is a good plugin architecture necessarily tied to the language? Is it best to use tools to create a plugin architecture, or to do it on one's own following models?
This is not an answer as much as a bunch of potentially useful remarks/examples.
One effective way to make your application extensible is to expose its internals as a scripting language and write all the top level stuff in that language. This makes it quite modifiable and practically future proof (if your primitives are well chosen and implemented). A success story of this kind of thing is Emacs. I prefer this to the eclipse style plugin system because if I want to extend functionality, I don't have to learn the API and write/compile a separate plugin. I can write a 3 line snippet in the current buffer itself, evaluate it and use it. Very smooth learning curve and very pleasing results.
One application which I've extended a little is Trac. It has a component architecture which in this situation means that tasks are delegated to modules that advertise extension points. You can then implement other components which would fit into these points and change the flow. It's a little like Kalkie's suggestion above.
Another one that's good is py.test. It follows the "best API is no API" philosophy and relies purely on hooks being called at every level. You can override these hooks in files/functions named according to a convention and alter the behaviour. You can see the list of plugins on the site to see how quickly/easily they can be implemented.
A few general points.
Try to keep your non-extensible/non-user-modifiable core as small as possible. Delegate everything you can to a higher layer so that the extensibility increases. Less stuff to correct in the core then in case of bad choices.
Related to the above point is that you shouldn't make too many decisions about the direction of your project at the outset. Implement the smallest needed subset and then start writing plugins.
If you are embedding a scripting language, make sure it's a full one in which you can write general programs and not a toy language just for your application.
Reduce boilerplate as much as you can. Don't bother with subclassing, complex APIs, plugin registration and stuff like that. Try to keep it simple so that it's easy and not just possible to extend. This will let your plugin API be used more and will encourage end users to write plugins. Not just plugin developers. py.test does this well. Eclipse as far as I know, does not.
In my experience I've found there are really two types of plug-in Architectures.
One follows the Eclipse model which is meant to allow for freedom and is open-ended.
The other usually requires plugins to follow a narrow API because the plugin will fill a specific function.
To state this in a different way, one allows plugins to access your application while the other allows your application to access plugins.
The distinction is subtle, and sometimes there is no distiction... you want both for your application.
I do not have a ton of experience with Eclipse/Opening up your App to plugins model (the article in Kalkie's post is great). I've read a bit on the way eclipse does things, but nothing more than that.
Yegge's properties blog talks a bit about how the use of the properties pattern allows for plugins and extensibility.
Most of the work I've done has used a plugin architecture to allow my app to access plugins, things like time/display/map data, etc.
Years ago I would create factories, plugin managers and config files to manage all of it and let me determine which plugin to use at runtime.
Now I usually just have a DI framework do most of that work.
I still have to write adapters to use third party libraries, but they usually aren't that bad.
One of the best plug-in architectures that I have seen is implemented in Eclipse. Instead of having an application with a plug-in model, everything is a plug-in. The base application itself is the plug-in framework.
http://www.eclipse.org/articles/Article-Plug-in-architecture/plugin_architecture.html
I'll describe a fairly simple technique that I have use in the past. This approach uses C# reflection to help in the plugin loading process. This technique can be modified so it is applicable to C++ but you lose the convenience of being able to use reflection.
An IPlugin interface is used to identify classes that implement plugins. Methods are added to the interface to allow the application to communicate with the plugin. For example the Init method that the application will use to instruct the plugin to initialize.
To find plugins the application scans a plugin folder for .Net assemblies. Each assembly is loaded. Reflection is used to scan for classes that implement IPlugin. An instance of each plugin class is created.
(Alternatively, an Xml file might list the assemblies and classes to load. This might help performance but I never found an issue with performance).
The Init method is called for each plugin object. It is passed a reference to an object that implements the application interface: IApplication (or something else named specific to your app, eg ITextEditorApplication).
IApplication contains methods that allows the plugin to communicate with the application. For instance if you are writing a text editor this interface would have an OpenDocuments property that allows plugins to enumerate the collection of currently open documents.
This plugin system can be extended to scripting languages, eg Lua, by creating a derived plugin class, eg LuaPlugin that forwards IPlugin functions and the application interface to a Lua script.
This technique allows you to iteratively implement your IPlugin, IApplication and other application-specific interfaces during development. When the application is complete and nicely refactored you can document your exposed interfaces and you should have a nice system for which users can write their own plugins.
I once worked on a project that had to be so flexible in the way each customer could setup the system, which the only good design we found was to ship the customer a C# compiler!
If the spec is filled with words like:
Flexible
Plug-In
Customisable
Ask lots of questions about how you will support the system (and how support will be charged for, as each customer will think their case is the normal case and should not need any plug-ins.), as in my experience
The support of customers (or
fount-line support people) writing
Plug-Ins is a lot harder than the
Architecture
Usualy I use MEF. The Managed Extensibility Framework (or MEF for short) simplifies the creation of extensible applications. MEF offers discovery and composition capabilities that you can leverage to load application extensions.
If you are interested read more...
In my experience, the two best ways to create a flexible plugin architecture are scripting languages and libraries. These two concepts are in my mind orthogonal; the two can be mixed in any proportion, rather like functional and object-oriented programming, but find their greatest strengths when balanced. A library is typically responsible for fulfilling a specific interface with dynamic functionality, whereas scripts tend to emphasise functionality with a dynamic interface.
I have found that an architecture based on scripts managing libraries seems to work the best. The scripting language allows high-level manipulation of lower-level libraries, and the libraries are thus freed from any specific interface, leaving all of the application-level interaction in the more flexible hands of the scripting system.
For this to work, the scripting system must have a fairly robust API, with hooks to the application data, logic, and GUI, as well as the base functionality of importing and executing code from libraries. Further, scripts are usually required to be safe in the sense that the application can gracefully recover from a poorly-written script. Using a scripting system as a layer of indirection means that the application can more easily detach itself in case of Something Bad™.
The means of packaging plugins depends largely on personal preference, but you can never go wrong with a compressed archive with a simple interface, say PluginName.ext in the root directory.
I think you need to first answer the question: "What components are expected to be plugins?"
You want to keep this number to an absolute minimum or the number of combinations which you must test explodes. Try to separate your core product (which should not have too much flexibility) from plugin functionality.
I've found that the IOC (Inversion of Control) principal (read springframework) works well for providing a flexible base, which you can add specialization to to make plugin development simpler.
You can scan the container for the "interface as a plugin type advertisement" mechanism.
You can use the container to inject common dependencies which plugins may require (i.e. ResourceLoaderAware or MessageSourceAware).
The Plug-in Pattern is a software pattern for extending the behaviour of a class with a clean interface. Often behaviour of classes is extended by class inheritance, where the derived class overwrites some of the virtual methods of the class. A problem with this solution is that it conflicts with implementation hiding. It also leads to situations where derived class become a gathering places of unrelated behaviour extensions. Also, scripting is used to implement this pattern as mentioned above "Make internals as a scripting language and write all the top level stuff in that language. This makes it quite modifiable and practically future proof". Libraries use script managing libraries. The scripting language allows high-level manipulation of lower level libraries. (Also as mentioned above)
I usually write web apps in PHP, Ruby or Perl. I am starting the study of Scheme and I want to try some web project with this language. But I can't find what is the best environment for this.
I am looking for the following features:
A simple way of get the request parameters (something like: get-get #key, get-post #key, get-cookie #key).
Mysql access.
HTML Form generators, processing, validators, etc.
Helpers for filter user input data (something like htmlentities, escape variables for put in queries, etc).
FLOSS.
And GNU/Linux friendly.
So, thanks in advance to all replies.
Racket has everything that you need. See the Racket web server tutorial and then the documentation. The web server has been around for a while, and it has a lot of features. Probably the only thing that is not included is a mysql interface, but that exists as a package on PLaneT (Racket package distribution tool).
UPDATE: Racket now comes with DB support, works with several DBs including mysql.
You may want to have a look at Clojure:
Clojure is a dynamic programming language that targets the Java Virtual Machine. [...] Clojure provides easy access to the Java frameworks, with optional type hints and type inference, to ensure that calls to Java can avoid reflection.
Clojure is a dialect of Lisp, and shares with Lisp the code-as-data philosophy and a powerful macro system.
Interop with Java is straightforward in Clojure, so you can re-use any existing Java libraries as you need. I'm sure there are plenty that are useful for web development.
clojure-contrib has an SQL API, and there is ClojureQL as well, which should cover your DB access needs.
There is a web framework for Clojure called Compojure under development. There may be others, too.
Clojure's source is available on github under the EPL. Getting it running on Linux is easy; I just clone the git repos and run ant.
You can do web development with guile scheme. Its standard library includes the (sxml simple) module that is very useful for html generation, manipulation, and parsing. The guile-www library adds support for http, cgi, etc. The guile-dbi library provides access to MySQL and other databases. With these building blocks, you can implement everything from simple cgi scripts to web applications with their own HTTP server.
Try Weblocks, a Common Lisp web framework:
http://weblocks.viridian-project.de/
I've written a pretty extensive tutorial/ebook on the topic: http://lispwebtales.ppenev.com/
Quick summary:
It uses Common Lisp
It uses the Restas framework
It has examples for pretty much most of basic web development, including DB access, authentication, HTML generation and templating.
Since the Restas documentation is pretty much out of date, my tutorial is the closest thing to up to date docs.
Shows a few of the more advanced features, like policies, which allow you to write pluggable interfaces, for instance you can write a data store layer, and write back-ends for different storage mechanisms with relative ease, the module system which allows you to write reusable components, like auth frameworks and things like that.
It covers things like installing lisp, setting up the ASDF build system and the quicklisp package manager etc.
It's free online, and as soon as I finish it it will be free on leanpub as well. The source is on https://github.com/pvlpenev/lispwebtales under a CC license, the source code is MIT. Not all of it is published yet, and I'm in the process of revising.
This may be what you are looking for.
http://www.plt-scheme.org/
http://docs.plt-scheme.org/web-server/index.html
http://common-lisp.net/project/cl-weblocks/
If you are interested in Common Lisp to be exact and do not want to go the weblocks route I would recommend the following setup:
Use SBCL on Linux but with multiple thread support
Use Hunchentoot as a web server which will provide you with all the server processing required including sessions and cookies
Use ClSql to communicate with MySql it has ample documentation and is very stable.
For the HTMl generation you can use Dr Edi Weitz Cl-WHO (very well documented).
Note all the above are under GPL or similar license (one that works more for lisp programs)
Gambit Scheme has its own solution to web apps as well. It uses the Spork framework, based o the Black Hole module system (both by Per Eckerdal).
Andrew Whaley has an initial tutorial on how to get Gambit, Black Hole and Spork running a web app under Apache using mod_proxy. You might want to take a look at that.
On a (possibly) related note, Gambit will also compile your stuff to C and then to an executable, if you feel so inclined.
Paul Graham (and friends) made a lisp dialect specifically for writing basic web applications. It's called Arc, and you can get it at arclanguage.org.
It's probably not suited for really big complex websites and I'm not sure what state it's database support is at but Paul Graham knows how to write web applications in lisp, so Arc will make the HTTP/HTML part easy for you while you spend most of your brain cycles learning the lisp way.
Weblocks is nice tool for building web apps in Common Lisp, but a bit too heavy-weight for me.
We use the following stack:
OpenMCL (open source Lisp, very nice)
Portable Allegroserve (web server, HTML generator)
Our own Rails-like tools for doing Ajaxy stuff (update: this has now been open sourced as WuWei)
A variety of CL libraries like cl-json, cl-smtp, md5
I use my own, customized version of Scheme, derived from MzScheme. It has a new, simple web-application framework, a built-in web-server (not the one that comes with MzScheme) and ODBC libraries. (http://spark-scheme.wikispot.org/Web_applications). The documentation may not be exhaustive, as this is more of a personal tool. But there are lots of sample code in the code repository.
Clojure is a Lisp dialect which may interest you. At this point there's a pretty decent web development stack. I can recommend a few things:
The leiningen dependency manager which makes is really easy to install and manage libraries that you're using. Pretty nice set of plugins for it too. There's even a plugin for Clojurescript, which is a language based on Clojure that compiles to Javascript.
The ring HTTP server abstraction. Its used in most actual web frameworks. Its a pretty good idea to learn that first before jumping into an actual framework.
hiccup is a HTML dsl/templating language written in Clojure. Its very expressive! Reminds me a bit of Jade, in a sense.
composure would have to be the most popular web framework for Clojure. Its essentially a routing library like express.js.
Let's see what can be done with Common Lisp.
The state of the Common Lisp ecosystem (2015) and the Awesome Common Lisp list show us a couple of modern frameworks (Caveman, Lucerne, all built on the new Clack web application server, an interface for Hunchentoot and other servers). Let's discuss with our own findings.
update 2019: there's a new tutorial on the Common Lisp Cookbook: https://lispcookbook.github.io/cl-cookbook/web.html It covers routing, template engines, building self-contained binaries, deployment, etc.
update: a bit later, I found out Snooze, by the creator of Sly or Emacs' Yasnippet, and had a much better impression than say Caveman. Declaring endpoints is just like declaring functions, so some things that were tedious in Caveman are obvious in Snooze, like accessing the url parameters. I don't have much experience with it but I recommend checking it out.
update june 2018: also don't miss the ongoing rewrite of Weblocks, it's going to be huge ! :D http://40ants.com/weblocks/quickstart.html Weblocks allows to build dynamic webapps, without a line of Javascript, without separating the back and front. It is components-based, like React but server-side. It's very alpha as of writing (june 2018), but in progress, and it's working, I have a couple simple web apps working.
A simple way of get the request parameters (something like: get-get #key, get-post #key, get-cookie #key).
I found easier the Lucerne way, it iss as simple as a with-params macro (real world example):
#route app (:post "/tweet")
(defview tweet ()
(if (lucerne-auth:logged-in-p)
(let ((user (current-user)))
(with-params (tweet)
(utweet.models:tweet user tweet))
(redirect "/"))
(render-template (+index+)
:error "You are not logged in.")))
Caveman's way has been less clear to me.
Mysql access
Caveman advertises database integration (with Fukamachi's Datafly and sxql).
You can just use clsql or the Mito ORM: https://lispcookbook.github.io/cl-cookbook/databases.html
HTML Form generators, processing, validators, etc.
I don't know if there are form generators out there. edit: there are: cl-forms and formlets, or again 1forms, working with Caveman2.
Caveman does not have one (issue raised in 2011).
Helpers for filter user input data (something like htmlentities, escape variables for put in queries, etc).
Ratify is an input validation library, not integrated into a framework though.
FLOSS and GNU/Linux friendly: ✓
Other web stuff
Speaking about web, there are other nice libraries in CL land:
web servers: Woo is a fast HTTP server, faster than Nodejs (beware of charts…), wookie is an async http server,
Dexador is an HTTP client
Plump, lquery and CLSS make it easy to parse html and query the DOM.
cl-bootstrap offers twitter-bootstrap shortcuts for the cl-who templating engine (which kind of replaces Jade/Pug, even though we have usual templates too).
Ajax in Lisp
(remember, with Weblocks, see above, we might not need those)
With ParenScript, we can write JavaScript in Common Lisp, without living our usual workflow, and we can thus use the fetch web API to write Ajax calls.
Clojure would be perfect for this. With some very short, clean code, you can implement some very complex applications, such as blogs or forums.
You might want to consider the awful web framework for Chicken Scheme.
Natively supports PostgreSQL and SQLite
Built-in easy support for sessions
Shortcuts for some webdev idioms, like the (ajax) procedure
Your app can be easily compiled to a static executable (via csc -static) for easier deployment
The collection of all chicken libraries (eggs) isn't as versatile as in some other programming languages, but isn't awful either
Is there a general procedure for programming extensibility capability into your code?
I am wondering what the general procedure is for adding extension-type capability to a system you are writing so that functionality can be extended through some kind of plugin API rather than having to modify the core code of a system.
Do such things tend to be dependent on the language the system was written in, or is there a general method for allowing for this?
I've used event-based APIs for plugins in the past. You can insert hooks for plugins by dispatching events and providing access to the application state.
For example, if you were writing a blogging application, you might want to raise an event just before a new post is saved to the database, and provide the post HTML to the plugin to alter as needed.
This is generally something that you'll have to expose yourself, so yes, it will be dependent on the language your system is written in (though often it's possible to write wrappers for other languages as well).
If, for example, you had a program written in C, for Windows, plugins would be written for your program as DLLs. At runtime, you would manually load these DLLs, and expose some interface to them. For example, the DLLs might expose a gimme_the_interface() function which could accept a structure filled with function pointers. These function pointers would allow the DLL to make calls, register callbacks, etc.
If you were in C++, you would use the DLL system, except you would probably pass an object pointer instead of a struct, and the object would implement an interface which provided functionality (accomplishing the same thing as the struct, but less ugly). For Java, you would load class files on-demand instead of DLLs, but the basic idea would be the same.
In all cases, you'll need to define a standard interface between your code and the plugins, so that you can initialize the plugins, and so the plugins can interact with you.
P.S. If you'd like to see a good example of a C++ plugin system, check out the foobar2000 SDK. I haven't used it in quite a while, but it used to be really well done. I assume it still is.
I'm tempted to point you to the Design Patterns book for this generic question :p
Seriously, I think the answer is no. You can't write extensible code by default, it will be both hard to write/extend and awfully inefficient (Mozilla started with the idea of being very extensible, used XPCOM everywhere, and now they realized it was a mistake and started to remove it where it doesn't make sense).
what makes sense to do is to identify the pieces of your system that can be meaningfully extended and support a proper API for these cases (e.g. language support plug-ins in an editor). You'd use the relevant patterns, but the specific implementation depends on your platform/language choice.
IMO, it also helps to use a dynamic language - makes it possible to tweak the core code at run time (when absolutely necessary). I appreciated that Mozilla's extensibility works that way when writing Firefox extensions.
I think there are two aspects to your question:
The design of the system to be extendable (the design patterns, inversion of control and other architectural aspects) (http://www.martinfowler.com/articles/injection.html). And, at least to me, yes these patterns/techniques are platform/language independent and can be seen as a "general procedure".
Now, their implementation is language and platform dependend (for example in C/C++ you have the dynamic library stuff, etc.)
Several 'frameworks' have been developed to give you a programming environment that provides you pluggability/extensibility but as some other people mention, don't get too crazy making everything pluggable.
In the Java world a good specification to look is OSGi (http://en.wikipedia.org/wiki/OSGi) with several implementations the best one IMHO being Equinox (http://www.eclipse.org/equinox/)
Find out what minimum requrements you want to put on a plugin writer. Then make one or more Interfaces that the writer must implement for your code to know when and where to execute the code.
Make an API the writer can use to access some of the functionality in your code.
You could also make a base class the writer must inherit. This will make wiring up the API easier. Then use some kind of reflection to scan a directory, and load the classes you find that matches your requirements.
Some people also make a scripting language for their system, or implements an interpreter for a subset of an existing language. This is also a possible route to go.
Bottom line is: When you get the code to load, only your imagination should be able to stop you.
Good luck.
If you are using a compiled language such as C or C++, it may be a good idea to look at plugin support via scripting languages. Both Python and Lua are excellent languages that are used to script a large number of applications (Civ4 and blender use Python, Supreme Commander uses Lua, etc).
If you are using C++, check out the boost python library. Otherwise, python ships with headers that can be used in C, and does a fairly good job documenting the C/python API. The documentation seemed less complete for Lua, but I may not have been looking hard enough. Either way, you can offer a fairly solid scripting platform without a terrible amount of work. It still isn't trivial, but it provides you with a very good base to work from.