I think (see below) that I would like to structure a Clojure project as multiple modules, with ordered dependencies - just like Maven lets me do with multi-modules projects.
But I can't see how to do this with Leiningen - all I can see is the checkouts fix described in the FAQ which doesn't seem to be as powerful.
Will lein do this? Should I be using Gradle instead? Or is this kind of thing not needed?
Some more context: I am wondering how to architect a modular application that supports plugins (which I imagine means jars dumped on the classpath). And am wondering to what extent I can structure that as a core + plugins (I am thinking I should be able to do something with Clojure's dynamic code loading and not have to go with Java/OSGi). So I guess the driving motivation for a single project comes from wanting some way of packaging everything (the core + default plugins) as a single blob that is easy for the end user, but which can also be divided up (and which is built and tested in fragments, testing the logical independence of each module). More general advice about this is welcome
Update
A possible solution that wasn't mentioned below is using a Maven plugin - I assume that supports everything Maven does, but compiles Clojure, so will work with nested modules, etc.
First, it does not seem like Leiningen supports a module hierarchy like Maven does. The checkouts are the next closest thing it has. It should be sufficient though to develop a modular application in Clojure though.
For the project structure, I would have an API project, a "core" project, the plugins themselves, and a separate packaging project. The core and the plugins should only depend on the API. Which build tool you use to create the packaging project is up to you. Gradle would probably be more effective at handling the packaging, however having the "checkout" functionality Leiningen offers could make development of the system as a whole easier.
I would take a look at the code for Leiningen and Noir to figure out how to effectively handle this.
For dynamically loading the plugins, I would start with looking how Noir handles it in two of their files:
server.clj has namespace loading for all files under a particular namespace. Under the hood it uses tools.namespace, but you can easily see how it's used to require every namespace under a particular base. This is how Leiningen handles custom tasks as well - the base definition for the task should be in the leiningen.$task namespace.
core.clj has what I would use for plugin registration. In summary, use a map under an atom and add plugins to that map. I would advice wrapping the registration with a macro to keep your code cleaner.
What I listed above should be sufficient if you don't need to handle adding plugins at run time. If you don't have every plugin on the classpath during start-up, I would recommend utilizing pomegranite to add entries to the classpath. You can see an example in classpath.clj.
Related
I'm working on a scala sbt project, and I am at a point where I want to assemble the whole thing to share a .jar-file with others, so that they can use it on their side. For my local testing, I'm doing so using the sbt-assembly plugin, that works nicely.
When sharing, I would though prefer to only share the parts that are important for the other party (the project has huge components that irrelevant at the current point, and I'd prefer not to share these for various reasons). Concretely, they will be executing one particular main file, so it would be enough to pack everything that this file depends on.
Is there a way to accomplish this? I'd also be interested in doing this on the code level (i.e., create a copy of the project that only contains dependencies of that main file), but also while assembling or even modifying the jar file after assembly is okay. I did not find the tools to achieve any of these.
As I said, I'm dealing with a scala sbt project, and I'm working with IntelliJ IDEA; I'd also be happy with an IDE tool that does the job.
I have many small Scala applications and I would like a central place to manage versions of common dependencies.
I know I can set up a Build.scala file and define multiple projects. I used to have that, but not all of these applications are related in that sense. They just happen to share a common software architecture.
How do I achieve this with SBT? Currently I'm managing multiple .sbt files that I batch update (like dependencies.sbt, common.sbt etc.).
My initial approach was to start writing a plugin, but I got stumped along the way with getting dependencies working within the plugin.
I wanted to collect plugin settings and build settings. But I never got one plugin to depend on a set of other plugins.
Anyone tried this before, how do I manage central application profiles (both plugins and settings)?
Do you mean different applications? You can then just put your settings into global.sbt. You can look for more detailed description here
Considering that I am developing an end-user software program (as an uberjar) I am wondering what my options are to make it possible for the user to download a plugin and load that during runtime.
The plugin(s) should come compiled and without source code, so sth. like load is not an option.
What existing libraries (or ways of Java...?) exist to build this on?
EDIT: If you are not sure I would also be satisfied with a way that costs a reboot/-start of the main-program. However, what is important is that the source-code won't be included in any JAR file (neither main application nor plugin-jars, see :omit-source of Leiningen documentation).
To add a jar during runtime, use pomegranate which lets you add a .jar file to the classpath. Plugins of your software should be regular Clojure libs that follow certain conventions that you need to establish:
Make them provide (e. g. in an edn) a symbol to an object implementing a constructor/destructor mechanism such as the Lifecycle protocol in Stuart Sierras component library. In runtime, require and resolve that symbol, start the resulting object and hand it over to rest your programs plugin coordination facilities.
Provide a public API in your program that allows the plugins to interact with it in ways that you coordinate asynchronously e. g. with clojure.core.async (don't let one plugin block the entire program).
Make sure that the plugins have a coordinated way to expose their functionality to each other only if they desire so to enable a high degree of modularity among your plugins. Make sure that your plugin loader is capable of detecting dependencies among plugins and is capable of loading and unloading them in the right order.
I've not tried it myself, but you should in theory be able to get OSGi to work with Clojure.
There's a Clojure / OSGi integration library here:
https://github.com/aav/clojure.osgi
If I were to attempt to role my own solution, I would try using tools.namespace to load and unload plugins. I'm not entirely sure it will work, but it's certainly in the right direction. I think one key piece is that the plugin jars will have to be "installed" in a location that's already on the classpath.
Again, this is only the start of one possible solution. I haven't tried doing this.
I was wondering if there is a way to develop Eclipse plugins in Clojure. To be clear, the question is not about using Eclipse to write Clojure code.
Both Eclipse and Clojure run on the JVM and I feel there should be way to leverage the power of Clojure (and it's libraries) to develop plugins. I was specifically looking at using Korma, but overall I would like to move complete plug-ins to clojure if there is a natural way to do it.
Counterclockwise, the Eclipse plugin for Clojure, is written in mixed Java and Clojure. It uses clojure.osgi 1.2.10 yet.
So it is a live proof of concept that it is possible. And AFAIK, Counterclockwise is used successfully by hundreds of people.
There are some constraints, tho: Clojure's namespace is "global" to some "root classloader". E.G. if you package Clojure inside a bundle named, say, myapp.clojure, then you'll probably have a bunch of other bundles which will require myapp.clojure. Say for example myapp.bundle1, myapp.bundle2. When you do so, and, from each bundle, load in memory (require) the bundles namespaces, each one will be loaded from within the right ClassLoader (namespaces of myapp.bundle1 will be loaded within the context classloader of myapp.bundle1, and namespaces of myapp.bundle2 will be loaded within the context classloader of myapp.bundle2). This is great, because it allows java interop to work okay.
But just remember that in the end, namespaces loaded from bundle1 & bundle2 will be held by the "global namespace world" in bundle myapp.clojure.
To be honest, this has not yet proven a problem for Counterclockwise. Because inside the same Feature, having the bundles share one single Clojure instance is almost okay.
The potential drawbacks are:
if you use third party libraries, e.g. tools.logging, you will not be able to have namespaces in myapp.bundle1 depend on version X of tools.logging, and at the same time have myapp.bundle2 depend on version Y of tools.logging. That is, inside your feature where you have a shared clojure via bundle myapp.clojure, you work as if OSGi rules did not apply, as webapps work, for example.
does not scale well if massively applied: if every Eclipse Feature were to repackage its own version of Clojure, there would be some waste of memory. But this drawback is more theoretical than practical, yet. And this is a problem that can be addressed later, when the need for it emerges.
Note that for an Eclipse RCP Product, as opposed to an Eclipse plugin, these drawbacks vanish.
If you want to see how Counterclockwise has repackaged clojure, and uses clojure.osgi, you can look at its sourcecode:
http://github.com/laurentpetit/ccw.clojure.git
http://github.com/laurentpetit/ccw.git
HTH,
-- Laurent
It seems it's not available in Eclipse 3.x, but is planned for Eclipse 4, as mentioned in http://wiki.eclipse.org/E4/Languages .
There's also a post here on Stack Overflow asking about development of Eclipse plugins in languages other than Java that may have more information that you'd find useful.
It's perfectly possible to write Eclipse plug-ins in Groovy or Scala. Since Clojure produces .class files, it should be no different. However, plugins are normally exported using PDE Build, which only handles Java by default, so you will have to write a customCallback.xml file which can compile Clojure (see http://www.michel-kraemer.com/scala-projects-with-eclipse-pde-build-2 for Scala build).
Background
I'm a developer of the Vrapper project.
Vrapper contains of 2 major parts
Vim-emulation library (vrapper.core)
Eclipse part that makes a good use of it
We want vrapper.core to be Eclipse-unaware, so it's reusable
outside of the Eclipse. Currently, we can "vrap" all sorts of Eclipse
text editors and our little mock text editor that we use for unit testing.
vrapper.core implements all sorts of Vim commands, modes, etc.
Those all communicate with Platform - an interface that abstracts out
underlying stuff (text editor, clipboards, settings system, etc.).
When mode is created for an editor it asks platform if there are extra
commands that are approperiate for underlying editor, currently edited file type, etc.
EclipsePlatform provides those commands using Eclipse extension points mechanism.
So, let's consider the following projects (there are more):
vrapper.core - Eclipse-independent code for Vrapper
vrapper.eclipse - Eclipse plug-in that depends on vrapper.core
surround.core - Eclipse-independent code that emulates surround.vim (Vim plug-in)
surround.eclipse - Eclipse fragment for vrapper.eclipse
that makes it provide commands form surround.core.
There are two ways we can deal with those:
One plug-in to rule them all
This is how it should look like from Eclipse's perspective.
There is one plug-in that contains code from vrapper.eclipse and vrapper.core,
and one fragment that contains code from surround.core and surround.eclipse.
Many plug-ins
There are 3 plug-ins
two OSGified libraries vrapper.core, surround.core
vrapper.eclipse
surround.eclipse fragment depends on vrapper.core in this case
Problems
Many plug-ins solution have some issues with lazy class loading that I don't understand.
It's beacause when instances of modes from vrapper.core are created they need
classes from surround.core to be created (via vrapper.eclipse -> surround.eclipse).
This works if you run stuff from Eclipse and select all plug-ins from run configuration,
but if one deploys features & plugins and run eclipse normally an exception is thrown
because classes from surround.core cannot be found.
It's something in the spirit of surround.core asking for extra commands from
dependent plug-ins creates implicit circular dependencies.
What I mean by implicit dependencies is that no core class depends on eclipse-specific classes in compile time.
Modes (like vim normal mode) are core classes. They contain commands. There are some commands specific for particular Eclipse editors (like run this JDT-specific refactoring). Those commands implement core interfaces, but their code (obviously) lives in eclipse-specific projects. When mode is created it asks underlying platform for some extra commands - those extra commands are implemented in eclipse plug-ins. This is when lazy class loading in eclipse make everything blow up in runtime - classes for extra commands are referenced by extension points, but they are not yet loaded. Boom, exception.
I tried to work this around by using "one plug-in to rule them all" approach.
Having just one plug-in seems to be much better solution to me, but I couldn't make it work cleanly.
Only thing that succeeded for me was quite an ugly hack.
All .core projects had an Ant task that created .jar file with their classes
and dropped it into corresponding *.eclipse project
*.eclipse projects included that jars and had them enlisted in MANIFEST files.
The problem with this ugly hack approach (besides of it being ugly hack) is
that development becomes quite painful. Eclipse code navigation, code coverage
and few other things in Eclipse stops working.
Summary
We have eclipse independent library + eclipse specific stuff architecture,
but we really need all of this to live in one plug-in (because there are some dependencies in both directions).
How do I make code from few projects live into one plug-in/fragment?
It turned out that adding Eclipse-BuddyPolicy: dependent to MANIFEST.MF files, reexporting some dependencies and turning one fragment into plugin (so there is plug-in dependency for BuddyPolicy to track) was the right solution.
Problem solved :-)
From reading this it sounds as if the actual problem is the fact that there are dependencies in both directions. Can't you refactor your projects to make only the Eclipse-spcific proejcts depend on the core projects, and not the other way around?