Use of Clojure macros for DSLs - macros

I am working on a Clojure project and I often find myself writing Clojure macros for DSLs, but I was watching a Clojure video of how a company uses Clojure in their real work and the speaker said that in practical use they do not use macros for their DSLs, they only use macros to add a little syntactic sugar. Does this mean I should write my DSL in using standard functions and then add a few macros at the end?
Update:
After reading the many varied (and entertaining) responses to this question I have realized that the answer is not as clear cut as I first thought, for many reasons:
There are many different types of API in an application (internal, external)
There are many types of user of the API (business user who just wants to get something done fast, Clojure expert)
Is there macro there to hide boiler plate code?
I will go away and think about the question more deeply, but thanks for your answers as they have given me lots to think about. Also I noticed that Paul Graham thinks the opposite of the Christophe video and thinks macros should be a large part of the codebase (25%):
http://www.paulgraham.com/avg.html

To some extent I believe this depends on the use / purpose of your DSL.
If you are writing a library-like DSL to be used in Clojure code and want it to be used in a functional way, then I would prefer functions over macros. Functions are "nice" for Clojure users because they can be composed dynamically into higher order functions etc. For example, you are writing a functional web framework like Ring.
If you are writing a imperative DSL that will be used pretty independently of other Clojure code and you have decided that you definitely don't need higher order functions, then the usage will be pretty similar and you can chose whichever makes most sense. For example, you might be creating some kind of business rules engine.
If you are writing a specialised DSL that needs to produce highly performant code, then you will probably want to use macros most of the time since they will be expanded at compile time for maximum efficiency. For example, you're writing some graphics code that needs to expand to exactly the right sequence of OpenGL calls......

Yes!
Write functions whenever possible. Never write a macro when a function will do. If you write to many macros you end up with somthing that is much harder to extend. Macros for example cant be applied or passed around.
Christophe Grand: (not= DSL macros)
http://clojure.blip.tv/file/4522250/

No!
Don't be afraid of using macros extensively. Always write a macro when in doubt. Functions are inferior for implementing DSLs - they're taking the burden onto the runtime, whereas macros allows to do many heavyweight computations in a compilation time. Just think of a difference of implementing, say, an embedded Prolog as an interpreter function and as a macro which compiles Prolog into some form of a WAM.
And do not listen to those who say that "macros cant be applied or passed around", this argument is entirely a strawman. Those people are advocating interpreters over compilers, which is simply ridiculous.
A couple of tips on how to implement DSLs using macros:
Do it in stages. Define a long chain of languages from your DSL to the underlying Clojure. Keep each transform as simple as possible - this way you'd be able to easily maintain and debug your DSL compiler.
Prepare a toolbox of DSL components that you will reuse when implementing your DSLs. It should include target languages of different semantics (e.g., untyped eager functional - it is Clojure itself, untyped lazy functional, first order logic, typed imperative, Hindley-Millner typed eager functional, dataflow, etc.). With macros it is trivial to combine properties of all that target semantics seamlessly.
Maintain a set of compiler-building tools. It should include parser generators (useful even if your DSLs are entirely in S-expressions), term rewriting engines, pattern matching engines, implementations for some common algorithms on graphs (e.g., graph colouring), etc.

Here's an example of a DSL in Haskell that uses functions rather than macros:
http://contracts.scheming.org/
Here is a video of Simon Peyton Jones giving a talk about this implementation:
http://ulf.wiger.net/weblog/2008/02/29/simon-peyton-jones-composing-contracts-an-adventure-in-financial-engineering/
Leverage the characteristics of Clojure and FP before going down the path of implementing your own language. I think SK-logic's tips give you a good indication of what is needed to implement a full blown language. There are times when it's worth the effort, but those are rare.

Related

Pure FP in Scala?

I was under the impression that there are folks out there that do write pure applications using Scalaz, but based on this example: [ stacking StateT in scalaz ], it looks like anything real would also be impossibly hairy.
Are there any guidelines or examples of real, modular, loosely-coupled, pure applications in Scala? I'm expecting that this means scalaz.effect.SafeApp and RWST over IO, but I'd like to hear from folks who have done it.
Thanks.
Edit: In the absence of an answer, I've started collecting resources as an answer below. If you have any examples or related links to contribute, please do.
i think you are mixing two different things. one is pure functional programming and second is scala type system. you can do 'pure' programming in any language, even in java. if the language is funvtional than you will have pure functional programming.
does it make your programs work faster? depends on the program - it scales better but for single threaded parts you will rather loose performance.
does it 'save your cognition'? it depends on how good you are in what you are doing. if you work with FP, monads, arrows etc on the daily basis then i assume it may help significantly. if you show the code to the OO developer he probably won't understand anything.
does it save the development time? as previously, i think it may but to be honest it doesn't matter that much. you more often read the code rather than write it
can you do useful stuff in PFP? yes, some companies makes money on haskell
and now, can it be done in scala? for sure. will anyone do it in scala? probably not because it's too easy to break the purity, because type system is too weak and because there are better, 'more pure' tools for it (but currently not on jvm)
I guess I will start collecting resources here, and update as I find more.
Functional Reactive Programming: stefan hoeck's blog, github, examples
Monadic effect worlds for interacting safely with mutable data. (tpolecat)
Mellow database access for Scala (tpolecat)
Dependency Injection without the Gymnastics (tony, rúnar)
Google search for "extends SafeApp"

Can monads replace macros when writing a redis json driver?

In this blog entry by Jim Duey - he provides a list of reasons that you'd want to use monads. One his reasons is this:
So what are some clues that a monadic solution is possible? It seems to me that anytime you're copying and pasting code to define a new function that's similar to an existing one, there might be a monad lurking.
This is actually quite similar to the justification for using a Clojure style macro.
In this presentation by Aaron Bedra - he talks about a use case for Macros when writing a redis driver to generate json. On slide 66 - he shows an example of this.
(defmacro defcommand
[name params]
(let [p (parameters params)]
`(defn ~name ~params
My question is - can Monads be used to solve this problem of duplicate code when generating json for redis calls instead of a macro?
Assumptions
I understand it is more idiomatic to choose a macro over a monad in Clojure. For the purpose of this question I'm choosing to ignore what is idiomatic and just look at what is possible.
anytime you're copying and pasting code to define a new function
that's similar to an existing one
I would create a higher order function to get rid of copy paste, as simple as that.
Now, Macro or Monad or even a simple higher order function solves a single purpose but at different conceptual levels. The purpose is to "abstract away a common pattern in a system".
The different conceptual levels at which you can see a pattern appearing in a system are:
Same chunk of code - Use function / higher order function
Same chunk of code but that require some sort of "compile time" preprocessing to make it abstract - Macro
Pattern of "A value inside a box with some context" and it fits with the monad laws - Monads. (Maybe, IO, List - all these are API patterns where we see a value inside a box with some context)
I know it sounds very "abstract", but once you practice enough thinking about how to abstract general patterns in your system, you will eventually get the intuition about which tool to use for which pattern.
Monads and Macros operate at different levels of abstraction.
I think of macros as "code monkeys": anytime I find myself writing a lot of boilerplate, I take a moment and think whether macros can help.
Monads on the other hand are very powerful abstractions generally used in order to isolate and compose side effects in a purely functional setting. (I'm not an expert on Monads so take this with a grain of salt).
The bottom line is that I think they are solving fundamentally different problems.
If we take the example you provided, writing a Redis driver, any function accessing the network is a candidate for an abstraction through monads - in this case, both the IO and Maybe (or Either) monads. But not because of repetition, but rather because your functions have side effects.
Now this is where Monads in clojure can get harder than it's worth: If you want to compose both monads, you need to use monad transformers. Obviously this is not enforced by clojure since it's a dynamic language so one can argue debugging composed monadic code in a dynamic language can be very hard. A static type system such as Haskell's would be of great help here.
So while a redis driver can definitely be written in a monadic style, I believe the point Aaron is making in the presentation is avoiding repetition, and for that, I believe macros are a better fit in Clojure.
I hope this is helpful.
EDIT:
I should also node that monads in Clojure would be hardly useful without the Haskell inspired do notation which is essentially made possible by macros, providing further proof that macros are at a different level of abstraction.

How to enforce Functional Programming on Scala

I'm starting to learn Functional Programming and would like to do so with Scala, not Haskell or Lisp.
But some people claim that learning Scala as the first functional language slows down your learning of Functional Programming, because Scala allows you to program both ways, and one tends to program the procedural way when confronted with a hard problem.
How can i make sure that 'm programming in a purely functional way? Maybe, due to not being able to properly distinguish both styles, I'll inadvertently program procedurally).
I know, for example, that I should only use vals and not vars.
The other answers have made some good points, but for an attempt to quickly get down some guidelines, here's how I'd start:
Firstly, some things to completely avoid:
Don't use the var keyword.
Don't use the while keyword.
Don't use anything in the scala.collection.mutable package.
Don't use the asInstanceOf method.
Don't use null. If you ever come across null (in someone else's code), immediately wrap it in a more appropriate datatype (usually Option will do nicely).
Then, a couple of things to generally avoid:
Be wary of calling anything with a return type of Unit. A function with a return type of Unit is either doing nothing, or acting only by side-effects. In some cases you won't be able to avoid this (IO being the obvious one), but where you see it elsewhere it's probably a sign of impurity.
Be wary of calling into Java libraries - they are typically not designed with functional programming in mind, and will often require you to abandon the functional approach.
Once you've avoided these things, what can you do to move your code to being more functional?
When you're performing direct recursion, look for opportunities to generalise it through the use of higher order combinators. fold is likely your biggest candidate here - most operations on lists can be implemented in terms of a suitable fold.
When you see destructuring operations on a data structure (typically through pattern matching), consider whether instead you can lift the computation into the structure and avoid destructuring it. An obvious example is the following code snippet:
foo match {
case Some(x) => Some(x + 2)
case None => None
}
can be replaced with:
foo map ( _ + 2 )
I dare say that your goal is already misleading:
I'm starting to learn Function Programming, and I really wanna learn
Scala, not Haskell or Lisp.
If you are really interested in learning concepts of function programming, then why not use a language such as Haskell that (more or less) does not allow you to use procedural or object-oriented concepts? In the end, the language is "just" a tool that helps you learning concepts of FP, you could just as well read loads of papers about FP. At least theoretically, I think it is usually easier to learn concepts of computer science with concrete tools at hand.
On the other hand, if you are interested in learning the language Scala, then why not use all features that it offers, regardless of whether they stem from the FP or the OO world?
In order to conclude with a somewhat practical advise: You could search for FP tutorials that use Scala or for blog entries etc. that describe how to realise certain FP-concepts in Scala and try to follow them. This way, it is less likely that you make use of non-FP concepts.
You don't buy a Ferarri to deliver furniture. Scala's fundamental strength is the fact that in your words, it goes both ways:). Whether or not you are programming in a functional style is decided by the techniques you use.
The best thing you can do is thoroughly review fundamental concepts of functional programming and seek the appropriate Scala implementation of the respective concepts. But if you want to program purely functional style, then go for Haskell, Lisp, Erlang, OCaml, or whatever other purely functional dialect.
Functional programming
Introduction
Functional thinking
Scala
If you want to learn Scala, then make sure to include both OO and FP in your learning curve. Lambda expressions + OO concepts + syntactic sugar made possible by IMHO the most advanced compiler on the face of the planet lead to something quite amazing. Take advantage of it!
I think learning is non linear process, it helps to see lots of ways of doing the same thing, also be opportunistic and use any learning resources that are available for you. For example Martin Odersky the creator of Scala offers a free course called "Functional Programming Principles in Scala" https://class.coursera.org/progfun-002/class/index there are some very high quality video lectures, and some really good assignments where the automated grader will tell you that your code is not functional enough and you loose style points because you are using var instead of val
I think the thing you want to focus on is learning the Functional Programming Paradigm and for me learning a paradigm is about learning what types of problems are easy to solve in one paradigm and are hard to solve in another paradigm. Focus on the paradigm and I think you will find that learning both about Haskell and Scala will teach you the functional paradigm faster, because you will be able to ask the question what are the common features between Scala and Haskell, what are the differences .... etc
I know, for example, that I should only use vals and not vars.
That's already a good start, other non-so-functional things to avoid are mutable collections and loops.
Have a look at immutable collections and recursion instead.
Of course, once you are familiar with the functional concepts, there might also be good reasons to use scala's non-functional features.

Tooling for expressive, feature rich numeric computations on the JVM

I am looking for numeric computation tooling on the JVM. My major requirements are expressiveness/readability, ease of use, evaluation and features in terms of mathematical functions. I guess I am after something like the Matlab kernel (probably including some basic libraries and w/o graphics) on the JVM. I'd like to be able to "throw" computional code at a running JVM and want this code to be evaluated. I don't want to worry about types. Arbitrary precision and performance is not so important.
I guess there are some nice libraries out there but I think an appropriate language on top is needed to get the expressiveness.
Which tooling would you guys suggest to address expressive, feature rich numeric computation on the JVM ?
From the jGroovyLab page:
The GroovyLab environment aims to provide a Matlab/Scilab like scientific computing platform that is supported by a scripting engine implemented in Groovy language. The GroovyLab user can work either with a Matlab-lke command console, or with a flexible editor based on the jsyntaxpane (http://code.google.com/p/jsyntaxpane/) component, that offers more convenient code development. Also, GroovyLab supports Computer Algebra based on the symja (http://code.google.com/p/symja/) project.
And there is also GroovyLab:
GroovyLab is a collection of Groovy classes to provide matlab-like syntax and basic features (linear algebra, 2D/3D plots). It is based on jmathplot and jmatharray libs:
Groovy has a smooth learning curve for Java programmers and a flexible syntax similar to Ruby. It is also pretty easy to write a DSL on it.
Though Groovy's performance is pretty good for a dynamic language, you can use static compilation if you are in the need for it.
Most of Mathworks Matlab is built on the Intel Math Kernel Library (MKL), which is (IMHO) the unbeatable champion in linear algebra computations. There is java support, but it costs 500 dollar (the MKL, not just the java support)...
Best second option if you want to use java is jblas, which uses BLAS and LAPACK, the industry standards for linear algebra.
Pure java libraries' performances are horrible apparently, see here...
Spire sounds like it's aiming at the area you're looking at. It takes advantage of a lot of recent scala features such as macros to get decent performance without having to sacrifice the expressiveness of being in a high level language.
There's also breeze, which is targeted at machine learning but includes a fair amount of linear algebra stuff.
Depending how much work you want to get into and what languages you're already familiar with, Incanter in the Clojure world might be worth a look. Also quickly evolving in Clojure right now is core.matrix, which aims to encapsulate high-level common abstractions in linear algebra implemented with various methods or packages.
You highlighted expressiveness in your post, and the nice thing about Clojure is that, as a Lisp, it is possible to make or extend DSLs to closely match problem domains. This is one of the big draws of the language (and of Lisps in general).
I'm the original author of core.matrix for Clojure. So I have a clear affiniy and much more knowledge in this specific space. That said, I'm still going to try and give you an honest answer :-)
I was the the same position as you a year or so back, looking for a solution for numeric computation that would be scalable, flexible and suitable for deployment as a clustered cloud service.
I ended up going with Clojure for the following reasons:
Functional Programming: Clojure is a functional programming language at heart, more so than most other language (although not as much as Haskell....). Lazy infinite sequences, persistent data structures, immutability throughout etc. Makes for elegany code when you are dealing with big computations.
Metaprogramming: I saw a need to do code generation for vector / computational experessions. Hence being a Lisp was a big plus: once you have done code generation in a homoiconic language with a "whole language" macro system then it's hard to find anything else that comes close.
Concurrency - Clojure has an impressive and movel approach to multi-code concurrency. If you haven't seen it then watch: http://www.infoq.com/presentations/Value-Identity-State-Rich-Hickey
Interactive REPL: Something I've always felt is very important for data work. You want to be able to work with your code / data "live" to get a real feel for its properties. Having a dynamically typed language with an interactive REPL works wonders here.
JVM based: big advantage for pragmantic purposes, because of the huge library / tool ecosystem and the excellent engineering in the JVM as a runtime platform.
Community: I saw a lot of innovation going on in Clojure, particularly around the general area of data and analytics.
The main thing Clojure was lacking at that time was a good library / API for matrix operations. There were some nice tools in Incanter, but they weren't very general purpose or performant. Hence I started developing core.matrix, which is shaping up to be an idiomatic Clojure-flavoured equivalent of NumPY / SciPY. Right now it is still work in progress but good enough for production use if you are careful.
In terms of low-level matrix support, I also maintain vectorz-clj, which is my attempt to provide a core.mattrix implementation that offers high performance vector/matrix operations while remaining Pure Java (i.e. no native dependencies). If you are interested in the performance of this, you may like to see:
http://clojurefun.wordpress.com/2013/03/07/achieving-awesome-numerical-performance-in-clojure/
My second choice after Clojure would have been Scala. I liked Scala's slightly greater maturity and decent static type system. Both the languages are JVM based so the library / tool side was a tie. It was probably the Lisp features that clinched it.
If you happen to have access to Mathematica, then it's fairly easy to get it working with the JVM by means of J/Link. For Clojure, Clojuratica is an excellent library to make that as seemless as possible, although it's not been maintained for a while and it may take some effort to get it working in modern environments again.

Language requirements for AI development [duplicate]

This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
Why is Lisp used for AI?
What makes a language suitable for Artificial Intelligence development?
I've heard that LISP and Prolog are widely used in this field. What features make them suitable for AI?
Overall I would say the main thing I see about languages "preferred" for AI is that they have high order programming along with many tools for abstraction.
It is high order programming (aka functions as first class objects) that tends to be a defining characteristic of most AI languages http://en.wikipedia.org/wiki/Higher-order_programming that I can see. That article is a stub and it leaves out Prolog http://en.wikipedia.org/wiki/Prolog which allows high order "predicates".
But basically high order programming is the idea that you can pass a function around like a variable. Surprisingly a lot of the scripting languages have functions as first class objects as well. LISP/Prolog are a given as AI languages. But some of the others might be surprising. I have seen several AI books for Python. One of them is http://www.nltk.org/book. Also I have seen some for Ruby and Perl. If you study more about LISP you will recognize a lot of its features are similar to modern scripting languages. However LISP came out in 1958...so it really was ahead of its time.
There are AI libraries for Java. And in Java you can sort of hack functions as first class objects using methods on classes, it is harder/less convenient than LISP but possible. In C and C++ you have function pointers, although again they are much more of a bother than LISP.
Once you have functions as first class objects, you can program much more generically than is otherwise possible. Without functions as first class objects, you might have to construct sum(array), product(array) to perform the different operations. But with functions as first class objects you could compute accumulate(array, +) and accumulate(array, *). You could even do accumulate(array, getDataElement, operation). Since AI is so ill defined that type of flexibility is a great help. Now you can build much more generic code that is much easier to extend in ways that were not originally even conceived.
And Lambda (now finding its way all over the place) becomes a way to save typing so that you don't have to define every function. In the previous example, instead of having to make getDataElement(arrayelement) { return arrayelement.GPA } somewhere you can just say accumulate(array, lambda element: return element.GPA, +). So you don't have to pollute your namespace with tons of functions to only be called once or twice.
If you go back in time to 1958, basically your choices were LISP, Fortran, or Assembly. Compared to Fortran LISP was much more flexible (unfortunately also less efficient) and offered much better means of abstraction. In addition to functions as first class objects, it also had dynamic typing, garbage collection, etc. (stuff any scripting language has today). Now there are more choices to use as a language, although LISP benefited from being first and becoming the language that everyone happened to use for AI. Now look at Ruby/Python/Perl/JavaScript/Java/C#/and even the latest proposed standard for C you start to see features from LISP sneaking in (map/reduce, lambdas, garbage collection, etc.). LISP was way ahead of its time in the 1950's.
Even now LISP still maintains a few aces in the hole over most of the competition. The macro systems in LISP are really advanced. In C you can go and extend the language with library calls or simple macros (basically a text substitution). In LISP you can define new language elements (think your own if statement, now think your own custom language for defining GUIs). Overall LISP languages still offer ways of abstraction that the mainstream languages still haven't caught up with. Sure you can define your own custom compiler for C and add all the language constructs you want, but no one does that really. In LISP the programmer can do that easily via Macros. Also LISP is compiled and per the programming language shootout, it is more efficient than Perl, Python, and Ruby in general.
Prolog basically is a logic language made for representing facts and rules. What are expert systems but collections of rules and facts. Since it is very convenient to represent a bunch of rules in Prolog, there is an obvious synergy there with expert systems.
Now I think using LISP/Prolog for every AI problem is not a given. In fact just look at the multitude of Machine Learning/Data Mining libraries available for Java. However when you are prototyping a new system or are experimenting because you don't know what you are doing, it is way easier to do it with a scripting language than a statically typed one. LISP was the earliest languages to have all these features we take for granted. Basically there was no competition at all at first.
Also in general academia seems to like functional languages a lot. So it doesn't hurt that LISP is functional. Although now you have ML, Haskell, OCaml, etc. on that front as well (some of these languages support multiple paradigms...).
The main calling card of both Lisp and Prolog in this particular field is that they support metaprogramming concepts like lambdas. The reason that is important is that it helps when you want to roll your own programming language within a programming language, like you will commonly want to do for writing expert system rules.
To do this well in a lower-level imperative language like C, it is generally best to just create a separate compiler or language library for your new (expert system rule) language, so you can write your rules in the new language and your actions in C. This is the principle behind things like CLIPS.
The two main things you want are the ability to do experimental programming and the ability to do unconventional programming.
When you're doing AI, you by definition don't really know what you're doing. (If you did, it wouldn't be AI, would it?) This means you want a language where you can quickly try things and change them. I haven't found any language I like better than Common Lisp for that, personally.
Similarly, you're doing something not quite conventional. Prolog is already an unconventional language, and Lisp has macros that can transform the language tremendously.
What do you mean by "AI"? The field is so broad as to make this question unanswerable. What applications are you looking at?
LISP was used because it was better than FORTRAN. Prolog was used, too, but no one remembers that. This was when people believed that symbol-based approaches were the way to go, before it was understood how hard the sensing and expression layers are.
But modern "AI" (machine vision, planners, hell, Google's uncanny ability to know what you 'meant') is done in more efficient programming languages that are more sustainable for a large team to develop in. This usually means C++ these days--but it's not like anyone thinks of C++ as a good language for AI.
Hell, you can do a lot of what was called "AI" in the 70s in MATLAB. No one's ever called MATLAB "a good language for AI" before, have they?
Functional programming languages are easier to parallelise due to their stateless nature. There seems to already be a subject about it with some good answers here: Advantages of stateless programming?
As said, its also generally simpler to build programs that generate programs in LISP due to the simplicity of the language, but this is only relevant to certain areas of AI such as evolutionary computation.
Edit:
Ok, I'll try and explain a bit about why parallelism is important to AI using Symbolic AI as an example, as its probably the area of AI that I understand best. Basically its what everyone was using back in the day when LISP was invented, and the Physical Symbol Hypothesis on which it is based is more or less the same way you would go about calculating and modelling stuff in LISP code. This link explains a bit about it:
http://www.cs.st-andrews.ac.uk/~mkw/IC_Group/What_is_Symbolic_AI_full.html
So basically the idea is that you create a model of your environment, then searching through it to find a solution. One of the simplest to algorithms to implement is a breadth first search, which is an exhaustive search of all possible states. While producing an optimal result, it is usually prohibitively time consuming. One way to optimise this is by using a heuristic (A* being an example), another is to divide the work between CPUs.
Due to statelessness, in theory, any node you expand in your search could be ran in a separate thread without the complexity or overhead involved in locking shared data. In general, assuming the hardware can support it, then the more highly you can parallelise a task the faster you will get your result. An example of this could be the folding#home project, which distributes work over many GPUs to find optimal protein folding configurations (that may not have anything to do with LISP, but is relevant to parallelism).
As far as I know from LISP is that is a Functional Programming Language, and with it you are able to make "programs that make programs. I don't know if my answer suits your needs, see above links for more information.
Pattern matching constructs with instantiation (or the ability to easily construct pattern matching code) are a big plus. Pattern matching is not totally necessary to do A.I., but it can sure simplify the code for many A.I. tasks. I'm finding this also makes F# a convenient language for A.I.
Languages per se (without libraries) are suitable/comfortable for specific areas of research/investigation and/or learning/studying ("how to do the simplest things in the hardest way").
Suitability for commercial development is determined by availability of frameworks, libraries, development tools, communities of developers, adoption by companies. For ex., in internet you shall find support for any, even the most exotic issue/areas (including, of course, AI areas), for ex., in C# because it is mainstream.
BTW, what specifically is context of question? AI is so broad term.
Update:
Oooops, I really did not expect to draw attention and discussion to my answer.
Under ("how to do the simplest things in the hardest way"), I mean that studying and learning, as well as academic R&D objectives/techniques/approaches/methodology do not coincide with objectives of (commercial) development.
In student (or even academic) projects one can write tons of code which would probably require one line of code in commercial RAD (using of component/service/feature of framework or library).
Because..! oooh!
Because, there is no sense to entangle/develop any discussion without first agreeing on common definitions of terms... which are subjective and depend on context... and are not so easy to be formulate in general/abstract context.
And this is inter-disciplinary matter of whole areas of different sciences
The question is broad (philosophical) and evasively formulated... without beginning and end... having no definitive answers without of context and definitions...
Are we going to develop here some spec proposal?