This past week I have been doing research into possible language/frameworks to use for a plugin architecture for my master's project. I feel as though I have learned a great deal, but I am still apprehensive in which direction I am comfortable going in.
Before summarizing my findings, I'd like to first describe the main requirements of my project--because I believe that it's possible that I've had tunnel vision on the implementation and maybe not on what the requirements call for itself.
My project is an IoT management platform, the idea is that the system will be comprised of services or modules, that provide communication, storage, security, events, device registration. Application developers will use these system services to create plugins to be loaded, unloaded as desired at runtime for end users to use with their devices.
I have done some tutorials with OSGI, some of which I had a lot of difficulty completing. I have read about plugin architecture implementations in C/C++ and golang, each seem to have their quirks and limitations.
Is using a plugin architecture for this project warranted? And if so, which framework/tools/language(s) would you use?
For architecture I think you should follow micro-service arch
and yes every language have their limitation then when you create service for high performance then use rust or c++ and when you want to create high performance api then create golang service and deploy them on cloud and kubernate.
For framework, if you still confused what framework is suite you then follow clean architecture that will help you when create service
ex. you create golang service with gin framework you can switch to fiber framework very easy.
ref.
https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html
https://github.com/bxcodec/go-clean-arch
Related
I am using flutter as my main framework for building cross-platform apps.
Until this very day, I used Firebase and it was wonderful but I want to expand my knowledge and learn to build full-stack mobile apps.
I was roaming around the web for several days now, and I can't seem to set my mind on backend language to go with flutter.
I am looking for a language that is up to date and much needed in the industry, so don't worry if it is overkill for small apps (I just want to practice for now). and in addition, I need it to go well with the bloc pattern recommended by Google.
I have heard that scala and play are great backend services. also, that ruby on rails is going well with the bloc pattern.
I would like to hear your thoughts and opinions about the situation, what would you choose and why? (In general, not between scala play and RoR)
Thanks a lot in advance!
IMHO, you should to define target tasks, that you will solve.
For example, our company engaged in computer vision and deep learning problems. We choose flask or fastapi python frameworks to build our backend rest-api prototypes, because it`s easy to integrate with tensorflow or pytorch solutions. If we have bottleneck somewhere, that microservices are rewriting in .Net.
If you looking for the most demanded language or framework, see upwork jobs or similar sites to understand, what does the market need today.
I'm developing a gateway that will be located between mobile web apps and web services on backend systems. The purpose of this gateway is to protect web apps against changes to backend web service api's, to introduce concurrency, transform messages, buffering, etc.
My proposed architecture is as follows:
Platform independent mobile web apps using PhoneGap (done)
Gateway is a web service using Scala for business logic and ZeroMQ for message passing (new)
Backends are existing web services (existing)
The gateway is purely responsible passing, translation, aggregation, etc. of messages and does not need to keep state or do user authentication at this point - it's simply responsible for being a single interface that knows how to talk to mobile apps on the one side and one or more services on the other side.
I'm strongly considering using Scala as the development language because it seems well suited for this type of application, but what would be the correct architecture for such a Scala service? I looked at frameworks such as Lift and Play and also considered doing a simple "java" based web service and just use Scala as to implement my business logic. I believe strongly in keeping things as simple as possible. I'm wary of complicated setups and thousands of lines of dead code in frameworks that may never be used. On the other side, limiting oneself to a 'role your own' solution and creating a lot of work and having to maintain code that may have been part of existing solutions is also not ideal.
Some things to consider: I'm the architect and developer, but my knowledge of Scala is limited to the first half of "Programming in Scala, Second Edition". Also, my time is very limited. Still, I want to get this right the first time.
I'm hoping that some clever gent or dame will provide me with insights to this type of solution and maybe a link or two to get started quick. I really need to get going FAST, but hope that the experience or insights of another professional could help me avoid pitfalls along the way. Any insights to development environments and tools will also be helpful. I have to develop on Mac (company rules) but will be deploying on Ubuntu server. I'm currently juggling between installs of Eclipse or Idea as IDE's and the scala compiler or sbt for building.
UPDATE
Thanks for all the potential answers below. I had a look at each and every suggestion and all of them have merit. The problem is now to bet on the right horse. Spray is possibly the simplest solution to the problem, but I also found Finagle. It seams like a stunning solution to my problem. I'm a bit concerned that it is built on top of Netty instead of Akka. Does anyone see any problem with that. I was hoping to keep my solution as purely Scala as possible, but Finagle appears to be the most mature of the lot. Any ideas?
It might be worth taking a look at Akka, which provides a lightweight framework for writing concurrent, fault-tolerant applications on the JVM, as well as useful built-in abstractions for writing web services. In addition, the Spray project looks like it could be useful for your purposes, providing HTTP server and client libraries on top of Akka (Although, unlike Akka itself, I haven't used this so myself so can't vouch for it).
If you're using a web service that is purely for other services, then an Akka based service like Spray or Blueeyes is probably your best bet. You can use Jerkson to do the JSON mapping to and from your case classes and use Akka-Camel to link to ZeroMQ.
As mistertim says, Akka is a good option. I'm biased, but I think Lift and its RestHelper is a fantastic way to go also (I've built several APIs for iPhone apps using Lift). Beyond those, I know several people who are very happy with Unfiltered.
Unfiltered is very interesting option if you want to keep it lightweight, as is not a full blown web framework.
To be honest, the documentation can use more detail, but to provide a web service you just write:
object MyService extends unfiltered.filter.Plan {
def intent = {
case req # GET(Path ("/myendpoint")& Params(params)) =>
for {
param1<-params("param1")
param2<-params("param2")
} yield ResponseString(yourMethod(param1,param2))
}
override def main(args:Array[String]) = unfiltered.jetty.Http.anylocal.filter(MyService).run();
}
Very useful if you don't want to use a framework and don't want to mix java/java annotations with Scala (otherwise, you have the usual java solutions)
Still, I would recommend to look into Akka if you think you think actors will fit your problem
If your system is middle-ware and get the things done quickly, then Spray will be a best. Spray was developed on top of Akka. Akka has ZeroMQ support.
in future if you are going to add some other web like module along with your middle-ware, then choosing Lift + Akka will be better because, Lift also provides Spray like web service stuff + it will be easy to start developing other modules.
You can choose SBT as your build, there are some project templates, template generation tools available, so you can get the project build very quickly.
In my experience , SBT + InteliJ Idea works well, have a look on sbt-idea plugin
Spray Project Template
Lift Project Template Generator
I am really torn between two different stacks with which to build a large application. One the one hand there is this option:
Node.js
express
coffee script
coffeekup
mongoose/mongodb
or
presistencejs/mysql
Play Framework w/ Scala
Anorm w/ mysql
or mongodb
The node.js path is appealing to me because i can write all of the server side code, views and client side code in coffeescript, which i already know. If i go down this road i am still not 100% sure which db path i would take. mongoose makes storing data quick and easy, but the lack of true relationships might be more difficult to work with given the data model i have in mind (very SQLish).
The Play Framework path is also appealing because i know the framework well when using Java, but i don't know much about Scala, so there would be a hit to productivity as i work through learning that language. The Anorm database access layer is appealing because i can write the SQL by hand which i would prefer, and have the results mapped to objects automatically, which saves a lot of effort.
I keep leaning towards node.js, but i'm not sold on the best db access layer to use. Anyone have any experience with any of this and can share some insight?
The stack you choose should depend upon the needs of your application. Let's look at Play vs. Node for their strengths:
Node
Real-time applications (chat, feeds)
Event-driven architecture
Can perform client-server duties (e.g. serve files), but not well-suited for this
Database management, testing tools, etc, available as additional packages
Play!
Client-server applications (website, services)
Share-nothing architecture
Can perform real-time duties (e.g. Websockets), but not well-suited for this
Database management (including migrations!), testing tools, etc, built into core
If your application more closely matches a traditional web-based model, Play is probably your best choice. If you need immediate feedback and real-time dynamic messaging, Node is the better choice.
For large traditional applications, seriously consider the Play! Framework because of the built-in unit and functional testing along with database migrations. If incorporated into the development process, these go a long way toward an end product that works as expected and is stable and error-free.
There are 10 major categories you should consider when comparing web frameworks:
Learn: getting started, ramp up, overall learning curve.
Develop: routing, templates, i18n, forms, json, xml, data store access, real time web.
Test: unit tests, functional tests, integration tests, test coverage.
Secure: CSRF, XSS, code injection, headers, authentication, security advisories.
Build: compile, run tests, preprocess static content (sass/less/CoffeScript), package.
Deploy: hosting, monitoring, configuration.
Debug: step by step debugger, profilers, logging,
Scale: throughput, latency, concurrency.
Maintain: code reuse, stability, maturity, type safety, IDEs.
Share: open source activity, mailing lists, popularity, plugins, commercial support, jobs.
Check out my talk Node.js vs Play Framework for a detailed breakdown of how these two frameworks compare across these 10 dimensions.
I am about to start building an app that will be used across all platforms. I will using monotouch and monodriod so I can keep things in .net
I'm a little lazy so I want to be able to reuse as much code as possible.
Lets say I want to create an application that stores contact information. e.g. Name & Phone number
My application needs to be able to retrieve data from a web service and also store data locally.
The MVVM pattern looks like the way to go but im not sure my approach below is 100% correct
Is this correct?
A project that contains my models
A project that contains my views,local storage methods and also view models which I bind my views to. In this case there would be 3 different projects based on the 3 os's
A data access layer project that is used for binding to services and local data storage
Any suggestions would be great.
Thanks for your time
Not specifically answering your question, but here are some lazy pointers...
you can definitely reuse a lot of code across all 3 platforms (plus MonoWebOS?!)
reusing the code is pretty easy, but you'll need to maintain separate project files for every library on each platform (this can be a chore)
MVVM certainly works for WP7. It's not quite as well catered for in MonoTouch and MonoDroid
some of the main areas you'll need to code separately for each device are:
UI abstractions - each platform has their own idea of "tabs", "lists", "toasts", etc
network operations - the System.Net capabilities are slightly different on each
file IO
multitasking capabilities
device interaction (e.g. location, making calls etc)
interface abstraction and IoC (Ninject?) could help with all of these
The same unit tests should be able to run all 3 platforms?
Update - I can't believe I just stumbled across my own answer... :) In addition to this answer, you might want to look at MonoCross and MvvmCross - and no doubt plenty of other hybrid platforms on the way:
https://github.com/slodge/MvvmCross
http://monocross.net (MVC Rather then Mvvm)
Jonas Follesoe's cross platform development talk: Has to be the most comprehensive resource out there at the moment. He talks about how best to share code and resources, abstract out much of the UI and UX differences, shows viable reusable usage of MVVM across platforms and nice techniques for putting together an almost automated build. (yes, that includes a way for you to compile you monotouch stuff on Visual Studio)
Best of all he has a available source code for the finished product and for a number of the major component individually placed in its own workshop project and a 50 + page pdf detailing the steps to do so.FlightsNorway on github
IMO the only thing missing is how best to handle local data storage across all platforms. In which case I would direct you to Vici Cool Storage an ORM that can work with WP7, MonoTouch and (while not officially supported) MonoDroid.
*Disclaimer* The site documentation isn't the most updated but the source code is available. (Because documentation is Kriptonite to many a programmer)
I think the easiest way to write the code once and have it work on all three platforms will probably be a web-based application. Check out Untappd for example.
You can start by looking at Robert Kozak's MonoTouch MVVM framework. It's just a start though.
MonoTouch MVVM
Scaffolding, what is it? Is it a Rails-only thing?
Scaffolding generally refers to a quickly set up skeleton for an app. It's not rails-only since other platforms have it as well. It's also not generally meant to be a "final" system; merely the first, smallest way to do it.
From Wikipedia:
Scaffolding is a meta-programming
method of building database-backed
software applications. It is a
technique supported by some
model-view-controller frameworks, in
which the programmer may write a
specification that describes how the
application database may be used. The
compiler uses this specification to
generate code that the application can
use to create, read, update and delete
database entries, effectively treating
the template as a "scaffold" on which
to build a more powerful application.
Just like a real scaffolding in a building construction site, scaffolding gives you some kind of a (fast, simplified, temporary) structure for your project, on which you can rely to build the real project.
It can be (and is today) used to describe many things - from abstracting DB layers, to web apps folder structures, and to generating and managing project dependencies .
It is not something that is specific to any language / technology, just like the term skeleton or boilerplate is platform agnostic.
It is just a term borrowed from real scaffolding (like mentioned above).
You build some fast, simplified, (sometimes external, sometimes temporary) structure that will help you to build the real, more complex, finalized structure under, above, inside or outside of that temporary structure .
.. And just like the real scaffolding, the scaffolding structure is meant to support the building process of the project, rather than the project itself (with some exceptions).
Scafolding is usually some type of code generation where you point it at a database, and the technology creates basic CRUD (create, read, update, delete) screens.
I believe that Wikipedia and some answers here provides a narrow and restricted view. Scaffolding is not just for CRUD operations on top of a database. Scaffolding has a broader objective to give you a skeleton app for any kind of technology.
Yeoman is a modern and useful tool for scaffolding. Using their own words:
The web's scaffolding tool for modern webapps
What's Yeoman?
Yeoman helps you to kickstart new projects, prescribing best practices
and tools to help you stay productive.
To do so, we provide a generator ecosystem. A generator is basically a
plugin that can be run with the yo command to scaffold complete
projects or useful parts.
Through our official Generators, we promote the "Yeoman workflow".
This workflow is a robust and opinionated client-side stack,
comprising tools and frameworks that can help developers quickly build
beautiful web applications. We take care of providing everything
needed to get started without any of the normal headaches associated
with a manual setup.
With a modular architecture that can scale out of the box, we leverage
the success and lessons learned from several open-source communities
to ensure the stack developers use is as intelligent as possible.
As firm believers in good documentation and well thought out build
processes, Yeoman includes support for linting, testing, minification
and much more, so developers can focus on solutions rather than
worrying about the little things.
That's it. Use scaffolding to create a quick-start application to work as an example or the foundation of your solution. It makes you productive faster them building things from scratch.
It is not a rails only term although I think it originated there (at least that is where I first heard it.)
Scaffolding is a framework that allows you to do basic CRUD operations against your database with little or no code. Generally, you then go through and add the code to manage the data the way you want replacing the scaffolding. It is generally only intended to get you up and running quickly.
No it is used in other technologies also such as ASP.NET MVC
it creates a basic layout from some predefined code which programmers uses in almost every project , Eg: for database data access it can make a crud method for create, read, update, delete operations
OR you might use it to create layout for your View/Html Code
Scaffolding is writing any piece of code that would not be part of the business logic but would help in unit testing and integration testing.
This is a software engineering term and not bound to any framework or programming language.
No, scaffolding is not the term for the specific platform, however many know this term in the context of Ruby on Rails or .NET
There are also plenty of tools that perform javascript scaffolding:
divjoy.com
flatlogic.com
scaffoldhub.com
yeoman.io
Those tools are also known as code-generators
Scaffolding is the term used when you don't want to create all parts of the structure such as models, views, etc. and want to generate them all in one go. A lot of frameworks use this technique, I studied about it while doing odoo but most of the references given were to ruby on rails :)