I have two questions regarding Quarkus:
Can quarkus be used with ANY java library when NOT using native mode? I don't understand the sense of quarkus extensions, for example, the extensions provided in https://github.com/quarkiverse for non native images. Are these extensions only needed when using quarkus in native mode?
I read Quarkus extensions nediness, but it vague describes if I can run any java lib in quarkus (when not using native images) without adaptions.
When comparing Quarkus with Wildfly, then it is obvious that Quarkus starts faster, BUT is quarkus also faster (and more memory friendly) while running the application in the long term (24 hours x 7 days)? I have read that, for example, speed and memory consumption on start is better in quarkus than wildfly, BUT when comparing runtime, then the story can be different because of quarkus non dynamical approach (e.g. quarkus bypasses JIT). Is that true? And if so, why?
Currently, I use Wildfly but I am considering to switch to Quarkus if (and only if), it supports all java libs out of the box in non-native mode (like wildfly) and is also better regarding speed and memory consumption while running the application in the long term (and not only at startup).
It is true that quarkus can be used with any java library when not using native mode. You are not utilizing Quarkus' real strength by not using extensions. Quarkus works on the philosophy of 'compile time boot' with which it is able to achieve fast start-up time. This is where extensions come into play. extensions tell quarkus what are the things that can be pre-processed even when not using native mode(https://quarkus.io/guides/writing-extensions#why-an-extension-framework)
As per the wiki(https://github.com/quarkiverse/quarkiverse/wiki) quarkiverse is for all the extensions that didn't find its place in quarkus-core project.
There is no binary answer. There are lot blogs on the quarkus experiments and all of them have got varying results. You will need to conduct your own experiments to see if quarkus really fits your usecase and performance expectations. but on the home page(https://quarkus.io/) though they only claim improvements in start-up time and memory consumption.
Quarkus does not bypass JIT, It just perform the same thing differently
Related
I have a small Akka application that passes many messages between its actors and each actor does some calculations on the data it receives. What I want is to profile this application in order to see which parts of the code take up most time and so on.
I tried VisualVM but I cannot really understand what's going on. I added a picture of the profiler output.
My questions are
What for example is this first line and why does it take up so much time? (scala.concurrent.forkjoin.ForkJoinPool.scan())
Can Akka applications because of their asynchronous behaviour be profiled well at all?
Can I see for instance how long one specific actor(-type) works for one specific message(-type) it receives?
Are there other best-practices for profiling Akka applications?
There are packages not profiled by default and it is their time that is accounted in the profile of scala.concurrent.forkjoin.ForkJoinPool.scan(). If all the hidden packages are allowed to be sampled, the true CPU time consumers will be revealed. For example, the following before/after illustrative profiles uncover that threads are put to sleep most of the time by sun.misc.Unsafe.park waiting to be unparked.
Akka applications can be profiled quite well with proper instrumentation and call tracing. Google's prominent Dapper, a Large-Scale Distributed Systems Tracing Infrastructure paper contains detailed explanation of the technique. Twitter created Zipkin based on that. It is open sourced and has an extension for distributed tracing of Akka. Follow its wiki for a good explanation of how to set up a system that allows to
trace call hierarchies inside an actor system;
debug request processing pipelines (you can log to traces, annotate them with custom key-value pairs);
see dependencies between derived requests and their contribution to resulting response time;
find and analyse slowest requests in your system.
There is also a new kid on the block, Kamon. It is a reactive-friendly toolkit for monitoring applications that run on top of the JVM, which is specially enthusiastic to applications built with the Typesafe Reactive Platform. That definitely means yes for Akka and the integration comes in the form of the kamon-akka and kamon-akka-remote modules that bring bytecode instrumentation to gather metrics and perform automatic trace context propagation on your behalf. Explore the documentation starting from Akka Integration Overview to understand what it can and how to achieve that.
Just a couple of days ago TypeSafe announced that TypeSafe console now is free. I don't know what can be better for profiling Scala/Akka applications. Of cause you can try JProfiler for JVM languages, I've used it with Java projects, but it's not free and for Java.
I was thinking about profiling/metrics in code since I also use Akka/Scala a lot for building production applications, but I also eager to hear alternative ways to make sure that application is healthy.
Metrics (like Dropwizard)
Very good tool for collecting metrics in the code, with good documentation and embedded support for Graphite, Ganglia, Logback, etc.
It has verbose tools for collecting in-app statistics like gauges, counter histograms, timings - information to figure out what is the current state of your app, how many actors were created, etc, if they are alive, what the current state is of majority of actors, etc.
Agree, it's a bit different from profiling but helps a lot to find roots of the problem, especially if integrated with some char building tool.
Profilers like (VisualVM, XRebel)
Since I'm a big fun of doing monitoring, it still answers a slightly different question - what are current insights of my application right now?
But there is quite another matter may disturb us - what of my code is slow (or sloppy)?
For that reason, we have VisualVM and another answers to this question - how to profile Akka actors with VisualVM.
Also, I'd suggest trying XRebel profiler that just adds a bit more firepower to process of figuring out what code makes app slower. It's also paid but on my project it saved a lot of time dealing with sloppy code.
New Relic
I'd suggest it for some playground projects since you can get some monitoring/profiling solutions for free, but on more serious projects I'd go for things I highlighted above.
So I hope, that my overview was helpful.
We are evaluating few options for developing a telecommunication related application platform (and migrating/consolidating some of the standalone apps into the new platform). One of our main concerns is the ability to handle high volume of requests during peak hours.
We feel the TorqueBox seems to an interesting solution worthy of consideration because:
Speed (Next to pure Java performance)
Faster development time over Java
Maintainability
Support for threads/concurrency even though it's Ruby
Faster/Easier front end development with Rails
...
RedHat supported and runs on JBoss (scalability, future development and ability to call Java if necessary)
Has anyone developed/deployed similar application(s) with JRuby/TorqueBox?
Any serious performance bottlenecks ahead? (or why we shouldn't use JRuby and should stick with Java?)
The answer is YES but be aware of memory leaks (gems, threadsafety issues, etc). You have to be familiar with tools like VisualVM, Eclipse MAT and/or NewRelic.
We're successfully using Torquebox on production for some clients on amazon EC2 handling 60k-80k visits per day (new c3 instances are great for Java).
Deployment is also an issue. We're unable to setup any kind of rolling restart because of memory consumption. So every time we deploy using Capistrano a full JBoss restart is needed (no big issue for us).
Bests,
Antonio
yes any mature Java web-server with JRuby is a valuable option. the details of handling high-loads on peak hours will really depend on what kind of app you'll be running, how much "hardware" can you afford to use but in general it's achievable but be aware there might still be some "gotchas" e.g. Ruby libraries (gems) that do not handle thread-safety well. you simply need to understand how to proceed than - which seems you do since you're want to use 'Celluloid.IO' :)
I wish to use the application known as Processing to create interactive graphs and figures to use in a presentation for the Arizona JSHS competition. I have used the widely popular LiveWeb application to be able to embed java into the power-point. But when I do the same thing with Java applications made with Processing it gives an error.
"Error: Unexpected identifier or string. http://localhost/applet/index.html"
I have tested it with locally hosted Java applets and Processing applets, both work run normally. But only the Java applet (which is very similar to the processing applet) works in LiveWeb.
Does anyone have suggestions to get around this? Do you think I could import processing library's into Eclipse and work from there?
One potential solution would be processing.js. It is a javascript version of processing that may fit your needs.
I'm interested in creating a web application and I've just done some research on the what makes a good web server. I've search through the facebook, twitter and foursquare. They share what software they used to build their infrastructure.
For me, some of the software used are new. I'd like to ask some questions here.
why create a back end server, isn't a web server running PHP is enough? Why use java/scala for backend? Do we really need RPC framework such as thrift/protocol buffer? What is that RPC framework used for? Is it used for communication between frontend and backend servers?
Really appreciate for those who answer my questions, or if there's some books you would suggest me to read.
Thank you.
It sounds as though you'd like to build a scalable backend infrastructure that ultimately will be used to do the following:
Serve content. This is the web server layer.
Perform some type of back end processing for user requests
coming in from the web server layer and communicate with the data store. Call this the application server layer.
Save session state and user data in a distributed, fault tolerant, eventually consistent key value store.
Also, it sounds as though you want to do this using commodity PC hardware.
This is a tall order.
Foursquare uses Scala with the Lift framework, jetty for their web server. Here's more. And more.
Facebook uses many different technologies. I know that for their data store they use HBase (they were using Cassandra)
Yahoo uses HBase to keep track of user statistics.
Twitter started as a Ruby-backend web site. They moved to Scala. Twitter is incrementally moving from mysql (I assume sharded) to Cassandra using their proprietary incremental database conversion tool.
As far as scaling on the application server and web server end, I know that what really counts is having a language that has the ability to spawn new user processes in user space and a manager process that assigns new worker processes the requests coming in. Think of it as running a very efficient company. The more work you've got coming in, the more people you hire. This is the Actor model. Some languages have actors built in,(erlang) others have actors implemented as frameworks(akka) or libraries (Scala native). Apparently, Scala's native actors are buggy so some people got together and implemented the akka framework for Scala and Java. There's a lot of discussion online regarding actors and which language and libraries one should use. Erlang has a lot going for it out of the box, however, Scala runs in the JVM and allows you to reuse a lot of the existing Java web libraries (which could have some issue if they happen to have static objects declared in them) Erlang has actors and the OTP libraries, but apparently does not have the rich libraries that Java has. So, for me it really boils down to Scala (with akka) or Erlang.
For the web server, with Scala, you can use any java app server. Foursquare uses jetty for most things. It's not written in Scala, but since Scala compiles down to bytecode that runs on the JVM it easily interops with any java app server.
People also say that there aren't that many Erlang programmers and that Erlang is harder to learn (functional programming vs imperative programming) Scala is functional and imperative at the same time (meaning you can do either)
Erlang is functional. Now, functional programming has a lot of things going for it as one expert functional programmer can get a lot more done than an expert imperative programmer. Yahoo stores was originally written and maintained in Lisp (functional language) by one man. On the other hand, imperative programming is easier to learn and used widely in a team setting. Imperative languages are good for some things, functional languages for
others. The right tool for the right job.
Back to the web server discussion, with Erlang, you can use yaws or you can run a framework (Chicago Boss)
Here's more on the Scala vs Erlang debate.
Another link.
More here.
And another.
Another opinion.
On the database end, you have a lot of choices. See here.
You can even eschew the database all-together and save your data in mnesia (Erlang's runtime data store)
My answer is not complete as this topic (scaling app servers, databases and web servers) is very complicated and full of debate. Some frameworks even blur the tiers (web server, application server, database) distinction and integrate a lot of the functionality of these layers within the framework itself.
For example, I encounter a lot of problems developing complex webapp using PHP only. PHP has no threads, php is lacking many good things that has scala, or another good modern language with rich syntax. PHP is slow comparing to compiled JVM language. PHP is less secure in my opinion. It is good to get a bunch of data and render as HTML page, but processing for high load is not its plus. RPC as you suggest serves as communication layer.
Can you give examples how you used gSOAP and how well it was integrated in your existing architecture? Have you found development bottlenecks with gSOAP?
We used gSOAP for a bunch of ARM clients to communicate with an AXIS Web Service server. Pros of gSOAP:
very powerful, supports nearly all Web Service constructs
easy to use, its abstraction of WS calls into functions removes all Web Service complexity to the programmer
elegant interfaces in both C and C++
However, we ran into several development bottlenecks:
when using custom datatypes like maps or sets, it takes quite some hacking to get the gSOAP compiler to handle them (marshal/unmarshalling). Especially bad with dynamic data structures.
debugging is hard because of its intrinsic complex network, parsing and memory allocation parts. Do everything possible to stick with static memory allocation.
the mailing list is alive, but developers are not very active on it. Simple questions can get answered quickly, but the toughest problems often go unanswered
forget about optimization. Linking in gSOAP eats about 1MB of memory at runtime (-Os). Runtime performance is fine on our 32MB linux-based ARM board, but there really is little to do about optimization if you need it.
We used gSOAP in a C++-based web server about 4 years back. Overall it worked fine. The only major issue was that the interface was in C and procedural (I understand it is difficult to design a good non-procedural interface). There may be a lot of repeated code while implementing the interface, for which you might have to use macros (we didn't explore the templates option too far then).
We are using gSoap to deploy a web service onto an embedde linux device running an ARM MX processor.
We are using gSOAP to consume a WCF based webservice from an application deployed on a linux device running on ARM processor. The experience is good to a large extent.
We used gSOAP in a web server on ARM ARM9 400MHz device.
gSOAP daemon connected to a database daemon through zeromq library, which is run on the same device.
It supports more than 1000 basic requests wich does not requre connection to database.
Disabling support for multi-referenced SOAP option by the WITH_NOIDREF define helped to decrease serialization time about 4 times faster on big requests with large number of serialization nodes.