To tie together various legacy applications, some mainframe-based, I'm trying to compare using an ESB like MQSeries, a WS-* approach, or something more RESTful.
Is there much substance to the idea that writing the interface to REST, instead of MQ or even WS-*, may have a secondary benefit of also taking us closer to web-enabling portions of the apps (for use by humans with browsers)?
Applications have been web-enabled for years before REST bacame a fad, and will do for years to come. I don't see much of a relationship between the two.
BTW, I didn't think of MQSeries as having to do with ESB - it's just a message queueing system, largely equivalent to MSMQ.
Related
I want to dive into the whole diversity of tools which provide connection between programs over the network.
To clarify the question, I divide it on subquestions:
Why some groups of programs (or specific tools/frameworks/approaches with programming languages where this frameworks can be used) were popular in each period of time? (I expect description of problems which were solved, description of tools, why those tools are considered as best solution to those problems at that time, why some tools lost popularity)
What is the entire history of software communication over the network? (tools/approaches popularity precisely to decades)
What are the modern solutions to this problem?
I can distinguish only two significant approaches.
RPC, RMI and their implementations (I saw this, but it is about concrete problem and specific tools to solve this problem, I want to see the place of this problem in the whole picture of interconnection programs over the network. I heard about implementations: ONC RPC, XML-RPC, CORBA, DCOM, gRPC, but which are active now? which are reasonable to use? which are preferable and why? I want answers not to be opinion based, so I accept answers like "technology A better than technology B for problem X because ..." only if there is reliable research/statistics or facts). I heard that RPC and RMI were popular 10 years ago. Are they still?
Web services: REST, SOAP.
Am I miss something? Maybe there are some technologies which solve problem completely new way? Maybe there are technologies which can be treated as replacement to RPC(RMI) and Web Services? Can we replace RPC(RMI) by REST for any task? Can we replace RPC(RMI) by REST only for modern tasks? Should I separate technologies not as RPC and Web Services, but in some other manner?
As a partial answer, I can give you my feedback on the use of RabbitMQ.
As explain here, it provides a lot of different ways to use it :
RPC by implementing a "callback" queue
One to one, one to many routing strategy to propagate your events through your whole infrastructure and target the right destination.
It comes with the ability to persist messages to avoid loosing data when a crash appears but also with some plugins to increase possibilities (e.g x-delayed plugin)
This technologie written in Erlang is powerful and is a must try in term of communication between programs.
To your question „Am I missing something“: yes.
Very popular communication patterns are the so-called Event-Driven or Message-Driven protocols. This type of protocols are often used in distributed systems such web applications, microservices and IoT-Environments. The communication is complete asynchronously and allows building scalable and loosely coupled systems.
There are many different frameworks and methods for Event-Driven systems like WebSockets, WebHooks, Pub-Sub and Messaging-Librarys like AcitveMQ, OpenMQ, RabbitMQ, ZeroMQ and MQTT.
Hope this info helps for your research.
I have been reading on difference between REST and SOAP. I see in many posts that SOAP is a better choice for distributed transactional resources.
Please give me a practical example of SOAP being used for distributed transaction.
SOAP has been the main player for many years inside enterprise applications simply because there was no alternative. REST came later.
Since SOAP is a protocol it is easier to build tools around it since you know how it behaves always (i.e. as the protocol is defined). For this reason and because it's mature as technology, a lot of other specifications were build around it, to cover any uses one might have for doing something with SOAP. See a list here. There are of course some for transactional semantics also. If you use
SOAP with a technology like Java or C# (which are heavyweight champions in the enterprise applications field) then you can have these transactional specifications already implemented in the framework or libraries and you just use them.
REST on the other hand is an architectural style of building applications. It's harder to limit it to a set of specifications. You can implement it in many ways. It is also going somehow against "the way of the SOAP" by staying away of creating new standards or specifications and instead just reusing the ones of the web. For this reason, there are no specs or tools to help you with transactional RESTful services. You have to build your own.
So when your application is build by self-contained web services, and these services need to cooperate on creating the applications outcome, and you need a distributed transaction to guarantee that outcome is consistent (all operations succeeding or none succeeding) then it's (more) practical to go for the technology that has the better tooling in supporting it.
I'm in the process of deciding how to write an online auction application. I would like to provide real-time updates to the site users. My background is with LAMP (although, in my case, the 'P' would be more for Perl than PHP). I've considered ActiveMQ, but I'm wondering if there are better options.
My primary concerns are scalability and speed. It could have several simultaneous auctions taking place, with [hopefully] many users participating in each auction. Whatever solution that I decide on would have to accommodate such a scenario. Of course, this is all in theory so I have no idea how many concurrent users that I might have, but I'd like to have the means to support tens of thousands of users.
Another concern is ease of implementation. I've spent the past few days reading docs and tutorials and, so far, nothing has come across as anything less than a bit of a pain in the rear to deal with, which is actually what has led me here to seek some advice.
I was hoping to use a web framework, such as Codeigniter (PHP) or Catalyst (Perl), because I intend to pay a contractor or two to help with some of the bulk of the coding, and I like the idea of having a framework to somewhat enforce a design pattern. However, the more that I look into this, I'm just not seeing an obvious solution to 1) use a framework, and 2) provide real-time auction updates (other than Tornado, I guess - maybe I'm answering my own question. ;)).
So, with all that said, short of using polling (which I'm not really interested in doing), is there a way that I can accomplish these real-time updates using a language like Perl or PHP for my server-side code? I know that ActiveMQ supports STOMP, and I actually have this working on my local machine (using Jetty since it requires a servlet to publish/consume messages from client-side javascript), but is there a better option here?
I'm sorry that I don't have a more direct question, but after several days of looking at docs and tutorials, I'm more lost than ever!
Part of your problem is that your mixing a variety of concepts together. If I read things correctly you have a problem statement of:
I'm building an online auction site and would like to insure that my visitors have real-time updates of prices on the items they are viewing.
Now between the Browser and the Server you'll probably use a Comet style request pattern to handle communications, you could also look at socket.io as a backup pattern. This polling will require a server that is able to handle lots of simultaneous open connections, which Tornado is a good candidate (there are others, but given you asked in relationship to Tornado it's good).
Now that we've gone from 1000+ of Browsers to a handful of Tornado servers, you need a way to communicate between them. In the the last of publish/subscribe message patterns you have a few choices:
RabbitMQ (AMQP)
ZeroMQ
Redis Pub/Sub
All three a good choices, with their own pros/cons. Personally I've used Redis and Rabbit on different projects and just toyed with ZeroMQ. The message broker is a whole decision tree that is going to be based on what you have available.
Is there anyone who has used or looked into using Jitterbit as well as BizTalk? If so, what are some pros and cons of each, and which one did you go with as your final solution?
Specifically, I'm looking for SAP integration, but any input would be appreciated.
Like Rob I have not heard about JitterBit until reading your question (thanks!), I have, however, been working with BizTalk, almost exclusively, for the past 9 years; for that reason I wasn't sure I should be responding, but as Rob did, and nobody else has, I figured it's worth a couple of cents....
From the little reading I've done it seems to me that JitterBit, apart from being an open source, which has it's pros and cons, is trying to lower the entry barrier by offering a relative simple solution with the promise of rapid development and drag-n-drop approach "with no custom code".
I'll take their promise at face value, as I know nothing about it, although I have my doubts, so let's assume developing with JitterBit is really easy, there's one thing I can clearly state - developing with BizTalk isn't.
But, and that's a bit but in my view, developing with BizTalk is somewhat difficult not because Microsoft did a bad job at it, on the contrary - developing with BizTalk is somewhat difficult because Microsoft wanted to create a tool that could realistically allow enterprises to solve their BPM and integration needs well, and, in my experience, these problems are almost never simple, so Microsoft had built a server that has many capabilities, is very strong and very flexible, at the cost of complexity.
So, while any experienced technical sales guy can give you a demo of an integration scenario that is very simple, and is developed in a few minutes using a lot of drag and drop and configuration, even in BizTalk, but is this a realistic enterprise-level solution? was it a realistic scenario that was demonstrated? from my experience the answer is almost exclusively no; the problems tend to be complex, and their require a more robust solution.
So, I guess the bottom line would be - if you're looking for a one off solution, and open source is something you guys work with - JitterBit is definitely worth looking at, seeing if it's capable of helping out and has, indeed, a short learning curve (it would be important to look at maintenance, monitoring, trouble shooting, instance management etc)
If, however, you believe, as is often the case, that your solution would grow to become a BPM/integration platform in your organisation, and you need something more robust - I would put my money on BizTalk being a better candidate.
I've done a fair bit of integration with SAP, starting with the old SAP DCOM connector. More recently I've been involved in the selection of an integration platform to serve in an Enterprise Service Bus pattern.
We did web service samples to connect to SAP on a number of platforms, including BizTalk, Mule, Netweaver, Webmethods and Tibco. Webmethods won out based on licensing and capability, though BizTalk and Netweaver both had very high marks.
Jitterbit was not part of the evaluation - in fact I had to look it up to be sure I understood your question.
If your goal is just to be able to call an RFC, the .NET SAP connector works well.
If your goal is to expose a web service to wrapper a process in SAP, then BizTalk is good, but I recommend you see if your organization already has netweaver licensed as there are many web services available directly from SAP with no coding.
My recommendation is to avoid Jitterbug and Mule for the enterprise for now - unless Open Source is actually a popular thing at your place of employment. Netweaver and BizTalk are very robust, polished products.
If you are looking for something you can ship easily, then Jitterbug may make more sense. Though generally I'd recommend you define it as a web service call, and look to your customers technology stack for the most appropriate integration technique.
More context of what you are looking to achieve will enable a more accurate answer.
Michael,
We use Jitterbit in our organization and we've been very successful with it in various projects. Our SAP projects use XI and Jitterbit has dramatically simplified the ability to integrate web service interfaces with the various protocols it supports.
In addition to an excellent price (and we now subscribe to Jitterbit for support) we realize great value out of the support service. If we have any questions during our implementations they seem to provide all the subject matter expertise included in the support cost, so we're quite self sufficient.
We still have many other integration solutions in our company including VB and Java programs; it's a mess, but we don't believe that any one platform will meet all of our different divisions' needs. We have been using open source, specifically Linux and Apache for many years now, although IBM and Microsoft are also prevalent here.
We went with Jitterbit as it supports protocols needed to integrate any modern system and with SOA / Web Services being our stated direction Jitterbit was a great fit for what we needed.
Given that Jitterbit is Open Source, I would encourage you to download it and try it out.
I will say it simply, I have been using biztalk and was one of the people that helped validate the 2006 training course. Biztalk by far one the best server applications for Business process that is available today. You do also have to factor in the price point is ridiculously low compared to what else is out there.
Are people still writing SOAP services or is it a technology that has passed its architectural shelf life? Are people returning to binary formats?
The alternative to SOAP is not binary formats.
I think you're seeing a surge in the desire to leave the complexities of WS-* behind in favor of REST and JSON, because they're much simpler to use and don't require frameworks to be used successfully. The problems that WS-* ostensibly tries to solve aren't problems for most users, but they have to pay for the complexity any way.
I still write WS-*–based services. Somewhat surprisingly, I've had less trouble with them when trying to inter-operate with less capable developers. This is because if I send them a WSDL file, they know how to crank it through their tool and get an API they can call, while being blissfully unaware what is happening under the hood. To give customers a REST-ful service, I have to start talking to them about HTTP and XML, which they really don't understand as well as they think they do, and then I start getting a headache.
In other words, to be successful with REST, both the service provider and consumer have to know what they're doing (and they can keep things simple and come up with a great, non–WS-* solution). With WS-* technologies, it can still succeed even if only one party has a clue.
I think, however, that REST-oriented standards that are much less complicated than current WS standards, will eventually emerge, and when that happens, comparable tools will be available too.
I think so. RESTful solutions are more and more sensible for the vast majority of use cases; the complexities of SOAP and other RPC technologies just aren't worth the effort anymore.
I wouldn't consider SOAP legacy at all. REST vs. SOAP is really just the continuation of the debate of COM/CORBA vs. HTTP POST/GET etc. SOAP is nothing more than an updated version of the same principles defined with C and C (contracts, providers, consumers etc.). It's just that has appeared to SOAP succeed (at least partially) where the other two failed (and it could be that SOAP just has a better marketing team), that is that SOAP really does allow to different systems to connect rather easily compared to it's predecessors. That being said, it still suffers from the same drawbacks that COM/CORBA did...it can get really complex.
I think REST is just coming back into style at the moment. It's nothing new, people are just taking another look at it. Look at the web. It's REST and it's been around for years. 5 years from now people are going to look back and say the same thing about it being legacy and the need to change. It's the nature of software development. Everything goes in cycles.
The debate about which one is better is going to be just like the tabs vs. spaces debate. There are going to be people on different sides swearing that one is better. Really in the end, they both accomplish the same goal. Sure one will be a better solution than the other in some situations, but in the end neither will be superior 100% of the time.
We were using SOAP, but since we control both messaging endpoints (thick client out on the web connecting to our servers) we decided that the "lingua franca" of XML wasn't offering any real benefit. Instead, we're experimenting with binary serialization via Google protocol buffers, and like everything we've learned so far. It's somewhat CORBA-esque, but doesn't make me grumpy the way CORBA did. Still haven't found the best fit for the RPC layer, but pretty sure the payload will be protocol buffers.
The point I'm trying to make is that if you control both sides of the conversation, there are significant efficiency advantages in bypassing the XML tax.
Yes, some people still are (and now it's 2011!). I think the main reason is that MS WCF automatically generates SOAP bindings. The horror.
It's impossible to define what the best technology solution is without considering what the problem is, in other words, what the context is. Both REST and SOAP have their place. If you have a high traffic site and a development audience who is comfortable with REST, then SOAP would be a bad choice, primarily because the message size is so incredibly bloated. If you have small scale site with a modest development budget, then SOAP will be a superior choice due to automatic proxy generation from WSDL. To make a fair comparison, it should be mentioned that implementing a REST conversation takes more development time and therefore is more expensive, a very relevant fact for your boss.
While it is true that SOAP is a more complicated protocol, in my experience this doesn't translate to maintainability issues. That's because messages ride on HTTP and can be easily debugged just like REST message, and the SOAP stacks available on major platforms are very solid.
The complexity of SOAP is of course an advantage if your requirements include sophisticated items like federated message security. On the other hand, these kind of requirements are not seen that often in my experience. The WS standards committee may have been vulnerable to some YAGNI issues. Now that web service communication is commonplace, it's turning out to be simpler that was originally envisioned.