I am trying to evaluate some technologies for implementing a communication process between some Ada modules with some C++/OpenGL modules. There is an (Windows XP) Ada application which communicates with a C++ application using COM, but I intend to switch the COM to a new technology. Some suggestions came up, such as direct Sockets, DSA, Polyorb, Corba and DSS/Opensplice.
DSA appears to be just Ada -implemented (not sure)
Polyorb has its last implementation on 2006, according to http://polyorb.ow2.org/
Corba someone argumented that it could be not simple enough to justify its complexity for implementing simple applications
DSS/Opensplice appears to be just C/C++ implemented, so an Ada binding should be done. It also looks to be not very simple to be implemented too.
Personally I like COM, but due to the migration, I'd rather take the sockets option due to its simplicity, and the interface architecture could be implemented very easily.
So, what you think? Could you please comment about these technologies or even suggest other more?
Thanks very much.
A big factor in your choice is the size and complexity of the system you're reengineering. Is it a broadly distributed system with lots of complex messages? Is it a relatively small system with a handful of mundane message exchanges?
For small systems I used to just roll-my-own socket-based comm modules. Now, though, I lean more towards ZeroMQ (brokerless) or STOMP (text-based). And there's some Ada support for these, zeromq-Ada and TOMI_4_Ada (supports both).
While these handle the distribution mechanics, you would still need to handle the serialization of the messages into transportable form.
CORBA/PolyORB and DDS solutions are rather heavyweight, but are complete solutions. If you don't fear IDL and managing brokers, they can do well for large-scale distributed systems. Yeah, there may need to be some Ada bindings built, but if you can get C headers or a C API to bind to, it's typically not too bad if you focus on just binding the functions and data structures you require. Rather than creating a comprehensive binding, liberally employ opaque and void pointers (void_ptr, opaque_structure_def_ptr) for structs and parameters whose internal contents you don't care about.
we intend to switch the COM to a new (suported) technology, since COM is not more supported by Microsoft
Whoever told you COM is no longer supported is totally clueless.
While COM has undergone many name changes (OLE, COM, OLE Automation, DCOM, COM+, ActiveX, WinRT) and extensions over the past decades, it is the single most important technology for MS platforms: past, present and future. The .NET runtime uses COM extensively. Much of the Win32 API is written in COM, and the portions that weren't, will be in Win8, since WinRT components are COM objects.
Also take a look at AMQP (RabbitMQ for server), there seems to be Ada library available for it http://www.gti-ia.upv.es/sma/tools/AdaBinding/index.php.
If you could find binding for Ada, Apache thrift might also be a lightweight option. Maybe you could even write your own binding, it should not be more difficult that rolling something of your own over the sockets.
If you do go sockets route, than I would suggest ZeroMQ as "supersockets".
One more option for your list should be to use Ada's distributed programming support, and write C/C++ wrappers to interface your C++ program into it.
I don't know that its the best option for your needs, but if your Ada compiler supports Annex E, it should be on the list.
Since this post, AdaCore published PolyORB on GitHub with regular updates :)
Related
What I want to do: I want to add communication capabilities to a couple of applications (soon to be jar libraries for Java) in Scala, and I want to do it in the most painless way, with no Tomcat, wars, paths for GET requests, RPC servers, etc.
What I have done: I've been checking a number of libraries, like Jetty, JAX-RS, Jackson, etc. But then I see the examples and they usually involve many different folders for configuration, WSDL files, etc. Most of the examples lack a main method and I don't have a clear picture about how many additional requirements may they have (e.g. Tomcat).
What I am planning to do: I'm considering to simply open a socket on the "server" to listen, then connect with the "client" and transfer some JSON, in both directions. This should be fairly standard so that I can use other programming languages in compatible ways (e.g. Python).
What I am asking: I would like to know whether there is some library that makes this easier. Not necessarily using raw sockets, but setting up some process communication in just a few lines, maybe not as simple as Node.js, but something similar.
Bonus: It would be cool to
be able to use other programming languages (e.g. Python) by using open standards
have authentication
But I don't really need any of those at this point.
I think you need RPC client/server system, I would suggest to take one of these two:
Finagle - super flexible and powerful RPC client/server from Finagle. You can define your service with Thrift, and it will generate stubs for client/server in scala. With Thrift it should be straightforward to add Python support.
Spray - much smaller library, focused on creating REST services. It's not so powerful as Finagle, however much easier. And REST allows you to use any other clients
Remotely - an elegant RPC system for reasonable people. Interesting and very promising project, however maybe difficult to start with because of extensive Scalaz+Shapeless+Macro usage
Honestly if you want something that is cross-language compatible, simple, straightforward, and concise then you do not want to use plain old sockets!
Check out dropwizard. It is amazing and I use it for small and large projects alike! It is usually configured by no more than a single configuration file. It supports authentication too!
Out of the box it gives you really great inter-process communication over JSON (using Jackson) and much much more. There is also pretty decent Scala support for dropwizard.
If you must roll your own then I'd recommend using Jackson for JSON parsing. It's super simple to use and also has great scala support.
If you've got a "controlled" use case where the client and server are on the same LAN and deployed in tandem, I'd (controversially) recommend Java RMI; it's dumb and JVM-specific (and uses a Java-specific protocol), but it's very simple to use.
If you need something more robust and cross-language, I'd recommend Apache Thrift. You write your interfaces in a platform-independent interface definition language, and it's very clear which changes are compatible and which are not; the thrift compiler generates skeleton interfaces for you to use, and then you just write an implementation of that interface and a couple of lines to start the server (as you can see from the example on the homepage). It's also got good support for async implementations if you need the performance. Thrift itself is reasonably standard and cross-platform, with its own binary protocol, or you can use JSON as a transport if you really want to (I'd recommend against that though).
RabbitMQ provides one easy way to do what you want without writing a server and implementing your own persistence, flow control, authentication, etc. You can brew or apt-get install it.
You start up a broker daemon process (i.e. manages message queues)
In the Scala producer, you can use Maven-provided Java API to send JSON strings without any fuss (e.g. no definition languages) to specified queues
Then in your other Scala program, connect to the broker, and listen for messages on the queue, and parse the incoming JSON
Because it is so popular, there are many tutorials online for different patterns you may want to use to distribute the messages, e.g. pub/sub, one-to-one, exactly-once delivery, etc.
This is more of a general quick question.
But in like C#,Python,C,C++.......etc
Are all the basic "Sockets" network programming essentially the same. Like do they all use the Berkley Sockets (i think thats what they are called) or does each language have it's own way of doing sockets.
Thanks
Sockets are platform-dependent, not language dependent. So, linux uses the BSD sockets alone, Windows offers both BSD sockets (used almost exclusively) and a M$ flavour of sockets (called WSA), dunno about others. It all boils to what is found under the hood - more exactly at the kernel level of the OS. The socket implementation found there will offer a first set of API in order to make them accessible to kernel/user space - usually through a shared object / dynamic linked library and thus "natively" supporting the c/c++ languages. All the other languages are relying on language specific bindings to those SO / DLL files offered by the OS natively for C/C++.
HTH
Sockets are basically the same in C, C++, Java, Ruby. They are slightly easier (because the build in classes handle the boiler plate) in higher level languages. If you can write Socket code in C, then you can do it anywhere if you have a reference to translate.
#Kellogs brings up a good point, Windows has their own Socket API which (typically) performs better (in my experiences) on Windows than the Posix implementations offered. The APIs are very similar. I would make the analogy of OpenGL to DirectX. If you really know one, then you can figure out the other with a reference.
Now a days we don't differentiate the language, the class library that can be accessed from the language and the underlying operating system. Here is my explanation
C,C++,C#,Java - Are just the languages has no specific support regarding the network programming.
Java class library, .Net Framework, C++ standard library - among this I think C# & Java provides some classes for network programming. C++ standard library does not provide any network programming classes (only iostreams for file, stdinput & strings are available). But BOOST library for C++ provides classes for network programming. I am not aware of the other libraries.
OS - The operating system proives a base api (mostly in C) that is utilised by the class libraries above. In case of windows it is the winsock api (WSA) and in case of unix it is the BSD socket api. I think windows also supports to some extent the BSD api so that the learning curve is less. But as #EnabrenTane said it is more than that.
Agree with kellogs above. Windows & most major POSIX-compliant OSs support the basic BSD socket API. In addition to the Berkeley socket, each platform provides additional proprietary API for performance & scalability enhancement. These extended APIs are not strictly confined to sockets only - they work on any I/O operations (i.e. disk files, named pipes etc.). For example, (in addition to traditional select() & poll()) Linux has it's own epoll mechanism, BSDs have kqueue, Windows has WSAevent*, WSAaAsync* and I/O completion ports APIs.
These differences are visible mostly in lower level languages like C, C++ or Pascal.
C#, Java, Python, Ruby et al. are somewhat "higher level" compared to C - they provide robust technologies to insulate you from the low-level APIs. Instead of fiddling directly with the bare-bones socket API, you can use the extensive run-time class libraries provided with each platform. For example, to write a TCP/IP server in Python, you could simply use the SocketServer class. Similarly in C#, you can use WebClient to downlad files of the net. In C++, you have the Boost library. Of course, there's nothing stopping you from using raw socket API directly in your app.
I am looking at linking a few applications together (all written in different languages like C#, C++, Python) and I am not sure how to go about it.
What I mean by linking? The system I am working on consists of small programs each responsible for a particular processing task. I need to be able to transfer a data set from one application to another easily (the data set in question is not huge, probably a few megabytes) and I also need some form of way to control the current state of the operation (This is where a client-server model rings a bell)
It seems like sockets or maybe SOAP would be a universal solution but just wanted to get some opinions as to what people think about this subject.
Comments/suggestions will be appreciated, thanks!
I personally take a liking towards ØMQ. It's a library that has a familiar BSD-sockets-like interface for passing messages, but you'll find it implements interesting patterns for distributing tasks.
It sounds like you want to arrange several processes in a pipeline. ØMQ allows you to do that using push and poll sockets. (And afterwards, you'll find it's even possible to scale up across multiple processes and machines with little effort.) Take a look at the guide to get started, and the zmq_socket(3) manpage specifically for how push and pull works.
Bindings are available for all the languages you mention.
As for the contents of the message, ØMQ doesn't concern itself with that, they are just blocks of raw data. You can use any format that suits you, such as JSON, or perhaps Protocol Buffers.
What I'm not sure about is the ‘controlling state’ you mention. Are you interested in, for example, cancelling a job halfway through?
For C# to C# you can use Windows Communication Foundation. You may be able to use it with Python and C++ as well.
You may also want to checkout named pipes.
I would think about moving to a model where you eliminate the issue by having centralized data that all of the applications look at. Keep "one source of the truth" so to speak.
Most outside software has trouble linking against C++ code, due to the name-mangling algorithm it uses for its symbols. For that reason, when interfacing with programs written in other languages, it is often best to declare wrappers to things as extern "C" or inside an extern "C" { block.
I need to be able to transfer a data set from one application to another easily (the data set in question is not huge, probably a few megabytes)
Use the file system.
and I also need some form of way to control the current state of the operation
Again, use the file system. A "current_state.json" file with a JSON serialized object is perfect for multiple languages to work with.
It seems like sockets or maybe SOAP would be a universal solution.
Perhaps. But it's overkill for this kind of thing. Your OS already has all the facilities you need. Just use the file system. It's very simple and very reliable.
There are many ways to do interprocess communication. As you said, sockets may be a universal solution. SOAP, i think, is somewhat an overkill. You may also use mailslots. I wrote C++ application using it a couple of years ago. Named pipes could be also a solution, but if you are coding on Windows, it may be difficult.
In my opinion:
Sockets
Mailslots
Are the best candidates.
So, there are several languages which will allow you to create a website, as long as you configure the server(s) well enough.
To my knowledge, there is:
PHP
ASP.NET
Ruby(on rails, what is
that all about?)
And thusly, my knowledge is limited. Ruby and ASP, I've only heard of, never worked with. If there are other languages, I suppose they have some way to make files containing the needed html. It would then suffice to add a line to the Apache config to associate the file-extension.
And if other languages: are there any notable characteristics about the one(s) you mention?
ANY language can be use to make a dynamic website - you could do it in COBOL or FORTRAN if you were twisted enough. Back in the olden days (about 10 years ago) most dynamic websites were done with CGI scripts - all you needed was a program that could read data from standard input and write data (usually HTML) to standard output.
Most modern languages have libraries and frameworks to make it easier. As well as the languages you have already mentioned, Java, C# and Python are probably the most common in use today.
Typically a web framework will have:
a way of mapping URLs to a class or function to handle the request
a mechanism for extracting data from a request and converting it into an easy to use form
a template system to easily create HTML by filling in the blanks
an easy way to access a database, such as an ORM
mechanisms to handle caching, redirections, errors etc
You can find a comparison of popular web frameworks on wikipedia.
How can you forget Java ? :)
Python
It runs on Windows, Linux/Unix, Mac OS X, and has been ported to the Java and .NET virtual machines.
Python is a perfect scripting language for web applications, e.g. via mod_python for the Apache web server. With Web Server Gateway Interface a standard API has been developed to facilitate these applications. Web application frameworks or application servers like Django, Pylons, TurboGears, web2py and Zope support developers in the design and maintenance of complex applications. Around libraries like NumPy, Scipy and Matplotlib, Python is a standard in scientific computing.
Among the users of Python are YouTube and the original BitTorrent client. Large organizations that make use of Python include Google, Yahoo!, CERN, NASA,and ITA.
This could be for your interest.
Virtually thru CGI all programming languages that produce output may use for web page generation.
Basically, you can use any language (if you are hosting your own server)
Very closely related and very interesting is this article where LISP has been used to build a very succesfull website.
Python has a 3rd party module CherryPy which can be used with or without a http server.
Amongst others: Erlang (YAWS, Mochiweb), Python
JSP has the advantage that it automatically wraps your code in a servlet, compiles that to bytecodes, then uses the just-in-time Java compiler to recompile critical sections into native object code. Not aware of any alternative which allows optimizes your work automatically in this way.
Also allows you to develop and deploy on any combination of Windows, Mac OS X, or Linux.
If you'd like to choose one for the beginning, you should check out PHP first. It gives you the basic clues about how dynamic sites work in general.
After you've become familiar with the basics, I recommend ASP.NET.
Fist off, you should know that ASP.NET is a technology and not a language. (It actually supports any language that can be used on the .NET platform.) Also it is not to be confused with classic ASP. (The old ASP was much more like PHP.)
ASP.NET is very easy to begin with, and after you have some clues about its concepts, you can always dig deeper and customize everything in it. The http://asp.net site is a very good starting point, if you are to learn it. I think it is really worth the effort, because even if you choose not to stick to it, it will give you some interesting ideas and concepts.
I tell you its most important advantages:
The code is compiled (and NOT interpreted like PHP), and it has a very good performance. (In a performace comparsion, it is 10-15 times faster. http://www.misfitgeek.com/pages/Perf_Stat_0809.htm)
It can be run on Windows without effort, and on Linux / Mac / etc using the Mono project.
It implements the Separation of Concerns principle very well.
It has most of the general functionality you'll need built-in. (Such as membership, roles, database management, and so on.)
Is there a general procedure for programming extensibility capability into your code?
I am wondering what the general procedure is for adding extension-type capability to a system you are writing so that functionality can be extended through some kind of plugin API rather than having to modify the core code of a system.
Do such things tend to be dependent on the language the system was written in, or is there a general method for allowing for this?
I've used event-based APIs for plugins in the past. You can insert hooks for plugins by dispatching events and providing access to the application state.
For example, if you were writing a blogging application, you might want to raise an event just before a new post is saved to the database, and provide the post HTML to the plugin to alter as needed.
This is generally something that you'll have to expose yourself, so yes, it will be dependent on the language your system is written in (though often it's possible to write wrappers for other languages as well).
If, for example, you had a program written in C, for Windows, plugins would be written for your program as DLLs. At runtime, you would manually load these DLLs, and expose some interface to them. For example, the DLLs might expose a gimme_the_interface() function which could accept a structure filled with function pointers. These function pointers would allow the DLL to make calls, register callbacks, etc.
If you were in C++, you would use the DLL system, except you would probably pass an object pointer instead of a struct, and the object would implement an interface which provided functionality (accomplishing the same thing as the struct, but less ugly). For Java, you would load class files on-demand instead of DLLs, but the basic idea would be the same.
In all cases, you'll need to define a standard interface between your code and the plugins, so that you can initialize the plugins, and so the plugins can interact with you.
P.S. If you'd like to see a good example of a C++ plugin system, check out the foobar2000 SDK. I haven't used it in quite a while, but it used to be really well done. I assume it still is.
I'm tempted to point you to the Design Patterns book for this generic question :p
Seriously, I think the answer is no. You can't write extensible code by default, it will be both hard to write/extend and awfully inefficient (Mozilla started with the idea of being very extensible, used XPCOM everywhere, and now they realized it was a mistake and started to remove it where it doesn't make sense).
what makes sense to do is to identify the pieces of your system that can be meaningfully extended and support a proper API for these cases (e.g. language support plug-ins in an editor). You'd use the relevant patterns, but the specific implementation depends on your platform/language choice.
IMO, it also helps to use a dynamic language - makes it possible to tweak the core code at run time (when absolutely necessary). I appreciated that Mozilla's extensibility works that way when writing Firefox extensions.
I think there are two aspects to your question:
The design of the system to be extendable (the design patterns, inversion of control and other architectural aspects) (http://www.martinfowler.com/articles/injection.html). And, at least to me, yes these patterns/techniques are platform/language independent and can be seen as a "general procedure".
Now, their implementation is language and platform dependend (for example in C/C++ you have the dynamic library stuff, etc.)
Several 'frameworks' have been developed to give you a programming environment that provides you pluggability/extensibility but as some other people mention, don't get too crazy making everything pluggable.
In the Java world a good specification to look is OSGi (http://en.wikipedia.org/wiki/OSGi) with several implementations the best one IMHO being Equinox (http://www.eclipse.org/equinox/)
Find out what minimum requrements you want to put on a plugin writer. Then make one or more Interfaces that the writer must implement for your code to know when and where to execute the code.
Make an API the writer can use to access some of the functionality in your code.
You could also make a base class the writer must inherit. This will make wiring up the API easier. Then use some kind of reflection to scan a directory, and load the classes you find that matches your requirements.
Some people also make a scripting language for their system, or implements an interpreter for a subset of an existing language. This is also a possible route to go.
Bottom line is: When you get the code to load, only your imagination should be able to stop you.
Good luck.
If you are using a compiled language such as C or C++, it may be a good idea to look at plugin support via scripting languages. Both Python and Lua are excellent languages that are used to script a large number of applications (Civ4 and blender use Python, Supreme Commander uses Lua, etc).
If you are using C++, check out the boost python library. Otherwise, python ships with headers that can be used in C, and does a fairly good job documenting the C/python API. The documentation seemed less complete for Lua, but I may not have been looking hard enough. Either way, you can offer a fairly solid scripting platform without a terrible amount of work. It still isn't trivial, but it provides you with a very good base to work from.