This is a copy of a post I've sent to the ZeroMQ mailing list. However, the question is raises it not specific to ZeroMQ, but more generally regarding the need for a 'mapping' layer over the networking functionality provided in WinRT to provide a more normal 'Berkeley Socket facade' for C++ code when compiling against WinRT:
Hi all, I've used ZeroMQ in a mobile app (see http://www.ibuzzedfirst.com ) previously, for the iPhone and Android versions, as those platforms support native/C++/Socket development, and therefore ZeroMQ.
For the WindowsPhone 7.5 (OS 7.1) version, I had to re-implement any required ZeroMQ functionality from scratch, as WinPhone 7.5 only supports C#, not C++ (it's effectively a C# Silverlight App). Also, WinPhone 7.5 only provides its own 'version' of Socket support ( http://msdn.microsoft.com/en-us/library/sb27wehh%28v=vs.95%29.aspx ) which only support Async versions of functions, e.g. ConnectAsync, SendAync, ReceiveAsync etc. However, the lack of C++ made this a moot point.
As such, for the WindowsPhone 7.5 version, I restricted the app to 'client' (Contestant) functionalty only, and didn't implement the 'server' (Quiz Master) part. This was because the client part of the app only sends and receives requests, replies and subscriptions to the server, whereas the server utilises the inherent multi-threaded multi-user functionality of ZeroMQ. It was (relatively) simple to recreate the ZeroMQ transport protocol/headers for client use, and use the WindowsPhone Socket support to provide comms.
Ok, now I'm looking at porting the app to WinRT on Windows 8. (The desktop/tablet version first - the Windows Phone 8 RT SDK isn't out yet, but will be similar). The good news is that C++ is supported in WinRT, yay! (Actually, it still isn't that simple, when writing C# only WinRT apps, you can compile for 'AnyCPU', as soon as you include a C++ portion, you have to build 3 different versions - x86/Win32, x64, and ARM versions, but that's a different problem).
Unfortunately though, like Windows 7/8 Phone, WinRT does NOT support 'normal' Berkeley Socket access, but instead offers its own 'version' of Socket programming, with discrete classes for different socket scenarios, e.g. StreamSocket for a connecting TCP client ( http://msdn.microsoft.com/en-us/library/windows/apps/windows.networking.sockets.streamsocket.aspx#methods ), StreamSocketListener for a bindable TCP server ( http://msdn.microsoft.com/en-us/library/windows/apps/windows.networking.sockets.streamsocketlistener.aspx#methods ), and DatagramSocket / DatagramSocketListener for the UDP versions. Furthermore, only async versions of all the methods are provided.
So it looks like, to get ZeroMQ to compile sucessfully on WinRT, I'm going to have to write a Facade layer that provdes a Berkeley Socket-like C++ interface, and underneath performs the necessary mapping to the version of Socket programming provided by WinRT.
Has anyone else started this journey or written a similar facade? Interested to hear everyone's thoughts, especailly as WinRT looks to quite a 'big thing'!
Though its far from complete or correct and has lots of bugs, but I have started this project here https://winrtsock.codeplex.com. Have not implemented any listen/accept as of yet
You might consider something like ACE. It provides slightly higher-level abstractions over sockets, and supports older windows embedded OSs such as WinCE and the like. It's open source, so you can try it out, hack it up to make it work and contribute back. Alternatively you can contact one of several commercial organizations or people that offer support and contract the work.
Related
What technology, frameworks, programming languages are involved? Must support for tcp inbound and outbound connections, read webcam and mic data.
Browser plugins have access to the operating systems native APIs, so there is no major difference to a stand-alone application regarding camera-, audio- and network-access. What you have to be aware of though is that your code might run in a less privileged process.
For Internet Explorer you will have to write content extensions, specifically ActiveX controls, using C++. For all other browsers you will write NPAPI plugins C++ or C.
On Windows you can combine the two plugin types into one DLL. FireBreath does that, which you should look into as it already abstracts over the browsers (and other problems) for you.
I am trying to evaluate some technologies for implementing a communication process between some Ada modules with some C++/OpenGL modules. There is an (Windows XP) Ada application which communicates with a C++ application using COM, but I intend to switch the COM to a new technology. Some suggestions came up, such as direct Sockets, DSA, Polyorb, Corba and DSS/Opensplice.
DSA appears to be just Ada -implemented (not sure)
Polyorb has its last implementation on 2006, according to http://polyorb.ow2.org/
Corba someone argumented that it could be not simple enough to justify its complexity for implementing simple applications
DSS/Opensplice appears to be just C/C++ implemented, so an Ada binding should be done. It also looks to be not very simple to be implemented too.
Personally I like COM, but due to the migration, I'd rather take the sockets option due to its simplicity, and the interface architecture could be implemented very easily.
So, what you think? Could you please comment about these technologies or even suggest other more?
Thanks very much.
A big factor in your choice is the size and complexity of the system you're reengineering. Is it a broadly distributed system with lots of complex messages? Is it a relatively small system with a handful of mundane message exchanges?
For small systems I used to just roll-my-own socket-based comm modules. Now, though, I lean more towards ZeroMQ (brokerless) or STOMP (text-based). And there's some Ada support for these, zeromq-Ada and TOMI_4_Ada (supports both).
While these handle the distribution mechanics, you would still need to handle the serialization of the messages into transportable form.
CORBA/PolyORB and DDS solutions are rather heavyweight, but are complete solutions. If you don't fear IDL and managing brokers, they can do well for large-scale distributed systems. Yeah, there may need to be some Ada bindings built, but if you can get C headers or a C API to bind to, it's typically not too bad if you focus on just binding the functions and data structures you require. Rather than creating a comprehensive binding, liberally employ opaque and void pointers (void_ptr, opaque_structure_def_ptr) for structs and parameters whose internal contents you don't care about.
we intend to switch the COM to a new (suported) technology, since COM is not more supported by Microsoft
Whoever told you COM is no longer supported is totally clueless.
While COM has undergone many name changes (OLE, COM, OLE Automation, DCOM, COM+, ActiveX, WinRT) and extensions over the past decades, it is the single most important technology for MS platforms: past, present and future. The .NET runtime uses COM extensively. Much of the Win32 API is written in COM, and the portions that weren't, will be in Win8, since WinRT components are COM objects.
Also take a look at AMQP (RabbitMQ for server), there seems to be Ada library available for it http://www.gti-ia.upv.es/sma/tools/AdaBinding/index.php.
If you could find binding for Ada, Apache thrift might also be a lightweight option. Maybe you could even write your own binding, it should not be more difficult that rolling something of your own over the sockets.
If you do go sockets route, than I would suggest ZeroMQ as "supersockets".
One more option for your list should be to use Ada's distributed programming support, and write C/C++ wrappers to interface your C++ program into it.
I don't know that its the best option for your needs, but if your Ada compiler supports Annex E, it should be on the list.
Since this post, AdaCore published PolyORB on GitHub with regular updates :)
This is more of a general quick question.
But in like C#,Python,C,C++.......etc
Are all the basic "Sockets" network programming essentially the same. Like do they all use the Berkley Sockets (i think thats what they are called) or does each language have it's own way of doing sockets.
Thanks
Sockets are platform-dependent, not language dependent. So, linux uses the BSD sockets alone, Windows offers both BSD sockets (used almost exclusively) and a M$ flavour of sockets (called WSA), dunno about others. It all boils to what is found under the hood - more exactly at the kernel level of the OS. The socket implementation found there will offer a first set of API in order to make them accessible to kernel/user space - usually through a shared object / dynamic linked library and thus "natively" supporting the c/c++ languages. All the other languages are relying on language specific bindings to those SO / DLL files offered by the OS natively for C/C++.
HTH
Sockets are basically the same in C, C++, Java, Ruby. They are slightly easier (because the build in classes handle the boiler plate) in higher level languages. If you can write Socket code in C, then you can do it anywhere if you have a reference to translate.
#Kellogs brings up a good point, Windows has their own Socket API which (typically) performs better (in my experiences) on Windows than the Posix implementations offered. The APIs are very similar. I would make the analogy of OpenGL to DirectX. If you really know one, then you can figure out the other with a reference.
Now a days we don't differentiate the language, the class library that can be accessed from the language and the underlying operating system. Here is my explanation
C,C++,C#,Java - Are just the languages has no specific support regarding the network programming.
Java class library, .Net Framework, C++ standard library - among this I think C# & Java provides some classes for network programming. C++ standard library does not provide any network programming classes (only iostreams for file, stdinput & strings are available). But BOOST library for C++ provides classes for network programming. I am not aware of the other libraries.
OS - The operating system proives a base api (mostly in C) that is utilised by the class libraries above. In case of windows it is the winsock api (WSA) and in case of unix it is the BSD socket api. I think windows also supports to some extent the BSD api so that the learning curve is less. But as #EnabrenTane said it is more than that.
Agree with kellogs above. Windows & most major POSIX-compliant OSs support the basic BSD socket API. In addition to the Berkeley socket, each platform provides additional proprietary API for performance & scalability enhancement. These extended APIs are not strictly confined to sockets only - they work on any I/O operations (i.e. disk files, named pipes etc.). For example, (in addition to traditional select() & poll()) Linux has it's own epoll mechanism, BSDs have kqueue, Windows has WSAevent*, WSAaAsync* and I/O completion ports APIs.
These differences are visible mostly in lower level languages like C, C++ or Pascal.
C#, Java, Python, Ruby et al. are somewhat "higher level" compared to C - they provide robust technologies to insulate you from the low-level APIs. Instead of fiddling directly with the bare-bones socket API, you can use the extensive run-time class libraries provided with each platform. For example, to write a TCP/IP server in Python, you could simply use the SocketServer class. Similarly in C#, you can use WebClient to downlad files of the net. In C++, you have the Boost library. Of course, there's nothing stopping you from using raw socket API directly in your app.
Is there a distributed application framework (commercial is okay as well) that supports iPhone / iPad ?
What I'm looking for in the framework:
Allows me to focus on the application logic
I don't have to code "low-level" network programming (I've done it too many times that I dont wanna do it again =p)
Should be actively maintained (popular would be nice)
Basically, I can then develop faster.
We plan to develop a soft real-time TCP/IP client/server application where there are many iPhone/iPad clients (30+) connected to single server over LAN. The server most likely will run Windows (unless the framework does not support it).
I've been looking around and I see:
MonoTouch WCF (still looks quite raw ?)
RemObjects (Mono + Objective-C)
Cocoa Distributed Objects
ZeroC Ice Touch (Objective-C)
RakNet ( ? included because it mentions iPhone, but will need to use C++)
Of course, there's also the option of using the plain old MonoTouch System.Net.Sockets
Or, CFNetwork (I dont plan to use this one)
I'm still deciding whether to use Objective-C or MonoTouch, but leaning towards MonoTouch since we will get the .NET framework, and not be tied into just the Mac world.
Please feel free to comment if I added anything that's not related to my question---I'm new to iPhone/iPad world.
We've used WCF/Monotouch with great success - there are some areas of the f/work that arent 100% but for most cases you should find working with WCF on monotouch a breeze.
The ability to share all of our data sync, model, tests etc between monodroid and monotouch and wm7 is seriously cool (with some working - this is easilly possible - you'll need to manage multiple prj files).
Be careful to manage calls to wcf services correctly, keep them to a minimum, keep the archetecture simple. We ended up with a fairly complex dto to minimise the amount of calls to the wcf services to sync the data - this was well worth it as the time needed to sync a device from scratch is now a fraction of what it was.
Using SSL to communicate with the server is a PITA but I think that's more a case of the way apple have managed it.
You need to be a bit more explicit on your requirements. If you need only object serialization (dehydration/hydration) over REST API, then anything that supports POX or JSON will work just fine for you. However, if you need RPC-style method invocation, with authentication, encryption/digital signature, transactions, etc, then you need one of those frameworks you listed above.
If you need a framework, I personally would lean towards the MonoTouch WCF, as it gives you the ability to move your client to other platforms later as well (Windows Phone 7 for example). Then again, as you said, it's a bit rough right now, and if Mono team decides in the future that they don't have the resources to invest in maintaining it, you might end up with having to move to another framework. Of course, there's also the drawback that you need to use MonoTouch for your application, and can't use Objective-C. Granted, with the recent changes in the iOS Developer Agreement, that's not that much of an issue, but it is still something to keep in mind.
(Disclaimer: I used to work on Microsoft's WCF team, so I am biased towards the product itself)
The other option I would go for, would be Cocoa Distributed Objects. However, that would be my choice if the server is also running on OS X. I know there's Bonjour for Windows, but I doubt it's optimized for server scenarios, and I also don't know how rich is Apple's RPC implementation on top of it for the Windows platform. So I would stay with Apple's technology only if I am building exclusively for Apple's platform.
Note that WCF and Distributed Objects would give you RPC-style functionality, but they won't help you with any particular scenarios. If you need/want even higher level of abstraction, for example you need presence information or multi-user chat, you will still need to implement those yourself. It might be worth at this point to look at frameworks that provide those features for you. An example would be RakNet (which you listed above), which abstracts the remoting level and builds additional features on top of it.
You can use JSON Touch + Vitche PHP Emission Framework which provides all server-side you need. Also you can use that Framework to access existing SOAP (WCF, Axis, etc) services.
You can use Google protocol buffers to implement RPC though you will need to do some network programming for transporting your messages anyway. It supports interface generation for C++, Java, Python and Objective-C and .NET so you can create a single set of RPC messages and get code for working with them for almost any mobile platform. Transport layer on your mobile platforms you will have to implement yourself.
http:// code.google.com/apis/protocolbuffers/ - main Protobuf page (C++, Java, Python)
http:// code.google.com/p/protobuf-net/ - Protobuf .NET mentioned in one of the comments
http:// code.google.com/p/metasyntactic/wiki/ProtocolBuffers - Protobuf for Obj-C
Has anybody done or seen a deployment of Apache Thrift in an iPhone app?
I am wondering if is a reasonable solution for a high-volume, low(er)-latency network service for iPhones compared to HTTP.
One noteworthy thing I found is a bug report about running Thrift on the iPhone, which seems to have been fixed. But that doesn't necessarily indicate that it's a done deal.
Thrift and HTTP aren't mutually exclusive. In fact thrift now ships with an HTTP transport implementation to use. It's also a really nice way to auto-generate server/client code that avoids a lot of marshalling/unmarshalling boilerplate while still being really fast. Its internal representation is basically binary JSON, so it's very similar to a RESTful web service (except being easier to code and much, much faster).
So... anyone able to answer the original question? If not, I'll dive in myself with thrift's included Cocoa support and see how it works on the iphone.
Just my two cents..
The accepted answer to this question, is an opinion to not use a technology, not an answer of whether it is possible.
Thrift, is an interface definition language, IDL, like Protobuf and Capt'n'Proto. They permit the definition of a client/server/server protocol which is platform agnostic. JSON and Plist don't provide the same level of type conformance.
Having previously lead an iOS team with 10Ms MAU using Google Protobuf v2.5 on iOS, Android, Windows, and server teams, I can attest that IDLs are great on mobile. Apple uses them for syncing iWork content.
My current team uses Thrift for iOS and Android clients, with a mostly Scala backend. I much prefer it to Protobuf.
We send Thrift payloads over HTTPS and WebSockets. Once you have defined (in Thrift) your our wire communication protocol (i.e. frame structure), it's very easy to evolve your APIs.
However, on iOS in particular there are some implementation issues. The current version of the library is quite poorly packaged, and if you hope to make an Objective-C framework (e.g. for iOS 8+), then you will not be able to out of the box with v0.9.2. This is because the library headers include local imports, (#import "TProtocol.h" instead of #import <Thrift/TProtocol.h>) with no umbrella headers. Worst of all, the Objective-C compiler generates very messy Objective-C classes, also including local imports from the Thrift library.
Some of these issues are pretty damning. It indicates to me that while use of an IDL is very much a good engineering decision, not many iOS teams are using Thrift, unless they're huge with the resources to write their own library.
I've always disliked frameworks that use a common interface definition that builds out both server and client code. It keeps both sides too much in lockstep where in reality server API changes must be very flexible in the versions of clients that are communicating with it.
There are helpful libraries that make JSON or PLIST communication over HTTP pretty easy, and decades of debugging and understanding the HTTP protocol and how to use it well. I would ignore that at your peril.
I have used thrift's objective c bindings for a large iPhone app with a few million users. As one of the posters mentioned we can use Http which gets the best of both worlds. However there is no asynchronous HTTP client for thrift. We had to build an event based wrapper to allow non-blocking I/O calls. The underlying layer still issues one call at a time which hit us in a big way because we have one server call that takes a long time but it does not block UI flow and another really fast one that does block UI flow. If the underlying layer is busy with the slow command our fast command just has to wait. I am trying to build asyc http in c++ which can then be used on the iPhone but that is someways off from being ready.
Thrift as an external API doesn't make sense. Use it internally rock and roll.