Has anybody done or seen a deployment of Apache Thrift in an iPhone app?
I am wondering if is a reasonable solution for a high-volume, low(er)-latency network service for iPhones compared to HTTP.
One noteworthy thing I found is a bug report about running Thrift on the iPhone, which seems to have been fixed. But that doesn't necessarily indicate that it's a done deal.
Thrift and HTTP aren't mutually exclusive. In fact thrift now ships with an HTTP transport implementation to use. It's also a really nice way to auto-generate server/client code that avoids a lot of marshalling/unmarshalling boilerplate while still being really fast. Its internal representation is basically binary JSON, so it's very similar to a RESTful web service (except being easier to code and much, much faster).
So... anyone able to answer the original question? If not, I'll dive in myself with thrift's included Cocoa support and see how it works on the iphone.
Just my two cents..
The accepted answer to this question, is an opinion to not use a technology, not an answer of whether it is possible.
Thrift, is an interface definition language, IDL, like Protobuf and Capt'n'Proto. They permit the definition of a client/server/server protocol which is platform agnostic. JSON and Plist don't provide the same level of type conformance.
Having previously lead an iOS team with 10Ms MAU using Google Protobuf v2.5 on iOS, Android, Windows, and server teams, I can attest that IDLs are great on mobile. Apple uses them for syncing iWork content.
My current team uses Thrift for iOS and Android clients, with a mostly Scala backend. I much prefer it to Protobuf.
We send Thrift payloads over HTTPS and WebSockets. Once you have defined (in Thrift) your our wire communication protocol (i.e. frame structure), it's very easy to evolve your APIs.
However, on iOS in particular there are some implementation issues. The current version of the library is quite poorly packaged, and if you hope to make an Objective-C framework (e.g. for iOS 8+), then you will not be able to out of the box with v0.9.2. This is because the library headers include local imports, (#import "TProtocol.h" instead of #import <Thrift/TProtocol.h>) with no umbrella headers. Worst of all, the Objective-C compiler generates very messy Objective-C classes, also including local imports from the Thrift library.
Some of these issues are pretty damning. It indicates to me that while use of an IDL is very much a good engineering decision, not many iOS teams are using Thrift, unless they're huge with the resources to write their own library.
I've always disliked frameworks that use a common interface definition that builds out both server and client code. It keeps both sides too much in lockstep where in reality server API changes must be very flexible in the versions of clients that are communicating with it.
There are helpful libraries that make JSON or PLIST communication over HTTP pretty easy, and decades of debugging and understanding the HTTP protocol and how to use it well. I would ignore that at your peril.
I have used thrift's objective c bindings for a large iPhone app with a few million users. As one of the posters mentioned we can use Http which gets the best of both worlds. However there is no asynchronous HTTP client for thrift. We had to build an event based wrapper to allow non-blocking I/O calls. The underlying layer still issues one call at a time which hit us in a big way because we have one server call that takes a long time but it does not block UI flow and another really fast one that does block UI flow. If the underlying layer is busy with the slow command our fast command just has to wait. I am trying to build asyc http in c++ which can then be used on the iPhone but that is someways off from being ready.
Thrift as an external API doesn't make sense. Use it internally rock and roll.
Related
I see that Go itself has a package net/http, which is adequate at providing everything you need to get your own REST APIs up and running. However, there are a variety of frameworks; the most popular maybe say gorilla.
Considering that one of the main things I need to do going forward is to build REST APIs that will access some back-end storage (databases, caches, etc.) to perform CRUD operation, is it good to go with Go's standard library itself, or should I consider using some frameworks?
Normally, people write a new library or framework which solves the problem present in the existing library. But a lot of the frameworks also tend to make things worse when actual demands are simple.
So I have few questions:
Is the basic library in go lang good enough to support basic to moderate functionality for REST?
If I do end up using the inbuilt library and tomorrow have to change it to use some framework (like a gorilla), how difficult/costly would that be?
Are frameworks really addressing the problems or just making simple problems complex?
I would be extremely grateful for someone to share his thoughts here (who has been through making this choice himself) while I research more of my own.
The net/http package is probably sufficient for most scenarios, but if you want to ease your development, you should use a third-party package, such as Gorilla.
For example, net/http's ServeMux does a great job at routing incoming requests for fixed URL paths but for pretty paths which use variables, you will need to implement a custom multiplexer while using Gorilla, you are getting this for free.
Another example is if you want to specify RESTful resources with
proper HTTP methods, it is hard to work with the standard
http.ServeMux, while with Gorilla's mux package,
requests can be matched based on URL host, path, path prefix,
schemes, header and query values, and HTTP methods.
One of the great benefits of Gorilla is that it is fully compatible with the net/http package and can be substituted in the future.
See 1.
I totally encourage you to use Gorilla's toolkit to develop REST services.
The built-in net/http package is sufficient to build a complete REST API. However, some of the libraries can make building an API slightly easier, particularly if the REST API is complex. Changing from the built-in facilities to any decent framework is relatively straightforward - they generally accept handlers of the http.Handler type.
In the end, though, this is an extremely situational choice. The best thing you can do is examine each available solution, contrast and compare, and build a proof of concept with the top options if you possibly can. First-hand experience will guide you best.
What I want to do: I want to add communication capabilities to a couple of applications (soon to be jar libraries for Java) in Scala, and I want to do it in the most painless way, with no Tomcat, wars, paths for GET requests, RPC servers, etc.
What I have done: I've been checking a number of libraries, like Jetty, JAX-RS, Jackson, etc. But then I see the examples and they usually involve many different folders for configuration, WSDL files, etc. Most of the examples lack a main method and I don't have a clear picture about how many additional requirements may they have (e.g. Tomcat).
What I am planning to do: I'm considering to simply open a socket on the "server" to listen, then connect with the "client" and transfer some JSON, in both directions. This should be fairly standard so that I can use other programming languages in compatible ways (e.g. Python).
What I am asking: I would like to know whether there is some library that makes this easier. Not necessarily using raw sockets, but setting up some process communication in just a few lines, maybe not as simple as Node.js, but something similar.
Bonus: It would be cool to
be able to use other programming languages (e.g. Python) by using open standards
have authentication
But I don't really need any of those at this point.
I think you need RPC client/server system, I would suggest to take one of these two:
Finagle - super flexible and powerful RPC client/server from Finagle. You can define your service with Thrift, and it will generate stubs for client/server in scala. With Thrift it should be straightforward to add Python support.
Spray - much smaller library, focused on creating REST services. It's not so powerful as Finagle, however much easier. And REST allows you to use any other clients
Remotely - an elegant RPC system for reasonable people. Interesting and very promising project, however maybe difficult to start with because of extensive Scalaz+Shapeless+Macro usage
Honestly if you want something that is cross-language compatible, simple, straightforward, and concise then you do not want to use plain old sockets!
Check out dropwizard. It is amazing and I use it for small and large projects alike! It is usually configured by no more than a single configuration file. It supports authentication too!
Out of the box it gives you really great inter-process communication over JSON (using Jackson) and much much more. There is also pretty decent Scala support for dropwizard.
If you must roll your own then I'd recommend using Jackson for JSON parsing. It's super simple to use and also has great scala support.
If you've got a "controlled" use case where the client and server are on the same LAN and deployed in tandem, I'd (controversially) recommend Java RMI; it's dumb and JVM-specific (and uses a Java-specific protocol), but it's very simple to use.
If you need something more robust and cross-language, I'd recommend Apache Thrift. You write your interfaces in a platform-independent interface definition language, and it's very clear which changes are compatible and which are not; the thrift compiler generates skeleton interfaces for you to use, and then you just write an implementation of that interface and a couple of lines to start the server (as you can see from the example on the homepage). It's also got good support for async implementations if you need the performance. Thrift itself is reasonably standard and cross-platform, with its own binary protocol, or you can use JSON as a transport if you really want to (I'd recommend against that though).
RabbitMQ provides one easy way to do what you want without writing a server and implementing your own persistence, flow control, authentication, etc. You can brew or apt-get install it.
You start up a broker daemon process (i.e. manages message queues)
In the Scala producer, you can use Maven-provided Java API to send JSON strings without any fuss (e.g. no definition languages) to specified queues
Then in your other Scala program, connect to the broker, and listen for messages on the queue, and parse the incoming JSON
Because it is so popular, there are many tutorials online for different patterns you may want to use to distribute the messages, e.g. pub/sub, one-to-one, exactly-once delivery, etc.
Is there a distributed application framework (commercial is okay as well) that supports iPhone / iPad ?
What I'm looking for in the framework:
Allows me to focus on the application logic
I don't have to code "low-level" network programming (I've done it too many times that I dont wanna do it again =p)
Should be actively maintained (popular would be nice)
Basically, I can then develop faster.
We plan to develop a soft real-time TCP/IP client/server application where there are many iPhone/iPad clients (30+) connected to single server over LAN. The server most likely will run Windows (unless the framework does not support it).
I've been looking around and I see:
MonoTouch WCF (still looks quite raw ?)
RemObjects (Mono + Objective-C)
Cocoa Distributed Objects
ZeroC Ice Touch (Objective-C)
RakNet ( ? included because it mentions iPhone, but will need to use C++)
Of course, there's also the option of using the plain old MonoTouch System.Net.Sockets
Or, CFNetwork (I dont plan to use this one)
I'm still deciding whether to use Objective-C or MonoTouch, but leaning towards MonoTouch since we will get the .NET framework, and not be tied into just the Mac world.
Please feel free to comment if I added anything that's not related to my question---I'm new to iPhone/iPad world.
We've used WCF/Monotouch with great success - there are some areas of the f/work that arent 100% but for most cases you should find working with WCF on monotouch a breeze.
The ability to share all of our data sync, model, tests etc between monodroid and monotouch and wm7 is seriously cool (with some working - this is easilly possible - you'll need to manage multiple prj files).
Be careful to manage calls to wcf services correctly, keep them to a minimum, keep the archetecture simple. We ended up with a fairly complex dto to minimise the amount of calls to the wcf services to sync the data - this was well worth it as the time needed to sync a device from scratch is now a fraction of what it was.
Using SSL to communicate with the server is a PITA but I think that's more a case of the way apple have managed it.
You need to be a bit more explicit on your requirements. If you need only object serialization (dehydration/hydration) over REST API, then anything that supports POX or JSON will work just fine for you. However, if you need RPC-style method invocation, with authentication, encryption/digital signature, transactions, etc, then you need one of those frameworks you listed above.
If you need a framework, I personally would lean towards the MonoTouch WCF, as it gives you the ability to move your client to other platforms later as well (Windows Phone 7 for example). Then again, as you said, it's a bit rough right now, and if Mono team decides in the future that they don't have the resources to invest in maintaining it, you might end up with having to move to another framework. Of course, there's also the drawback that you need to use MonoTouch for your application, and can't use Objective-C. Granted, with the recent changes in the iOS Developer Agreement, that's not that much of an issue, but it is still something to keep in mind.
(Disclaimer: I used to work on Microsoft's WCF team, so I am biased towards the product itself)
The other option I would go for, would be Cocoa Distributed Objects. However, that would be my choice if the server is also running on OS X. I know there's Bonjour for Windows, but I doubt it's optimized for server scenarios, and I also don't know how rich is Apple's RPC implementation on top of it for the Windows platform. So I would stay with Apple's technology only if I am building exclusively for Apple's platform.
Note that WCF and Distributed Objects would give you RPC-style functionality, but they won't help you with any particular scenarios. If you need/want even higher level of abstraction, for example you need presence information or multi-user chat, you will still need to implement those yourself. It might be worth at this point to look at frameworks that provide those features for you. An example would be RakNet (which you listed above), which abstracts the remoting level and builds additional features on top of it.
You can use JSON Touch + Vitche PHP Emission Framework which provides all server-side you need. Also you can use that Framework to access existing SOAP (WCF, Axis, etc) services.
You can use Google protocol buffers to implement RPC though you will need to do some network programming for transporting your messages anyway. It supports interface generation for C++, Java, Python and Objective-C and .NET so you can create a single set of RPC messages and get code for working with them for almost any mobile platform. Transport layer on your mobile platforms you will have to implement yourself.
http:// code.google.com/apis/protocolbuffers/ - main Protobuf page (C++, Java, Python)
http:// code.google.com/p/protobuf-net/ - Protobuf .NET mentioned in one of the comments
http:// code.google.com/p/metasyntactic/wiki/ProtocolBuffers - Protobuf for Obj-C
I'm just getting into the client-server data sync stage of my iPhone app project, and have managed to get my CoreData data model loading on both the iPhone client and my TurboGears server (which is good). I'm now beginning to tackle the problem of sync'ing data between the server and multiple clients, and while I could roll my own, this seems like one of those problems that is quite general and therefore there should be frameworks or libraries out there that provide a good deal of the functionality.
Does anyone know of one that might be applicable to this environment (e.g. Objective-C on iPhone, pyobjc / Python on the server)? If not, does anyone know of a design pattern or generally-agreed upon approach for this stuff that would be a good road to take for a self-implementation? I couldn't find a generally accepted term for this problem beyond "data synchronization" or "remote object persistence", neither of which hit much useful on Google.
I did come across the Funambol framework which looks like it provides this exact type of functionality, however, it is C++ / Java based and therefore seems like it might not be a good fit for the specific languages in my project.
Any help much appreciated.
Since you are using TurboGears already, take a look at the RestController documentation. Using RESTful services has become a widely adopted architecture with many implementations for both clients and servers. Matt Gemmell's MGTwitterEngine is a good example of the client implementation of a specific API, Twitter.
If I have make an application, how would I make it interact with a database on a server of mine? I'm kind of lost, and don't know where to start.
Vague question yields a vague answer.
It entirely depends on what kind of interaction you need. Lots of constant queries? Few and far between? Client side cache? Real time updates? All of these questions will impact the answer.
The easiest way is to go with an AJAX style HTTP based client/server type of interaction. Sticking a database behind a web server has been done about a bazillion times and, thus, you'll find lots and lots of examples and, even, solutions with a few google searches.
You don't really need to use JavaScript (the J in AJAX). Instead, send over an HTTP request that encapsulates your query and have the server respond with an XML document containing the answer.
If that won't work for you -- too much overhead, need to relay binary information (for which XML sucks), etc.. -- then you'll wan to go with more direct access to the database. That is harder and can range from porting a client library to the iPhone to creating your own wire protocol.
A significantly harder problem and you'll have to deal with networking issues -- firewalls, NAT, proxies, etc... -- that are generally already solved with HTTP.
Stick with HTTP until you prove that it won't work. Much simpler.
Search for "http request" in the iPhone docs. HTTP client APIs are included.
You might try Apple's "Getting Started with Networking & Internet" or "Introduction to the URL Loading System", in addition to the Networking section of the iPhone Application Programming Guide. There are also several sample applications that handle networking of various types within the ADC's iPhone section.
Additionally, there are a number of open source iPhone applications out there that handle networking, including my own.
I had a similar question regarding a rails app - the answers there may help you.
What is the best approach for building an iphone client for a rails app?
But the answer really depends on your knowledge of iphone programming and server side programming, plus how your database is set up at the moment - but most likely you're going to need to write some kind of webservice / REST API that allows a remote client to do things with your database.
There are many frameworks available for that on the server side - if you're starting from scratch, ruby on rails may be a good choice.
On the iphone side, you'll probably want to start by reading up on NSURLConnection, and various request/response formats that you can use with it. I've found exchanging plists between the phone and server to be a pretty easy approach.