I'm looking for a server side component, preferably java, that will allow me to subscribe to pubsubhubbub feeds through javascript. I understand that subscribers are server side applications in the standard rest/pubsubhubbub format, but Google seems to have created a ajax bridge that looks quite handy.
Unfortunately, I'm dealing with data that simply cannot leave our servers, let alone go through Google's.
Is anyone aware of a (preferably free) server side proxy for pseudo javascript pubsubhubbub subscribers?
Reference: http://code.google.com/apis/feed/push/docs/index.html#hiworld
I know for a fact that Kwwika and Pusherapp are working on this. I can intro you with these guys if you want.
If not, I believe this should be relatively easy to build with stuff like Node.JS for example. This code on Github should be a good first start. Things like this have been built with it.
We (superfeedr) are trying to get more people building similar things...
I'm looking for a server side
component, preferably java, that will
allow me to subscribe to pubsubhubbub
feeds through javascript
There is a java implementation]1 of the subscribe part available. But the hub-part hasn't yet been implemented in java which is needed to subscribe to the feed which should be private. For the javascript(jquery) part I would just use simple long-polling.
Is anyone aware of a (preferably free)
server side proxy for pseudo
javascript pubsubhubbub subscribers?
I don't think a free solution like that exists (yet). Even google's push API isn't open yet.
Unfortunately, I'm dealing with data
that simply cannot leave our servers,
let alone go through Google's.
There isn't yet an implementation of the HUB-part of the pubsubhubbub protoccol. But if it is internally I also don't think you need this kind of fan-out the hub(specification) is offering(broadcast to other servers).
I think you could just use A comet framework like Atmosphere to suspend connection and broadcast feed diff. I think this can be written quick with the Atmosphere framework(1 day you will have a working prototype).
You can see an example using a combination of Superfeedr and Kwwika within a web application that lets you subscribe to any RSS feed or track keywords within RSS feeds here:
http://superfeedr.kwwika.com
And you can get the source code in GitHub here:
http://github.com/kwwika/ASP.NET-MVC-PubSubHubbub-Subscriber/tree/Kwwika-Superfeedr-Demo
Related
this is a general 'what technologies are available' question.
My company provides a web application with a RESTful API. However, it is too slow for my needs and some of the results are in an awkward format.
I want to wrap their restful server with a proxy/adapter server, so when you connect to the proxy you get the RESTful API I wish the real one provides.
So it needs to do a few things:
passthrough most requests
cache some requests
do some extra requests on the original server to detect if a request is cacheable
for instance: there is a request for a field in a record: GET /records/id/field which might be slow, but there is a fingerprint request GET /records/id/fingerprint which is always fast. If there exists a cache of GET /records/1/field2 for the fingerprint feedbeef, then I need to check the original server still has the fingerprint feed beef before serving the cached version.
fix headers for some responses - e.g. content-type, based upon the path
do stream processing on some large content, for instance
GET /records/id/attachments/1234
returns a 100Mb log file in text format
remove null characters from files
optionally recode the log to filter out irrelevant lines, reducing the load on the client
cache the filtered version for later requests.
While I could modify the client to achieve this functionality, such code would not be re-usable for other clients (different languages), and complicates the client logic.
I had a look at whether clojure/ring could do it, and while there is a nice little proxy middleware for it, it doesn't handle streaming content as far as I can tell - the whole 100Mb would have to be downloaded. Also it doesn't include any cache logic yet.
I took a look at whether squid could do it, but I'm not familiar with the technology, and it seems mostly concerned with passing through requests rather than modifying them on the fly.
I'm looking for hints where I might find the correct technology to implement this. I'm mostly language agnostic if learning a new language gets me access to a really simple way to do it.
I believe you should choose a platform that is easier for you to implement your custom business logic on. The following web application frameworks provide easy connectivity with REST APIs, and allow you to create a web application that could work as a REST proxy:
Play framework (Java + Scala)
express + Node.js (Javascript)
Sinatra (Ruby)
I'm more familiar with Play, of which I know it provides utilities for caching you could find useful, and is also extendable by a number of plugins.
If you are familiar with Scala, you could have a also have a look at Finagle. It is a framework build be Twitter's infrastructure team to provide protocol-agnostic connectivity. It might be an overkill for REST to REST proxy, but it provides abstractions you might find useful.
You could also look at some 3rd party services like Apitools, which allows to create a proxy programmatically (in lua). Apirise is a similar service (of which I'm a co-founder) that intends to do provide similar functionalities with a user-friendly UI.
Beeceptor does exactly what you want. It plugs in-between your web-app and original API to route requests.
For your use-case of caching a few responses, you can create a rule. That way it shall not hit the original endpoint.
The requests to original APIs can be mocked, and you can inspect response
You can simulate delays.
(Note: it is a shameless plug, I am the author of Beeceptor and thought it should help you and other developers.)
https://github.com/nodejitsu/node-http-proxy is looking useful - although I don't yet know if it can stream process for transcoding.
Alright, so a better title here may have been "Progressive Enhancement with REST in CakePHP", but at least now I'll know you didn't read the question if your answer just refers to the difference between the two ;)
I'm pretty familiar with REST and how to integrate it with CakePHP, but I'm not 100% on board with how to still maintain a conventionally functioning website. Using Router::mapResources sounds like a great idea, but this creates a problem with maintaining the "gracefully degradation" version of the site, because both POST requests to /resource/ AND GET requests for /resource/add will route to the same action (add). Clearly I'll want this action to return a JSON object if they're using the REST api, but if they're using the degraded version of the site (no JS perhaps), it should be a add form, right?
What's the best way to deal with this. Do you route your REST requests to other action names using Router::resourceMap()? Do you do that crazy hack I saw to have the /api/ prefix part of the resourceMap so you can use api_action functions? Do you have the actions handle both REST and conventional requests via checking isAjax()? If so, how do you ensure that you can rely on the browser to properly support the other two request types?
I've searched around quite a bit but haven't found anything about how to keep conventional requests available in Cake along side REST, so if anyone has any advice or experience, I'd love to hear it!
CakePHP uses extension routing as well, via Router::parseExtension() so;
/test/action will render views/test/action.ctp
/test/action.html also
/test/action.json will render views/test/json/action.ctp
/test/action.xml will render views/test/xml/action.ctp
If all views are designed to handle the same data as set by your controller, you'll be able to show a regular HTML form and handle the posted data the same way as you'd handle the AJAX request.
You'll probably might have to add checks if any data is posted/submitted inside the /add, /edit, /delete actions to prevent items being deleted without a form being posted (haven't tested that though, it might be that cake blocks these urls if mapresources is set for the controller)
REST in CakePHP:
http://book.cakephp.org/2.0/en/development/rest.html
(Extension) Routing
http://book.cakephp.org/2.0/en/development/routing.html#file-extensions
I am in the process of scoping the development of an iPhone app for a client. Among other things, the app will allow users to browse through and place orders on specific (tangible) products.
The client has a website that currently does a similar thing and due to their limited budget and the fact that the website runs on a third-party proprietary platform which they have no control over, we are investigating possible alternatives to building a web service.
On the website, user registration and authentication, as well as order placing is done through POST requests via secure HTTP. The response is always a formatted HTML page which will contain strings indicating whether the request was successful or not, and if there was an error, what the error is etc.
So provided I can replicate the POST requests on the phone, and parse the HTML responses to read the results of each request, do you think this is an acceptable alternative to building a web service to handle this?
Apart from the possibility of pages changing (which we can manage) and the fact that I will probably have to download and parse a relatively large HTML response, are there any other drawbacks to this solution and is there anything else that I might be missing?
Many thanks in advance for your thoughts.
Cheers,
Rog
You could create an intermediary server that will communicate with the client server, and on it expose some REST web services with json (small overhead and easy to handle) responses that will be consumed by the iPhone app.
So, you're going to parse HTML and formulate POSTs off a third-party server, and pray that they don't even so much as rename a form field.
Your question is in two parts:
Do I think that a miracle is an acceptable solution? I don't.
Do I think that aside from the fact a miracle is required, are there any other drawbacks? None that I can think of.
You didn't ask, but this is a terrible course of action. Two suggestions.
I spy an assumption that the providers of the third-party platform aren't interested in enabling third-party applications by providing an API. They have a very good business reason for this, which is that it promotes platform lock-in. Reach out to their support department and have a talk with them.
You have to sell the client on building an intermediary web service. To at least try to mitigate the damage that changes on this third-party platform can do to your app, I recommend that you build and operate a proxy that receives requests from your applications, and proxies them over to the third-party platform. You should build into this client-server protocol a means for returning "we are in maintenance mode, go away" messages to apps, for that inevitable day when the third-party server changes something that breaks your app (they swapped the billing and shipping address pages, for instance) and you have to rush through an update through Apple to deal with it.
The proxy could be written in something more flexible and easy to bash stuff out in, such as PHP, Python, Perl, or Ruby. It could be hosted at Amazon in a micro instance.
p.s. This question is inappropriately tagged objective C.
HTML is the worst because of parsing (1-2secs per page), memory, and changes, but you already know that. Check in advance that ALL the data you need is exposed on the HTML.
If you use an intermediary server you are moving work elsewhere and you have another server to maintain. I would only do that if memory is an issue. Check How To Choose The Best XML Parser for Your iPhone Project for memory/performance/xpath support. libxml2 is a good option, but it depends on your needs. And maybe you'll want to check ASIHTTPRequest features before using the SDK.
I think utilising the web language of JSON would contribute to the diminishing of the parsing time. By building a REST service that, when sent a GET request, returns the correct information for easy sorting, you could then display the output a lot faster than that of parsing straight HTML.
I prefer JSON over XML, but everyone has their personal preference. You should look at a few very good libraries that are built specifically for parsing purposes of both XML and JSON.
For XML I recommend using the inbuilt libxml parser. Albeit, this can sometimes deem very difficult to use. A simple Google search will bring up a heap of results that relate specifically to what parser should be used depending on what task is to be completed.
As for a JSON parser, I recommend SBJSON. I am currently using it one of the biggest projects I have undertaken and it is definitely working perfectly for my use.
If you need a good way to connect to a RESTful web service, you should try LRResty.
Don't go for a parsing solution on the iPhone for 4 reasons:
Server can change their design and break your application (AppStore submition is long) + They can also detect that the request are sent from an application based on user agent which you have to update the application to change it.
Some of the requests might be made thru Javascript so you not only have to parse (X)HTML but also Javascript request (which can be in the form of XMLHttpRequest, but don't have to)
Long term evolution of the mobile market : maybe your client want (or will want) an application for android, Blackberry, Bada OS (Samsung), Symbian (Nokia/ OVIStore), Java Mobile or Windows Phone 7?
Of course network traffic, Memory and CPU needed to parse HTML (look the time it takes to the browser to do it?)
Regarding the traffic, if the application will not have a huge traffic you can home-host your proxy. Or you can find some provider to host it for you. I guess you won't need more than a couple of Megabytes of storage but maybe traffic. For less than 100€/year you can find some with unlimited traffic (like OVH Pro plan or Infomaniak). But if you want to go Java have a look at Google App Engine : you pay only if your traffic is important and if your application generate many CPU Cycles. If not : you don't have to pay. And it's hosted on Google server : reliable.
If the client is open, you could consider the paypal API.
Looking for ideas on how to implement a kiosk style/full screen client that will display messages sent to it, in realtime or push based. Basically, think a 911 dispatch center call board. I was thinking a xmpp bot to display, and making a simple xmpp client that can only send messages to the bot.
This is for a very small emergency disaster agencies war room, and only needs to be able to display simple messages entered in from one of the computers in the building. Is XMPP a good solution for this?
An IM protocol like XMPP is an acceptable solution for this.
XMPP seems to fit the bill, it is mature and has many clients that support it so it would not be necessary to write a client, just set up a regular IM client to send to the "buddy" that is the big board.
Are you looking at the problem backwards? Describe what you want to do a little more THEN seek advice on protocols. It smells like you might be designing around a protocol rather than designing around your requirements.
This should be as simple as a single HTML page, running full screen, using Strophe.js and an XMPP account. Strophe is an easy-to-use XMPP library in Javascript.
Something like the basic.{html,js} example here should be pretty much what you want:
http://code.stanziq.com/cgit/strophe/strophejs/tree/
Sounds like a simple pubsub setup (XMPP will work for this) where the clients are all publishers and the War room is the only subscriber. This eliminates the need for rosters so it keeps the intial configuration pretty simple.
I don't know what language you prefer to use, but it would be rather simple in Smack using the pubsub API and any XMPP server you prefer that supports the pubsub extension. (You will need to build Smack from source though as that particular API is new and not in the release version yet.)
I'm working on an iPhone project that needs to receive data from a PHP script during execution. My first thought was to use sockets/streams on either end to connect the two, but I am having trouble finding information on how to do this from the iPhone side.
Has anyone been down this path that could point me towards some useful resources or offer some advice? The official documentation seems to be geared more towards desktop apps and uses code that doesn't seem to be supported on the iPhone (namely NSHost).
Update: The intended use of this app is to receive log messages from an executing script, so I can't use a simple HTTP request with JSON or XML. Many cases will involve the page being loaded by another client, where the script would relay/push log messages to the iPhone.
Polling is evil. You'll chew through batteries doing that.
You might consider running an HTTP server on the iPhone. Check out this blog post; it has an implementation of an HTTP server in Cocoa as well as example code for using it for two-way communication.
The PHP CURL library (can't link it because the site doesn't trust me yet, just search php.net for it) is a (relatively) simple, easy way to make http requests with a PHP script.
Why don't you just use HTTP? Create an ad-hoc protocol with XML or JSON, use POST for upstream data transmission. I'm a fan of JSON for this sort of thing, personally. The PHP, instead of returning a webpage in HTML for rendering, should just return your data in a JSON format.