How do you do continous deployment in an AJAX application with lots of client side interaction and local data? - deployment

We have an app that is written in PHP. The front end uses javascript heavily. Generally, for normal applications that require page reloads, continuous deployment is not really an issue, because:
The app can be deployed with build tags: myapp-4-3-2013-b1, myapp-4-3-2013-b2, etc.
When the user loads a page (we are using the front controller pattern), we can inject the buildtag and the files are loaded from the app directory with the correct build tag.
We do not need to keep the older builds around for too long because as the older requests finish, they will move to the newer build tags.
The issue with database and user data being incompatible is not very high as we move people to the newer builds after their requests finish (more on this later).
Now, the problem with our app is that it uses AJAX heavily for smooth page loads. In addition, because there is no page refresh at all when people navigate through the application, people can keep their unsaved data in a their current browser session and revisit it as long as the browser has not been refreshed.
This leads to bigger problems if we want to achieve continuous deployment:
We can keep the user's buildtag in their session (set when they make the first request) and only switch to newer buildtags after the logout and login again. This is obviously bad, because if things like the database schema changes or the format of files to be written to disk changes in a newer build, there is no way to reconcile this.
We force all new requests to a newer build tag, but there is a possibility we change client side javascript and will break a lot of things if we force everyone with a session onto the new build tags immediately.
Obviously, the above won't occur with every build we push and hopefully will not happen a lot, but we want to build a fool proof process so that every build which passes our tests can be deployed. At the same time, we want to make sure that every deployed and test passing build does not inadvertently break in clients with running sessions cause a whole bunch of problems.
I have done some investigation and what google does (at least in google groups) is that they push a message out to the clients to refresh the application (browser window). However, in their case, all unsaved client side data (like unsaved message, etc) would be lost.
Given that applications that uses AJAX and local data are very common these days, what are some more intelligent ways of handling this that will provide minimal disruption to users/clients?

Let me preface this that I haven't ever thought of continuous deployment before reading your post, but it does sound like quite a good idea! I've got a few examples where this would be nice.
My thoughts on solving your problem though would be to go for your first suggestion (which is cleaner), and get around the database schema changes like this:
Implement an API service layer in your application that handles the database or file access, which is outside of your build tag environment. For example, you'd have myapp-4-3-2013-b1, and db-services folders.
db-services would provide any interaction with the database with a series of versioned services. For example, registerNewUser2() or processOrder3().
When you needed to change the database schema, you'd provide a new version of that service and upgrade your build tag environment to look at the new version. You'd also provide a legacy service that handles the old schema to new schema upgrade.
For example, say you registered new users like this:
registerNewUser2(username, password, fullname) {
writeToDB(username, password, fullname);
}
And you needed to update the schema to add the user's date of birth:
registerNewUser3(username, password, fullname, dateofbirth) {
writeToDB(username, password, fullname, dateofbirth);
}
registerNewUser2(username, password, fullname) {
registerNewUser3(username, password, fullname, NULL);
}
The new build tag will be changed to call registerNewUser3(), while the previous build tag is still using registerNewUser2().
So the old build tag will continue to work, just that any new users registered will have a NULL date of birth. When an updated build tag is used, the date of birth is written to the database correctly.
You would need to update db-services immediately, as soon as you roll out the new build tag - or even before you roll out the build tag I guess.
Once you're sure that everyone is using the new version, you can just delete registerNewUser2() from the next version of db-services.
This will be quite complicated to make sure that you are correctly handling the conversion between old API and new API calls, but might be feasible if you're already handling continuous deployment.

Related

JAX-WS service and client versioning

My group is getting into web services for the first time. What is the usual way to control versioning issues with web services and clients? We're generating our services and clients using in IBM's RAD using the wizards it presents. The services and clients work well enough, but we're wondering how managing these will become a task going forward.
-If a service changes, and it's interface doesn't change I presume older clients would never know the difference and would work fine.
-If a service changes to add a new property to a parameter, would older clients work with it if they didn't need to set that value?
-How do people handle version control of web services as they grow in number and the number of apps using their clients grows? What is best practice on this sort of thing?
There are two types of versioning that we use. One that is visible to clients (public) and one that isn't (private). This comes from (what you already touched upon) whether clients are affected or not.
If clients are affect by, for example change in XSD Schema definitions or functionality of the WS changes in such way that clients must also modify their end, we change the public version. We increment the version number in WS context root, which means that it will have different URL than the previous version. Also make sure that the code archive, in our case they are war, have also the public version, as not to overwrite the previous deployment.
For example our WS called foo is in public version 2. Its URL is http://ourserver:8000/foo_2 and war file is called foo_2. We modified our XSD Schema, so the clients must react to the change. We update the version and now URL is http://ourserver:8000/foo_3 and war file is called foo_3. The previous version still exists, while clients can slowly transition to the new version.
If the change does not force any action from clients, then we call this private versioning. This usually shows up, in combination with public version, as a part of a project name. Using previous example, we have a WS foo, with private version 5 and public 2. Our project for this service is called WS_foo_2_5. We now change the order in which we store incoming data. This does not effect the clients, so we change the private version. In effect we have project WS_foo_2_6. We create a code archive from it called foo_2 and deploy it with URL set to http://ourserver:8000/foo_2. This way we modified the version, without changing anything from clients POV.

Beginner GXT issues

We have a working web application, which has been developed with ExtJS for client side, and Struts, Spring, Hibernate for server side. now, we are considering to migrate to GXT (or may be GWT itself). The thing is I'm very new to GWT/GXT. and we are trying to decide whether we go down this road or not.
1) Until now, we have 2 domains for our web-app. one is that the application (Struts+...) have been deployed to, and the other is mainly a cookie-less custom CDN. The transfer between client and server is mostly XHR requests, sending/receiving JSON and/or JSONP. But with the new approach ahead of us, I began to understand that we are supposed to have only ONE domain, for the whole GXT application. Is it correct or I forgot to consider something here?
and if not, Is it possible that we deployed just part of the application (i.e. com.ourcompany.webapp.gxt.server.*) to the main server, and the contents that have been compiled and generated by the GWT compiler to the other CDN-like domain?
2) The other big issue we are facing is that the current application is consists of mostly 3 huge modules. One is responsible for "SignIn", the other is for "Webtop", and the third one is "Modules which each users has access to". The latter has been generated on the server due to "access rights" of each users, and obviously could be different from one user to the other.
The only thing I could find on this matter, which might be related is Code Splitting. Although I'm not totally sure if this would be the right solution for this.
We want that the application, on Start Up, checks whether user has been logged in or not. if not, loads the SignIn sets of javascript files (i.e webapp.signin.nocache.js), then after user has entered the correct username/password, unloads this signin file and loads webtop.nocache.js AND modules.nocache.js.
I would be really appreciated if you could help me out.
1) If your GWT app is loaded from a different domain than you have to face the same origin policy. You can not do a xhr to a different domain. You could use the ScriptTagProxy to get around this. But it does not feel very netural.
2) You can use CodeSplitting in order to automatically load a particular part of your application dynamically. All you have to do is to warp your splitt point into an async call.
A detailed compile report gives you a pretty good overview how well code splitting is working.
But CodeSplitting does not unload already loaded code. If its really importend to do so you have to redirect the user to another url in order to load the appropriate user depended module.
Once Javascript code has been loaded and executed its impossible to remove the code from the browsers memory.
Grettings,
Peter

Elegant way to detect if GWT application is out of date and auto-refresh browser?

After many support cases we are realizing the biggest problem we have with our GWT-based application is that users are leaving it open for weeks at a time. This means when we do a hotfix every week or two the RPC stubs are out of sync and cause silent exceptions to be thrown making the site look "broken". Does anybody know of a way to auto-detect and avoid this issue? A few ideas I have had are...
On catching an RPC mismatch exception refresh the browser.
When loading the host page inject the version number in source control the build came from, have a status checker/timer that check that the number did not change. When it does reload.
Reload on an arbitrary timer (perhaps twice daily).
Any ideas?
I'd like to present a fourth option.
Create an RPC Proxy and UI Object proxy that all UI requests and RPC requests are routed through. This way whenever this proxy detects that something is out of date it can dynamically load the widget or change it's expected RPC models.
This is pretty how Vaadin does things and it works great. Vaadin is a UI toolkit built on GWT in case you're not aware. We have several long running production applications using this over the last couple of years and we have made some tweaks in their UI Def language (UIDL) to add version mismatch.
This diagram is a good representation of what they do and if you don't want to build something like this yourself I'd of course recommend moving to Vaadin.
Implement security that logs users out after an hour of idle time. Assuming your releases are overnight or on weekends, the users are logging in after the release. No need to refresh the app.
this is viable especially if your site eventually needs users to login.

Persistence on an online click once application

Is there a way to persist an string from an online click once application. I saw something about isolated file storage as answers to other questions. But none of them specify if it works also for online apps (I really don't think so).
I think that something like a cookie will work. Is there something like that available?
The application must run only online (is triggered with some parameters), but for each user it needs to save a file with specific information asked to him. Once the app runs for the very first time it must not ask for that info to the user.
Thanks.
You can store the information in LocalApplicationData. Just create a directory with either your application name or your company name, stick the string in a file, and read it from there. This article shows you how to persist this data, and not have it impacted by ClickOnce updates. It will work even though your application is online-only. (Online-only C/O apps are still installed, it just means it always runs it from the URL, and requires the user to be connected in order to install the app.)

How to apply database updates after deployment?

i know this is an often asked question on these boards. And usually the question has been about how to manage the changes being made to the database before you even get around to deploy them.Mostly the answer has been to script the database and save it under sourcecontrol and then any additional updates are saved as scripts under version control too.(ex. Tool to upgrade SQL Express database after deployment)
my question is when is it best to apply the database updates , in the installer or when the new version first runs and connects to the database? note this is a WinApp that is deployed to customers each have their own databases.
One thing to add to the script: Back up the database (or at least the tables you're changing!) before applying the changes.
As a user I think I'd prefer it happens during the install, and going a little further that the installer can roll itself back in the event of a failure. My thinking here is that if I am installing an update, I'd like to know when the update is done that it actually is done and has succeeded. I don't want a message coming up the next time I run it informing me that something failed and I've potentially lost all my data. I would assume that a system admin would probably also appreciate install time feedback (of course, that doesn't matter if your web app isn't something that will be installed on a network). Also, as ראובן said, backing up the database would be a nice convenience.
You haven't said much about the architecture of the application, but since an installer is involved I assume it's a client/server application.
If you have a server installer, that's where you want to put it, since the database structure is only going to change once. Since the client installers are going to need to know about the change, it would be nice to have a way to detect the database version change, and for the old client to be able to download the client update from the server automatically and apply it.
If you only have a client installer, I still think it's better to put it there (maybe as a custom action that fires off the executable for updating the database). But it really isn't going to matter, because conceptually one installer or first-time user of the new version is going to have to fire off the changes to the database anyway. The database changes are going to put structural locks on the database so, in practical terms, everyone is going to have to be kicked off the system at that time for the database update to be applied.
Of course, this is all BS if it's not client-server.