I'm learning about the REST architecture style, and there are some things I don't understand when it comes to developing a back-end api for clients.
I've read about various approaches to versioning an http api, which all make sense, but how do you indicate to a client, when he's using an outdated version of your api, that he needs to update his version? Is there a way to do this without physically contacting the client and telling him that he needs to update his version?
I was thinking there might be some way to require the client to indicate his current version and give an appropriate message if it's outdated. Is this standard or even feasible?
Typically, clients update under one of two circumstances. Either they want functionality that's available in a more recent version, or you're dropping support for a prior version.
If you're planning on dropping support for an API version, you should definitely be notifying any customers you can find proactively. If they rely on your API version, and it disappears with no warning, they're going to be former customers.
In the vast majority of cases, clients of your API will not be scanning network traffic looking for a header or other indicator that the API is changing. Asking them to do so is non-standard and almost certainly not feasible.
Also, dropping support for an API version is a major shift. It causes upheaval in all of your clients, forcing them to make a code change in their applications by a date of your choice. It's not something to be done lightly.
Related
I have an internal service that exposes few APIs and few clients using these APIs. I have to make some breaking changes and redesign this service's API.
What are some of the best ways to maintain backward compatibility for these clients while making these changes? (I known it's not ideal but most things in the world aren't, right?)
One solution I can think of is having a config based on which the clients either talk to the old API or the new. This allows me to merge the client code immediately and then enable the new API through the config when the time is right for me.
I want to find out if there are more solutions out there that's in practice when making such breaking changes.
The most common way is to introduce versioning in your API, e.g:
http://api.example.com/ (can default to an older version for backwards compatibility)
http://api.example.com/v1
etc...
See more information and examples here: https://restfulapi.net/versioning/
I'm trying to figure out a scenario, but I can't find any relevant information on the web.
Let's say I am deploying an Android Application (v1.0.0) with the backend (v1.0.0).
At some point, I will make some changes and update the app to v1.0.1 and the backend to v1.0.1 and they will work perfectly.
But how can I also support the previous version of the application (maybe the new server version provides another format of response for one specific request)?
I thought of having separate deployments for every version of the server, but for many updates, that would mean a big resources impact.
Also, forcing the users to update doesn't seem a good option in my opinion.
Essentially you can go multiple ways of doing it. Really depends on your requirements, but usually the reality is a mixture of the things below.
As a rule of thumb, think in data model that will be held compatible. If the contract can not be kept or your realize major changes are needed, introduce a new version of API. If old clients can not be supported, force update. You can also agree on a policy on how long to support each previous versions and then force update, this will make your life much easier and simpler, than maintaining tens of versions of APIs.
Backwards compatible data model
You must think backwards with each release:
Think of incremental modelling with each release cycle. So you can keep things.
When you forget about it and you need to switch behavior based on data:
Happened to me in my trainee years. Simply you have to decode which version it might be, based on the data if you forget to add protocol version. From the beginning you can always have a version field on the data. Moreover, you can also add a set of compatible parser versions.
Include protocol version in data:
{
"data": [ arbitrary data model],
"protocolVersion": "v1"
}
Based on the protocol version, you can decide how to process the data on the server side. You don't need to keep client version in mind, only the protocol's. Since it can be that you release: 1.0.0, 1.0.1, 1.1.0, and you only change protocol in 1.2.0.
But I think the problem is the that as data changes subsequently, so does behavior on server side processing.
Versioned API
Sometimes you see different paths for major versions:
/api/v1/query
/api/v2/query
Used when backwards compatibility is broken or after total reconsideration. Therefore not every version will have an increment.
Use a query parameter with the client version:
/api/query?v=1.0
/api/query?v=1.1
Similar to previous one, just written differently. Although I think this is not the way you want to go down.
Track client releases and used service versions
In reality there are multiple requests and multiple versions of services being used all times per one client version. This makes things even harder to maintain.
All the time, document and track. Version and release management is very important. Know the git hash from which version you built, many time it can get confusing and you just changed only one parameter after release as a quick fix and the day after nothing is compatible anymore.
Map all service versions to the client version at each release, know which commit you really built and use tagging and release management.
Test everything before each release
Have clear requirements for your backwards compatibility. Maybe you only support two older versions, then test with all the two clients, new client with the to be released server. Document everything. And when you meet your criteria for release, go with it.
Summary
Reality is a mixture of solutions. Have clear requirements on the backward compatibility. Therefore you can test before each release. When it is necessary, force update. Keep track of releases, and document each client versions with all the services being used with their versions.
Use switch case at the server for each different version of the app.
I've been reading up on versioning strategies for ReST APIs, and something none of them appear to address is how you manage the underlying codebase.
Let's say we're making a bunch of breaking changes to an API - for example, changing our Customer resource so that it returns separate forename and surname fields instead of a single name field. (For this example, I'll use the URL versioning solution since it's easy to understand the concepts involved, but the question is equally applicable to content negotiation or custom HTTP headers)
We now have an endpoint at http://api.mycompany.com/v1/customers/{id}, and another incompatible endpoint at http://api.mycompany.com/v2/customers/{id}. We are still releasing bugfixes and security updates to the v1 API, but new feature development is now all focusing on v2. How do we write, test and deploy changes to our API server? I can see at least two solutions:
Use a source control branch/tag for the v1 codebase. v1 and v2 are developed, and deployed independently, with revision control merges used as necessary to apply the same bugfix to both versions - similar to how you'd manage codebases for native apps when developing a major new version whilst still supporting the previous version.
Make the codebase itself aware of the API versions, so you end up with a single codebase that includes both the v1 customer representation and the v2 customer representation. Treat versioning as part of your solution architecture instead of a deployment issue - probably using some combination of namespaces and routing to make sure requests are handled by the correct version.
The obvious advantage of the branch model is that it's trivial to delete old API versions - just stop deploying the appropriate branch/tag - but if you're running several versions, you could end up with a really convoluted branch structure and deployment pipeline. The "unified codebase" model avoids this problem, but (I think?) would make it much harder to remove deprecated resources and endpoints from the codebase when they're no longer required. I know this is probably subjective since there's unlikely to be a simple correct answer, but I'm curious to understand how organisations who maintain complex APIs across multiple versions are solving this problem.
I've used both of the strategies you mention. Of those two, I favor the second approach, being simpler, in use cases that support it. That is, if the versioning needs are simple, then go with a simpler software design:
A low number of changes, low complexity changes, or low frequency change schedule
Changes that are largely orthogonal to the rest of the codebase: the public API can exist peacefully with the rest of the stack without requiring "excessive" (for whatever definition of of that term you choose to adopt) branching in code
I did not find it overly difficult to remove deprecated versions using this model:
Good test coverage meant that ripping out a retired API and the associated backing code ensured no (well, minimal) regressions
Good naming strategy (API-versioned package names, or somewhat uglier, API versions in method names) made it easy to locate the relevant code
Cross-cutting concerns are harder; modifications to core backend systems to support multiple APIs have to be very carefully weighed. At some point, the cost of versioning backend (See comment on "excessive" above) outweighs the benefit of a single codebase.
The first approach is certainly simpler from the standpoint of reducing conflict between co-existing versions, but the overhead of maintaining separate systems tended to outweigh the benefit of reducing version conflict. That said, it was dead simple to stand up a new public API stack and start iterating on a separate API branch. Of course, generational loss set in almost immediately, and the branches turned into a mess of merges, merge conflict resolutions, and other such fun.
A third approach is at the architectural layer: adopt a variant of the Facade pattern, and abstract your APIs into public facing, versioned layers that talks to the appropriate Facade instance, which in turn talks to the backend via its own set of APIs. Your Facade (I used an Adapter in my previous project) becomes its own package, self-contained and testable, and allows you to migrate frontend APIs independently of the backend, and of each other.
This will work if your API versions tend to expose the same kinds of resources, but with different structural representations, as in your fullname/forename/surname example. It gets slightly harder if they start relying on different backend computations, as in, "My backend service has returned incorrectly calculated compound interest that has been exposed in public API v1. Our customers have already patched this incorrect behavior. Therefore, I cannot update that computation in the backend and have it apply until v2. Therefore we now need to fork our interest calculation code." Luckily, those tend to be infrequent: practically speaking, consumers of RESTful APIs favor accurate resource representations over bug-for-bug backwards compatibility, even amongst non-breaking changes on a theoretically idempotent GETted resource.
I'll be interested to hear your eventual decision.
For me the second approach is better. I have use it for the SOAP web services and plan to use it for REST also.
As you write, the codebase should be version aware, but a compatibility layer can be used as separate layer. In your example, the codebase can produce resource representation (JSON or XML) with first and last name, but the compatibility layer will change it to have only name instead.
The codebase should implement only the latest version, lets say v3. The compatibility layer should convert the requests and response between the newest version v3 and the supported versions e.g v1 and v2.
The compatibility layer can have a separate adapters for each supported version which can be connected as chain.
For example:
Client v1 request: v1 adapt to v2 ---> v2 adapt to v3 ----> codebase
Client v2 request: v1 adapt to v2 (skip) ---> v2 adapt to v3 ----> codebase
For the response the adapters function simply in the opposite direction. If you are using Java EE, you can you the servlet filter chain as adapter chain for example.
Removing one version is easy, delete the corresponding adapter and the test code.
Branching seems much better for me, and i used this approach in my case.
Yes as you already mentioned - backporting bug fixes will require some effort, but at the same time supporting multiple versions under one source base (with routing and all other stuff) will require you if not less, but at least same effort, making system more complicated and monstrous with different branches of logic inside (at some point of versioning you definetely will come to huge case() pointing to version modules having code duplicated, or having even worse if(version == 2) then...) .
Also dont forget that for regression purposes you still have to keep tests branched.
Regarding versioning policy: i would keep as max -2 versions from current, deprecating support for old ones - that would give some motivation for users to move.
Usually, introduction of a major version of API leading you in a situation of having to maintain multiple versions is an event which does not (or should not) occur very frequently. However, it cannot be avoided completely. I think it is overall a safe assumption that a major version, once introduced, would stay latest version for relatively long period of time. Based on this, I would prefer to achieve simplicity in the code at the expense of duplication as it gives me better confidence of not breaking previous version when I introduce changes in latest one.
How do services like github,twilio,algolia,stormpath maintain rest api's along with SDK's for different languages? Do they generate such code using tools like enunciate or are they maintaining the client code themselves? I guess for github ,they are open sourced client libraries. My questions are:
How to sync between rest api changes and corresponding SDK changes.
What are the best practices for versioning of rest apis,as well as their sdk's ?What are the common pitfalls one must be aware of?
At Algolia we’ve developed a dozen of API clients on top of our REST API. Honestly, I must say we suffered a lot to create all of them /o\ I hope the following bullet points will help:
Why did we create our own API clients instead of just using libraries/tools to generate them (which is pretty common for REST API clients)?
be able to reach a 99.99% SLA, we decided to implement a “retry-strategy” in our API clients -> the Algolia index is always replicated on 3 machines -> if one goes down the API client will retry on the other ones -> this cannot be handled by generic libraries
to reach optimal performances, we wanted to be sure we control the way the HTTP keep alive works -> most of generic libraries doesn’t handle that as well
we wanted to force HTTPS as soon as possible
How did we proceed?
at the beginning, we were not super fluent in all those languages; so we started to look at other API clients implemented in each language to understand best practices
we got the help from 1 guy for Node.js and 1 for python
but it was really not OK until we decided to move all of them to Travis.CI + plug code coverage to reach 80-95% code coverage + automated tests -> obviously, we spot a lot of bugs :)
as soon as we release a new feature, we need to update all our API clients -> pretty painful…
to ease the README generation, we’ve developed a small tool generating the README for all languages. It’s super Algolia-specific but you can have a look at https://github.com/algolia/algoliasearch-client-readme-generator -> pretty painful to bootstrap as well :)
Things we learned:
maintaining all of them ourself make them act exactly the same, whatever the language -> super appreciable from a customer POV
if your community is building API clients, that’s cool; but they may not go so deep in the tests -> we’re testing ALL features in ALL API clients -> quality
we would do the same again if we needed to
I'm developing an application that consists of a 'fat' javascript client backed by a JSON/REST server for data access. The data is stored in mongodb but the system should be flexible enough to switch to a relational backend if required.
I started out with pintura as the server side framework but recently ran into some problems (specifically with the perstore/filestore). I noticed that one problem has even been reported (including a fix) over a month ago, but there has been no reply to it and the issue is still present.
There seems to be relatively low activity in this project so I was wondering if many people are actually using it, and how stable the framework is.
Does anybody here have experience with this framework or know of an alternative framework that has similar capabilities?
I agree the project and the website/blog does not seem to be active, although the perstore repository does have recent activity. I'd contact the author there since your problem seems more related to that.
A search for REST on http://search.npmjs.org/ will show quite a few results, although I cannot recommend any from experience.