I'm looking for a convention on how to serialize my data when I have a (long) list of items that I want to POST to the server.
For example, if I have a resource /users and I wanted to POST a new one to it, I'd http-encode the fields for the new user and put it in the request body like this: name=foo&age=20
But if I have a list of users, like this [{ name: 'foo', age: 20 }, { name: 'bar', age: 10 }], is there a conventional way of POSTing this?
I'm thinking name[0]=foo&age[0]=20&name[1]=bar&age[1]=10 but I can't find anything to back it up. What do web servers usually accept/expect?
Quick question which may change my answer: Are you POSTing directly from an HTML form or are you expecting something more sophisticated (e.g. javascript processsing, or not even a web-based client)
If you have a sophisticated enough client, you could just construct a JSON string and POST with a content type of application/json. Then whatever resource is processing the POST could use any number of json libraries to read the posted string and process as is.
Further Rambling:
What framework/languages are you using to construct your REST service? Do they have built-in functionality/conventions to help you?
For example if you're using JAX-RS to build your service, there is a built in annotation #FormParam which can be used to process posted forms... for example: if you posted the following with a content type of application/x-www-form-urlencoded: name=foo&age=20&name=bar&age=10
You could retrieve parallel lists on the service side via:
#POST
#Consumes("application/x-www-form-urlencoded")
public void createUsers(#FormParam("name") List<String> name, #FormParam("age") List<String> age) {
// Store your users
}
But you would then have to deal with the question of what if one list is shorter/longer than the other, how do you resolve that? What happens if a new field is required or optional to create a list of users? (But as I mentioned initially, a JSON array of JSON objects would solve that issue... there are a number of libraries out there that support automagic JSON deserialization in JAX-RS or there is also the option of creating your own MessageBodyReader.
(Disclaimer on the next section: I don't know rails, my experience is more in the Java services world... I'm basing this on this guide). It looks like Rails has a convention of name[]=foo&name[]=bar to process posted data into arrays automagically, and a similar convention to populate structure like user[name]=foo&user[age]=20... Perhaps if you are on rails there is some way to use/abuse both of these features to get the desired result?
Other REST frameworks and languages may have their own conventions and functionality :)
Rails serializes forms on a format not unlike what you suggest. If you have a nested model it encodes it like this:
name=theo&company[name]=acme
(the equivalent JSON would be {"name": "theo", "company": {"name": "acme"}})
I can't say that I've seen a Rails application sending arrays, but there's no reason why it wouldn't work (worst case you would end up with a hash with string keys).
PHP has another convention, if you want to send an array you do
names[]=alice&names[]=bob&names[]=steve
But I don't know how you do nested objects that way.
The HTTP spec, or if it's the URI spec, not sure which atm, actually specifies that if you pass the same argument multiple times you get array of values (instead of the last-wins behaviour of most application frameworks). You can see this in the API docs for Jetty, for example: http://api.dpml.net/org/mortbay/jetty/6.1.5/org/mortbay/jetty/Request.html#getParameterValues(java.lang.String)
However, most of this applies to GET requests, not necessarily POST (but perhaps application/x-url-encoded should adhere to the same standards as GET).
In short, I don't think there is a standard for doing this, POST bodies are a bit of a wild west territory. I think, however, that either you should go with JSON, because it's made to describe structures, and application/x-url-encoded is not, or you should try to represent the structure of your data better, something like:
users[0][name]=foo&users[0][age]=20&users[1][name]=bar&users[1][age]=10
That has some kind of chance of actually being interpretable by a Rails app out of the box, for example.
Related
Let's say I want to make a RESTful interface, and I want to work with foos based upon their IDs. Nothing new here:
GET /api/foo1 returns a representation (e.g. using JSON) of foo1.
DELETE /api/foo1 deletes foo1.
etc.
Now let me tell you that a "foo" is a collection type thing. So I want to be able to add a "bar" to a "foo":
PUT /api/foo1/bar3 adds bar3 to foo1.
GET /api/foo1/bar3 returns a representation of foo1.
DELETE /api/foo1/bar3 removes bar3 from foo1.
DELETE /api/foo1 deletes foo1 altogether.
Now the question remains: what does GET /api/foo1 do? Does it it simply return a representation of foo1 as I originally assumed in this question? Or does it return a list of bars? Or does it return a representation of foo1 that is both a description of foo1 as well as includes a list of all contained bars?
Or should GET /api/foo1 merely return a representation of foo1 as I assumed at the beginning, and require a PROPFIND request to list the bars inside foo1 (the approach taken by WebDAV)? But then in order to be consistent, wouldn't I have to change all my other list-type functionality to PROPFIND, directly contradicting all those thousands of RESTful tutorials that say to use GET /api/foo1 to list the contents?
After some pondering, I think the best conceptual explanation from a RESTful perspective is that usually the "thing" is not the same thing as its "collection". So while in the WebDAV world a directory/ might be the same thing that holds its files, in the RESTful world I might have a separate directory/files/ subpath for the contained files. That way I can manipulate directories separately from the files that hold.
Consider a RESTful API for a farm which contains barns. The endpoint farm/api/barns/ might return a list of barns, one of which would be farm/api/barns/bigredbarn. Naively I would think that retrieving farm/api/barns/bigredbarn/ would provide me a list of animals in the barn, which is what prompted this question.
But really the animals in the barn is only one aspect of the Big Red Barn. It might contain vehicles and hay:
farm/api/barns/bigredbarn/animals/
farm/api/barns/bigredbarn/vehicles/
farm/api/barns/bigredbarn/haybales/
With this approach the dilemma I faced does not arise.
The semantics of webdav has never really been reconciled with the idioms of RESTful interfaces.
In theory, GET should retrieve a representation of a state of a resource and PROPFIND should be used to retrieve the members of a collection.
So you should do this:
GET /api/foo1/ - returns state of foo1 only
PROPFIND /api/foo1/ - returns the members of foo1
Most front end devs would freak out if you told them to use PROPFIND, although its completely supported in browser js implementations.
Personally i use a webdav/json gateway, where requests are made using RESTful idioms, but routed to my webdav implementation
For example i would do this:
GET /api/foo1/_PROPFIND?fields=name,fooProp1,fooProp2
And that would return
[
{ name : "bar1", fooProp1: "..", fooProp2 : ".."},
{ name : "bar2", fooProp1: "..", fooProp2 : ".."}
]
One advantage of this approach is that client's get to control the json properties returned. This is good because a rich API will have a lot of properties, but in most situations clients dont need all of them.
The routes and their operations in RESTfull API are completely designed by the developers. It's the developer who decides what to return while requesting a specific route say, GET /api/foo1.
And the developer should design every route including /api/foo1/bar. There is no specific rule on what a particular route should do. If your API is an open-source project make a clean and clear documentation of every route.
Don't waste your time thinking about the old school strategies.
I'm designing a RESTful api for a large collection of reporting data, I would like to pass a complex set of parameters like in the codeblock below. I'm debating between using POST and GET for this endpoint. Team member seem to favor GET but I am not sure the best way of passing this amount of data as GET parameters, best idea so far is to have one GET parameters called something like jsonparams that would have all of the below json encoded
{
"filters":
[
{
"field": "metric-name",
"gt": (float/int),
"lt": (float/int)
},
{
"field": "metric-name-2",
"gt": (float/int),
"lt": (float/int)
}
],
"sort":
[
{
"field": "metic-name",
"order": "ASC"/"DESC"
},
{
"field": "metic-name-2",
"order": "ASC"/"DESC"
}
]
"limit": 100,
"offset": 0
}
POST is the method to use for any operation that isn't standardized by HTTP. Retrieval is standardized by the GET method, so using POST to retrieve information that corresponds to a potential resource is never RESTful. In theory, you should use GET, regardless of how convoluted your URI turns out to be.
However, since you're performing a query for which there isn't a single resource you could perform a GET to, it seems fine to use POST, as long as you're aware of the disadvantages and your documentation is clear about it. Frankly, I think using a POST is much clearer than encoding that payload as a JSON + base64 and sending it as a querystring merely for purism.
The real issue with using POST is when people use it in such a way that it avoids or prevents using a real URI. That doesn't seem to be the issue in your case, since you have a valid URI for the collection, but the semantics of your query are too complex to be expressed easily.
If you decide to go with GET, there's a catch. While the HTTP specs establish no limit for URIs, most implementations do, and you might hit that limit if you need to feed all those parameters as a querystring. In that case, it's RESTful to circumvent limitations of the underlying implementation, as long as that's decoupled from your application. A convention for doing what you want would be to use the POST method with the payload you described above, and the X-HTTP-Method-Override: GET header.
If you're adding the data to a resource or creating a resource use POST. GET is to get a already existing resource, not change the state of the resources.
Update: While POST request are fine to update a resource, if the action is idempotent (meaning will not result in creating new resource and every time you issue the request with the same parameters and data you can guarantee to the same resulting resource), then it's recommended to use a PUT. If it's not idempotent, but you're not replacing the entire resource, use PATCH if only updating part of the resource.
If the argument to go with a crazy serialized GET parameter based request for some kind of percived simplicity, you're not going to be adhearing to REST.
Now, if you're retrieving resources only (no creation), use GET. While I prefer human typable parameters, it's not required. If your situation is 100% retrieval you could encode the entire set into into a giant encoded param string, but I'd suggest at least splitting it out a bit for improved sanity by doing something like:
/resource?filters=urlencoded_filter_array&sort=urlencoded_sort_array&offset=0&count=100
Or you could go more explicit like:
/resource?filter1=urlencoded_filter_json&filter2=urlencoded_filter_json .... sort2=urlencoded_sort_json&offset=0&count=100
or finally (my favorite) a completely explicit broken out set of params
/resource?filter1_field=bah&filter1_gt=1.0&filter1_lt<2&filter2_field=boo&filter2_lt...
I like the final one because there's no encodeing/decoding of json and then url encoding the entire json string. This format is easy to decipher in access logs and trouble shoot. It's also very cacheable, even if the parameter order gets changed, some proxy caches can still work with this, whereas encoding some of the filters in a json object if they get moved around they look like entirely different values as far as proxies would be concerned. For me it's the most REST friendly (if that makes any sense), even though the first 2 examples are fine REST GET requests.
The added work to parse the parameter name isn't really that much fuss. A simple method could convert your json into the parameter string and another simple one could re-hydrate the json object from the explict filter1_xyz format.
I'm wondering where to put the edit, and new keywords by rest application. I'll use express-resource in a project, and the default settings are these:
GET /forums -> index
GET /forums/new -> new
POST /forums -> create
GET /forums/:forum -> show
GET /forums/:forum/edit -> edit
PUT /forums/:forum -> update
DELETE /forums/:forum -> destroy
There is a problem with this solution: there is not a real resource behind the new and edit. I mean the URL-s refer to resources, and after any slash is a sub-resource.
For example:http://my.example.com/users/1 represents:
var firstUser = {
name: "John Smith",
birthDate: new Date(1952,10,4),
hobbies: ["skiing", "football"],
...
}
And http://my.example.com/users/1/birthDate represents:
firstUser.birthDate
But by http://my.example.com/users/1/edit there is no such property:
firstUser.edit
So something is wrong with this conception.
Where is the real place of these keywords? In queryString or in headers?
From the perspective of a REST API these do not exist anywhere as they are not related to the representation of resources. They are actions upon resources and therefore expressed by the HTTP methods used. They would not bee needed if you were to create an external client that uses the API.
There is likely a need to provide some support for this type of functionality so that something like a UI could be presented, but that is the concern of the particular application and not the API itself. At that point it becomes discretionary but I would certainly avoid using headers as that would be pretty outside of conventional practice. But by headers it appears that you actually meant URI path. Out of those 2 I would say the path is the better option since it clearly defines any type of UI as a distinct resource and would keep it apart from the clean API, while using a query string on a part of the API would more tightly (mistakenly) associate it with the underlying resource.
Some update after a year or so:
edit and new should be hyperlinks
these links are just the representations of possible operation calls
by following them it is possible to manipulate the resource they belong by sending representations of the intended resource state, and/or by calling the proper methods
these terms are no resources, they don't have their own URL (resource identifier)
Thanks the advices Matt Whipple!
I think the best way to learn REST is reading the Fielding dissertation. There are many tutorials out there, but the authors of most of these does not really understand REST.
There is a lot of discussions out there about REST URI design, but none replied precisely to my question.
Let's say I have some lists that contains tasks and / or other lists (list = node and task = leaf).
We can have something like
/lists/{id_list1}/lists/{id_list2}/lists/{id_list3}/tasks/{id_task1}
or maybe a shorter version :
/lists/{id_list1}/{id_list2}/{id_list3}/{id_task1}
But what about really deep trees ?
I was also thinking about the parent/child relation which could be translated like
/lists/{id_list_parent}/{id_list_or_task_child}
but I feel like something's missing...
What would be a smart design for REST URI trees ?
EDIT
Sorry for my late answer !
So I think I was mixing 2 needs : API URIs and browser URIs.
Here's how I see things :
On the API side, I'll have only those URIs for both write and read methods (the '/id' at the end is not required, for create for instance) :
/lists/id
/tasks/id
On my browser however, I'll have something like this :
/lists/id/lists/id/tasks/
Only the first part (/lists/id) will be interpreted by the server, the second part (lists/id/tasks/) will be used by my client side javascript to find the tasks of the sublist of the list received from the server.
What do you think of this approach, does something feel wrong in it ?
Is there any need to represent the tree via the URIs? When they can simply be references to the nodes themselves and the relationships are modeled via links in the hypermedia type.
Other than that, I think something like:
/lists/{nodeId}/children
and it can return a list of the direct children to the parent.
You can also do something like:
/lists/{nodeId}/children?depth=3
and return the next three levels of the tree in your result.
/lists/{nodeId}/children?depth=infinite
can return the entire branch from that node.
Using the URI to enforce this constraint doesn't seem correct. I would accept a PUT request that has some kind of payload (JSON, XML, etc.).
PUT /tasklist
lists: [{
id: 1234,
id: 12345,
tasks: [{
id: foo,
id: bar,
id: baz
}]
}]
The URL seems to indicate that you are trying to do too much here (IMHO). Typically, you would be interacting with a single resource, while your use case implies that you are operating on several contextually tiered resources at one time.
Im developeing a RESTful Service in which Processes can be executed and proivde a resulting calculation. For this i have modeled the process itself as a Resource (Example: /processes/translate). I want to execute the process by sending a GET request with appended Input Parameter as Query Parameter (Example: /processes/translate?input1=xxxx&input2=xxxxx).
Each process has different Input Parameter which are defined during the process creation in the backend. My Question is how should i document or describe which inputs are needed to execute a process in machine readable form. For Example in XML.
Until now ive integrated atom:link elements in the Representation. i thought that maybe including XFORM could be a soluttion?
Best Regards
Andre
I would not model this with a GET. While it's the easier solution, it's also (IMO) the least RESTful. I would have clients POST a document describing what they want you to translate and your service sends them back a URI where their answer can be found (some translations might take a while).
Example (ommiting a lot of HTTP headers/context)
POST /processes/translate
Content-Type: application/xml
...
<translation-request>
<input1 type="type1">....</input1>
<input2 type="type5">....</input2>
</translation-request>
Response:
200 OK
Content-Location: /processes/translate/jobs/1234
....
That's always an interesting question. We have a project called RESTx (http://restx.org), with which you can create RESTful web services very easily. You can write custom component code in either Java or Python and then create RESTful resources by sending parameter sets to the server, which are then stored. Each parameter set gets its own URI, though, so you can always just run the code with those parameters by accessing the new parameter set's URI.
Importantly, the entire RESTful API, is automatically created. RESTx examines the component code and then assembles the API description. We decided to describe parameters in a way that is human as well as machine readable. You can see examples of what that looks like in a browser or in plain JSON.
I'm the lead developer on that, so please feel free to contact me about any questions you might have.