We currently have 4 content types that might be included in a delivery. In about 8-12 months we will probably have another 2-4 content types. I'm working on a public REST api now and wondering whether I can write the api in such a way that the future additions won't require a version bump?
Currently we are thinking a delivery would return a json result kind of like this:
{
"dateDelivered": "2016-01-01",
"clientId": "000001",
"contentCounts" : {
"total" : 100,
"articles": 75,
"slideshows": 25
... // other content types as we add them
}
"content" : {
"articles" : "http://api.example.com/v0/deliveries/1234/content/articles",
"slideshows" : "http://api.example.com/v0/deliveries/1234/content/slideshows",
... // other content types as we add them
}
}
That defines contentCounts and content as objects with an optional property for each available content type. I suppose I could define that as an array of objects for each content type, but I don't see how that would really change anything.
Is there any reason it be a breaking change if in the future the result object looked more like this:
{
"dateDelivered": "2016-01-01",
"clientId": "000001",
"contentCounts" : {
"total" : 150,
"articles": 75,
"slideshows": 25,
"events": 25,
"videos": 25
}
"content" : {
"articles" : "http://api.example.com/v0/deliveries/1234/content/articles",
"slideshows" : "http://api.example.com/v0/deliveries/1234/content/slideshows",
"events" : "http://api.example.com/v0/deliveries/1234/content/events",
"videos" : "http://api.example.com/v0/deliveries/1234/content/videos"
}
}
Breaking change is a relative notion.
It is breaking the client code that does not account for those changes.
In your case, if a client of your REST API has hardcoded the content types, then he will need to change his code to account for new content types.
In that sense, his code is broken because it does not handle all of it.
In another sense, his code is not broken because as long as you do not remove content types, his code will continue to work for the content types it supports.
If the client code is smart enough to iterate through properties and be flexible about the changes, it is fine.
In any case, if you plan on changing the model, you should mention it.
Whether it is adding, removing or renaming those content types, if the client knows it, he can write a client that will successfully use your REST API. In that case, NO, it would not be a breaking change because is has a dynamic structure (content types can vary) but in a structured way.
Related
Dolibarr has a module for restful APIs.
The API explorer seems to show all the CRUD tasks for each module like orders, stock and customer.
But to CREATE a record, the sample VALUE for the POST method shows as:
{
"request_data": [
"string"
]
}
What are the specific field attributes that should go in here?
Where can I look up the field requirements?
You should take a look at the attributes of the Commande class:
https://github.com/Dolibarr/dolibarr/blob/develop/htdocs/commande/class/commande.class.php
The object should be something like this :
{
"date_commande" : "0000-00-00 00:00:00",
"date_livraison" : "0000-00-00 00:00:00",
"attribute3": "and so on"
}
When you need a parameter like
{ "request_data": [ "string" ] } for a POST API, all you have to do is to call the similar API to get a record (so the same API with the GET method). The result can be cut and paste to be used to create a new record (just change the id and ref in the answer retreived by the GET).
This is the case: A webshop in which I want to configure which items should be listed in the sjop based on a set of parameters.
I want this to be configurable, because that allows me to experiment with different parameters also change their values easily.
I have a Product collection that I want to query based on multiple parameters.
A couple of these are found here:
within product:
"delivery" : {
"maximum_delivery_days" : 30,
"average_delivery_days" : 10,
"source" : 1,
"filling_rate" : 85,
"stock" : 0
}
but also other parameters exist.
An example of such query to decide whether or not to include a product could be:
"$or" : [
{
"delivery.stock" : 1
},
{
"$or" : [
{
"$and" : [
{
"delivery.maximum_delivery_days" : {
"$lt" : 60
}
},
{
"delivery.filling_rate" : {
"$gt" : 90
}
}
]
},
{
"$and" : [
{
"delivery.maximum_delivery_days" : {
"$lt" : 40
}
},
{
"delivery.filling_rate" : {
"$gt" : 80
}
}
]
},
{
"$and" : [
{
"delivery.delivery_days" : {
"$lt" : 25
}
},
{
"delivery.filling_rate" : {
"$gt" : 70
}
}
]
}
]
}
]
Now to make this configurable, I need to be able to handle boolean logic, parameters and values.
So, I got the idea, since such query itself is JSON, to store it in Mongo and have my Java app retrieve it.
Next thing is using it in the filter (e.g. find, or whatever) and work on the corresponding selection of products.
The advantage of this approach is that I can actually analyse the data and the effectiveness of the query outside of my program.
I would store it by name in the database. E.g.
{
"name": "query1",
"query": { the thing printed above starting with "$or"... }
}
using:
db.queries.insert({
"name" : "query1",
"query": { the thing printed above starting with "$or"... }
})
Which results in:
2016-03-27T14:43:37.265+0200 E QUERY Error: field names cannot start with $ [$or]
at Error (<anonymous>)
at DBCollection._validateForStorage (src/mongo/shell/collection.js:161:19)
at DBCollection._validateForStorage (src/mongo/shell/collection.js:165:18)
at insert (src/mongo/shell/bulk_api.js:646:20)
at DBCollection.insert (src/mongo/shell/collection.js:243:18)
at (shell):1:12 at src/mongo/shell/collection.js:161
But I CAN STORE it using Robomongo, but not always. Obviously I am doing something wrong. But I have NO IDEA what it is.
If it fails, and I create a brand new collection and try again, it succeeds. Weird stuff that goes beyond what I can comprehend.
But when I try updating values in the "query", changes are not going through. Never. Not even sometimes.
I can however create a new object and discard the previous one. So, the workaround is there.
db.queries.update(
{"name": "query1"},
{"$set": {
... update goes here ...
}
}
)
doing this results in:
WriteResult({
"nMatched" : 0,
"nUpserted" : 0,
"nModified" : 0,
"writeError" : {
"code" : 52,
"errmsg" : "The dollar ($) prefixed field '$or' in 'action.$or' is not valid for storage."
}
})
seems pretty close to the other message above.
Needles to say, I am pretty clueless about what is going on here, so I hope some of the wizzards here are able to shed some light on the matter
I think the error message contains the important info you need to consider:
QUERY Error: field names cannot start with $
Since you are trying to store a query (or part of one) in a document, you'll end up with attribute names that contain mongo operator keywords (such as $or, $ne, $gt). The mongo documentation actually references this exact scenario - emphasis added
Field names cannot contain dots (i.e. .) or null characters, and they must not start with a dollar sign (i.e. $)...
I wouldn't trust 3rd party applications such as Robomongo in these instances. I suggest debugging/testing this issue directly in the mongo shell.
My suggestion would be to store an escaped version of the query in your document as to not interfere with reserved operator keywords. You can use the available JSON.stringify(my_obj); to encode your partial query into a string and then parse/decode it when you choose to retrieve it later on: JSON.parse(escaped_query_string_from_db)
Your approach of storing the query as a JSON object in MongoDB is not viable.
You could potentially store your query logic and fields in MongoDB, but you have to have an external app build the query with the proper MongoDB syntax.
MongoDB queries contain operators, and some of those have special characters in them.
There are rules for mongoDB filed names. These rules do not allow for special characters.
Look here: https://docs.mongodb.org/manual/reference/limits/#Restrictions-on-Field-Names
The probable reason you can sometimes successfully create the doc using Robomongo is because Robomongo is transforming your query into a string and properly escaping the special characters as it sends it to MongoDB.
This also explains why your attempt to update them never works. You tried to create a document, but instead created something that is a string object, so your update conditions are probably not retrieving any docs.
I see two problems with your approach.
In following query
db.queries.insert({
"name" : "query1",
"query": { the thing printed above starting with "$or"... }
})
a valid JSON expects key, value pair. here in "query" you are storing an object without a key. You have two options. either store query as text or create another key inside curly braces.
Second problem is, you are storing query values without wrapping in quotes. All string values must be wrapped in quotes.
so your final document should appear as
db.queries.insert({
"name" : "query1",
"query": 'the thing printed above starting with "$or"... '
})
Now try, it should work.
Obviously my attempt to store a query in mongo the way I did was foolish as became clear from the answers from both #bigdatakid and #lix. So what I finally did was this: I altered the naming of the fields to comply to the mongo requirements.
E.g. instead of $or I used _$or etc. and instead of using a . inside the name I used a #. Both of which I am replacing in my Java code.
This way I can still easily try and test the queries outside of my program. In my Java program I just change the names and use the query. Using just 2 lines of code. It simply works now. Thanks guys for the suggestions you made.
String documentAsString = query.toJson().replaceAll("_\\$", "\\$").replaceAll("#", ".");
Object q = JSON.parse(documentAsString);
I'm making a database on theses/arguments. They are related to other arguments, which I've placed in an object with a dynamic key, which is completely random.
{
_id : "aeokejXMwGKvWzF5L",
text : "test",
relations : {
cF6iKAkDJg5eQGsgb : {
type : "interpretation",
originId : "uFEjssN2RgcrgiTjh",
ratings: [...]
}
}
}
Can I find this document if I only know what the value of type is? That is I want to do something like this:
db.theses.find({relations['anything']: { type: "interpretation"}}})
This could've been done easily with the positional operator, if relations had been an array. But then I cannot make changes to the objects in ratings, as mongo doesn't support those updates. I'm asking here to see if I can keep from having to change the database structure.
Though you seem to have approached this structure due to a problem with updates in using nested arrays, you really have only caused another problem by doing something else which is not really supported, and that is that there is no "wildcard" concept for searching unspecified keys using the standard query operators that are optimal.
The only way you can really search for such data is by using JavaScript code on the server to traverse the keys using $where. This is clearly not a really good idea as it requires brute force evaluation rather than using useful things like an index, but it can be approached as follows:
db.theses.find(function() {
var relations = this.relations;
return Object.keys(relations).some(function(rel) {
return relations[rel].type == "interpretation";
});
))
While this will return those objects from the collection that contain the required nested value, it must inspect each object in the collection in order to do the evaluation. This is why such evaluation should really only be used when paired with something that can directly use an index instead as a hard value from the object in the collection.
Still the better solution is to consider remodelling the data to take advantage of indexes in search. Where it is neccessary to update the "ratings" information, then basically "flatten" the structure to consider each "rating" element as the only array data instead:
{
"_id": "aeokejXMwGKvWzF5L",
"text": "test",
"relationsRatings": [
{
"relationId": "cF6iKAkDJg5eQGsgb",
"type": "interpretation",
"originId": "uFEjssN2RgcrgiTjh",
"ratingId": 1,
"ratingScore": 5
},
{
"relationId": "cF6iKAkDJg5eQGsgb",
"type": "interpretation",
"originId": "uFEjssN2RgcrgiTjh",
"ratingId": 2,
"ratingScore": 6
}
]
}
Now searching is of course quite simple:
db.theses.find({ "relationsRatings.type": "interpretation" })
And of course the positional $ operator can now be used with the flatter structure:
db.theses.update(
{ "relationsRatings.ratingId": 1 },
{ "$set": { "relationsRatings.$.ratingScore": 7 } }
)
Of course this means duplication of the "related" data for each "ratings" value, but this is generally the cost of being to update by matched position as this is all that is supported with a single level of array nesting only.
So you can force the logic to match with the way you have it structured, but it is not a great idea to do so and will lead to performance problems. If however your main need here is to update the "ratings" information rather than just append to the inner list, then a flatter structure will be of greater benefit and of course be a lot faster to search.
I am attempting to use the Confluence REST API to correct the tree structure of pages on a Confluence site. I can do this manually, however there are quite a few pages and I already have the tree structure in another XML file.
Currently I am able to upload pages and images using the confluence REST API. Definitions are located here.
I have attempted to use the /rest/api/content/{id} PUT command. This requires the content to be in the body of the request. Attaching the content I had received from a previous GET request I find I have to update the version count.
The above all works. Adding to the ancestor list however doesn't seem to. If I add the parent page to the list of ancestors I get formatting errors reported back.
Added to this if I manually attach the page to another page and request its contents I find that the parent appears twice in the ancestor list. Also the history count does not increase.
The resulting JSON returned from a GET request for the page:
http://someconfluencesite.com/rest/api/content/10031361?expand=ancestors (200)
{
"id" : "10031361",
"type" : "page",
"title" : "Automatic Action Usage Updates",
"ancestors" : [{
"id" : "10031327",
"type" : "page",
"title" : "Action Lists"
}, {
"id" : "10031327",
"type" : "page",
"title" : "Action Lists"
}, {
"id" : "10031328",
"type" : "page",
"title" : "Actions Tab"
}, {
"id" : "10031327",
"type" : "page",
"title" : "Action Lists"
}, {
"id" : "10031327",
"type" : "page",
"title" : "Action Lists"
}, {
"id" : "10031328",
"type" : "page",
"title" : "Actions Tab"
}
]
}
The tree this is for looks like the following:
Space
Action Lists
Actions Tab
Automatic Action Usage Updates
This leads me to believe I am using the wrong command.
I have attempted to contact Confluence support however their authorization emails are not arriving in my inbox. So I have come here :)
So my question is what is the REST call for creating a page tree in confluence? After that, what is the format of the request body?
Edit:
The following PUT request gets a successful return code. Yet the returned object does not have the ancestor attached (not unusual as in all GET requests you have to expand to get it). The version number does not update. The two pages are not hooked to each other either.
{
"id":"10552520",
"type":"page",
"title":"Correct Page Title",
"ancestor":[
{
"id":"10552522",
"type":"page"
}],
"version":
{
"number":"2"
}
}
The nice thing the above does though is delete all the contents of the page.
The following POST call results in the page being created but having no ancestors. The ancestor exists with the id supplied. Strangely this also is created without any content in the page.
{
"type":"page",
"title":"Correct Title",
"space":{"key":"SpaceKey"},
"ancestor":[{"id":"10553655","type":"page"}],
"body":{"storage":{"value":"<p>New Page </p>","representation":"storage"}}
}
Putting the above into the REST API browser also results in the children not being attached to the parent.
So it appears the answer to my question is in the question itself.
The JSON needed to have "ancestors" not "ancestor" as the array name for the ancestors array. Once this was changed it all works for a POST request.
So if your reading this and having similar issues make sure that all the element names in the JSON your passing are correct. If they aren't, they simply are ignored.
{
"id":"10552520",
"type":"page",
"title":"Correct Page Title",
"ancestors":[
{
"id":"10552522",
"type":"page"
}
],
"version":
{
"number":"2"
}
}
It's "ancestors", not "ancestor".
To move an existing page first get it's version, space, etc.
Then call PUT with version + 1, space, ancestors, title.
pageaschild = { "type":"page",
"title":name,
"space":{"key":space},
"ancestors":[{"id":ancestorid,"type":"page"}],
"version":{"number":version}};
$.ajax({type:'PUT', url:baseURL + pageId, contentType:"application/json;charset=utf-8", data:JSON.stringify(pageaschild)});
I have a collection with following document:
{
"_id" : ObjectId("51f1fd2b8188d3117c6da352"),
"cust_id" : "abc1234",
"ord_date" : ISODate("2012-10-03T18:30:00Z"),
"status" : "A",
"price" : 27,
"items" : [{
"sku" : "mmm",
"qty" : 5,
"price" : 2.5
}, {
"sku" : "nnn",
"qty" : 5,
"price" : 2.5
}]
}
I want to use "$where" in the fields of "items", so something like this:
{$where:"this.items.sku==mmm"}
How can I do it? It works when the field is not of array type.
You don't need a $where operator to do this; just use a query object of:
{ "items.sku": mmm }
As for why your $where isn't working, the value of that operator is executed as JavaScript, so that's not going to check each element of the items array, it's just going to treat items as a normal object and compare its sku property (which is undefined) to mmm.
You are comparing this.items.sku to a variable mmm, which isn't initialized and thus has the value unefined. What you want to do, is iterate the array and compare each entry to the string 'mmm'. This example does this by using the array method some which returns true, when the passed function returns true for at least one of the entries:
{$where:"return this.items.some(function(entry){return entry.sku =='mmm'})"}
But really, don't do this. In a comment to the answer by JohnnyHK you said "my service is just a interface between user and mongodb, totally unaware what the field client want's to store". You aren't really explaining your use-case, but I am sure you can solve this better.
The $where operator invokes the Javascript engine even though this
trivial expression could be done with a normal query. This means unnecessary performance overhead.
Every single document in the collection is passed to the function, so when you have an index, it can not be used.
When the javascript function is generated from something provided by the client, you must be careful to sanetize and escape it properly, or your application gets vulnerable to code injection.
I've been reading through your comments in addition to the question. It sounds like your users can generically add some attributes, which you are storing in an array within a document. Your client needs to be able to query an arbitrary pair from the document in a generic manner. The pattern to achieve this is typically as follows:
{
.
.
attributes:[
{k:"some user defined key",
v:"the value"},
{k: ,v:}
.
.
]
}
Note that in your case, items is attributes. Now to get the document, your query will be something like:
eg)
db.collection.find({attributes:{$elemMatch:{k:"sku",v:"mmm"}}});
(index attributes.k, attributes.v)
This allows your service to provide a way to query the data, and letting the client specify what the k,v pairs are. The one caveat with this design is always be aware that documents have a 16MB limit (unless you have a use case that makes GridFS appropriate). There are functions like $slice which may help with controlling this.