Hello Internet Denizens,
I was reading through a nice database design article and the final determination on how to properly generate DB primary keys was ...
So, in reality, the right solution is probably: use UUIDs for keys,
and don’t ever expose them. The external/internal thing is probably
best left to things like friendly-url treatments, and then (as Medium
does) with a hashed value tacked on the end.
That is, use UUIDs for internal purposes like db joins, but use a friendly-url for external purposes (like a REST API).
My question is ... how do you make uniquely identifiable (and friendly) keys for external purposes?
I've used several APIs: Stripe, QuickBooks, Amazon, etc. and it seems like they use straight up sequential IDs for things like customers, report IDs, etc for retrieving information. It makes me wonder if exposing UUIDs as a security risk is a little overblown b/c in theory you should be able to append a where clause to your queries.
SELECT * FROM products where UUID = <supplied uuid> AND owner/role/group/etc = <logged in user>
The follow-up question is: If you expose a primary key, how do people efficiently restrict access to that resource in a database environment? Assign an owner to a db row?
Interested in the design responses.
Potential Relevant Posts for Further Reading:
Should I use UUIDs for resources in my public API?
It is not a good idea to expose your internal ids to the outside. You should either encode them (with some algorithm) or have a look up table.
Also, do not append parameters provided by user (or URL) to your SQL query (UUIDS or not), this is prone to SQL injection. Use parameterized SQL queries for that.
Related
I'm thinking about a REST API design. There are several tables in my database. For example Customer and Order.
Of course - each Order has its Customer (and every customer can have many Orders).
I've decided to provide such an interface
/api/v1/Customers/ -- get list of Customers, add new Customer
/api/v1/Customers/:id: -- get Customer with id=:id:
/api/v1/Orders/ -- get list of Orders, add new Order
/api/v1/Orders/:id: -- get Order with id=:id:
It works flawlessly. But my frontend has to display a list of orders with customer names. With this interface, I will have to make a single call to /api/v1/Orders/ and then another call to /api/v1/Customer/:id: for each record from the previous call. Or perform two calls to /api/v1/Orders/ and /api/v1/Customers/ and combine them on the frontend side.
It looks like overkill, this kind of operation should be done at the database level. But how can/should I provide an appropriate interface?
/api/v1/OrdersWithCustomers
/api/v1/OrdersWithCustomers/:id:
Seems weir. Is it a right way to go
There's no rule that says you cannot "extend" the data being returned from a REST API call. So instead of returning "just" the Order entity (as stored in the backend), you could of course return an OrderResponseDTO which includes all (revelant) fields of the Order entity - plus some from the Customer entity that might are relevant in your use case.
The data model for your REST API does not have to be an exact 1:1 match to your underlying database schema - it does give you the freedom to leave out some fields, or add some additional information that the consumers of your API will find helpful.
Great question, and any API design will tend to hit pragmatic reality at some point like this.
One option is to include a larger object graph for each resource (ie include the customer linked to each order) but use filter query parameters to allow users to specify what properties they require or don't require.
Personally I think that request parameters on a restful GET are fine for either search semantics when retrieving a list of resources, or filtering what is presented for each resource as in this case
Another option for your use case might be to look into a GraphQL approach.
How would you do it on the web?
You've got a web site, and that website serves documents about Customers, and documents about Orders. But your clients aren't happy, because its too much boring, mistake-prone work to aggregate information in the two kinds of documents.
Can we please have a document, they ask, with the boring work already done?
And so you generate a bunch of these new reports, and stick them on your web server, and create links to make it easier to navigate between related documents. TA-DA.
A "REST-API" is a facade that makes your information look and act like a web site. The fact that you are generating your representations from a database is an implementation details, deliberately hidden behind the "uniform interface".
I work with a banking database, which is structured like this:
Table Primary Key Unique Keys Foreign Keys
-------------------------------------------------------------
BANK ID BIC
CUSTOMER ID CUSTNO, PASS, CARD BANK
ACCOUNT ID IBAN BANK, CUSTOMER
I want to design a clean REST API, but I run into following problems:
Should I put resources in a hierarchy, or rather flat? The problem with the hierarchy might be that the client only knows the ACCOUNT ID, but does not know the CUSTOMER ID, so how is he supposed to get the resource?
/banks/{id}/customers/{id}/accounts{id}
or
/banks/{id}
/customers/{id}
/accounts{id}
The primary key in each table is the database ID. It is an internal ID and has no business meaning. Is it correct to use it as the default URI of the resource?
Each object has its own set of unique keys. For example, CUSTOMER can be identified by his CUSTNO, PASS or CARD. Each client only has a subset of these keys. Should I define a sub-resource per key or provide a lookup service that will give the proper URI back?
/customers/id/{id}
/customers/custno/{custno}
/customers/pass/{pass}
/customers/card/{card}
or
/lookup/customer?keyType=card&keyValue=AB-303555
(gives back customer {id})
I am asking what is the truly RESTful way, what is best practice. I haven't found proper answers yet.
I am asking what is the truly RESTful way, what is best practice.
REST doesn't care what spellings you use for your identifiers.
/ef726381-dd43-4017-9778-83cee2bbbd93
is a perfectly RESTful URI, suitable for any use case.
Outside of some purely mechanical concerns, general purpose consumers treat a URI as a single opaque unit. There's no notion of a consumer extracting semantic information from the URI -- which means that any information encoded into the identifier is done at the server's discretion and for its use alone.
For cases where information known to the client needs to be included in the target-uri of the request, we have URI Templates, which are a sort of generalization of a GET form in HTML. So a way to think about your problem is to consider what information the client has, and how they would put that information into a form.
HTML's form processing rules are pretty limiting -- general URI templates have fewer constraints.
/customers/id/{id}
/customers/custno/{custno}
/customers/pass/{pass}
/customers/card/{card}
Having multiple resources sharing common information is normal in REST -- your resource model is not your data model. So this could be fine. It's even OK to have multiple resources that share representations. You could have them stand alone, or you could have them share a Content-Location, or a canonical link relation, or you could simply have those resources redirect to the canonical resource.
It's all good.
So you mean if a UUID can be a valid URI, then a table autonumber key can be too?
Yes, exactly.
Note that if you want the lifetime of the URI to extend beyond the lifetime of your current implementation, then you need to design your identifiers with that constraint in mind. See Cool URIs Don't Change.
The clients don't care what the URI is, they just want the link to work again when they need it.
I want to ask what is the most RESTful way for queries, I have this existing API
/entities/users?skip=0&limit=100&queries={"$find":{"$minus":{"$find":{"username":"markzu"}}}}
Easily the first parts of the query, skip and limit are easily identifiable however I find the "queries" part quite confusing for others. What the query means is to
Find every User minus Find User entities with username 'markzu'
The reason it is defined this way is due to the internal database query behavior.
Meaning in the NoSQL database we use, the resource run two transactional queries, first is to find everything in the User table minus a find User with a username that was specified (similar to SQL) -- boolean operations. So in other words, the query means, "fetch every User except username 'markzu' "
What is the proper way to define this in RESTful way, based on standards?
What is the proper way to define this in RESTful way, based on standards?
REST doesn't care what spelling you use for resource identifiers, so long as your choice is consistent with the production rules defined in RFC 3986.
However, we do have a standard for URI Templates
A URI Template is a compact sequence of characters for describing a range of Uniform Resource Identifiers through variable expansion.
You are already aware of the most familiar form of URI template -- key-value pairs encoded in the query string.
?skip=0&limit=100&username=markzu
That's often a convenient choice, because HTML understands how to process forms into url encoded queries.
It doesn't look like you need any other parameters, you just need to be able this query from others. So a perfectly reasonable choice might be
/every-user-except?skip=0&limit=100&username=markzu
It may help to think "prepared statement", rather than "query".
The underlying details of the implementation really shouldn't enter into the calculation at all. Your REST API is a facade that makes your app look like an HTTP aware key value store.
Let's take the following resource in my REST API:
GET `http://api/v1/user/users/{id}`
In normal circumstances I would use this like so:
GET `http://api/v1/user/users/aabc`
Where aabc is the user id.
There are times, however, when I have had to design my REST API in a way that some extra information is passed with the ID. For example:
GET `http://api/v1/user/users/customer:1`
Where customer:1 denotes I am using an id from the customer domain to lookup the user and that id is 1.
I now have a scenario where the identifier is more than one key (a composite key). For example:
GET `http://api/v1/user/users/customer:1;type:agent`
My question: in the above URL, what should I use as the separator between customer:1 and type:agent?
According to https://www.ietf.org/rfc/rfc3986.txt I believe that the semi-colon is not allowed.
You should either:
Use parameters:
GET http://api/v1/user/users?customer=1
Or use a new URL:
GET http://api/v1/user/users/customer/1
But use Standards like this
("Paths tend to be cached, parameters tend to not be, as a general rule.")
Instead of trying to create a general structure for accessing records via multiple keys at once, I would suggest trying to think of this on more of a case-by-case basis.
To take your example, one way to interpret it is that you have multiple customers, and those customers each may have multiple user accounts. A natural hierarchy for this would be:
/customer/x/user/y
Often an elegant decision like this can be made, that not only solves the problem but also documents your data-model in a way that someone can easily see that users belong to customers via a 1-to-many relationship.
I have a restful service for the documents, where the documents are stored in mongodb, the restful api for the document is /document/:id, initially the :id in the api is using the mongodb 's object id, but I wonder deos this approach reveal the database id, and expose the potential threat, should I want to replace it with a pseudonymity id.
if it is needed to replace it the pseudonymity id, I wonder if there is a algorithmic methods for me to transform the object id and pseudonymity id back and forth without much computation
First, there is no "database id" contained in the ObjectID.
I'm assuming your concern comes from the fact that the spec lists a 3 byte machine identifier as part of the ObjectID. A couple of things to note on that:
Most of the time, the ObjectID is actually generated on the client side, not the server (though it can be). Hence this is usually the machine identifier for the application server, not your database
The 3 byte Machine ID is the first three bytes of the (md5) hash of the machine host name, or of the mac/network address, or the virtual machine id (depending on the particular implementation), so it can't be reversed back into anything particularly meaningful
With the above in mind, you can see that worrying about exposing information is not really a concern.
However, with even a small sample, it is relatively easy to guess valid ObjectIDs, so if you want to avoid that type of traffic hitting your application, then you may want to use something else (a hash of the ObjectID might be a good idea for example), but that will be dependent on your requirements.