I am very new to backbone.js (and MVC with javascript), and while reading several resources about backbone.js to adopt it in my project, I now have a question: how can I detect when multiple users(browsers) attempt to update? (and prevent it?)
My project is a tool for editing surveys/polls for users who want to create and distribute their own surveys. So far, my web app maintains a list of edit-commands fired by browser, sends it to the server, and the server does batch update.
What I did was, each survey maintains a version number and browser must request update with that version number, and if the request's version number does not match with the one in the server, the request fails and the user must reload his page (you know, implementing concurrent editing is not easy for everyone). Of course, when the browser's update was successful, it gets new version number from the server as ajax response, and one browser can request update to server only when its past update request is done.
Now, I am interested in RESTful APIs and MV* patterns, but having a hard time to solve this issue. What is the best / common approach for this?
There is a common trick instead of using versions, use TIMESTAMPS in your DB and then try to UPDATE WHERE timestamp = model.timestamp. If it returns zero result count - use appropriate HTTP 409 (conflict) response and ask the user to update the page in save() error callback. You can even use the local storage to merge changes, and compare the non-equivalent side by side.
Related
I am looking for a REST API to do following
Search based on parameters sent, if results found, return the results.
If no results found, create a record based on search parameters sent.
Can this be accomplished by creating one single API or 2 separate APIs are required?
I would expect this to be handled by a single request to a single resource.
Which HTTP method to use
This depends on the semantics of what is going on - we care about what the messages mean, rather than how the message handlers are implemented.
The key idea is the uniform interface constraint it REST; because we have a common understanding of what HTTP methods mean, general purpose connectors in the HTTP application can do useful work (for example, returning cached responses to a request without forwarding them to the origin server).
Thus, when trying to choose which HTTP method is appropriate, we can consider the implications the choice has on general purpose components (like web caches, browsers, crawlers, and so on).
GET announces that the meaning of the request is effectively read only; because of this, general purpose components know that they can dispatch this request at any time (for instance, a user agent might dispatch a GET request before the user decides to follow the link, to make the experience faster).
That's fine when you intend the request to provide the client with a copy of your search results, and the fact that you might end up making changes to server local state is just an implementation detail.
On the other hand, if the client is trying to edit the results of a particular search (but sometimes the server doesn't need to change anything), then GET isn't appropriate, and you should use POST.
A way to think about the difference is to consider what action you want to be taken when an intermediate cache holds a response from an earlier copy of "the same" request. If you want the cache to reuse the response, GET is the best; on the other hand, if you want the cache to throw away the old response (and possibly store the new one), then you should be using POST.
Theres a ton of videos and websites trying to explain backend vs frontend, but unfortunately none of them explains it in a way that you know how to develop a backend - driven website (at least I haven't found anything good).
So, I wanted to ensure that I understood it and kindly ask you to confirm or correct me on this topic.
Example:
I wanted to build Mini - Google. I have a Database containing 1000 stored websites.
Assumption #1:
Everytime I type something into the search bar, the autofill suggestions change. This means, everytime i type, another website / API gets called returning the current autofill suggestions. On a developer site, this means the website e.g. is a Python script which gets called with the current word typed in as a Parameter and is returning all suggestions as e.g. JSON:
// Client Side Script
function ontype(input):
suggestions = get("https://api.googlemini.com/suggestions?q=" + str(input))
show(suggestions)
Assumption #2:
This also means I could manually call the website containing the Python script, providing a random word and it would always return a JSON containing the autofill suggestions for that word.
Question #1:
If A#1 turns out true but A#2 turns out false, how could I prevent a user from randomly accessing the "API" while still returning results when called by a script?
Assumption #3:
After pressing enter, my website googlemini.com/search?... would be called. As google.com/search reloads everytime searching for a new query (or going to page 2 etc.), I assume, instead of calling an API, when the server gets the client request, it first searches through its database, sorts the results and then returns a whole html as a static webpage:
// Server Side Script
#app.route("/search")
function oncall():
query = getparam("q")
results = searchdatabase(query)
html = buildhtml(results)
return html
Question #2:
Often, I hear (or at least understand it this way) that database and webserver are 2 seperate servers. How would that work? Wouldn't that mean the database server needs to be accessible to the web too (of course it would have security layers etc., but technically it would)? How could I access the database server from the webserver?
Question #3:
Are there, on a technical basis, any other ways to build backend services?
That's it. I would also appreciate any recommendations like videos, websites or others to learn how to technically setup and / or secure backend servers.
Thanks in advance.
For your first question you can yes there is a way to prevent miss use.
What you can do is add identifier to api like Auth token to identify a user and every time a user access the api you can save the count on the server n whenever the count has exceeded a limit within a time span you can reject the call. And the limit can be set in such a way that it doesn't trouble the honest user and punishes the wrong one. There are even more complex and effective methods but this is the basic idea.
For question number to let me explain you a simple concept a database is a very efficient, resourcefull and expensive data storage solution we never want it to be used in a general sense as varible store or something. We always want to access the database in call get the data process the data update the data. So we do it data way and its not necessary you make sepreate server for data base. The thing is we mostly make databse to be accessible to various platforms android, ios, windows. So its better to add some abstraction and keep data base as a separte entity.
For the last, I am not well aware about what you meant by other but I am listing some backend teechnologies, some of these might be used in isolation some of these not some other tools as well.
Django
FLask
Djnago rest
GraphQL
SQL
PHP
Node
Deno
I'm building a REST API and I'm trying to keep it as RESTful as possible, but some things are still not quite clear for me. I saw a lot of topic about similar question but all too centered about the "simple" problem of updating data, my issue is more about the business logic around that.
My main issue is with business logic triggered by partial update of a model. I see a lot of different opinion online about PATCH methods, creating new sub-ressources or adding action, but it often seems counter productive with the REST approach of keeping URI simple and structured.
I have some record that need to be proceeded ( refused, validated, partially validated ..etc ), each change trigger additional actions.
If it's refused, an email with the reason should be sent
if it's partially validated, the link to fulfill the missing data is sent
if it's validated some other ressources must be created.
There is a few other change that can be made to the status but this is enough for the example.
What would be a RESTful way to do that ?
My first idea would be to create actions :
POST /record/:id/refuse
POST /record/:id/validate ..etc
It seems RESTful to me but too complicated, and moreover, this approach means having multiple route performing essentially the same thing : Update one field in the record object
I also see the possibility of a PATCH method like :
PATCH /record/:id in which I check if the field to update is status, and the new value to know which action to perform.
But I feel it can start to be too complex when I will have the need to perform similar action for other property of the record.
My last option, and I think maybe the best but I'm not sure if it's RESTful, would be to use a sub-ressource status and to use PUT to update it :
PUT /record/:id/status, with a switch on the new value.
No matter what the previous value was, switching to accepted will always trigger the creation, switching to refused will always trigger the email ...etc
Are those way of achieving that RESTful and which one make more sense ? Is there other alternative I didn't think about ?
Thanks
What would be a RESTful way to do that ?
In HTTP, your "uniform interface" is that of a document store. Your Rest API is a facade, that takes messages with remote authoring semantics (PUT/POST/PATCH), and your implementation produces useful work as a side effect of its handling of those messages.
See Jim Webber 2011.
I have some record that need to be proceeded ( refused, validated, partially validated ..etc ), each change trigger additional actions.
So think about how we might do this on the web. We GET some resource, and what is returned is an html representation of the information of the record and a bunch of forms that describe actions we can do. So there's a refused form, and a validated form, and so on. The user chooses the correct form to use in the browser, fills in any supplementary information, and submits the form. The browser, using the HTML form processing rules, converts the form information into an HTTP request.
For unsafe operations, the form is configured to use POST, and the browsers therefore know that the form data should be part of the message-body of the request.
The target-uri of the request is just whatever was used as the form action -- which is to say, the representation of the form includes in it the information that describes where the form should be submitted.
As far as the browser and the user are concerned, the target-uri can be anything. So you could have separate resources to handle validate messages and refused messages and so on.
Caching is an important idea, both in REST and in HTTP; HTTP has specific rules baked into it for cache invalidation. Therefore, it is often the case that you will want to use a target-uri that identifies the document you want the client to reload if the command is successful.
So it might go something like this: we GET /record/123, and that gives us a bunch of information, and also some forms describing how we can change the record. So fill one out, submit it successfully, and now we expect the forms to be gone - or a new set of forms to be available. Therefore, it's the record document itself that we would expect to be reloading, and the target-uri of the forms should be /record/123.
(So the API implementation would be responsible for looking at the HTTP request, and figuring out the meaning of the message. They might all go to a single /record/:id POST handler, and that code looks through the message-body to figure out which internal function should do the work).
PUT/PATCH are the same sort of idea, except that instead of submitting forms, we send edited representations of the resource itself. We GET /record/123, change the status (for example, to Rejected), and then send a copy of our new representation of the record to the server for processing. It would therefore be the responsibility of the server to examine the differences between its representation of the resource and the new provided copy, and calculate from them any necessary side effects.
My last option, and I think maybe the best but I'm not sure if it's RESTful, would be to use a sub-resource status and to use PUT to update it
It's fine -- think of any web page you have ever seen where the source has a link to an image, or a link to java script. The result is two resources instead of one, with separate cache entries for each -- which is great, when you want fine grained control over the caching of the resources.
But there's a trade - you also need to fetch more resources. (Server-push mitigates some of this problem).
Making things easier on the server may make things harder on the client - you're really trying to find the design with the best balance.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I am thinking about buildning a REST API with both websockets and http where I use websockets to tell the client that new data is available or provide the new data to the client directly.
Here are some different ideas of how it could work:
ws = websocket
Idea A:
David get all users with GET /users
Jacob add a user with POST /users
A ws message is sent to all clients with info that a new user exist
David recive a message by ws and calls GET /users
Idea B:
David get all users with GET /users
David register to get ws updates when a change is done to /users
Jacob add a user with POST /users
The new user is sent to David by ws
Idea C:
David get all users with GET /users
David register to get ws updates when a change is done to /users
Jacob add a user with POST /users and it gets the id 4
David receive the id 4 of the new user by ws
David get the new user with GET /users/4
Idea D:
David get all users with GET /users
David register to get ws updates when changes is done to /users.
Jacob add a user with POST /users
David receive a ws message that changes is done to /users
David get only the delta by calling GET /users?lastcall='time of step one'
Which alternative is the best and what are the pros and cons?
Is it another better 'Idea E'?
Do we even need to use REST or is ws enought for all data?
Edit
To solve problems with data getting out of sync we could provide the header"If-Unmodified-Since"https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Unmodified-Sinceor "E-Tag" https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag or both with PUT requests.
Idea B is for me the best, because the client specifically subscribes for changes in a resource, and gets the incremental updates from that moment.
Do we even need to use REST or is ws enought for all data?
Please check: WebSocket/REST: Client connections?
I don't know Java, but I worked with both Ruby and C on these designs...
Funny enough, I think the easiest solution is to use JSON, where the REST API simply adds the method data (i.e. method: "POST") to the JSON and forwards the request to the same handler the Websocket uses.
The underlying API's response (the response from the API handling JSON requests) can be translated to any format you need, such as HTML rendering... though I would consider simply returning JSON for most use cases.
This helps encapsulate the code and keep it DRY while accessing the same API using both REST and Websockets.
As you might infer, this design makes testing easier, since the underlying API that handles the JSON can be tested locally without the need to emulate a server.
Good Luck!
P.S. (Pub/Sub)
As for the Pub/Sub, I find it best to have a "hook" for any update API calls (a callback) and a separate Pub/Sub module that handles these things.
I also find it more resource friendly to write the whole data to the Pub/Sub service (option B) instead of just a reference number (option C) or an "update available" message (options A and D).
In general, I also believe that sending the whole user list isn't effective for larger systems. Unless you have 10-15 users, the database call might be a bust. Consider the Amazon admin calling for a list of all users... Brrr....
Instead, I would consider dividing this to pages, say 10-50 users a page. These tables can be filled using multiple requests (Websocket / REST, doesn't matter) and easily updated using live Pub/Sub messages or reloaded if a connection was lost and reestablished.
EDIT (REST vs. Websockets)
As For REST vs. Websockets... I find the question of need is mostly a subset of the question "who's the client?"...
However, once the logic is separated from the transport layer, than supporting both is very easy and often it makes more sense to support both.
I should note that Websockets often have a slight edge when it comes to authentication (credentials are exchanged once per connection instead of once per request). I don't know if this is a concern.
For the same reason (as well as others), Websockets usually have an edge with regards to performance... how big an edge over REST depends on the REST transport layer (HTTP/1.1, HTTP/2, etc').
Usually these things are negligible when it comes time to offer a public API access point and I believe implementing both is probably the way to go for now.
To summarize your ideas:
A: Send a message to all clients when a user edits data on the server. All users then request an update of all data.
-This system may make a lot of unnecessary server calls on behalf of clients who are not using the data. I don't recommend producing all of that extra traffic as processing and sending those updates could become costly.
B: After a user pulls data from the server, they then subscribe to updates from the server which sends them information about what has changed.
-This saves a lot of server traffic, but if you ever get out of sync, you're going to be posting incorrect data to your users.
C: Users who subscribe to data updates are sent information about which data has been updated, then fetch it again themselves.
-This is the worst of A and B in that you'll have extra round trips between your users and servers just to notify them that they need to make a request for information which may be out of sync.
D: Users who subscribe to updates are notified when any changes are made and then request the last change made to the server.
-This presents all of the problems with C, but includes the possibility that, once out of sync, you may send data that will be nonsense to your users which might just crash the client side app for all we know.
I think that this option E would be best:
Every time data changes on the server, send the contents of all the data to the clients who have subscribed to it. This limits the traffic between your users and the server while also giving them the least chance of having out of sync data. They might get stale data if their connection drops, but at least you wouldn't be sending them something like Delete entry 4 when you aren't sure whether or not they got the message that entry 5 just moved into slot 4.
Some Considerations:
How often does the data get updated?
How many users need to be updated each time an update occurs?
What are your transmission
costs? If you have users on mobile devices with slow connections, that will affect how often and how much you can afford to send to them.
How much data gets updated in a given update?
What happens if a user sees stale data?
What happens if a user gets data out of sync?
Your worst case scenario would be something like this: Lots of users, with slow connections who are frequently updating large amounts of data that should never be stale and, if it gets out of sync, becomes misleading.
I personally have used Idea B in production and am very satisfied with the results. We use http://www.axonframework.org/, so every change or creation of an entity is published as an event throughout the application. These events are then used to update several read models, which are basically simple Mysql tables backing one or more queries. I added some interceptors to the event processors that update these read models so that they publish the events they just processed after the data is committed to the DB.
Publishing of events is done through STOMP over web sockets. It is made very simple is you use Spring's Web Socket support (https://docs.spring.io/spring/docs/current/spring-framework-reference/html/websocket.html). This is how I wrote it:
#Override
protected void dispatch(Object serializedEvent, String topic, Class eventClass) {
Map<String, Object> headers = new HashMap<>();
headers.put("eventType", eventClass.getName());
messagingTemplate.convertAndSend("/topic" + topic, serializedEvent, headers);
}
I wrote a little configurer that uses Springs bean factory API so that I can annotate my Axon event handlers like this:
#PublishToTopics({
#PublishToTopic(value = "/salary-table/{agreementId}/{salaryTableId}", eventClass = SalaryTableChanged.class),
#PublishToTopic(
value = "/salary-table-replacement/{agreementId}/{activatedTable}/{deactivatedTable}",
eventClass = ActiveSalaryTableReplaced.class
)
})
Of course, that is just one way to do it. Connecting on the client side may look something like this:
var connectedClient = $.Deferred();
function initialize() {
var basePath = ApplicationContext.cataDirectBaseUrl().replace(/^https/, 'wss');
var accessToken = ApplicationContext.accessToken();
var socket = new WebSocket(basePath + '/wss/query-events?access_token=' + accessToken);
var stompClient = Stomp.over(socket);
stompClient.connect({}, function () {
connectedClient.resolve(stompClient);
});
}
this.subscribe = function (topic, callBack) {
connectedClient.then(function (stompClient) {
stompClient.subscribe('/topic' + topic, function (frame) {
callBack(frame.headers.eventType, JSON.parse(frame.body));
});
});
};
initialize();
Another option is to use Firebase Cloud Messaging:
Using FCM, you can notify a client app that new email or other data is
available to sync.
How does it work?
An FCM implementation includes two main components for sending and
receiving:
A trusted environment such as Cloud Functions for Firebase or an app server on which to build, target and send messages.
An iOS, Android, or Web (JavaScript) client app that receives messages.
Client registers its Firebase key to a server. When updates are available, server sends push notification to the Firebase key associated with the client. Client may receive data in notification structure or sync it with a server after receiving a notification.
Generally you might have a look at current "realtime" web frameworks like MeteorJS which tackle exactly this problem.
Meteor in specific works more or less like your example D with subscriptions on certain data and deltas being sent out after changes only to the affected clients. Their protocol used is called DDP which additionally sends the deltas not as overhead prone HTML but raw data.
If websockets are not available fallbacks like long polling or server sent events can be used.
If you plan to implement it yourself i hope these sources are some kind of inspiration how this problem has been approached. As already stated the specific use case is important
The answer depends on your use case. For the most part though I've found that you can implement everything you need with sockets. As long as you are only trying to access your server with clients who can support sockets. Also, scale can be an issue when you're using only sockets. Here are some examples of how you could use just sockets.
Server side:
socket.on('getUsers', () => {
// Get users from db or data model (save as user_list).
socket.emit('users', user_list );
})
socket.on('createUser', (user_info) => {
// Create user in db or data model (save created user as user_data).
io.sockets.emit('newUser', user_data);
})
Client side:
socket.on('newUser', () => {
// Get users from db or data model (save as user_list).
socket.emit('getUsers');
})
socket.on('users', (users) => {
// Do something with users
})
This uses socket.io for node. I'm not sure what your exact scenario is but this would work for that case. If you need to include REST endpoints that would be fine too.
With all great information all the great people added before me.
I found that eventually there is no right or wrong, its simply goes down to what suits your needs:
lets take CRUD in this scenario:
WS Only Approach:
Create/Read/Update/Deleted information goes all through the websocket.
--> e.g If you have critical performance considerations ,that is not
acceptable that the web client will do successive REST request to fetch
information,or if you know that you want the whole data to be seen in
the client no matter what was the event , so just send the CRUD events
AND DATA inside the websocket.
WS TO SEND EVENT INFO + REST TO CONSUME THE DATA ITSELF
Create/Read/Update/Deleted , Event information is sent in the Websocket,
giving the web client information that is necessary to send the proper
REST request to fetch exactly the thing the CRUD that happend in server.
e.g. WS sends UsersListChangedEvent {"ListChangedTrigger: "ItemModified" , "IdOfItem":"XXXX#3232" , "UserExtrainformation":" Enough info to let the client decide if it relevant for it to fetch the changed data"}
I found that using WS [Only for using Event Data] and REST
[To consume the data ]is better because:
[1] Separation between reading and writing model, Imagine you want to add some runtime information when your data is retrieved when its read from REST , that is now achieved because you are not mixing Write & Read models like in 1.
[2] Lets say other platform , not necessarily web client will consume this data.
so you just change the Event trigger from WS to the new way, and use REST to
consume the data.
[3] Client do not need to write 2 ways to read the new/modified data.
usually there is also code that reads the data when the page loads , and not
through the websocket , this code now can be used twice , once when page
loads , and second when WS triggered the specific event.
[4] Maybe the client do not want to fetch the new User because its showing currently only a view of old Data[E.g. users] , and new data changes is not in its interest to fetch ?
i prefer the A, it allows client the flexibility whether or not to update the existing data.
also with this method, implementation and access control becomes much more easier.
for example you can simply broadcast the userUpdated event to all users, this saves having a client list for do specific broadcasts and the Access Controls and Authentications applied for your REST Route wont have to change to reapplied again because the client is gonna make a GET request again.
Many things depends on the what kind of application you are making.
I'm developing an application that needs to be syncronized with remote database. The database is connected to the a web-based application that user able to modify some records on the web page.(add/remove/modify) User also able to modify the same records in mobile application. So each side (server - client) must be keep the SAME latest records when an user press the sync button in mobile app. Communication between server and client is provided by Web Serives.(SOAP) and i am not able to change it because of it is strict requirements. (i know this is the worst way that can be used). And another requirement is the clients are not able to delete the server records.
I already be familiar with communicating web service (NSURLConnection), receiving data (NSData) and parsing it. But i could not figure out how the syncronization procedure should be. I have already read this answer which about how i can modify server and client sides with some extra attributes (last_updated_date and is_sync)
Then i could imagine to solve the issue like:
As a first step, client keeps try to modify the server records with sending unsyncronized ones. New recoords are directly added into DB but modified records shoud be compared depending on last_updated_date. At the end of this step, server has the latest data.
But the problem is how can manage to modify the records in mobile app. I thought it in a two way:
is the dummiest way that create a new MOC, download all records into this and change with existing one.
is the getting all modified records which are not in client side, import them into a new MOC and combine these two. But in this point i have some concerns like
There could be two items which are replicated (old version - updated version)
Deleted items could be still located in the main MOCs.
I have to connect multiple relationships among the MOCs. (the new record could have more than 4 relationships with old records)
So i guess you guys can help me to have another ideas which is the best ??
Syncing data is a non-trivial task.
There are several levels of synchronization. Based on your question I am guessing you just need to push changes back to a server. In that case I would suggest catching it during the -save: of the NSManagedObjectContext. Just before the -save: you can query the NSManagedObjectContext and ask it for what objects have been created, updated and deleted. From there you can build a query to post back to your web service.
Dealing with merges, however, is far more complicated and I suggest you deal with them on the server.
As for your relationship question; I suggest you open a second question for that so that there is no confusion.
Update
Once the server has finished the merge it pushes the new "truth" to the client. The client should take these updated records and merge them into its own changes. This merge is fairly simple:
Look for an existing record using a uniqueID.
If the record exists then update it.
If the record does not exist then create it.
Ignoring performance for the moment, this is fairly straight forward:
Set up a loop over the new data coming in.
Set up a NSPredicate to identify the record to be updated/created.
Run your fetch request.
If the record exists update it.
If it doesn't then create it.
Once you get this working with a full round trip then you can start looking at performance, etc. Step one is to get it to work :)