If I correctly understand how the whole memcache technology works, it caches queries to an external component of a website, most often a database. But a static site does not use a database. So does it make sense to use memcache with a static side framework such as Jekyll and Octopress?
You're better off using Nginx and/or Varnish if your site is mostly static. The typical use case for memcached is for caching database data, it's not limited to that, but Varnish in particular is great for caching entire HTML pages and all of its associated static content (js, css, images, etc).
Yes. you can use. u can cache entire page. all depends on your application
Related
I've done almost all of my web projects using Java(Spring MVC + Thymeleaf) and it's MVC technologies. Recently I've heared about REST and started learning some stuff about it. I realized that it is one of the coolest things I've ever seen in my whole life!(I'm just joking around, it's definitely not)
All we need is just to parse data to json type and then return it to the frontend. And in frontend we no longer need to use Model and it's objects. Frontend can get all required data in nice-to-work-with json type using one single GET request!
We don't need to use some weird Thymeleaf constructions to handle errors or to iterate through the list in our template! We can handle all events and process all data using javascript and it's frameworks. It is much more powerful.
Does there exist something I missed? When to use Model? When to use json-type data?
These are two (somewhat overlapping) approaches to frontend development: generating pages on the backend and on the frontend.
Using a backend model and Thymeleaf templates (or any other HTML templates), you generate your web page on the server side. This means the following benefits
you can write frontend and backend logic in one language (Java);
you can enforce data and security constraints in one place - on the backend, using Java code or configuration;
in many cases, it's faster for the user to get their first page, since the rendering happens on the server, and the user gets an already rendered page;
But this approach has the following drawbacks:
the server has to render the pages, which means more load on the server;
most of the modern web sites and applications use JavaScript anyway, so you'll have to write at least some JavaScript;
server-generated pages don't even come close to what's possible to render in the browser.
Providing a REST API for a JavaScript frontend, you have the following benefits:
you unload your server, since most of the UI related work happens on the user's device;
you can achieve much more with modern frontend frameworks;
navigating on an already rendered UI is faster, since you don't need for the server to render every page, you only need to get a relatively small JSON response and then update the page dynamically.
But this approach has the following tradeoffs:
the user generally waits longer to see their first page, as the browser needs to download a lot of scripts and then spend time on dynamic rendering of the page;
in a large application, you need to either have a full-stack developer team, or two teams working on frontend and backend separately;
you have to write your UI logic in JavaScript. Make of it what you will :)
you have to sometimes duplicate the constraints both on the frontend and on the backend: e.g., if a user can't edit a field, you have to both show it as read-only on the frontend, and add validation in your REST service, since the user may try to access your REST API directly and bypass the frontend validation;
you have to be more aware of securing your REST API in general.
I need to programmatically interact with a WebObjects website and extract data from the responses. The particular WebObjects site I am scraping uses component actions and stores sessions in cookies (not urls). This means that all urls look something like this:
http://example.com/WOApp/WebObjects/WOApp.woa/wo/7.0.0.0.29.1.1.1
My first questions are:
Does urls like this not completely destroy local and shared caching opportunities (cachable constraint in REST)? I imaging the only effective caching with such urls is the WebObjects server itself.
Isn't addressability broken as well? Each resource does have a unique endpoint, but it changes constantly. Furthermore (I think) that WebObjects also makes too old URLs invalid since they "time-out" after a period of time. I'm not sure whether this applies only to urls with sessions though.
Regarding the scraping I am not sure whether it's possible to extract any meaningful endpoints from the website. For example, with a normal website I would look through the HTML and extract the POST urls, then use them in my scraper by posting directly to them instead of going through the normal request-response cycle.
In this case I obviously cannot use any URLs extracted from the HTML since they are dynamically generated on each request, but I read something about being able to access WebObjects components directly if the security settings have not been set to disallow this (see https://developer.apple.com/legacy/library/documentation/LegacyTechnologies/WebObjects/WebObjects_3.5/PDF/WebObjectsDevGuide.pdf, p. 53 "Limitations on Direct requests"). I don't understand exactly how to do this though or if it's even possible.
If it's not possible what would be a good approach then? The only options I can think of is:
Using a full-blown browser client to interact with the website (e.g. WatiR or Selenium) and extract & process the HTML from their responses
Manually extracting the dynamic end-points by first request the page where they are on and then find the place in the HTML where they're located. Then use them afterwards as if they were "static".
I am interested in opinions on how to approach this scenario since I don't believe any of the solutions above are particularly good.
You've asked a number of questions, and I'll see if I can cover each in turn.
Does urls like this not completely destroy local and shared caching
opportunities (cachable constraint in REST)? I imaging the only
effective caching with such urls is the WebObjects server itself.
There is, indeed, a page cache within the WebObjects application server, and you're right to observe that these component action URLs probably thwart any other kind of caching. Additionally, even though the session ID is not present in the URL, you'd need the session ID in the cookie to re-create the same page, so having just that URL would get you a session restoration error from the application server.
Isn't addressability broken as well? Each resource does have a unique
endpoint, but it changes constantly.
Well, yes, on the face of it this is true. You've given a component action URL as an example, and they're tied to the session.
Furthermore (I think) that
WebObjects also makes too old URLs invalid since they "time-out" after
a period of time. I'm not sure whether this applies only to urls with
sessions though.
Again, all true. Component action URLs generate sessions, and sessions time out.
At this point, let me take a quick diversion. I'm assuming you're not the owner of the WebObjects application—you're talking about having to scrape a WebObjects app, and you've identified some ways in which this particular app doesn't conform to REST principles. You're completely right—a fully component-action-based WebObjects application won't be RESTful. WebObjects pre-dates REST by a few years. Having said that, there are ways in which a WebObjects application can be completely RESTful:
Using session-less direct actions gives a degree of REST-like behaviour, and would certainly solve the problems you identify with caching, addressability and expiry.
Using the ERRest framework to create a 100% RESTful application.
Of course, none of this will help you if you're just trying to scrape a legacy application.
Regarding the scraping I am not sure whether it's possible to extract
any meaningful endpoints from the website. For example, with a normal
website I would look through the HTML and extract the POST urls, then
use them in my scraper by posting directly to them instead of going
through the normal request-response cycle.
Again, if it's a fully component action-based application, you're right—all those URLs will be dynamically generated and useless to you.
In this case I obviously cannot use any URLs extracted from the HTML
since they are dynamically generated on each request, but I read
something about being able to access WebObjects components directly if
the security settings have not been set to disallow this…
That's talking about getting a component to render directly from its template with some restrictions:
As you note, the application can easily prevent it from happening at all.
As mentioned on p.53, the user input and action-invocation phases of rendering the component are skipped, which probably means this approach would be limited to rendering a component that didn't have any dynamic content anyway. This might be of some very limited use to you, though you'd need to know the component names you were interested in, and they wouldn't normally be exposed anywhere.
I'm not sure you're going to find anything better than the types of high-level functional approaches you've already suggested above, such as automating at the browser level with Selenium. If what you need is REST-style direct addressability of resources within the application, you're not going to get that unless you can re-write the application to use direct actions or ERRest where you need them.
A little late, but could help.
I use the Apache's mod_ext_filter (little modified) to pre/post filter the requests/responses from our WebObjects application. The filter calls PHP scripts and can read the dynamical hyperrefs and other things from the HTML pages. The scripts can also modify the HTTP requests, so we can programatically add/remove parameters from the request to implement new workflows in front of the legacy app and cleanup the requests before they will reach WebObjects. It is also possible to handle an additional database within the scripts and store some things over multiple requests.
So you can get the dynamically created links (maybe a button's name or HTML form destination) and can recognize these names within the request.
It is also possible to "remote control" such applications with little scripts like "click on the third button on the page". The only thing you need is a DOM parser to get the structure of the HTML pages and then rebuild the actions which the browser would do (i.e. create the HTTP request manually and send it as POST to the extracted form destination href). The only problem is the Javascript code, which we analyze and reprogram within PHP (i.e. enable/disable input elements, so they will not be transmitted within the requests)
There were some problems within the WebObjects Adapter Module for Apache. It still uses Content-Length within the HTTP header, which you cannot change in mod_ext_filter. If you change the HTML or the parameters within the request, the length of the content will not longer match. But it is possible to change that.
Theoretically it could also be possible to control such an closed-source legacy application from a new UI on a tablet or smartphone, which delegates the user interaction to the backend WebObjects app.
The scripts depends on the page structure, so if your WebObjects app will be changed, you have to correct some things in the scripts (i.e. third button could be now the fourth button).
It should also be possible to add a Restful interface in front of the application and query the data from the legacy app by the filter scripts.
I own a Play 2.1 application.
Initially, I used the default template mechanisms from Play 2.1 until I .. learned AngularJS.
Now, I clearly want my client side to be an AngularJS app.
However, while surfing the net, I find there are no clear way to achieve it:
Letting Play behave as a simple RESTful application (deleting the view folder) and making a totally distinct project to build the view (AngularJS app initialized by grunt.js).
Advantage: Likely to be less messy, and front and backend teams would work easily separately.
Drawback: Need another HTTP server for the AngularJS app.
Try to totally integrate AngularJS app with the traditional Play's workflow.
Drawback: With a pretty complex framework like AngularJS, it would lead to a confusion of templates managementfor instance : scala.html (for Play) / tpl.html (for Angular) ... => messy.
Making a custom folder within the play project but distinct from the initial folders created by the Play scaffolding. Let's call it myangularview instead of traditional view for instance. Then, publish static contents generated by grunt.js into the Play's public folder, in order to be reachable from browser through Play's routing.
Advantage: SRP between components is still fairly respected and no need to use another light HTTP server for the client-side like in 1.
I pointed out my own views of advantage and drawbacks.
What would be a great way to achieve the combination of Play with Angular?
Yes, I'm answering to my own question :)
I came across this way of doing:
http://jeff.konowit.ch/posts/yeoman-rails-angular/
Rails?? No matter the framework is, the need remains exactly same.
It advocates a real separation between APIs (backend side), and front-end side (in this case making AJAX calls to backend server).
Thus, what I've learned is:
During development phase, a developer would use two servers: localhost on two distinct ports.
During production phase, the front-end elements would be encompassed into the whole backend side (the article would deal with a kind public folder, aiming to serve static contents: HTML, angular templates (for instance), CSS etc... Advantage? => dealing with a one and unique serving server's APIs exposition as well as static assets for the UI.
With this organization, some tools like Yeoman would be able to bring some really awesome handy things to developers like for instance: the livereload feature. :):)
Of course, during development phase, we end up with two different domains, (localhost:3000 and localhost:9000 for instance) causing issues for traditional ajax requests. Then, as the article points out, a proxy may be really useful.
I really find this whole practice very elegant and pleasant to work with.
There was an interesting discussion on the play mailinglist a couple of days ago about frontend-stack/solution, could be something in it for you, quite some people using angular it seems: https://groups.google.com/forum/#!searchin/play-framework/frontend/play-framework/IKdOowvRH0s/tQsD9zp--5oJ
For an intranet application instead of using a proper web server, it has been planned to just deploy static html pages by sharing it over the network. This static html page is continously updated in the background by a program running on that PC and people access this html page by typing the shared file path directly in the browser. A maximum of 10 simultaneous users are expected to access it.
Can any one see the disadvantages or advantages in this type of deployment?
The problem is that this is what Web server side programming languages are designed to solve. One disadvantage is that this makes the process overly complicated. Web servers can be pretty light weight if the issue is not wanting extra administrative overhead. It sounds like you'd be writing a lot of data unnecessarily. I'd suggest using some Web based programming language/platform like PHP, Perl, ASP.NET, etc to just serve up content.
I am working on a experimental website (which is accessible through web browser) that will act as a front-end to a restful interface (a sub-system). The website will serve as an interface between a user and the restful interface, as it will make http requests to the restful interface for almost all database operations. Authentication will probably be done using openid and authorization for the database operations will be done via oAuth.
Just out of curiousity, is this a feasible solution or I should develop two systems that accesses the database in parallel (i.e. the website has its own data access logic, and the restful interface has another data access logic)? And what are the pros/cons if I insist on doing it this way (it is just an experiment project for me to learn things like how OpenID and oAuth work in real life anyway) besides there will be more database queries and http requests generated for each transaction?
Your concept sounds quite feasible. I'd say that you'll get some fairly good wins out of this approach. For starters you'll get a large degree of code reuse since you'll be able to put other front ends on top of the RESTful service. Additionally, you'll be able to unit test this architecture with relative ease. Finally, you'll be able to give 3rd party developers access to the same API that you use (subject possibly to some restrictions) which will be a huge win when it comes to attracting customers and developers to your platform.
On the down side, depending on how you structure your back end you could run into the standard problem of granularity. Too much granularity and you'll end up making lots of connections for very little amounts of data. Too little and you'll get more data than you need in some cases. As for security, you should be able to lock down the back end so that requests can only be made under certain conditions: requests contain an authorization token, api key, etc.
Sounds good, but I'd recommend that you do this only if you plan to open up the restful API for other UI's to use, or simply to learn something cool. Support HTML XML and JSON for the interface.
Otherwise, use a great MVC framework instead (asp.net MVC, rails, cakephp). You'll end up with the same basic result but you'll be "strongerly" typed to the database.
with a modern javascript library your approach is quite straightforward.
ExtJS now has always had Ajax support, but it is now able to do this via a REST interface.
So, your ExtJS user interface components populate receive a URL. They populate themselves via a GET to the URL, and store update via POST to the URL.
This has worked really well on a project I'm currently working on. By applying RESTful principles there's an almost clinical separation between the front & backends - meaning it would be trivial undertaking to replace other. Plus, the API barely needs documenting, since it's an implementation of an existing mature standard.
Good luck,
Ian
woow! A question from 2009! And it's funny to read the answers. Many people seem to disagree with the web services approach and JS front end - which has nowadays become kind of standard, known as Single Page Applications..
I think the general approach you outline is quite feasible -- the main pro is flexibility, the main con is that it won't protect clueless users against their own ((expletive deleted)) abuses. As most users are likely to be clueless, this isn't feasible for mass consumption... but, it's fine for really leet users!-)
So to clarify, you want to have your web UI call into your web service, which in turn calls into the database?
This is exactly the path I took for a recent project and I think it was a mistake because you end up creating a lot of extra work. Here's why:
When you are coding your web service, you will create a library to wrap database calls, which is typical. No problem there.
But then when you code your web UI, you will end up creating another library to wrap calls into the REST interface... because otherwise it will get cumbersome making all the raw HTTP calls.
So you essentially created 2 data access libraries, one to wrap DB and the other to wrap the Web service calls. This basically doubles the amount of work you do, because for every operation on a resource, you will end up implementing in both libraries. This gets tiring real fast.
The simpler alternative is to create a single library that wraps access to the database, as before, then use that library from BOTH the web UI and web service.
This is assuming that your web UI and web service reside on the same network and both have direct access to the backend database server (which was the case for me). In this setup having both go directly to the database is also a lot more efficient then having the UI go through the web service.