It successfully fetches page itself, but then breaks on whatever page tries to load: images, styles, fonts, js files, API calls – whatever. Every time something different. And it says that resources are "Temporarily unreachable".
And sometimes it successfully loads and renders entire page with no errors.
Their doc says that "Temporarily unreachable" means that either server took too long to respond, or that fetch was cancelled "because too many consecutive requests were made to the server for different URLs".
The page I tested is completely loaded within 1.5-2s. Is it too long?
It makes 20 requests: 1 html, 4 css files (3 of them are 3-party font-face), 6 js files, 4 api calls (1 failed, it's intentional), 4 font files, 1 image. Total data size is 2.5Mb. Is it too much?
I checked every failed resource with their "robots.txt tester" – each of them is allowed for googlebot.
I don't have any noindex/nofollow directives nowhere on the site.
And I remind, that sometimes it just succeeds, like everything is ok.
With all of that, I have 3 questions:
Do I have to care of google's rendering at all? If I will just pre-render my html (with phantomjs or whatever) for googlebot, won't it be enough for normal indexing?
If I need google's rendering – do I have to care of that random fails? If at least sometimes fetch-and-render succeeds, maybe, it means that my site will be indexed normally?
If I have to care about that fails – what else do I have to do to make it work stably?! Such random behavior doesn't make any sense and doesn't give me any clues about that.
you may want to see this related post:
https://webmasters.stackexchange.com/questions/118727/what-else-can-i-test-when-troubleshooting-a-fetch-issue-in-google-search-console/118730#118730
My thinking is that the failed API call may have some bearing, or more likely, DNS or shared hosting issues because of the randomness of your Fetch results. Some people have reported better performance when having a robots.txt file (even if empty or just User-agent: * ), and others have found that it was just overuse of the fetch tool on their domain.
Related
I am working on the web vitals for a website and I was checking the Chrome Developer Tools the Network tab. The website loads fully, but I see that in the network tab, the server requests keep on increasing and the resources requested go up to 7.8MB and the website has a slider that keeps repeating in the network. How can I check why so many requests are made?
Here is the picture of the network tab of the website.
I see that the resource names are slide-X.jpg. Without seeing the website or its code, I can only guess that there's a carousel on the page that cycles through images. If the images aren't cacheable, they'd continue to be loaded over the network. Otherwise if they are cacheable, I'd expect to see no network requests at all or at worst a 304 HTTP "Not Modified" response code.
So I'd recommend confirming what kinds of widgets are on the page like a carousel with repetitive behavior and checking the cache control headers of static content like images to avoid the need to load the images each time. Personally, I think carousels are bad UX so I'd even suggest you consider removing it all together! Regardless, you should still cache your content more efficiently.
I added a few workbox.routing.registerRoute using staleWhileRevalidate to my app and so far it has passed most lighthouse tests under PWA. I am not currently using Precaching at all. My question is, is it mandatory? What am I missing without Precaching? workbox.routing.registerRoute is already caching everything I need. Thanks!
Nothing is mandatory. :-)
Using stale-while-revalidate for all of your assets, as well as for your HTML, is definitely a legitimate approach. It means that you don't have to do anything special as part of your build process, for instance, which could be nice in some scenarios.
Whenever you're using a strategy that reads from the cache, whether it's via precaching or stale-while-revalidate, there's going to be some sort of revalidation step to ensure that you don't end up serving out of date responses indefinitely.
If you use Workbox's precaching, that revalidation is efficient, in that the browser only needs to make a single request for your generated service-worker.js file, and that response serves as the source of truth for whether anything precached actually changed. Assuming your precached assets don't change that frequently, the majority of the time your service-worker.js will be identical to the last time it was retrieved, and there won't be any further bandwidth or CPU cycles used on updating.
If you use runtime caching with a stale-while-revalidate policy for everything, then that "while-revalidate" step happens for each and every response. You'll get the "stale" response back to the page almost immediately, so your overall performance should still be good, but you're incurring extra requests made by your service worker "in the background" to re-fetch each URL, and update the cache. There's an increase in bandwidth and CPU cycles used in this approach.
Apart from using additional resources, another reason you might prefer precaching to stale-while-revalidate is that you can populate your full cache ahead of time, without having to wait for the first time they're accessed. If there are certain assets that are only used on a subsection of your web app, and you'd like those assets to be cached ahead of time, that would be trickier to do if you're only doing runtime caching.
And one more advantage offered by precaching is that it will update your cache en masse. This helps avoid scenarios where, e.g., one JavaScript file was updated by virtue of being requested on a previous page, but then when you navigate to the next page, the newer JavaScript isn't compatible with the DOM provided by your stale HTML. Precaching everything reduces the chances of these versioning mismatches from happening. (Especially if you do not enable skipWaiting.)
I've been noticing a very quirky trend lately and I'm baffled by it. In the past month or two, I've begun to notice sites breaking without a referer header.
As background: you'll of course remember the archaic days where referer headers were misused to do a whole bunch of things from feature detection to some misguided appearance of security. There are still some legacy sites that depend on it, but for the most part refer headers have been relegated to shitty device detection.
Imagine my surprise when not one, but three modern websites are suddenly breaking without a referer.
Codepen: pen previews and full page views just break (i.imgur.com/3abXqsC.png). But editor view works perfectly.
Twitter: basically every interactive function breaks. If you try to tweet, retweet, favourite, etc. you get a generic no-descriptive error (i.imgur.com/E6tIKFo.png). If you try to update a setting, it just flat out refuses (403) (i.imgur.com/51e2d0M.png).
Imgur: It just can't upload anything (i.imgur.com/xCWpkGX.png) and eventually gives up (i.imgur.com/iO2UlR6.png).
All three are modern websites. Codepen was already broken since I started using it so I'm not sure if it was always like that, but Twitter and Imgur used to work perfectly fine with no referer. In fact I had just noticed Imgur breaking.
Furthermore, all of them only generate non-descriptive error messages, if at all, which do not identify the problem at all. It took a lot of trial and error for me to figure it out the first two times, now I try referer headers as one of the first things. But wait! There's more! All it takes to un-bork them is to send a generic referer that's the root of the host (i.e. twitter.com, codepen.io, imgur.com). You don't even need to use actual URLs with directory paths!
One website, I can chalk it up to shitty code. But three, major, modern websites - especially when they used to work - is a huge head scratcher.
Has anybody else noticed this trend or know wtf is going on?
While Referer headers don't "add security", they can be used to trim out attempts from browsers (that play by refer rules) which invoke the request. It's not making the site "secure" from any HTTP attempt, but it is a fair filter for browsers (running on behalf of, possibly unsuspecting, users) acting-as proxies.
Here are some possibilities:
Might prevent hijacked (or phished) users, and/or other injection attacks on form POSTS (non-idempotent requests), which are not constrained to Same-Origin Policy.
Some requests can leak a little bit of information, event with Same-Origin Policy.
Limit 3rd-party use of embedded content such as iframes, videos/images, and other hotlinking.
That is, while it definitely should not be considered a last line of defence (eg. it should not replace proper authentication and CSRF tokens), it does help reduce some exposure of undesired access from browsers.
I built a website for someone and I used https://gtmetrix.com to get some analytics, mainly because the wait time is huge (~20 sec) without having any heavy images. Please find attached a screenshot here:
http://img42.com/05yvZ
One of my problems is that it takes quite a long time to perform the 301 redirect. Not sure why, but if someone has a key to the solution I would really appreciate. At least some hints to search would be nice.
The second problem is after the redirection, the waiting time is still huge. As expected I have a few plugins. Their javascripts are called approx. 6 secs after the redirection. Would someone please show me some directions on where to search please?
P.S. I have disabled all plugins and started from a naked plain Twenty Eleven theme, but I still have waiting times during redirection and smaller delay after redirection.
Thanks in advance
But a few suggestions:
1 and 2.) If the redirect is adding noticeable delays; test different redirect methods. There are several approaches to this -- including HTML meta and server side (ie PHP) methods -- I typically stick to server side; if it's showing noticeable delays using a server side method, this may be a great indicator that your experiencing server issues - and may be very well your server all along causing your speed issues; contact your host provider.
3.) Take a look at the size of your media. Images and Video; also Flash if your using any. Often cases it's giant images that were just sliced / saved poorly and not optimized for web in a image editing software like PhotoShop. Optimize your images for web and re-save them at a lower weight to save significant on load time. Also, many cases nowadays and you can avoid using clunky images in general by building the area out using pure CSS3. (ie. Odd repeatable .gifs to create gradients or borders etc.)
I'm developing an offline web-app for a client of ours, designed to run on an iPad in airplane mode, mounted on a stand. It has no server-side dynamic pages, only a static HTML page, many JavaScript components to handle navigation and interactivity, and a bunch of small graphics assets. The whole website (static html + css + js + graphics) weighs exactly 8.3Mb.
I'm caching the whole site via an offline.manifest declared in my single HTML file, this manifest references absolutely all the files under the root directory, so that all files needed are cached.
I'm not using localStorage, IndexedDB or other offline-storage techs in my JS code. Apart from the "automatic" caching, I don't store anything on-device.
So atfer checking my webserver logs, when my client installs the webapp on its iPad homescreen, it downloads all the files once, then never downloads anything from my server afterwards. That's fine, exactly what he wanted in the first place : a full offline webapp.
Then, how comes that after several minutes of testing from my client, his iPad asks him to “increase local storage from 10Mb to 25Mb” ???
FYI, the app consists of a kind of quizz: one welcome screen, 19 question screens, one result screen ; the user can navigate backwards/forwards in the questions sequence, but they're created and nullified on-the-fly so as to minimize memory footprint. Anyway I don't believe this problem has to do with RAM access, only with "hard", permanent, cached storage.
I've noticed that with all apps, it's like the iPad has to realize it has everything, and waits a few seconds to realize that it's going to go over it's app limit.
it would be nice to have it default to a larger amount, or let you set it up with a larger amount to begin with.
Seems like my client doesn't have this problem anymore. As I'm not in direct, physical contact with him, I can't tell what he did to get rid of it.