Historically one of the reasons is that search engines can’t be sure how and to what degree search users will be able to replicate the same result.
So in worst case search engines would cloak users – read and index content on a site, that the search users following a link from the search engine to a site cannot see. Naturally they don’t want to give this kind of bad experience to search users.
As I have pointed out before you always need to read between the lines when you listen to Google. I would not go as far as to call Google liars but it is a fact that …
- Very few of the thousands of people working on the many Google products know anything about search – and even less about SEO.
- Google is not in the SEO business – they are in the search engine business. Its not the same! Their job is to build a great search engine and search user experience. Our job as SEOs is to market our sites. Many of Googles recommendations are therefore naturally more focused on what we as webmasters can do to help them build a better search engine. Fair enough. But just don’t believe that is the same as the truth about how you get the best possible results in SEO.
Jeff Whelpley is one of Googles leading engineers working on the Angular team. He said on the Angular U conference, in June 2015:
If you search for any competitive keyword terms, it’s always going to be server rendered sites. And the reason is because although Google does index client-side rendered HTML, it’s not perfect yet and other search engines don’t do it as well. So if you care about SEO, you still need to have server-rendered content.
- Fetch and render via Google Search Console – does it render properly?
- Is URL indexed by Google?
- Is URL’s content visible in Google’s cache?
- Are links displayed properly in Google’s cache?
- Search for unique content from framework’s page.
- Check if ”*framework*/test/” URL was crawled.
Among the key findings was:
- The fact that Google Cache content doesn’t mean it is also indexed!
- Google Search Console’s Fetch and Render doesn’t work the same as Googlebot
You can read the entire experiment here but I recommend that you start by reading my comments below.
I think they did a great job with this experiment. However, there are a few minor flaws and recommendations that I would like to comment:
- Also, in addition to the above you cannot base a test on “security by obscurity” – that because they told noone about test URL’s noone found them,accessed them or in worst case linked to them or submitted them to Google. Obscurity is not security.
Regarding my first two comments it is actually not so important. The least interesting part of this experiment is if Google read and follow the link. Not that it is not important that they do, but if they read the content, they will will in all likelihood also see the link. I am not so nervous about this.
How about Bing, Yandex etc?
The experiment was entirely focused on Google. Being the most dominant search engine in most regions that is understandable but you should not forget about Bing, Yandex if you are in Russia or other local search engines that may be important in your target region.
Repeating the test reveal some very surprising results!
So I decided to check some of Bartosz Góralewicz results …
Below you see the original chart with the results from the study.
I am mostly interested to find out if the URL get indexed, cached and the content is indexable. So I tested this again. Here is my results.
As you can see my results do not entirely match the results Bartosz Góralewicz got. Actually, it is worse than what he got. I am not saying there is anything wrong with his data. I am sure this is what he got but I can also assure you the above is what I got – when I tested.
That is just not good enough for you to base your SEO strategies on!
I have seen the opposite before – that Google have cached a text that is not indexed and searchable but not this other way around.
But as with almost all SEO there is a good explanation for this . I will explain, but lets first look at two different examples of this …
If you look at the code behind the page it is easy to see why the above is happening. The text is with React loaded directly in the code and stored in a variable (Hello) and then “inserted” into the DIV ID “app-root”.
With Angular 1 I noticed the same thing – content not being in the Google cache but indexed and searchable.
However, with Angular 1 the text is not embedded directly in the code as it is with React 1 – Google can’t just read it by looking in the code. Instead, the text is found in the external JS file /script.js that is linked to from the embedded code.
So, apparently, Google is – in my tests, reading the texts they can see directly in embedded code or external files, but is not executing the script and inserting the texts in the right placeholders. So it do not appear in the cache but is searchable.
If this type of “code indexing” is good enough to rank for anything competitive I did not test but I suspect not.
Its actually similar to Flash or frames. There are applications where it may be useful – SEO is just not one of them.