Blog
Can Google index javascript-generated content? Results
Last week we set up an experiment to see what kind of Javascript-generated content Google would index.
We created a test page with three kinds of content:
- Content loaded by simple inline javascript using document.write().
- Content loaded automatically through an Ajax call.
- Content loaded through an Ajax call in response to user action.
Our findings are as follows:
1. Inline Javascript using document.write(): YES – Google can (and does) find and index this content.
2 & 3. Content loaded through Ajax: NO – Google does not index this content.
Conclusions:
Content that is written to your document using Javascript’s document.write function will be indexed by Google.
But content that is loaded asynchronously, for example via an Ajax call, is not indexed.
If you have important content that you need to get the search engines to find, don’t use Ajax – or at least don’t rely on Ajax alone. If you’re going to use Ajax in your website or application, Google provides some guidelines for getting your Ajax content crawled.
Can Google follow Javascript links?
Last week we set up some experiments to try to answer the question “Can Google find (and index) pages with no inbound links?“
The results are now in and they’re quite surprising.
In short, Google is much smarter than we thought. It knows that <a> tags are not the only way to create a link. It can (and does) find and index content linked only through Javascript.
Click here to view the results in full.
Can Google find pages with no inbound links?
Generally, Google and other search engines find new pages to add to their indices by following links from one web page to another.
Some search engines, including Bing and Google, also allow webmasters to submit URLs directly, meaning that your site may get indexed even if there are no links pointing to it from the “outside world”. (Links like these are called “inbound links” in the trade.)
But we wanted to know: can Google ever find and index pages that have no inbound links, and that are never sent directly to Google via its site submission form?
We set up an experiment to find out.
In fact, the page you are reading right now is part of that experiment.
We wanted to test the following scenarios:
- A page with no inbound links, but whose URL is to be found in plain text. Here is just such an example:
http://www.adherewebdesign.com/experiment/orphans/url-only-no-link.php
- A page with exactly one inbound link, but which has the rel=”nofollow” attribute added. In theory such links are not followed or indexed by Google. But in practice? Let’s see. Here is our case-study:
http://www.adherewebdesign.com/experiment/orphans/link-nofollowed.php - A page with no conventional inbound links, but which can be accessed by clicking what you might call a JavaScript pseudo-link, like this: <a href=”javascript:document.location=’http://www.example.com/example.html’”>a javascript pseudo-link</a>.
Here it is in action: a javascript pseudo-link - Identical to number 3, but with the pseudo-link attached to something other than an html <a> tag.
Here it is in action, using a <span> tag: a clickable javascript pseudo-link - A page with no inbound links in pure html, but with an inbound link that is javascript generated, rather like .
- Finally, we wanted to test whether Google follows links from PDFs. So here is the PDF, which contains exactly one link to a page with no other inbound links.
Check back here in a couple of weeks to see the results of these experiments.
Does Google index content generated by JavaScript?
We were curious to find out: does Google read and index javascript-generated content?
Driven in part by the rise of the smartphone, websites are becoming more “appy”, with data and content loading asynchronously through Ajax calls.
So it’s becoming more important to find out whether your jQuery / Javascript / Ajax-loaded content gets indexed by the search engines.
We couldn’t find a definitive answer online.
So we thought: there’s one way to find out for certain: do an experiment. We’re interested in three different kinds of content, really:
- Content loaded by simple inline javascript using document.write().
- Content loaded automatically through an Ajax call.
- Content loaded through an Ajax call in response to user action.
We’ll post again in a few days’ time with the results.
Why I’ll watch any old crap
I’m often in arguments with a friend, who we’ll call Rob S, no wait, R Shaw, about online video. My friend is a top video producer, he owns a very good video production company that produces High quality video content in HD, broadcast quality and so on. We’ve had many discussions about how best to translate some of his services to online video. But we always argue about this.
Rob’s natural reaction is to focus heavily on quality of production, beautiful filming with high quality equipment, expertly lit, awesome sound quality and high definition. I frustrate him by telling him I really don’t care about that, and he doesn’t understand me. (sob)
I realise the truth with online video is at this point in time, if I see something perfectly filmed that isn’t grainy, that isn’t pixelated, that isn’t washed out, blurry or with sound not fit for purpose then I immediately distrust it, jumping to the conclusion this must be evil corporate advertising designed to strip me of my well earned money by luring me into buying the Slambaster ubercoolitude 3000 from an online retailer.
But if I see a handheld mobile phone video of a dog chasing deer across a motorway I’ll watch it (almost) every time. I believe it to be real you see, not that reality is what I necessarily look for in video, in fact fiction is my preferred format on television, but I am distrustful of internet video.
I must point out I do not mean the quality of streamed tv or movie online video should be poor, no, I expect and wonder at just how good this is becoming with modern broadband connections. But use of video on websites I automatically look for a level of authenticity, of realism, or spontaneity that can only come from someone passionate pointing a cheap filming device at their subject and feeling proud enough of the quality of the CONTENT to share with the world. No pretence, no editing or endless manipulation of subtext, just interesting bit sized content I can digest quickly and move on.
I’m tempted to point to a song about a pterodactyl (although technically not video) but I’ll let you find that for yourself.
Like me, like me, like me. I’m an interesting person, no really I am
If you heard something on the news, say for example that Paris was dimming its street lights to save energy, would you shout that snippet of information out at a party, proclaim yourself a leading expert on energy conservation and request social and global coverage for your ability to listen and repeat?
Imagine the conversation:
You – “Paris is going to dim its street lights a night by 6%, saving ten billion tons of fossil fuels a year.”
Marginally interested fellow guest – “is that so?”
You – “Yes. You like me now, please like me. Tell all your friends about me and my wondrous knowledge. In fact tell everyone you know, or meet in the street about how clever I am, how interesting I am and just how much you damn well like me. Also, tell others my ground breaking Paris fact, but be sure to quote ME when you tell them this information, then maybe they’ll like me too.”
Marginally interested fellow guest (now trying to leave) – “rrrriiiiigggghhhht. Got to go, forgot I’m having a new wheely bin delivered today.”
You wouldn’t repeat a snippet of common knowledge, proclaim it as your own (despite it being all over all news sources across he globe) and expect everyone you meet to hail you as the bringer of knowledge, the sooth sayer of modern society. You wouldn’t do it at a party, so why do it online?
I often get asked “should I be using twitter and Facebook on my website and getting likes and stuff”. My answer is fairly simple, if you have something new to share, something unique to say, or something interesting to post that puts something in a different light that people WILL find interesting then, yes, by all means share it, and enable others to do so too. If you are going to regurgitate stuff you’ve heard elsewhere, changing the wording just enough so you think you can fool Google into believing its original content then do not bother. Shouting like me, like me, share me, share me on poorly recycled non news is a sin. Don’t waste your time, and ours, filling up the internet with copycat garbage in an attempt to make search engines think you’re a fountain of knowledge.
If you want to be a fountain of knowledge, then be one, be unique, say something interesting and let the public do the rest.
Will Raspberry Pi revitalise the creativity of our Gen-Y kids???
Over recent months, the anticipation of the full release of ‘Raspberry Pi’ has grown to feverish levels. For those of you that don’t know, Raspberry Pi is a low-priced ($25-$35), credit card sized computer that can plug directly into your TV and keyboard, has the capability to play Hi-definition video, has USB and ethernet connectivity and runs utilising the Fedora (Linux) operating system. To quote from the official Raspberrypi website, “We want owning a truly personal computer to be normal for children” and “We want to see it used by kids all over the world to learn programming”.
In theory it’s a fantastic idea aimed directly at improving the computer science curriculum practised within our schools. A hark back to the days I remember using BBC Micros, C64′s and ZX Spectrums whereby you were encouraged, even forced to try your hand at basic programming if you wanted it to do the most straightforward of tasks. These days, our children are taught the in’s and out’s and up’s and down’s of MS Office and Social media websites. There is no doubt that for many kids, it will stimulate their curiosity and creativity as to what they can achieve but the cynic in me thinks that for the vast majority – the interest span will be very limited and the desire to rush home and jump on their Xbox, PS3 of Wii is simply too strong for our Generation Y offspring!
Is this the parents fault? Maybe. Is this the schools fault? Maybe. Has this forward thinking Uk based charitable company (Raspberrypi) left it too late to try and address this problem? Again, maybe. All schoolkids I see now have Smartphones, Facebook and Twitter accounts and even my own god-daughter has an ipad2 – at the age of 7!!! (Where’s mine I cry???)
Can we really get them to delve deeper beneath the shiny exteriors and to see how these things tick? I for one will encourage it and my order will be placed, but in reality, I fear that it will be Us, the parents, the children of the 80′s, who will end up making the most of this technology…….
How To Not Annoy People with Adwords Remarketing
Ever feel like certain adverts are following you round the internet, lurking on websites that are totally unrelated, jumping out at you when you’re not interested?
You’re right, they are, Google Adwords and many other advertising platforms make it easy to target you with ads based on sites you’ve visited, if you visits greenwidgets.com, a cookie can be placed in your browser so that adverts for green widgets can be served up to you on any other site that rents advertising space.
With my marketing hat on, I love this, it’s a great way to make sure that you show relevant ads to people who have already visited your site. But with my consumer hat on, I hate remarketing with a passion, just because I visited your site once for 10 seconds before I realised it was rubbish and left does not mean I want to see your equally rubbish adverts everywhere I go. Imagine going in to a shop, glancing round and leaving only to be followed around for the next week by people with advertising placards who jump in to your line of sight at every opportunity.
By all means use remarketing, but if you don’t want to irritate potential customers to the point where they view your brand as a creepy stalker, make sure you take the appropriate precautions.
1) Cap the number of daily impressions.
By default, there is no cap on the maximum number of times per day that a visitor will see your ads. Change this, do it now. What cap you place on the number of impressions is up to you, but I would suggest no more than 8 per day. Not only will this help avoid annoying people but if you’re paying CPM it will keep your costs down.
2) Vary your ads
Seeing ten different ads for the same product is slightly less annoying than seeing the same ad ten times. It also gives you useful data on which type of ads get the best CTR.
3) Don’t preach to the choir.
I use a certain well known hosted ecommerce solution. I’m already a customer and I like the product, I recommend whenever I’m asked, they really don’t need to advertise to me, and yet everywhere I go I’m harassed by the same ads.
Fortunately this is easy to avoid, you can set multiple audiences within adwords, so as well as the “visited my site once” group, you can have a “has an account” group. Now all you have to do is exclude the second group and your loyal customers won’t get bugged by ads for something they already buy and you won’t waste money doing it.
There is an exception; if you are selling a consumable item that people buy on a regular cycle – like razor blades – then you can target them, but be clever about it. Create a “people who bought razor blades” group with a cookie duration of a bit less than the average buying cycle and another “people who bought razor blades” group with a longer cookie duration. Include the second, exclude the first and you’ll be showing your ads to people who bought razorblades a while ago and are probably about to run out. Win.
4) Be specific
If I land on your site via a search for a specific product and you show me ads for a load of other stuff I never even looked at then the chances are I won’t click. Identify high traffic landing pages and create audiences specific to them, then show relevant ads. Your CTR will be higher and you won’t waste money showing ads for perfume to people who came to your site looking for spanners.
Adwords remarketing is a beautiful, powerful tool capable of intricate, finely detailed campaigns that show the right ads to the right people at the right time. Use it that way, don’t use it as blunt instrument.
Homepage settings for multi-language sites in Drupal
When creating multi-language websites in Drupal you can use a number of modules to provide languages and translations. These include the Locale and i18n modules. However you may get stuck wondering why, despite translating your homepage node (for example node/1) into another language, it does not switch to the translated node when you choose a different language – like the other pages of your site do.
This is because in the site settings you have one node set as the front page, and Drupal does not recognise this needs to be dynamic. So there is a small change needed to your settings.php file in order to use a multilingual variable as the homepage, instead of a fixed node.
Step 1:
In your settings.php file add the following code, this creates a multilingual variable:
This variable can be set up for each language when i18n is enabled.
*/
$conf['i18n_variables'] = array(
// Different front page for each language
‘site_frontpage’,
);
Step 2
Now go to your Admin > Site Information page and you will see that next to the front page box there is now a note that this is a multilingual variable:
This means you can set a DIFFERENT front page node for each language simply by visiting this settings page from within each language, i.e.
fr/admin/settings/site-information
en/admin/settings/site-information
us/admin/settings/site-information
etc…
Handy htaccess and htpassword access restrictions
Using a htaccess file to restrict access to your website can be extremely useful for both development and live environments. Here are a couple of my favourites:
Restrict access by IP address
This snippet only allows access to the site to people browsing from the IP address specified (given here as aaa.bbb.ccc.ddd):
order deny,allow
deny from all
allow from aaa.bbb.ccc.ddd
</limit>
Restrict access by password
This snippet goes in the .htpasswd file (not htaccess) and allows users to login to access the site:
It’s that simple!