As to the article, I feel I had to write what I had to write, as it is no good pointing this kind of thing in comments on other blogs.
I just don't want people to think that if they have used robots.txt or noindex follow for their duplicate content issues that they have "SEOed" their site.
igorthetroll
· 2 years ago
Andy, really I have seen many SEO bloggers chalenged this, including Rand Fishkin at SeoMoz and his grandfather Si, but by far you did the best job. You hit the nail right on the head.
Home Run
Patrick Altoft
· 2 years ago
To be honest my preferred method of removing content is sometimes just to cloak it and 301 googlebot to the homepage and let other users see it as normal (not recommended but not too dangerous).
Robots.txt isn't useful as you say because if a page has enough juice to get indexed you need to pass the juice to another page.
AndyBeard
· 2 years ago
For some of the links it is possibly a viable solution, but I am not sure I would want to do that for all the pages that many people suggest are blocked with Robots.txt or meta tags.
There is also a risk of diluting anchor text pointing to the home page.
Hobo
· 2 years ago
Nice post Andy, and yesterdays.
Andy I have a question that's going round my head but perhaps you have come across before.
If a page has 10 links going to 6 pages, (ie 5 links go to one page each and the other 5 go to the 6th) do you think the "link equity" is split 10 ways (with the 6th page getting the most) or is it split 6 ways evenly to the 6 unique urls?
Shaun
AndyBeard
· 2 years ago
It is something I have never tested myself, and I have seen conflicting opinions not only on number but also on placement.
Though this isn't a conclusive test, it will be interesting to see if the anchor text from this comment is passed to the home page.
Stephen Cronin
· 2 years ago
Andy, great post! I have a question: How Google treats feeds? Does it crawl XML? Does it know what feeds are and ignore them? Or does it actually pass PR to the feed, with PR passed onwards via links in the feed content? I link to my main feed on every page of my site. Should I nofollow that link?
Sorry if this is a 'silly question', but I really don't know the answer to this...
AndyBeard
· 2 years ago
Stephen, Feedbuner offer an option to have a feed blocked from search engines. I did it a while back as a test, but if most of the links from your feed point back to your own content, it isn't actually a major problem. Think: tags, categories, related posts, internal links
Links to comment feeds on every page are more of a problem. A while back it seems Google knocked them all into supplemental, and Matt Cutts I seem to remember stating that Google understands blog structure quite well.
I would still use add_link_attribute plugin on as many unnecessary features as possible, especially on navigation on those duplicate content pages.
chris
· 2 years ago
In terms of a blog
leaks for the same piece of content when using CMS systems such as Wordpress which create site-wide links in the sidebar when using poorly designed themes, plugins, and especially Wordpress Widgets.
So we should remove things like, categories, archives ect.. and use a noindex on the frontpage to improve our seo?
AndyBeard
· 2 years ago
Chris there isn't a single method of setting up a blog for SEO, a lot depends on how you want to place emphasis on particular content.
I have mentioned before that if you were a gambler, the following would apply.
The biggest problem is that different themes handle things link sidebar elements differently, and Wordpress widgets.
One of the key tools is the add_link_attribute plugin.
Eventually I hope to have some better solutions with a friendly interface.
Chris
· 2 years ago
Good lord there is a lot to learn. GREAT post. Thanks very much. Bookmarked.
Local SEO Guide
· 2 years ago
Quite a mind-blowing post Andy. Glad to see you are thinking overtime.
Lisa Stewart- Higher Rankings
· 2 years ago
Wow- Best post I have read in weeks. Really exceptional analysis, Andy- Thanks for raising the bar!
Chip
· 2 years ago
I knew some of that stuff myself, but i wasn't aware of the robots.txt facts. Although it's more than obvious.
Good article indeed, this is an evergreen.
AndyBeard
· 2 years ago
Chip it wasn't obvious for me, and coming up with an explanation so that I wasn't ripped to shreds by lots of SEOs wasn't easy either.
John Sadler
· 2 years ago
Excellent reference to the Dan Thesis book. Thanks Andy!
Mike Taylor
· 2 years ago
Wow, great post. I am going to have to re-read this at least twice to take it all in. Thanks.
Seo Blog
· 2 years ago
Excelent post!
Drew Stauffer
· 2 years ago
Andy have you ever designed your own Wordpress theme for the public? One that is properly developed with SEO in mind?
AndyBeard
· 2 years ago
Drew it is a challenge I am working on but making it so it is usable by the majority of bloggers is something very hard to achieve.
Lord Matt
· 1 year ago
I have been working on a project called "the one theme" inspired by CSS Zen Garden. The idea would be to create a single markup that could converted to theme WordPress, Blogger, NucleusCMS, typo, etc and then the pretty stuff is done only by CSS. I'd love to create the "perfect" SEO base with that markup (or designed with SEO in mind). If only to separate the design and the HTML (making wordpress themes safer).
Michael Streko
· 2 years ago
Andy you in depth posts and views never cease to amaze. Great post & even better for the person who stumbled this (where i found it)
Carsten Cumbrowski
· 2 years ago
"Extensive use of Nofollow and other forms of dynamic linking are the only way to effectively prevent duplicate content pages in some way having a effect on your internal linking structure and juice flow. The Wikipedia page on Nofollow really isn't correct."
Hi Andy,
I contributed a substantial part of the article to nofollow at Wikipedia and have it on my watch list and keep it updated as much as I can and time permits me to do so.
Please elaborate what exactly is not correct in the current article. From what I take away from your post, could you argue that the article does not explain additional uses of nofollow, like the control of flow of linkjuice within your own website.
This could be mentioned and makes sense IMO. It would at least provide some positive aspects to the whole thing and that webmasters seem to make the best possible thing out of this new tool (in contradiction to search engines who's repurpose of the nofollow attribute causes more problems than anything else).
Thanks
igorthetroll
· 2 years ago
Roy, exactly! rel="nofollow" best SEO practises is to control internal flow of PR iteration - juice...rand says this in his post as well.
AndyBeard
· 2 years ago
Carsten the specific sections are these ones
nofollow is a non-standard HTML attribute value used to instruct search engines that a hyperlink should not influence the link target's ranking in the search engine's index. It is intended to reduce the effectiveness of certain types of spamdexing, thereby improving the quality of search engine results and preventing spamdexing from occurring in the first place.
I think it is currently clear that the scope is a little deeper now.
What nofollow is not for
The nofollow attribute value is not meant for blocking access to content or preventing content to be indexed by search engines. The proper methods for blocking search engine spiders to access content on a website or for preventing them to include the content of a page in their index are the Robots Exclusion Standard (robots.txt) for blocking access and on page Meta Elements that are designed to specify on an individual page level, what search engine spider should or should not do with the content of the crawled page.
I think this needs to be revised in light of what Matt said in the interview on Stone Temple.
Nofollow has been used for the control of juice since it was created almost 3 years ago, though references I know of are all on private email lists or private access web documents. am not sure how you can reference them but it would be more appropriate than referencing current discussion other than that Google is on record to say they are ok with it.
SEO Agency
· 2 years ago
This is the best post I've read in months! Absolutely brilliant collection of gems, organised in a readable and useful way. I visit here every now and again, but for what it's worth you've won my RSS subscription, andy!
James Lee
· 2 years ago
I must admit that I'm coming around to understand Google's position on the whole paid linking situation. Their seems to be a cottage industry evolving around manipulating back links and page rank.
Sites like PPP are really harming the whole search results for everyone. I find that I might want to do a write up on one of my blogs talking about someone's site and linking to them. This situation ruins that.
In reality this all has derailed what page rank was supposed to be about...getting credit for others linking to your site honestly...which can't really happen now.
Graham
· 2 years ago
Thanks for the article Andy,
The whole duplicate content issue is a really difficult one to deal with - the worst thing is you never know when the algorithms are gonnna change and you have to re-think everything again.
cheap laptops
· 2 years ago
Wow Andy. Im not exaggerating when I say this has got to be the best post of have read in a very long time on an SEO blog (and I have all the main ones on my feed). This really changes my mindset on a lot of issues and opens my eyes to things that I may have been doing wrong in the past. Ive gota say congratulations on an excellent post. Keep it up.
Alex Gogan
· 2 years ago
Excellent post, have book market this and will be reading this a lot. It will be given in snips to my tech and desgin guys to follow. Thanks again.
Valentine
· 2 years ago
Thanks for putting the time together to write this. These longer posts are often the best.
Pet Lover
· 2 years ago
Wow. Definitely need to bookmark this one and read it over a second time after I get home from work. It answers a lot of questions I had about NoIndex pages. Very, very good post!
Thanks Andy!
Kamerovy
· 2 years ago
Nice article. Especially the part about Dangling Pages was very helpfull for me.
David DeAngelo
· 2 years ago
This is an eye opener. I always thought linking to external sites does not do any good to a sites/page pagerank. Infact, it seems now that we could be losing out by having dangling pages
Utah SEO Pro
· 2 years ago
Andy, I will have to agree with everyone else in stating that this is one of the best, if not the best, pieces of content I've read on here. I've relied on robots.txt for duplicate content issues but not anymore. Thank you!
Seotaal Seo Blog
· 1 year ago
Nice article, was very helpfull. Keep on blogging :).
Vinayak
· 1 year ago
Hi, I found your article very interesting and helpful. Made some changes to my blog following some of the advices. A big thanks!
Tim - Money Making Truth
· 1 year ago
Excellent in depth article, have had to make a few changes to my blog after reading this.
Jack Humphrey
· 1 year ago
This got my vote for the Semmy Award. Best one hands down! Great job.
AndyBeard
· 1 year ago
Much appreciated Jack, though making it to the final was reward in itself with such notable judges and so much competition.
gossipm
· 1 year ago
Quite a lot of info to take in and digest. Still trying to understand seo and how it works. This post was definitely helpful. Thanks.
Julie Francis
· 1 year ago
This no follow tag appears to be as dangerous as updating a plugin when you have a lot of other plugins, you never know quite how it's going to fit in.I think I'll wait another six months for the knowledge level to rise before I tackle no follow. Can you find and replace a whole blog then? Hell, I've got no rank to share so I may as well send my juice on.
Your excellent instruction is bookmarked though, I'll be back.
CloudStrife
· 1 year ago
Firstly , thanks for the wonderful pictures you displayed , I fully get what you mean very easily. It is definitely not an easy job to get so much traffic so easily . The most important item in a website is : Users
SEO Expert
· 1 year ago
Andy - once again you cease to impress with the effort and diligence to writing a great post. We are starting to use some of these tools and graphics for our internal training and documentation.
What comes as second nature to us as internet marketers is not often as easily grasped by our employers and clients. i can't tell you how many times I have pencil or white-board drawn the same chicken scratch tactical approaches for link structures and now I can just point them to your work here.
Thanks for taking such effort to put out awesome content and diagrams. I have bookmarked your site on some social bookmarking sites and will spread the word to all I work with in industry.
Best, Joe
Affordable Search Engine Optim
· 1 year ago
I had a customer who was linking to Wikipedia in almost every single post that he wrote. All dofollow links, of course. Your explaination of why to nofollow Wikipedia references (with a little creative addition or two) really helped him understand.
That is one of the better pages, certainly far better than the Wikipedia page. Unfortunately like everywhere else it didn't take into consideration dangling pages, which mess up everyone's thought process when they actually stop to think about what is happening.
Edward Beckett
· 1 year ago
@Andy
True ... However, If read very carefully ... we may deduce from Ian's writing that dangling pages are not a wise move ...
I try to use the implementation from his fifth example, which fully suggests that child pages should support the parent(s).
Unfortunately, this too leads to child pages that "sacrifice" their PR ... to the benefit of the parent ...
Блог о дизайне и
· 1 year ago
Very good article. Keep on eriting such helpful articles. I like to read english artickes because it more helpful then russian blogs :))
Indianapolis SEO
· 1 year ago
Wow Andy, Everytime I start to think I understand the game someone like yourself throws me a little "wrench/news" that reminds me that the world is not flat. Your argument on the issues of dangling pages is tough to agree 100% without just drinking the kool-aid.
zania
· 1 year ago
As so many others said above. This post is really useful and a real eye opener for someone trying her best to grapple with SEO and being thwarted at every turn with conflicting information. Like Julie, I think I'll wait a couple of months for the knowledge level to accumulate before I even attempt to tackle the pros and cons of no follow and dangling pages, so this post is bookmarked until then ;)
SEO Pune
· 1 year ago
Andy, great post..especially for SEO beginners
AndyBeard
· 1 year ago
Are you claiming to be an SEO beginner, as you made some of these mistakes with a number of pages on your site?
Gas Saver Guide
· 1 year ago
I'm glad there's Matt around to fill me in on this stuff all the time. Interesting little loopholes.
Trevor Stolber
· 1 year ago
This just goes to show the importance of testing in SEO. I actively test lots of things to get hard and fast answers to many of the things that remain speculative in SEO.
Cheap Laptops
· 1 year ago
Good information about SEO, and very valid information on how faster SEO works.
Nick
· 1 year ago
Great post - very informative and easy to read. I am still pretty new to SEO so have bookmarked your blog as I obviously still have lots and lots to learn. Thanks.
Webrevolve
· 9 months ago
Excellent post a very interesting read, i too have never been an Akismet fan what so ever, can't understand why it comes pre-packaged with wordpress, everytime i upload a fresh install of wordpress i forget that i like to remove the plugin from the wp-content folder
sethgoldstein (Seth Goldstein)
· 7 months ago
SEO Linking Gotchas Even The Pros Make: I am going to attempt to debunk almost every Wordpress SEO "Expert&.. http://tinyurl.com/2wsuds
emmaseoagency
· 5 months ago
as a seo beginner i appreciate your post very much, thanks!
All Comments
I also have a matter with Akismet.
http://www.phsdl.net/phsdl-vs-akismet-complaint...
Thank you,
Igor
As to the article, I feel I had to write what I had to write, as it is no good pointing this kind of thing in comments on other blogs.
I just don't want people to think that if they have used robots.txt or noindex follow for their duplicate content issues that they have "SEOed" their site.
Home Run
Robots.txt isn't useful as you say because if a page has enough juice to get indexed you need to pass the juice to another page.
There is also a risk of diluting anchor text pointing to the home page.
Andy I have a question that's going round my head but perhaps you have come across before.
If a page has 10 links going to 6 pages, (ie 5 links go to one page each and the other 5 go to the 6th) do you think the "link equity" is split 10 ways (with the 6th page getting the most) or is it split 6 ways evenly to the 6 unique urls?
Shaun
Though this isn't a conclusive test, it will be interesting to see if the anchor text from this comment is passed to the home page.
Sorry if this is a 'silly question', but I really don't know the answer to this...
I did it a while back as a test, but if most of the links from your feed point back to your own content, it isn't actually a major problem.
Think: tags, categories, related posts, internal links
Links to comment feeds on every page are more of a problem. A while back it seems Google knocked them all into supplemental, and Matt Cutts I seem to remember stating that Google understands blog structure quite well.
I would still use add_link_attribute plugin on as many unnecessary features as possible, especially on navigation on those duplicate content pages.
So we should remove things like, categories, archives ect.. and use a noindex on the frontpage to improve our seo?
I have mentioned before that if you were a gambler, the following would apply.
6+1=7
3+4=7
I have provided a number of solutions in my Wordpress SEO masterclass
http://andybeard.eu/2007/06/wordpress-seo-maste...
The biggest problem is that different themes handle things link sidebar elements differently, and Wordpress widgets.
One of the key tools is the add_link_attribute plugin.
Eventually I hope to have some better solutions with a friendly interface.
Good article indeed, this is an evergreen.
Hi Andy,
I contributed a substantial part of the article to nofollow at Wikipedia and have it on my watch list and keep it updated as much as I can and time permits me to do so.
Please elaborate what exactly is not correct in the current article. From what I take away from your post, could you argue that the article does not explain additional uses of nofollow, like the control of flow of linkjuice within your own website.
This could be mentioned and makes sense IMO. It would at least provide some positive aspects to the whole thing and that webmasters seem to make the best possible thing out of this new tool (in contradiction to search engines who's repurpose of the nofollow attribute causes more problems than anything else).
Thanks
I think it is currently clear that the scope is a little deeper now.
I think this needs to be revised in light of what Matt said in the interview on Stone Temple.
Nofollow has been used for the control of juice since it was created almost 3 years ago, though references I know of are all on private email lists or private access web documents.
am not sure how you can reference them but it would be more appropriate than referencing current discussion other than that Google is on record to say they are ok with it.
Sites like PPP are really harming the whole search results for everyone. I find that I might want to do a write up on one of my blogs talking about someone's site and linking to them. This situation ruins that.
In reality this all has derailed what page rank was supposed to be about...getting credit for others linking to your site honestly...which can't really happen now.
The whole duplicate content issue is a really difficult one to deal with - the worst thing is you never know when the algorithms are gonnna change and you have to re-think everything again.
Thanks Andy!
I found your article very interesting and helpful. Made some changes to my blog following some of the advices.
A big thanks!
Your excellent instruction is bookmarked though, I'll be back.
What comes as second nature to us as internet marketers is not often as easily grasped by our employers and clients. i can't tell you how many times I have pencil or white-board drawn the same chicken scratch tactical approaches for link structures and now I can just point them to your work here.
Thanks for taking such effort to put out awesome content and diagrams. I have bookmarked your site on some social bookmarking sites and will spread the word to all I work with in industry.
Best,
Joe
To add to the heirchial site design argument ...
Long, Long, Ago ...
Someone described in Great Detail ...
The Google PageRank Algorithm ...
And, how it works ...
True ... However, If read very carefully ... we may deduce from Ian's writing that dangling pages are not a wise move ...
I try to use the implementation from his fifth example, which fully suggests that child pages should support the parent(s).
Unfortunately, this too leads to child pages that "sacrifice" their PR ... to the benefit of the parent ...
Everytime I start to think I understand the game someone like yourself throws me a little "wrench/news" that reminds me that the world is not flat. Your argument on the issues of dangling pages is tough to agree 100% without just drinking the kool-aid.
Like Julie, I think I'll wait a couple of months for the knowledge level to accumulate before I even attempt to tackle the pros and cons of no follow and dangling pages, so this post is bookmarked until then ;)