Comments on: Not ALL Wikipedia Links NoFollow

By: Skitzzo

Skitzzo — Wed, 24 Jan 2007 23:42:19 +0000

Sorv, the links of your name are not nofollowed, only links entered in the comments and to be honest, I’m not sure where that’s set up. I’ll try and track it down though.

If anyone knows where to look please feel free to share.

By: AG

AG — Wed, 24 Jan 2007 18:59:06 +0000

Lea: Many wikis probably do; I’m not that well versed in the details. However, en.WP is the single most heavily used wiki around, by some sizable margin, and one of the ways to cope with the traffic is a very heavy layer of caching - it simplifies matters enormously if only a small proportion of pageviews require a database query, rather than them all.

Basically, the HTML rendering is done a stage earlier than you think it is. My understanding is that almost all pages the casual reader sees* are lumps of pre-generated HTML stored in the cache servers and sent out to readers; whenever the page is edited, or any “component” (templates, images) included into the page is itself altered, the HTML is regenerated from the source by a central server and sent to the caches.

But if you change the way the HTML parser renders the source, it doesn’t know to invalidate the existing HTML - it’s all been done, after all, and nothing has changed in the source. So old things like this can persist for a short while even when most pages have changed over, simply because the system hasn’t needed to regenerate them.

If you want to force the change, to see if it’s working okay, use http://en.wikipedia.org/w/index.php?title=PAGENAME&action=purge - this forces a purge of the cache - and reload.

As to untouched older pages periodically regenerating - I think they are, but I don’t know for sure. This would presumably stop it persisting *too* long, but cache servers can do strange things.

*logged-in users may be handled differently, I vaguely recall.

By: Trond Sorvoja

Trond Sorvoja — Wed, 24 Jan 2007 10:17:28 +0000

It is a server side cache issue, all exernal links has the fraking nofollow attribute. That being said, when are you going to remove the nofollow on the comments in this blog?

By: Lea

Lea — Wed, 24 Jan 2007 05:35:23 +0000

Not a cache issue?
I thought I saw the same thing on some pages, but when I checked again the next day they were nofollowed. I assumed it was my cache clearing, but who knows?

AG: I would have thought the point of a wiki was that every page was generated on the fly from the db?
Remember the editor does NOT enter anchor tags, but wiki markup, so all that needs to be done is alter the way the anchor element is generated from the wiki markup

By: AG

AG — Wed, 24 Jan 2007 01:10:48 +0000

To modify every “a href…” to include nofollow tags, Wikipedia needs to effectively regenerate the code for anywhere from one point six to three million pages. (Figures are a bit vague because I can’t recall if nofollow was already enabled on non-article pages). As that’d put undue strain on the system, it doesn’t do it at once; it does it when the cached copy of any given page is cleared out and refreshed. So if the page you’re looking at hasn’t been altered since the change - “sabotage” was last edited on Jan 12th - chances are it hasn’t been recached yet, and so the links haven’t been “nofollowed”.

Wikipedia (and MediaWiki generally) would indeed *like* to have some kind of selective enabling/disabling/whitelisting/whatever for nofollow tags and URLs; please send them any suggestions which can easily scale up to an active community of fifty thousand people and three million pages…