2001: A Browser Odyssey
December 31, 2001
Want to dig even deeper? Post to the new MacEdition Forums (beta)!
Like almost every other Web content producer over 30, I started off in another field. Without giving too much away, that field involves a lot of number-crunching. And those skills are presumably what makes me frustrated with the quality of freely available browser stats. I don’t think they measure things properly. I don’t think they are adequately counting certain minor browsers. And I wonder if their audiences, biased by their own admission, tell us enough about the browser usage of other audiences. So I do my own.
It’s no wonder so many Web developers take a conservative line when making choices about what technologies to support, believing those 1998-vintage books that tell you it’s too early to do anything with CSS because of all the Netscape 3 users out there, or that kludgy, non-standard workarounds are necessary to make your site look good for the majority of browsers. But sometimes the truth will set you free. As would be obvious from some of my earlier columns, I put considerable effort into analyzing MacEdition’s logs and understanding our audience. That’s why I know that nearly 80 percent of our pageviews come from users of reasonably modern browsers with good CSS support, and 60 percent from browsers that get the box model right. That means that the majority of users will benefit from standards-oriented design practices, and the rest can be accommodated relatively easily, too. It’s also why I always knew that accessibility for users of text browsers and non-standard devices is an issue. I can’t rely on testing in the big two browsers; the data say that MacEdition’s audience is way more diverse than that.
Doing my own detailed log analysis has other benefits as well. By tracking browser usage week to week, I’ve also picked up some interesting trends. This year has seen some big changes in browser usage, suggesting that the one-browser Web has not yet arrived, but the standards-friendly Web may be just around the corner.
The first of these trends was the introduction of version 6 browsers, both by Microsoft and the Netscape/Mozilla team. (Opera 6 for Windows is in beta, but functionally it’s a version 5 browser according to current reports.) Netscape 6 / Mozilla had a slow start because of the crud performance of 6.0 and 6.01. I think a lot of people wrote off Netscape then. But as we saw in August, the improvements in Version 6.1 and its successors generated a lot more interest. Since August, we have found that Mozilla-based browsers including Netscape 6.x account for around 5 percent of our pageviews, and nearly 7 percent in early December. Adding in Konqueror accounts for another half a percentage point. (This share, and the others I’ll discuss in this column, are reported after excluding the obvious search engine spiders from the total.)
Internet Explorer 6 for Windows has also attracted a slice of our readers, averaging around 5 percent to 6 percent of MacEdition’s pageviews. Sites that are not focussed on Macs, and that therefore have a higher proportion of Windows users in their readership, are presumably attracting an even higher proportion of IE6. Strangely, if you looked at theCounter.com’s stats for September and October – long after IE6 was released – you would be led to believe that almost nobody is using IE6. I don’t think that’s true, and I believe the real explanation is that theCounter.com hadn’t properly updated its log analyzers to identify IE6. The big jump in “Netscape (compatible)” you see in its logs is almost certainly the uptake of IE6. Sure enough, in the November and December data, IE6 turned up with a large share, and “Netscape (compatible)” fell again. Now you know why I don’t trust these sorts of instant statistics. The increasing usage of Netscape 6, Mozilla and IE6 means that many more users will see the benefit of Web design that adheres to the W3C recommendations.
The other big shift in browser usage this year was the collapse in usage of Netscape 4, currently the biggest roadblock to Web designers using a standards-oriented approached without proprietary hacks. From around 30 percent at the end of last year, more recently the multiple versions of Netscape 4 have been accounting for around 11 percent of MacEdition’s pageviews. Since Mac users tend to use non-Microsoft browsers more often than their Windows counterparts, it’s likely that other Web sites have seen something similar but perhaps not as striking. The users that remain seem committed to Netscape 4; in the last week of November 2001, of the 11.8 percent of our pageviews that were from Netscape 4, around 9.8 percentage points were from Version 4.7 or higher, and 7.3 percent was from Netscape 4.75 or higher. The newly released Version 4.79 already had about 1.5 percentage points of our pageviews, or one in seven of all Netscape 4 users. These people are downloading the most recent versions as they come out. This obliterates the idea that the usage of Netscape 4 is attributable to user inertia and unwillingness to upgrade. Never mind your anecdotes about old machines in public libraries; they aren’t the full story – these folk are using Netscape because they want to, and they are upgrading for bug fixes. Otherwise, the stats would show more Netscape 4.5 and 4.6, but that’s not what the numbers say. So do us all a favor, kids, and upgrade to a browser that doesn’t exhibit the biggest bug of all – being Netscape 4!
The early 4.0x versions that caused us so much trouble at launch have virtually disappeared out of the statistics. They are now accounting for around 0.1 percent of our pageviews – only detectable at all because we have tens of thousands of pageviews a week – whereas it was more like half a percentage point at the beginning of the year. In November, total Netscape 4 usage held fairly steady around a 12 percent share. However, there have been similar periods through this year, before this browser’s share resumed its marked decline. I predict that sometime in 2002, most Web authors should be able to start designing standards-compliant sites that degrade gracefully to a bland look in Netscape 4, instead of feeling obliged to hack things to look in Netscape 4 like they do in IE6 – the usage of Netscape 4 will have dropped to that rump of die-hards using Netscape 4.7999996.
What of the older browsers? Well, if your audience is anything like ours, you can forget IE3. It barely even makes it into our logs, at less than 0.1 percent for the whole year. The three weeks where it actually got to around 0.2 percent all looked to be spiders and robots spoofing their user agent tag. Similarly, Netscape 3 wobbled around one quarter of one percent earlier this year, and has dropped down to sub-0.1 percent levels more recently. Internet Explorer 4.x has also seen a collapse in usage from 4 percent at the beginning of the year to around 1 percent at the beginning of December. Don’t forget, most of these browsers aren’t meant to be fully Y2K compliant, so what they are doing in our logs on the eve of 2002 is a mystery to me. Cyberdog, the mid-1990s vintage OpenDoc-based browser from Apple, still appears in our logs – but it too is declining steadily, from about half a percent at the beginning of the year to a barely detectable level similar to that of IE3. All of these browsers will cope with basic table layouts and degrade gracefully when presented with CSS, and the early IE versions will do some CSS, too.
Meanwhile, text browsers like Lynx and WannaBe are consistently in the logs, accounting for maybe 0.3 percent of pageviews each, and the AvantGo browser for handhelds accounts for about 1 percent (although you have to be careful of the caching peaks from its home servers before you take AvantGo usage literally). This is yet another reason to design for diversity, instead of nested-table messes that only work on big screens of desktop computers.
Fake “old browsers” can be a real issue. One week in June, IE4.0 for Windows apparently jumped up to 4 percent of pageviews, after being around 2 percent in previous weeks and on a downwards trend all year. Now, 4.0 is not a widely used version on Windows; usage is almost all 4.01, not 4.0, so I was already suspicious. Further examination of the raw logs showed that all of those extra pageviews were coming from a single IP address, and it was clearly spidering through the site. There was no way that was a human reader of our articles. If I wasn’t so diligent about looking at the logs, I would have been misled into believing that older browsers account for a greater share of our traffic than they really do. I am sure that some of the Web pundits and authors who sneer at standards-based practices have been similarly misled.
Two minor browsers – Opera and iCab – have good enough standards support, despite some remaining bugs and omissions. Sadly, they have failed to gain traction amongst users, with each accounting for a fairly steady 2 percent of pageviews. This is a pity because both of them are very suitable for those users with older machines, and I have never once had iCab crash on me. I can’t say the same for the big two browsers.
If none of this evidence can convince you to use standards-compliant
markup, at least consider the big slice of user agents who really need
balanced tags, and really don’t need four thousand font tags and
nested tables – search engine spiders and other robots. I’ve
excluded their traffic from the pageview shares mentioned here, but they
account for as much as 10 percent of our traffic, and even more in
some weeks. Many of these spoof themselves as old versions of Netscape and
IE, especially InfoSeek’s Slurp spider. My Analog configuration file
has 75 separate aliases just to separate out all the robots (just append this file to your
file to do the same). If you don’t have sensible markup practices,
sensible entry into Google. And as our referrer logs show,
without that, you’re nothing.
— CodeBitch (email@example.com) is the grumpy cow who does the HTML production for MacEdition. Despite her grumpiness, she wishes all her readers a very happy New Year. Read other articles by CodeBitch