2001: A Browser Odyssey
December 31, 2001
Feedback Farm
Have something to say about this article? Let us know below and your post might be the Post of the Month! Please read our Official Rules and Sponsor List.
Forums
Want to dig even deeper? Post to the new MacEdition Forums!
Like almost every other Web content producer over 30, I started off in another field. Without giving too much away, that field involves a lot of number-crunching. And those skills are presumably what makes me frustrated with the quality of freely available browser stats. I don’t think they measure things properly. I don’t think they are adequately counting certain minor browsers. And I wonder if their audiences, biased by their own admission, tell us enough about the browser usage of other audiences. So I do my own.
It’s no wonder so many Web developers take a
conservative line when making choices about what technologies to support,
believing those 1998-vintage books that tell you it’s too early to do
anything with CSS because of
all the Netscape 3 users out there, or that kludgy, non-standard
workarounds are necessary to make your site look good for the majority of
browsers. But sometimes the truth will set you free. As would be obvious
from some of my
earlier columns, I put considerable effort into analyzing
MacEdition’s logs and understanding our audience. That’s why I
know that nearly 80 percent of our pageviews come from users of
reasonably modern browsers with good CSS support, and 60 percent from
browsers that get the box model right. That means that the majority of
users will benefit from standards-oriented design practices, and the rest
can be accommodated relatively easily, too. It’s also why I always
knew that accessibility for users of text browsers and non-standard devices
is an issue. I can’t rely on testing in the big two browsers; the
data say that MacEdition’s audience is way more diverse than that.
Doing my own detailed log analysis has other benefits as well. By tracking browser usage week to week, I’ve also picked up some interesting trends. This year has seen some big changes in browser usage, suggesting that the one-browser Web has not yet arrived, but the standards-friendly Web may be just around the corner.
The first of these trends was the introduction of version 6 browsers, both by Microsoft and the Netscape/Mozilla team. (Opera 6 for Windows is in beta, but functionally it’s a version 5 browser according to current reports.) Netscape 6 / Mozilla had a slow start because of the crud performance of 6.0 and 6.01. I think a lot of people wrote off Netscape then. But as we saw in August, the improvements in Version 6.1 and its successors generated a lot more interest. Since August, we have found that Mozilla-based browsers including Netscape 6.x account for around 5 percent of our pageviews, and nearly 7 percent in early December. Adding in Konqueror accounts for another half a percentage point. (This share, and the others I’ll discuss in this column, are reported after excluding the obvious search engine spiders from the total.)
Internet Explorer 6 for Windows has also attracted a slice of our readers,
averaging around 5 percent to 6 percent of MacEdition’s
pageviews. Sites that are not focussed on Macs, and that therefore have a
higher proportion of Windows users in their readership, are presumably
attracting an even higher proportion of IE6. Strangely, if you looked at
theCounter.com’s stats for September
and October
– long after IE6 was released – you would be led to believe
that almost nobody is using IE6. I don’t think that’s true,
and I believe the real explanation is that theCounter.com hadn’t
properly updated its log analyzers to identify IE6. The big jump in
“Netscape (compatible)” you see in its logs is almost certainly
the uptake of IE6. Sure enough, in the November
and December
data, IE6 turned up with a large share, and “Netscape
(compatible)” fell again. Now you know why I don’t trust these
sorts of instant statistics. The increasing usage of Netscape 6, Mozilla
and IE6 means that many more users will see the benefit of Web design that
adheres to the W3C
recommendations.
The other big shift in browser usage this year was the collapse in usage of Netscape 4, currently the biggest roadblock to Web designers using a standards-oriented approached without proprietary hacks. From around 30 percent at the end of last year, more recently the multiple versions of Netscape 4 have been accounting for around 11 percent of MacEdition’s pageviews. Since Mac users tend to use non-Microsoft browsers more often than their Windows counterparts, it’s likely that other Web sites have seen something similar but perhaps not as striking. The users that remain seem committed to Netscape 4; in the last week of November 2001, of the 11.8 percent of our pageviews that were from Netscape 4, around 9.8 percentage points were from Version 4.7 or higher, and 7.3 percent was from Netscape 4.75 or higher. The newly released Version 4.79 already had about 1.5 percentage points of our pageviews, or one in seven of all Netscape 4 users. These people are downloading the most recent versions as they come out. This obliterates the idea that the usage of Netscape 4 is attributable to user inertia and unwillingness to upgrade. Never mind your anecdotes about old machines in public libraries; they aren’t the full story – these folk are using Netscape because they want to, and they are upgrading for bug fixes. Otherwise, the stats would show more Netscape 4.5 and 4.6, but that’s not what the numbers say. So do us all a favor, kids, and upgrade to a browser that doesn’t exhibit the biggest bug of all – being Netscape 4!
The early 4.0x versions that caused us so much trouble at launch have virtually disappeared out of the statistics. They are now accounting for around 0.1 percent of our pageviews – only detectable at all because we have tens of thousands of pageviews a week – whereas it was more like half a percentage point at the beginning of the year. In November, total Netscape 4 usage held fairly steady around a 12 percent share. However, there have been similar periods through this year, before this browser’s share resumed its marked decline. I predict that sometime in 2002, most Web authors should be able to start designing standards-compliant sites that degrade gracefully to a bland look in Netscape 4, instead of feeling obliged to hack things to look in Netscape 4 like they do in IE6 – the usage of Netscape 4 will have dropped to that rump of die-hards using Netscape 4.7999996.
On the Mac side, there have also
been some interesting developments. The introduction of Mac OS X has
changed the browser population significantly, and has probably been
responsible for a lot of the reduction in usage of Netscape 4. At
first, the only browsers available for the new operating system were a
preview version of IE5.1, and NeXTStep stalwart OmniWeb. OmniWeb also went
to Version 4.0 early this year, although on the basis of its support for
Cascading Stylesheets, JavaScript and the DOM, and HTML 4.0-specific features
like id and title attributes, it should be considered a third-generation
browser like, say, IE3 or Netscape 4. After a peak in the middle of
the year, OmniWeb has been losing traction amongst OS X users, or at least
the ones that read MacEdition. Of course, this could partly be because our
design is only marginally friendly to OmniWeb’s broken CSS implementation, or because I’ve said so many rude things about it
in the past. The recent betas of Version 4.1 have added background
colors for DIV tags as well as table cells, so MacEdition no longer looks
so stupid in this browser. Perhaps this will encourage a few people back to
OmniWeb. On the other hand, more recently the Mozilla family of browsers
has been gaining share amongst OS X users at the expense of OmniWeb.
What of the older browsers? Well, if your audience is anything like ours, you can forget IE3. It barely even makes it into our logs, at less than 0.1 percent for the whole year. The three weeks where it actually got to around 0.2 percent all looked to be spiders and robots spoofing their user agent tag. Similarly, Netscape 3 wobbled around one quarter of one percent earlier this year, and has dropped down to sub-0.1 percent levels more recently. Internet Explorer 4.x has also seen a collapse in usage from 4 percent at the beginning of the year to around 1 percent at the beginning of December. Don’t forget, most of these browsers aren’t meant to be fully Y2K compliant, so what they are doing in our logs on the eve of 2002 is a mystery to me. Cyberdog, the mid-1990s vintage OpenDoc-based browser from Apple, still appears in our logs – but it too is declining steadily, from about half a percent at the beginning of the year to a barely detectable level similar to that of IE3. All of these browsers will cope with basic table layouts and degrade gracefully when presented with CSS, and the early IE versions will do some CSS, too.
Meanwhile, text browsers like Lynx and WannaBe are consistently in the
logs, accounting for maybe 0.3 percent of pageviews each, and the AvantGo
browser for handhelds accounts for about 1 percent (although you have to be
careful of the caching peaks from its home servers before you take
AvantGo usage literally). This is yet another reason to design for
diversity, instead of nested-table messes that only work on big screens of
desktop computers.
Fake “old browsers” can be a real issue. One week in June, IE4.0 for Windows apparently jumped up to 4 percent of pageviews, after being around 2 percent in previous weeks and on a downwards trend all year. Now, 4.0 is not a widely used version on Windows; usage is almost all 4.01, not 4.0, so I was already suspicious. Further examination of the raw logs showed that all of those extra pageviews were coming from a single IP address, and it was clearly spidering through the site. There was no way that was a human reader of our articles. If I wasn’t so diligent about looking at the logs, I would have been misled into believing that older browsers account for a greater share of our traffic than they really do. I am sure that some of the Web pundits and authors who sneer at standards-based practices have been similarly misled.
Two minor browsers – Opera and iCab – have good enough standards support, despite some remaining bugs and omissions. Sadly, they have failed to gain traction amongst users, with each accounting for a fairly steady 2 percent of pageviews. This is a pity because both of them are very suitable for those users with older machines, and I have never once had iCab crash on me. I can’t say the same for the big two browsers.
If none of this evidence can convince you to use standards-compliant
markup, at least consider the big slice of user agents who really need
balanced tags, and really don’t need four thousand font tags and
nested tables – search engine spiders and other robots. I’ve
excluded their traffic from the pageview shares mentioned here, but they
account for as much as 10 percent of our traffic, and even more in
some weeks. Many of these spoof themselves as old versions of Netscape and
IE, especially InfoSeek’s Slurp spider. My Analog configuration file
has 75 separate aliases just to separate out all the robots (just append this file to your manconf.cfg
file to do the same). If you don’t have sensible markup practices,
or you do everything in JavaScript and Flash, you won’t have a
sensible entry into Google. And as our referrer logs show,
without that, you’re nothing.