Ricci Street < Port 80 < Customhouse || search | sitemap | help gazette | theater | bistro

|

Caching other pages Ricci Street's Site Traffic

What happens

You're online at home using an account provided by your local ISP. You dial in, you are unobtrusively given an IP number for your session, and off you surf. When you request this Ricci Street web page, the request goes through your ISP first. To keep bandwidth down, your ISP may well cache web pages that customers have asked for recently. If this page is cached, your ISP's computer will ask the Ricci Street computer when this page was last updated. If the most recent version is cached, you'll see it off your ISP's server, which is quicker and cheaper. There's absolutely no difference in the page and everyone is happy.

Except that the full transaction is not recorded in the Ricci Street server logs. According to the W3C, which wrote the protocol:

If the client has done a conditional GET and access is allowed, but the document has not been modified since the date and time specified in If-Modified-Since field, the server responds with a 304 status code and does not send the document body to the client.

The purpose of this feature is to allow efficient updates of local cache information ... without requiring the overhead of multiple HTTP requests ... and minimizing the transmittal of information already known by the requesting client (usually a caching proxy).

the problem

The Ricci Street server logs undercount. By how much? I don't know. However, so do every other server's logs, and the undercount is consistent.

the cause

Not only are the undercounts gross, but I don't know how gross they are. A recent article on the ClickZ Network, Cold Hard Cache, by Keith Pieper, explains the problem:

No one can be exact, but NetRatings figures that 30 percent of content comes out of browser cache. ... According to caching appliance vendors, ISPs and corporations can save anywhere from 10 to 80 percent in bandwidth reduction just by employing caches. ... Factor in increased customer satisfaction from faster connections.

Pieper concludes, "Caching creates this hidden black hole in the internet that you can't control and have no clue how to get around."

Does caching affect toLearn.net? No question. If you look on the host lists of the marketing and math logs, you'll see proxy servers from AOL, corporations, and non-U.S. universities.

Caching is becoming more common. As Michael Dillon notes in the January 4, 1999, Internet World:

In other countries, such as Australia, not only are cache servers already universally deployed, they are also connected into a national Web caching hierarchy using the Inter-Cache Protocol. By year's end, North America will have a similar architecture -- in effect, a caching network stop the Internet.

a solution

Aggregate stats are certainly comparable from month to month to show overall trends. Don't ask the server logs to do too much. Keep in mind the undercount when drawing conclusions about traffic.