Watching the World Wide Web
Web Growing Pains: The Price of Success
Maintaining a successful Web site can consume increasing amounts of money
and resources. Make sure you budget for both.
Many people judge the success of their Web site by how much traffic it generates.
But it may not be obvious that the more traffic you generate, the more money
it costs to keep your site current and your customers happy. In cyberspace,
this is the price of success.
More traffic generally forces you to use more people to run your site and
answer the questions you get from your customers; more horsepower for your
server to ensure that your site runs fast enough to keep up with the traffic;
and a faster communications line to ensure that customers can get through.
If your site is generating traffic but not a lot of revenue, selling these
additional expenses to management can be a difficult task, especially if
those expenses aren't budgeted.
When our site went live in January 1995, we had no idea how much traffic
we'd generate. When we were averaging 13,000 hits per day within two weeks
of going live, we were flabbergasted. Traffic was much higher than we expected.
However, sales were much lower. It seemed that people were visiting our
site, finding the information they wanted, and then returning to traditional
channels (bookstores) to buy the products. In itself, this was fine with
us; like other publishers, we like to promote our resellers. But it didn't
resolve our basic conflict. More cyber-traffic means more questions from
more readers and, hence, more people to answer those questions in a timely
manner. Our policy is to answer questions within 24 hours; cyber-customers
are not a patient lot.
Upgrading on the Fly
By June, we were averaging more than 30,000 hits per day. Through careful
load measurement and system performance statistics, we knew that it was
time to upgrade our server, a single-processor Sparc 20. We added a second
processor and doubled the RAM to 128MB. We also increased hard disk capacity
from 5GB to more than 10GB.
Obviously, you can't upgrade your server in the middle of the day, because
traffic is too high. That's the equivalent of closing a restaurant at lunchtime.
It's better to upgrade in the middle of the night on a weekend, even though
your system administrators might not like it. And when you upgrade, rather
than confusing customers by being completely off-line, use a spare server,
which could be as simple as a 486 running Linux. Our spare server, a 486,
broadcasts a message that says the site is currently down for upgrades and
asks you to visit again in a few hours.
The upgrade process could take only a few minutes, but it could take hours
if things don't go smoothly. Our preferred upgrade time is 2 a.m. on Sundays.
In the interest of the sanity of our system administration staff, however,
we tend to perform minor upgrades at 1 a.m. in the middle of the week and
save the middle of the night weekend chores for major upgrades that bring
the system down for long periods of time.
Increasing horsepower also required us to upgrade our staff. We hired two
more people in July to keep up with the traffic and updating the site to
meet the demand. The site's success made this a fairly easy sell to upper
It's important to continually update the site, or traffic will not return
and will not increase. If you're experiencing steady or declining traffic
over a long period of time, chances are you're not updating the material
on your Web site often enough.
By August, our traffic had ballooned to 100,000 hits per day. Because we
still didn't generate much revenue, we created a frequently asked questions
list to help users get answers online, without having to submit a question.
This measure saved us from having to hire an additional technical support
We redesigned our site in September to accommodate the addition of several
divisions of our company. We ran a test site in a hidden location on our
main server for several weeks, adding material and tweaking the new site.
When it came time to take the new site live, we had to change the URL of
the test site to the URL of our main site and take the old site offline.
We weren't sure how long this would take, but it required the services of
a programmer on the West Coast working with our system administrator. The
programmer agreed to come into his office at 6 a.m. Pacific time. To our
surprise, everything went smoothly, taking a total of only eight minutes.
By November, traffic had increased to 200,000 hits per day. Support is still
an issue, as is maintaining more than 20,000 pages of information about
more than 6,000 books and software products. To make our server run more
smoothly, we upgraded the operating system from Solaris 2.3 to Solaris 2.4.
We also added two secondary servers to handle traffic that puts unusually
heavy loads on our server. This includes majordomo requests, which generate
a lot of system overhead due to the fact that we have more than 40 mailing
lists on our server and high-bandwidth data, such as documents using Adobe's
PDF format and our experiments with new technologies such as Virtual Reality
Markup Language (VRML) and Java.
More, More, More
Traffic, by the way, has not increased at a constant rate. In the beginning,
there were periods of growth followed by periods of level or somewhat lower
traffic. I likened it to a stairstep approach. However, starting in June,
traffic began to increase dramatically and showed no sign of leveling off
until the 200,000 hit mark in October. Our system administrators constantly
monitor the site for performance to identify when the server and/or the
communications line needs to be upgraded.
Because the cost difference wasn't too great, we installed a T-1 line from
the beginning. A T-1 works at 1.544 megabits per second at a cost of less
than $2,000 per month. Our T-1 is currently at no more than 50 percent of
capacity through the day, so we feel that it can last for another few months
of traffic increases (traffic is currently increasing by about 40 percent
per month) before we have to add another, which we've budgeted for this
For next year, we've also budgeted an upgrade to a new server with faster
processors, using the Sparc 20 as a secondary server for more complicated
applications. And we're going to increase the size of our staff to help
us maintain our site.
I hope our experience demonstrates that it's crucial to put the costs of
a successful Web site into your budget. Otherwise, you could be setting
yourself up for failure.
Jordan Gold is publisher of online services for Macmillan
Digital USA, the multimedia arm of Macmillan Publishing, in Indianapolis,
IN. He can be reached at firstname.lastname@example.org.
The Web address is http://www.mcp.com.