The rise and fall of the Gopher protocol

It was mid-March 1992, and Mark McCahill had never been to San

Diego before. Back home in Minneapolis, the skies had been

dumping snow for six months, and would keep at it for several

more weeks. McCahill checked into the Hyatt Islandia, an 18-story

high-rise hotel overlooking Mission Bay. There were palm trees,

he recalls. Boy, was it nice.

McCahill was then in his mid-30s and managing the Microcomputer

centre at the University of Minnesota Twin Cities, which

facilitated the emerging use of personal computers on campus. He

and Farhad Anklesaria, a programmer in the centre, had been

invited to address the 23rd Internet Engineering Task Force

(IETF), an elite convocation of academics and government

officials from around the world who were literally deciding how

the internet should work.

The gods of the internet, McCahill says, though in other circles

they would have gone unnoticed. In his memoirs, another internet

pioneer, Tim Berners-Lee, describes these gatherings as people

in T-shirts and jeans, and at times no footwear. They would meet

in different small rooms and talk excitedly.

The internet then, as now, was a vast array of information stored

in random computers around the world, only there was no easy or

consistent access. It was difficult even to discover what was out

there - there were no good search engines. The most popular

protocol, or method of retrieving information from another

computer, was FTP (file transfer protocol), the primitive,

labour-intensive equivalent of knocking on someone's door and

asking if you could carry away his piano.

The IETF had been convening since 1986 to iron out these issues,

which had prevented the internet from becoming the Intergalactic

Network its originators had foreseen, instead remaining the

limited domain of physicists and the military. But this meeting

felt different. For the first time, the internet seemed on the

verge of going public.

On March 18, in a conference room of the hotel, Berners-Lee

presented one possible breakthrough: the World Wide Web. It was

evening. Many of the 530 conference attendees had already gone to

the bar or to dinner. To the curious who stayed behind,

Berners-Lee explained that the Web could be used to connect all

the information on the internet through hyperlinks. You could

click on a word or a phrase in a document and immediately

retrieve a related document, click again on a phrase

in that document, and so on. It acted like a web laid over the

internet, so you could spider from one source of information to

another on nearly invisible threads.

Two other programs with the potential to expand access to the

internet WAIS and Prospero were discussed in the same

session. In the reports of people who saw the presentation, the

Web did not come across as the best of them, or even as

particularly promising.

The next day, in the light of the afternoon, McCahill and

Anklesaria presented the Internet Gopher. It was simple enough to

explain: With minimal computer knowledge, you could download an

interface the Gopher and begin searching the internet,

retrieving information linked to it from anywhere in the world.

It was like the Web but more straightforward, and it was already

working.

In fact, most attendees needed little introduction to Gopher the

software had been out for months. It was the developers they were

curious about, the Minnesotans who had created the first popular

means of accessing the internet. People wed never met were

telling us how they were using our stuff and adding things to

it, McCahill says. We had no idea how big Gopher was going to

be until we experienced this firsthand and realised that growth

could be exponential for a while.

In the years that followed, the future seemed obvious. The number

of Gopher users expanded at orders of magnitude more than the

World Wide Web. Gopher developers held gatherings around the

country, called GopherCons, and issued a Gopher T-shirt worn by

MTV veejay Adam Curry when he announced the networks Gopher

site. The White House revealed its Gopher site on Good Morning

America. In the race to rule the internet, one observer noted,

Gopher seems to have won out.

McCahill's father was an executive for Conoco, the oil company,

which moved him around the country about every two years.

McCahill was in junior high when the family finally settled in

the Twin Cities. He graduated from the U in 1979 with a BA in

chemistry, spent a year studying effluent in rivers, and realised

he liked the computer analysis (heavy number smashing) more

than the chemistry itself (kind of dirty). So he took a job in

the Us Microcomputer centre, programming some of Apples first

personal computers.

The Twin Cities were a proto-Silicon Valley then, with a long

history of producing some of the worlds most powerful computers

at UNIVAC, Control Data Corporation, and Engineering Research

Associates, which supported the work of Honeywell, IBM, and other

local tech firms. The Minnesota Educational Computing Consortium,

or MECC, formed in 1973 to get computers in schools and created

software for them most famously The Oregon Trail. By the early

1980s, when only a fraction of schools in neighbouring states had

computers, there were about three or four in every public school

in Minnesota.

When McCahill began working in the Microcomputer centre, a turf

battle was heating up a religious war, McCahill calls it

between the high priests of computing who oversaw the Us

venerable mainframes, the enormous machines that once occupied

entire rooms, and the growing cadre of personal-computer

converts. Microcomputer guys were as far out of the mainstream

as you could get and still be a part of the Us computer centre,

McCahill says. Anklesaria, who had earned a doctorate in genetics

before gravitating to computer science, says the Microcomputer

centre was a splinter group when he joined in late 1987. The

mainframe was still the only thing the Mac was considered a

toy.

McCahill sided with PCs. The idea of democratizing access to

computing, putting computers in the hands of everyday people

that resonated with me, and that was part of all the early PC

stuff, he says. If you were interested in PCs, you just

absorbed that attitude by osmosis. It was in the air and the

water.

McCahill, who had long hair then that he now pulls into a

ponytail, spent his free time wind-surfing on Lake Calhoun, and

says the PC revolution looked like a good wave to ride it

would enable me to do what Ive tried to do ever since: take

technology that is cutting edge and get it to the point that its

palatable to mom and dad and English majors.

In the late 1980s, McCahill and Anklesaria developed the first

popular internet email system, called POPMail. It was partly for

selfish reasons, McCahill says. I wanted more people using

email so I didn't have to walk down the hall to my mailbox to

collect my phone messages on little slips of paper. Instead,

secretaries could send an email. For that matter, so could

callers.

At the same time, the U was determined to network its computers on

the internet in a so-called campus-wide information system, or

CWIS, and the schism was delaying development. By early 1991, a

committee of more than 20 department heads and computing

specialists had been meeting for months, producing a long list of

demands including the use of mainframe computers but zero

code. They had some complicated shit for doing searches that I

didn't want to do, McCahill says. Farhad didn't want to write

it at all.

But I had to show something, Anklesaria says. So he stripped the

program down to its simplest parts a basic protocol for making

information in one place available somewhere else. He cobbled it

together on a Mac, writing a server (a program enabling a

computer to serve up requested files) and a client (how most of

our computers are programmed, enabled to search for and request

those files). And I said, This stuff kind of works. Since we

had nothing else, we went for it.

McCahill pushed for a full-text search engine something we now

take for granted and borrowed the gist of one from a computer

system called NeXT, which had recently been invented by Steve

Jobs. We had this marriage of Farhads super-simple protocol for

saying give me a list of items, a menu, says McCahill, and my

thing of having a way of searching, and we glued those two

together.

It was plain text no pictures, given that modem speeds were so

slow. And it was organized like the one information source most

people alive in the early 1990s were familiar with: a library,

with similar subjects grouped together. You just pointed your

gopher, as the lingo went, to any Gopher site you wanted to

explore, and there you were, burrowing through the internet. It

was so simple that just about anyone could make it work, even an

English major.

It was one of the rare times when we both looked at each other

and said, Holy shit, weve got a really good idea here,

McCahill recalls. Anklesaria called it the Internet Gopher, a

triple play on words: the Us mascot, a critter that digs, and a

go-fer one who fetches. We figured that if we called it

Gopher, the committee couldnt complain, McCahill says. Its

the school mascot!

Sir Tim Berners-Lee he was knighted in 2004 was born in London

in 1955, less than a year before McCahill. His parents were both

mathematicians. While McCahill's father was being shipped from

Colorado to Oklahoma to Minnesota, Sir Tims parents were

developing the worlds first commercially available

computer.

Berners-Lee grew up with the internet, or at least the concept of

it. As a child, he built mock computers out of cardboard boxes.

He came home from high school one day to find his father writing

a speech on how computers might someday make intuitive

connections, linking information the way the brain uses random

associations to link thoughts.

He was a teenager when the internet began as the ARPAnet,

connecting a handful of computers at Stanford and other

universities. It was designed as a defense, a secure means of

communication should the Soviets destroy the American telephone

system, though its practical purpose was to allow scientists to

use the computing power at another facility for massive

calculations.

Berners-Lee earned a degree in physics from Oxford where he

built his first computer with parts from an old television, a

calculator, an electronics kit, and a car battery then worked

as a software engineer for a few years before taking a job with

CERN, the famous particle-physics lab in Geneva, Switzerland. By

then, in 1984, the internet had gone global; CERN was the largest

internet node in Europe.

Yet using the internet was problematic even within the CERN lab.

When you wanted information, Berners-Lee later recalled, often

it was just easier to go ask people while they were having

coffee. He decided he could do better. He had never forgotten

his fathers research on the brain, and when he wove the Web, it

was based on this holistic, serendipitous, strangely rewarding

experience of surfing from one vaguely related idea to the

next.

He finished a model of the Web in 1989, but for years it went

nowhere. The concept was too abstract and it only worked on NeXT

computers. But in early 1991, just as McCahill and Anklesaria

were conceiving the Internet Gopher in Minnesota, the first Web

servers outside of CERN were switched on.

After outlining the Gopher protocol, McCahill and Anklesaria

pulled together four programmers who worked in the Microcomputer

centre to write the software. McCahill wanted it in a hurry. Not

because he was racing Berners-Lee, but because he wanted it over

with. He wanted it before the next CWIS committee meeting, in a

month.

The programmers were young guys, mostly in their 20s and, like

McCahill, mostly huge Nirvana fans. Paul Lindner, a coding

wunderkind from northern Minnesota who was dubbed the Gopher Dude

for his evangelism, had long metal-head hair and signed Gopher

emails with lyrics like You have to spit to see the shine from

Babes in Toyland. Early Gopher servers were named Mudhoney,

Danzig, and Anthrax. The sole outlier in the microcomputer mosh

pit was Bob Alberti, a programmer who named a server Indigo, as

in the Indigo Girls.

The centre was in Shepherd Labs, a hulking cement building built

like a tank in 1968 on the Us Minneapolis campus, with concrete

floors and no windows. Early on, it was used for NASA materials

research. There were pipes with strange fluids running through

them, Lindner recalls.

The microcomputer team, in addition to developing software for the

U, ran a showroom for students and faculty interested in buying a

Mac, taught computer training classes, tested software, and

served as a help centre for people with PC problems walk-in and

call-in. Everyone would answer phones at least one day a week,

Lindner says, even if you were programming. That way you were

close to the pain you were inflicting on people if programmers

today still took calls, wed have more user-friendly

software.

Gopher, however, was claiming more and more of their time. It

became infectious, Lindner says. How can we build this into the

science fiction of our dreams, access to all the information in

the world, the library of everything

The team, in 36-hour sessions fueled by beer, pizza, and speed

metal, finished writing Gopher in about three weeks. They

installed the first computer running a Gopher server a Mac

SE/30, a little droid of a computer with an iPad-size monitor

built in in a narrow hallway between their offices and the

showroom, in a closet with metal shelves. It became known as the

Mother Gopher.

The committee meeting where the team first presented the Gopher

protocol was a disaster, literally the worst meeting Ive ever

seen, says Alberti. I still remember a woman in pumps jumping

up and down and shouting, You cant do that!

Among the teams offenses: Gopher didn't use a mainframe computer

and its server-client setup empowered anyone with a PC, not a

central authority. While it did everything the U required and

then some, to the committee it felt like a middle finger. Youre

not supposed to have written this! Alberti says of the groups

reaction. This is some lark, never do this again! The Gopher

team was forbidden from further work on the protocol.

After the meeting, McCahill leaned on the director of the computer

centre, a Chinese-American man named Shih Pau Yen, who had

supported Gopher all along. I said I would quit before I stopped

working on the coolest thing wed ever created, McCahill says.

Yen ran interference, and the Gopher team kept working on it in

their own time.

In this bureaucracy of little fiefdoms, where everyone had a hard

time working together, the one thing that could unify us was

telling everyone else off, Lindner says. That was our rallying

cry.

Finally, in April 1991, still unable to persuade the U to take on

Gopher, Lindner released it into the wild. He made the Gopher

software available via FTP the most popular way to share

information on the internet at the time and wrote a brief,

quiet announcement on an internet mailing list: Hey, weve got

this thing, come and get it.

Within months, the team was hearing from Gopher users around the

country. It was the first viral software, Alberti says. All

these people started calling the U and pestering the president

and other administrators, saying, This Gopher thing is great,

when are you going to release a new version And the

administrators said, What are you talking about

The ban was lifted. Never numbering more than six core members,

the team fanned out to conferences to spread the Gopher gospel,

while continuing to improve and diversify the protocol. In the

process they ended up laying the foundation for much of how we

navigate the internet. The first hyperlinks. The first bookmarks.

McCahill, thinking of windsurfing, even coined the term surf the

internet.

A video of veejay Adam Curry wearing an Internet Gopher World

Tour T-shirt on MTV.

Within a year, there were hundreds of Gopher servers. Berners-Lee,

who had publicly introduced the World Wide Web a few months after

Gophers debut, used Gopher to do it. People look at the World

Wide Web today and think it sprang out of Tim Berners-Lees

forehead, Alberti says. But the fact is, the only way he was

able to spread the word about the Web is because the Internet

Gopher was there to allow people to download his files, find a

discussion group, and talk about it.

We had the right product at the right time, McCahill says.

People were looking to expand the internet beyond physicists

stuff. Gopher could do that. It was simple to use, it could

network lots and lots of computers. It gave people a reason to

say, hey, this internet is good.

Lindner was solicited for side gigs twice he went to Ecuador to

set up Gopher for the countrys fledgling internet. Thats when

I knew we were really onto something, he says, when I was

helping wire a whole country.

Al Gore, then a U.S. senator, came to visit. Four GopherCons, held

between 1992 and 1995, drew reps from the New York Times, the

World Bank, Microsoft, and other global heavyweights. The Gopher

T-shirt, black and scribbly, listed the names of places with

Gopher servers on the back, in the style of rock tour shirts. It

was an apt metaphor, as Gopher team member Daniel Torrey told the

Pioneer Press in 1996: We thought we were rock stars.

Some team members dreamed of fortune to go with their fame. But

the internet was not yet open for business. It had been built on

dot-mil and dot-edu, on public funds. Programmers shared source

code; if you needed something, someone gave it to you. A dot-com

address was considered crass. It was as though all of TV was

PBS, Lindner says. No commercials.

Still, Alberti raised the profit potential of Gopher with Shih Pau

Yen. Before coming to work in the Microcomputer centre, Alberti

had helped create the first online multi-player role-playing game

called Scepter of Goth, an ancestor of World of Warcraft and

the like. I said we should take this private, we should make a

business of this and make some money off it, Alberti recalls.

He looked at me like Id just grown another head.

Eventually, though, the U did want some money for itself. At

GopherCon 93, Yen announced that for-profit Gopher users would

need to pay the U a licensing fee: hundreds or thousands of

dollars, depending on the size and nature of their business. Many

users felt betrayed. In the open-source computing spirit of the

day, they had contributed code to Gopher, helping the team keep

up with the times. Now they were being asked to pony

up.

The reaction deflated the team. As hard as they were expected to

work on Gopher, they were never relieved of other duties,

including answering the Us help line. We never got additional

funding, we were going broke, Alberti says. We had the whole

internet yelling at us, when are you going to update your

software, when are you going to put images on Gopher pages, and

make this smoother and better And were like: Were six

guys!

For a while, the U threatened to get rid of them altogether, a bid

to outsource the universitys computer work, provoking one

programmer to bug a computer in Morrill Hall, the Us

administrative centre, so the team could listen in on

discussions.

Asking for a contribution seemed reasonable. When it backfired,

the team posted a defensive letter to Gopher users in March 1993:

There has been a lot of hysteria, misinformation, and rumor

floating around. In a time where we are having budgets slashed,

it is impossible to justify continued (increasing) resources

being allocated to Gopher development unless some good things

result for the University of Minnesota. This is a fact of life.

Before you go off and flame once more, ask yourself if you want

to get YOUR particular server going with as little fuss and

expense as possible or if you just want to stir up the

soup.

That socially killed Gopher, Alberti says of the licensing fiasco.

Yet it wasnt the end. In 1993, Gopher was still far more popular

than the World Wide Web, and Gopher traffic grew by 997 percent.

But the Web was starting to catch up that year, it grew by

341,634 percent.

At the San Diego internet conference, in 1992, Berners-Lee had

pulled McCahill and Anklesaria aside on the last day of meetings

and asked if they wanted to collabourate on a Web/Gopher hybrid.

Some melding of their different designs an internet

super-system though it wasnt clear how that might

work.

Tim is a great guy, but hes a little odd, a little scattered,

McCahill says. Talking to him is like a ball of twine

experience. The Web you see now, thats how he thinks.

McCahill told Berners-Lee that he would need to look at the Web

more closely. But it wasnt much to look at when McCahill went

back to Minnesota and examined it. There were no graphics yet. It

was still only running on NeXT computers. I wasnt feeling it,

McCahill says. I told him, Tim, I dont think so. Of course, I

look back and say, I might have been wrong.

Soon enough, the Web did have pictures and was available on more

platforms. In 1993, the first popular Web browser, Mosaic, was

introduced for sale, breaking the commercial taboo of the

internet and suggesting to McCahill at least that tech

investors had taken sides. The fix was in, he says.

In 1994, modem speeds doubled, and the interminable rendering of

images on the Web once dubbed the World Wide Wait greatly

accelerated. PCs began to be sold with these faster modems built

in. To anyone looking for a simple, even crude explanation for

the Webs rise, this is it: the ability to view a reasonable

facsimile of a naked woman in the privacy of your own home.

Thats what came to drive a lot of the internet, Alberti says.

Porn.

The Internet Gopher, with his text-only menu and gloss-less,

institutional mien, couldnt keep up. He had fallen off the wave,

and almost overnight was revealed as a buck-toothed square,

ignored by the girls on the beach, his surfboard held together by

duct tape. Obsolesced, as one observer put it in 1994. A

has-been.

I remember the exact moment I knew I was no longer on the right

track, says Lindner. It was September 9, 1993. I was invited to

give a talk about Gopher at Princeton, and I had my slides all

printed up on my little university-budget black-and-white foils.

The person presenting before me was talking about the future of

the Web, with full-color LCD projection. I said, I think I see

where things are going.

Mark McCahill's entry on the University of Minnesotas Wall of

Discovery, which celebrates the accomplishments of students and

faculty.

For McCahill, the realization happened on the street. I saw a URL

on the side of a bus, he says. Thats when I knew the Web was

all about advertising. Gopher was not good for advertising. I

knew it would start winding down.

In the spring of 1994, Web traffic overtook Gopher traffic for the

first time. Gopher, within a few months, began to decline. Its

reign had lasted three years.

The coup de grâce came from the U itself. In 1995, Dr. John

Najarian, a renowned transplant surgeon at the university, was

indicted for tax evasion, embezzlement, and fraud. Although

Najarian was later cleared of all charges, the National

Institutes of Health (NIH) put the U on probational status,

threatening tens of millions of dollars in grants and the

schools viability as a top research institution. It was an all

hands on deck situation, McCahill says.

The issue was essentially simple: bad accounting. Paperwork was

literally on paper, and transaction records sat around for weeks

before making it into a ledger, if they made it in at all. The

Gopher programming team was diverted to creating a more

accountable accounting system for the U, what turned out to be

the first Web-based transaction program. Thats what my kick-ass

development shop did for a year and a half, McCahill says, to

show the NIH that we were cleaning things up. By the time they

finished, the Internet Gopher was dead.

In the beginning, when the Mother Gopher was new and there were no

other Gopher servers to link to, Gopherspace was empty. The

Gopher team, to demonstrate the usefulness of their invention,

dumped in a cookbook and searched for eggplant. As late as 1993,

the most popular information in Gopherspace included recipes,

weather, phone books, and movies of chemical reactions.

Gopher represented a simpler, more naïve time, says one of its

modern-day fans. Others call it a purer way of navigating the

internet, of making structure out of chaos.

Today, there are about 140 Gopher servers still out there, many of

them relatively new. The tech world is not a sentimental place,

but it does appreciate simplicity and irony in equal measure

many of these servers were set up on April Fools

Day.

Farhad Anklesaria: You have to be at the right time and place to

have your technology take off and become popular.

Cameron Kaiser, perhaps the Internet Gophers greatest advocate

these days (he runs a Gopher support site called the Overbite

Project), figures he spends about 25 percent of his internet time

in Gopherspace. Its actually rather nice to have a small

ecosystem because no ones running annoying ads in Gopherspace or

trying to track your browsing habits, he says. The protocol

makes the former hard and the latter almost impossible.

As the Web has become synonymous with the internet, and we conduct

more of our lives online, weve learned to abide the

misinformation, solicitations, and scams that thrive in its chaos

though it has changed us and society in ways we are only

beginning to understand. If Gopher had won out, who knows how

things would be different.

The Gopher team, who might understandably be disappointed, seem

genuinely OK with how things turned out. More than OK, actually,

as though they expected it. They were first, after all. And in

the march of technology, the first are eventually last. This is

the natural order of things, Anklesaria says. You have some

building blocks the natural thing to do is build on them.

Thats how civilization bootstraps and how we progress.

If they are surprised by anything, its that the Web was the

system that surpassed them and has outlasted everything else.

There was a protocol that was better than either the Web or

Gopher, they say, called Hyper-G, developed in Austria, which

never got off the ground. You have to be at the right time and

place to have your technology take off and become popular, says

Anklesaria. If youre at the base of a wave, you have a chance

to rise with it. But if youre already on top of the wave and the

wave has broken, thats not good if youre too early, you wont

last.

Bob Alberti: The only way [Tim Berners-Lee] was able to spread

the word about the Web is because the Internet Gopher was there

to allow people to download his files, find a discussion group,

and talk about it.

On July 1, 1997, or maybe in 2009, most likely in the 2000s,

someone walked into the closet in Shepherd Labs and unplugged the

Mother Gopher. No one can agree on the date. No one from the

Gopher team was around.

While I was at the U, McCahill says, it was a point of pride

that wed keep running the server. But McCahill left in 2007 to

become a systems architect at Duke University, developing

instructional and research computing technology. Lindner left in

1996, having followed a Gopher side gig to Geneva, Switzerland,

where he worked for the United Nations, and is now a software

engineer for Google. Anklesaria retired last spring. Of the core

Gopher team members, Alberti is the only one still at the U, as

an information security architect.

At its peak, the Mother Gopher consisted of 10 Apple IIci

computers. But when it was finally euthanized, who knows what

shape it was in. There was no ceremony. Nothing was carted off to

a museum. Gopherspace simply became emptier, and the world

without the Web became harder to imagine.