The NASA Web portal we had designed was exactly what we felt that NASA management wanted: a new face for the Agency, engaging, interactive, and upbeat; a real change from the staid, informational Web site that NASA had already.
I was the project manager for the redesign. I was the one who would say, "We're going live," who would push the button, so to speak. I felt great about what we had accomplished under an incredibly tight deadline.
On January 2, when we still had about four months to go on our schedule, we had given a presentation at Headquarters. Upper management had loved our vision for the new site; they loved it so much that they said they didn't like our schedule and, instead of deploying the site in four months, they wanted it up and running in four weeks. We hadn't even signed contracts with our subs yet, but we were asked to deliver the portal on February 3rd.
Though we were in a hurry-up mode of operation, we had spent the time to find the right place for everyone on the team, so that every team member knew they could trust the people around them to do their jobs. My Jet Propulsion Laboratory (JPL) team included Douglas Hughes, Jay Dyson, Ellie Trevarthen, and Greg Williams. We worked in conjunction with Brian Dunbar and the Public Affairs Office.
We all worked around the clock; no one took any time off. In that month, we were able to pull together a well-constructed, solid Web site. It didn't have all the bells and whistles that it would have had over the four-month development time, but we had a plan for adding in all those bells and whistles eventually.
|The team may have been tired from the marathon of getting the site online, but they reconvened instantly for this new dash.
The team decided to launch the new portal on a Friday night, January 31st, just before midnight here in California -- traditionally a slow time for the Web. Though the Shuttle was scheduled to land the next morning and we knew the public would look to the site for information, we would be able to give the site a test run before most people saw it.
We invited friends from around the world to join us as we logged on. When the Web site came up on our computers the music we built into the splash page started. On the telecom, every couple of minutes we heard the music start in the different remote locations. We would ask, "Which country was that?" And the answer would come back: "Oh, this is Panama." "It's Calgary." "It's France."
We all had been working 16- to 20-hour days for weeks. Many of us -- myself included -- were about ready to collapse, but it was still a celebratory moment for us. Most of the team was online until about 2 a.m., and then I sent everyone home for a well-deserved rest.
But we didn't rest for long. A few hours later I got a call at home. It was someone at NASA Headquarters who told me, "We've lost the Shuttle." I was asleep when the phone rang. It was early in the morning, and I was so tired that it took me a moment to understand what I had just heard. And then it hit me: Columbia was gone.
Then and now
I can't help but compare the whole experience to the last Shuttle tragedy, and contrast it with the helplessness I felt then.
In 1986, I was working on the Voyager project at the JPL, and we were gearing up for a planetary encounter. We had all sorts of press folks there in the room with us because planetary encounters come few and far between. We had the NASA TV on in the background and were all watching the Challenger launch. When we lost Challenger, there was just this stunned silence and disbelief. It was the longest time before anybody could admit what had happened.
The team I was on during Challenger wasn't directly involved in the manned missions, and we had no official role in helping the Agency through the crisis. Because we didn't know what we could do to make a difference, we felt a sense of complete helplessness.
This time, with Columbia, I felt the team I was leading, in our own small way, could do something to help.
I knew immediately that we would need to make changes on the Web site. The way we designed the site was completely inappropriate for the incredibly tragic event unfolding. Our flashy rock-n-roll intro showed, of all things, a Shuttle flowing through the sky. I knew immediately that it had to come off the site, as soon as possible. And I knew that we would need to prepare for an onslaught of traffic; like us, the rest of the world would be looking to NASA for answers.
Driving back to the office, I got on my cell phone and called my leads on the project at eTouch and Critical Mass, explaining to them, "I know you're exhausted, but we need to get back to work. We've got to make the site appropriate for the moment."
By the time I had gotten into the office, which was only about fifteen minutes away, the team had come back to me with a new proposal for how they could, within the next hour or so, bring together a Web site that would reflect what we were struggling with at that moment as an Agency and as a Nation. The team may have been tired from the marathon of getting the site online, but they reconvened instantly for this new dash.
On the fly that Saturday morning we completely redesigned the interface to the site. We developed new graphics and started posting new Columbia contents, including posting warnings about staying away from the debris and informing people whom to contact if they came across any.
Instead of the 200,000 hits that we thought we would get that first weekend, we had 220 million hits from all around the world. My greatest concern was our search capability. Normally we would get about 6,000 requests a day to the NASA search engine. We had built the initial deployment to handle about 20,000 hits a day, wanting to make certain that we had plenty of capacity. That first day we got 1.5 million search requests.
Redesigning graphics was easy. But you can't scale up a search engine on the spur of the moment.That first day, the search started to melt -- it started taking half a minute, a minute, and then longer for people to get a response. This kind of load on the system was unprecedented, and it looked as though we didn't have the ability to get people to the content they wanted.
I did two things to address this problem. First, I reduced demand on the search by asking my designers to build navigation on the home page to get people immediately to Columbia. They created a rotating banner of a flag at half-staff that linked directly to the main Columbia page. People looking for content could immediately see that they could get what they were looking for by clicking on a big image at the top of their screen.
That handled a huge number of the search queries because we were directing people to where they needed to be, rather than having them get there by way of a search. But that wasn't enough. Many searches from our site were still timing out. I needed to enhance our capacity -- and I needed to do it immediately.
Because we had been pushing so hard on the site, I think I knew the home phone numbers for every NASA Webmaster. I called two people who ran other large search engines for Agency sites, Scott Glasser and Jeff Cobb. I asked, "Hey guys, can we offload some of our hits to you?" They were still doing fine because they weren't the primary search engines for NASA, and they agreed.
One of the search engines had actually been scaling back, in anticipation of the new portal capabilities. They had already taken some of their servers offline and boxed them up. In no time at all, they went into the office and put the servers back online.
|Columbia crew members David Brown, Rick Husband, Laurel Clark, Kalpana Chawla, Michael Anderson, William McCool, and Ilan Ramon
By mid-morning, when our search function required too long a response time (more than 45 seconds), we programmed the site to bring up a message directing users to the two other NASA sites. Together, we were able to successfully handle all requests through the three search capabilities.
We wanted to make certain that the site stayed up, functioning optimally, so that the folks who were trying to communicate what was happening at the Agency could say, "If you need some information, go to www.nasa.gov." If we could demonstrate that we could handle the unprecedented traffic, then they would have a tool they knew they could use.
None of us left the office for two days, making sure that the folks who were posting information about Columbia had all the support they needed, and making sure that the press conferences were streamed online.
Our team played a small role compared to so many others, but I am immensely proud of their dedication, and it made me realize, yet again, that NASA is not just America's space agency. Those weren't just our astronauts; those were the world's astronauts. Through the Web, the world could mourn their loss together.
- Under conditions of uncertainty and tight deadlines, one important capability of project teams is quick adaptability.
- Building a cohesive team upfront is always crucial to project success -- but when a quick response is required, cohesive teamwork becomes even more important.
How do you prepare your team to cope with surprises?