| CONTROL ALT DELETE...Escaping the Black Hole |
Seven Lessons For Doing Technology Right
Lesson 1: PROJECT -- Complete Small, Short-Term, High Value Projects
Lesson 2: CUSTOMER – Deliver a Transcendental Experience to Your Customers
Lesson 3: TEAM -- Use The Right Team
Lesson 4: TECHNOLOGY -- Leverage technology to enable business, not disable it.
Lesson 5: USER – Empower The User To Create Value, Not Waste Time
Lesson 6: VALUE -- Understand How To Harvest Value From Technology
Lesson 7: CHANGE -- Get People To Think Differently As You Turn The Ship
|
Interested in Previewing the Pre-Release?
|
|
|
INTRODUCTION
Computers pervade our lives in ways that were unimaginable decades ago. Our power grid, air traffic control systems, healthcare, defense, entertainment, and food supply are inextricably intermeshed and intertwined with hardware and software. For most of us, most of the time, these computer systems have helped to make our lives longer, more efficient, and in many ways more interesting. Modern civilization has grown as dependant upon computer technology as much as we depend other great human inventions such as money, clothing, and language.
Yet, people die every day because of malfunctioning technologies. Troops in conflict are waiting for ammunition that is weeks late, a patient receives an improperly labeled drug in her surgery, and the fire department responds to the wrong address. The global Supply Chain, healthcare, and public safety systems are all technology-based, and when those technologies don’t work, disaster looms. Our survival depends increasingly on our ability to collectively balance ourselves on technology’s tightrope. An important step forward would be to improve our understanding of the progenitor of technology, the technology project.
When we connect to our online banking system via the internet, we interact with a technology that resulted from a complex multi-month or multi-year endeavor tying together computers, software, networks, and communications equipment. The master chef is the project manager, responsible for making the technology palatable both to the users and to the executives or officials commissioning the project. The chef directs teams of workers, consultants, and vendors to assemble the ingredients into the technology. Time is of the essence, and confusion often prevails, leading to delays and overruns. Delayed technology quickly becomes as unwanted as food aging under a heat lamp.
As with office buildings, bridges, and airplanes, the technologies that power our modern economy are built with blueprints, tools, and great human energy. What happens when blueprints are wrong, tools don’t work, or human energy is misapplied? Great waste results and risk becomes an unintended feature of what was built. On a technology project, unlike those that produce buildings, bridges, and airplanes, true progress toward completion is much more difficult to gauge; you cannot look inside a computer room at a Fortune 500 company and have any idea how much time is still left on a project. Nor can you know what pressures will result in shortcuts taken or safeguards avoided in order to meet a deadline.
The truth is that technology often fails because of the complexity of the human interactions surrounding its design, implementation, and use. There is no cookbook or formulaic approach for doing technology right; success depends upon a deeper understanding of a technology project’s individuals, their motivations, and the interplay among executives, consultants, and vendors, combined with keen timing, operating disciplines, and an ability to adjust to highly fluid situations. To translate valuable assets into harmonious results, a wise techno-chef begins by clarifying and reinforcing goals and objectives, and then uncovering efficient pathways to the results. When this occurs, all parties benefit. Resources are pulled into alignment and success is beckoned. Sadly, there are few great chefs out there and mediocrity often prevails.
When failure pervades, who profits? The profiteers are typically the “greasers” and those being greased. Vendors sometimes entice technology’s buyers with sports tickets, expensive dinners, and even the occasional opportunity for “extra-family” fun. Those buyers, typically technology professionals, are empowered by business executives or government officials who choose not to understand the complexities and the risks of what they are commissioning. When management outsources the buying of these services and products to the technology professionals who are not generally accountable for business results, dreamy goals are concocted. High-fives and wide smiles often accompany those first big vendor payments. In reality, those first slugs of big money are pittances compared to the money and time generally required to actually push a gargantuan project over the finish line. Often times, by the time the race is won, the race doesn’t really matter anymore, because the only people who really won were the ones putting on the race.
Business people blame the technology people but technology projects generally fail because of the business people. Good technology is the servant of good business and not the other way around. It is incumbent upon the business people to take responsibility for what they attempt and what they want. Vendor promises ought be scrutinized, not celebrated. By employing common sense, sobriety, and simple control mechanisms, business people would have a lot less to stew and sue about and a lot more to celebrate.
The technology projects to develop our nuclear weapons, tax collection, and health care, systems can involve thousands of people, hundreds of months, and billions of dollars. In reality, many multi-year projects never get completed. They just soldier on, consuming cost and time. The Standish Group reported that only 35% of software projects started in 2006 were successful in meeting deadlines, budgets, and user objectives. Why? Is it bad technology? Bad project management? Bad consulting? What about the opportunity costs of not having those systems available to us sooner? Are these failures inevitable or are they largely preventable? Is there a better way?
Consider the following examples:
GOVERNMENT
FBI Virtual Case File
Computer failures in governments can happen on a grand scale. Consider the FBI’s Virtual Case File (VCF) system debacle. VCF was designed to help the FBI to more rapidly confront terror threats by enabling its agents to share information. Taxpayer funded and built by Science Applications International Corp or SAIC in San Diego, VCF was supposed to help FBI agents quickly share information on bad guys and emerging threats. The premise was that if agents could communicate more thoroughly and faster electronically, they would be able to solve cases more quickly.
VCF was the third leg of a major FBI initiative, named Trilogy, to modernize its information systems, following the attacks against the United States on September 11, 2001. Trilogy’s first two legs were massive, but reasonably doable: first, upgrade the FBI’s hardware and operating systems software, and second, upgrade the communications infrastructure. Upgrading hardware and software is akin to going to the government version of Sam’s Club and buying tens of thousands of PC’s, and installing tens of thousands of copies of an operating system such as Windows or Linux. Upgrading infrastructure meant becoming a private Internet Service Provider or ISP, like America On-Line or Earthlink, so the FBI could hook up reliably and securely to their private version of the internet while still being able to access the public internet. A lot of activities, yes. Challenging, not really. These projects don’t fail very often because the critical path, or those steps which must be taken toward project completion, are readily apparent. For example, it is obvious that the computer must be plugged in and then turned on before the operating system can be installed. Projects like this are characterized by a high “duh” factor, because of the ease of seeing the pathway to completion.
The third leg of Trilogy proved to be much more challenging. The objective of the VCF system was to replacing the FBI’s outdated problem solving and investigation systems. Why was this so challenging? Because designing a system that manages cases, helps to catch criminals, and enables collaboration among agents around the globe is incredibly complex. If one could wrap a whiteboard around the Pentagon, a la Jean Paul and Cristo and the Bundestag, one might be able to draw the flowchart that would govern all of the processing, decisioning, and reporting steps. Such a system is too ambitious, too complex and too undoable. It has a low “duh” factor because too many interesting sounding approaches are posited without it being obvious when something is ridiculous. In other words, the steps toward developing large technologies are not nearly as easy to chart out on a critical path. Despite the vagueness of the technology, in June, 2001, the FBI awarded the build contract to SAIC.
Predictably, the VCF never materialized. The Office of the Inspector General, the OIG, completed a review of the FBI’s Trilogy project and concluded that between April 2001 and December 2004, the FBI spent more than $170 ($51 million more than the original budget) million on VCF with little to show for it. The FBI refused to accept major portions of the work SAIC delivered on its first three delivery dates. Much of this money went to SAIC and its subcontractors, and yet in February 2005, the OIG stated, “We concluded that the delays and cost growth in completing the Trilogy project were partially attributable to: 1) design modifications the FBI made as a result of refocusing its mission from traditional criminal investigations to preventing terrorism, 2) poor management decisions early in the project, 3) inadequate project oversight, 4) a lack of sound IT (Information Technology) investment practices, and 5) other lessons learned over the course of the project.”
Following the OIG report, Zalmai Azmi, the Chief Information Officer (CIO) of the FBI, responded that “the VCF project remains the highest IT priority for the FBI, and we are developing an implementation plan that will result in deployment of a fully functional investigative case and records management system.” The FBI then commissioned a $2 million independent evaluation of VCF by computer experts at Aerospace Corp. to determine whether VCF should be saved or scrapped.
Predictably, the Aerospace audit encouraged the FBI to decide to scrap VCF in January 2005. In the meantime the FBI continues to use its antiquated, mostly manual, processes for handling complex cases. Azmi, who became CIO in May 2004, has been working on better defining the technologies that the post 9/11 modernization was supposed to accomplish. In March 2006, the FBI awarded Lockheed Martin a six-year, $305 million contract for Sentinel, the FBI’s replacement technology for VCF. Maybe it’s a move in the right direction, but the terrorists have been given a reprieve because Sentinel isn’t scheduled to materialize until 2009.
A primary driver of VCF’s failure was the lack of discernable clarity describing exactly how VCF would enable the FBI’s agents to fight crime. Foggy perspectives of a technology’s objectives is a clear recipe for its failure. Layers of uncertainly create more and more complexity, eventually leading to a Black Hole that sustains itself, sucking in money, time, reputations, and in this case, the opportunity cost of not solving cases and catching criminals. A key flaw in the FBI practices is that they used a cost plus contract and did not materially tie vendor payments to measurable progress toward VCF’s completion. Whose fault was this debacle? SAIC? The FBI? Will Sentinel rise from VCF’s ashes or burn up on those same flames?
IRS
The IRS is an information agency in the purest sense. It maintains hundreds of millions of records on individuals and businesses. In 1962, when John F. Kennedy was President and people looked forward to watching new episodes of Andy Griffith, the IRS rolled out some lines of computer code. It was the IRS’s first main foray into the then new world of computers. These lines of code were written in IBM’s Assembly Language Compiler or ALC. Amazingly, these lines of code or their modified descendants are still supporting the analysis of our tax returns. Are historic or legacy systems necessarily bad? After all, old wine tastes better than new and even aged meat or cheese can be more pleasing than younger alternatives. With technology, however, this rarely holds true. In computing especially, nostalgia is a poor substitute for modernity.
Why is the IRS running some of the same computer systems that it used at a time when computing power was literally one ten-billionth of what it is today? The IRS attempted to modernize before. In 1977, the IRS sought $1.8 billion to develop its Tax Administration System or TAS. This system was intended to partly augment and partly replace the original IRS systems from the 1960’s. Congress refused ongoing funding for the system several times, and in 1978 the IRS withdrew its efforts. The IRS backtracked because it felt that, “concern over Privacy policy, First Amendment rights, a general fear of over-computerization, and the resulting dehumanization of Government” made its case to raise the money too challenging. By the late 1970’s the IRS was managing its data on a variety of computers from IBM, Honeywell, GE, and Control Data. Its massive Individual Master File and Business Master File were repositories of taxpayer data. Its systems weren’t integrated and, in fact, the IRS had a true sneaker-net in which data was transferred among three systems via magnetic tapes on reels. Data came in on paper, was keypunched by hand, and then the data tapes and punch cards were hand-carried among the systems. Documentation was inconsistent, and though the system sweated, heaved, and strained, it didn’t completely collapse, but it caused the IRS to leave a lot of money on the table.
By 1995 the IRS apparently failed to examine five million returns. In 1997, estimates ranged up to $150 billion per year that the IRS’s “klutzy” computer systems failed to collect. The General Accounting Office (GAO) in 1995 blasted the IRS for spending eight years and two billion dollars on what amounted to “marginal improvements.” The National Research Council (NRC) and the GAO hacked away at the IRS for not having a robust technology strategy nor having the executives on board who could execute such a strategy if it existed. In 1996, Larry Summers, then Deputy Treasury Secretary, told the Subcommittee on Treasury, Postal Service and General Government, “The modernization effort has been sobering."
The IRS tried again. The Internal Revenue Service sought funding for its billion dollar "Business System Modernization" or BSM program. BSM was designed to modernize the IRS systems for collecting taxes, auditing returns and helping taxpayers with questions. In 1999, the IRS kicked off the development of its Customer Account Data Engine, or CADE component of the BSM. CADE was designed to be a modern mechanism for storing and managing customer data. The IRS hired Computer Sciences Corporation or CSC, a venerable consultancy to do the work. The project was more than CSC could handle on its own, so they brought in IBM, Lucent, Northrop Grumman, Unisys and SAIC to help, a common practice in the technology development industry.
By 2005, the CADE system had yet to be deployed. Neither the IRS nor its customers could go online and easily look up a balance as one can do with most banks and credit card companies. The aforementioned group of high powered vendors and consultants, continued to miss deadlines. In the past six years the revolving door of CIO’s office hasn’t stopped spinning. According to the IRS Oversight Board, made up of independent tax and technology executives, the fault is wide and nearly unbounded. The IRS failed to manage its vendors and never really understood what CSC was doing. CSC was humbled and cowed by the enormity of the project. Paul Cofoni, a division president at CSC told a United States House Subcommittee, “I have never encountered a program of the size and complexity of the Business Systems Modernization program at the IRS.” Scary, isn’t it? So ask yourself why CSC accepted the job in the first place and then continued to stay on it long after they realized that it was a fiasco. The reason is simple. The IRS continued to pay the bills.
What does this all mean? Why should you care? Because, the IRS systems are responsible for collecting $5.5 billion dollars per day, to fund the government of the leader of the free world. If IRS systems collapse or suffer a major outage, do we want to contemplate the ripple effects? No money to pay for our troop’s meals in Iraq. No money to pay our air traffic controllers. If the federal government shuts down or more realistically, slows down, then the state and local governments will follow, followed by businesses, and then our banks. When we slow down in the US, the whole world slows down. A decade ago, dread reigned as it was thought that Year 2000 or Y2K computer issues would shut down our economy. The cataclysm failed to materialize but rather than inspiring a sense of greater awareness of the risks of technology dependency, a greater level of complacency or “I’m sure it will work” now pervades.
BUSINESS
Hewlett-Packard
It was a classic American success story. The founders even tossed a coin to see whose name would be first on the stationery. Started in 1939 by Stanford classmates Bill Hewlett and Dave Packard, Hewlett-Packard was visionary and significant. From electronic test equipment to durable scientific and business calculators to LaserJet printers, HP developed a reputation for innovativeness and excellence. The company became very dynamic and some say out of control by the time Carly Fiorina took the helm in 1999. In 2000 HP tried and failed to purchase the consulting division of PriceWaterhouseCoopers for $18 billion. In 2002, her apparent desire to write a massive check seemed sated when HP bought Compaq Computer, its longstanding competitor, for $25 billion. Since Bill and Dave flipped that coin, HP has grown to an eighty billion dollar per year company with one hundred and fifty thousand employees.
Size can be a burden though. In Carly’s tenure, HP’s market value dropped by 30%. In February, 2005 the shareholders had enough and Fiorina was out. Why? Perhaps because bigness and dumbness are often close cousins. In a Banc of America Securities meeting, Fiorina admitted that HP’s internal Fusion project cost it $400 million in revenue. When the FBI or the IRS fail on a big project, one could argue that they are not experts at this. But HP? A company that has its own large division of management and technology consultants? How could they fail?
HP’s Fusion project was designed to reduce the number of instances of its SAP enterprise software from forty to six. An instance of SAP is simply an independent version of the software with its own data and users, running separately from other instances. Why did HP undergo this project? Ostensibly to simplify itself. It figured that it would make sense to integrate both HP’s and Compaq’s many disparate business management systems into a few integrated systems.
SAP is a German software manufacturer, the largest manufacturer in the world of ERP or Enterprise Resource Planning software, the type that companies and governments use to keep track of their employees, cash, and customers. Nearly every Fortune 500 company uses SAP or Oracle’s ERP as a central nervous system for its business information. Companies are so dependant upon their ERP systems that a major overhaul or switchover to a new vendor is nearly akin in complexity to a human brain transplant. This thrills the consulting firms that handle these overhauls and switchovers, but causes clients typically spend a multiple of their vendor-guided project budgets.
In the end, maybe Fusion has simplified operations within HP, but at great cost to HP’s bottom line. In its September 10, 2004 third quarter SEC filing, HP referred to “internal execution issues” as a reason that its Enterprise Servers and Storage business had suffered a major blow. During its third quarter’04 earnings call HP was reluctant to point out why its order fulfillment systems weren’t working. It only came to light later that the Fusion systems consolidation was the culprit and couldn’t respond to the orders that customers were placing. Apparently HP decided to use “hope” as a strategy, rather than securing its operations with a backup system to handle critical order flow. Fiorina claimed that lost revenue in the Enterprise Server business was only $40 million, which was reportedly $10 million more than the cost of the entire Fusion project.
At least HP had the forethought that its SAP project might not go smoothly and it overbuilt inventory because it wanted to make sure that orders wouldn’t get lost in its manufacturing process. Mostly the overbuilding worked, but the system did lose up to 20% of the orders for HP’s servers and once data starts to lose integrity, the problems tend to mushroom, rather than resolve. Servers are mostly a commodity market and when HP couldn’t ship, IBM and Dell were more than happy to pick up the extra business.
Most of the server instance consolidations went well, according to HP. Thirty-four worked fine, but the thirty-fifth was the disaster. In an interview with Computerworld, Gilles Bouchard, the CIO and Executive Vice President of Global Operations at HP, remarked that “Shit Happens.” Bouchard’s job was spared by three other senior executives at HP lost their jobs. In reality, HP did a pretty good job at this major project, they only missed their revenue target by $400 million, a third of which was due to the Fusion project. Other than a lost CEO, many millions in business, and a lot of anxiety, HP positioned itself for a recovery.
FoxMeyer Drug
FoxMeyer Drug wasn’t so lucky. In 1995, FoxMeyer was a $5 billion per year drug company. A year later the company was bankrupt. FoxMeyer management blamed it on SAP and the consulting firm that managed its main implementation project, Andersen Consulting. Andersen Consulting, now known as Accenture, was spun off from the venerable Arthur Andersen, the accounting firm, ostensibly because the consulting partners grew tired of sharing their larger fees and earnings with their audit and tax partners who were working under more mundane fee structures.
FoxMeyer embarked on a $100 million dollar project to replace its distribution system with a system an ERP from SAP. Simultaneously, FoxMeyer undertook a project to replace its warehouse automation system with technology from McHugh-Freeman. In 1994, FoxMeyer’s CIO, Robert R. Brown, told Computerworld, "We are betting our company" on these systems. FoxMeyer was anticipating a $40 million per year savings and with that assumption it bid low and won a huge contract with a large consortium of teaching hospitals to deliver drugs. Within a few months, FoxMeyer was parceling out pieces of that contract that it couldn’t service to its competitors. FoxMeyer took a $34 million dollar charge and its stock went from 26 to 3.
Why did the SAP/Arthur Andersen project fail? Was the project wildly ambitious? Did the vendors and consultants “back up the truck” at FoxMeyer and unload every consultant they had who was “on the beach” or “in their stables”? Could large systems from separate vendors easily work together? Why did FoxMeyer defer the reviews of Andersen’s work to Andersen’s own consultants? Why wasn’t FoxMeyer’s management more involved?
On August 27, 1996 FoxMeyer filed for bankruptcy. Six years later the bankruptcy trustee and Accenture settled out of court and the lawsuit was dismissed on August 8, 2002. In the end, perhaps McKesson, FoxMeyer’s former competitor and eventual acquirer won the battle because it picked up FoxMeyer’s assets on the cheap.
Others
In July, 2004 the Department of Veteran’s Affairs pulled the plug on a $472 million technology for their Bay Pines Medical Center in Tampa. The system was intended for eventual use at all VA hospitals. Bay Pines shut down the project when it became clear that its users weren’t using the system. Bearingpoint, a descendant consulting arm of KPMG, another of the former Big Six firms that was investigated by Congress for accepting incentive bonuses for keeping the project on schedule while failing to train the Bay Pines employees.
Failures abound on technology projects. A decade ago AMR Corp, the parent of American Airlines, Budget Rent A Car, Hilton, and Marriot pulled the plug on a reservation system that they collectively spent four years and $125 million developing. During the Christmas holiday in 2004, Comair, a subsidiary of Delta Airlines claimed that computer snafus caused it to cancel 1100 flights, stranding more than 30,000 passengers. Enron, Grainger, Oxford Health, Nike, and Hershey are a part of a select group of companies whose technology project failures made it into the media. The truth is that most failures are never reported, yet they occur within the most trusted and critically important organizations, businesses, and governments.
As of January, 2008, the problem worsens. Credit Card issuer GE Money reported that a computer backup tape had been lost from Iron Mountain, a well-known information archive. That tape contained data from 650,000 customers including 150,000 Social Security numbers from J.C. Penney and a number of other well-known retailers. In 2006, the names and credit-card numbers of 243,000 Hotels.com customers were on a laptop computer stolen from an Ernst & Young employee’s locked car trunk in Texas. Sure these events are random, but it kept on happening to Ernst & Young. The firm had a laptop stolen in New Jersey containing data on Goldman Sachs employees and another stolen in Florida containing employee data from several other companies. Personal data on more than 26 million veterans was lost when a laptop was stolen from the home of a VA employee. Tragically, the pace of these breaches appears to be accelerating.
**********
Should we panic? If eBay fails or goes into the Black Hole, it probably won’t result in social catastrophe. If the water purification system fails for Los Angles County, it could. As our technologies become more pervasive and intertwined, we are more vulnerable than ever. We rightfully worry about global warming, but the risk of technology failure is both more urgent and more addressable. Technology-driven calamity, at least to some extent, is merely a question of if, and not when.
Technologies must be designed with risk reduction in mind. The faster we move forward, the more likely it is that we’ll lose control of technology in our thirst for progress. The face of failure probably won’t be that of SkyNet from the Terminator movies, instead it will be the dark cash machines and the angry delivery driver selling boxes of Cheerios from the back of his truck that he didn’t deliver to the 7-Eleven because he knew he wasn’t going to be paid.
Our existence has become so umbilicaled to our technology that a few days of service interruption would permanently tear the paper-thin fabric of our security blanket. Yet, hope can triumph fear. Doing technology right is not undoable, but the challenge is paradoxical. It requires balancing a myopic perspective of the details of what is being sought with a holistic understanding of how the technology will interact with its neighbors. The wise seeker will find that clarity is essential but the virtuoso techno-chef understands that simplicity is paramount.
Safe, productive, and resilient technologies can only be birthed from well-planned and well-executed technology projects. Our challenge is to properly train the techno-chefs who will ensure that enough tasty and empowering technologies will be produced to satisfy our rapidly expanding appetites.
CONTROL ALT DELETE…Escaping the Black Hole is colorful case study describing a Midwestern manufacturer’s technology misadventures. The CEO, Biff Harper, works with a super consultant, Jack Bluto, to turn around a failing technology project. Biff is facing his greatest challenge to his career and company. Enter Jack, who understands technology from the crook’s perspective. Biff and Jack come together to lead Biff’s company out of its mire. Along the way the duo encounter fear, sex, criminality, and triumph. Jack teaches Biff the seven master lessons for doing technology right. These master lessons consist of sixty-three sub-lessons that are woven into the case study, but also bulleted so the reader can return for reference. This book is designed to be a fun read for a non-technology audience, but also a serious how-to guide for any manager, employee, executive, supplier, or consultant who is involved in a technology project.
The lessons herein are neither industry nor project specific. The lesson content is somewhat synchronized with the technology of the moment but is also relevant for the technology of yesterday and is designed to be useful for the technology of tomorrow. It is written to arm the third millennial worker with the tools to both think about and do technology right.
Enjoy the story and good luck on your project!
Jon Bellman
© 2008 Jon Bellman All Rights Reserved
Re-distribution in any form or commercial use without permission is not allowed.
|
|