Information Throughout the Hospital


[Sidebar]: Notes Technical Checklist

Kaiser Permanente Southern California has embarked on an ambitious project to develop an interoperable infrastructure that will provide messages to any system that needs them.
Complicating the transport task was the fact that Kaiser, like most other large organizations, uses a variety of networks
to connect its systems.
Kaiser Permanente Southern California is a health care industry giant, with 10 medical center hospitals serving 2.2 million patients from San Diego to Bakersfield. At its headquarters in Pasadena, Ron Williams, lead programmer analyst, is working on a vital piece of Kaiser's effort to deliver more service at less cost. Williams, who joined Kaiser in 1991, is the architect of a new messaging infrastructure designed to link a variety of clinical information systems in an open network, using the ANSI standard Health Level 7 (HL7) protocol.

The objective sounds simple. When a new patient is admitted to a Kaiser facility, a variety of clinical systems need to know something about the patient, who may require meals, X-rays, and other care and services. The problem is that health care providers traditionally have purchased the best system for each task, only to incur headaches later in linking them together.

"The problem we're attempting to address is that clinical systems need access to legacy and mainframe data," Williams says. "Traditionally, we've written interfaces from platform to platform. But nowadays we have a lot of clients that need data in all kinds of disparate platforms, and the strategy of writing custom interfaces between each of them is getting to be ridiculous."

Williams personally has written interfaces that handle 300,000 transactions a day. He wants to do this sort of thing almost automatically. "The bottom line is we're looking for a means of having a coherent interface we can basically plug any client into and provide access to our other systems with a simple API [application programming interface] and a simple infrastructure. That is the impetus to develop what we call HL7 Services."

Standard Messaging

If Williams is the architect of the HL7 messaging infrastructure, Randi Ferraro, technical design specialist and a 15-year Kaiser veteran, is the project's software and standards expert. "HL7 gives us a powerful way of taking an in-patient event and passing messages about that event to a variety of systems that may be interested in it," says Ferraro.

For instance, dietary managers need to plan a new patient's meals. Radiology might require information if the new admission needs X-rays. Unfortunately, current clinical systems come from vendors that use different protocols for sending or receiving data. This creates a need for something like the HL7 standard, which governs addressing, routing, and security of medical message traffic.

"HL7 Services is the router that sends the message to any system with a legitimate interest in it," Ferraro says. "It starts to approach what we call plug and play. You can bring any system into the network so long as it's HL7-compliant, and you are less bound to the older systems."

Tom Fleischman, executive vice president and CIO for Kaiser Permanente Southern California, says HL7 is a critical ingredient in achieving an ultimate objective that may be years away: the integration of an open clinical information network that frees medical providers from dependency on proprietary solutions.

"Historically, many companies, including Kaiser, didn't worry about clinical systems," Fleischman says. They concentrated instead on standard data processing applications such as payroll, accounting, and general ledger. But with outside forces pushing for "managed care"--a buzzword that implies more health care for less money--providers have tried to improve the efficiency of medical information systems as well.

"We're striving for a clinical system that puts automation at the fingertips of a provider," Fleischman says. "Rather than a doctor or nurse scribbling notes from a physical or a medical history on a piece of paper, they could do it through point-and-click browsers and templates."

Given the twin needs to add new clinical applications and link them for greater efficiency, Fleischman says there is no real choice but to integrate applications into a coherent environment. "No single vendor can deliver a [comprehensive] suite of applications," he says. "HL7 is the standard we hope to use to get all the different systems talking together."

Industry Agreement

According to Ferraro, the HL7 standards effort began in March 1987, when a small group of health care users formed a committee that came under the umbrella of the American National Standards Institute (ANSI). "This began as a user initiative," says Ferraro, who joined the HL7 committee about three years ago. She says vendors eventually realized that a standard data format for clinical information systems would be good for them, too.

"With HL7 standards in place for passing messages, vendors could concentrate on building functionality into their systems rather than writing custom interfaces for each client," Ferraro explains. "That opened up their systems to a wider variety of people."

Since its beginnings in 1987, more than 1,300 providers and vendors have joined the HL7 committee, which meets three times a year and updates the standard every 18 months. Anchored by medical providers such as the American Nursing Association and the Blue Cross system, as well as Kaiser, HL7 has become a given in health care sales.

"We assume HL7 compliance in new applications," says Williams. "If a vendor comes to us with a clinical application and cannot speak HL7, they get a low priority on our shopping list."

Williams' own motivation for wanting to implement the HL7 alternative for clinical systems is simple. "I've already written more interfaces than I cared to," he jokes.

Building Momentum

In fact, just prior to starting work on the HL7 pilot, Williams helped create an electronic medical records (EMR) application, an electronic version of the file folder used to maintain a patient's medical history. The EMR application required access to IBM mainframe data. Williams accomplished this using a screen-scraping technique--that is, creating a virtual 3270 terminal to cull the needed data, then pasting it into the EMR application. The process worked, but throughput was extremely slow.

"The maximum volume you can get is about two transactions a second, and that is if everything is optimal, which it never is," he says. "If you're looking for volume it's a miserable way to go."

His experience with the EMR project convinced him that if clinical systems ever were to be knitted together, Kaiser needed to create a ubiquitous environment that people could plug into. Given the momentum behind HL7 and Ferraro's involvement with the standards process, the choice was obvious. "We knew that any application that could talk HL7, with a minimum amount of coding changes to the application, could read and write in our HL7 Services environment," Williams says.

Of course, discussing an HL7 environment proved far easier than actually creating one, because while HL7 defines a standard data format for clinical applications, it provides no specific transport mechanism to move data between applications. "The challenge for us was to create an environment where we could have ubiquitous access using the standardized HL7 protocol and underlying networking protocols that we have available in-house or would have available shortly," he says.

Complicating the transport task was the fact that Kaiser, like most other large organizations, uses a variety of networks to connect the systems it wanted to bring into the HL7 environment. On the mainframe side, Williams was dealing with IBM ES/9000s and various Tandem systems. The same HL7 environment had to reach server computers running Sybase's DB Live software. And Kaiser was riddled with other networks, all woven into the TCP/IP backbone that connects 10 major medical centers and more than 55 offices in Southern California.

"If you can think of a physical network, we probably have it somewhere in our infrastructure," Williams says. "We have Token Ring, Ethernet, T-1s, T-3s, FDDI, and a few things that I don't want to think about. We have built the HL7 environment on top of these other protocols."

Enter DCE

To serve as the HL7 transport mechanism in this heterogeneous network environment, the team chose the Distributed Computing Environment (DCE), with its remote procedure call (RPC) technology, over TCP/IP. "In a financial environment or a clinical environment, you need security and network management, and you need services associated with a distributed computing environment," Williams says, adding that TCP/IP lacks many of the necessary management and security features. "TCP/IP, or IP really, provides the way our low-level packets move around, but at a higher level we need to manage the information in a way that will enable us to control its access and guarantee its delivery."

He says DCE allows Kaiser to create access control lists for every client in the HL7 environment. Thus he can control what data a given client can and can't access, without having to restrict the overall HL7 message flow. DCE also provides a coherent means of encrypting sensitive data and instituting authentication procedures to make sure intruders can't "spoof" the system.

"DCE provides the means of having nonsecure traffic flow, just as it would under TCP/IP," Williams says. "But it also enables us to provide full authentication of the clients, both when they enter the environments and as they are running, so at any given time we can verify that the client is who he claims to be and is getting information that truly belongs to him.

"And where we have truly sensitive information," he continues, "DCE also provides a mechanism for encrypting data so we can guarantee not only that the person who is supposed to be receiving that data is receiving it, but that nobody else is going to be able to read it off the wire."

Williams says DCE also provides a more elegant means of accomplishing the RPCs that are at the heart of some distributed computing and which let remote users access data from other sources on the network. "Anyone who's done TCP/IP programming or socket programming knows you have to do a lot of things to establish a connection," Williams says. "In DCE there's a number of things you have to do as well, but it's better encapsulated, and the model itself is a synchronous, client/server, request/acknowledgment type of conversation."

Above all, DCE provides independence from the underlying physical network. "I was looking for a common base-level layer, if you will, that will insulate me from whatever is underlying," Williams says. "I don't have to worry whether I have Token Ring, Ethernet, or whatever."

Kaiser expects to run two versions of DCE on two types of server. The servers will mirror one another to provide redundancy for the messaging system and, in the event the HL7 environment is expanded, performance comparisons. According to Williams, Kaiser plans to acquire one DCE implementation from Digital Equipment Corp. that will run on Alpha servers and a second from IBM that will run AIX on RS/6000 servers. The exact timing of these acquisitions will depend on the progress of the project.

Eventually, Williams wants run-time versions of DCE that can operate on any client hardware, regardless of operating system. "The run time provides the ability to have RPC running as an application on that box with authentication and all the security services," he says. "Somebody sitting at a Windows NT box, for instance, can log into the DCE environment as a full-fledged participant."

Testing the Concept

Given Kaiser's commitment to demand HL7-compliant applications and the decision to use DCE to stitch the HL7 environment together, the chief task of the pilot project is to build RPC capability into the systems involved in the test. At present, they include the mainframe, which is passing admissions, discharge, and transfer (ADT) information to several clinical systems. These clinical systems include a dietary system, an EMR application, a surgical information system, a radiological information system, a laboratory system, and an acuity system (which helps predict what medical services a patient is likely to require given his or her condition upon admission).

An early version of the HL7 environment has been up and running since September. In this initial test, the applications use HL7 data formats, but none of them were running in native DCE because that software was not yet available. Instead, Williams and his team created a message queue system that relied on a series of custom interfaces to pass HL7 data.

Williams was scheduled to install DCE on the main HL7 server in December. At that point, the applications will be modified, removing the custom messaging queue and installing a client proxy. The client proxy interfaces the application into DCE. "Each of the client drivers inside the data channel will be modified to talk RPC," Williams says. "I remove one layer from the client, plug in the DCE interface to that, and the clients will be talking RPC instead of talking to my existing message queues."

Although an implementation schedule has not been set, Kaiser intends to replace the client proxies in each application with native DCE. "When the clients can talk RPC, we will remove that proxy and make the client a full member of the DCE and HL7 Services environment, and they can talk directly to our processes," he says. "That will be transparent to the client. The users don't know anything about our underlying protocols."

The hardware components of the messaging system begin with the mainframe environment, which stores and dispenses much of the data of interest to the client systems that are already included or slated for inclusion in the HL7 system. Williams says the HL7 environment treats the IBM ES/9000 and fault-tolerant Tandem mainframes as servers on the messaging system, with most of the traffic flowing one way, from mainframe to clients.

In the project's early stages, only four client systems receive data on the HL7 environment. They include acuity systems at two locations, and the EMR application and the dietary system described below. Williams says the next big step in the project will be acquiring the DCE implementations and servers from DEC and IBM. He expects to include several other client systems in the HL7 pilot in early 1996, bringing to about 100 the total number of seats in the messaging environment. The system is designed to run wherever possible on existing hardware, however modest. For instance, the dietary system runs on a PC XT with DOS. "The amazing thing is it doesn't do badly," Williams says.

On-Line Diet

Cindy Crawford, assistant director of nutrition services at the Kaiser facility in Riverside, CA, has been testing the HL7 environment since September and using it on a daily basis since mid-October. Her staff must keep track of 600 to 900 meals per day at two Kaiser facilities in Fontana and Riverside. That means making sure the right patient gets the right meal delivered to the right room. Before joining the HL7 pilot project, her staff kept track of all those meals on index cards. "We were keeping a card file on patient dietary preferences, and we used to have to manually fill in the cards and manually transfer them to another file when the patient moved from one room to another," Crawford recalls.

The process of automating this system began when Kaiser brought in meal tracking software from CBord Dietary Systems of Ithaca, NY. At first this HL7-compliant application was stitched into the pilot network using a custom messaging interface. Then it went to a client proxy when the DCE server was installed in December. Crawford cares mainly that the HL7 network automatically sends her dietary software a message whenever a patient is admitted, discharged, or transferred.

"As soon as it hits the mainframe, we're seeing it on our software," she says. With the HL7 message in hand, the CBord software automatically updates the meal order and prints out a menu, which includes a patient's name, dietary preference, and room number to minimize wrong deliveries. "We've done away with the card file," Crawford says.

Going Paperless

David Waller is business systems project leader for an even more ambitious clinical application linked to the HL7 environment. He is helping a small outpatient clinic in Moreno Valley, CA, implement a prototype EMR application developed by Oceania, Inc., of Palo Alto, CA. EMR is designed to replace handwritten file folders that doctors and nurses use to record and store medical histories.

"Today, somebody has to pull all those file folders and deliver them to the doctors," Waller says. "Over time, as the EMR expands, the need for a paper chart would diminish or ultimately disappear."

He says the EMR prototype is linked to the HL7 environment to receive patient admission and registration information, as well as walk-in appointments and cancellations. In 1996, the prototype EMR software will also be installed at Kaiser's internal medicine unit in Riverside. The HL7 environment will allow physicians at both facilities to read and write data to a patient's EMR, doing away with the need to duplicate or transfer file folders from one facility to another.

"We know there are people who have a primary care physician in Moreno Valley and are followed by a cardiologist in Riverside," Waller says. "With EMR, progress notes [recorded by a physician at either site] would be visible to everyone on the system."

It all adds up to better service, says HL7 expert Ferraro. "If the ADT system says I have a patient in the hospital who is having a birthday, we like to send them a special birthday tray." This is a small touch but one that might spell the difference in the patient's mind between routine and exceptional service.

The same open system will also allow Kaiser to mix and match clinical systems without having to rewrite interfaces and transport protocols each time a piece of the network changes. "Suppose I change my ADT system," Ferraro says. "In an HL7 environment, it's no problem. You can change your data systems from time to time as you need new technology."

The HL7 pilot and its associated clinical systems are test projects, but they are being closely watched by Kaiser management. "Some of these things may come to fruition," says CIO Fleischman, "but we don't know yet if the pilot projects will scale up."

Even if the pilots go well, implementing solutions across Kaiser's 10 medical centers would take years under the best of circumstances. Although Fleischman realizes that seamless clinical information systems may be a long way off, he considers the HL7 messaging infrastructure a crucial step toward achieving that end. "The true value comes from putting all of our clinical information systems together," he says. "HL7 is the standard we hope will achieve that."

* Tom Abate covers science and technology for the San Francisco Examiner. He can be reached at