Information Throughout the Hospital
BY TOM ABATE
Kaiser Permanente Southern California has embarked on an ambitious project
to develop an interoperable infrastructure that will provide messages to
any system that needs them.
Complicating the transport task was the fact that Kaiser,
like most other large organizations, uses a variety of networks
Kaiser Permanente Southern California is a health care industry giant, with
10 medical center hospitals serving 2.2 million patients from San Diego
to Bakersfield. At its headquarters in Pasadena, Ron Williams, lead programmer
analyst, is working on a vital piece of Kaiser's effort to deliver more
service at less cost. Williams, who joined Kaiser in 1991, is the architect
of a new messaging infrastructure designed to link a variety of clinical
information systems in an open network, using the ANSI standard Health Level
7 (HL7) protocol.
to connect its systems.
The objective sounds simple. When a new patient is admitted to a Kaiser
facility, a variety of clinical systems need to know something about the
patient, who may require meals, X-rays, and other care and services. The
problem is that health care providers traditionally have purchased the best
system for each task, only to incur headaches later in linking them together.
"The problem we're attempting to address is that clinical systems need
access to legacy and mainframe data," Williams says. "Traditionally,
we've written interfaces from platform to platform. But nowadays we have
a lot of clients that need data in all kinds of disparate platforms, and
the strategy of writing custom interfaces between each of them is getting
to be ridiculous."
Williams personally has written interfaces that handle 300,000 transactions
a day. He wants to do this sort of thing almost automatically. "The
bottom line is we're looking for a means of having a coherent interface
we can basically plug any client into and provide access to our other systems
with a simple API [application programming interface] and a simple infrastructure.
That is the impetus to develop what we call HL7 Services."
If Williams is the architect of the HL7 messaging infrastructure, Randi
Ferraro, technical design specialist and a 15-year Kaiser veteran, is the
project's software and standards expert. "HL7 gives us a powerful way
of taking an in-patient event and passing messages about that event to a
variety of systems that may be interested in it," says Ferraro.
For instance, dietary managers need to plan a new patient's meals. Radiology
might require information if the new admission needs X-rays. Unfortunately,
current clinical systems come from vendors that use different protocols
for sending or receiving data. This creates a need for something like the
HL7 standard, which governs addressing, routing, and security of medical
"HL7 Services is the router that sends the message to any system with
a legitimate interest in it," Ferraro says. "It starts to approach
what we call plug and play. You can bring any system into the network so
long as it's HL7-compliant, and you are less bound to the older systems."
Tom Fleischman, executive vice president and CIO for Kaiser Permanente Southern
California, says HL7 is a critical ingredient in achieving an ultimate objective
that may be years away: the integration of an open clinical information
network that frees medical providers from dependency on proprietary solutions.
"Historically, many companies, including Kaiser, didn't worry about
clinical systems," Fleischman says. They concentrated instead on standard
data processing applications such as payroll, accounting, and general ledger.
But with outside forces pushing for "managed care"--a buzzword
that implies more health care for less money--providers have tried to improve
the efficiency of medical information systems as well.
"We're striving for a clinical system that puts automation at the fingertips
of a provider," Fleischman says. "Rather than a doctor or nurse
scribbling notes from a physical or a medical history on a piece of paper,
they could do it through point-and-click browsers and templates."
Given the twin needs to add new clinical applications and link them for
greater efficiency, Fleischman says there is no real choice but to integrate
applications into a coherent environment. "No single vendor can deliver
a [comprehensive] suite of applications," he says. "HL7 is the
standard we hope to use to get all the different systems talking together."
According to Ferraro, the HL7 standards effort began in March 1987, when
a small group of health care users formed a committee that came under the
umbrella of the American National Standards Institute (ANSI). "This
began as a user initiative," says Ferraro, who joined the HL7 committee
about three years ago. She says vendors eventually realized that a standard
data format for clinical information systems would be good for them, too.
"With HL7 standards in place for passing messages, vendors could concentrate
on building functionality into their systems rather than writing custom
interfaces for each client," Ferraro explains. "That opened up
their systems to a wider variety of people."
Since its beginnings in 1987, more than 1,300 providers and vendors have
joined the HL7 committee, which meets three times a year and updates the
standard every 18 months. Anchored by medical providers such as the American
Nursing Association and the Blue Cross system, as well as Kaiser, HL7 has
become a given in health care sales.
"We assume HL7 compliance in new applications," says Williams.
"If a vendor comes to us with a clinical application and cannot speak
HL7, they get a low priority on our shopping list."
Williams' own motivation for wanting to implement the HL7 alternative for
clinical systems is simple. "I've already written more interfaces than
I cared to," he jokes.
In fact, just prior to starting work on the HL7 pilot, Williams helped create
an electronic medical records (EMR) application, an electronic version of
the file folder used to maintain a patient's medical history. The EMR application
required access to IBM mainframe data. Williams accomplished this using
a screen-scraping technique--that is, creating a virtual 3270 terminal to
cull the needed data, then pasting it into the EMR application. The process
worked, but throughput was extremely slow.
"The maximum volume you can get is about two transactions a second,
and that is if everything is optimal, which it never is," he says.
"If you're looking for volume it's a miserable way to go."
His experience with the EMR project convinced him that if clinical systems
ever were to be knitted together, Kaiser needed to create a ubiquitous environment
that people could plug into. Given the momentum behind HL7 and Ferraro's
involvement with the standards process, the choice was obvious. "We
knew that any application that could talk HL7, with a minimum amount of
coding changes to the application, could read and write in our HL7 Services
environment," Williams says.
Of course, discussing an HL7 environment proved far easier than actually
creating one, because while HL7 defines a standard data format for clinical
applications, it provides no specific transport mechanism to move data between
applications. "The challenge for us was to create an environment where
we could have ubiquitous access using the standardized HL7 protocol and
underlying networking protocols that we have available in-house or would
have available shortly," he says.
Complicating the transport task was the fact that Kaiser, like most other
large organizations, uses a variety of networks to connect the systems it
wanted to bring into the HL7 environment. On the mainframe side, Williams
was dealing with IBM ES/9000s and various Tandem systems. The same HL7 environment
had to reach server computers running Sybase's DB Live software. And Kaiser
was riddled with other networks, all woven into the TCP/IP backbone that
connects 10 major medical centers and more than 55 offices in Southern California.
"If you can think of a physical network, we probably have it somewhere
in our infrastructure," Williams says. "We have Token Ring, Ethernet,
T-1s, T-3s, FDDI, and a few things that I don't want to think about. We
have built the HL7 environment on top of these other protocols."
To serve as the HL7 transport mechanism in this heterogeneous network environment,
the team chose the Distributed Computing Environment (DCE), with its remote
procedure call (RPC) technology, over TCP/IP. "In a financial environment
or a clinical environment, you need security and network management, and
you need services associated with a distributed computing environment,"
Williams says, adding that TCP/IP lacks many of the necessary management
and security features. "TCP/IP, or IP really, provides the way our
low-level packets move around, but at a higher level we need to manage the
information in a way that will enable us to control its access and guarantee
He says DCE allows Kaiser to create access control lists for every client
in the HL7 environment. Thus he can control what data a given client can
and can't access, without having to restrict the overall HL7 message flow.
DCE also provides a coherent means of encrypting sensitive data and instituting
authentication procedures to make sure intruders can't "spoof"
"DCE provides the means of having nonsecure traffic flow, just as it
would under TCP/IP," Williams says. "But it also enables us to
provide full authentication of the clients, both when they enter the environments
and as they are running, so at any given time we can verify that the client
is who he claims to be and is getting information that truly belongs to
"And where we have truly sensitive information," he continues,
"DCE also provides a mechanism for encrypting data so we can guarantee
not only that the person who is supposed to be receiving that data is receiving
it, but that nobody else is going to be able to read it off the wire."
Williams says DCE also provides a more elegant means of accomplishing the
RPCs that are at the heart of some distributed computing and which let remote
users access data from other sources on the network. "Anyone who's
done TCP/IP programming or socket programming knows you have to do a lot
of things to establish a connection," Williams says. "In DCE there's
a number of things you have to do as well, but it's better encapsulated,
and the model itself is a synchronous, client/server, request/acknowledgment
type of conversation."
Above all, DCE provides independence from the underlying physical network.
"I was looking for a common base-level layer, if you will, that will
insulate me from whatever is underlying," Williams says. "I don't
have to worry whether I have Token Ring, Ethernet, or whatever."
Kaiser expects to run two versions of DCE on two types of server. The servers
will mirror one another to provide redundancy for the messaging system and,
in the event the HL7 environment is expanded, performance comparisons. According
to Williams, Kaiser plans to acquire one DCE implementation from Digital
Equipment Corp. that will run on Alpha servers and a second from IBM that
will run AIX on RS/6000 servers. The exact timing of these acquisitions
will depend on the progress of the project.
Eventually, Williams wants run-time versions of DCE that can operate on
any client hardware, regardless of operating system. "The run time
provides the ability to have RPC running as an application on that box with
authentication and all the security services," he says. "Somebody
sitting at a Windows NT box, for instance, can log into the DCE environment
as a full-fledged participant."
Testing the Concept
Given Kaiser's commitment to demand HL7-compliant applications and the decision
to use DCE to stitch the HL7 environment together, the chief task of the
pilot project is to build RPC capability into the systems involved in the
test. At present, they include the mainframe, which is passing admissions,
discharge, and transfer (ADT) information to several clinical systems. These
clinical systems include a dietary system, an EMR application, a surgical
information system, a radiological information system, a laboratory system,
and an acuity system (which helps predict what medical services a patient
is likely to require given his or her condition upon admission).
An early version of the HL7 environment has been up and running since September.
In this initial test, the applications use HL7 data formats, but none of
them were running in native DCE because that software was not yet available.
Instead, Williams and his team created a message queue system that relied
on a series of custom interfaces to pass HL7 data.
Williams was scheduled to install DCE on the main HL7 server in December.
At that point, the applications will be modified, removing the custom messaging
queue and installing a client proxy. The client proxy interfaces the application
into DCE. "Each of the client drivers inside the data channel will
be modified to talk RPC," Williams says. "I remove one layer from
the client, plug in the DCE interface to that, and the clients will be talking
RPC instead of talking to my existing message queues."
Although an implementation schedule has not been set, Kaiser intends to
replace the client proxies in each application with native DCE. "When
the clients can talk RPC, we will remove that proxy and make the client
a full member of the DCE and HL7 Services environment, and they can talk
directly to our processes," he says. "That will be transparent
to the client. The users don't know anything about our underlying protocols."
The hardware components of the messaging system begin with the mainframe
environment, which stores and dispenses much of the data of interest to
the client systems that are already included or slated for inclusion in
the HL7 system. Williams says the HL7 environment treats the IBM ES/9000
and fault-tolerant Tandem mainframes as servers on the messaging system,
with most of the traffic flowing one way, from mainframe to clients.
In the project's early stages, only four client systems receive data on
the HL7 environment. They include acuity systems at two locations, and the
EMR application and the dietary system described below. Williams says the
next big step in the project will be acquiring the DCE implementations and
servers from DEC and IBM. He expects to include several other client systems
in the HL7 pilot in early 1996, bringing to about 100 the total number of
seats in the messaging environment. The system is designed to run wherever
possible on existing hardware, however modest. For instance, the dietary
system runs on a PC XT with DOS. "The amazing thing is it doesn't do
badly," Williams says.
Cindy Crawford, assistant director of nutrition services at the Kaiser facility
in Riverside, CA, has been testing the HL7 environment since September and
using it on a daily basis since mid-October. Her staff must keep track of
600 to 900 meals per day at two Kaiser facilities in Fontana and Riverside.
That means making sure the right patient gets the right meal delivered to
the right room. Before joining the HL7 pilot project, her staff kept track
of all those meals on index cards. "We were keeping a card file on
patient dietary preferences, and we used to have to manually fill in the
cards and manually transfer them to another file when the patient moved
from one room to another," Crawford recalls.
The process of automating this system began when Kaiser brought in meal
tracking software from CBord Dietary Systems of Ithaca, NY. At first this
HL7-compliant application was stitched into the pilot network using a custom
messaging interface. Then it went to a client proxy when the DCE server
was installed in December. Crawford cares mainly that the HL7 network automatically
sends her dietary software a message whenever a patient is admitted, discharged,
"As soon as it hits the mainframe, we're seeing it on our software,"
she says. With the HL7 message in hand, the CBord software automatically
updates the meal order and prints out a menu, which includes a patient's
name, dietary preference, and room number to minimize wrong deliveries.
"We've done away with the card file," Crawford says.
David Waller is business systems project leader for an even more ambitious
clinical application linked to the HL7 environment. He is helping a small
outpatient clinic in Moreno Valley, CA, implement a prototype EMR application
developed by Oceania, Inc., of Palo Alto, CA. EMR is designed to replace
handwritten file folders that doctors and nurses use to record and store
"Today, somebody has to pull all those file folders and deliver them
to the doctors," Waller says. "Over time, as the EMR expands,
the need for a paper chart would diminish or ultimately disappear."
He says the EMR prototype is linked to the HL7 environment to receive patient
admission and registration information, as well as walk-in appointments
and cancellations. In 1996, the prototype EMR software will also be installed
at Kaiser's internal medicine unit in Riverside. The HL7 environment will
allow physicians at both facilities to read and write data to a patient's
EMR, doing away with the need to duplicate or transfer file folders from
one facility to another.
"We know there are people who have a primary care physician in Moreno
Valley and are followed by a cardiologist in Riverside," Waller says.
"With EMR, progress notes [recorded by a physician at either site]
would be visible to everyone on the system."
It all adds up to better service, says HL7 expert Ferraro. "If the
ADT system says I have a patient in the hospital who is having a birthday,
we like to send them a special birthday tray." This is a small touch
but one that might spell the difference in the patient's mind between routine
and exceptional service.
The same open system will also allow Kaiser to mix and match clinical systems
without having to rewrite interfaces and transport protocols each time a
piece of the network changes. "Suppose I change my ADT system,"
Ferraro says. "In an HL7 environment, it's no problem. You can change
your data systems from time to time as you need new technology."
The HL7 pilot and its associated clinical systems are test projects, but
they are being closely watched by Kaiser management. "Some of these
things may come to fruition," says CIO Fleischman, "but we don't
know yet if the pilot projects will scale up."
Even if the pilots go well, implementing solutions across Kaiser's 10 medical
centers would take years under the best of circumstances. Although Fleischman
realizes that seamless clinical information systems may be a long way off,
he considers the HL7 messaging infrastructure a crucial step toward achieving
that end. "The true value comes from putting all of our clinical information
systems together," he says. "HL7 is the standard we hope will
* Tom Abate covers science and technology for the San
Francisco Examiner. He can be reached at email@example.com.