by Stephen R. Walli, Special Technical Contributor to UniNews online
Introduction
Application source code portability is one of the cornerstones of most open systems definitions. The intention is that if an application is written to a particular model of source code portability, it can port relatively easily to any platform that supports the portability model. This model is typically based on source code portability standards such as POSIX and ANSI C, and specifications that include these standards such as The Open Group's Single UNIX Specification.The strength of this model is that the investment in the application's development is not lost in re-deployment to new architectures. Every useful application outlives the platform on which it was originally developed and deployed. Re-writing the application to new platforms every five years or so is not an option. There are already too many new applications in the queue to be written without adding the burden of costly re-writes every few years to address newer, faster hardware platforms.
The POSIX standard is so successful as to have become ubiquitous. There is no major operating system available today that has not received its NIST FIPS 151-2 certificate demonstrating it provides the POSIX.1 interface. This not only includes all the major flavors of traditional UNIX systems, but systems as diverse as IBM's MVS (which has also branded to The Open Group's UNIX 95 brand), Digital's VMS, and Microsoft's Windows NT.
Windows NT is designed so that a micro-kernel architecture supports different user and programmer interfaces through environment subsystems. Individual application processes interact with the appropriate subsystem in a client/server relationship, with the subsystems servicing requests for kernel functions.
Microsoft's primary subsystem is the Win32 subsystem, expressing the Win32 user and programmatic interfaces. Windows NT also ships an OS/2 subsystem and a POSIX subsystem. The Microsoft POSIX subsystem was an exact implementation of the POSIX.1 standard interface and it's required ANSI C library interfaces. It was not particularly accessible, but it proved that the environment subsystem design was flexible and powerful.
If Windows NT could support a POSIX subsystem, why not an entire UNIX environment subsystem? The Windows NT kernel provides the complete functionality of an operating system. There is nothing that is described in the POSIX standards or Open Group specifications that could not be manifested through an environment subsystem.
A Porting Solution
If Windows NT could support a POSIX subsystem, why not an entire UNIX environment subsystem? The Windows NT kernel provides the complete functionality of an operating system. There is nothing that is described in the POSIX standards or Open Group specifications that could not be manifested through an environment subsystem. Indeed a number of functions are exposed through the Microsoft POSIX and OpenNT subsystems that are not available through the Win32 interface. This is the core strength of the environment subsystem approach -- by creating a subsystem that communicates directly with the kernel, complete POSIX/UNIX semantics can be provided for such fundamental capabilities as fork()/exec(), signals, and file system access. This allows a complete porting and run-time environment to be developed. A number of alternatives to the subsystem approach have been tried, using UNIX-like libraries in the Win32 space to solve the porting problem, but none of these provide the complete required semantics to directly solve the porting and deployment problem.OpenNT (a product of Softway Systems, San Francisco, California, the company for which I work), is a complete UNIX environment subsystem comprised of several components. At its core is the OpenNT subsystem that runs as a peer to the Win32 subsystem. It is a replacement for the Microsoft POSIX subsystem and supports considerable functionality beyond that described by POSIX.1 and delivered in the Microsoft POSIX subsystem. A command shell and complete set of utilities provides a traditional user command-line and scripting environment. The software development kit (SDK) provides a wrapper around the Microsoft Visual C compiler environment, allowing applications code created on traditional UNIX systems to be ported directly to Windows NT. (A port of gcc is almost complete.) The OpenNT X11R6 server supports remote and local X11R5 and Motif applications running with the OpenNT subsystem.
With the release of OpenNT 2.0 the following base functionality is supported at the system call level:
- complete POSIX.1 and ANSI C API.
- BSD sockets
- X11R5 client libraries
- OSF/Motif 1.2.4 client libraries
- SVID IPC (shared memory, semaphores, message queues)
- memory mapped files
- curses
- pseudo-terminals
- libraries supporting a myriad of traditional functions from BSD and SVID flavors of UNIX.
The command and scripting environment supports 200+ utilities, including the Korn Shell, csh, and a complete port of Perl 5.
Overall Architecture
The OpenNT subsystem is a true native subsystem on Windows NT, the same as the Win32 subsystem that provides the services to Win32 applications. It runs as a peer to the Win32 subsystem and provides a number of interesting ways to interoperate with the rest of WindowsNT.The subsystem architecture is comprised of the subsystem process itself, a console session manager and a DLL through which POSIX processes communicate with the OpenNT subsystem. The subsystem is responsible for providing system services to OpenNT application processes requesting them. Process control, signal processing, file system access, and input/output functionality is provided via the subsystem.
OpenNT processes communicate with the subsystem via the same fast Local Procedure Call (LPC) and shared memory mechanisms used by Win32 applications communicating with the Win32 subsystem. There is no performance penalty compared to the Win32 subsystem as they each use similar mechanisms to provide kernel functionality to their client processes.
Terminal (tty) semantics are provided in console windows with OpenNT. For each process group and session, a console session manager is started that manages the processes I/O to a console window.
The subsystem sits on top the Windows NT file system (NTFS). There is no container file system mimicking UNIX file system semantics. NTFS provides the functionality required by the POSIX standards (file name case sensitivity, group ownership, and hard links). This functionality is not reachable from the Win32 world at this time, but is easily available via the OpenNT subsystem.
The socket functionality is provided through the subsystem through Winsock, so the TCP/IP stack in place on Windows NT is used.
Integration with Win32
A question often asked about OpenNT is "how does one use Win32 interfaces in a ported application?" The Windows NT architecture is such that applications that are a client of one environment subsystem cannot make calls that are supported by another subsystem.This mixing of development paradigms is not a feature if protecting the application development investment is of interest to an organization. Adding interfaces from another programming model destroys the application's forward and backward portability to other architectures, locking the application to the platform providing the mixed model.
However, few useful applications exist in a vacuum. Once deployed, the need to share data between applications becomes a necessity, and new applications are created in the space between existing applications. The model for integration between the Win32 world and OpenNT happens at a higher level than the application programming interface. Applications source code is ported directly to Windows NT to run with the OpenNT subsystem. This protects the application investment. Integration with the Win32 world can then take place in a number of ways:
- NTFS: The Win32 and OpenNT worlds share a common file system. Files created in one world can easily be reached in the other. As well, the security and auditing features provided by Windows NT and managed through the Win32-based administration tools are all available to OpenNT ported applications.
- Win32exec: An early feature demanded by customers was the ability to execute Win32 based applications from the OpenNT world. A Win32exec ability was added that allows Win32 GUI and command-line applications to be executed from within an OpenNT process. This means the lp utility becomes a simple shell script wrapper around the Win32 PRINT.EXE command. All the functions available on the network printing system are instantly available in a manner most appropriate to the architecture, while providing an interface most appropriate to a traditional UNIX environment.
- Desktop Interaction: An OpenNT tty is a Win32 console. This means large windows and screen buffers, scroll bars and cut-and-paste are all available at the user interface. An OpenNT console window behaves like an xterm. Cut-and-paste between Win32-based applications and OpenNT applications is flawless.
- Winsock: Applications can communicate in client/server fashion using sockets and the TCP/IP protocol. An Apache web server running on the OpenNT subsystem communicates with the Internet Explorer running on the Win32 subsystem.
- Security: As a native environment subsystem, OpenNT fits cleanly into the robust security model presented on Windows NT. There is no /etc/passwd or /etc/groups files, but rather all authentication happens against the Windows NT security subsystem.
From an end-user's point of view, he simply has a "desktop" full of applications. He cares which subsystem the application is communicating with for kernel services about as much as he cares in which language the programmer wrote the source code.
Client/server applications can be architected to use either the Win32 GUI for client code, written on Winsock in the Win32 subsystem (runnable on Windows NT and Windows 95), or maintain the traditional UNIX client code with its X11 or simple character interface. Server code is maintained as traditional UNIX code running on the OpenNT subsystem. The long term flexibility of the solution is completely in the customer's hands.
Client/server applications can be architected to use either the Win32 GUI for client code, written on Winsock in the Win32 subsystem (runnable on Windows NT and Windows 95), or maintain the traditional UNIX client code with its X11 or simple character interface. Server code is maintained as traditional UNIX code running on the OpenNT subsystem. The long term flexibility of the solution is completely in the customer's hands.
A real example of this is the OpenNT port of the Apache web server. The Apache server source code plus all the existing Perl, CGI, html scripting from the UNIX environment comes directly into the Windows NT world on the OpenNT subsystem. The Microsoft Internet Explorer or Netscape browser are Win32 "clients" that can then be used to connect to the web server.
Likewise, the telnet daemon shipped with OpenNT 2.0 is a direct port of a traditional UNIX telnetd running on the OpenNT subsystem. It further runs as a Windows NT service, controlled by either the OpenNT command line service utility or the Win32 GUI service control applet in the Windows NT Control Panel. Any telnet client, local or remote, Win32-based or UNIX-based, can connect to it.
Porting Experiences
Application code created in a traditional UNIX environment ports very easily to OpenNT. Much of the early utility code came from the original 4.4BSD-Lite distribution. Several larger examples of application code that has ported within a few hours include:xv
Xv is an X11 based JPEG and GIF viewer with a full file browser built-in. It's about 75,000 lines of code. It was ported in a few hours.apache
Apache is the popular public domain web server. The distribution is about 45,000 lines of code. The platform specific "stanza" in the configuration header was the same dozen lines as most other platforms.gnu code
A number of gnu utilities have been ported to OpenNT. The gawk utility source code ships as an example with the product. It is about 17,000 lines and ported after the minimal changes to update <varargs.h> to <stdarg.h> were made.Perl 5.0
The complete Perl 5 distribution (78,000 lines) was ported in a few hours. The configuration scripts are the only way to set the options for the build, and this required the most time. Once the configure scripts had built the "build" scripts to build the makefiles, the source code build essentially ran to completion.Summary
It has become too expensive to continually rewrite applications to move them from system to system; and source code portability is an important tool to protect existing applications investments. OpenNT provides the facilities of a traditional UNIX system on Windows NT, so that existing applications developed on traditional UNIX systems can be directly brought to Windows NT and deployed. Using the environment subsystem architecture of Windows NT, a peer environment to the Win32 world exists, and is integrated to that world in a manner most logical to both.Up-to-date information about OpenNT can be found at: http://www.opennt.com
About the Author:
Stephen R. Walli is vice-president, R&D for Softway Systems, Inc. He was the author of "Go Solo", X/Open's 1995 book on the Single UNIX Specification and UNIX 95. He is a long time participant in the POSIX standards efforts at both the IEEE and ISO. He can be reached at stephe@opennt.com.