Putting People Power Into Virtual RealityCompetition among design technologies is as keen as that of car marques. At Ford, engineers combine virtual reality with old-fashioned human verification.
To make sure that the cars we design are the ones our customers will want to buy and drive with satisfaction, we maintain a design practice which "keeps a human in the loop" by frequent use of prototyping for design and verification. However, the competitive nature of a consumer-focused industry requires constant reductions in the time it takes to bring a new vehicle to market.
Ford engineers perform interference checking between vehicle components using Digital Buck, an entirely electronic prototyping tool. This virtual reality system, based on tessellated data, provides real-time stereo viewing and animation.
Traditional physical prototyping methods present a significant challenge to reducing design time. The problem is not so much that the physical prototype takes too long to create, but rather that the act of physically prototyping a component removes the design from the electronic medium. Thus, a great need exists for effective electronic human-in-the-loop prototyping techniques, which, in a broad sense, describes the field of virtual reality (VR).
Developing effective VR applications requires much more than creating a human interface to an electronic model. In their least effective form, virtual prototypes present all the design bottlenecks of physical prototypes with less than half the fidelity. VR developers have to understand the constraints of the design process so they can make the appropriate tradeoffs to optimize the return value of the technology. Only if the cost of implementation is significantly less than the return value will the technology be considered for implementation. If the cost ratio of VR is not favorable, other competing technologies are waiting for the resources. The number of new technologies available today far outstrips our ability to implement them.
The design and manufacturing of a vehicle is referred to as a program, and the length of time of the design phase, from the time concepts for the vehicle are discussed until the first production vehicle rolls off the assembly line, is known as the cycle time. Program cycle times vary widely based on the complexity of the vehicle and the amount of design content carried over from previous programs. An entirely new vehicle may have a cycle time of four years and involve as many as 6,000 people. Thus, if an auto manufacturer were to release two new vehicles per year and allow for changes in the overall cycle plan, there would be approximately 10 programs in progress at all times.
Full-scale holograms can be generated directly from CAD models, giving Ford engineers unencumbered visual interaction with concepts and styling. Shown here is the hybrid powertrain of the new P2000 high-mileage vehicle.
The automotive design process combines three factors that make it unique from other industries and particularly well suited to VR applications. First, given the complexity of the product and the magnitude of the process, high-quality results will require detailed understanding of intermediate data on a regular basis. Achieving this understanding, given the time constraints, requires a reliance on electronic prototypes. Second, delivering products to meet rapidly changing customer needs will require fundamental design and engineering changes that can be accomplished only by restructuring the design process. Third, efficiently managing production runs of 10,000 to one million units for such a complex product places a significant emphasis on integrating design for manufacturing techniques up front in the process. Thus, the automotive process requires frequent CAE models, rapid adaptability, and concurrency of all phases of design.
At Ford, getting tens of thousands of people to work in the same design environment generally leads to differences of opinion. Regardless of the difficulties, however, design tool evolution has to be a constant process; we are always preparing for the next generation of technology. Ideally, the process should determine which tools are developed and used, but when program deadlines are at stake, there is no choice but to use the most robust tool at hand. New tools are constantly being developed and integrated, and the constraints placed on this development are worth describing.
A common data structure is required. Due to the extreme degree of concurrency in the design process (the manufacturing facility is constructed before the product concept is complete), there must be as few barriers to simultaneous design as possible. Even slight differences in model formats between tools require translations. These will be constant sources of error and delay. New technologies are significantly easier to integrate if they function on the general foundation data structures of the process.
Two-way data transfer must be simple. Each area with design responsibility is progressively and simultaneously refining its portion of the product model. To facilitate the correct evaluation of design tradeoffs, changes must be propagated quickly throughout the product design system. This so-called change process, more than any other factor, determines the cycle time of a new product. New technologies that hinder data communication are difficult to use.
Simulation cycle time includes pre- and postprocessing, and must be small. A program moves quickly and decisions are made based on the best available information. Once decisions are made, the change process makes them progressively more expensive to revisit. The impact that any technique, VR or otherwise, has on a program is maximized when the time window required to collect the data, preprocess it into required models, run the simulation, and capture the results in a meaningful, transmittable form is minimized. A technology that requires significant transformation of the common data model makes this difficult to achieve.
Haptic technology is used to create force-feedback constraints based directly on the CAD model. The desktop configuration (left) is used for component layout and assembly prove-out. The immersive configuration (below) lets engineers perform ergonomic evaluations directly on full-scale vehicles as soon as conceptual design begins.
The quality of the simulation must match the tolerances required. Interestingly, tool developers tend to overapply this constraint some of the time. Many virtual technologies already have adequate fidelity to perform certain design tasks, particularly those measuring human interaction with conceptual geometry.
The simulation must contain all the pertinent information. Within the context of a particular decision a designer is interested in only a certain subset of information that can be called pertinent. A successful application will be developed to have high bandwidth along those channels of information flow that the designer considers important to the problem being addressed.
Simulations must be robust to withstand low quality or incomplete data. Producing an optimum design in a concurrent environment in a short time requires that information on design constraints be passed from one activity to another. The cross verification of design constraints is the so-called feasibility process. The hallmark of feasibility testing is that it must be accomplished with low-quality data that does not meet design tolerances or, worse still, incomplete data that does not yet constitute a proper CAD model. Any technology must be robust to withstand unmatched surface boundaries, duplicate nodes, swapped surface normals, and a host of other unanticipated problems.
The technology must work for industrial-size models. The models are massive compared to those that may be used to validate research algorithms. For example, a production vehicle chassis may have 3,000 to 5,000 surface patches and occupy 50 to 100 MB; a low-quality tessellation of a vehicle exterior may have 80,000 polygons. If a technology has to simplify such models in order to function properly, it will be severely affected by all six preceding constraints.
One of the challenges in describing the integration of virtual reality into an industrial design process is to define just what VR means. To some, VR refers to a specific collection of technologies—a head-mounted stereo display, glove input device, and audio. To others, it is a system that differs from traditional simulation systems to the extent in which real-time interaction is facilitated, the perceived visual space is three-dimensional rather than two-dimensional, the human-machine interface is multimodal, and the operator is immersed in the computer-generated environment.
In the automotive design process, VR can take on many forms. In this context, an application will be said to incorporate VR technology if it enables the user to "experience" a computer model, or some aspects of it, by putting the "human in the loop." Due to the complexity of the overall model, such a VR application will not be an all-encompassing simulation of the entire product or process. Instead, the diversity of the process does allow for individual components of the design to be decoupled for detailed examination. In some cases, the computer model will be a close replica of reality, in others, an abstraction. Regardless, a VR tool should let the user interact with the model in a manner that resembles a real-life interaction.
Some of the VR projects at Ford are in the research stage; others have already been integrated into Ford's product development process. Particular attention is given to how each implementation satisfies the design guidelines illustrated in the previous sections, and to when a tradeoff has been made to compromise between integration and real-time performance. At first glance, it might seem that some applications are redundant, but the requirements of different user groups make the applications quite different.
A Ford engineer uses a Powerwall from Silicon Graphics to examine a full-scale stereoscopic view of the aerodynamic flow for a Formula One car calculated with a proprietary CFD code.
Within Ford the application that most closely matches the conventional definition of virtual reality is Digital Buck. The term "buck" is used in the automotive industry to describe a full-scale prototype. Usually, this would be an entire vehicle, but the term is also used in the context of any subcomponent. Thus, Digital Buck is an entirely electronic prototyping tool that provides real-time stereo display with an emphasis on real-time visualization of complex models. The Digital Buck team of Juliet Kraal, Elisabeth Baron, and leader Dave Roberts, engineers within the CAD Product Information Department, are collaborating with Engineering Animation Inc. of Ames, Iowa, on the development of this tool.
One of the critical advantages of human interaction with an electronic model is that only those surfaces that will actually be touched are required. Unlike a physical buck, no substrate of any kind is required and those surfaces that are used can be moved at will. This is a tremendous advantage that allows human-in-the-loop evaluations early in the design cycle when there is not enough data to build a physical prototype. Certainly, the electronic prototypes are incomplete, but they contain all the pertinent information an expert engineer requires to make good decisions. The expert user is more interested in tool functionality than tool fidelity. Therefore, the tool can use a tessellated mesh in order to achieve real-time interaction. The mesh itself is regenerated daily from the central CAD model. Depending on the size of the model, Digital Buck requires one or two processors of a Unix workstation.
Given this architecture, Digital Buck is used primarily to verify rather than to modify a design. (For instance, it is used to verify that components do not interfere with each other or violate packaging constraints. Components that the customer will use must be checked against ergonomic constraints.) Changes made to a tessellated model are difficult, if not impossible, to communicate back to a structured CAD model. For example, when a NURBS model is tessellated, the analytical relationships between patches and the patch boundaries themselves are lost. Thus, any change applied to the tessellated model would be sent back to the NURBS model without reference to these key components.
With a different user community in mind—stylists and modelers—Tom Scott, the director of the Ford Advanced Design Studio, is leading an advanced "replacement reality" research project aimed at producing a different kind of 3-D visualization tool. To stylists, experiencing the sight lines and proportions of the vehicle is paramount, and must be done as unobtrusively as possible, thus excluding head-mounted displays and other bulky devices worn by the user. Instead, holography is used to provide a full-scale, unencumbered stereo view of a vehicle model or components. To date, full-size and half-scale holographic representations of entire vehicles have been generated. All of these representations are full parallax views.
The Ford Vibration Simulator is used to evaluate the road feel of a vehicle at high fidelity. This configuration allows engineers to test-ride and compare a large number of vehicles in a short time.
In future releases, the user will be able to interactively shape the CAD model as if he or she were sculpting the clay model, thus potentially making this tool the digital clay buck of the future. The fact that the tool can be used to interact with incomplete models represents another significant advantage compared to physical bucks. Moreover, in a desktop configuration, such a tool could also be used to gain valuable customer feedback and to visualize craftsmanship issues before any type of model is created. In contrast to the Digital Buck, which ultimately contains an electronic version of every component in the vehicle, the pertinent information for styling includes only visible geometry, the so-called customer surfaces. These can be captured at a stage when the CAD model is still incomplete and uncomplicated. The smaller size of the model allows this technology to be applied directly to the NURBS model, allowing a tighter integration with the CAD model.
In many cases, engineers are not so much interested in reality, so to speak, as in understanding complex data sets. As part of the Virtual Wind Tunnel project led by Gary Strumolo, a senior staff technical specialist in the Ford Research Laboratory, a complex CAE code has been developed to analyze the pressures and vibrations due to airflow over a vehicle directly from a geometric model. The resulting data is rich with information, but overly complex. To alleviate this problem, a Powerwall from Silicon Graphics Inc. of Mountain View, Calif., is used to stereoscopically immerse the user in the resulting three-dimensional flow fields. The 8 x 10-foot screen of the Powerwall is back-projected in stereo from an Onyx workstation that has dual graphics pipes and two processors. Traditional alternatives would not permit such a revealing "walk" through the data.
The time saved in running the airflow analysis on a computer model and the accuracy of relative analyses between design variations put the stress on rapidly forming new CAD test models. A challenge for successful integration of this technology is to reduce the overall simulation cycle time. In other words, the time required to generate the necessary input models must be reduced and is the subject of ongoing investigation.
The applications described so far emphasize visual feedback, yet the visceral experience of driving a car is many-faceted. Not only do customers see the car, but they also touch it, hear it, and smell it. To achieve the full customer experience directly from a computer model requires information traditionally conveyed visually to be transmitted through the appropriate sensory channel.
In the Haptic Buck research performed by the authors, visual and haptic (force-feedback) sensory inputs are combined to provide an experience in which an engineer can look at and touch a computer model, mimicking the interaction with a physical prototype. In its immersive configuration, the Haptic Buck uses a grounded stereoscopic display registered with a large 6-degree-of-freedom haptic device. The immersive workspace is capable of representing the interior of an entire vehicle.
The Haptic Buck targets the same range of applications as the Digital Buck, but provides a complementary set of answers to engineering problems that require evaluation of human proprioception (the unconscious perception of movement and spatial orientation arising from stimuli within the body itself) in interacting with a vehicle's interior and components. Since emphasis is on tactile perception, simulations can be run with limited and incomplete geometric information taken from the early stages of the design process. For example, it is essential to assess the reachability of different parts of the instrument panel directly from constraints developed for the product concept, known as package data.
In its desktop configuration, this application uses a flat screen with or without stereo glasses and a variety of small haptic devices ranging from 3 to 6 degrees of freedom. In this form, it is a powerful tool for collision-free path generation and verification of assembly tasks, an area where visual-only simulations have fallen short of the mark.
In this CAD model of a manufacturing cell on the plant floor, a crane assists the user in handling heavy payloads and is free to move about the work cell except for the green, yellow, and red funnel cones. These are virtual constraints that guide the payload to the correct position on the passing assembly line.
The Haptic Buck has been tailored to the tessellated data structures better suited for the fast rendering of large models and to real-time collision detection. For this reason, like the Digital Buck, it has limited ability to propagate geometric changes to the central CAD model. At the same time, many of the decisions that can be made with this tool do not relate to product geometry as much as component location and path planning. One significant challenge, however, is the relatively poor quality of the tessellated mesh, in terms of its mathematical definition. The time required to correct duplicate or missing facets, for example, must be realistically included in the simulation cycle time. On the plus side, though, is the fact that the decisions being made do not rely on a haptic touch fidelity that is too difficult to achieve, so a wide variety of force-feedback devices can be used.
Flexing the traditional definition of VR beyond immersive environments opens up many areas of the design process to the benefit of VR tools targeted at very specific aspects of the CAD model. In these cases, a physical buck is often used in conjunction with a sophisticated computer model. Ford has a suite of tools to study human factors, including the Driving Interaction, Vibration, and Acoustical simulators.
The Driving Simulator project, led by Jeff Greenberg, a staff technical specialist in the Ford Research Laboratory, seats the driver inside a physical cab surrounded by a projection screen that provides a faithful visual replica of the entire visible external environment. Ford engineers use this platform to characterize human driving performance and comfort, and to experiment with different driver interfaces and tasks. The tradeoff between model complexity and faithfulness is carefully balanced to allow for real-time interaction while, at the same time, guaranteeing the soundness of the collected data. A physical set of active vehicle controls provides the tactile immersion. The audio feedback does not have to be perfect, but only representative. Of greater importance is the user's ability to focus in the vehicle when all senses are active. Simulation cycle time is a limitation, however, since this approach requires an existing prototype and significant setup time. At the same time studies that would be impractical on the test track, such as driver drowsiness, can now be simulated in depth.
Ford's Vibration Simulator and Acoustical Simulator complement the Driving Simulator. The Vibration Simulator, led by Ray Meier, a senior technical specialist in the Ford Research Laboratory, provides a full-scale realistic sensation of sitting in a car seat, holding the steering wheel, and feeling the vibration one would sense from the road. No visual images, no audio, and no dynamic road handling are reproduced—just the playback of this single decoupled realm (actually, 11 degrees of freedom) of the driving experience. In fact, it is the decoupling of the experience that makes this tool so successful for product evaluation. Future challenges are to generate vibration data directly from the CAD model. This, of course, will require a sophisticated interpretation of mechanical properties inferred from CAD model geometry.
The Acoustical Simulator, developed by Scott Amman and Mike Blommer, technical specialists in the Ford Research Laboratory, replicates the audio signal that a customer perceives while seated in different locations within a vehicle. In this case, however, the signal can be either taken from a road recording or generated directly from the CAD model.
Ford's augmented manufacturing research, led by Tom Pearson, a senior technical specialist in advanced manufacturing technology development, is an example of VR applied directly to the manufacturing process. Intelligent assist devices—conventional lift and hoist machinery guided by sophisticated control devices—are proposed to amplify human performance in material handling and assembly tasks. Virtual constraints that describe the acceptable boundaries within which valuable parts may move within an assembly cell are obtained directly from the CAD models of the assembly line and the vehicle. These constraints are then enforced by the intelligent assist devices so that a worker is free to move a heavy part throughout the work cell with the assist device, but is prevented from moving the part into regions where worker safety is compromised or the product may be damaged. In such a scenario, virtual surfaces and real auto components share the same workspace, and constraint information is conveyed to the user haptically.
Any successful application of virtual reality must address many issues related to the quality of the simulation. Specific examples include expanded field of view from display devices, stable position sensing from nongrounded displays, and accurate registration of multiple display devices and physical components. And beyond these issues are the psychological and sociological barriers to implementation that any new technology must overcome.
The most successful VR applications at Ford have targeted very specific tasks within the product development cycle, have balanced the seven design process constraints described here, and have added significant value to the product by shortening the design cycle and increasing the human-in-the-loop understanding of the product design—all without incurring a high process cost. While there may someday be a so-called "killer application" that revolutionizes the design industry, it is more likely that the steady, incremental migration VR implementations will provide the consistent long-term evolution required for VR to be considered a successful technology.
