In vocational training books, experimental construction, and increasingly learning software, are typical media. Usually, these media have little links, what means that much mental effort must be spent for associating the differently mediated portions of knowledge. Beyond the widely proposed advantages of multimedia learning software, there are two shortcomings of these systems. Firstly, learning only with virtual media leaves out the concrete handling with tangible objects and the experience of physical phenomena. Secondly, learning with computer-based media has some tendency to leave out team oriented problem solving and collaboration because the typical PC/Workstation hardware does not provide appropriate multi user usability. The project BREVIE presented here, aims at overcoming both of these drawbacks with a new kind of technology. This is capable of coupling closely diverse learning media, such as simulation software, hypertext and multimedia as well as tangible and graspable experimental boards. This is the basis for setting up collaborative learning environments usable for groups of students. The concepts of this new kind of learning media are presented, and its application to courses in pneumatics as a subject in the domain of industrial/vocational training is discussed.

Keywords

Learning Environments, Vocational Training, Pneumatics, Virtual Reality, Real Reality

Introduction

Engineering disciplines such as mechanical engineering or production engineering have a close association to reality, that is, beyond the theoretical knowledge the students gain in their studies there is always the need to transfer this knowledge to the physical reality where ideas and concepts become tangible. In particular this is true for vocational training where students are prepared to do tasks with a practical application-oriented background. Therefore, it is very common in vocational training to teach theoretical knowledge as well as its use in typical fields of application. However, in several projects with participants from vocational training schools we recognised the demand for a learning media which meets the requirement of covering both learning objectives: acquisition of basic theoretical knowledge and the ability to make use of this knowledge for solving practical problems. Conventional learning methods usually employ different media such as books, blackboards and - with increasing acceptance - multimedia learning software and digital simulators. Furthermore, we often find physical construction kits for making practical experiments with small models of real world situations. On the one hand, for the students this is the point where they apply their knowledge for building a real artefact. On the other hand, building something helps gaining, confirming and manifesting the learned. Thus, we see there is a diversity of things used as learning media for teaching. Although it might be advantageous to have a broad range of alternatives, there remains the difficulty of putting them together in a sound user-oriented way.

From our point of view, this is particularly true for the use of physical construction kits which have been proven to be a highly valuable tool for getting insight to the functionality of technical systems. In conventional teaching practice, however, these construction kits are used with loose links to other learning media. Therefore, we propose a new kind of technology that interfaces construction kits to other computer-based media. This interface mechanism bridges the physical reality and the computer-internal virtual reality by recognising what users do, or have done, with their hands. This means practically, the computer tracks user activities and their acting with their hands on graspable physical parts. This tracking allows the recognition of user operations in the real world and an update of a parallel virtual world in the computer. By this, it is possible to keep an image in the computer of what users do in the real environment. This image can be used to link other computer-based systems such as simulators or multi-media software to that what exists outside the computer.

The Educational Multi-Media project BREVIE (Bridging Reality and Virtuality with a Graspable User Interface), presented here, makes use of this kind of bridge. The project aims at coupling construction kits for pneumatics used as learning media for teaching in the area of production control and automation. These construction kits are linked to application software for simulating dynamic systems and to an online help function that provides information about the components of pneumatic circuits. The combination of a physical construction kit, simulation software and multimedia systems constitutes a new kind of learning environment which can be used simultaneously by several users. With the construction kit they can build models, then the interface creates a virtual image, and this can further be used for experiments with a simulator. Furthermore, while building models, users may ask the computer for information about single components, simply by selecting them in reality. Because the user attention is not focussed to visual computer output only, but also to a physical model which can be viewed and accessed from different positions, the learning environment may be used by small groups of students. This has additional positive effects on the learning process. The students can learn from each other, they solve problems collaboratively and beyond the pure technical knowledge, they gain some experience in social behaviour (Bruns 1996; Hornecker 1998).

Related Work

In 1993 Bruns et al. laid the foundations for a new class of user interface in shop floor and handicraft working (Bruns 1993, Bruns et al. 1993). The main characteristic of the above described "Real Reality User Interface (RR)", as they called it, is the use of the user's hand as a manipulator of physical objects in the real environment. Appropriate interface devices like data gloves and tracking systems are used for capturing the users' hand movements and operations. The term Real Reality emphasises the difference to Virtual Reality, where the user immerses in and is surrounded by a visual interface. Real Reality means staying in and experiencing the real world where all human senses are attracted, and communication within groups is encouraged. The interface becomes a passive observer and is ideally not noticed by its users.

A comparable approach frequently referenced to is Augmented Reality (AR) where the user's view of his physical environment is merged with computer-generated images (Feiner et al. 1993; Milgram et al. 1995). In contrast to RR, the main emphasis of AR is enhancing reality with information stored in the computer but not on creating new models collaboratively in groups.

Although the idea of combining physical entities and virtual objects is investigated by several researchers, their aims as well as their areas of application differ from ours crucially. Fitzmaurice et al. (1995 & 1996) propose a Graspable User Interface. They use tracking sensors as physical handles (bricks) for controlling virtual objects. The bricks are located on a flat screen and are logically linked to their visible virtual counterparts, thus moving a brick with a hand yields to a movement of the attached virtual object. They propose a new kind of drawing program with the option of using several bricks simultaneously. At MIT's Media Lab this approach is currently improved in the Tangible Media project (Ishii & Ullmer 1997; Ullmer & Ishii 1997). Instead of flat 2D models they additionally use stereo vision and 3D geometrical models, displayed on a desk-like device. The movements of physical handles such as cubes or pyramids are mapped to their graphical counterparts. By doing so, graphical user interfaces are enhanced with physical embodiments of their elements. Even farther reaching is the concept of Ubiquitous Computing (Weiser 1993). Computational functionality is embedded in many physical artefacts and spread throughout the real environment. A behaviour construction kit (Resnick 1993; Resnick et al. 1996) allows the composition of models consisting of computerised LEGO bricks equipped with electronic sensors. These can be programmed with LEGO/Logo, so they can interact with users or other physical objects.

The Real Reality concept can be applied to a variety of problem domains. The artec research centre at the University of Bremen is particularly interested in developing new tools for coping with complexity in the area of industrial production. We are currently developing a Real Reality construction kit for planning factory layouts (Brauer 1996; Brauer & Bruns 1996; Schäfer et al. 1997). Furthermore, for 3D geometrical modelling and surface shaping different approaches are investigated that make use of the flexibility and dexterity of the human hand to compose models from modular building blocks or to create shapes from a workable mass.

A Collaborative Learning Environment

From our point of view, new learning media should take account of the increasing requirements young people are faced to if they become active in a job. Vocational training, today, has to prepare students for jobs dealing with highly complex technology. The domain of production control and automation is a good example for this. Networked dynamic logistic systems of plants and machines are high-tech artefacts which are hard to understand and to control. As a contribution to this application domain we propose a collaborative learning environment for teaching and learning pneumatics.

Pneumatics, like hydraulics, can be considered as a basic technology in the area of industrial production. In vocational training of pneumatics one can find several kinds of learning materials for teaching. For us, the most compelling is the very popular pneumatic construction kit which can be used for building functional pneumatic circuits. These work with compressed air much like under real conditions in a production environment. The circuits are built by placing elements such as cylinders and valves on a switchboard, these are connected with tubes, and under pressure condition the circuit has some dynamic behaviour according to the laws fluid dynamics.

 

Figure 1: A simple pneumatic circuit consisting of two cylinders, valves and some tubes, on the left built with a construction kit, on the right created with a simulator.

A pre-study showed that there are some difficulties to build functionally correct circuits with the construction kit only from the theoretical knowledge previously learned, and that there are crucial differences between building a circuit with real components and creating a circuit diagram with an editor of a simulator. However, both capabilities are required from engineers or skilled workers in real life. They have to be able to read and understand the abstract level of a symbolic language as well as the concrete level of physical parts and elements. Therefore, in our design, we are building a learning environment around the physical level, where groups of users can build circuits and change easily their view on the composed circuit. Based on the computer's internal image of the physical circuit, additional representations can be generated, linked together and displayed according to the users' needs.

We are currently working on a recognition system which makes use of computer vision technologies to capture the circuit built with the construction kit. This results in a geometrical and a symbolic description of the circuit. This can be imported to a 3D Viewer and a simulation software package for pneumatics. With simple actions, like pointing at components, the users can surf in virtuality to more detailed information about the functionality and behaviour of pneumatic parts.

Figure 2: The collaborative learning environment for pneumatics with its different views on the system. While building a circuit the users may employ the computer for getting information or for simulating what they have composed.

Acknowledgements

I gratefully acknowledge the contributions of my colleagues as well as our project partners to the progress of our research. Our work on the Real Reality concept is gratefully being sponsored by the German Research Community (DFG Br1556/2-3, Br 1556/3-2), the European Community (TAP MM1002), and the University of Bremen.

References

Brauer, Volker (1996). Simulation Model Design in Physical Environments. ACM SIGGRAPH, Computer Graphics, Vol. 30, No. 4, Nov. 1996, pp. 55-56.

Brauer, Volker & Bruns, Willi (1996). Bridging the Gap between Real and Virtual Modeling - A New Approach to Human-Computer Interaction. Conference Papers of the 2. IFIP 5.10 Workshop on Virtual Prototyping, Arlington Tx, 96.

Bruns, W., Heimbucher, A. & Müller, D. (1993). Ansätze einer erfahrungsorientierten Gestaltung von Rechnersystemen für die Produktion. Universität Bremen, artec Arbeitspapier Nr. 21, März 1993.

Bruns, Willi (1993). Zur Rückgewinnung von Sinnlichkeit. Technische Rundschau, Heft 29/30, Juli 1993, pp. 14 - 18.

Bruns, Willi (1996). Grasping, Communicating, Understanding - Connecting Reality and Virtuality. AI & Society, Oct. 1996; auch als artec Paper Nr. 44, Universität Bremen, pp. 6 - 14.

Feiner, S., MacIntyre, B. & Seligmann, D. (1993). Knowledge-Based Augmented Reality. Communications of the ACM, Vol. 36 No. 7, 1993, pp. 53 - 62.

Fitzmaurice, G., Ishii, H. & Buxton, W. (1995). Bricks: Laying the Foundations for Graspable User Interfaces. Proceedings of the ACM CHI Conference, 1995.

Fitzmaurice, George W. (1996). Graspable User Interfaces. PhD Thesis, University of Toronto, Department of Computer Science.

Hornecker, Eva (1998). Coupling Physical Artifacts and Abstract Representations. ED-Media 98. Freiburg, Germany.

Ishii, Hiroshi & Ullmer, Brygg (1997). Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. Proceedings of the CHI '97.

Milgram, P., Drascic, D., Grodski, J., Restogi, A., Zhai, S. & Zhou, C. (1995). Merging Real and Virtual Worlds. Proc. of the IMAGINA '95, Monte Carlo, Feb. 1995.

Resnick, M., Martin, F., Sargent, R. & Silverman, B. (1996). Programmable Bricks: Toys to think with. IBM Systems Journal, Vol. 35, Nos. 3 & 4, 1996, pp. 443 - 452.

Resnick, Mitchel (1993). Behavior Construction Kits. Communications of the ACM, Vol. 36 No. 7, 1993,
pp. 64 - 71.

Schäfer, K. Brauer, V. & Bruns, W. (1997). A new Approach to Human-Computer Interaction - Synchronous Modelling in Real and Virtual Spaces. Proc. of the Designing Interactive Systems Conference (DIS '97), Amsterdam, August 1997.

Ullmer, Brygg & Ishii, Hiroshi (1997). The metaDESK: Models and Prototypes for Tangible User Interfaces. Proc. ACM Symp. on User Interface Software and Technology, UIST '97.

Weiser, Mark (1993). Some Computer Science Issues in Ubiquitous Computing. Communications of the ACM, Vol. 36, No. 7, pp. 74 - 84.

Web-Sites

BREVIE: http://www.brevie.uni-bremen.de

RUGAMS: http://www.artec.uni-bremen.de/field1/rugams

EUGABE: http://www.artec.uni-bremen.de/field1/eugabe

artec: http://www.artec.uni-bremen.de