«ISSN 1613-4907 artec | Forschungszentrum Nachhaltigkeit Enrique-Schmidt-Str. 7 Postfach 330 440 28334 Bremen A generalized concept to blend physical ...»
F. Wilhelm Bruns
-Enabling Mixed Realityartec-paper Nr. 82
artec | Forschungszentrum Nachhaltigkeit
Postfach 330 440
A generalized concept to blend physical systems with their virtual counterparts, being their
computer-internal representation or a functional continuation will be introduced. Hyper-Bonds
combine the unified
systems representation of bond graphs with their concrete implementation in a “hyper-connection” between physical phenomena of the computer-external environment and the logical structure of computer-internal representations in a vivid way. The theory is laid down and a first application in a learning environment for electro-pneumatics, allowing a free determination of boundaries, is shown. Further applications and extensions are highlighted.
1. Introduction Mixed realities are becoming more and more popular. They represent a kind of coupling of real world phenomena to various information, represented within a computer. This might take place in the sense that information is displayed and overlaid to real phenomena (see Augmented Reality research: Weiser 1993, Wellner 1993), in the sense that real phenomena are used as handles to manipulate information (see Graspable User Interface research: Fitzmaurice et al
1995) or in the sense that both worlds are bi-directional tight together (Ishii & Ullmer 1997, Brave et al 1998). The focus of this paper is the later approach.
In some previous work, we introduced the concept of complex objects (Bruns 2000) being objects with a real concrete part coupled to various virtual representations by means of graspor image-recognition, Fig. 1. The fruitful application of this concept in several environments for learning and systems design has been demonstrated in some school applications (Grund & Grote 2000). There is also strong evidence from other authors, that constructivist learning based on concrete objects has many advantages (Breretron & McGarry 2000). At least this can be observed for certain learning styles.
Fig. 1: Complex Object This coupling so far has been only unidirectional, giving the possibility to build and change real systems and synchronously generate their functional representatives. Simulation could then be carried out with the virtual model and compared with the behavior of the real system, but no interactive bi-directional connection was realized. However it was possible, to download the controlling algorithm, driving the virtual model, as a whole to a Programmable Logic Controller (PLC) and drive the physical system by means of sensors and actors, Fig. 2. Our first extension of this concept was a bi-directional pneumatic link between the virtual and the real model, being able tosense and generate all relevant air pressure and flow phenomena via one connection (Bruns&Gathmann 1999), Fig. 3.
The concept is now generalized towards further physical phenomena, from pneumatics, hydraulics, electrics, mechanics, thermodynamics. The unifying concept allowing this approach is the Theory of bond graphs (Paynter 1961, Karnopp 1995). Its implementation is done by PLC-technology.
At first, a short recall of bond graphs is given and their usefulness for this approach shown. A short motivation for Hyper-Bonds from a system dynamics point of view follows. In section 3 the detailed concept of Hyper-Bonds is introduced. Some applications and perspectives finish this contribution.
2. Usefulness of Bond Graphs
Paynter introduced the theory of bond graphs as a unifying view on physical phenomena from a continuity of power-flow perspective. Power flows through system components and connections by way that the product of effort and flow is continuous, following typical laws of energy conservation and power-flow continuity. Effort (e) is the driving force for flow and can be a pressure difference, force and torque, electrical potential difference, temperature difference etc. Flow (f) can be a flow of material, momentum, electric current, entropy. Bond graphs are networks with edges of (e,f)-connection and nodes with constant e (0-node) or f (1node). Bond graphs can be used to describe dynamic behaviors of different physical domains with one formalism (Fig. 4-5), similar the way we use differential equations for all kinds of physical phenomena. In fact, for calculation means, bond graphs are transferred to systems of differential equations and then integrated (symbolic or numerical). For engineering purposes bond graphs have several advantages as they are vivid and preserve important constraints (Cellier et al 1995). One important feature of bond graphs is the input-output relation of effort and flow, seen from a physical component perspective. Components are always connected by bonds having the value pair e and f where one of them can be seen as input the other as output from a cause-reaction point of view. However, which one is input and which one output can only be determined from an overall systems view by causal reasoning. Knowing e and f at one connection, resulting from calculation or measurement, allows a cutting of the system in two parts and a separate investigation.
Fig. 4: Bond graph for equivalent mechanical and electrical systems Fig. 5: Bond graph for a pneumatic cylinder with different levels of abstraction Bond graphs are especially useful for the description of mechatronic systems, where powerflow is transformed and exchanged between different forms of energy, Fig. 6. Several simulators now support the notation of bond graphs.
The feature of effort and flow, determining a systems behavior can be used fruitfully, to implement arbitrary cuts in an overall system, realizing one part in reality, the other one in virtuality and provide a mechanism to sense and generate arbitrary efforts and flows. This will be demonstrated.
3. Systems and Boundaries
In physical science, and not only there, we learn the principle of cutting a system at well defined boundaries and replacing the external influences by some observable and measurable relevant variables, reducing the investigation to the internal dynamics of the rest. In laboratory work we use this principle to construct reproducible experiments, but also mentally we use it to think about systems in hypothesis and mental experiments (consider the principle of virtual work applied by d’Alembert). For future laboratories, being more and more penetrated by computers, a free and easy distribution of a system between reality and virtuality seems to have some advantages (Fig. 7). Certain well known aspects of a system can be represented in a formal way by algorithms in the computer, others to be investigated in more detail and uncertainty are represented in reality, but coupled to a dynamic surrounding. This would allow completely new forms of easy experimental work and learning.
4. Hyper-Bonds: a sensing and generating Mechanism In order to provide arbitrary boundary conditions, we must have a mechanism to switch between source and sink and to generate one phenomenon and sense the other. We chose to generate the effort and sense the flow, because this has the broadest application area, as we will see. The alternative would yield some different implementation. The implementation for electrics (voltage and current) and pneumatics (pressure and volume-flow) is given in Fig. 8, together with its abstract bond graph representation.
Fig. 8: Hyper-Bond for Voltage/Current- or Pressure/Volume-Flow As all necessary components are available in a standard mechatronics construction kit, we built a reduced version of a prototype (Fig. 9) and then multiplied and miniaturized it.
The integration into a virtual construction and simulation environment has been done by some parallel digital and analog I/O ports, Fig. 10. To be more flexible, the I/O can be handled by a separate box or PLC, connected to the PC via bus or serial interface.
5. Applications and further Perspectives The new concept of Hyper-Bonds is being applied in a learning environment for electro-pneumatics (Fig. 11), where students can work on complex systems, freely switching between virtuality and reality 1. Especially if one is interested in a work-process-oriented learning, using authentic situations and always wants to see the complex context, it is of high value, to be able to select certain interesting aspects of a system, put them as real components on ones desk, but still have them connected to and integrated in the overall system. As the virtual model can easily be distributed over different locations (only with some restrictions in time behavior), one has the possibility to have one complex virtual system materialized in parts at one location and in parts at other locations. This opens completely new perspectives for distributed task oriented learning in a co-operating group. For systems development our concept would support an incremental implementation and testing of complex devices. For service and maintenance it would support the stepwise investigation and repair of disintegrated parts.
We have shown so far how to implement Hyper-Bonds in the domains of electrics and pneumatics, the implementation for hydraulics, thermodynamics and mechanics should follow.
EU-IST Project DERIVE (Distributed Real and Virtual Learning Environment for Mechatronics and TeleService Fig 11: DERIVE Learning Environment References Brauer, Volker. Simulation Model Design in Physical Environments. Computer Graphics ACM Siggraph vol 30, no. 4, pp. 55-56, 1996 Brave, S., Ishii, H., Dahley, A.: Tangible Interfaces for remote Collaboration and Communication. Proc. Of CSCW ’98, Nov. 14-18, 1998 Breretron, M., McGarry, B.: An Observational Study of How Objects Support Engineering Design Thinking and Communication: Implications for the design of tangible media.
CHI 2000 Conf. Proceedings, acm press, pp. 217-224, 2000 Bruns, F. W.: Zur Rückgewinnung von Sinnlichkeit - Eine neue Form des Umgangs mit Rechnern, Technische Rundschau, 29/30, 1993, pp. 14-18 Bruns, F. W.: Complex Construction Kits for Coupled Real and Virtual Engineering Workspaces. CoBuild `99 Bruns, F. W., Gathmann, H.: Auto-erecting Agents for a collaborative Learning Environment.
Proc. of 8th IEEE Int. Workshops on Enabling Technologies: Infrastructures for Collaborative Enterprises, June 16-18, Stanford, 1999, 287-288 Cellier, E. F., Elmqvist, H., Otter, M.: Modeling from Physical Principle. In W. S. Levine (Ed): The Control Handbook, CRC Press, Boca Raton, 1995, pp 99-108 Fitzmaurice, G. W., H.. Ishii, W. Buxton. Bricks: Laying the Foundations for Graspable User Interfaces. CHI’95 Mosaic of Creativity, 1995, pp. 442-449 Ishii, H., B. Ullmer, B. Tangible Bits: Toward Seamless Interfaces between People, Bits and Atoms. CHI´97, Atlanta, Georgia, 1997 Karnopp, D. C., Margolis, D. L., Rosenberg, R. C.: System Dynamics – A unified Approach.
John Wiley, New York, 1990 Paynter, H. M.: Analysis and Design of Engineering Systems, MIT Press, Cambridge, MA, Weiser, M.: Some Computer Science Issues in Ubiquitous Computing. In: Communications of the ACM, 36/7, July 1993 Wellner, P.: Interacting with Paper on the DigitalDesk. In: Communications of the ACM, 36/7, July 1993 Acknowledgement
I would like to thank my colleagues contributing to this concept in theory and practice:
Herman Gathmann, Hauke Ernst, Jürgen Huyer, Kai Schmudlach, Rainer Pundt.
Our research is being supported by the Deutsche Forschungsgemeinschaft DFG (G-Nr. Br 1556/2-3, G-Nr. Br1556/3-3) and the EU (MM1002 and IST2031).
Further related publications
Bruns, F. W. (2001): Learning in mixed Realities. Proceedings of i3 Spring Conference:
(inter)facing tomorrow. Workshop on Moving between the physical and digital:
exploring and developing new forms of mixed-reality user experience, 23.-25. 4.
2001, Porto Bruns, F. W. (2001): Web-based cooperative Teaching and Learning of Mechatronics in vocational Education – New Forms of mixed Realities. Proceedings of i3 Spring Conference: (inter)facing tomorrow. Workshop on Web Based Platforms for Cooperative Learning, 23.-25. 4. 2001, Porto Hornecker, E. / Robben, B. / Bruns, F.W. (2001): Technische Spielräume: Gegenständliche Computerschnittstellen als Werkzeug für erfahrungsorientiertes, kooperatives Modellieren. In: Ingo Matuschek/ Annette Henninger/ Frank Kleemann (Hrsg.) (2001): Neue Medien im Arbeitsalltag. Empirische Befunde, Gestaltungskonzepte, Theoretische Perspektiven. Opladen, Westdeutscher Verlag (im Erscheinen) Schäfer, K., Bruns, F. W. (2001): PLC-Programming by Demonstration with graspable Models. Proceedings of 6th IFAC Symposium on Cost Oriented Automation. Berlin, 8.-9. 10. 2001, in print Bruns, F. W. (2000): Complex Objects and anthropocentric Systems Design. In: L.M.
Camarinha-Matos, H. Afsarmanesh, H.-H. Erbe (Ed.), Advances in networked Enterprises: Virtual Organizations, Balanced Automation and Systems Integration, Kluwer Academic Publishers, S. 249-258 Hornecker, E. / Bittner, P. (2000): Kritisch studieren... und dann? Editorial der FIfFKommunikation 1/2000. (S. 3, S.8) Hornecker, E. / Bittner, P. (2000): Vom kritischen Verhältnis zur Berufspraxis in der Informatik - Ergebnisse einer Befragung. FIfF-Kommunikation 1/2000, (S. 33-39) Hornecker, E. / Robben, B. / Bruns, F.W. (2000): Technische Spielräume: Gegenständliche Computer-Schnittstellen als Werkzeug für erfahrungsorientiertes, kooperatives Modellieren. In: Neue Medien Im Arbeitsalltag -- Empirische Analysen, Gestalterische Impulse, Theoretische Befunde. Interdiziplinäre Tagung der DFGForschergruppe "Neue Medien im Alltag" in Chemnitz, 10. und 11. November 2000.
Workshopreader Müller, D. (2000): Hypermediagestützte Simulationssysteme - Neue Software-Werkzeuge in
der beruflichen Bildung. In: Arbeitsgemeinschaft Berufliche Bildung (Hrsg.):
Hochschultage Berufliche Bildung 1998. Neusäß. S. 197-208 Müller, D./Ernst, H. (2000): Eine Konzeptstudie zur Realisierung des Autorensystems MentorAuthor. Forschungsbericht, Universität Bremen. 2000