WWW.ABSTRACT.XLIBX.INFO
FREE ELECTRONIC LIBRARY - Abstract, dissertation, book
 
<< HOME
CONTACTS



Pages:   || 2 | 3 | 4 | 5 |   ...   | 13 |

«A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE STUDIES (Department of ...»

-- [ Page 1 ] --

PERSONALIZED FACE ANIMATION FRAMEWORK

FOR MULTIMEDIA SYSTEMS

by

Ali Arya

B.Sc., Tehran Polytechnic, Iran, 1989

A THESIS SUBMITTED IN PARTIAL FULFILMENT OF

THE REQUIREMENTS FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

in

THE FACULTY OF GRADUATE STUDIES

(Department of Electrical and Computer Engineering) We accept this thesis as conforming to the required standard.

THE UNIVERSITY OF BRITISH COLUMBIA

February 2004 © Ali Arya, 2004 Abstract Advances in multimedia-related technologies are enabling new applications such as virtual agents, video conferencing, visual effects in movies, and virtual players in computer games. Such applications are, in turn, motivating much research in digital character and face animation. This thesis addresses an important area in this field, Personalized Face Animation which is concerned with creating multimedia data representing the facial actions of a certain character, such as talking, expressions, and head movements. Much success has been achieved for this purpose using 3D head models (general and customized to specific individuals) and also view morphing based on 2D images. The model acquisition and computational complexity of 3D models, and large image databases for 2D methods, however, are major drawbacks. The thesis addresses these issues along with other important ones, mainly realism, authoring tools, content description, and architecture of the whole face animation system.

We propose a comprehensive framework for personalized face animation which we call ShowFace. ShowFace integrates a component-based architecture, well-defined interfaces, helper objects and tools with a simple, yet effective, approach to content generation. These are paired with a language for describing face animation events. ShowFace

is designed to satisfy the following basic requirements of face animation systems:

• Generalized decoding of short textual input into multimedia objects that minimizes the model complexity and database size

• Structured content description for face activities like talking, expressions, and head movement, their temporal relation, and hierarchical grouping into meaningful stories

• Streaming for continuously receiving and producing frames of multimedia data

• Timeliness issues

• Compatibility with existing standards and technologies and

• Efficiency with regards to algorithms and required data

ShowFace achieves this objective by introducing:

ii

• Feature-based image transformations along with a 2D image-based method for creating MPEG-4 compatible and realistic facial actions. This is accomplished without the need for a complicated 3D head model and/or large databases of 2D images

• A face modeling language which is an XML-based language. It is compatible with MPEG-4 standard and specifically designed for face animation It is also capable of describing spatial and temporal relations of facial actions, behavioural templates, and external event handling.

• A component-based structure for development of animation applications. This structure has a well-defined interface, independently usable components, and streaming capability and

• A comprehensive set of evaluation criteria for face animation systems The thesis review basic concepts and related work in the area of face animation. Then the ShowFace system is introduced and its contributions are thoroughly discussed. A comparative evaluation of the system features and performance is also provided.

iii Table of Contents ABSTRACT

TABLE OF CONTENTS

LIST OF TABLES

LIST OF FIGURES

LIST OF ABBREVIATIONS

ACKNOWLEDGEMENTS

1. INTRODUCTION

1.1. BACKGROUND AND MOTIVATIONS

1.2. PERSONALIZED FACE ANIMATION

1.2.1. Problem Model

1.2.2. Objectives

1.3. THESIS STRUCTURE

2. RELATED WORK

2.1. CONTENT DESCRIPTION

2.2. CONTENT CREATION

2.2.1. Visual Content

2.2.2. Audio Content

2.3. SYSTEM ARCHITECTURE

2.4. EVALUATION CRITERIA

3. FACE MODELING LANGUAGE

3.1. DESIGN IDEAS

3.2. FML DOCUMENT STRUCTURE

3.3. MODELING ELEMENTS

3.4. STORY-RELATED LANGUAGE CONSTRUCTS

3.4.1. FML Time Containers and Moves

3.4.2. Iterations in FML

3.5. EVENT HANDLING AND DECISION MAKING

3.6. COMPATIBILITY

3.7. CASE STUDIES

3.7.1. Static Document

3.7.2. Event Handling

3.7.3. Dynamic Content Generation

4. FEATURE-BASED IMAGE TRANSFORMATIONS

4.1. BACKGROUND: OPTICAL FLOW-BASED APPROACH

4.1.1. Structural Overview

4.1.2. Optical Flow Field

iv 4.1.3. Map Validation and Filtering

4.1.4. Image Warping and Final Morph

4.1.5. Experimental Results

4.2. FEATURE-BASED APPROACH: BASIC CONCEPTS





4.3. FACIAL STATES AND FEATURES

4.3.1. Identifying Facial States and Transitions

4.3.2. Detecting Facial Features

4.3.3. Feature Translation Functions

4.4. MAPPING FACIAL FEATURES

4.5. IMAGE WARPING

4.5.1. Facial Regions

4.5.2. Warp Function

4.5.3. Newly Appeared Areas

4.6. TEXTURE TRANSFORMATION

4.7. SUMMARY OF FIX FEATURES AND ADVANTAGES

5. SHOWFACE SYSTEM

5.1. SYSTEM ARCHITECTURE

5.1.1. Requirements

5.1.2. Basic Structure

5.1.3. Streaming

5.2. SPEECH SYNTHESIS

5.3. APPLICATION DEVELOPMENT

5.3.1. SF-API

5.3.2. ShowFacePlayer

6. EVALUATION AND EXPERIMENTAL RESULTS

6.1. EVALUATION CRITERIA

6.1.1. Criteria Categories

6.1.2. Content

6.1.3. Architecture

6.1.4. Development

6.2. EXPERIMENTAL RESULTS

6.2.1. Test Procedure

6.2.2. Realism

6.2.3. Graphic Capabilities

6.2.4. Speech Generation

6.2.5. Timeliness

6.2.6. Descriptiveness

6.2.7. Compatibility

6.2.8. Modularity

6.2.9. Computational Simplicity and Efficiency

6.2.10. Input Requirements

7. CONCLUDING REMARKS

7.1. OBJECTIVES AND CONTRIBUTIONS

7.2. FUTURE RESEARCH

vBIBLIOGRAPHY

vi List of Tables Table 1-1. Example Applications for Virtual Software Agents

Table 2-1. Facial Action Coding System, Partial List of Action Units

Table 4-1. Visemes List

Table 4-2. Feature Lines and Points (FaceFeature Structure)

Table 4-3. Features Associated with Face Patches

Table 4-4. Features Comparison for FIX vs. Typical 3D and 2D Methods

Table 5-1. Basic Filter Interfaces

Table 5-2. ShowFace API Methods for Filters

Table 5-3. CDSGraph SF-API Class

Table 6-1. Evaluation Criteria

Table 6-2. Evaluation Criteria, Revisited

Table 6-3. Summarized Comparative Evaluation

Table 6-4. Realism in Different Animation Approaches

Table 6-5. Graphic Capabilities of Different Animation Approaches

vii List of Figures Figure 1-1. Some Applications of Virtual Software Agents

Figure 1-2. Personalized Face Animation System

Figure 1-3. Personalized Face Animation and Related Domains

Figure 2-1. Interoperability in XMT

Figure 2-2. Some Approaches to Object Modeling

Figure 2-3. 3D Head Models from Photographs

Figure 2-4. Mesh-warping for Face Animation

Figure 2-5. Feature-based Morphing

Figure 2-6. Creating Visual Speech in MikeTalk

Figure 2-7. Video Rewrite

Figure 2-8. Facial Image Transformations Based On Static 2D Prototypes

Figure 3-1. FML Timeline and Temporal Relation of Face Activities

Figure 3-2. FML Document Map

Figure 3-3. FML Model and Templates

Figure 3-4. FML Time Containers and Primitive Moves

Figure 3-5. FML Iteration

Figure 3-6. FML Decision Making and Event Handling

Figure 3-7. Static Iterative FML Document

Figure 3-8. Events and Decision Making in FML

Figure 3-9. JavaScript Code for FML Document Modification

Figure 4-1. Moving/Talking Head with Correlation-based Optical Flow

Figure 4-2. Sample Results of Optical Flow-based Approach

Figure 4-3. Feature-based View Generation

Figure 4-4. Feature-based Image Transformation

Figure 4-5. Facial Expressions

Figure 4-6. 3D Head Movements

Figure 4-7. Facial Features

Figure 4-8. Feature Detection.

viii Figure 4-9. Using Feature Translation Functions

Figure 4-10. Mapping Vector for Feature Islands

Figure 4-11. Newly Appeared Areas for Head Movement and Talking

Figure 4-12. Texture Transformation

Figure 5-1. ShowFace System

Figure 5-2. ShowFaceStudio

Figure 5-3. Sample DirectShow Filter Graph

Figure 5-4. Using DirectShow Filters

Figure 5-5. ShowFace Filter Graph

Figure 5-6. Smooth Connection of Diphones

Figure 6-1. ShowFace Test Procedure

Figure 6-2. Image Transformation Results

Figure 6-3. Comparing Transformed and Real Images

Figure 7-1. Some Applications of Face Animation

Figure 7-2. Face Animation System Requirements and ShowFace Contributions............... 105 ix List of Abbreviations 2D Two Dimensional 3D Three Dimensional API Application Programming Interface AU Action Unit AVI Audio-Video Interleaved BEAT Behaviour Expression Animation Toolkit CML Cognitive Modeling Language COM Component Object Model DOM Document Object Model ECA Embodied Conversational Agent EDL Edit Decision List EPG Electronic Program Guide FACS Facial Action Coding System FAP Face Animation Parameters FDP Face Definition Parameter FFT Fast Fourier Transform FIX Feature-based Image Transformations FML Face Modeling Language FTF Feature Translation Function MPEG Motion Picture Experts Group MPML Multimodal Presentation Markup Language OCI Object Content Information OF Optical Flow PCF Perspective Calibration Function SDK Software Development Kit SF-API ShowFace Application Programming Interface SMIL Synchronized Multimedia Integration Language SMPTE Society of Motion Picture and Television Engineers

–  –  –

xi Acknowledgements The work presented in this thesis is done under the supervision of Dr. Babak Hamidzadeh and Dr. Rabab Ward, at the Department of Electrical and Computer Engineering (ECE), University of British Columbia (UBC). Their effective guidance and insightful ideas, throughout the entire project, are greatly appreciated. This research has also benefited from invaluable helps by Dr. David Lowe of UBC Department of Computer Science (CS), to whom the author offers the most sincere gratitude. The author also wishes to thank the heads and professors at UBC departments of ECE and CS, and Institute for Computing, Information, and Cognitive Systems (ICICS), for the wonderful learning and research opportunity they have provided to the students.

Special thanks go to ECE, CS, and ICICS staff, in particular Doris Metcalf (ECE Graduate Program Clerk), Cathleen Holtvogt (ECE Administrative Clerk), and Kirsty Barclay (ICICS Technical Writing Advisor), for their administrative and technical support, and also other ECE and CS graduate students for the active, supportive, and friendly environment they have created.

Last but not least, the author would like to thank his dear friends, Mr. Kouros Goodarzi and his wife Parisa, for reviewing and proofreading the final manuscript, and their love, friendship, and patient support. The kindness of these and other friends and family members have made this research, and a lot more, possible. Thank you all !

–  –  –

1.1. Background and Motivations From the humble cavemen painting on the walls of their caves to the esteemed masters of the École des Beaux Arts in 19th century Paris, to the animators of recent motion pictures such as Antz and The Matrix, visual artists have long been creating views that represent ideas, events, and characters. Regardless of what they represent in their work, and how they choose to do so, these artists share one common role: visualization. In the absence of other means, their creative minds and hands were what they had to bring ideas into form. The invention of camera brought a new player into this game.

Cameras opened up a new approach to visualization, that of actually “recording” views instead of “creating” them 1, and gradually, they proved to be quite reliable and precise in doing it. Although the technology was very limited at the beginning, it was not hard to foresee its improvement over time, as it actually happened. The ability of the new devices to record and display scenes posed an important question to visual artists: if it is possible to record subjects, is it still significant to draw or paint them? Modern art has a definite, positive answer to this question. The key to this answer lies in realizing that visualization is not simply a mirroring of the “real” world. It is also about representing things that do not exist externally, or are hard to capture on camera, or representing existing subjects but in different ways (for instance, introducing the effect of the artist’s viewpoint and impressions).

In one sense, recording can be considered a special type of creating content, but here we use the term “creation” to mean generating “new” content, even if based on existing materials.

This realization led to a variety of styles that form what we now call modern art. In 1890,

Vincent Van Gogh wrote [101]:



Pages:   || 2 | 3 | 4 | 5 |   ...   | 13 |


Similar works:

«Teorema Vol. XVIII/1, 1999, pp. 19-41 Searle on Perception* Manuel García-Carpintero RESUMEN En el curso de su discusión de la percepción, Searle critica las teorías representacionales. En este artículo argumento que, si bien sus críticas pueden ser adecuadas contra una forma de tales teorías, quizás la más frecuentemente defendida por los filósofos de la percepción, una versión que será esbozada aquí escapa a ellas. Una segunda cuestión que presento concierne a si la teoría de...»

«UNVEILING THE MOLECULAR MECHANISMS REGULATING THE ACTIVATION OF THE ERBB FAMILY RECEPTORS AT ATOMIC RESOLUTION THROUGH MOLECULAR MODELING AND SIMULATIONS Andrew J. Shih A DISSERTATION in Bioengineering Presented to the Faculties of the University of Pennsylvania In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Advisor Dr. Ravi Radhakrishnan, PhD, Associate Professor of Bioengineering Graduate Group Chair Dr. Susan S. Margulies, PhD, Professor of Bioengineering...»

«ISLAM, DEMOCRACY, AND GOVERNANCE: SUDAN AND MOROCCO IN A COMPARATIVE PERSPECTIVE By WALEED MOUSA A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA Copyright 2005 by Waleed Mousa There is not a single night in which I do put my head over the pillow without thinking of the poor in my nation and in the world in large. It is to those impoverished people, to my parents...»

«The Use of Newspapers as a Source for Musicological Research: A Case Study of Dublin Musical Life 1840–44 Catherine Ferris Thesis submitted to National University of Ireland, Maynooth for the Degree of Doctor of Philosophy Department of Music National University of Ireland, Maynooth Maynooth Co. Kildare April 2011 Head of Department: Professor Fiona M. Palmer Supervisor: Professor Barra Boydell Contents Acknowledgements i Abbreviations ii List of Tables iii Part I Chapter 1: Introduction 1...»

«KUSCO-MURPHY INTERNATIONAL PRODUCTS Proudly Australian Made and Australian Owned, Naturally. KUSCO-MURPHY Australian Made Natural Haircare Products Company Profile KUSCO-MURPHY INTERNATIONAL PRODUCTS is a Melbourne based manufacturer of natural haircare products. At KUSCO-MURPHY we have married ancient philosophy with modern technology to bring a no fuss, antihype hair care range to the consumer. KUSCO-MURPHY provides that „wow‟ factor and a point of difference across our unique range of...»

«DENY THY FATHER, YET SEEK TO PLEASE HIM?: SUBVERSIVE SHAKESPEARE AND THE AUTHORITATIVE DESIRE OF SHAKESPEAREAN TEEN FILMS by NATALIE JONES LOPER SHARON O’DAIR, COMMITTEE CHAIR DAVID AINSWORTH BARBARA BRICKMAN JEREMY BUTLER HAROLD WEBER A DISSERTATION Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department of English in the Graduate School of The University of Alabama TUSCALOOSA, ALABAMA Copyright Natalie Jones Loper 2012 ALL RIGHTS...»

«Rethinking Humanism by Nadine Picone-Wightman, BA word count: 14,340 A thesis submitted in partial fulfilment of the requirements for the degree of Bachelor of Arts with Honours in Philosophy School of Philosophy University of Tasmania October, 2012 Declarations and Permissions I declare that all material in this thesis is my own work except where there is clear acknowledgement or reference to the work of others and I have complied with and agreed to the University statement on Plagiarism and...»

«¡Quiero estudiar! Mexican Immigrant Mothers’ Participation in Their Children’s Schooling—and Their Own by Alice Anne Miano A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Education in the Graduate Division of the University of California, Berkeley Committee in Charge: Professor Glynda Hull, Co-chair Professor Guadalupe Valdés, Co-chair Professor Sarah Warshauer Freedman Professor William Hanks Spring, 2010 
 
 
...»

«‘Guttersnipes’ and ‘Eliterates’: City College in the Popular Imagination Philip Kay Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy under the Executive Committee of the Graduate School of Arts and Sciences COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK © 2011 Philip Kay All rights reserved (This page intentionally left blank) ABSTRACT ‘Guttersnipes’ and ‘Eliterates’: City College in the Popular Imagination Philip Kay Young people go to...»

«Regulation of Drosophila intestinal regeneration by the Hippo pathway Rachael Louise Shaw University College London and Cancer Research UK London Research Institute PhD Supervisor: Nicolas Tapon A thesis submitted for the degree of Doctor of Philosophy University College London October 2012 Declaration I, Rachael Shaw, confirm that the work presented in this thesis is my own. Where information has been derived from other sources, I confirm that this has been indicated in the thesis. Abstract...»

«suhrkamp taschenbuch wissenschaft 1978 Absicht Bearbeitet von Joachim Schulte, G. E. M. Anscombe 1. Auflage 2010. Taschenbuch. 164 S. Paperback ISBN 978 3 518 29578 6 Format (B x L): 10,8 x 17,6 cm Gewicht: 110 g Weitere Fachgebiete Philosophie, Wissenschaftstheorie, Informationswissenschaft Philosophie: Allgemeines Westliche Philosophie: 20./21. Jahrhundert schnell und portofrei erhältlich bei Die Online-Fachbuchhandlung beck-shop.de ist spezialisiert auf Fachbücher, insbesondere Recht,...»

«CULTURE AND SUBCULTURE IN THE TURKISH POLICE FORCE Thesis submitted for the degree of Doctor of Philosophy at the University of Leicester by Muammer YILDIZ Scarman Centre University of Leicester JUNE 2001 UMI Number: U144983 All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material...»





 
<<  HOME   |    CONTACTS
2016 www.abstract.xlibx.info - Free e-library - Abstract, dissertation, book

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.