Bibliographies for Exam

http://www.utdallas.edu/ah/programs/graduate/doctoralexams.html

——————————————————

Creating Interactive and Immersive Visual Music based on Dynamic Models

My research centers on the use of dynamic simulation models within the interactive 3D virtual environment. The goal of the research is to investigate a wide array of dynamic models for the purpose of creating a new hybrid form of art that combined visual art, music, and computer game. This work is positioned within the area of Aesthetic Computing, which fosters research in arts-based representations of computing structures (e.g., models, formalisms, program, and data structures). The accurate term to describe this hybrid form of art including but not limited to “interactive visual music”, “audio-visual art”, “visual music game art,” and “computational art.”

The examples of dynamic models and art include the following.

(1) The Lotka-Volterra (also known as the predator–prey) simulation model is a coupled set of two ordinary differential equations, which when translated into Forrester’s Systems Dynamics 2D graph-based methodology, can be represented within 3D virtual environment in combination with (musical) sounds.

(2) “Dining philosophers problem” is used in computer science to demonstrate the concept of concurrency and shared resources. It was implemented using computing models such as Finite State Machines and Petri Nets. The “dining philosophers problem” could be replaced by “jazz musicians improvisation” to demonstrate the concurrency and rules of harmony in jazz music. The result artwork is a visualization of live jazz music improvisation.

Besides the written dissertation, the primary outcome of the research is a collection of interactive artworks in a form of downloadable software/games.

The contributions to the knowledge and the novelty of the research/artworks include the following.

(1) The artwork is downloadable via internet, thus, it is no longer depend on the judgment of art critic/theorist/curator, art museum, and gallery.

(2) The new finding, adapting, and implementing of the abstract concepts from science (such as Lotka-Volterra simulation model) for creating the novel artworks is publishable on the peer-reviewed technical journals (or conferences) for art/music technology such as Computer Music Journal, International Computer Music Conference, and New Interface For Musical Expression.

(3) Creating interactive visual music with domain specific tools/languages such as Max/MSP/Jitter or VVVV is quite a straightforward task to start. But creating interactive visual music with game engine is a novel practice because there is no dedicated music synthesis functionality in any game engine. The creator needs to implement the digital musical messages (such as Open Sound Control) and send them to music synthesizers. To create an easy-to-use software for the users/players, the creator could ported the open-source C++ music synthesis toolkit to the game engine.

(4) The music in the typical computer games are usually composed and rendered as audio files, but in this research, they are computed and synthesized as audio signals in real-time. The music synthesis parameters are mapped to the variables in the simulation model, or the scene containing the dynamic model, or controlled by the interaction of the user/player.

(5) While the main research goal is to find the new techniques and methods for creating artwork, the assets (digital files, scripts, and codes) behind the artwork and the artwork itself could be used in the classroom to teach abstract concepts in computer science and dynamic modeling.

 

 

 —————————————————–

Paul Fishwick

Hybrid Modeling for Computer Music

.

Buys, Jan. 2011. “Generative Models of Music for Style Imitation and Composer Recognition.” Master Thesis, Computer Science Division, Department of Mathematical Sciences, University of Stellenbosch.

Castiglione, Filippo. 2006. “Agent Based Modeling.” Scholarpedia 1: 02/24/2016-1562. doi:doi:10.4249/scholarpedia.1562.

Conklin, Darrell and John G. Cleary. 1988. Modelling and Generating Music using Multiple Viewpoints
. University of Calgary: Knowledge Sciences Institute, Department of Computer Science.

Costalonga, Leandro L., Rosa M. Vicari, and Evandro M. Miletto. 2008. “Agent-Based Guitar Performance Simulation
.” Journal of the Brazilian Computer Society 14 (3).

Eigenfeldt, Arne and Ajay Kapur. 2008. “An Agent-Based System for Robotic Musical Performance.” Genova, Italy, Conference on New Interfaces for Musical Expression, .

Fishwick, Paul A. 2006. Aesthetic Computing, edited by Paul A. Fishwick. Cambridge, Mass.: MIT Press.

———. 1995. Simulation Model Design and Execution : Building Digital Worlds. Englewood Cliffs, N.J.: Prentice Hall.

Flynt, John P. and Ben Vinson. 2005. Simulation and Event Modeling for Game Developers. Boston, MA: Thomson Course Technology.

Forsyth, Jon and Juan P. Bello. 2013. “Generating Musical Accompaniment using Finite State Transducers.” Maynooth, Ireland, Proceedings of the International Conference on Digital Audio Effects (DAFx-13), September 2013.

Jurish, Bryan. 2004. “Music as a Formal Language.” Graz, Austria, First International pd~Convention, September, 2004.

Kapadia, Mubbasir, Nuria Pelechano, Jan Allbeck, and Norm Badler. 2015. Virtual Crowds: Steps Toward Behavioral Realism. 1st ed. San Rafael, California: Morgan & Claypool.

Loy, Gareth. 2006. “Chapter 9: Composition and Methodology.” In Musimathics: The Mathematical Foundations of Music, Volume 1, 285-406. Cambridge, Massachusetts: MIT Press.

Mark, Dave. 2009. Behavioral Mathematics for Game AI. Boston, MA: Course Technology PTR.

Marzinotto, Alejandro, Michele Colledanchise, Christian Smith, and Petter Ögren. 2014. “Towards a Unified Behavior Tree Framework for Robot Control.” IEEE International Conference on Robots and Automation, Hong Kong, DiVA Portal, 05/31/2014 – 06/07/2014.

McCormack, Jon. “Nodal, Generative Music Software, Research & Development.”, accessed April 16, 2015, www.csse.monash.edu.au/~cema/nodal/research.html.

Merz, Evan X. 2014. “Implications of Ad Hoc Artificial Intelligence in Music.” North Carolina State University, USA, AAAI: Tenth Artificial Intelligence and Interactive Digital Entertainment Conference, 10/03-10/07.

Miranda, Eduardo Reck, ed. 2011. A-Life for Music: Music and Computer Models of Living Systems. 1st ed. Middleton, Wisconsin, USA: A-R Editions.

———. 2001. Composing Music with Computers. Newton, MA, USA: Butterworth-Heinemann.

Namee, Brian M. 2012. “Agent Based Modeling in Computer Graphics and Games.” In Computational Complexity: Theory, Techniques, and Applications, edited by Robert A. Meyers. 1st ed., 604-621. New York: Springer.

Nierhaus, Gerhard. 2009. Algorithmic Composition: Paradigms of Automated Music Generation. New York: Springer.

Rabin, Steve, ed. 2015. Game AI Pro 2. 1st ed. Boca Raton, FL: CRC Press.

———. 2013. Game AI Pro: Collected Wisdom of Game AI Professionals, edited by Steve Rabin. 1st ed. Baca Raton, FL, USA: A K Peters/CRC Press.

Reynolds, Craig. 1987. “Flocks, Herds, and Schools: A Distributed Behavioral Model.”Computer Graphics, SIGGRAPH ’87 Conference Proceedings, .

Scheper, Kirk Y. W., Sjoerd Tijmons, Cornelis C. de Visser, and Croon, Guido C. H. E. de. 2016. “Behavior Trees for Evolutionary Robotics.” Artificial Life 22 (1): 23-48.

Unemi, Tatsuo and Daniel Bisig. 2004. “Playing Music by Conducting BOID Agents – a Style of Interaction in the Life with A-Life.”Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems, .

Wilensky, Uri and William Rand. 2015. An Introduction to Agent-Based Modeling: Modeling Natural, Social, and Engineered Complex Systems with NetLogo. 1st ed. Boston, MA: MIT Press.

 

—————————————————

Ryan McMahan

Virtual Reality and Interactive Sounds

.

Barbosa, Jeronimo, Joseph Malloch, Marcelo M. Wanderley, and Stéphane Huot. 2015. What does ‘Evaluation’ mean for the NIME community? Baton Rouge, LA, USA, Proceedings of the International Conference on New Interfaces for Musical Expression, May 31-June 3.

Berthaut, Florent, Myriam Desainte-Catherine, and Martin Hachet. 2010. An immersive environment for hierarchical live-looping. Sydney, Australia, Proceedings of the International Conference on New Interfaces for Musical Expression, .

Bharitkar, Sunil and Chris Kyriakakis. 2006. Immersive audio signal processing. Information technology: Transmission, processing and storage. 1st ed. New York, NY: Springer.

Bidlack, Rick, Dorata Blaszczak, and Gary Kendall. 1993. An implementation of a 3D binaural audio system within an integrated virtual reality environment. Tokyo, Japan., International Computer Music Conference, .

Bouënard, Alexandre. 2009. Synthesis of music performances: Virtual character animation as a controller of sound synthesis. Ph.D. in Computer Science, European University of Brittany.

Bouënard, Alexandre, Sylvie Gibet, and Marcelo M. Wanderley. 2009. Hybrid inverse motion control for virtual characters interacting with sound synthesis – application to percussion motion. Amsterdam, Netherlands, Proceedings of the International Conference on Computer Animation and Social Agents (CASA), June 2009.

Bowman, Doug A., Ernst Kruijff, Joseph J. LaViola Jr., and Ivan Poupyrev. 2005. 3D user interfaces : Theory and practice. Boston: Addison-Wesley.

Hamilton, Robert. Carillon: interactive musical performance environment built within the unreal engine. https://ccrma.stanford.edu/~rob/carillon/ (accessed 02/09/2016, 2015).

———. 2014. Perceptually coherent mapping schemata for virtual space and musical method. Ph.D. in Computer-based Music Theory and Acoustics Center for Computer Research in Music and Acoustics, Stanford University, Department of Music.

———. 2013. Sonifying game-space choreographies with UDKOSC. Graduate School of Culture Technology, KAIST, International Conference on New Interfaces for Musical Expression, .

Lages, Wallace, Mahdi Nabiyouni, Javier Tibau, and Doug A. Bowman. 2015. Interval player: Designing a virtual musical instrument using in-air gestures.3D User Interfaces (3DUI), 2015 IEEE Symposium on, 23-24 March.

Lentz, Tobias, Dirk Schroder, Michael Vorlander, and Ingo Assenmacher. 2007. Virtual reality system with integrated sound field simulation and reproduction. Journal on Advances in Signal Processing no. 1: 1-19.

Ma, Jian. 2014. Evaluation and comparison of head-mounted displays in immersive virtual environments. M.S., The University of Texas at Dallas, Graduate Program in Computer Science.

Mäki-Patola, Teemu, Juha Laitinen, Aki Kanerva, and Tapio Takala. 2005. Experiments with virtual reality instruments. Vancouver, BC, Canada, Proceedings of the International Conference on New Interfaces for Musical Expression, .

Mihelj, Matjaž, Domen Novak, and Samo Beguš. 2013. Virtual reality technology and applications. 1st ed.Springer.

Moore, Alec G., Michael J. Howell, Addison W. Stiles, Nicolas S. Herrera, and Ryan P. McMahan. 2015. Wedge: A musical interface for building and playing composition-appropriate immersive environments. Arles, France, IEEE Symposium on 3D User Interfaces, 3/23/15 – 3/24/15.

Polfreman, Richard. 2009. Frameworks 3D : Composition in the third dimension. Pittsburgh, PA, Proceedings of the International Conference on New Interfaces for Musical Expression, .

Valbom, Leonel and Adérito Marcos. 2007. An immersive musical instrument prototype. IEEE Computer Graphics and Applications 27, no. 4: 14-19.

Yang, Jay N. Bjork’s first 360 degree virtual reality video. https://virtualrealityreporter.com/bjork-360-vr-video-stonemilker/ (accessed 02/03/2016, 2016).

 

———————————————————-

Xiaohu Guo

Computer Graphics, Animation, and Game

.

Akenine-Moller, Tomas, Eric Haines, and Naty Hoffman. 2008. “Real-Time Rendering.” In . 3rd ed., 1045A K Peters/CRC Press.

Aristidou, Andreas and Joan Lasenby. 2011. “Inverse Kinematics Solutions Using Conformal Geometric Algebra.” In Guide to Geometric Algebra in Practice, edited by L. Dorst and J. Lasenby. London: Springer.

———. . 2009. Inverse Kinematics: a review of existing techniques and introduction of a new fast iterative solver: A technical report (CUEDF-INFENG, TR-632) submitted to the Department of Information Engineering at University of Cambridge.

Bourg, David M. and Bryan Bywalec. 2013. Physics for Game Developers, 2nd ed. ed. Sebastopol, CA: O’Reilly Media.

Budge, Bill. 2014. “Reducing Cumulative Errors in Skeletal Animations.” In Game Programming Gems 7, edited by Scott Jacobs. 1st ed. Boston, MA: Course Technology.

Buss, Samuel R. 2009. Introduction to Inverse Kinematics with Jacobian Transpose, Pseudoinverse and Damped Least Squares Methods. University of California, San Diego, USA: Survey article (unpublished).

Conger, David. 2004. Physics Modeling for Game Programmers. Boston, MA: Course Technology / Cengage Learning.

Eberly, David H. 2010. Game Physics, 2nd ed. Burlington: Taylor & Francis.

Emperore, Katax. 2015. Unreal Engine Physics EssentialsPackt Publishing.

Fratacangeli, Marco. 2011. “GPGPU Cloth Simulation Using GLSL, OpenCL, and CUDA.” In Game Engine Gem 2, edited by Eric Lengyel. 1st ed., 365-378. Boca Raton, FL: CRC Press.

Hayes, Jeremy. 2014. “Advanced Particle Deposition.” In Game Programming Gems 7, edited by Scott Jacobs. 1st ed. Boston, MA: Course Technology.

Kallmann, Marcelo. 2008. “Analytical Inverse Kinematics with Body Posture Control “ Computer Animation and Virtual Worlds (CAVW) 19, no. 2: 79-91.

Kane, Frank. 2011. “Modeling, Lighting, and Rendering Techniques for Volumetric Clouds.” In Game Engine Gem 2, edited by Eric Lengyel. 1st ed., 21-43. Boca Raton, FL: CRC Press.

———. . 2011. “Simulation of Night-Vision and Infrared Sensors.” In Game Engine Gem 2, edited by Eric Lengyel. 1st ed., 45-54. Boca Raton, FL: CRC Press.

Kim, Chang-Hun, et al. 2015. “Real-Time Visual Effects for Game Programming.” In Gaming Media and Social Effects. 1st ed., 227. Singapore: Springer Singapore.

McCord, Ryan A. 2012. “Techniques and Aesthetics of Human Inverse Kinematics for Fighting Simulations.” Master of Interactive Technology, Southern Methodist University, The Guildhall.

Parent, Rick. 2012. Computer Animation, 3rd ed. Waltham, MA, USA: Morgan Kaufmann.

PV, Satheesh. 2016. Unreal Engine 4 Game Development Essentials, 1st ed. Birmingham, UK: Packt Publishing.

Reynolds, Craig. 1987. “Flocks, Herds, and Schools: A Distributed Behavioral Model.”. Computer Graphics, SIGGRAPH ’87 Conference Proceedings.

Shreiner, Dave, et al. 2013. OpenGL Programming Guide: The Official Guide to Learning OpenGL, Version 4.3, 8th ed. Ann Arbor, Michigan: Addison-Wesley Professional.