WorldMaker Universe:: Software Framework for the Development of Computational Artworks

The following is an excerpt taken from:
Hosale, Mark-David, "Worldmaking as a conceptual framework for computational art", Worldmaking as Techné: Participatory Art, Music, and Architecture, Edited by Mark-David Hosale, Sana Murrani, and Alberto de Campo, Riverside Architectural Press, Hardcover, 433pp, 2018 ISBN: 978-1-988366-09-8

 

From a practical point of view, the WMU as a software framework facilitates the creation of expressive and emergent behaviour in interactive installation environments by encapsulating commonly used elements of the software design of interactive environments into a ready to use set of abstractions. The implementation of the WorldMaker Universe as software is based on an abstract a modular system that can function as a distributed suite of applications. A large portion of the concepts and terminology that help define this framework are taken from design patterns, as used in computer science, such those found in Design Patterns: Elements of Reusable Object-Oriented Software[1].

 

 

Figure 1: The WMU framework designed for the development of computational artworks.

The design of the Universe framework is based on an augmentation of the commonly known Model-View-Controller (MVC) design pattern[2]. The intent of the MVC is to organize an application into three areas that focus on the application logic (Model), user input and display (View), and the control mapping (Controller), which is responsible for maintaining the communications between the model and the view.

The primary additions to MVC help define the WMU framework more specifically in terms of the technologies and protocols required for its implementation. The WMU framework is not a standalone API or software framework, but is built upon common media oriented frameworks and toolkits found in computational art, such as OpenFrameworks[3], OpenSoundControl[4], MySQL[5], etc., which contain the basic elements of this system, but require augmentation in order to satisfy the needs of the WMU. The following is not a comprehensive model, but an overview of key elements needed to implement the WMU[6].

Model

Operations as described in the conceptual model of the WMU framework are implemented in the Model component of the software framework (Fig. 1). The Model contains all of the processes that are used to determine the unfolding logic of the world based on the generative techniques of formalization, indeterminacy, and combinatoriality as described above. The Model also makes decisions about what events to play and when they should be played. The Model uses internal algorithms, data gathered from databases, and virtual representations of the view components to determine these actions. All of the information processing and representation of the installation components within the Model is intended to be abstract in order to keep the data model and the representation of the data as separate as possible. Again emphasizing variability and independence facilitates scalability and the reuse of the same model in various types of works.

Databasing

Query is a database class that provides an interface to querying the database engine being used by the Model, such as scenes, histories, sequences, or any other data used by the WMU. Databases are primarily responsible for the storage and retrieval of event sequences and data type definitions, which are used by the view classes for the execution of state and transition updates.

Events

Event is an event handler class that handles the sequencing of events. This can happen through a generative algorithm, a sequencer, or combination of the two. Event handling is realized as a Composite pattern that organizes events into hierarchical structures for clarity and modular transformation (Fig. 2). Playback is achieved through a recursive unraveling of a sequence’s branches and executing the events at the indicated relative time-point.

 

 

Figure 2: Diagram of the Composite Pattern Sequencer, which manages the unraveling of hierarchical events in time.

Embedded State Machine

The Embedded State Machine (Fig. 3) is responsible for keeping track of changing state, and determining transitions between events. The Embedded Transition State is governed by the operations of the WMU world. From the point of view of the conceptual model the Embedded State Machine is part of the responsibility of the transforms as it is used to interpret how state changes will be executed, acting as the glue for actualized events from the operations to the views. From a design pattern perspective, this is modelled as a combined state[7] and strategy[8] pattern, which facilitates the encapsulation of various logic systems, and various groupings of independent logic systems for the execution of states and transitions.

 

Figure 3: Diagram of the Embedded State Machine.

Controller

The controller handles mapping, scale, translation, and routing from the Views to the Model. From the point of view of the conceptual model the controller falls in the domain of transforms maintaining a tight connection between the virtual and actual aspects of a view to complete the circuit of the flow of information between these domains.

Views

View classes contain the logic of the various views of the personae of the WMU. Views consist of anything that an end user makes contact with within the physical and virtual space of the world. Views are defined as either input or output views. It is not unusual for an object or system to be composed of several input and output views. In this case, feedback is handled through the Model and Controller.

Views are designed to be as independent as possible of the software that implements the Model and Controller of the framework to facilitate the potential of Views being distributed among several applications and/or computers. To maintain this independence, views have virtual counterparts that are implemented in the Model and often have specialized data types associated with them in order to keep track of their type and state. This virtual model is also used to facilitate feedback between the views.

As a tool for analysis

In addition to being a tool for the creation of computational artworks, the WMU has the potential to be a method for analysis. There is a need in computational art for descriptors that extend beyond the experiential qualities of a work (personae) and look at the generative processes (operations) and structure of information (transforms) that inform the behaviour of a work. The separation of these characteristics in the WMU lend themselves well to comparing experientially disparate works that share common approaches in their operational and transformational application.

While the development of frameworks like the WMU for the creation of computational artworks is fairly common, few examples exist that apply similar tools in analysis. One of the closest examples can be found in A framework for understanding generative art[9] [10] (FGA)[11]. Like the WMU, the FGA separates processes and the transformation of data from its expression.[12] However, the WMU framework is broken down into three categories: operations, transforms, personae; whereas in the FGA the framework has four categories: entities, processes, environmental interaction, and sensory outcomes.[13] However, on closer reading it becomes clearer that, while there are differences in the language used in the WMU and FGA, the approach to analysis is quite similar between the two.

For a start, operations of the WMU and entities of FGA are similar in their description. Both terms represent the domain of unactualized generative algorithms independent of expression.[14] Knowledge objects (Ding an sich) in the WMU are entities in FGA, with a distinction made between those entities that are readily apparent and those that are not (noumena).

The nature of knowable and unknowable noumena is highlighted in the processes of the FGA. Here again we see parallels with the role of transformations in the WMU in the recognition of the role of processes in actualization: “…there is not necessarily a direct relationship between entity properties and the perceived outcomes of a generative artwork; these properties are often perceived only via a mapping.”[15]

The biggest difference that remains, at least on first glance, is the absence of a representation of personae within the FGA. Instead the FGA divides personae into two distinct domains: environmental interaction, and sensory outcomes.[16] Clearly these domains map readily to input and output views. However, the WMU takes these concepts further by attempting to abstract modes of interactions, and the concept of agent, agency, subject and observer as an experiential domain of percepts and affects.

The parallel nature of these two approaches emphasizes a common need for analysis tools that take in account the behavioural qualities of a work, as well as their conceptual underpinnings. Where these approaches begin to diverge is in the WMU’s metaphysical description of the framework as a system itself. This level of description is absent from the FGA. This is likely a symptom of the differing goals of the two approaches. The initial goal of the WMU is in the creation of work, and in that goal there is an attempt to encapsulate the conceptual motivations of making a work as an embodied ontology. And by further extension of the concept of worldmaking, to address how these works can become worlds, through the immersion of body and mind in the experience. By encapsulating these conceptual drivers with a practical framework the ultimate goal of the WMU is to infuse the making of work with the process of concept building (techné).

My goal in presenting the FGA is not to advocate one approach over the other, but to recognize the similarity of the work of the FGA and build upon it in order to show the potential of this kind of approach in the analysis of generative/computational art. Assuming the goal of the FGA (at least as it was presented in the aforementioned publication) is solely to be used as a tool for analysis, it is important for the FGA to remain more general in its descriptions and approach. So while the approach of the WMU is more expansive in its definition, it could prove to be more cumbersome for the general analysis of other works unless those works had an agenda of worldmaking at its core. The scope is narrow because the WMU framework is the product of a single artists’ practice, whereas the FGA is derived from the intersection of a number of researchers’ contributions. That being said, while the FGA has a more broad and general approach, using the WMU as a tool for analysis would be a valuable pursuit in finding qualities in works that align with the more specific approach of the WMU, if for any reason than to help improve the WMU itself.

The FGA has been given a large amount of attention because of the high-level of alignment between the WMU and the FGA. Besides the FGA there are a number of compelling texts that addresses the role of frameworks in making computational art. One example can be found in the anthology, Interacting: Art, Research and the Creative Practitioner[17] . In particular, the chapter by Linda Candy, Research and Creative Practice[18], which discusses art-making in terms of a research practice that is founded on conceptual frameworks at the core of their development and outcomes. Research-creation is another way of framing the goal of integrating making and thinking as a single action in the creation of work. Another work, What Is Generative Art?[19], co-authored by one of the editors of Interacting: Art, Research and the Creative Practitioner analysis the history of terminology, aesthetics, and varying practices within the field of generative/computational art.

Other work in this area that is notable includes approaches that use narrativity as a tool in understanding the form and structure of computational art works. Examples include, Brenda Laurel’s Computers as Theatre[20] , Janet Murray Hamlet on the holodeck: The future of narrative in cyberspace[21], Mark Meadows’ Pause & effect: the art of interactive narrative[22], and Mitchell Whitelaw’s, System stories and model worlds: A critical approach to generative art[23]. All of these works, with the exception of the last, had a direct influence on the development of the WMU in its early stages.

-----------------------

[1] Gamma, Erich, Richard Helm, Ralph Johnson, and John Vlissides. 1995. Design Patterns : Elements of Reusable Object-Oriented Software. Addison-Wesley Professional Computing Series. Reading, Mass.: Addison-Wesley.

[2] Ibid. pp. 4-6

[3] http://www.openframeworks. cc (accessed May 15th, 2016)

[4] http:// www.opensoundcontrol.cc (accessed May 15th, 2016)

[5] http://dev.mysql.com (accessed May 15th, 2016)

[6] For a more comprehensive model see: Hosale, M.D., 2008. pp. 83-111

[7] Gamma, et. al. 1995, pp. 305-314.

[8] Ibid., pp. 315-324.

[9] Dorin, Alan, Jonathan McCabe, Jon McCormack, Gordon Monro, and Mitchell Whitelaw. "A framework for understanding generative art." Digital Creativity 23, no. 3-4 (2012): 239-259.

[10] It should be noted that the FGA does not use the term computational art, but instead uses the term generative art. As used in this framework, the term computational art is quite similar, in spirit, to generative art. The real difference is the larger goal of the WMU, being worldmaking, whereas in the FGA the focus is on the generative methodologies. Despite this difference - both systems could be used for the other purpose and are quite comparable in their approach.

[11] My acronym.

[12] Dorin, et.al. 2012. p. 239.

[13] ibid. p. 244

[14] ibid pp. 245-246

[15] ibid. p. 244

[16] ibid. pp. 246-247.

[17] Candy, Linda and Ernest Edmonds. eds. Interacting: Art, Research and the Creative Practitioner. Libri Publishing, Oxford, U.K., 2011.

[18] Ibid. pp. 33- 57

[19] Boden, M.A. and Edmonds, E.A., 2009. What is generative art?. Digital Creativity, 20 (1-2), pp.21-46.

[20] Laurel, Brenda. Computers as theatre. Addison-Wesley, 1993.

[21] Murray, Janet Horowitz. Hamlet on the holodeck: The future of narrative in cyberspace . Simon and Schuster, 1997.

[22] Meadows, M.S., 2002. Pause & effect: the art of interactive narrative. Pearson Education.

[23] Whitelaw, Mitchell. "System stories and model worlds: A critical approach to generative art." Readme 100 (2005): 135-154.