ArTbitrating JaVOX:

an evolutionary environment for visual and sound composition

Artemis Moroni, Dr.

Renato Archer Research Center, DRVC/CenPRA, Brazil.

email: Artemis.Moroni@cenpra.gov.br

Jônatas Manzolli, Dr.

Interdisciplinary Nucleus of Sound Studies, NICS/Unicamp, Brazil

Fernando Von Zuben, Dr.

School of Electrical and Computer Engineering, FEEC/Unicamp, Brazil

Abstract

ArTbitrariness is presented as an initiative of upgrading the esthetical judgment through evolutionary computation techniques and others population based techniques. ArTbitrariness arose from the attempt to emulate computational creativity applied to artistic production in the visual and sound domains. ArTbitrating JaVOX appeared from the merging two other environments, VOX POPULI and Art Lab, which are all described. Questions concerning to the modeling of a poetics for artistic production in an automatic environment are placed and important aspects to model creativity are presented.

1. Introduction

How can computers have anything to do with creativity? The first person to denounce this apparent absurdity was Ada, Lady Lovelace, the friend and collaborator of Charles Babbage. She realized that Babbage´s “Analytical Engine” – in essence, a design for a digital computer – could in principle “compose elaborate and scientific pieces of music of any degree of complexity or extent.” But she insisted that the creativity involved in any elaborate pieces of music emanating from the Analytical Engine would have to be credited not to the engine, but to the engineer. As she put it, “The Analytical Engine has no pretensions whatever to originate anything. It can do [only] whatever we know how to order it to perform.” [1]

Since then several authors [2, 3, 4, 5] tried in some way to bring creativity into computers systems, applying different techniques like production systems, shape grammars and, more recently, genetic algorithms. Nowadays, a new generation of computational researchers is discovering that by using simulated evolution techniques to create new composition systems, it is relatively easy to obtain novelty, often complex novelty, but it is correspondingly difficult to rein in the direction that novelty takes. The results of this still-young approach are frequently more frightening than pleasing. This is a consequence of the structure/novelty tradeoff. The challenge faced by the designers is how to bring more structure and knowledge into the compositional loop, while trying to take people out [6]. But is it really desirable to take people out?

Following, the concept of ArTbitrariness [7] will be presented. In section 3, arTbitrating JaVox, an evolutionary environment applied to visual and sound composition is described. In section 4. questions addressing poetics in a computational creative environment are placed. Next, in section 5, similarities between the creative and the evolutionary processes are commented. Section 6 approachs important aspects for internalizing creativity in a system. Finally, the conclusios are presented.

2. What is arTbitrariness?

ArTbitrariness refers to the initiative of upgrading the esthetical judgment through evolutionary computation techniques and others population based techniques for exploratory search, and is interpreted as an iterative interactive optimization process [7]. The main goal of arTbitrariness is to avoid to leave to the artist what can (already) be optimized and to avoid to leave to the machine what can’t be optimized (yet). We can say that arTbitrariness addresses an arbitrary point among subjectivity and objectivity, with its associated automation capability, as presented in figure 2.

total automation

Figure 2. arTbitrariness as an arbitrary point between subjectivity and objectivity

If aesthetical appreciation would governed only by subjective opinion, it would not be possible to obtain (partially) automatic shapes of artistic production, with some aesthetical value, without a complete integration of the artist with the machine. On the other side, if the general rules did not allow the maintenance of a set of liberty degrees of expression, therefore automation could be complete, despite of the possible design complexity.

Since none of the extremes appropriately describes the artistic production process, we can conclude that there is space to automate the exploration of the liberty degrees of expression, this one through a man-machine interaction, such as in the attendance of general rules. In few words, the liberty degrees can be modeled such as optimizing problems of combinatory mathematics and the general rules can be mathematically formalized and inserted in computational systems, as constraints or directions to be followed by the machine. The freedom of expression will be so understood as an exploratory search for the best combination of the free attributes among all possibilities. This scene is characterized by the existence of a huge number of possible solutions, or possible combination of the free attributes.

After the proposition of a search space that contains possible solutions, a search tool is applied to look for promissory regions in the space, in which there are possible good solutions or combinations of free attributes with more aesthetical value than others from less promissory regions. There are very strong search algorithms, but among the factors that justify the choice of evolutionary computation techniques is the fact these algorithms apply population search techniques. But, independent of this, the search algorithms require the definition of an individual evaluation for each solution. The automation of this evaluation process would require of the machine to be able to deterministically evaluate the aesthetical quality of each individual in the current cycle, or generation. Instead of delegating this task to the machine, or to give the machine the evaluation ability, what is done is to require to an interaction with the artist, in such a way that the automatic solutions are presented to the artist and that he or she evaluates the solutions according to his or her subjectivity. In this context, the search for freedom of expression is directly linked to the exploratory power of the machine and the efficiency of the human/machine interaction process.

The concept of ArTbitrariness arose from the attempt of computationally emulate creativity applied to artistic production in the visual and sound domains. Two composition systems were developed, Vox Populi, in sound domain, and Art Lab, in visual domain. Interesting results appeared from both. Emergent questions were: what criteria to automate when looking for creative composition? What does assure the quality of the composition? How to recognize an interesting result, or how to supply the system with an automatic judgment capability?

3. arTbitrating JaVox

ArTbitrating JaVOX appeared from the merging two other environments, VOX POPULI and Art Lab. VOX POPULI, an evolutionary environment for sound composition, uses the computer and the mouse as real-time music controllers, acting as an interactive computer-based musical instrument. It explores evolutionary computation in the context of algorithmic composition and provides a graphical interface that allows changing the evolution of the music by using the mouse [8]. In VOX POPULI, an interactive pad supplies a graphical area in which bi-dimensional curves can be drawn. The pad control allows the composer to conduct the music through drawings, suggesting metaphorical “conductor gestures” when conducting an orchestra. By different drawings, the composer can experience the generated music and conduct it, trying different trajectories or sound orbits. The trajectory affects the musical fitness evaluation and the reproduction cycle of the genetic algorithm that is being applied to sound generation.

By its time, Art Lab is an experimental visual composition environment that allows the creation of frames of geometric figures and evolve them. This repeated interaction between artist and computer is implemented as an interactive and iterative population-based search, allowing the artist to search hyperspaces of possible compositions, sometimes very complex ones, by means of genetic algorithms. After each iteration, Art Lab´s interface permits the user/artist to evaluate the frames (by attributing a grade to each of them) or to promote mutations on selected frames for the next evolutionary cycle.

Conceptually, Art Lab has proven to be creative. According to Margaret Boden [1], there are three main types of creativity: combinatory, exploratory and transformational. The first mode, combinatory, involves new or improbable combinations of ideas. The second mode, exploratory, involves the generation of new ideas by the exploration of structured conceptual spaces. The second and the third mode are strongly linked, the distinction of one from the other is a question of interpretation. The third mode, transformational, involved the transformation of one or more dimensions in the space, in such a way that new structures that could not have occurred before can be generated.

Figure 3. The “solid arcs” of the frames in the bottom arose from the merging of the shapes of the frames in the top.

In its first version, Art Lab could generate compositions of shapes like arcs or “solid boxes”, but it did not have “solid arcs” among its primitive shapes. The compositions presented in the bottom of figure 3, with solid arcs, appeared after some evolutionary cycles, and are “descendants” of the two frames in the top. New “unexpected forms”, in the sense that they did not exist in that domain emerged. It is important to emphasize that creativity is being referred here in a very strict sense, or what Margaret Boden calls the “psychological sense” of creativity (P-creativity) and Gardner call “small creativity”, in opposition to the other historical sense of creativity (H-creativity) or “big creativity”. In accordance with Margaret Boden [1], a valuable idea is P-creative if the person in whose mind it arises could not have it before; it does not matter how many times other people have already had the same idea. By contrast, a valuable idea is H-creative if it is P-creative and no one else has ever had it before.

Figure 4. From left to right, the visual and sound compositions: 400 red and blue sticks; Instability; 510 arcs.

In ArTbitrating JaVox, the features of both environments, Vox Populi and ArtLab are being merged allowing the automatic production of visual and sound compositions, which is a machine possibility not so easy for human beings. The problem now is how to associate the visual features with sound features. In figure 4, three visual and sound compositions built with the simple graphical primitives line and arc are shown. Some possibilities are immediate. For example, we can easily think in associating sound trajectories to a composition like Jackson Pollock´s Number 33 action painting depicted in Figure 4, but how to associate sound attributes to the different colors and width of the lines? Any possible choice is arbitrary. This attempt refers to the poetic question in a computational environment, following next.

Figura 4 – Jackson Pollock (1949) Number 33

4. Computational Poetics

The initial question in this paper was: can the computers be creative? And, if they can be creative and moreover, artistically creative, what it would be the computational poetics? According to Pareyson [9], poetics is art program and poetics, explicit or implicit, is indispensable to the artistic activity, since that the artist can pass without a concept of art but not without an ideal of art.

Another problem appears, how to model poetics? Perhaps Pareyson himself can suggest an answer to this question: the artist is the first critic of himself and therefore he exercises, while creating, the critical thought. Now, the problem is the critical thought, how to model it? Here, there is already some development. Baluja, Pomerleau and Jochem [10] have trained a neural network to replace the human critic in an interactive image evolution system [5]. The network `watches´ the choices that a human user makes when selecting two-dimensional images from one generation to reproduce in the next generation, and over time learns to make the same kind of aesthetic evaluations as those made by a human user. When the trained network is put in place of the human critic in the evolutionary loop, interesting images can be evolved automatically. Gibson and Byrne [11] suggested a similar approach for very short music fragments. With learning critics of this sort, whether applied to images or music, even less structure will end up in artificial creators, because it must get there indirectly via the trained fitness-evaluating critic that learned its structural preferences from a user-selected training set. Perhaps there is a way to model poetics. Following, common aspects in the evolutionary and the creative processes are presented.

5. Creativity and cultural evolution

According to Csikszentmihalyi, creativity is the cultural equivalent of the process of genetic changes that result in biological evolution, where random variations take place in the chemistry of our chromosomes, below the threshold of consciousness. These changes result in the sudden appearance of a new physical characteristic in a child, and if the trait is an improvement over what existed before, it will have a greater chance to be transmitted to the child´s descendants. Most new traits may disappear after a few generations, but a few do improve survival chances, and it is these that account for biological evolution.

In cultural evolution there are no mechanisms equivalent to genes and chromosomes. Therefore, a new idea or invention is not automatically passed on to the next generation. Instructions for how to use fire, or the wheel, or atomic energy are not built into the nervous systems of the children born after such discoveries. Each child has to learn again from the start. The analogy to genes in the evolution of culture are memes [12], or units of information that we must learn if culture is to continue.

But in artificial evolution it is possible to simulate the process of competition and selection and let candidate “ideas”, or solutions, fight for room in future generations. Moreover, random generation can be used to search for new solution in a manner similar to natural evolution. An evaluation function can be used to determine the relative merit, or value, of each initial solution. When applying an evolutionary algorithm more than one parent solution can be used to generate a new candidate solution. One way to do this is by taking parts of two parents and putting them together to form an offspring. For example, the first part of one parent might be viewed as idea, and similarly so with the second half of the second parent. Sometimes this recombination can be very useful, but sometimes things do not work out so well. The appropriateness of every solution depends on the problem at hand, in this case to attend to an aesthetic appeal. The hardest part is to model the evaluation function: when is it art? Probably, because of this so many authors still use the human judgement to evaluate the authors automatically generated by those systems [13].

Next, important aspects concerning creativity modelling are presented.

6. Aspects to internalize creativity

According to Csikszentmihalyi [14], a person, to be creative, has to internalize the entire system that makes creativity possible. In other words, the person has to learn the rules and the domain, as well as the selection criterion. This is what we want to do in a machine. There are three important aspects to consider:

1) a huge data base, or the type of necessary memory;

2) to be able of catalyzing ideas;

3) to get rid of the garbage.

Todd and Latham [4], in Form Grow system, built two data bases, a form data base and a gene data base, that can be seen as phenotype and genotype databases. An evolutionary system with this kind of resource is able to deal with aspect 1. The other two aspects are treated by the evolutionary system dynamics. The successive genetic cycles promote the catalyzing ideas, using the best solutions found in the previous generations. This takes care of aspects 2 and 3.

Until the moment, JaVox does not have a data base to store the best solutions. In other words, it does not have memory. The next step will be to add a data base in order to store the best solutions. Perhaps this will be the key point for the treatment of the computational poetics.

7. Conclusion

The concept of ArTbitrariness as an iterative interactive optimization process for upgrading the esthetical judgment through evolutionary computation techniques and others population based techniques for exploratory search is presented. The environment ArTbitrating JaVox, an evolutionary environment for visual and sound composition emerged from two other evolutionary environments, VOX POPULI, an interactive environment for computational composition, and Art Lab, applied to visual domain. The features of both environments, Vox Populi and Art Lab are being merged in ArTbitrating JaVox, in order to enable the system to the production of visual and sound compositions. Until the moment JaVox does not have memory, it is only an exploratory tool in visual and sound domains, but interesting results have appeared. ArTbitrating JaVox is available at http://www.geocities.com/Artemis_Moroni/JaVox.

Acknowledgements

We would like to thank to Daniel Gurian Domingues for his strong support in JaVox development. In this sense, we would like to thank also Guilherme Ferreira dos Santos and Leonardo Laface de Almeida. Daniel Gurian Domingues, Guilherme Ferreira dos Santos and Leonardo Laface de Almeida are supported by CNPq PIBIC program. Fernando Von Zuben is supported by CNPq grant 300910/96-7. Jônatas Manzolli is supported by CNPq program of Productivity in Research. Artemis Moroni is supported by CenPRA.

References

[1] Boden, M. Dimensions of Creativity. New York: The MIT Press, 1996.

[2]Cohen,H. Colouring without Seeing: a Problem in Machine Creativity, http://crca.ucsd.edu/~hcohen, 1999.

[3] Gero, J. S. Conceptual Designing as a sequence of situated acts, in Smith, I. (ed.) Artificial Intelligence in Structural Engineering, Springer, Berlin, pp. 165 –177, 1998.

[4] Todd, S. & Latham, W. “The Mutation and Growth of Art by Computers” in Bentley, P. (ed.), Evolutionary Design by Computers, San Francisco: Morgan Kaufmann, 1999.

[5] Sims, K. “Evolving Three-Dimensional Morphology and Behaviour” em Bentley, P. (ed.), Evolutionary Design by Computers, San Francisco: Morgan Kaufmann, 1999.

[6] Todd, P. M. & Werner, G. M. “Frankensteinian Methods for Evolutionary Music Composition” em Griffith, N. & Todd, P. M. (eds) Musical Networks: Parallel Distributed Perception and Performance, Cambridge: The MIT Press, 1999.

[7] Moroni, A., Von Zuben, F.J. & Manzolli, J., “ArTbitration: Human-Machine Interaction in Artistic Domains”, Leonardo, 35(2), pp. 185 – 188, 2002b.

[8] Moroni, A., Manzolli, J., Von Zuben, F.J. & Gudwin, R. "Vox Populi: An Interactive Evolutionary System for Algorithmic Music Composition", Leonardo Music Journal, 10, pp. 49-54, 2000.

[9] Pareyson, L. Os Problemas da Estética. São Paulo: Martins Fontes, 1989.

[10] Baluja, S., Pomerleau, D. & Jochem, T. “Towards automated artificial evolution for computer generated images”, Connection Science, 6(2 – 3), 325 – 354, 1994.

[11] Gibson, P. M. & Byrne, J. A. “NEUROGEN, musical composition using genetic algorithms and cooperating neural networks”, London: Proceedings of the IEE Second International Conference on Artificial Neural Networks, pp. 309 – 313, 1991.

[12] Dawkins, R. The Selfish Gene. Great Britain: Oxford University Press, 1989.

[13] Michalewicz, Z. & Fogel, D. B. How to Solve It: Modern Heuristics. New York: Springer, 1998.

[14] Csikszentmihalyi, M. Creativity: Flow and the Psychology of Discovery and Invention. New York: HarperPerennial, 1996.