MSC: a generative art system integrating music and video through magic squares at compositional models

Paul A. Oehlers

Department of Computer Science, Audio Technology, and Physics

American University, Washington, DC, USA

oehlers@american.edu

Chris Mich

Production Department

QVC, West Chester, PA, USA

chris_mich@comcast.net

Abstract

MSC is a generative art composition system in which routes through a magic square are selected. A magic square consists of a series of numbers arranged so that the sum of each row, column, and diagonal are the same amount. This enabled the composer and filmmaker to independently select routes, choosing identical routes in some sections and differing ones at other points. These routes are then assembled to form the structure of the piece. The first resulting collaborative film was MFL with both live and electro-acoustic music that employs formalization to determine structure.

1. Background

Completed in 1999, Magic Square Composition (MSC) was conceived as a computer assisted system that would enable independent expression by composer and filmmaker, generating different realizations of artworks using the same mathematic constraints. Our primary goal was to develop a system that provided the same structure for all visual and auditory components without dictating content. MSC was designed in the tradition of exquisite corpses in visual art, as no discussion regarding the selection of material is necessary once the structure is determined. The composer only sees the completed film once it is synchronized with the music. The filmmaker only hears the final version of the music once he had completed the film.

Background of Magic Squares

In order to examine the construction of MSC, it is necessary to understand the basic principles of magic squares. A magic square is a square matrix of n rows and columns; the first of n² integers are arranged so that the sum of each row, column,, and diagonal equal the same amount. A five by five or order five square, for example, would contain the numbers 1 through 25, each occurring only once. Adding the numbers of each row, column, or diagonal would generate the same total, 75. An order six square, in comparison, would contain the numbers 1 through 36, and the sum of each of the rows, columns, and diagonals would each equal 111. Magic squares can be classified into three types: odd, doubly even (n divisible by four), and even (n even, but not divisible by four). [1]

Before constructing our system, we examined other methods employed by other artists and composers and manifestations of their respective magic square artworks. The compositional system of Sir Peter Maxwell Davies is typified in Ave Maris Stella, written in 1975. Davies uses the magic square of the Moon, one of the
“Ptolemic Magic Squares” in De Occulta Philosophia, a book on magic by Heinrich Cornelius Agrippa von Nettesheim, written in 1531. [2] As this is an order nine square, Davies reduced the square by modulo 9 to form a Latin square, a square of matrix n rows and columns, whose cells contain n different symbols so that no symbol appears more than once in any row or column. He then correlated these numbers to the pitches of a plainsong with the same title and used the positions in the square to control durations. Although the piece contains nine movements, the use of the square is only to control these local aspects of the piece. [3]

In the research presented on the compact disc recording Banjaxed, Zack Browning developed a system that employed routes through the magic square, but the individual positions within the square do not correspond to a specific theme. The number seventeen may appear twice with a different set of themes, densities, and orchestration parameters. The system remains constant over a number of compositions written beginning in 1986. [4] The documented history made this body of work ideal for study. Browning’s earlier experiments in the use of magic squares are also worth noting, particularly due to his departure from his style associated with his earlier experiments with magic squares. Browning attempted to integrate magic squares into every aspect of his music, but foundthe more appealing aspects of using the square was to create a means by which the change in the individual positions was apparent. [5] Other artworks by Paul Klee and others were studied, but a discussion of these is beyond the limitations of this paper.

Background of the second realization of MSC: MFL

The first realization of a composition using MSC was for experimental purposes. It resulted in a composition for orchestra, which revealed several imperfections in the preliminary composition system. These were corrected before the second run was started. MFL for piano, film, and electronics was begun as the result of a commission by composer/’pianist Mei-Fang Lin. The title comes from her initials.

1.3 Selecting the magic square for MFL

The particular magic square employed in the construction of MFL is the magic square of the sun, one of the “Ptolemic Magic Squares” in De Occulta Philosophia. The magic square of the Sun in an order six, singly even square, as when divided through the center intersection of the square forms quarter squares that are of an odd order or root. While most magic squares are associative, (containing skewly related numbers opposite one another), this particular square is the only one of the Ptolemic squares that is not. Figure 1 shows this particular magic square.

Figure 1. The magic square of the sun

We selected this magic square due to its use in other systems. The symbolism of using this square was indeed one factor, however, by analyzing the results of earlier systems, particularly Browning’s, the results of these other systems made the selection of this particular square apparent.

5.1 Construction

As it was critical that the timing for the music and the film remain consistent, every global element of MSC is determined by the magic square. Routes through the magic square are mapped onto both the musical and visual structure. The unique location of each number within the square and its position within the route is paralleled in the musical score by a particular style, density, amplitude, timbre, and orchestration. The individual numbers in the square are used to represent the number of beats of that section.

2.1 Selection visual imagery in the film component of MFL

In the film, individual positions determine the scene or scenes and visual effects or filter. Although the specific process of correlating the scene or combination of scenes corresponds to the individual position of the square is different, the principle in the same. The unifying element of all of the scenes is their inclusion of a type of human propulsion. In total there are twenty-four scenes, six scenes filmed at each of four different locations.

Figure 2. Locations in the film MFL

Los Angeles Area Hibernia State Park, Pennsylvania

Burbank Airport Running in an outdoor ampitheatre

Running next to the Amtrak train Tree

Bicycle sequence Running in the Park

Walking in the apartment Handball Player (front)

Tough guy walking up stairs Handball Player (back)

Construction Handball Player (feet)

West Chester, Pennsylvania Exton, Pennsylvania

Woman by the blocks Septa Train sequence

Woman smoking Truck on the train tracks

Airport/woman extreme close-up Traffic

Woman with the flashlight Truck turning

Feet/Chris Mich close-up sequence Underneath the Train Station

Film reel sequence Looking down at the tracks

2.2 Constructing Musical Themes

The six themes to MFL were designed to contain micro-and macro-cellular structre. This was accomplished through weighted probabilities, primarily Markov chains, for each series of possibilities. The themes were also placed in specific frequency ranges that isolated them from one another to enable the pianist to simultaneously perform several of them. The frequency ranges of each theme remain a static parameter throughout the piece.

As MFL employs an order six magic square, six distinct musical themes were created. Each of the themes is of differing lengths, in order to assure asynchronous repetition against the others. The basic unit of measurement dictating tempo is beats per minute with quarter note equaling 144. Figure 3 described the themes in regards to these parameters.

Figure 3. Basic structure of the themes in MFL

Music theme in MFL	Duration (in beats)	Range (in pitch names)	Micro-cellular structure	Macro-cellular structure
A	28	F sharp 2 – G3, C5 – D flat 6	Aaabbbb aabbb aabb aaabbb abbbbb	AABBA
B	39	A flat 4 – F 6	abcbcd cbabcd bcdb’c’ba bcb’c’d’ bcbc’d’ cdbcd	AA’A”BB’A”’
C	25	E flat 2 – A flat 3	aaa’bb’ a”a”’b”b”’ cabb’b’bc aa’b’b a”a”’b”’b” c’cdadcbb’d	ABA’B’
D	24	G5 – D7	aaa’bb’ a”a”b”b” cadbb’b’bc aa’b’b a”a”’b”’b’ c’cdadcbb’d	ABA’B’
E	22	E1 – E2	abcd aa’ab’a” cdc cde abcde	ABCC’A’
F	30	D2 – E flat 6	abcaa babd abd’b abd’b gaefg’a”b g’aha” gab’ aij	ABB’CC’D

The micro-cellular cells do not remain consistent in respective themes. The cell “a” in theme A is different from “a” in theme B. The cells are roughly all the same length from an eighth note in duration to a dotted quarter note.

2.3 Constructing scenes in the film

Routes through the square were determined subjectively. The equal value of each of the routes enabled us to select routes independently and still have point where elements could be unified. After the musical themes were affixed to individual positions, eleven different routes were selected, giving MFL a duration of just under nine minutes. Although we were both aware of these routes, we kept information regarding the correlation of a specific theme or themes to the individual positions within the square separate

2.4 Determining Density of Musical Materials

After the musical themes were completed, the density of each location within the square was reduced to modulus 3. This generates the following results:

Figure 4. The magic square of the sun expressed in modulus 3

The numbers with a remainder of 0 were then represented as 3. These numbers are analogous to the quantity of thematic ideas present in each square location. As there were not a proportional number of thematic combinations as there were for numbers with certain remainders, a Markov chain was employed to determine which theme would fit in the blank spaces.

Figure 5. The corresponding themes to the individual positions in

the magic square of the sun

2.5 Determining musical parameters

The quantity of thematic ideas in an individual square location as well as the position of the number in the route determined the instrumentation and orchestration of a particular thematic idea. If the first position in the route contained one thematic idea, for example, the instrumentation for that position would be piano and electronics. If there were two themes present, however, the position would be electronic playback alone. As seen in the grid in figure 6, the choices of instrumentation all follow this predetermined form.

Figure 6. Instrumentation of the routes in MFL

Position in route	1	2	3	4	5	6
One theme	Piano and electronics	Electronics	Piano	Electronics	Piano and electronics	Piano or Piano and electronics
Two themes	Piano and electronics	Piano or Piano and electronics	Electronics or Piano and electronics	Piano	Piano	Electronics
Three themes	Piano and electronics	Piano	Piano and Electronics	Electronics	Piano and electronics	Piano

Initial attempts were made to realize a version of the piece with an equal amount of the each of the three possible options, but it immediately became apparent that this was not a viable option. First, the pianist would not perform during extended periods. Second, during the portions with three themes, the pianist would be forced to perform three themes simultaneously. Third implementing the system would not be consistent with the balanced versus imbalanced paradigm already set in the themes and density within the square.

In addition to the choices of instrumentation determined by the square, the location of the number within the square also determined the type of compositional sieve employed, the type of digital signal processing (reverberation, spectral extraction, spectral dynamics, etc.), and dynamic shape of the individual location. The resulting modifiers ensured that while musical ideas were repeated, the combination of these elements was unique to each occurrence of individual locations within the magic square. As can be seen in figure 7, the listing of the different musical elements of the first route through the square reveals the level of formalization of these components in the structure of the piece.

Figure 7. Sieves and digital signal processing techniques in the first route of MFL

Route 1	31	12	13	24	30	1
Sieve	A1	B3 C3 F3	E1	A2 C1 D3	B1 D1 E1	F1
DSP	None	No electronics present	No electronics present	Spectral dynamics	Spectral extraction	No electronics present

2.6 Organizing data

After the data was compiled for each of position in the square and order of routes, a linear version of the piece was generated. In the music this realization was constrained by a series of predetermined rules:

1. When a theme is presented alone, it begins on beat 1

2. A theme must cycle through completely before beginning on beat 1

3. When a theme is not played, it is suspended or paused. When it resumes, it begins on the next sequential beat.

These rules assured that the themes would all be presented in a method that does not favor a particular theme.

2.7 Structuring the film

Although I did not the filmmaker, I can speak as a collaborative artist in general terms, how the filmmaker, Chris Mich, created the film. He chose to create twenty-four separate images (six times four). These can be divided into four groups based upon locations where the images were filmed. As seen in Figure XXXX,

Figure 9. Musical themes and film scenes in the first route of MFL

Route 1	31	12	13	24	30	1
Film	All	Tree	Flashlight	Running in an outdoor ampitheatre	Film reel sequence	Septa train sequence

3 Presentation of Research

3. 1 Presentation as independent components

In addition to its presentation as a live performance piece and as a film, the music component of MFL has been used as part of the film score for the independent feature film, Most High by director Marty Sader. The film premiered at the Indiefest Film Festival in Chicago, where it won every award for which it was eligible, including the grand prize (the Vision Award) and the Sundance Audience Award. This was followed by a selection to the Hamptons International Film Festival where Most High won the Golden Starfish, the largest independent film prize in the United States, and a selection to the Atlanta international Film Festival, where the film won the Grand Jury Prize. The film is commercially available in North America by Netflix.

3.2 Presentation at Film and Music Festivals

Presentations of MFL at film festivals include screenings of the movie (with recorded soundtrack performed by pianist Augustus Arnone) at the Berlin International Film Festival, ESPN’s EXPN Tube 2000 Film Festival, the Light Plays Tricks 3 Short Film Festival, and the Angelciti Film Festival. In addition, the film has screened on the Independent Images television show on PBS in the United States, and was the highest rated or best film of the year on several internet movie websites.

It has also been presented as a live performance work at music festival including the Society for Electro-acoustic Music in the United States 2002 National Conference, Electronic Music Midwest, the MAVerick Festival 2003, the Midwest Composers Symposium, and the Ukrainian Institute of Modern Art in Chicago. These performances were by several different pianists.

3.3 Future Research

As all of the parameters of MSC are part of a formalized process, the addition of other parameters is a relatively simple process. In the next composition employing similar formalized Magic Square techniques, the addition of location as a formal parameter is the obvious next step. The current 5.1 encoding of multi-channel audio, providing 720 possible options This only considers static locations. In order to apply a more formal approach, a system to determine perceived location would be constructed.

A longer format film or video is another option for the use of magic squares as compositional models. This would also provide an interesting study as to how the passage of time is perceived in magic square compositions when applied to longer structures. The use of magic squares in interactive or non-linear artworks is another possibility. As a route is played, the user could select the next route to be executed. This would generate a large number of possible realizations of the piece, while also providing a set number of variables at the local level. (i.e. the theme would all remain the same).

Clearly there are several possible directions and courses of action to pursue in the construction and realization of other magic square compositions. The complex nature of the magic square allows for many different approaches to create original artworks. As a larger body of works is completed, it will become more apparent which future course of study we will take.

4 References

[1] Andrews, W. S. 1960. Magic Squares and Cubes. New York Dover Publications

[2] Agrippa, Heinrich Cornelius. 2005. De Occulta Philosophia. New York: Llewellen Publications

[3] Warnaby, J. 2005 Peter Maxwell Davies. London: Max Opus.

[4] Browning, Z 1999. Banjaxed. Capstone Records: New York

[5] Browning, Zack. 2006. Phone Interview. Unpublished