GA2: a Programming Environment for
Abstract Generative Fine Art
Philip Galanter, BA, MFA
Interactive Telecommunications Program, New
York University, New York, USA.
looking to use computers to create generative works, especially those artists
inclined towards abstraction, often face an uncomfortable choice in the
selection of software tools. On the one hand there are a number of commercial
and shareware programs available which implement a few techniques in an easy to
use GUI environment. Unfortunately such programs often impose a certain look or
style and are not terribly versatile or expressive. The other choice seems to
be writing code from scratch, in a language such as c or Java. This can be very
time consuming as every new work seems to demand a new program, and the
artist's ability to write code can seldom keep pace with his ability to imagine
new visual ideas.
describes a software system created by the author called GA2 that has been
implemented in the Matlab software environment. By layering GA2 over Matlab the
artist can take advantage of a very mature programming environment which
includes extensive mathematical libraries, simple graphics routines, GUI
construction tools, built-in help facilities, and command line, batch mode, and
GUI modes of interaction. In addition, GA2 is very portable and can run on
Macintosh, Windows, and Unix systems with almost no incremental effort for
GA2 is a work in
progress and an extension of the completed GA1 environment. It is medium
independent, and can be used for all manner of image, animation, and sound
production. GA1 includes a complete set of genetic algorithm operations for
breeding families of graphical marks, a database function for managing and
recalling various genes, a set of statistical operations for creating various
distributions of marks on a canvas or animation frame, a unique
Markov-chain-like operator for generating families of visually similar lines or
paths, and a complete L-System implementation. GA2 extends GA1 by adding more
generative techniques such as tiling and symmetry operations, Thom's cusp
catastrophe, and mechanisms inspired by complexity science notions such as
cellular automata, fractals, artificial life, and chaos. All of these
techniques are encapsulated in genetic representations.
software system I created for creating generative art I named GA1. My intent was
not to create a software tool for general distribution, but rather to create
something I could use for my own artwork.
At the same time I wanted GA1 to provide a general extensible
environment that would yield rewards across a number of projects in a number of
forms and styles. Having said that, GA1
(and now GA2) is generally useful, and I hope to make it available to some
student artists at NYU in the fall semester of 2001.
For the sake of economy, one trade off I
willingly made was to not require that GA1 produce its results in real
time. Like most commercial computer
animation programs, GA1 executes in phases, the last one being a rendering
phase where a single minute of graphics or sound may take a number of minutes,
or even hours, to compute. GA1 is also
not intended to be the engine for interactive works. By lifting any real time requirements GA1 can be programmed at a
high level, and software rigor and elegance do not have to take a back seat to
GA1 is not a single
program, but rather a system of related software modules. GA1 includes subroutine libraries, commands
that can be used interactively, point-and-click GUI tools, and batch oriented
compute-intensive programs. Because GA1
is more like a soup than a monolithic system, any technical overview will be a
bit disorienting at first. Perhaps the
most important aspect of GA1 is that it presents a modular paradigm that can
embrace a wide variety of techniques and algorithms, including (I hope!) many
that have not yet been invented.
Because GA1 is
implemented in Matlab, it is portable across a number of hardware and operating
system platforms, including Mac OS, Windows 95/98/NT, and Unix. For example, initial work can be done on an
inexpensive Mac, and later compute-intensive batch jobs to create a final
animation can be run on a large Silicon Graphics “number cruncher”.
GA1 is not
intended to be an all-in-one package.
By design it is intended to be used with and leverage off of
commercially available software packages, taking advantage of them where they
are useful, and optimizing the artists time by programming only what is
GA1 is not a
static environment, but rather an evolving system that allows the
artist-programmer to build, tweak, and modify tools as needed on the fly.
For purposes of
illustration very simple examples are presented here rather than rich final
compositions. GA1 allows the artist to
assemble any combination of these techniques to create both stills and
animations that are complex and subtle.
Viewed as a
design tool GA1 offers a number of advantages:
Genetic database function - Various styles, line thickness,
color combinations, sonic characteristics, motion tendencies, and so on are
encapsulated in genes. Sets of genes are
concatenated together to form chromosomes, and a single chromosome can be
expressed multiple times to make numerous marks or sounds which exhibit
variation within the same aesthetic look or feel. GA1 includes utilities for
the creation, management, cataloging, and recall of these chromosomes. The
point here is that once I have developed a given look, it’s in the database and
it can be reused later at any time.
Variation generation - Given a library of chromosomes,
i.e. individuals within a given species, I can breed and mutate given
chromosomes to create stylistic variations.
So if something looks or sounds almost but not quite right, I can
generate a number of variations to select from and perhaps breed again. For a
recent series of large stills presented as transparencies in lightboxes, I used
3 species of marks, with about 200 individuals to choose from within each
Non-repeating patterns – GA1 can fill an arbitrary planar
space of any shape with a given design, look, or style without using any tiles
or other form of repetition.
A nonlinear production response - most computer based design tools
are somewhat similar to manual tools in that for a single physical gesture you
create a single mark. Whether using
Photoshop, Painter, or Illustrator the digital product is still in a sense
handmade. The creation of new artwork
is somewhat more productive, but is still very time consuming manual
labor. GA1 can yield a nonlinear return
on the artist’s effort. Once a look or
style is developed, with very little incremental effort the designer can
decorate any number of objects, surfaces, or installations.
A massively parallel architecture – Much of GA1 is compute intensive,
but structured in a highly parallel fashion.
While the parallel structure is not currently exploited the promise
remains that future hardware with multiple processors will yield a nearly
linear improvement in execution time.
1.1 Matlab as a Programming Environment for Artists
Matlab is a
programming environment with a number of features that recommend it for use by
generative artists. While it can be
used for compute intensive “batch” processing, any part of the Matlab language
can be used from the command window interactively. As users write their own routines they can be bundled into so
called “toolboxes” so they are fully integrated into the interactive
environment. Matlab also has an
extensive help system, and user authored help can be seamlessly integrated into
that aspect of the system as well. In
short, Matlab can be highly customized to create a digital workspace closely
fitted to the user’s needs.
Figure 1 – The
MATLAB command line interface.
In the example
shown here a random number function has been called to create a 5 by 5 matrix
of random numbers between 0 and 1.
are overloaded to allow array operations without having to tediously code
explicit loops. In addition Matlab has
a number of mathematical libraries, freeing the programmer from having to
reinvent common functions. In addition
the Matlab routines are often (but not always!) faster than the routines most
users would code at a lower level.
provides built-in engineering graphics tools, and a set of Graphic User
Interface (GUI) routines for the creation of point-and-click controls. Matlab’s graphics are not adequate for
formal fine art work, but in GA1 Matlab graphics and GUI tools are used for the
preparation of control surfaces, paths, fields, and other spatial data types.
systems typically produce works given an initial set of conditions and a
mechanism by which those initial conditions are set into motion to interact and
develop. This interaction produces a
result that is, at least to some extent, surprising even to the artist. The degree to which the artist gives up
control or shapes the result is, in itself, a choice the artist makes. Some systems, by design, allow for little
artistic control, while others are designed to be artist driven.
The model used
by GA1 allows for a great deal of flexibility with regards to artistic control
versus machine autonomy. In addition
GA1 allows for the production of both 2D still and motion picture with sound
works...and could be extended to include other forms such as sculpture.
Figure 2 – GA1
data flow as used to create a video with sound.
Most of the
programming used to implement GA1 has been done in an engineering software
environment called Matlab. In this
example GA1 (via Matlab) generates two files, a script file for the Fractal
Design (now Corel) Painter application, and a Midi file for the MAX music
application with the real time audio MSP extension. In a sense the composition is “finished” when GA1 creates these 2
files. The next steps are fairly
mechanical. Painter interprets the
script file generating a large number of video-sized frames. A MAX+MSP patch I’ve created executes the
Midi file synthesizing a sound file.
The sound files and frames are then matched together on a Media 100
video editing system for the final picture and sound.
For the most
part GA1 is independent of the final rendering mechanism. It would be a fairly straight forward matter
to add device drivers for painting machines, numerical control milling or
stereolithography machines, or other industrial rapid prototyping tools for the
creation of objects rather than pixel based images.
Figure 3 – GA1
internals diagram. Chromosomes and
controls interact and are expressed as complex data structures. These, in turn, via drawing methods and device
drivers create scripts for rendering devices or software.
implemented in Matlab is summarized in the above diagram. In the first step a “seed” for each mark or
object is placed on the (virtual) canvas using one of many distribution methods. Each seed corresponds to an (x,y) canvas
location along with a z coordinate which indicates the back-to-front order in
which the marks are drawn. Each seed
has an entry in the Seed Table, which can later be used to quickly sort and
select seeds by type and location.
(Type here is a proxy for species id number, an individual's id number,
or an arbitrary grouping number specific to the piece.) The Seed Table is linked by pointers with a
much larger Seed Structure Table that is empty when the seed is first
In this system
each seed and corresponding mark or object drawn has a chromosome with genes
which determine how the mark is grown, how it is decorated, and in a motion
picture, how it moves about. Related
growth and motion methods are typically specific to a given species, but often
call a library of shared utility and custom math functions. Along with chromosomes the environment also
includes surfaces and fields which control and modify the innate behavior of a
seed based on its (x,y) location. The
environment provides input to the growth and motion methods, the results of
which are stored as complex data structures that completely describe the final
topology, look, and motion path for each seeds mark.
(Note: In the
context of this paper the term “topology” is used in an informal way to refer
to a sort of stick figure approximation of the final form without any
decoration such as color, paint texture, or the like. It is not a reference to the mathematical notion where malleable
shapes retain abstract relationships.)
Once the seed
structures are completed, a species and medium specific drawing method is used
to render each mark. By providing
alternate drawing methods and drivers, devices such as painting and milling
machines could be used rather than the virtual painting machine offered by
running Fractal Design Painter with
GA1 system includes a number of functions and operations that are described
3.1 The Chromosome Library and Breeding
Each species can
have a number of individuals, and each of these individuals will have a unique
chromosome in that species’ format. New
individuals can be created by taking a single individuals chromosome and introducing
mutations, or by taking the chromosomes from two individuals and breeding them
yielding new chromosomes via crossover recombination.
It is also
possible to extract specific genes from a given individual from one species and
to splice it into the chromosome for an individual of a different species. There is, however, the following
The genetic data
structures used in GA1 are not pure bit strings, as used in classical genetic
algorithm classifier programs, but rather are higher level data structures
specific to a given function. Thus, for
example, a color gene from a given individual can only be spliced to replace a
color gene in another individual. GA1
has both species specific genes and standardized genes common across all
species, such as those for color, absolute length, relative width, and so
on. For the purposes of gene splicing
only the standardized genes can be spliced from one individual to another. Within a species, however, the chromosome
can be crossed over, mutated, or spliced at any gene site.
3.2 Surfaces and Fields
fields are controls the artist can interactively create with GA1 and then use
in compositions. Multiple surfaces and
fields can cover the virtual canvas, and each can influence an aspect of the way
the seed at given point grows. This is
not unlike the way that nutrient concentration in the soil (a GA1 surface) or
the prevailing wind patterns (a GA1 field) will ultimately influence the form a
plant will take. The growth and motion
methods for each seed sample the local surface and field values for use as the
given genes are expressed from genotype to phenotype.
Depending on the
species, surfaces and fields can be used to modulate all aspects of color,
site, shape, orientation, motion speed, audio pitch or timbre, growth or motion
direction, and so on. The chromosome
from a single individual can thus be expressed in a number of ways across the
canvas. This can be under the direct
control of the artist by his manually designing the control surfaces and
fields, or such variation can emerge on its own by having GA1 dynamically
create control surfaces and fields during execution.
Figure 4 – A
control surface and the same control surface converted to a field.
The GUI control
panel shown at the bottom of the page allows the artist to sculpt a 3D surface
with arbitrary slopes, peaks, valleys, etc.
Usually the surfaces are depicted using an elevation map such as the one
shown. When displayed in color red
areas are the highest altitudes, and blue the lowest. The surface can be thought of as assigning a value for every
point on the canvas, some high (red) and some low (blue).
A field, such as
the one shown here, assigns 2 values for every point on the canvas, both a
magnitude and a direction. The artist
can create a field from a surface by calculating the surface gradient via the
GUI control panel. An intuitive way to
think about this is if a marble was placed on one of the red peaks below it
would roll into one of the blue valleys.
The corresponding field shows arrows (force vectors) leading away from
the red peaks and into the blue valleys.
Figure 5 – The
control panel used to create and sculpt control surfaces and fields.
3.3 Seed Distribution
One of the first
steps in any GA1 composition is the distribution of seeds. Seeds simply indicate the spot on the canvas
where a particular kind of mark will be made or at least started.
method is the random distribution of seeds within a border. The diagrams shown here are from tests made during
the early development of GA1. As can be
seen in the plot, when seeds are distributed within a circle using uniform
random numbers, the distribution appears “lumpy” rather than “smooth”.
When using GA1
seeds are often first randomly distributed, and then iteratively moved about to
achieve an even distribution as defined by a stopping function. In this case the distances between each seed
and its nearest neighbors are measured.
Seeds that are “too far away” are moved a bit closer, and seeds that are
“too close” are moved a bit further away.
After many cycles of adjustment the variation in distances settles down
to a constant within a predefined tolerance.
In this case variation was measured as the standard deviation of the
distance between seeds
satisfied the stopping function, the newer “smoother” distribution of seeds is
used in subsequent steps.
Figure 6 – A
uniform random distribution of seeds before and after smoothing.
Seeds need not,
however, be uniformly distributed. A control
surface can be used as a density function to define areas where the seeds are
placed in greater or smaller numbers.
3.4 Path Generation
Paths are a
basic data type used where there is a need to create brush strokes, motion
paths, or any other aspect described by a curved or straight line. A given path gene can be expressed multiple
times to create an arbitrary number of paths that are unique yet similar in
Path genes for
any species can be developed using the GA1 path tool as shown on this page. It allows the artist to generate path genes
by trial and error. The genotype is set
to random values within a set of constraints defined by the artist. The gene is then expressed multiple times,
each initially starting in the same direction.
Where the phenotype (the visible form) shows promise the corresponding
gene can be added to the gene pool of the given species.
GA1 includes a
proprietary mechanism for encoding as a gene tendencies which result in
families of similar looking paths. A
single routine executes all of the path genes for all species. If two
individuals are bred and crossover occurs at the site of the path gene, the
resulting genes will generate paths that show similarities to both parents.
Figure 7 – 3 path genes expressed multiple
times in each plot demonstrating variation.
path genes can be expressed in an environment where there are barriers. The resulting path will avoid the barriers
by executing turns that are consistent with its typical overall shape and
genes can be expressed in the presence of a field. Just as the prevalent yearly wind conditions will, over time,
warp the way a tree grows, fields can direct the way a path develops without
robbing it of its intrinsic shape or character.
While not shown
here, path genes can encapsulate fairly complex behaviors. For example, a given gene could give rise to
a repertoire of behaviors such as a mix of small clockwise loops, large
counter-clockwise loops, and long straight connecting strokes.
L-Systems are a
family of grammar based methods which can be used to simulate and generate a
wide variety of natural branching structures.
As a part of GA1 L-Systems are not the primary center of attention, but
simply one of a number of modules or alternatives for generating form. Also the
use of L-Systems in GA1 isn’t specifically intended for the creation of
representational art, such as drawings of trees and other plants, but rather
the use of natural branching forms reinterpreted for use in abstract art.
Figure 8 – Two
L-System genes developed and expressed in the GA2 environment
GA1 includes a
robust L-System engine, and a method for encoding L-System grammar in a genetic
structure. GA1 L-Systems include
stochastic and contextual functions. Parametric L-Systems will be supported in GA2.
and path genes are growth related genes which only contribute topology. A full rendering also requires decoration
genes that contribute a look, and usually some additional smoothing which
eliminates most of the hard angles.
GA2 is a
software project currently underway and an extension of the intent and
mechanisms of GA1. Following are a
number of improvements and additions either under consideration or in active
development. A primary thrust in the
development of GA2 is the incorporation and application of methods from the
realm of complexity science.
4.1 Adoption of the Open Source Platforms Octave and the GIMP
While the use of
commercial software as part of the GA1 environment yielded quick results and
minimized the amount of programming required before actual artwork could be
created, some of the problems of using commercial software were also
encountered. Support for Matlab on the
Macintosh has been dropped. This is
unfortunate for computer artists, as the Mac is the leading platform for many
related forms of media production such as music and video production. Its unclear whether Matlab support will
return with the advent of the Macintosh OS X.
Painter has had its ups and downs in terms of vendor support. In addition, Painter’s scripting
capabilities have been problematic when working with large files, such as some
recent 12,000 by 9,000 pixel resolution images.
The open source
community now offers viable alternatives to both, at least relative to GA2
requirements. Octave is a sort of
Matlab clone in terms of its programming language, and may allow for an easy
port of GA2. However, Octave’s graphics
support is acknowledged to be inferior to Matlab.
GIMP (or “the
GIMP”) is an open source image-editing package that includes paint tools, and
can be used in a manner similar to Photoshop or Painter. GIMP can also be used via scripts and can
run as an image-rendering server. This
would allow GA2 to be used for web based art projects involving the dynamic
creation of generative art.
4.2 Iterative Processing Allowing Computer Vision
GA1 is a one way
pipeline where the generative aspects are executed, and then a rendering system
creates the final output. Methods are
being explored to allow GA2 to iteratively generate elements, render them, read
in the resulting image, use computer vision techniques to (for example)
identify areas in the image for further work, generate additional elements,
render them, and so on. This may be
done by connecting either Octave or Matlab to GIMP via Perl scripts. Its unlikely this could be done using the
current version of Painter.
Such a mechanism
would also allow GA2 to process real world photography or video footage. One can imagine, for example, artwork
created via an autonomous generative art rotoscoping system.
4.3 Tiling and Symmetry Procedures
A number of
routines are in development to support symmetry operations about a point, about
a line, or filling a plane, as are a number of tiling procedures. Along with the obvious uses for tiles, one
can use tiles as another method to distribute seeds. In addition, rule based systems can process regular tiles and
combine them to form complex areas or contours. Finally exotic tiling patterns can be used to explore cellular
automata which are not in the typical square grid configuration.
4.4 Cellular Automata
in GA1 and GA2 are simply arrays of floating point values that approximate a
surface but are not continuous. (When
values for a specific point on the surface are needed, however, they are
calculated via Gaussian interpolation, and thus the surface can be used as
functionally continuous). Since
surfaces are simply rectangular arrays they can also be used as the grid for
cellular automata. A set of generic
cellular automata routines will allow artworks, especially animations, to
exhibit various forms of emergent behavior.
For example reaction-diffusion processes could be simulated which would
then act as a control surface to create the kind of organic looking stripes and
spots one sees in seashells, fish, and other animals.
4.5 Thom’s Cusp Catastrophe
catastrophe theory has not lived up to its initial promise as a broad tool for
model building, it remains a useful artistic tool for situations where one
wants a generative behavior which exhibits bifurcation, hysteresis, and
sensitivity to initial conditions.
Basic functionality of this kind would be most useful in time based art,
but could also be useful in creating processes which exhibit emergent behaviors
which ultimately result in a single image.
4.6 Fractal Noise Generation and Chaotic Simulation
One of the
oldest generative art strategies is the use of chance operations. The GA2 implementation of additional random
number generators exhibiting non-uniform distributions or autocorrelation, such
as 1/f noise, will be fairly straightforward and quite useful. The accurate simulation of truly chaotic
dynamical systems is, of course, an exercise in frustration. But as an area of active research for the
computer artist, the development of software systems that are “chaotic enough”
is certainly worth exploring. The
“feel” of chaos in a time based work, whether visual or sonic, is very fertile
ground for the artist.