Paragraphs about Papers for CS680 for 2/26/97

The animator can control the fish by specifying individual characteristics, while letting the model control all the details. Thus, the animator can specify has fearful, lusty and hungry the fish will be. The fish can tend to school, or not, and the fish can be predator or prey. Similarly, the animator can specify what kinds of environments the fish likes, such as warm or cold water, and light or darkness. However, there is clearly more work to be done in unifying all the behaviors. In this paper, the predators were only hungry, rather than lusty or fearful, and never ate each other. It would be interesting to see simulations where that wasn't the case. Having looked at the videos, the results are impressive and very realistic.

Genetic algorithms provide a way of "evolving" virtual creatures which will perform a given task very well. For instance, the creatures are provided with the ability to sense light; some can be evolved to follow the light. Movement, such as swimming, walking, and jumping can also be evolved for. An advantage is that the animator need not understand or design the model or motion. The appearance of the model can be made to comform to certain limits by including those details in the genetic language and in the fitness tests. The main method the animator has for affecting the results is by selecting successful creatures as the seed for another evolution and by specifying the fitness criteria.

The computation power necessary is rather impressive, requiring a CM-5 for the evolutions. However, the resulting creatures perform as desired, and are significantly more varied than most people are likely to come up with.

Commentary 6

Due: 2/26/97

This paper is describing a system for creating Artificial Fishes. These Artificial Fishes are autonomous entities that have a set of different systems that help them to produce there behavior. The strength of this system is that the use does not have to give a intricate inputs or have specific understanding of how the fish is supposed to move to be able to use the system. That is the animator can describe robust models of ecosystems without having to know the simulation details. This provides a useful layer of abstraction away from the simulation and allows the animator to concentrate at the job at hand, designing the desired system.

This system is constructed from a bottom up approach. It starts to build the model from the appearance and also the physics of how the animal would move in its environment. To this end the model of the fishes is a "spring-mass system". This allows the system to model behavior that will be realistic in its look and relation to a physical properties of a real environment. This system is broken up into three parts the motor system, the perception system and the behavior system.

The motor system of the fish resembles a large model or puppet that is controlled by a series of strings or springs in this case that will control the movement of the fish or creature being processed. This is processed by the Lagrange equation. The perception system is made up from a vision sensor and a temperature sensor. This allows the creature to determine how much light is present within a predetermined field of view. The behavior system is based on a system of states that the creature can be in. This system was the most interesting because it allows for the most variation in actions. By modifying this system the user has total control over how the creature interacts within it environment.

"Evolving Vitual Creatures" by Karl of Sims Thinking Machines Corporation

This paper is describing a system for creating evolving creatures using genetic algorithms. These creatures are then produced by a series of genetic trials. These trails seek to maximize the creatures overall fitness.

The system used is made up from three different sensor types. They are Joint angles, Contact sensors and Photosensors. These give the system control over the movement of the joints; collision detection and detection of light. To give the impression of a memory in the creature the system implies a Neuron. This serves to allow the creatures to interact move realistically. These two components are combined in a Effector so that the inputs from the sensors can be scaled and used by the creature. The paper listed four examples of how one might create a fitness criteria. These are swimming, walking, jumping and following. Evolution was accomplished by either mating and by randomly mutating the genes from the parents. This allowed for random creation of the virtual creatures.

Like all engineering questions. There is a good side to the process of evolving these shapes and there is a bad side. The good side is that the user does not need to know the complexities and the control parameters that are used in creating the model. The bad side is that this gives the users little control over what is going on within the creature or mechanism that is begin produced. I don't see the use of this for ascetic reasons. You lose control over what the design of the creator is and aesthetic what it looks like. I think that it may be a good way to generate new ideas and way of thinking about designing creators that we may not have though about naturally though.

Evolving Virtual Creatures -- Karl Sims

Sims presents a method for creating virtual creatures that move and behave in simulated three-dimensional worlds with the use of genetic algorithms. He defines a genetic language that uses nodes and connections as primitive elements to represent directed graphs which describe the morphology and "brains" (neural circuitry) of the creatures.

This paper was intriguing and smart in the sense that Sims does not simply attempt to re-create nature as we know it, instead he succeeds in building an environment that contains novel creatures. This seems quite important in the respect that Sims not only addresses the mechanics of building and animating the creatures but that he manages to add another dimension to the computer. In essence, the computer becomes the world for these creatures and we become mere observers, or voyeurs looking in on their existence. The computer is effectively removed from its role as a mere tool.

Of particular interest to me was the evolution of the creatures depending on on which task or behavior was to be optimized. For example, the creatures could swim, walk, jump and follow. However, although these are actions that we readily associate with predefined movements (e.g., walking by putting on foot in front of the other), the creatures evolve different appendages and means by which to successfully accomplish the task (e.g., using corners of their parts). The vast variety of evolved creatures goes beyond that which we could pre-model or predict.

Another aspect of the paper that I found interesting was the notion of family's of creatures that propagated from one generation to the next with only the fittest surviving (the fittest being those who were successfully performing the tasks). This mirrors our own and basically all of natures existence. Sims discusses the possibilities of experimenting with other types of fitness evaluation methods though this seems difficult because we can only base judgment on what we know from our world. Perhaps there would be some way to evolve fitness criteria within the "virtual" world as well as the actual evolution of the creatures.

I keep imagining being immersed in a virtual reality environment and interacting with these creatures. Would I proceed to the next generation?

Artificial Fishes: Physics, Locomotion, Perception, Behavior -- Xiaoyuan Tu and Demetri Terzopoulos

In this paper the authors propose a framework for animation that achieves the motion evident in certain natural ecosystems with minimal input from an animator. Specifically, they focus on developing a framework in which fish, in their natural environment (the water) can be simulated. The main points include (1) key frames are not needed and (2) the fish are autonomous and fully functional.

The fish are modeled as spring mass systems based on their actual physics in the real world and achieve locomotion via control algorithms that exert forces on the model depending on the state of the systems parts in relationship to their interaction with the environment (i.e., the water).

An interesting part of the system is the incorporation of sensory perception, specifically the vision sensor. This separates the paper from others dealing solely with locomotion and adds an aspect of intelligence to the creatures being modeled. I particularly liked the idea of the creatures ability to query a database when trying to identify an object and the creatures ability to interrogate the physical simulation to obtain information such as instantaneous velocities of objects of interest. However, I did not quite understand how they accomplished the latter.

The authors incorporate a behavior subsystem to mediate between the motor subsystem and the perception subsystem. For example, this effectively allows a fish who has a mating habit (habits are pre-determined) to avoid an obstacle and then resume the mating behavior.

The framework seemed rather constrained but the results (mpeg movies) were quite impressive. The authors succeeded in achieving realistic, natural looking motion.

Artificial Fishes: Physics, Locomotion, Perception, Behavior

This paper describes a model of physics-based animation of life-like fishes. The fish are 3D models of connected springs. Three main motor controllers control the motion of the fish through water. The behavior is modeled through the Mental State (hunger, libido, and danger levels) and the Intention Generator (setting goals for the fish). The society of artificial fishes has different types of fish, predators, prey, and pacifists. All together, the outcome is very interesting. Watching the movies of the authors' work shows that the fishes are very life-like with many realistic features. I believe that physics-based modeling is a great way to learn about complex behaviors of life, but it takes a model that is extremely accurate in order to have meaningful results. Unfortunately, you need a computer as efficient and powerful as the brain of what we are modeling (this is conjecture). The authors have taken a big step toward simulating true models of fish.

Evolving Virtual Creatures, by Karl Sims

In this paper Karl Sims describes a genetic language used to "grow" virtual creatures. The creatures are autonomous, evolving semi-lifelike 3D models that can learn and develop behaviors. The most interesting part is the use of genetic algorithms for the evolution of the creatures. These creatures are very impressive. They can learn to follow, walk, run, jump. They can reproduce, live and die. The creatures are created from a "genotype" directed graph that can be modeled with 3D blocks (phenotype). The creatures also have an evolving virtual brain that controls the behavior. Not much attention is paid to social interaction (other than mating) which is a key part in evolution, but nonetheless this is a great paper on the showing the possibilities of simulating realistic evolving creatures.

They key idea is to create autonomous, fully functional artificial creatures whose appearance and interaction are closely related to their real ones. For this, locomotion, perception behavior are combined in this work. This work uses fishes as its study o bject.

The fish model is a an animated spring-mass system with muscles that can contract to produce motion. The motor system achieves a muscle-base hydrodynamics locomotion by simulating the forces of interactions of the deformable body with its environment.

Physics of the fish model and locomotion:

Some basic assumptions are used: caudal swimming uses posterior muscles, turning uses anterior muscles. To get realistic fish locomotion, they use a model consisting of 23 nodal point masses and 91 springs. The spring arrangement maintains the structural stability of the body, at the same time it allows it to flex. To control the mechanics of the model, they use for each node i, a mass mi, a position Xi(t), velocity vi(t). They also define for each spring Sij, its rest length, deformation. The force that it exerts on node i is fij and -fij on node. They use an implicit Euler method to integrate the differential equations of motion. Each time step involves the resolution of some system of equations.

The movement is simulated by contracting springs, (decreasing their length. For example the swinging of the fish tail is achieved by periodically contracting the swimming segment springs on one side of the body while relaxing their counterparts on the other side.

It sets in motion a volume of water, the inertia of it produces reaction force normal to the fish body, an proportional to the displaced volume of water.This makes the fish go forward. So in this way they can simulate how and in what direction the artificial fish will move according to its current position and tail movement.

Motor controllers: the fish has three MC's. The swim-MC produces straight swimming (control the segment muscles), left/right-turn-MC's execute turns (control the turning segment muscles)..., the sinusoidal body shape when the fish swims is achieved because the swim-MC produces periodic muscle contractions in the posterior swim segment which lag 180 degrees behind those of he anterior swim segment, producing such effect.

They also include an explanation about how the fish turns using its anterior muscles. (2.3)

Modeling the pectoral fins was simplified, they influence the fish motion but not in the detail they are using it the simulation. They preferred to simplify the model.

About the sensory perception:

Vision sensors: covers a 300 degree spherical angle (uses one sensor). It extract from the 3D virtual world only the most useful info (color, size, distance and object identification). Geometry, material property, illumination are available to the fish (this info is part of the rendering info). The vision system can search in the database to identify objects, also the fish can interrogate the physical simulation (instantaneous velocities, objects of interest). So no extraction of information from images, no binocular vision.

Temperature sensors: sample the temperature at the center of the body fish.

About the behavioral model.

They define habits for each fish (like darkness, cold/ warmth preferences, schooling, is male/female...)

With respect to mental states: they define hunger, libido, fear. Certain choices produce fishes that are hungry all the time, or sexual mania.

With respect to the intention generator: it determines the immediate goal-directed behavior of the fish. More or less it works as follows.

Checks collision-> it generates "avoid collision". Some timid fishes takes evasive action very quickly. A tight sensitivity regions produce a courageous fish that take evasive action at the last second.

Check for predators-> generates school intention (depending if the fish has school habits or escape intention is generated).

If there is not enough fear (they use a threshold), hunger and libido are calculated. So if one of them are over a threshold, the eat or mate intention are generated.

Wander, leave intention are other states. Depending on the intention, the perceptual focus mechanism is invoked, which dictates to what things the fish will pay attention.

The intention generator issues an intention based on the fish habits, mental state and sensory information.. Then it chooses and executes a behavior routine, then it runs the appropriate motor controllers.

The persistence of intention is also analyzed, so they use a simple short term memory mechanism to avoid switching goals very often.

Every intention is described in terms of actions -> animation.

To improve the realism of the simulation, they use different kind of fishes. Predators, prey, pacifist are defined. They are simplified, for example, in predators, escape, school and mate intentions are disabled. They provide a very good description of the main habits -behavior.

In general I think that this is a very good animation model that combines a right amount of physically-based characteristics with behavior variables. It indeed cut the work for a final animator, but some control is lost (it really does not matter to much for most of the applications).

In general this paper presents a technique for creating virtual creatures that can move and behave in a 3D physical environment. To describe the morphology and the neural structure of the creatures, a genetic language is used which uses nodes and connections to represent directed graphs.

The user loses some control on the animation, (although his influence is still maintained by specifying the fitness criteria) but a gain in automating the creation of complexity is obtained. In this work, the optimization determines the creature morphologies and their control system, which is a new approach according to its author. Also the approach is fully physically based.

Directed graphs are used here to represent the virtual creatures. Nodes represent rigid parts, dimension s determine the physical shape of the part, joint types determine how the relative motion between the part and its parent is defined.Each connection has information of th position, orientation scale and reflection of a child part relative to its parent.

It uses a virtual brain which way of working is not biologically plausible, but it is simple and effective for the simulation. It basically accepts sensor values and provide output effector values.

Sensors provide information of a specific part or aspect of th environment relative to it. They use basically 3: joint angle, contact sensors, and photosensor.

Neural nodes are used to give virtual creatures the possibility of arbitrary behavior. In this way the creature has something like a internal state determined by history and other parameters.

Effectors receive values form sensors or neurons. This values is used to simulate muscle forces. Each effector controls a degree of freedom of a joint. Effectors have their own parameters (e.g.max strength)

For the physical simulation, they used articulated body dynamics, numerical integration collision detection, collision response, friction and viscous fluid effect. To calculate the accelerations they use the velocities and external forces of each hierarchy of connected rigid parts. The shapes of parts are represented by rectangular solids only. For collision for example, parts whose world-space bounding boxes intersect are tested for penetrations. Collisions with a ground plane are also tested. Connected parts can interpenetrate but not rotate completely through each other. Also a viscosity effect is used for simulations underwater.

With respect to the behavior, it dictates the way creatures evolve. They evolve by optimizing for a specific behavior.A fitness value is used to determine the success of a behavior.

Before creatures are simulated for fitness evaluation, some are discarded because of viability checks. Some fitness methods are used, swimming for example, uses speed, straight swimming evaluators, continuing movement evaluators are used as fitness values. Walking, jumping, and following are also used in this paper.

Creatures evolve by creating an initial population of genotypes. Random, existing genotypes from previous evolutions are used here. For survival, only those individuals whose fitness fall within the survival percentile are reproduced, their offspring replace those individuals that did not survive. SO it will generate better and better individuals, (most of the time). Something very important is that the offspring are generated by combining graph genotypes of surviving creatures, plus some mutation. This can actually create better-adapted individuals, even better that a simple combination of the characteristics of their parents.

Despite of this idea, populations of inter-related creatures often converge to homogeneity, so it is necessary to perform many separate evolutions if we want diversity.

The results were really interesting, showing the development of specialized body parts for specific behaviors. The way the genetic language works allows the definition of a very big variety of possible creature-behavior units.

Altohough the approach is not very biologically plausible (for neuroscientists or biologist) it is good for animation. Provide good ideas for further research. I particulary like the way the genotype produces the phenotype

Using this idea, a lot of new forms of interesting strategies for solving a problem can be generated, for example locomotion strategies. By using other types of sensors the creature can easily be improved. No detailed specification have to be defined by the user.

The main problem here is how to define good fitness functions, the user may think that his function is good for something, but it may also create secondary effects. Or the function may not be as accurate as we think, creating not desirable effects. According to its author, the user do not need to know too much about the system, but what is it necessary to know in order to define the fitness functions?.

Complexity, without big effort of the user, but complexity in this way can create redundancy of lose of efficiency (computations), I think that for now it is OK if the realism is high. Increment in complexity may be the only feasible way. If by using normal step by step methods the human cannot create this complexity then it seems like the only way.

A lot of food for though in this work. This approach is a very interesting step toward the creation of virtual environments-creatures that can be extended in many ways.

Stan Sclaroff

Created:  Jan 21, 1997

Last Modified: Jan 30, 1997