Evolving Finite-State Machines Controllers for the Simulated Car

Evolving Finite-State Machines Controllers for the Simulated Car Racing

Championship

Bruno H. F. Macedo

Gabriel F. P. Araujo

Gabriel S. Silva

Matheus C. Crestani

Yuri B. Galli

Guilherme N. Ramos

Faculty of Technology, University of Bras

ılia, Brazil

Department of Computer Science, University of Bras

ılia, Brazil

Abstract

Autonomous vehicles have many practical applications, but the de-

velopment of software controllers for such use has several difﬁcul-

ties. This work presents a ﬁnite state-machine model with evolved

parameters as a suitable solution for a self-driving car, an approach

that enables a clear division of behaviors in states, providing an

easy way to test different conﬁgurations and simplifying the search

for better controllers by allowing changes only in selected states.

A 5-state and a 3-state drivers were evolved through genetic algo-

rithm, and a learning module developed to improve their behaviors.

These were compared to each other and to AUTOPIA, the current

state of the art controller for the Simulated Car Racing Champi-

onship, using The Open Racing Car Simulator. Results showed that

the proposed model has potential for racing, besting one of AU-

TOPIA’s qualifying marks, and provide insights on developing and

conﬁguring the model.

Keywords: ﬁnite-state machine, genetic algorithm, TORCS, sim-

ulated car racing

Author’s Contact:

{bhfmacedo,gabriel.araujo.5000}@gmail.com

gabriel [email protected]

{programadorﬁles,yurigalli}@gmail.com

gnramos@unb.br

1 Introduction

Recent history has seen an increasing amount of resources and

scientiﬁc research being applied to the automation of day to day

tasks [Terzic et al. 2008; Bejczy 2012]. One of the more desirable

goals is the advent of self-driving cars, which should drive safely

and efﬁciently, within trafﬁc laws [Carbaugh et al. 1997; Barth

et al. 2013], improving on the current issues of trafﬁc in urban roads

by aiming at shorter response times, better fuel consumption, and

lower levels of pollution. Such drivers, however, face several difﬁ-

cult challenges such as perception, navigation and control [Luettel

et al. 2012].

Besides the complexity of executing the task, the process of de-

veloping driving algorithms is also complicated since testing is a

frequent necessity and real driving systems are expensive. Creating

and testing solutions can be aided by realistic car racing simulators

which can closely simulate the roads, other vehicles, and their com-

plex interactions [Caldeira et al. 2013]. Such software tools also

present a well deﬁned environment which may be used not only

for applications of machine learning results, such as neuroevolu-

tion [Cardamone et al. 2010] or human pose recognition [Shotton

et al. 2011], but also for comparing Artiﬁcial Intelligence (AI) so-

lutions for speciﬁc problems such as path planning [de Freitas et al.

2012], controlling human non-playable character [Lucas 2008], car

racing [Loiacono et al. 2010], among others.

The Open Racing Car Simulator (TORCS) is a modern, modular,

highly-portable multi-player, multi-agent car simulator [Wymann

et al. 2014], one of the most advanced racing games available, and

frequently used as a platform for comparing driver solutions in the

Simulated Car Racing (SCR) Championship [Loiacono et al. 2010;

Loiacono 2012]. TORCS/SCR controllers handle a car through

an sensor/actuator interface, similar to the ones used in real au-

tonomous vehicles, albeit much simpler. This abstraction resembles

how the robot Stanley [Thrun et al. 2006] handled itself to become

the ﬁrst winner of the DARPA Challenge for autonomous driving

cars. This paper uses a ﬁnite-state machine (FSM) controller to ex-

ploit the advantages of a divide-and-conquer approach by consid-

ering the task’s various situations as distinct states (racing, getting

back on track, etc.).

FSM is a well-known mathematical model with widespread ap-

plications such as controlling air conditioning systems [Bernaden

1999], highway surveillance systems [HyunKim et al. 2013], and

even simulated car racing [Perez et al. 2009]. Such solutions are im-

plemented as software and conﬁgured according to a speciﬁed set

of parameters, and deﬁning the best possible conﬁguration is usu-

ally a huge combinatorial problem for which exhaustive systematic

searches become unfeasible.

Several machine learning approaches have been applied to this

issue (heuristic algorithms [Quadﬂieg et al. 2014], evolutionary

algorithms [Nallaperuma et al. 2014], modular fuzzy architec-

tures [Onieva et al. 2012], sensory-to-motor couplings [Butz and

onneker 2009], among others). One of the most successful opti-

mization methods is Genetic Algorithm (GA), which has been ap-

plied to a myriad of problems such as non-linear systems identiﬁca-

tion [Wang et al. 2003], biomedicine prosthesis development [Wang

et al. 2009], agent behavior forecasting [Bharathi et al. 2007], im-

proving classiﬁers [Pedrycz and Succi 2005], and others.

This work uses a GA for searching for optimal parameter settings

for the proposed FSM based driver, using TORCS as test bed. The

rest of this paper is structured as follows: Section 2 introduces the

TORCS/SCR, ﬁnite-state machines, and Genetic Algorithms con-

cepts involved; Section 3 details the proposed controller model.

Section 4 describes the evaluation and validation process; and Sec-

tion 5 presents concluding remarks.

2 Related Work

Scientiﬁc research has been enormously aided by the evolution of

simulators, which greatly simplify and reduce initial development

costs. These tools have been widely used in several areas, such

as medical education [Zhang et al. 2007], aiding decision mak-

ing [Levy et al. 2002], aviation industry [Duncan and Feterle 2000],

and automotive research and development [Xue et al. 2011]. In the

ﬁeld of Machine Learning (ML), simulations are specially helpful

for training and learning through experimentation.

Advanced car simulators model several elements of a vehicular dy-

namics, including inertia, suspension types, differentials, friction,

aerodynamics, and others [Wymann et al. 2014]. These models

represent an approximation of real systems, and the differences be-

tween model and real system results that stems from the simpliﬁ-

cations made have to be considered [Whitton and Moseley 2012],

so these tools cannot completely replace experimentation on ac-

tual cars. However, current simulators provide such realistic expe-

riences, and enable a simpler, faster way to experiment that they are

ideal for the initial development phase.

2.1 TORCS & SCR

The Open Racing Car Simulator

is a platform that is renowned for

its highly credible physics modeling engine and yet user-friendly

interface with very customizable environment for car racing simu-

lations [Loiacono et al. 2013; Wymann et al. 2014], and has widely

http://torcs.sourceforge.net/

SBC – Proceedings of SBGames 2015 | ISSN: 2179-2259

Computing Track – Full Papers

XIV SBGames – Teresina – PI – Brazil, November 11th - 13th, 2015

used in AI research for developing and comparing solutions [Loia-

cono et al. 2010]. It considers factors such as collision, traction,

aerodynamics, and fuel consumption; and provides several circuits,

vehicles, and controllers [Loiacono et al. 2010; Loiacono 2012],

enabling all kinds of possible in-game situations. Additionally, it

is open source, with an active community, making it possible to

suggest modiﬁcations of its source code (and encouraging them) to

better suit speciﬁc needs. It has been used as a standard platform

for simulated racing since 2007 [Loiacono 2012].

TORCS tracks

differ in categorization according to the surface

they simulate, and there are three types available: road, with asphalt

surface in wide variety of environments; oval, which also have as-

phalt but are generally wider and have smoother curves than road;

and dirt, which are usually short and bumpy tracks with dirt or ice

surfaces. Each type has its own characteristics that inﬂuence the dy-

namics of the cars and the necessary driver abilities during a race.

There are also several cars available

, each with its speciﬁc weight,

suspension, differentials, tires, aerodynamic and other proper-

ties [Wymann et al. 2014]. The AI controllers available with

TORCS are called robots, and they are linked to the game engine

and have access to its internal model. External drivers are called

controllers, and can these can be dynamically linked to the game

engine, like the robots, or communicate through the SCR interface.

The Simulated Car Racing Championship is a competition between

controllers built on TORCS [Loiacono et al. 2013]. It was the

ﬁrst simulated car racing championship organized as a joined event

of major scientiﬁc conferences: IEEE Congress on Evolutionary

Computation, the ACM Genetic and Evolutionary Computation

Conference, and the IEEE Symposium on Computational Intelli-

gence and Games [Loiacono et al. 2010], and has been accepted by

researchers as suitable platform for evaluation and comparison of

controllers [Wymann et al. 2014].

Controllers are ranked according to performance during the cham-

pionship, which consists of several races on different tracks di-

vided into legs, spread through the conferences [Loiacono et al.

2010]. Each race has three stages: warm-up, where the controllers

race alone on the track before being evaluated (to encourage on-

line learning); qualifying, where the driver races alone for a ﬁxed

amount of time; and the main race, where the best controllers in

the previous stage race against each other and are scored using the

Formula 1 point system

The software for SCR extends the TORCS architecture by structur-

ing it as a clientserver application, incorporating real time process-

ing, and physically separating the driver code and the race server

through a sensors/actuators model abstraction layer [Loiacono et al.

2010]. These changes provide an even more interesting environ-

ment for researchers, they enable any kind of controller implemen-

tation (as long as it can communicate via UDP connections) and

deﬁne a clear abstract sensor/actuator interface between controller

and simulated car, which can be easily adapted for testing the con-

troller with a different simulator or even a real car.

Car control is based on the drivers perceptions and the available

actions; inputs are data about the car status (gear, fuel level, etc.),

the race status (lap time, distance raced, etc.), and car surroundings

(track borders, obstacles, etc.); outputs are steering, acceleration

and breaking, gear. There are 19 range ﬁnders available, these pro-

vide distance readings between the car axis and the track and their

conﬁguration can be customized by the developer. Further details

for sensor and actuators are presented in the Simulated Car Rac-

ing Championship Competition Software Manual [Loiacono et al.

2013].

SCR controllers have been developed using several AI techniques,

such as neural networks, fuzzy logic, potential ﬁelds, and genetic

algorithms [Loiacono 2012]. In the competition, the race tracks

are unknown and several drivers incorporate machine learning pro-

cedures to improve their performance in the Championship [Loia-

cono et al. 2010], attempting to not only develop a controller that

http://www.berniw.org/trb/tracks/tracklist.php

http://www.berniw.org/trb/cars/carlist.php

http://www.formula1.com/

can handle tracks with diverse characteristics, but exploiting the in-

formation available during the stages. These procedures can essen-

tially be online, which learn by moving about the environment and

observing the results, or ofﬂine, which learn solely by simulating

actions within an internal model [Mitchell 1997].

Several approaches to implementing a controller have taken part

in SCR, including imitating the human behavior [Cardamone et al.

2009; Muoz et al. 2010]. Mr. Racer, which won the last three cham-

pionships (2011 to 2013), employs several heuristics and black-box

optimization methods in a a modular structure in order to repro-

duce human-like mechanisms [Quadﬂieg et al. 2014], and applies

a Covariance Matrix Adaptation Evolution Strategy (CMA-ES), to

evolve parameters ofﬂine.

In the GECCO leg of the 2013 competition

, AUTOPIA’s perfor-

mance stood out from the other competitors. It implements a fuzzy

architecture with gear, steering and speed control modules which

are optimized ofﬂine by a genetic algorithm and an online learning

mechanism for landmarking lane exit points to avoid leaving the

track [Onieva et al. 2012].

Not surprisingly, many participants use combinations of ofﬂine

and online learning, as well as a modular approach to implement-

ing them [Loiacono et al. 2010; Perez et al. 2009; Cardamone

et al. 2009]. Modularity may advance development and optimiza-

tion by allowing parts of the controller to be improved indepen-

dently[Quadﬂieg et al. 2014; Onieva et al. 2009], or to separate

driving behaviors and work on them separately.

2.2 Finite-State Machines

One of the most simple and intuitive models for implementing dif-

ferent behaviors is a ﬁnite-state machine, a model which views a

system as a black box with a number of inputs and outputs with a

ﬁnite number of possible states, of which it can be in only one at

any given instant [Belzer 1992]. FSMs have been widely used in AI

application [Millington and Funge 2006], mostly due to its inher-

ent characteristics of ﬂexibility, modularity, and intuitive behavior,

among others [Buckland 2005].

States are usually implemented with hard-coded rules concerning

a speciﬁc situation [Buckland 2005], which in turn demands small

amounts of processor time, and can be easily implemented in dif-

ferent manners. The different driving behaviors can then be coded

in parallel with less effort, and easily translated into different states

of a FSM. Such approach has successfully been used in SCR [Loia-

cono et al. 2010; Perez et al. 2009].

The proper conﬁguration of states and transitions, however, are a

more complex problem. Several possible solutions exist to solve

this, from a hard coded approach bases on trial and error to different

machine learning techniques (or the composition of both).

2.3 Genetic Algorithms

GAs are a particular kind of genetic optimization mechanism, in-

spired by evolutionary algorithms from Darwin’s theory of natural

selection [Holland 1984]. They are probabilistic search procedures

designed to work on large spaces [Goldberg and Holland 1988],

and have been successfully applied in many areas such as non-

linear systems identiﬁcation [Wang et al. 2003], prosthesis develop-

ment [Wang et al. 2009], economic dispatch [Bharathi et al. 2007],

neuroevolution [Stanley et al. 2005], data classiﬁers [Pedrycz and

Succi 2005], and others.

This well-known approach works by generating successor solutions

by repeatedly mutating and recombining parts of the best currently

known solutions, replacing a fraction of the population by off-

spring [Mitchell 1997]. This is interesting because it works with

little information on the problem’s domain, the only requirements

related speciﬁcally to the problem are a representation of a solution

for it and a means for evaluating it’s quality (ﬁtness).

http://www.slideshare.net/dloiacono/gecco13scr

SBC – Proceedings of SBGames 2015 | ISSN: 2179-2259

Computing Track – Full Papers

XIV SBGames – Teresina – PI – Brazil, November 11th - 13th, 2015

In the context of a simulated self-driving car controller, a solution

can be seen as a representation of some of the driver’s parameters

(or conﬁgurations) and the ﬁtness a measure of its performance,

considering speed, safety, fuel consumption, or whatever is the de-

veloper’s interest. Considering SCR, the ﬁtness depends on the

stage of the competition; in warm-up it could be anything from dis-

tance raced or average speed to the amount of information gathered

from the track to be used in the following stages, which have their

own metrics. In the qualifying stage, ﬁtness should be the distance

driven during the given simulation time as that is what deﬁnes the

participants to the race. In the race, though the winner gets more

points, several other factors must be considered into play, such as

average speed, car damage, and opponents position (and points in

the Championship).

3 FSMDriver

Driving a car can be intuitively divided into several behaviors, and

thus provide a straightforward implementation as a ﬁnite-state ma-

chine model. Such model’s conﬁguration parameters can be op-

timized through a genetic algorithm, and the TORCS/SCR plat-

form provides a standard measure of driver quality, while its sen-

sor/actuator interface enables it to be readily replaced by actual

robotic prototypes (or other more advanced simulators) for fur-

ther testing. This work proposes to use these ideas for evolving

a controller for a car, the FSMDriver, as one more step towards

autonomous vehicles.

3.1 Driving Behaviors

A state in the FSM model considered here deﬁnes a racing be-

havior, deciding the driver’s output solely based on the given sensor

input (a Moore machine [Ajzerman et al. 1971]). The FSM imple-

ments a transition function that, at every game tick, analyzes the

sensor inputs, as deﬁned by SCR’s interface, and decides which

state best represents the current situation, triggering a change in

states if necessary. The current state then processes the sensor read-

ings and deﬁnes the proper actuator outputs according to its imple-

mentation.

Intuitively, a single Driving state could be a deﬁnite solution for

such FSM model, but self-driving cars are obviously involved in

several distinct situations, each of which could be translated into a

state and, if possible, developed independently from the others. Not

only does this approach make it easier to develop and test changes,

it also enables parallel implementation.

Considering TORCS/SCR as the testing environment, it is clear that

two antagonistic situations which require speciﬁc behaviors exist:

Racing, for situations where the car is in “proper” racing condi-

tions, and Recovery, for when it is in such situation that distinct

actions are required to return to the racing conditions. These can be

further divided into the more speciﬁc behaviors for implementation:

Straight Line: for racing straight ahead;

Curve: for racing through a curve;

Out of Track: for recovering from leaving the track or facing

the wrong direction; and

Stuck: for recovering from being unable to move forward.

Each of these states implements its racing policies as follows.

Straight Line attempts to go as fast as possible parallel to

the track axis by accelerating at full throttle, and changing gears

according to RPM thresholds. Curve steers towards the direction

towards the track sensor with the largest reading (see Figure 1a)

with 60% throttle, braking in case it starts to slide in the X axis.

Out of Track attempts to return to the track, facing the right

direction, adjusting its speed and steering according to its current

orientation concerning the track (see Figure 1b). Stuck tries to

get unstuck by using the reverse gear and hard steering for a limited

amount of game ticks and then switching to Out of Track or a

Racing state.

The transition function for this FSMDriver, presented in Algo-

rithm 1, is quite intuitive. The car is considered stuck if it can

only move in a very slow speed or not at all for some duration,

it is considered inside the track by direct sensors reading and in a

curve if the readings from the range ﬁnders in both sides are not too

different.

Algorithm 1 FSMDriver Transition

if car is stuck then

state ⇐ Stuck

else

if car is inside track then

if car is in curve then

state ⇐ Curve

else

state ⇐ Straight Line

end if

else

state ⇐ Out of Track

end if

Initially, random conﬁgurations for the states’ parameters were set

and manually adjusted until an acceptable behavior was achieved,

i.e. such a conﬁguration in which a car could successfully com-

plete a race. In the beginning, the Recovery states (Out of

Track and Stuck) were constantly triggered, and therefore were

ﬁrst to be adjusted for improved behavior. Eventually their settings

were such that the Racing behaviors could be focused on. These

too were manually and arbitrarily set through trial and error, consid-

ering the driver’s performance (time and damage) and skill (visual

analysis of tests).

After several iterations, this model performed better than some of

the less capable robots available in TORCS, showcasing the FSM-

Driver’s potential for controlling a car.

3.2 FSMDriver5

Initial tests showed that transitioning from Straight Line to

Curve was too swift, hampering performance because the car

consistently entered curves at such high speeds that the controller

was unable to turn and exited the track, so the additional behavior

Approaching Curve was implemented.

Approaching Curve positions the driver towards the outside

of the incoming curve and tries to race at a speed proportional to

the curvature, so less braking would be necessary in Curve. Thus,

a 5-state FSMDriver model (FSMDriver5, illustrated in Figure 2a)

was ready for testing. The range sensors were arbitrarily conﬁgured

at every 10 degrees from −90 to 90 relative to the car’s front axis

(that is, a uniform distribution from the left to the right of the car),

as shown in Figure 2b.

Sensor readings provide an the controller’s perception of the car’s

position of location in the track and, since Straight Line aims

to drive parallel to the central axis of the track, the changes in how

the range ﬁnders’ input are spread out around their mean value can

be used to switch between states because the difference between

frontal and lateral sensors is larger when driving in a straight line

than in a curve. This variance of the readings is computed as fol-

lows:

variance =

i=1

[(RF

− µ)

] (1)

where RF

is the input from the i-th range ﬁnder and µ is the mean

value of all such readings. The resulting value is considered in the

transition function to deﬁne the current state of the Racing behav-

ior, as described in Algorithm 2.

After several iterations adjusting the conﬁgurations, this model also

had an acceptable performance. However, testing also revealed that

SBC – Proceedings of SBGames 2015 | ISSN: 2179-2259

Computing Track – Full Papers

XIV SBGames – Teresina – PI – Brazil, November 11th - 13th, 2015

(a) Sensor input in Curve. (b) Angle between car and track axis in Out of Track.

Figure 1: Range ﬁnders readings for FSMDriver model.

(a) States and transitions. (b) Range ﬁnders conﬁguration.

Figure 2: FSMDriver5 model.

Algorithm 2 FSMDriver5 Transition

if car is stuck then

state ⇐ Stuck

else

if variance is in Straight Line thresholds and the cur-

rent state is not Straight Line then

state ⇐ Straight Line

else if variance is not in Approaching Curve thresholds

and current state is not Curve then

state ⇐ Approaching Curve

else if variance is in Approaching Curve thresholds

and current state is not Approaching Curve then

state ⇐ Approaching Curve

else if variance > 0 then

state ⇐ Curve

else

state ⇐ Out of Track

end if

the transitions between states required were the bottleneck for im-

proved racing. Not only some of the triggers needed a more detailed

analysis to avoid eventual erratic behavior, but the sheer number

parameters considered for transitioning, each with its triggers (see

Figure 2a), and their impact on the model’s behavior during a com-

plete race caused the whole model to be thought over.

Analyzing FSMDriver5’s transitions occurrences within a race, it

was clear that the Recovery behavior had two distinct situa-

tions, which were acceptably handled by the current conﬁgura-

tion, and that the major issue was frequent triggering between the

Racing states, which inevitably resulted in the car leaving the

track and jeopardizing its performance.

3.3 FSMDriver3

To reduce the model’s complexity, and considering the intuitive

division of behaviors proposed, a new model where the deﬁned

Recovery behaviors were maintained and the Racing behaviors

were joined into a new state called Inside Track. Figure 3a

illustrates this FSMDriver3 model.

Recovery, though essential for championship disputes, rarely oc-

curred after FSMDriver5 the accepted initial conﬁguration (through

trial and error). The new Racing state implements a very straight-

forward idea, accelerate to an arbitrary target speed towards the

largest open space inside track limits, including both the straight

and the curved portions of the track. The target speed is propor-

tional to the largest sensor reading, and braking is activated if the

current speed is greater than the target speed. Gear changing works

based on RPM threshold as in FSMDriver5.

In order to have more detailed information on track segment straight

ahead, which will guide the movement, more range ﬁnders are

placed in front of the car in a uniform distribution (as shown in

Figure 3b), contrasting with FSMDriver5’s equidistant placement.

This different approach to driving lead to a reimplementation of the

transition function, as shown in Algorithm 3:

The new transition dismisses the computation of the variance and

is clearly more intuitive. Initially, conﬁgurations resembling FSM-

Driver5’s parameters were set and manually adjusted until a behav-

ior similar to the original driver was achieved. As expected, this

SBC – Proceedings of SBGames 2015 | ISSN: 2179-2259

Computing Track – Full Papers

XIV SBGames – Teresina – PI – Brazil, November 11th - 13th, 2015

(a) States and transitions. (b) Range ﬁnders conﬁguration.

Figure 3: FSMDriver3 model.

Algorithm 3 FSMDriver3 Transition

if car is stuck then

state ⇐ Stuck

else

if car is within track limits then

state ⇐ Inside Track

else

state ⇐ Out of Track

end if

process was quicker than for FSMDriver5 and soon a conﬁguration

better than some robots was available.

Considering both FSMDriver3 and FSMDriver5 models as accept-

able drivers, the next step is to deﬁne proper conﬁgurations as to

have them behave acceptably as racing controllers, maximizing

their performance in the TORCS/SCR environment.

3.4 Ofﬂine Learning: Evolving Parameters

To search for an optimal solution, a genetic algorithm was selected

between possible techniques not only due to its successful applica-

tion in similar (and very different) problems, but also because of

its more general nature: only a representation of a solution and a

means to measuring its quality must be speciﬁcally deﬁned for the

task. Considering the evolution of the drivers’ parameters, a solu-

tion (or an individual) was represented by a string of bits, which

compose the ﬂoating point parameters of the FSMDrivers’ models.

FSMDriver5 has, overall, 22 parameters that need to be speciﬁed

in the transition function and states. The function considers 4

threshold values deﬁning 2 intervals, one for identifying the vari-

ance levels for Straight Line and another for Approaching

Curve. Straight Line also considers 4 threshold values, one

for a minimal racing gear, and 3 for identifying RPM values that

indicate when to increase/decrease gear.

Approaching Curve has 3 parameters: a desired position

to enter the upcoming curve (in relation to the track axis), a

lower speed threshold, and a maximum steering value. Out of

Track considers 7 parameters: two angle thresholds deﬁning an

acceptable angle interval for returning to the track, 3 speed thresh-

olds for changing gears, a maximum breaking threshold to avoid

skidding, and decelerating intensity to reduce the speed before en-

tering the curve.

Stuck has 4 parameters, one for deﬁning a starting distance be-

fore considering the car stuck, one for tracking the current speed to

determine if it is still stuck, and two for deﬁning game ticks thresh-

olds, establishing an amount of time to consider before triggering a

transition to this state.

FSMDriver3, on the other hand, has the same 11 for Out of

Track and Stuck as FSMDriver5, and 6 additional for Inside

Track. There is one for a minimal racing gear, 3 for identify-

ing RPM values to increase/decrease gear, one for deﬁning a ratio

between car speed and range ﬁnder reading for setting the target

speed, and a base speed value to control brake and acceleration ac-

tuators.

The solution is then modeled as a string of enough bits to represent

each ﬂoating point value for as many parameters as necessary, ac-

cording to the model. The GA also requires a ﬁtness function to

evaluate the solutions, the same for either model. There are several

possibilities (damage taken, similarity to a human driver, etc.), but

his work will focus on the standard metric for SCR: the distance

raced during a ﬁxed time period.

3.5 Online Learning: Staying in Track

An online learning module was included in FSMDriver3 trying to

improve its performance, in an attempt to reduce damage and time

loss for racing out of the track. Whenever the driver enters Out

of Track, the speed and location of this occurrence are recorded

and used in the Racing state. If the speed is over a threshold, the

model assumes the driver has left the track due to excessive speed

and uses this information to slow down the driver in subsequent laps

when approaching the recorded places, trying to remain inside the

track.

This behavior does not interfere with the breaking actions in the

model, but does affect the driver’s performance and, thus, with

the ofﬂine learning process. Though any controller can also learn

where it leaves the track (if so), the differences in implementation

of the Racing states in the FSMDrivers considered here are such

that slowing down before the recorded places implies on excessive

additional complexity in FSMDriver5 and, therefore, this learning

module was not implemented for this controller.

SBC – Proceedings of SBGames 2015 | ISSN: 2179-2259

Computing Track – Full Papers

XIV SBGames – Teresina – PI – Brazil, November 11th - 13th, 2015

4 Experimental Results

The drivers were developed in C++

and tested in TORCS

(v1.3.6)

and SCR

(v2.1) with the standard car model for SCR: car1-

trb1 [Loiacono et al. 2013]. The experiments’ goals are twofold:

a) compare the performances between the two proposed ﬁnite-state

machine models; and b) compare their performance to the current

state-of-the-art AI controller and to a standard TORCS robot.

The FSMDrivers’ parameters are ﬁne tuned through evolution via

genetic algorithm, aiming to improve their racing performances by

changing their values. The GA used standard conﬁgurations from

literature, the population size was arbitrarily set as 30 [Grefen-

stette 1986], which leads to 616 generations for a moderate prob-

lem complexity in which the optimal population size was coded as

bitstrings [Rylander 2002]. Selection is done through elitism by

keeping the best 4 individuals from a generation unchanged, and

the other 26 are generated from the best 10 individuals’ reproduc-

tion. This step is done by randomly choosing a pair from the top

10 that are cloned with a 5% chance or generate a pair of offspring

with a 95% probability [Grefenstette 1986]. These offspring are

produced through crossover by partitioning the parents’ bitstrings

at a random position and swapping them. Finally, mutation (a sim-

ple bit ﬂip) is applied to every bit in every individual of these 26

with a 1% probability [Grefenstette 1986].

The ﬁtness of each individual is computed as the distance covered

in the race. The GA communicates with the test platform, simulat-

ing races with the drivers in the desired tracks in TORCS, through

operating system calls to the command line interface and the ﬁtness

communicated through shared memory

The two FSMDriver models had their parameters evolved through

the GA and compared to each other. This considers only the

Racing states, as Out of Track and Stuck are considered

adequate with the manual settings. During this evolution, the learn-

ing module in FSMDriver3 is disabled in order to not interfere with

the learning, and no parameter of this module was included model’s

bitstring representation. To analyze the learning module’s effect,

two instances of the 3-state FSM controller were tested, the FS-

MDriver3+ with it enabled, and disabled in FSMDriver3. When

enabled, the learning module’s threshold speed is arbitrarily set to

85 km/h. All controllers have 5 laps for warm-up, during which

they can adapt to the track, in accordance to the stipulated 2015

Championship rules

Since it is unknown which tracks will be used during the SCR

Championship, a more general and versatile controller is desirable

and, thus, the controllers were evolved considering their perfor-

mance in several tracks types. Three basic driver proﬁles were de-

sired: a road driver, which should perform well in road and oval

tracks, a dirt driver for dirt roads and a mixed driver, which should

perform well in both types.

For the road driver proﬁle, four road tracks were selected from the

standard TORCS distribution list to compose the training set in the

evolution process. The tracks were arbitrarily selected for certain

features, which might lead to better drivers: Spring, the longest

track available; Wheel 2, with many sharp curves; E-Track 3, a fast

track with sharp curves; and Forza, a fast circuit with smoother

curves.

Analogously, four dirt tracks were selected for evolving a dirt pro-

ﬁle driver: Dirt 2, Dirt 6, Mixed 1, and Mixed 2, which present

variations of high speed and curve styles. For the mixed driver, two

of each were selected: Spring, Wheel 2, Dirt 2, and Mixed 1.

The best solutions evolved for each proﬁle where then evaluated in a

set of six tracks, different from the ones used before, which present

Code available at https://github.com/bruno147/

fsmdriver

http://sourceforge.net/projects/torcs/

http://sourceforge.net/projects/cig/files/SCR\

%20Championship/Server\%20Linux/

Code available at https://github.com/bruno147/

driver-ga

http://cs.adelaide.edu.au/

optlog/SCR2015/

distinct typed, speeds, and curve features for better analysis. These

tracks, shown in Figure 4 and available on the standard TORCS set,

were used in previous SCR Championships [Onieva et al. 2009].

They are two road tracks: Street 1 and CG Speedway 1; one oval

track: D-Speedway; and three dirt tracks: Dirt 1, Dirt 3, and Dirt 4.

4.1 Single Race Results

Table 1 presents the distance covered results for these tests for each

FSMDriver model and proﬁle (best results per track in bold). The

ﬁtness function in this case is the sum of distances covered in all

tracks, considering 10.000 game ticks per track, as per the SCR

rules, so higher values are desirable. The table also includes, for

direct comparison with the current state of the art, results for AU-

TOPIA in three of the same tracks for the same 10.000 game ticks

metric, as presented in [Onieva et al. 2009]. The ‘*’ symbol indi-

cates that the driver does not have data available for the track.

Considering the 5-states FSMDriver in road/oval tracks, it is clear

that the road driver and mixed driver have better results than dirt

driver, since this proﬁle uses road tracks for evaluating individuals

during evolution. Looking at the results for dirt tracks, all proﬁles

have similar performances except for the dirt driver in Dirt 1. FS-

MDriver5’s road driver and mixed driver are practically equivalent

and at least as good as its dirt driver, and all proﬁles are quite infe-

rior to the other controllers shown in Table 1.

Looking at FSMDriver3 in road/oval tracks, it is also clear that the

road driver and mixed driver have better results than dirt driver,

as was expected. Considering the results for dirt tracks, mixed

driver’s performance was signiﬁcantly better than the others in Dirt

1 and Dirt 3, but road driver produced the best result in Dirt 4. Vi-

sual analysis of showed that the faster, more aggressive road driver

would leave the track often in Dirt 4’s sharper, shorter curves (see

Figure 4f), which lead to it racing a longer distance. Road driver’s

result for Street 1 was also the best all around.

FSMDriver3+ proﬁles (with learning module enabled) also per-

formed as expected in road/oval tracks. For these, FSMDriver3+

had shorter distances than FSMDriver3 for Street 1 (5%) and D-

speedway (18.8%), and a slightly longer for CG Speedway num-

ber 1 (0.7%). This is also expected as the driver learns to slow

down in order to stay in the track. For dirt tracks, FSMDriver3+’s

mixed driver had the best results, better than FSMDriver3 for Dirt

1 (13.9%) and Dirt 3 (18.4%), though worse in Dirt 4 (16.9%).

Comparing the results for road/oval tracks with AUTOPIA

([Onieva et al. 2009]), the current state of the art in SCR, FSM-

Driver3’s road driver was about 11.8% better in Street 1 while

FSMDriver3+ was 6.2% better. In CG Speedway number 1, FS-

MDriver3 and FSMDriver3+ were only 2.5% and 1.8% worse, re-

spectively. In D-speedway, these numbers rise to 15.5% and 31.3%.

Thus, the 3-state FSMDriver model showed potential for competing

on road tracks.

In order to learn more about FSMDriver’s endurance, a new set of

tests using the distance raced in 10 laps as metric was executed,

and the results presented in Table 2. The values represent the time

needed to complete the laps, so lower values are desirable, the

‘†’ symbol indicates that the driver does not complete the 10 laps

required, and the ‘*’ symbol indicates that the driver does not have

data available for the track.

Table 2 also includes results for the robot Berniw Hist4, provided in

with the TORCS distribution. This was not included in the previous

tests because TORCS only allows a race with the available robots

to run for a given amount of laps (not game ticks). This robot has

intermediate performance, and was arbitrarily chosen from other

robots to provide insights on the controllers performance compared

to a robot’s and to AUTOPIA, as presented in [Onieva et al. 2012].

Direct comparison is not exactly fair because Berniw Hist4 does not

the standard car car1-trb1, it uses the legacy TZ2.

Looking at results for FSMDriver5, only the road driver completed

10 laps in CG Speedway number 1, all other instances had excessive

damage. The 3-state FSMDriver, on the other hand, completed all

track in at least 2 proﬁles.

SBC – Proceedings of SBGames 2015 | ISSN: 2179-2259

Computing Track – Full Papers

XIV SBGames – Teresina – PI – Brazil, November 11th - 13th, 2015

(a) Street 1 (b) CG Speedway number 1 (c) D-Speedway (d) Dirt 1 (e) Dirt 3 (f) Dirt 4

Figure 4: Test tracks, road tracks (a, b), oval (c), and dirt tracks (d, e, and f) [Wymann et al. 2014].

Table 1: Distance covered (in meters) racing alone for 10.000 game ticks.

Track

Driver Street 1 CG Speedway number 1 D-speedway Dirt 1 Dirt 3 Dirt 4

FSMDriver5

road 3822.76 4114.66 3427.11 2145.49 2205.97 3260.19

dirt 1267.83 2057.47 2936.82 1072.92 2205.82 3260.33

mixed 3822.99 4114.80 3427.06 2145.75 2205.83 3260.31

FSMDriver3

road 7925.60 8745.49 13196.50 3978.01 3451.26 6757.83

dirt 5103.41 5093.82 8589.13 3525.84 4905.58 5590.78

mixed 5864.48 8714.58 11694.60 5052.38 4920.32 3797.83

FSMDriver3+

road 7532.93 8804.73 10720.20 3678.89 5199.46 3193.44

dirt 4738.76 5102.92 8197.55 3525.84 4214.30 5057.18

mixed 6780.43 8089.41 10943.10 5752.90 5824.93 5614.62

AUTOPIA [Onieva et al. 2009] 7091.8 8970.4 15612.3 * * *

Table 2: Time elapsed (in seconds) racing alone for 10 laps.

Track

Driver Street 1 CG Speedway number 1 D-speedway Dirt 1 Dirt 3 Dirt 4

FSMDriver5

road † 816.6 † † † †

dirt † † † † † †

mixed † † † † † †

FSMDriver3

road 1086.28 483.17 607.35 † 1150.09 †

dirt † 1274.81 840.02 597.33 1045.62 1307.52

mixed 1096.95 510.86 612.00 408.57 811.96 2157.95

FSMDriver3+

road 1225.24 484.58 709.74 † 1108.48 †

dirt 1658.47 1291.31 915.86 597.33 1055.21 1360.71

mixed 1208.84 534.87 646.68 384.93 939.93 1930.19

Berniw Hist4 1143.77 605.76 656.24 460.95 872.97 1127.45

AUTOPIA [Onieva et al. 2012] * * * 339.3 742.4 796.5

Considering FSMDriver3 in road/oval tracks, it is also clear that the

road driver and mixed driver have better results than dirt driver, as

expected. Considering the results for dirt tracks, mixed driver’s per-

formance was signiﬁcantly better than the others in Dirt 1 and Dirt

3, while dirt driver was almost twice as fast in Dirt 4. Due to the

track’s shape, the mixed driver’s more aggressive style makes it fre-

quently leave the track, which leads to the slower Recovery con-

ditions, causing this result.

FSMDriver3+ proﬁles also performed as expected in road/oval

tracks. For these, FSMDriver3+ required a longer time than FS-

MDriver3 for Street 1 (12.8%) and D-speedway (0.3%), but it did

ﬁnish CG Speedway number 1 while FSMDriver3 did not. This is

also expected as the driver learns to avoid leaving the track and,

thus, receive damage. For dirt tracks, FSMDriver3+’s mixed driver

had better than FSMDriver3 for Dirt 1 (5.8%), but worse in Dirt 3

(15.8%) and Dirt 4 (14.1%).

Considering the robot Berniw Hist4, it seems that the 3-state FS-

MDriver is better controller, despite the “handicap” of using SCR’s

interface for driving and not having access to additional information

from the simulator (such as track layout).

Finally, comparing the results for dirt tracks with AUTOPIA

([Onieva et al. 2012]), FSMDriver3’s mixed driver was about

20.4% worse in Dirt 1 while FSMDriver3+ was 13.2%. In Dirt 3,

FSMDriver3 and FSMDriver3+ were only 9.4% and 26.6% worse,

respectively. In Dirt 4, these numbers rise to 170.9% and 142.3%.

4.2 Damages

Table 3 shows the damage results for the 3-state models for analysis

of the learning module. The module produced interesting results in

reducing the damage taken in Street 1, Dirt 1, and Dirt 3 tracks,

suggesting that it indeed has the potential for avoiding it. This is

interesting not only because damage is an important factor for the

pointing system, but also because - in excess - it may lead to the

controller abandoning the race.

Analyzing results for CG Speedway number 1, the learning mod-

ule increases damage in 10.000 game ticks, but decreases it in the

longer 10 lap period. Considering that the difference in racing con-

ditions of FSMDriver3 and FSMDriver3+ (from Tables 1 and 2) is

very small, it may be inferred that the learning module has little

impact on the performance but a signiﬁcant effect on damage for

longer races, also indicating its potential for improving FSMDriver.

However, results for D-speedway and Dirt 4 show that FSM-

Driver3+ takes a lot more damage than FSMDriver3, showing that

its behavior and effects on the controller must be further investi-

gated. Considering Dirt 4, visual analysis showed that though FS-

MDriver3+ leaves the track more often that its counterpart, this ac-

tually leads to it racing a longer distance, which is the ﬁtness func-

tion

SBC – Proceedings of SBGames 2015 | ISSN: 2179-2259

Computing Track – Full Papers

XIV SBGames – Teresina – PI – Brazil, November 11th - 13th, 2015

Table 3: Damage points.

10.000 game ticks 10 Laps

Track Proﬁle FSMDriver3 FSMDriver3+ FSMDriver3 FSMDriver3+

Street 1

road 1629 505 6510 4188

dirt 202 903 8778 8279

mixed 167 80 3051 2202

CG Speedway number 1

road 5 0 5 0

dirt 247 344 5826 1118

mixed 0 0 0 1009

D-speedway

road 2 348 1001 4447

dirt 1 1 1 715

mixed 1 1 1 1

Dirt 1

road 4083 798 10044 10796

dirt 764 764 2677 2677

mixed 3217 37 7042 62

Dirt 3

road 947 412 4050 2870

dirt 48 44 577 545

mixed 585 155 2966 2064

Dirt 4

road 911 7098 1048 10382

dirt 0 63 454 2220

mixed 2407 3507 2542 3678

4.3 Analysis

As expected, the controllers evolved on road/oval tracks were

fastest. dirt tracks provide a more difﬁcult environment, sudden

braking often results in unwanted behavior, skidding is noticeably

more common then. The controllers evolved in this end of the evo-

lution process tends to drive in lower speeds to stay within track

boundaries. Mixed drivers tend to evolve some of the faster, reck-

less behavior for road driver, while keeping some of the carefulness

of a dirt driver, producing intermediate results.

Though the FSMDriver5 conﬁguration proposed seems intuitively

correct, the performance - obviously - dependent which states are

activated during the race, and, thus, how these states were handled

in the evolution process. A more detailed analysis of race logs in-

dicate that the parameters in the transition function, which deﬁne

the current state, are very susceptible to the inputs, resulting in sev-

eral changes in the states which greatly impacts the evolution of

the states’ parameters and, consequently, of those in the function.

By having multiple states acting inside the track the 5-state model

struggles to deﬁne the boundaries of different typed of track seg-

ments, and mismatching between perceived and actual states would

often cause the car to leave the track.

This suggests a different approach to development, such as adjust-

ing one state at a time (independently) and then the transition func-

tion or augmenting the FSM from a successful 3-state conﬁgura-

tion. Additionally, the variance approach is also very sensitive to

the car’s position at the moment, implying that a different, more

robust metric for establishing the type of track segment the car is

in should be pursued. Another possibility, since the search space is

larger than FSMDriver3’s, is to increase the number of individuals

and/or generations in the genetic algorithm’s conﬁgurations.

Considering the 3-state model, it was seen that it quickly achieved

an intermediate level performance in TORCS (compared to FSM-

Driver5 and Berniw Hist4). The less complex ﬁnite state machine

model is more compatible with the tested optimization method,

since it has a 23% smaller search space and with the simpler tran-

sition function, which enables completely independent evolution of

the states.

Considering the 10.000 game ticks results, the learning module

produces slightly slower drivers, but both FSMDriver3 and FSM-

Driver3+ had interesting results in all tracks tested, and they bested

AUTOPIA’s qualifying mark for Street 1. This is very signiﬁcant

because it implies that FSMDriver could qualify to the SCR Cham-

pionship’s racing stage (after the qualifying stage) ahead of the cur-

rent state of the art.

Analyzing the results for 10 laps, FSMDriver5 has obviously poor

driving skills for longer races, taking excessive damage and being

unable to complete any. The 3-state models, on the other hand were

able to ﬁnish the all tracks. FSMDriver3+ ﬁnished Street 1 while

FSMDriver3 did not, which could indicate that the learning module

produces a more careful driver that takes less damage, prolonging

its endurance (a desired feature during the race stage). In these

tests, AUTOPIA was clearly a better driver than the others, despite

not having supremacy in the qualifying.

A visual comparison between FSMDriver and AUTOPIA in a sim-

ulation using TORCS graphical interface revealed that AUTOPIA’s

brake, acceleration and recovery policies are better than the ones

achieved with FSMDriver. AUTOPIA barely leaves and track and

is more stable in curves in dirt tracks.

4.4 SCRC 2015

The evolved FSMDriver3+ was submitted to the Simulated Car

Racing Championship 2015 leg, as part of GECCO 2015, under

the name Need4SS

. It performed surprisingly well for such early

stage of development, coming in second place in a few races and

obtaining 144 points in the Noisy Mode Race Stage - only 4 points

behind the 2nd runner up SnakeOil.

5 Conclusions

Self-driving cars are one of the most interesting promises of tech-

nology, but the process of developing a software controller is com-

plex and expensive. Advanced car racing simulators present a well

deﬁned environment for developing such drivers and comparing Ar-

tiﬁcial intelligence solutions, and The Open Racing Car Simula-

tor/Simulated Car Racing Championship platforms have been suc-

cessfully used to this end.

This work presented two versions of ﬁnite-state machine con-

trollers, an interesting approach that clearly breaks the task into

distinct, independent behaviors (states), facilitating testing differ-

ent conﬁgurations and simplifying the search for better conﬁgura-

tions. This search was executed by applying a genetic algorithm

with default conﬁgurations found in the literature to evolve the con-

trollers’ parameters in the proposed models. The evolved controller

was submitted to the 2015 Simulated Car Racing Championship

and ﬁnished in 4th place.

Experimental results showed that proﬁling the evolution process ac-

cording to the track type has signiﬁcant impact on the performance.

The 5-state model had an overly complex implementation, in which

the transition function hampered the evolution of the states param-

eters. The 3-state controller had a simple, intuitive model which

http://cs.adelaide.edu.au/

optlog/SCR2015/

index.html

SBC – Proceedings of SBGames 2015 | ISSN: 2179-2259

Computing Track – Full Papers

XIV SBGames – Teresina – PI – Brazil, November 11th - 13th, 2015

was performed well in races, surpassing one of AUTOPIA’s marks

which could result in it qualifying ahead of the current state of the

art in SCR Championship’s racing stage.

The learning module potentially leads to better drivers when dam-

age is a concern, with small impact on the performance. However,

its behavior handling sharper curves needs further investigation.

Other ways to improve the controllers endurance are longer periods

in the ﬁtness function and integration of harsher brake policies or

path planning to avoid the car leaving the track. Additionally, a bet-

ter breaking system, such as anti-lock breaking system or traction

control system, could lead to improved performance by reducing

skidding and providing more stable breaking and curve handling.

Considering the ofﬂine learning, different techniques could be in-

vestigated and compared to the GA’s results. The genetic algorithm

could also be improved through a ﬁne tuning of its parameters, and

a less general approach to representing a solution for the model (re-

placing ﬂoating point representations by parameter speciﬁc ones),

or even incorporating fuzzy logic to handle ordering issues with

some parameters could simply solve this ordering issue.

Finally, further development on handling opponents must be pur-

sued as focus must be set on developing skills for the racing stage

of the championship, specially for dealing with opponents.

References

AJZERMAN, M. A., GUSEV, L. A., ROZONOER, L. I.,

SMIRNOVA, I. M., AND TAL, A. A. 1971. Logic automata

and algorithms. Math. Sci. Eng. Academic Press, New York,

NY.

BARTH, M., BORIBOONSOMSIN, K., AND WU, G. 2013. The

potential role of vehicle automation in reducing trafﬁc-related

energy and emissions. In Connected Vehicles and Expo (ICCVE),

2013 International Conference on, 604–605.

BEJCZY, A. K. 2012. New challenges for application of robotics

and automation in space research. 7th IEEE International Sym-

posium on Applied Computational Intelligence and Informatics.

BELZER, J., Ed. 1992. Encyclopedia of computer science and

technology. 25: Supplement 10. Dekker, New York.

BERNADEN, A. 1999. Variable air volume control using a ﬁnite

state machine. In Control Conference (ECC), 1999 European,

31–36.

BHARATHI, R., KUMAR, M. J., SUNITHA, D., AND PRE-

MALATHA, S. 2007. Optimization of combined economic and

emission dispatch problem a comparative study. International

Power Engineering Conference.

BUCKLAND, M. 2005. State-driven agent design. In Programming

Game AI By Example, 2nd ed., W. Publishing, Ed. Wordware,

Plano, TX, ch. 2, 43–45.

BUTZ, M., AND L

ONNEKER, T. 2009. Optimized sensory-motor

couplings plus strategy extensions for the torcs car racing chal-

lenge. In Computational Intelligence and Games, 2009. CIG

2009. IEEE Symposium on, 317–324.

CALDEIRA, C., ARANHA, C., AND RAMOS, G. 2013. Torcs

training interface: An auxiliary api for developing torcs drivers.

XII Simp

osio Brasileiro de Jogos e Entretenimento Digital, 13.

CARBAUGH, J., GODBOLE, D., AND SENGUPTA, R. 1997. Tools

for safety analysis of vehicle automation systems. In Ameri-

can Control Conference, 1997. Proceedings of the 1997, vol. 3,

2041–2045 vol.3.

CARDAMONE, L., LOIACONO, D., AND LANZI, P. L. 2009.

Learning Drivers for TORCS through Imitation Using Super-

vised Methods. Computational Intelligence and Games, 2009.

CIG 2009. IEEE Symposium on.

CARDAMONE, L., LOIACONO, D., AND LANZI, P. 2010. Learn-

ing to drive in the open racing car simulator using online neu-

roevolution. Computational Intelligence and AI in Games, IEEE

Transactions on 2, 3 (Sept), 176–190.

DE FREITAS, L. G., REFFATTI, L. M., DE SOUSA, I. R., CAR-

DOSO, A. C., CASTANHO, C. D., BONIF

ACIO, R., AND

RAMOS, G. N. 2012. Gear2d: An extensible component-based

game engine. In Proceedings of the International Conference on

the Foundations of Digital Games, ACM, New York, NY, USA,

FDG ’12, 81–88.

DUNCAN, J. C., AND FETERLE, L. C. 2000. The use of per-

sonal computer-based aviation training devices to teach aircrew

decision making, teamwork, and resource management. In Pro-

ceedings of the IEEE 2000 National Aerospace and Electronics

Conference, 421–426.

GOLDBERG, D., AND HOLLAND, J. 1988. Genetic algorithms and

machine learning. Machine Learning 3, 2-3, 95–99.

GREFENSTETTE, J. 1986. Optimization of control parameters for

genetic algorithms. Systems, Man and Cybernetics, IEEE Trans-

actions on 16, 1 (Jan), 122–128.

HOLLAND, J. H. 1984. Genetic Algorithms and Adaptation. In

Adaptive Control of Ill-Deﬁned Systems, vol. 16 of NATO Con-

ference Series, 317–333.

HYUNKIM, D., KWON, J., CHEN, K., AND CHOI, K. 2013. Fi-

nite state machine for vehicle detection in highway surveillance

systems. In Frontiers of Computer Vision, (FCV), 2013 19th

Korea-Japan Joint Workshop on, 84–87.

LEVY, D., CHALOUPKA, F., GITCHELL, J., MENDEZ, D., AND

WARNER, K. 2002. The use of simulation models for the

surveillance, justiﬁcation and understanding of tobacco control

policies. Health Care Management Science 5, 2, 113–120.

LOIACONO, D., LANZI, P. L., TOGELIUS, J., ONIEVA, E.,

PELTA, D. A., BUTZ, M. V., L

ONNEKER, T. D., CARDA-

MONE, L., PEREZ, D., S

AEZ, Y., PREUSS, M., AND QUAD-

FLIEG, J. 2010. The 2009 Simulated Car Racing Champi-

onship. IEEE Transactions on Computational Intelligence and

AI in Games 2 (June).

LOIACONO, D., CARDAMONE, L., AND LANZI, P. L. 2013. Sim-

ulated Car Racing Championship: Competition Software Man-

ual. ArXiv e-prints (Apr.).

LOIACONO, D. 2012. Learning, evolution and adaptation in rac-

ing games. In Proceedings of the 9th Conference on Computing

Frontiers, ACM, New York, NY, USA, CF ’12, 277–284.

LUCAS, S. M. 2008. Computational intelligence and games: Chal-

lenges and opportunities. International Journal of Automation

and Computing 5, 1, 45–57.

LUETTEL, T., HIMMELSBACH, M., AND WUENSCHE, H.-J.

2012. Autonomous ground vehicles – concepts and a path to the

future. Proceedings of the IEEE 100, Special Centennial Issue

(May), 1831–1839.

MILLINGTON, I., AND FUNGE, J. 2006. Decision making. In

Artiﬁcial Intelligence for Games, 2nd ed., Elsevier, Ed. Morgan

Kaufmann Publishers, San Francisco, CA, ch. 5, 309–311. Sec-

tion 3.

MITCHELL, T. M. 1997. Machine Learning. McGraw-Hill.

MUOZ, J., GUTIERREZ, G., AND SANCHIS, A. 2010. A human-

like torcs controller for the simulated car racing championship.

In Computational Intelligence and Games (CIG), 2010 IEEE

Symposium on, 473–480.

NALLAPERUMA, S., NEUMANN, F., BONYADI, M. R., AND

MICHALEWICZ, Z. 2014. Evor: An online evolutionary al-

gorithm for car racing games. In Proceedings of the 2014 Con-

ference on Genetic and Evolutionary Computation, ACM, New

York, NY, USA, GECCO ’14, 317–324.

ONIEVA, E., PELTA, D., ALONSO, J., MILANES, V., AND PEREZ,

J. 2009. A modular parametric architecture for the torcs racing

SBC – Proceedings of SBGames 2015 | ISSN: 2179-2259

Computing Track – Full Papers

XIV SBGames – Teresina – PI – Brazil, November 11th - 13th, 2015

engine. In Computational Intelligence and Games, 2009. CIG

2009. IEEE Symposium on, 256–262.

ONIEVA, E., PELTA, D., GODOY, J., MILAN

ES, V., AND

RASTELLI., J. P. 2012. An Evolutionary Tuned Driving System

for Virtual Car Racing Games: The AUTOPIA Driver. Inter-

national Journal of Intelligent Systems, Wiley-Blackwell 27, 3,

217–241.

PEDRYCZ, W., AND SUCCI, G. 2005. Genetic granular classiﬁers

in modeling software quality. Journal of Systems and Software

76, 3 (June), 277–285.

PEREZ, D., RECIO, G., SAEZ, Y., AND ISASI, P. 2009. Evolving

a fuzzy controller for a car racing competition. Computational

Intelligence and Games (CIG).

QUADFLIEG, J., PREUSS, M., AND RUDOLPH, G. 2014. Driv-

ing as a human: a track learning based adaptable architecture

for a car racing controller. Genetic Programming and Evolvable

Machines 15, 4, 433–476.

RYLANDER, S. G. B. 2002. Optimal population size and the ge-

netic algorithm. Population 100, 400, 900.

SHOTTON, J., FITZGIBBON, A., COOK, M., SHARP, T., FINOC-

CHIO, M., MOORE, R., KIPMAN, A., AND BLAKE, A. 2011.

Real-time human pose recognition in parts from single depth im-

ages. In Proceedings of the 2011 IEEE Conference on Computer

Vision and Pattern Recognition, IEEE Computer Society, Wash-

ington, DC, USA, CVPR ’11, 1297–1304.

STANLEY, K. O., BRYANT, B. D., AND MIIKKULAINEN, R.

2005. Real-time neuroevolution in the NERO video game. IEEE

Transactions on Evolutionary Computation 9, 6, 653–668.

TERZIC, I., ZOITL, A., AND FAVRE, B. 2008. A survey of dis-

tributed intelligence in automation in european industry, research

and market. IEEE International Conference on Emerging Tech-

nologies and Factory Automation.

THRUN, S., MONTEMERLO, M., DAHLKAMP, H., STAVENS, D.,

ARON, A., DIEBEL, J., FONG, P., GALE, J., HALPENNY, M.,

HOFFMANN, G., LAU, K., OAKLEY, C., PALATUCCI, M.,

PRATT, V., STANG, P., STROHBAND, S., DUPONT, C., JEN-

DROSSEK, L.-E., KOELEN, C., MARKEY, C., RUMMEL, C.,

VAN NIEKERK, J., JENSEN, E., ALESSANDRINI, P., BRAD-

SKI, G., DAVIES, B., ETTINGER, S., KAEHLER, A., NEFIAN,

A., AND MAHONEY, P. 2006. Stanley: The robot that won the

darpa grand challenge: Research articles. J. Robot. Syst. 23, 9

(Sept.), 661–692.

WANG, Q., SPRONCK, P., AND TRACHT, R. 2003. An overview

of genetic algorithms applied to control engineering problems.

In International Conference on Machine Learning and Cyber-

netics, vol. 3, 1651–1656.

WANG, S., HONG, J., LIU, Z., WANG, W., AND ZHANG, E.

2009. Method of human-imitating posture generation for me-

chanical hand prosthesis. In Reconﬁgurable Mechanisms and

Robots, 2009. ReMAR 2009. ASME/IFToMM International Con-

ference on, 428–433.

WHITTON, N., AND MOSELEY, A., Eds. 2012. Using games to

enhance learning and teaching: a beginner’s guide. Routledge,

New York.

WYMANN, B., ESPI

E, E., GUIONNEAU, C., DIMITRAKAKIS, C.,

COULOM, R., AND SUMNER, A., 2014. TORCS, The Open

Racing Car Simulator. http://www.torcs.org.

XUE, H., WU, H., MENG, X., AND LIU, W. 2011. Simulation

research and programming of torque ﬂuctuation of automotive

steering system. In Second International Conference on Me-

chanic Automation and Control Engineering, 4385–4387.

ZHANG, F., DU, Z., SUN, L., AND JIA, Z. 2007. A new novel

virtual simulation system for robot-assisted orthopedic surgery.

IEEE International Conference on Robotics and Biomimetics.

SBC – Proceedings of SBGames 2015 | ISSN: 2179-2259

Computing Track – Full Papers

XIV SBGames – Teresina – PI – Brazil, November 11th - 13th, 2015