Charles A. G. Madeira

e-mail: charlesandrye at gmail.com
webpage: http://charles.madeira.free.fr

Welcome to my research homepage. Here you will find information about my academic research and links to papers that I have written and presentations I have given.

Research Interests

My research interests focus on the design of autonomous agents for complex real-world problems such as modern computer games and digital television. In this context, I am interested in some large domains:

Machine learning (specially reinforcement learning)
Decision-making under uncertainty
Abstraction
Multiagent systems.

PhD Work

The aim of my PhD research was to investigate the challenges posed by the application of reinforcement learning to modern strategy games. During the course of my research I developed STRADA, a novel integrated learning approach to the automatic design of adaptive behavioral strategies for modern strategy games. STRADA combines state-of-the-art techniques from several areas of machine learning with new ideas. In particular, it investigates two main issues: (1) complexity reduction of the problem by decomposing the decision-making and abstracting state and action spaces; and (2) acceleration of the process of learning from interaction by generalizing the value function and bootstrapping the acquisition of experience. Solutions to these issues are combined into an efficient learning system, whose performance is demonstrated on the task of learning valuable behavioral strategies for a commercial wargame called Battleground.

Supervisors:
Assistant Professor: Vincent Corruble
Full Professor: Jean-Gabriel Ganascia

Group of Cognitive Agents and Automated Symbolic Learning (ACASA)
Laboratory of Computer Science (LIP6)
University of Paris 6 (Pierre et Marie Curie)

Post-doctoral Work

My post-doctoral research consisted in studying machine learning techniques in order to design an innovative recommendation system for the French Terrestrial Digital TV. My work was part of a project of research and innovation in audiovisual and multimedia, called BUIS (Simple and Intelligent Set-top Box), that had as partners the LIP6 Laboratory, the LIMSI Laboratory and the IWEDIA Company.

Background

Before coming to Paris, I developed a game framework called FORGE V8 as my master thesis at the Center of Informatics (CIn) of the Federal University of Pernambuco (UFPE) - Recife / Brazil. FORGE V8 permitted to start a study of Artificial Intelligence and computer games that gave me interesting directions to my research work.

Adaptive Agents for Modern Strategy Games

An important goal of artificial intelligence in the field of games is the design of artificial opponents which propose real challenges to human players. In this context, some domains such as machine learning have already obtained excellent results when applied to classical games in the last years. Despite this, when we face to real world simulations such as modern strategy games, which can be seen as modern extensions of classical games, state-of-the-art techniques of artificial intelligence cannot be efficiently applied because of the orders of magnitude more complex than those found within the framework of classical games. These modern games require the players to manage a high number of units placed in a very sophisticated environment so as to achieve collectively a common goal. Designing interesting solutions for such applications need to tackle simultaneously several challenging issues (resource management, decision-making under uncertainty, spatial and temporal reasoning, opponent modeling, coordination between units, etc.), each of which can represent an important research problem in itself.

Recommendation Systems for Digital Television

Digital Television (DTV) is a new broadcasting technology that offers television with movie-quality picture and sound. DTV uses digital modulation data and requires a specially designed television set to be decoded. For instance, a standard receiver with a Set-top Box (STB), or a PC fitted with a television card. DTV STBs offer multiple programming choices and interactive capabilities which allow recording several hours of DTV programs. This make the task of selecting a program very involved and time consuming. Therefore, we proposed through a project called BUIS (Simple and Intelligent STB), funded by the network for research and innovation in audiovisual and multimedia (RIAM), the investigation of machine learning and graphical user interface techniques in order to design an innovative real-time recommendation system that enables DTV viewers to, quickly and in a user-friendly manner, select DTV programs.

Doctoral Thesis

Madeira, C.: (2007) Agents adaptatifs dans les jeux de stratégie modernes : une approche fondée sur l'apprentissage par renforcement, Ph.D. Thesis, LIP6, Université Pierre et Marie Curie (Paris 6), Paris, France.

Madeira, C.: (2007) Adaptive Agents for Modern Strategy Games: an Approach Based on Reinforcement Learning, Extended Summary in English of Ph.D. Thesis.

Thesis Proposal

Madeira, C.: (2004) Techniques d'Intelligence Artificielle pour la conception d'un adversaire automatique dans des jeux de stratégie de type wargame : Le Projet Napolectronic.

Papers

Madeira, C., and Corruble V.: (2009) STRADA : une approche adaptative pour les jeux de stratégie modernes. Revue d'Intelligence Artificielle, 23(2-3):293-326.

Ganascia J.-G., Madeira, C., and Fouladi K.: (2008) An Adaptive Cartography of DTV Programs. Changing Television Environments, Lecture Notes in Computer Science, 5066:253-262.

Madeira, C., Corruble V., and Ramalho G.: (2006) Designing a Reinforcement Learning-based Adaptive AI for Large-Scale Strategy Games. In Proceedings of the Second Conference on Artificial Intelligence and Interactive Digital Entertainment, Marina del Rey, CA, USA.

Madeira, C., Corruble V., and Ramalho G.: (2005) Generating Adequate Representations for Learning from Interaction in Complex Multiagent Simulations. In Proceedings of the IEEE/WIC/ACM International Joint Conference on Intelligent Agent Technology, Compiègne, France.

Madeira, C., Corruble V., Ramalho G., and Ratitch B.: (2004) Bootstrapping the Learning Process for the Semi-automated Design of a Challenging Game AI. In Proceedings of the AAAI Workshop on Challenges in Game AI, San Jose, CA, USA.

Corruble V., Madeira, C., and Ramalho G.: (2002) Steps Toward Building a Good AI for Complex Wargame-Type Simulation Games. In Proceedings of the 3rd International Conference on Intelligent Games and Simulation, London, UK.

Madeira, C., Vieira, M., Menezes, T., Silva, D., Ramalho, G., and Ferraz, C.: (2001) FORGE V8: Um Framework para Jogos de Computador e Aplicações Multimídia. In Proceedings of the Seventh Brazilian Symposium on Multimedia and Hypermedia Systems, Florianópolis, SC, Brazil.

Madeira, C., Araujo, A., Macedo, H., Andrade, R., Cavalcanti, D., Ferraz, C., and Ramalho, G.: (2000) NetMaze: Um Jogo de Ação Multimídia Distribuído. In Proceedings of the Sixth Brazilian Symposium on Multimedia and Hypermedia Systems, Natal, RN, Brazil.

Macedo, H., Araujo, A., Cavalcanti, D., Andrade, R., Madeira, C., and Ferraz, C.: (2000) Evaluating Multi User Distributed Action Games Architectures on a CORBA Platform. In Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications, Las Vegas, NV, USA.

Dissertations

Madeira, C.: (2001) Forge V8: Um framework para o desenvolvimento de jogos de computador e aplicações multimídia. Master Thesis, CIn, Universidade Federal de Pernambuco (UFPE), Recife, PE, Brazil.

Madeira, C.: (1998) Concepção e Implementação da Plataforma Básica de Comunicação do Sistema SAGRI. Undergraduate report, DIMAp, Universidade Federal do Rio Grande do Norte (UFRN), Natal, RN, Brazil.

Agents adaptatifs dans les jeux de stratégie modernes : une approche fondée sur l'apprentissage par renforcement
Thesis Presentation at the LIP6.
The presentation in French of my Ph.D. thesis. The presentation outlines the STRADA approach. STRADA is a novel integrated learning approach to the automatic design of adaptive behavioral strategies for modern strategy games. The full thesis document is available in pdf.

Adaptive Agents for Modern Strategy Games: an Approach Based on Reinforcement Learning
Thesis Presentation at the LIP6.
The presentation in English of my Ph.D. thesis.

Generating Adequate Representations for Learning from Interaction in Complex Multiagent Simulations
A Paper Presentation at IAT 2005.
This is the presentation of my paper by the same title at IAT 2005. It presents same ideas of an abstraction method proposed to reduce decision-making complexity in modern strategy games.

Bootstrapping the Learning Process for the Semi-automated Design of a Challenging Game AI
A Paper Presentation at AAAI-04 Workshop on Game AI.
This is a presentation I gave at the AAAI-04 Workshop on Challenges in Game AI held in San Jose, California, USA. It presents a bootstrap mechanism proposed for learning by levels of a hierarchical decision-making.

Techniques d'Intelligence Artificielle pour la conception d'un adversaire automatique dans des jeux de stratégie de type wargame : Le Projet Napolectronic
Thesis Proposal Presentation at the LIP6.
The presentation of my Ph.D. thesis proposal. The presentation outlines a course of research to develop an approach to the design of adaptive strategies for modern strategy games.

Apprentissage par renforcement
A general presentation about Reinforcement Learning given to the ACASA group.

Systèmes de recommandation pour la TV numérique
A general presentation about Digital Television Recommender Systems given to the ACASA group.

Outils de traitements des données
First semester 2005/2006
MIAGE of University of Paris 12 (Val-de-Marne)

Programmation orientée objet avec Java
Second semester 2005/2006
MIAGE of University of Paris 12 (Val-de-Marne)