Wouter CAARLS

Wouter Caarls obtained a Master's degree in Artificial Intelligence from the University of Amsterdam in 2002, and continued to do a PhD at the Delft University of Technology, on the subject of intelligent cameras, completed in 2008. After postdoctoral fellowship positions working on microscope automation and robotics, he is now an assistant professor working on reinforcement learning for robotics at the Pontifical Pontifical Catholic University of Rio de Janeiro.

Informações coletadas do Lattes em 21/05/2022

Acadêmico

Formação acadêmica

Doutorado em Applied Sciences

2002 - 2008

Delft University of Technology
Título: Automated Design of Application-Specific Smart Camera Architectures
Orientador: Pieter Jonker

Mestrado em Artificial Intelligence

1997 - 2002

Universiteit van Amsterdam, UvA
Título: Genetic Algorithm Visualisation,Ano de Obtenção: 2002
Orientador: Peter Sloot
Coorientador: Jaap Kaandorp.

Formação complementar

2005 - 2005

Extensão universitária em One-month internship (Shorin Kyo). (Carga horária: 160h). , NEC Corporation, NEC, Japão.

2002 - 2002

Extensão universitária em One-month internship (Richard Kleihorst). (Carga horária: 160h). , Philips Research Eindhoven, PRE, Holanda.

Idiomas

Inglês

Compreende Bem, Fala Bem, Lê Bem, Escreve Bem.

Português

Compreende Razoavelmente, Fala Razoavelmente, Lê Bem, Escreve Razoavelmente.

Alemão

Compreende Razoavelmente, Fala Razoavelmente, Lê Razoavelmente, Escreve Pouco.

Holandês

Compreende Bem, Fala Bem, Lê Bem, Escreve Bem.

Áreas de atuação

Grande área: Outros / Área: Robótica, Mecatrônica e Automação.

Grande área: Ciências Exatas e da Terra / Área: Ciência da Computação / Subárea: Sistemas de Computação/Especialidade: Arquitetura de Sistemas de Computação.

Participação em bancas

Aluno: Rafael Fontanella Katopodis

Caarls, W.; LIMA, P. M. V.; FIGUEIREDO, D. R.Daniel Ratton Figueiredo. REINFORCEMENT LEARNING WITH WEIGHTLESS NEURAL NETWORKS. 2022. Dissertação (Mestrado em Engenharia de Sistemas e Computação) - Universidade Federal do Rio de Janeiro.

Orientou

Luis Carlos Parra Camacho

Reinforcement learning for a bipedal robot; Início: 2022; Dissertação (Mestrado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro; (Orientador);

Igor Ferreira da Costa

VIsual navigation for an agricultural robot; Início: 2021; Dissertação (Mestrado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro; (Orientador);

Matheus Nascimento

Underwater docking using visual navigation; Início: 2019; Dissertação (Mestrado profissional em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro; (Orientador);

Franklin Cardeoso

Reinforcement learning for human-machine interaction; Início: 2022; Tese (Doutorado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro; (Orientador);

Gabriel Baruque

Machine learning for medical imaging; Início: 2020; Tese (Doutorado em Doutorado em Engenharia Elétrica - Pontifícia Universidade Católica, RJ) - Pontifícia Universidade Católica do Rio de Janeiro; (Orientador);

Gabriel Lins Tenório

Vision-based agricultural robotics; Início: 2019; Tese (Doutorado em Doutorado em Engenharia Elétrica - Pontifícia Universidade Católica, RJ) - Pontifícia Universidade Católica do Rio de Janeiro; (Orientador);

Evelyn Conceição Santos Batista

Underware petroleum line inspection; Início: 2019; Tese (Doutorado em Doutorado em Engenharia Elétrica - Pontifícia Universidade Católica, RJ) - Pontifícia Universidade Católica do Rio de Janeiro; (Orientador);

Adalberto Oliveira

Combining robust control and reinforcement learning; Início: 2019; Tese (Doutorado em Doutorado em Engenharia Elétrica - Pontifícia Universidade Católica, RJ) - Pontifícia Universidade Católica do Rio de Janeiro; (Orientador);

Guilherme Eduardo

Deep Reinforcement Learning for Quadrotor Trajectory Control in Virtual Environments; 2021; Dissertação (Mestrado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro,; Orientador: Wouter Caarls;

Pedro Chataignier

Assistive Guidance System for the Visually Impaired Using Object Detection; 2020; Dissertação (Mestrado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro,; Orientador: Wouter Caarls;

Franklin Cardeñoso Fernandez

Deep Reinforcement Learning for Haptic Shared Control in Unknown Tasks; 2020; Dissertação (Mestrado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro,; Orientador: Wouter Caarls;

Felipe Ferraz Martins

Navigation and pest identification systems for the Soybot agricultural robot; 2020; Dissertação (Mestrado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro,; Coorientador: Wouter Caarls;

Guilherme Neri de Souza

Sliding mode control for single- and multi-legged robots; 2020; Dissertação (Mestrado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro,; Coorientador: Wouter Caarls;

Yessica Rosas Cuevas

Modeling and control of a quadcopter for autonomous navigation in agricultural fields; 2020; Dissertação (Mestrado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro,; Coorientador: Wouter Caarls;

Evelyn Conceição Santos Batista

Um estudo de transfer learning em deep reinforcement learning em ambientes robóticos simulados; 2019; Dissertação (Mestrado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro,; Orientador: Wouter Caarls;

Gabriel Lins Tenório

Applications of Deep Learning for Crop Monitoring: Classification of Crop Type, Health and Maturity; 2019; Dissertação (Mestrado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro,; Orientador: Wouter Caarls;

Eric Luz

Reinforcement learning for chemical plant control; 2019; Dissertação (Mestrado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro,; Orientador: Wouter Caarls;

SAMARA ALVAREZ ALVES

Negociação no mercado financeiro utilizando a rede neural sem peso WiSARD; 2017; Dissertação (Mestrado em Programa de Pós-Graduação em Informática) - Universidade Federal do Rio de Janeiro,; Coorientador: Wouter Caarls;

Bruno PEREIRA PALMA

PRM-EB UM NOVO ALGORITMO PARA A OTIMIZAÇÃO DE PRM BASEADO NA TEORIA DE APOSTAS; 2016; Dissertação (Mestrado em Programa de Pós-Graduação em Informática) - Universidade Federal do Rio de Janeiro,; Coorientador: Wouter Caarls;

Jorn Postma

Speeding Up Reinforcement Learning with Graphics Processing Units; 2015; Dissertação (Mestrado em Biomechanical Design) - Delft University of Technology,; Orientador: Wouter Caarls;

Bruno Sousa Campos da Costa

Trading Between Sampling and Computation in Reinforcement Learning; 2015; Dissertação (Mestrado em Informática) - Universidade Federal do Rio de Janeiro,; Coorientador: Wouter Caarls;

Hendrik Meijdam

Learning while preventing mechanical failure due to random motions; 2013; Dissertação (Mestrado em Biomechanical Design) - Delft University of Technology,; Orientador: Wouter Caarls;

Bas Vennemann

Sample-Efficient Reinforcement Learning for Walking Robots; 2013; Dissertação (Mestrado em Biomechanical Design) - Delft University of Technology,; Orientador: Wouter Caarls;

Martijn Zeestraten

Robot-learning using a Tree-based Policy Representation; 2013; Dissertação (Mestrado em Biomechanical Design) - Delft University of Technology,; Coorientador: Wouter Caarls;

Mart Moerdijk

Learning to walk using minimum prior knowledge: And a small hexapod robot; 2013; Dissertação (Mestrado em Biomechanical Design) - Delft University of Technology,; Orientador: Wouter Caarls;

Jurren Pen

Evolutionary Co-Optimisation of Control and System Parameters for a Resonating Robot Arm; 2012; Dissertação (Mestrado em Systems and Control) - Delft University of Technology,; Coorientador: Wouter Caarls;

Bart van Vliet

Accelerating reinforcement learning on a robot by using subgoals in a hierarchical framework; 2011; Dissertação (Mestrado em Biomechanical Design) - Delft University of Technology,; Coorientador: Wouter Caarls;

Merel van Diepen

Avoiding failure states during reinforcement learning; 2011; Dissertação (Mestrado em Biomechanical Design) - Delft University of Technology,; Coorientador: Wouter Caarls;

Jan van der Horst

Development and implementation of a real-time stereo camera; 2006; Dissertação (Mestrado em Applied Physics) - Delft University of Technology,; Coorientador: Wouter Caarls;

Floris Mantz

A Robust Behaviour-Based Hierarchical Vision System with Local Colour Tables and Use of Behaviour and Location Information; 2005; Dissertação (Mestrado em Applied Physics) - Delft University of Technology,; Coorientador: Wouter Caarls;

Renata Oliveira

Aprendizado por Reforço Profundo Hierárquico; 2017; Tese (Doutorado em Engenharia Elétrica) - Pontifícia Universidade Católica do Rio de Janeiro,; Orientador: Wouter Caarls;

Ivan Koryakovskiy

Safer Reinforcement Learning for Robotics; 2014; Tese (Doutorado em Mechanical, Maritime and Materials Engineering) - Delft University of Technology,; Coorientador: Wouter Caarls;

Luiz Felipe Monteiro de Albuquerque Fonseca

Otimização do Controle por Câmera de um Projeto Ball and Plate; 2020; Trabalho de Conclusão de Curso; (Graduação em Engenharia de Controle e Automação) - Pontifícia Universidade Católica do Rio de Janeiro; Orientador: Wouter Caarls;

LUIZ ANTONIO MARTINS

BIAROBOT: ROBÔ AUTÔNOMO DE INFORMAÇÕES; 2019; Trabalho de Conclusão de Curso; (Graduação em Engenharia de Controle e Automação) - Pontifícia Universidade Católica do Rio de Janeiro; Orientador: Wouter Caarls;

Guilherme Siqueira Eduardo

DESIGNING A COMPUTER VISION SYSTEM FOR HOOP DETECTION TO AID MAV NAVIGATION; 2018; Trabalho de Conclusão de Curso; (Graduação em Engenharia de Controle e Automação) - Pontifícia Universidade Católica do Rio de Janeiro; Orientador: Wouter Caarls;

Manoel Feliciano da Silva Neto

Nícolas Mocaiber Cardoso Cury

SISTEMA DE POSICIONAMENTO INDOOR POR VISÃO COMPUTACIONAL, CONSTRUÇÃO E CONTROLE DE ROBÔ CAPAZ DE IDENTIFICAÇÃO E CONDUÇÃO DE ALVO ATÉ DETERMINADO DESTINO; 2018; Trabalho de Conclusão de Curso; (Graduação em Engenharia de Controle e Automação) - Pontifícia Universidade Católica do Rio de Janeiro; Orientador: Wouter Caarls;

Vinicius Tostes Seixas

SISTEMA DE CONTROLE EMBARCADO DO ROBÔ BIPEDE LEO2; 2017; Trabalho de Conclusão de Curso; (Graduação em Engenharia de Controle e Automação) - Pontifícia Universidade Católica do Rio de Janeiro; Orientador: Wouter Caarls;

Rodrigo Sávio Gonçalves de Menezes

COMUNICAÇÃO APLICADA UTILIZANDO MODBUS; 2020; Iniciação Científica; (Graduando em Engenharia de Controle e Automação) - Pontifícia Universidade Católica do Rio de Janeiro; Orientador: Wouter Caarls;

Erick Mateus Silva de Oliveira

ATUALIZAÇÃO DE HARDWARE E SOFTWARE DE UM ROBÔ MÓVEL PARA AGRICULTURA DE PRECISÃO; 2020; Iniciação Científica; (Graduando em Engenharia de Controle e Automação) - Pontifícia Universidade Católica do Rio de Janeiro; Orientador: Wouter Caarls;

Produções bibliográficas

TENORIO, GABRIEL LINS ; Caarls, Wouter . Automatic visual estimation of tomato cluster maturity in plant rows. MACHINE VISION AND APPLICATIONS , v. 32, p. 78, 2021.
SANTIAGO, LEANDRO ; VERONA, LETICIA ; RANGEL, FABIO ; FIRMINO, FABRÍCIO ; MENASCHÉ, DANIEL S. ; Caarls, Wouter ; BRETERNITZ JR, MAURICIO ; KUNDU, SANDIP ; LIMA, PRISCILA M.V. ; FRANÇA, FELIPE M.G. . Weightless Neural Networks as Memory Segmented Bloom Filters. NEUROCOMPUTING , v. 416, p. 292-304, 2020.
Caarls, Wouter . Deep Reinforcement Learning with embedded LQR Controllers. IFAC-PAPERSONLINE , v. 53, p. 8063-8069, 2020.
KORYAKOVSKIY, IVAN ; KUDRUSS, MANUEL ; VALLERY, HEIKE ; BABUSKA, ROBERT ; Caarls, Wouter . Model-Plant Mismatch Compensation Using Reinforcement Learning. IEEE Robotics and Automation Letters , v. 3, p. 2471-2477, 2018.
CALLI, BERK ; Caarls, Wouter ; WISSE, MARTIJN ; JONKER, PIETER P. . Active Vision via Extremum Seeking for Robots in Unstructured Environments: Applications in Object Recognition and Manipulation. IEEE Transactions on Automation Science and Engineering , v. 15, p. 1810-1822, 2018.
CALLI, B. ; Caarls, W. ; WISSE, M. ; JONKER, P. . Viewpoint optimization for aiding grasp synthesis algorithms using reinforcement learning. Advanced Robotics , v. 32, p. 1077-1089, 2018.
REIFFERS-MASSON, ALEXANDRE ; HARGREAVES, EDUARDO ; ALTMAN, EITAN ; Caarls, Wouter ; MENASCHÉ, DANIEL S. . Timelines are Publisher-Driven Caches. Performance Evaluation Review , v. 44, p. 26-29, 2017.
KORYAKOVSKIY, IVAN ; KUDRUSS, MANUEL ; BABU?KA, ROBERT ; Caarls, Wouter ; KIRCHES, CHRISTIAN ; MOMBAUR, KATJA ; SCHLÖDER, JOHANNES P. ; VALLERY, HEIKE . Benchmarking model-free and model-based optimal control. Robotics and Autonomous Systems (Print) , v. 92, p. 81-90, 2017.
KORYAKOVSKIY, IVAN ; VALLERY, HEIKE ; BABU?KA, ROBERT ; Caarls, Wouter . Evaluation of physical damage associated with action selection strategies in reinforcement learning * *I. Koryakovskiy, H. Vallery and R.Babu?ka were supported by the European project KOROIBOT FP7-ICT-2013-10/611909.. IFAC-PAPERSONLINE , v. 50, p. 6928-6933, 2017.
PALMA, BRUNO ; LIMA, CABRAL ; Caarls, Wouter ; VETTORAZZI, DANILO . Wich Probabilistic Roadmap method should be used by a robot in an actual environment? An analysis of the main methods through simulations. Revista IEEE América Latina , v. 14, p. 2020-2025, 2016.
WOLFSLAG, W.J. ; PLOOIJ, M.C. ; Caarls, W. ; VAN WEPEREN, S. ; LOPES, G.A.D. . Dissipatively actuated manipulation. Control Engineering Practice , v. 34, p. 68-76, 2015.
Caarls, Wouter ; SCHUITEMA, ERIK . Parallel Online Temporal Difference Learning for Motor Control. IEEE Transactions on Neural Networks and Learning Systems , v. 27, p. 1-1, 2015.
AKMAN, OYTUN ; POELMAN, RONALD ; CAARLS, WOUTER ; JONKER, PIETER . Multi-cue hand detection and tracking for a head-mounted augmented reality system. Machine Vision and Applications (Internet) , v. 24, p. 931-946, 2013.
CAARLS, W. ; RIEGER, B. ; DE VRIES, A.H.B. ; ARNDT-JOVIN, D.J. ; JOVIN, T.M. . Minimizing light exposure with the programmable array microscope. Journal of Microscopy (Print) , v. 241, p. 101-110, 2011.
KANTELHARDT, SVEN R. ; CAARLS, WOUTER ; DE VRIES, ANTHONY H. B. ; HAGEN, GUY M. ; JOVIN, THOMAS M. ; SCHULZ-SCHAEFFER, WALTER ; ROHDE, VEIT ; GIESE, ALF ; ARNDT-JOVIN, DONNA J. . Specific Visualization of Glioma Cells in Living Low-Grade Tumor Tissue. Plos One , v. 5, p. e11323, 2010.
CAARLS, WOUTER ; SOLEDAD CELEJ, M. ; DEMCHENKO, ALEXANDER P. ; JOVIN, THOMAS M. . Characterization of Coupled Ground State and Excited State Equilibria by Fluorescence Spectral Deconvolution. Journal of Fluorescence , v. 20, p. 181-190, 2010.
CELEJ, M. SOLEDAD ; CAARLS, WOUTER ; DEMCHENKO, ALEXANDER P. ; JOVIN, THOMAS M. . A Triple-Emission Fluorescent Probe Reveals Distinctive Amyloid Fibrillar Polymorphism of Wild-Type -Synuclein and Its Familial Parkinson-s Disease Mutants. Biochemistry , v. 48, p. 7465-7472, 2009.
HAGEN, GUY M. ; CAARLS, WOUTER ; LIDKE, KEITH A. ; DE VRIES, ANTHONY H.B. ; FRITSCH, CORNELIA ; BARISAS, B. GEORGE ; ARNDT-JOVIN, DONNA J. ; JOVIN, THOMAS M. . Fluorescence recovery after photobleaching and photoconversion in multiple arbitrary regions of interest using a programmable array microscope. Microscopy Research and Technique (Online) , v. 72, p. 431-440, 2009.
ARNDT-JOVIN, D.J. ; KANTELHARDT, S.R. ; CAARLS, W. ; DE VRIES, A.H.B. ; GIESE, A. ; JOVIN, T.M. . Tumor-Targeted Quantum Dots Can Help Surgeons Find Tumor Boundaries. IEEE Transactions on Nanobioscience , v. 8, p. 65-71, 2009.
CAARLS, W. . Skeletons and Asynchronous RPC for Embedded Data and Task Parallel Image Processing. IEICE Transactions on Information and Systems , v. E89-D, p. 2036-2043, 2006.
HAGEN, GUY M. ; LIDKE, KEITH A. ; Rieger, Bernd ; Lidke, Diane S. ; Caarls, Wouter ; ARNDT-JOVIN, DONNA J. ; JOVIN, THOMAS M. . Dynamics of Membrane Receptors: Single-molecule Tracking of Quantum Dot Liganded Epidermal Growth Factor. Single Molecule Dynamics in Life Science. 1ed.: Wiley-VCH Verlag GmbH & Co. KGaA, 2009, v. , p. 117-130.
OLIVEIRA, RENATA ; Caarls, Wouter . A History-based Framework for Online Continuous Action Ensembles in Deep Reinforcement Learning. In: 13th International Conference on Agents and Artificial Intelligence, 2021, Online Streaming. Proceedings of the 13th International Conference on Agents and Artificial Intelligence, 2021. p. 580.
BATISTA, EVELYN ; PACHECO, MARCO ; Caarls, Wouter ; FORERO, LEONARDO . Training an Agent to Find and Reach an Object in Different Environments using Visual Reinforcement Learning and Transfer Learning. In: 13th International Conference on Agents and Artificial Intelligence, 2021, Online Streaming. Proceedings of the 13th International Conference on Agents and Artificial Intelligence, 2021. p. 732.
FELIPE F. MARTINS ; THIAGO M. CARVALHO ; ALIMED CELECIA ; Adalberto I. S. Oliveira ; Gustavo B. P. Barbosa ; MARLEY M. B. VELLASCO ; WOUTER CAARLS ; KARLA FIGUEIREDO ; ANTONIO C. LEITE . Sistema de navegação autônoma para o robô agrícola Soybot. In: XV Simpósio Brasileiro de Automação Inteligente, 2021, Online, 2022.
GARCIA OLIVEIRA, RENATA ; Caarls, Wouter . Comparing Action Aggregation Strategies in Deep Reinforcement Learning with Continuous Action. In: Congresso Brasileiro de Automática 2020, 2020. Anais do Congresso Brasileiro de Automática 2020, 2020. v. 2.
Leandro Santiago ; Leticia Verona ; Fabio Rangel ; Fabrício Firmino ; MENASCHE, D. S. ; Wouter Caarls ; Mauricio Breternitz Jr ; Sandip Kundu ; Priscila M.V. Lima ; FRANCA, F. M. G. . Memory Efficient Weightless Neural Network using Bloom Filter. In: uropean Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, 2019, Bruges. ESANN 2019 proceedings,, 2019.
TENORIO, GABRIEL ; MARTINS, F. F. ; CARVALHO, T. M. ; LEITE, A. C. ; FIGUEIREDO, K. ; VELLASCO, M. ; CAARLS, W. . Comparative Study of Computer Vision Models for Insect Pest Identification in Complex Backgrounds. In: Developments in eSystems Engineering, 2019, Kazan. DeSE proceedings, 2019.
BARBOSA, W. ; Adalberto I. S. Oliveira ; Gustavo B. P. Barbosa ; LEITE, A. C. ; FIGUEIREDO, K. ; VELLASCO, M. ; CAARLS, W. . Design and Development of an Autonomous Mobile Robot for Inspection of Soy and Cotton Crops. In: Developments in eSystems Engineering, 2019, Kazan. DeSE proceedings, 2019.
Adalberto I. S. Oliveira ; CARVALHO, T. M. ; MARTINS, F. F. ; LEITE, A. C. ; FIGUEIREDO, K. ; VELLASCO, M. ; CAARLS, W. . On the Intelligent Control Design of an Agricultural Mobile Robot for Cotton Crop Monitoring. In: Developments in eSystems Engineering, 2019, Kazan. DeSE proceedings, 2019.
CARVALHO, T. M. ; TENORIO, G. L. ; FIGUEIREDO, K. ; VELLASCO, M. ; CAARLS, W. . Comparison of Machine Learning Models for Total Dengue Cases Prediction. In: XVI Encontro Nacional de Inteligência Artificial e Computacional, 2019, Salvador. Proc. XVI Encontro Nacional de Inteligência Artificial e Computacional, 2019.
TENORIO, GABRIEL ; COSTA DA SILVA, EDUARDO ; VILLALOBOS, CRISTIAN ; MENDOZA, LEONARDO ; Caarls, Wouter . Improving Transfer Learning Performance: An Application in the Classification of Remote Sensing Data. In: 11th International Conference on Agents and Artificial Intelligence, 2019, Prague. Proceedings of the 11th International Conference on Agents and Artificial Intelligence, 2019. v. 2. p. 174.
ALVES, SAMARA A. ; Caarls, Wouter ; LIMA, PRISCILA M.V. . Weightless Neural Network for High Frequency Trading. In: 2018 International Joint Conference on Neural Networks (IJCNN), 2018, Rio de Janeiro. 2018 International Joint Conference on Neural Networks (IJCNN), 2018. p. 1.
CARDENOSO FERNANDEZ, FRANKLIN ; Caarls, Wouter . Parameters Tuning and Optimization for Reinforcement Learning Algorithms Using Evolutionary Computing. In: 2018 International Conference on Information Systems and Computer Science (INCISCOS), 2018, Quito. 2018 International Conference on Information Systems and Computer Science (INCISCOS), 2018. p. 301.
M. F. S. Neto ; G.S. Eduardo ; COSTA DA SILVA, EDUARDO ; W. Caarls . Computer vision based solutions for mav target detection and flight control. In: 10th international micro air vehicle competition and conference (IMAV), 2018, Melbourne. Proc. 10th international micro air vehicle competition and conference (IMAV). Melbourne: RMIT Unmanned Aerial Systems Research Team, 2018. v. 1. p. 309-314.
CAARLS, WOUTER ; Eduardo Hargreaves ; MENASCHE, D. S. . Q-caching: an integrated reinforcement-learning approach for caching and routing in information-centric networks. In: XXXIV Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos, 2016, Salvador, Bahia, Brazil. Anais do SBRC 2016. Porto Alegre: Sociedade Brasileira de Computação, 2016. p. 366-379.
Kimberly McGuire ; Masato Tsukada ; Boris Lenseigne ; Wouter Caarls ; Masato Toda ; Pieter Jonker . A Novel Method for Simultaneous Acquisition of Visible and Near-Infrared Light Using a Coded Infrared-Cut Filter. In: Computer Analysis of Images and Patterns, 2015, Valletta. 16th International Conference, CAIP 2015, Valletta, Malta, September 2-4, 2015 Proceedings, Part I, 2015. v. 9256. p. 174-185.
Bruno Costa ; Wouter Caarls ; Daniel Sadoc Menasche . Dyna-MLAC: Trading Computational and Sample Complexities in Actor-Critic Reinforcement Learning. In: Brazilian Conference on Intelligent Systems, 2015, Natal. Proc. Brazilian Conference on Intelligent Systems, 2015.
BHARATHEESHA, MUKUNDA ; Caarls, Wouter ; WOLFSLAG, WOUTER JAN ; WISSE, MARTIJN . Distance metric approximation for state-space RRTs using supervised learning. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), 2014, Chicago. 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2014. p. 252-257.
PEN, JURREN ; Caarls, Wouter ; WISSE, MARTIJN ; BABUSKA, ROBERT . Evolutionary co-optimization of control and system parameters for a resonating robot arm. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), 2013, Karlsruhe. 2013 IEEE International Conference on Robotics and Automation. p. 4195.
MEIJDAM, H. J. ; PLOOIJ, M. C. ; CAARLS, W. . Learning while preventing mechanical failure due to random motions. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2013), 2013, Tokyo. 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. p. 182.
CALLI, BERK ; Caarls, Wouter ; JONKER, PIETER ; WISSE, MARTIJN . Comparison of extremum seeking control algorithms for robotic applications. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2012), 2012, Vilamoura-Algarve. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. p. 3195.
VAN VLIET, B. ; CAARLS, W. ; SCHUITEMA, E. ; JONKER, P.P. . Accelerating reinforcement learning on a robot by using subgoals in a hierarchical framework. In: Benelux Conference on Artificial Intelligence, 2011, Ghent. Proc. 23rd Benelux Conference on Artificial Intelligence, 2011.
SCHUITEMA, E. ; CAARLS, W. ; WISSE, M. ; JONKER, P.P. ; BABUSKA, R. . The Effects of Large Disturbances on On-Line Reinforcement Learning for a Walking Robot. In: Benelux Conference on Artificial Intelligence, 2010, Luxembourg. Proc. 22nd Benelux Conference on Artificial Intelligence, 2010.
CAARLS, W. ; JONKER, P.P. ; CORPORAAL, H. . Algorithmic skeletons for stream programming in embedded heterogeneous parallel image processing applications. In: , 2006, Rhodes Island. . p. 9 pp..
CAARLS, W. ; JONKER, P.P. ; CORPORAAL, H. . Skeletons and Asynchronous RPC for Embedded Data- and Task Parallel Image Processing. In: IAPR Conference on Machine Vision Applications, 2005, Tsukuba Science City. Proc. 9th IAPR Conference on Machine Vision Applications, 2005. p. 384-387.
BROERS, H. ; CAARLS, W. ; JONKER, P.P. ; KLEIHORST, R.P. . Architecture Study for Smart Cameras. In: EOS Conference on Industrial Imaging and Machine Vision, 2005, Munich. Proc. EOS Conference on Industrial Imaging and Machine Vision, 2005. p. 39-49.
MANTZ, F. ; JONKER, P.P. ; CAARLS, W. . Behavior-Based Vision on a 4 Legged Soccer Robot. In: Robocup conference, 2005, Osaka. Lecture Notes in Computer Science, 2005. v. 4020. p. 480-487.
JONKER, P.P. ; CAARLS, W. . Application Driven Design of Embedded Real-Time Image Processors. In: Advanced Concepts for Intelligent Vision Systems, 2003, Ghent. Proc. Advanced Concepts for Intelligent Vision Systems, 2003. p. 1-8.
CAARLS, W. ; JONKER, P.P. . Benchmarks for SmartCam Development. In: Advanced Concepts for Intelligent Vision Systems, 2003, Ghent. Proc. Advanced Concepts for Intelligent Vision Systems 2003, 2003. p. 81-86.
HAGEN, GUY M. ; CAARLS, W. ; THOMAS, M. ; HILL, A. ; LIDKE, K.A. ; Rieger, Bernd ; FRITSCH, CORNELIA ; VAN GEEST, B. ; JOVIN, THOMAS M. ; ARNDT-JOVIN, DONNA J. . Biological applications of an LCoS-based PROGRAMMABLE ARRAY MICROSCOPE (PAM). In: Photonic West 2007, BiOS, 2007, San Jose, CA. Proc. SPIE, 2007. v. 6441.
CAARLS, W. ; JONKER, P.P. ; CORPORAAL, H. . Data- and Task Parallel Image Processing on a Mixed SIMD-ILP Platform using Skeletons and Asynchronous RPC. In: PROGRESS Workshop on Embedded Systems, 2004, Nieuwegein. Proc. 5th PROGRESS Workshop on Embedded Systems, 2004.
CAARLS, W. ; JONKER, P.P. ; CORPORAAL, H. . SmartCam Design Framework. In: PROGRESS Workshop on Embedded Systems, 2003, Nieuwegein. Proc. 4th PROGRESS Workshop on Embedded Systems, 2003.
CAARLS, W. ; JONKER, P.P. ; CORPORAAL, H. . SmartCam: Devices for Embedded Intelligent Cameras. In: PROGRESS Workshop on Embedded Systems, 2002, Utrecht. Proc. 3rd PROGRESS Workshop on Embedded Systems, 2002.
CALLI, B. ; CAARLS, W. ; LEI, Q. ; WISSE, M. ; JONKER, P.P. . SMAG: Simultaneous Modeling and Grasping. In: RSS 2013 Workshop: Manipulation with Uncertain Models, 2013, Berlin. RSS 2013 Workshop: Manipulation with Uncertain Models, 2013.
DE BEULE, P.A.A. ; DE VRIES, ANTHONY H. B. ; CAARLS, W. ; ARNDT-JOVIN, DONNA J. ; JOVIN, THOMAS M. . A Generation-3 Programmable Array Microscope with Digital Micro-Mirror Device. In: 54th Annual Meeting of the Biophysical Society, 2010, San Fransisco, CA, USA. Biophysical Journal, 2010. v. 98. p. 178a-178a.
HAGEN, GUY M. ; CAARLS, W. ; LIDKE, K.A. ; DE VRIES, ANTHONY H.B. ; FRITSCH, CORNELIA ; BARISAS, B. GEORGE ; ARNDT-JOVIN, DONNA J. ; JOVIN, THOMAS M. . FRAP and Photoconversion in Multiple Arbitrary Regions of Interest Using a Programmable Array Microscope (PAM). In: 58th Annual Meeting of the Biophysical Society, 2009, San Fransisco, CA, USA. Biophysical Journal, 2009. v. 96. p. 281a-281a.
CAARLS, W. ; CELEJ, M. SOLEDAD ; DEMCHENKO, ALEXANDER P. ; JOVIN, THOMAS M. . Multiwavelength ratiometric fluorescence sensing. In: Methods and Applications of Fluorescence 2009, 2009, Budapest. Proc. Methods and Applications of Fluorescence 2009, 2009. p. 184-184.
CAARLS, W. ; DE VRIES, ANTHONY H. B. ; ARNDT-JOVIN, DONNA J. ; JOVIN, THOMAS M. . Arbitrary and Dynamic Patterning in a Programmable Array Microscope. In: Focus on Microscopy 2009, 2009, Krakow. Proc. Focus on Microscopy 2009, 2009. p. 137-137.
JOVIN, THOMAS M. ; HAGEN, GUY M. ; CAARLS, W. ; ARNDT-JOVIN, DONNA J. . Live cell microscopy of growth-factor dependent signal transduction pathways with a Programmable Array Microscope (PAM). In: 17th Annual Meeting of the German Society of Cytometry (DGfZ), 2007, Regensburg. Cytometry Part A, 2007. v. 71A. p. 745-746.
ARNDT-JOVIN, DONNA J. ; HAGEN, GUY M. ; CAARLS, W. ; HILL, A. ; JOVIN, THOMAS M. . Biological applications of an LCoS-based programmable array microscope (PAM). In: 17th Annual Meeting of the German Society of Cytometry (DGfZ), 2007, Regensburg. Cytometry Part A, 2007. v. 71A. p. 512-512.
CAARLS, W. . Parallel DYNA. 2013. (Apresentação de Trabalho/Simpósio).
CAARLS, W. . Parallel Real-Time Reinforcement Learning. 2011. (Apresentação de Trabalho/Simpósio).
CAARLS, W. . GPU Programming Paradigms. 2010. (Apresentação de Trabalho/Simpósio).

Outras produções

CAARLS, WOUTER ; KORYAKOVSKIY, I. ; Manuel Kudruss . Generic Reinforcement Learning Library. 2015.

Projetos de pesquisa

2019 - Atual

Aprendizado por Reforço em Conjunto, Descrição: Muitas áreas da tecnologia precisam da otimização de sequências de decisões. Planejamento de processos de produção, gestão de tráfego, agendamento de medicações e controle de robôs são só alguns exemplos desse tipo de tarefa. Sabendo o modelo da dinâmica do sistema, a área de pesquisa operacional possui diversas ferramentas para realizar a otimização. Porém, em muitas instâncias, o modelo é desconhecido ou inexato, como um robô móvel encontrando um novo ambiente ou objeto a ser manipulado. Nesse caso, o problema se torna um de aprendizado por reforço. Em aprendizado por reforço, a otimização da tarefa acontece usando experiência do mundo real. Através de tentativa e erro, o algoritmo descobre ações a ser reforçadas (que trazem custos baixos) e ações a serem evitadas (que trazem custos altos). A abordagem é baseada no método de aprendizagem por animais e seres humanos, e foi aplicada com êxito a uma grande variedade de tarefas. O problema é que precisa de muitas tentativas antes de aprender, e os erros podem danificar o sistema. Recentemente, foram desenvolvidas estratégias para diminuir a quantidade de experiência necessária para o aprendizado. O foco nessas estratégias é o uso efetivo da informação acumulada. Originalmente, os algoritmos usaram cada experiência só uma vez, e só para situações muito parecidas. Agora existem algoritmos que conseguem reusar experiência obtida no passado e generalizar essa experiência para situações diferentes. Porém, ainda precisam de muitos dados. A ideia principal desse projeto é extrair ainda mais informação das experiências, através o uso de conjuntos. Podemos aprender um conjunto de (sub)tarefas ao mesmo tempo, aprender usando um conjunto de modelos inexatos além do mundo real, e aprender com algoritmos e representações diferentes. Assim, usamos os dados para muitas finalidades ao mesmo tempo. Se as finalidades são ligadas à mesma tarefa global, isso pode diminuir a quantidade de experiência necessária para aprendê-la.. , Situação: Em andamento; Natureza: Pesquisa. , Alunos envolvidos: Doutorado: (1) . , Integrantes: Wouter Caarls - Coordenador / Renata Garcia Oliveira - Integrante.
2014 - 2016

KOROIBOT, Descrição: The goal of the KoroiBot project is to enhance the ability of humanoid robots to walk in a dynamic and versatile fashion in the way humans do. Research and innovation work in KoroiBot will mainly target novel motion control methods for existing hardware, but it will also derive optimized design principles for next generation robots. By doing so, KoroiBot addresses the ambitious goals set for the humanoid robots of the 21st century which are supposed to work and replace humans e.g. in households, disaster sites or space missions but which still lack the very fundamental ability to walk in a human-like fashion. Compared to all the intelligence that humanoids have to acquire to perform these tasks, the demand for an improved walking performance seems simple, but it is in fact very challenging, and the motion abilities of contemporary humanoids are still far behind their human role models. Human gaits are at the same time efficient, robust and versatile but gaits of humanoids or bipedal robots are at best good in one of these areas. This problem is not only linked to the present hardware, but also to a large extent to the control principles and the software used.. , Situação: Concluído; Natureza: Pesquisa. , Integrantes: Wouter Caarls - Integrante / WISSE, MARTIJN - Integrante / Heike Vallery - Coordenador / Ivan Koryakovskiy - Integrante.
2014 - 2016

On-line Model-learning Policy Search, Descrição: Reinforcement learning is a powerful method for learning control policies in a variety of applications such as robotics, scheduling, and traffic and network congestion control. Because the environments in which such systems must work are forever changing, it is infeasible to pre-program a solution that works in all cases. Through trial and error, a reinforcement learning agent optimizes a control policy for the desired task without prior knowledge of the environment. However, especially in robotics its applicability has thus far been limited by long learning times. In this project, we aim to develop fully on-line model-learning policy search techniques, thereby combining the low number of trials of model-learning policy search with the short computation time of on-line methods. We have recently developed such an on-line model learning method in the context of value-based reinforcement learning, where we achieved a speedup of two orders of magnitude over standard on-line techniques. An effective combination of the two should allow effcient learning of complex control policies for systems with many state variables.. , Situação: Concluído; Natureza: Pesquisa. , Alunos envolvidos: Graduação: (1) . , Integrantes: Wouter Caarls - Integrante / Daniel Sadoc Menasche - Coordenador.
2013 - 2017

Factory-in-a-day, Descrição: Be it the packing and quality checking of fruit, the polishing of steel moulds or the filling of a spray-painting machine, all these processes have one thing in common: they are usually done manually because there is no robot or automated process that can do the job as efficient as a human worker. Today, setting up a robotic system takes at least 3 months and the costs are immense. SMEs usually only have small production batches due to seasonal on-off production. State-of-the-art systems don?t provide the flexibility they need to stay competitive on a global market. For these reasons SMES in Europe rarely use advanced robot technology. Using innovative design templates and parameterized learning by demonstration through domain-specific task libraries, the Factory-in-a-day project aims to reduce the installation time to a single day.. , Situação: Concluído; Natureza: Pesquisa. , Integrantes: Wouter Caarls - Integrante / Martijn Wisse - Coordenador.
2007 - 2010

FLUOROMAG, Descrição: The project of the consortium has two elements. The first is the development of other classes of still smaller NPs, i.e. with sizes below 10 nm (less than a millionth of a cm): fluorescent noble-metal "nanodots" and magnetic NPs. These materials are superior to conventional fluorophores in that they exhibit extreme photo- and chemical stability. The nanodots should have reduced toxicity and greater target accessibility than quantum dots, yet offer a similar detection sensitivity. They will be derivatized and tested for specific recognition of biomolecules such as tumor markers (for breast cancer) and global viral disease (Hepatitis C and Dengue Fever). Other core-shell "onion-like" NPs developed by the partner in Santiago de Compostela have diverse and strong magnetic properties and will be tested for their application in micro-chip and MRI diagnostics. In a parallel effort, several of the partners will optimize the design and performance of a new type of high-speed, sensitive, optically sectioning microscope known as the Programmable Array Microscope (PAM), for use in both the basic research and medical communities. The PAM is very versatile in that it implements many imaging modalities and has been under development in the Molecular Biology Dept. for the past 10 years. It has single-NP sensitivity, and is ideally suited for measurements of thick samples such as tissue slices and patterned arrays, important objects for diagnostic tests.. , Situação: Concluído; Natureza: Pesquisa. , Integrantes: Wouter Caarls - Integrante / DE VRIES, ANTHONY H. B. - Integrante / HAGEN, GUY M. - Integrante / JOVIN, THOMAS M. - Integrante / ARNDT-JOVIN, DONNA J. - Coordenador / Arturo López-Quintela - Integrante / Vinod Subramaniam - Integrante / Quentin Hanley - Integrante.
2006 - 2012

FALCON, Descrição: The challenge to develop a fully integrated and automated logistics warehouse as a complex ?system-of-systems?, is to increase performance under stochastic conditions whilst maintaining reliability. Basically the research projects comprises of three lines of attention: 1. Through high-level simulation, system-level requirements can be decomposed and propagated to the component level. 2. Developing specific mechatronic components, as well as the redesign and evaluation of both the overall system- and component architectures. 3. Reliable integration of components into systems requires a system-level control approach that involves the appropriate sensors and actuators, as well as relevant in formation architectures to synchronize the virtual world with the real world. In addition to these research topics, the integrity of the relationship between the conceptual design and its actual implementation is addressed as a crucial condition for achieving the correct level of dependability of the resulting system-of-systems.. , Situação: Concluído; Natureza: Pesquisa. , Integrantes: Wouter Caarls - Integrante / AKMAN, OYTUN - Integrante / JONKER, PIETER - Coordenador / Maya Rudinac - Integrante.
2002 - 2008

SMARTCAM, Descrição: The advent and subsequent popularity of low cost, low power CMOS vision sensors enables us to integrate processing logic on the camera chip itself, thereby creating so-called smart sensors. The SmartCam project investigates these new opportunities and contributes to a better and more quantitatively guided design trajectory. In particular, it will investigate the impact of current applications, define relevant architectural parameters and develop an architectural template, enhance existing application mapping environments for SIMD (Single-Instruction, Multiple-Data) and ILP (Instruction-Level Parallel) processors, and perform two case studies. The work will focus on creating an environment for exploring the design space parametrized by the architectural template and integrating this with the application mapping environment.. , Situação: Concluído; Natureza: Pesquisa. , Alunos envolvidos: Mestrado acadêmico: (0) Doutorado: (2) . , Integrantes: Wouter Caarls - Integrante / Pieter Jonker - Coordenador / Henk Corporaal - Integrante / Hamed Fatemi - Integrante.

Prêmios

2002

Cum Laude, University of Amsterdam.

Histórico profissional

Endereço profissional

Pontifícia Universidade Católica do Rio de Janeiro, Reitoria, Departamento de Engenharia Elétrica. , Pontifícia Universidade Católica - PUC, Gávea, 22451900 - Rio de Janeiro, RJ - Brasil, Telefone: (21) 35271231

Experiência profissional

2014 - 2016

Universidade Federal do Rio de Janeiro

Vínculo: Scholarship, Enquadramento Funcional: Bolsista Jovem Talento Ciência sem Fronteiras, Carga horária: 40

2009 - 2014

Delft University of Technology

Vínculo: , Enquadramento Funcional: Postdoctoral research fellow, Carga horária: 38, Regime: Dedicação exclusiva.

2007 - 2009

Max Planck Institute for Biophysical Chemistry

Vínculo: Formal labor contract, Enquadramento Funcional: Postdoctoral research fellow, Carga horária: 40, Regime: Dedicação exclusiva.

2019 - Atual

Pontifícia Universidade Católica do Rio de Janeiro, PUC-Rio

Vínculo: Formal labor contract, Enquadramento Funcional: Professor Adjunto I, Carga horária: 40

2016 - 2019

Pontifícia Universidade Católica do Rio de Janeiro, PUC-Rio

Vínculo: Formal labor contract, Enquadramento Funcional: Professor Assistente I, Carga horária: 40, Regime: Dedicação exclusiva.