Selected Publications by Topic

Unsupervised Computer Vision

Rico Jonschkowski, Austin Stone, Jonathan T. Barron, Ariel Gordon, Kurt Konolige, and Anelia Angelova. What Matters in Unsupervised Optical Flow. ECCV, 2020. Code, PDF, BibTeX, ECCV Oral, featured in Computer Vision News' Best of ECCV,

TL;DR: A thorough study into the core components of unsupervised optical flow produces in a simple method that sets a new state of the art


Rico Jonschkowski and Austin Stone. Towards Object Detection from Motion. NeurIPS Workshop on Robot Learning: Control and Interaction in the Real World, 2019. PDF, BibTeX

TL;DR: A method for learning to detect objects from a few minutes of video without image annotations

Algorithmic Priors

Peter Karkus, Anelia Angelova, Vincent Vanhoucke, and Rico Jonschkowski. Differentiable Mapping Networks: Learning Structured Map Representations for Sparse Visual Localization. arXiv:2005.09530, 2020. PDF, BibTeX

TL;DR: combining spatial structure and end-to-end learning for mapping and localization


Rico Jonschkowski, Divyam Rastogi, and Oliver Brock. Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors. Proceedings of Robotics: Science and Systems, 2018. PDF, BibTeX

TL;DR: the particle filter structure encoded in a recurrent deep network


Rico Jonschkowski and Oliver Brock. End-To-End Learnable Histogram Filters. Workshop on Deep Learning for Action and Interaction at NeurIPS, 2016. PDF, BibTeX, (extension of the paper below)

Rico Jonschkowski and Oliver Brock. Towards Combining Robotic Algorithms and Machine Learning: End-To-End Learnable Histogram Filters. Workshop on Machine Learning Methods for High-Level Cognitive Capabilities in Robotics at IROS, 2016. PDF, BibTeX, (preliminary version, see extended version above)

TL;DR: a way to combine the algorithmic structure of Bayes filters with the end-to-end learnability of neural networks

State Representation Learning

Rico Jonschkowski, Roland Hafner, Jonathan Scholz, and Martin Riedmiller. PVEs: Position-Velocity Encoders for Unsupervised Learning of Structured State Representations. New Frontiers for Deep Learning in Robotics Workshop at RSS, 2017. PDF, BibTeX, Paper Award winner

TL;DR: unsupervised learning of where things are and how they are moving


Sebastian Höfer, Antonin Raffin, Rico Jonschkowski, Oliver Brock, and Freek Stulp. Unsupervised Learning of State Representations for Multiple Tasks. Workshop on Deep Learning for Action and Interaction at NeurIPS, 2016. PDF, BibTeX

TL;DR: a learning method for automatic detection of multiple reinforcement tasks and extraction of state representations from raw observations


Rico Jonschkowski and Oliver Brock. Learning State Representations with Robotic Priors. Autonomous Robots. Springer US 39(3):407-428, 2015. Code, Video of Robot Experiment, PDF, BibTeX, (extension of the two papers below)

Rico Jonschkowski and Oliver Brock. State Representation Learning in Robotics: Using Prior Knowledge about Physical Interaction. Proceedings of Robotics: Science and Systems, 2014. Talk, PDF, BibTeX, (preliminary version, see extended version above)

Rico Jonschkowski and Oliver Brock. Learning Task-Specific State Representations by Maximizing Slowness and Predictability. Proceedings of the 6th International Workshop on Evolutionary and Reinforcement Learning for Autonomous Robot Systems (ERLARS), 2013. PDF, BibTeX, (preliminary version, see extended version above)

TL;DR: state representations can be learned from raw sensory input by making these representations consistent with prior knowledge about interactions governed by physics = robotic priors

Amazon Picking Challenge

Rico Jonschkowski, Clemens Eppner*, Sebastian Höfer*, Roberto Martín-Martín*, and Oliver Brock. Probabilistic Multi-Class Segmentation for the Amazon Picking Challenge. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016. Code, PDF, BibTeX, IROS Best Paper Award Finalist

TL;DR: in-depth analysis of the object-segmentation method used in our winning entry to the 2015 Amazon picking challenge with lessons that should be useful towards more generic robotic perception


Clemens Eppner*, Sebastian Höfer*, Rico Jonschkowski*, Roberto Martín-Martín*, Arne Sieverling*, Vincent Wall* and Oliver Brock. Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems. Proceedings of Robotics: Science and Systems, 2016. Talk, PDF, BibTeX, RSS Best Systems Paper Award winner

TL;DR: four aspects that improve our exploration and understanding of how to build robotic systems: modularity vs. integration, computation vs. embodiment, planning vs. feedback, generality vs. assumptions

Dissertation

Rico Jonschkowski. Learning Robotic Perception Through Prior Knowledge. Dissertation, Technische Universität Berlin, 2018. PDF

This dissertation summarizes my pre-2018 work listed above. My committee consisting of Manfred Opper, George Konidaris, Marc Toussaint, and Oliver Brock graded it with summa cum laude and the Department for Electrical Engineering and Computer Science at TU Berlin awarded it the Best Dissertation Prize of the Dr. Wilhelmy-Stiftung.

Curriculum Vitae

05/2018 - present

Research scientist at Robotics at Google, Mountain View, California.

10/2012 - 05/2018

Research associate and PhD student at RBO, TU Berlin (Advisor: Oliver Brock).
Taught courses: Fundamentals of Robotics, Robotics, Advanced Robotics, Robotics Seminar, Robotics Project, Algorithms and Datastructures.

05/2018 Dr. rer. nat. (German PhD equivalent, summa cum laude)
Thesis: Learning Robotic Perception Through Prior Knowledge.

01/2017 - 04/2017 PhD Intern at DeepMind, London.

10/2007 - 09/2012

Study of computer science at FU Berlin.

09/2012 Master of Science (grade: 1.0, major: robotics/AI, minor: psychology).
Thesis: New Approaches to Temporal Abstraction in Hierarchical Reinforcement Learning.

01/2012 - 09/2012 Research assistant at MLR, FU Berlin (Advisor: Marc Toussaint).

2011 Study abroad at UNSW, Sydney.

10/2009 - 09/2011 Teaching assistant at FU Berlin.
Taught courses: Functional Programming, Object-Oriented Programming, Computer Science and Society, Software Engineering

01/2011 Bachelor of Science (grade: 1.4, major: computer science, minor: philosophy).
Thesis: Control of autonomous humanoid soccer robots with XABSL.

2008 - 2011 Member of RoboCup team FUmanoids (Advisor: Raul Rojas).

Awards, Prizes, Scholarships

2018 Prize from the Dr. Wilhelmy-Stiftung for the best dissertation in electrical engineering and computer science at TU Berlin.

2017 Paper Award winner at New Frontiers for Deep Learning in Robotics Workshop at RSS
Rico Jonschkowski, Roland Hafner, Jonathan Scholz, and Martin Riedmiller. PVEs: Position-Velocity Encoders for Unsupervised Learning of Structured State Representations. New Frontiers for Deep Learning in Robotics Workshop at RSS, 2017.

2016 Best Paper Award finalist at IROS
Rico Jonschkowski, Clemens Eppner, Sebastian Höfer, Roberto Martín-Martín, and Oliver Brock. Probabilistic Multi-Class Segmentation for the Amazon Picking Challenge. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016.

2016 Best Systems Paper Award winner at RSS
Clemens Eppner, Sebastian Höfer, Rico Jonschkowski, Roberto Martín-Martín, Arne Sieverling, Vincent Wall, and Oliver Brock. Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems. Robotics: Science and Systems (RSS), 2016.

2015 Winner of the Amazon Picking Challenge at ICRA15.

2015 AAAI-15 Robotics Fellowship

2011 PROMOS scholarship from FU Berlin

2011 4th place RoboCup Worldcup, 2nd place RoboCup German Open, 1st place Technical Challenge @ RoboCup German Open, 1st place RoboCup Iran Open

2010 2nd place RoboCup Worldcup, 1st place Technical Challenge @ RoboCup Worldcup, 1st place RoboCup Iran Open

2008 2nd place RoboCup German Open

2007 1st place RoboCup Junior German Open

Organized Events

Nature vs. Nurture in Robotics (ICRA16 Workshop)
Slides, Transcript of a radio interview