Johannes Ackermann

I am on the job market for postdocs or industry positions starting Fall 2026 or Spring 2027, please reach out if you have an opening!

I am a final-year PhD student at the University of Tokyo, working on Reinforcement Learning supervised by Masashi Sugiyama, and a part-time researcher at RIKEN AIP.

I previously interned at Sakana AI and at Preferred Networks. Prior to starting my PhD, I worked on applied ML for Optical Communication at Huawei, obtained a B.Sc. and M.Sc. in Electrical Engineering and Information Technology from the Technical University of Munich and wrote my Master’s Thesis at ETH Zurich’s Disco Group.

I am particularly interested in the nature of “tasks” in RL, here defined as the combination of transition and reward function:

• Task Specification: In LLM post-training tasks are often specified indirectly via reward models or with other LLMs as judges. We showed that reward models learned from human preferences (RLHF) need off-policy corrections [COLM1]. I am also particularly excited about our recent preprint [arXiv1], which shows that gradient regularization effectively prevents reward hacking in both RLHF and LLM-as-a-Judge, showing how we can deal with poory specified rewards/tasks! I also (co-)investigated different ways to accumulate rewards, beyond the simple discounted sum [RLC3], such as range, min, max, variance, etc.

• Changing Tasks: How can we deal with changing tasks during dataset collection for Offline RL [RLC1] or changing dynamics shift during deployment [RLC2]?

• Structure of Tasks: In Multi-Task RL, all tasks are usually treated as equally (dis)similar. I investigated how to identify and use task relations, by learning continuous task spaces [Thesis, Chapter 3] and task clusterings [ECML-PKDD1].

I’m always happy to chat about research, so feel free to reach out by e-mail or socials!