Can I read Continual Task Allocation in Meta-Policy Network via Sparse Prompting on EtoBox?

Continual Task Allocation in Meta-Policy Network via Sparse Prompting by Yang, Yijun; Zhou, Tianyi; Jiang, Jing; Long, Guodong; Shi, Yuhui is a scholarly article available to read on EtoBox.

What is Continual Task Allocation in Meta-Policy Network via Sparse Prompting about?

How to train a generalizable meta-policy by continually learning a sequence of tasks? It is a natural human skill yet challenging to achieve by current reinforcement learning: the agent is expected to quickly adapt to new tasks (plasticity) meanwhile retaining the common knowledge from previous tasks (stability). We address it by "Continual Task Allocation via Sparse Prompting (CoTASP)", which learns over-complete dictionaries to produce sparse masks as prompts extracting a sub-network for each task from a meta-policy network. CoTASP trains a policy for each task by optimizing the prompts and the sub-network weights alternatively. The dictionary is then updated to align the optimized prompts with tasks' embedding, thereby capturing tasks' semantic correlations. Hence, relevant tasks share more neurons in the meta-policy network due to similar prompts while cross-task interference causing forgetting is effectively restrained. Given a meta-policy and dictionaries trained on previous tasks, new task adaptation reduces to highly efficient sparse prompting and sub-network finetuning. In experiments, CoTASP achieves a promising plasticity-stability trade-off without storing or replaying

Author

Yang, Yijun; Zhou, Tianyi; Jiang, Jing; Long, Guodong; Shi, Yuhui

Published

2023

Language

EN