RMM: Reinforced memory management for class-incremental learning
Class-Incremental Learning (CIL) [38] trains classifiers under a strict memory budget: in each incremental phase, learning is done for new data, most of which is abandoned to free space for the next phase. The preserved data are exemplars used for replaying. However, existing methods use a static an...
Saved in:
Main Authors: | , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2021
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/6790 https://ink.library.smu.edu.sg/context/sis_research/article/7793/viewcontent/NeurIPS2021_Submission_Class_Incremental_Learning__2___1_.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-7793 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-77932022-01-27T09:58:59Z RMM: Reinforced memory management for class-incremental learning LIU, Yaoyao SUN, Qianru SUN, Qianru Class-Incremental Learning (CIL) [38] trains classifiers under a strict memory budget: in each incremental phase, learning is done for new data, most of which is abandoned to free space for the next phase. The preserved data are exemplars used for replaying. However, existing methods use a static and ad hoc strategy for memory allocation, which is often sub-optimal. In this work, we propose a dynamic memory management strategy that is optimized for the incremental phases and different object classes. We call our method reinforced memory management (RMM), leveraging reinforcement learning. RMM training is not naturally compatible with CIL as the past, and future data are strictly non-accessible during the incremental phases. We solve this by training the policy function of RMM on pseudo CIL tasks, e.g., the tasks built on the data of the 0-th phase, and then applying it to target tasks. RMM propagates two levels of actions: Level-1 determines how to split the memory between old and new classes, and Level-2 allocates memory for each specific class. In essence, it is an optimizable and general method for memory management that can be used in any replaying-based CIL method. For evaluation, we plug RMM into two top-performing baselines (LUCIR+AANets and POD+AANets [28]) and conduct experiments on three benchmarks (CIFAR-100, ImageNet-Subset, and ImageNet-Full). Our results show clear improvements, e.g., boosting POD+AANets by 3.6%, 4.4%, and 1.9% in the 25-Phase settings of the above benchmarks, respectively. The code is available at https://gitlab.mpi-klsb.mpg.de/yaoyaoliu/rmm/. 2021-12-01T08:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/6790 https://ink.library.smu.edu.sg/context/sis_research/article/7793/viewcontent/NeurIPS2021_Submission_Class_Incremental_Learning__2___1_.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Artificial Intelligence and Robotics Databases and Information Systems |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
Artificial Intelligence and Robotics Databases and Information Systems |
spellingShingle |
Artificial Intelligence and Robotics Databases and Information Systems LIU, Yaoyao SUN, Qianru SUN, Qianru RMM: Reinforced memory management for class-incremental learning |
description |
Class-Incremental Learning (CIL) [38] trains classifiers under a strict memory budget: in each incremental phase, learning is done for new data, most of which is abandoned to free space for the next phase. The preserved data are exemplars used for replaying. However, existing methods use a static and ad hoc strategy for memory allocation, which is often sub-optimal. In this work, we propose a dynamic memory management strategy that is optimized for the incremental phases and different object classes. We call our method reinforced memory management (RMM), leveraging reinforcement learning. RMM training is not naturally compatible with CIL as the past, and future data are strictly non-accessible during the incremental phases. We solve this by training the policy function of RMM on pseudo CIL tasks, e.g., the tasks built on the data of the 0-th phase, and then applying it to target tasks. RMM propagates two levels of actions: Level-1 determines how to split the memory between old and new classes, and Level-2 allocates memory for each specific class. In essence, it is an optimizable and general method for memory management that can be used in any replaying-based CIL method. For evaluation, we plug RMM into two top-performing baselines (LUCIR+AANets and POD+AANets [28]) and conduct experiments on three benchmarks (CIFAR-100, ImageNet-Subset, and ImageNet-Full). Our results show clear improvements, e.g., boosting POD+AANets by 3.6%, 4.4%, and 1.9% in the 25-Phase settings of the above benchmarks, respectively. The code is available at https://gitlab.mpi-klsb.mpg.de/yaoyaoliu/rmm/. |
format |
text |
author |
LIU, Yaoyao SUN, Qianru SUN, Qianru |
author_facet |
LIU, Yaoyao SUN, Qianru SUN, Qianru |
author_sort |
LIU, Yaoyao |
title |
RMM: Reinforced memory management for class-incremental learning |
title_short |
RMM: Reinforced memory management for class-incremental learning |
title_full |
RMM: Reinforced memory management for class-incremental learning |
title_fullStr |
RMM: Reinforced memory management for class-incremental learning |
title_full_unstemmed |
RMM: Reinforced memory management for class-incremental learning |
title_sort |
rmm: reinforced memory management for class-incremental learning |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2021 |
url |
https://ink.library.smu.edu.sg/sis_research/6790 https://ink.library.smu.edu.sg/context/sis_research/article/7793/viewcontent/NeurIPS2021_Submission_Class_Incremental_Learning__2___1_.pdf |
_version_ |
1770576069458395136 |