Intractability of Continual Learning using Parametric Models with Unlimited Replay BufferResearch Paper
In continual learning a model is trained on a stream of tasks and must keep performing well on every task seen so far; losing accuracy on earlier tasks is called catastrophic forgetting. A common remedy is a replay buffer that stores past data. With an unlimited buffer every earlier dataset remains available, so forgetting can no longer be…