深度學習在各種領域已有重大的突破。然而，如果要讓機器有像人類一般的學習能力，像是終身學習中，要在避免災難性遺忘(catastrophic forgetting)的情況下有效率地學習新事物、或甚至是根據稀少的資訊快速學習新知等，尚還有一段路要走。為了解決如此的問題，本論文根據腦科學理論提出了「快思慢想者」模型來實現機器的終身學習與少樣本學習能力。腦神經科學理論中，影響大腦做決定與運作包含了兩個系統：一是一個快速、直覺式的反應家、一是緩慢、但深思熟慮的決策家。因此，我們的快思慢想者模型也包含了兩個部分：(一)一個是記憶擴充的快速類神經網絡，稱作「快思者」(Fast Thinker, FT)、(二)二是較慢的自適應類神經網絡，稱作「慢想者」(Slow Thinker, ST)，被訓練為測試時有能力輕易適應所有任務的網絡。上述兩個網絡系統透過慢想者反饋給快思者的訊息相輔相成，使得快思者能學習如何給提供慢想者所需的表徵。此外，快思者與慢想者是同時進行交錯的訓練、讓兩者的互動反饋達到最好的效果。本論文使用置換MNIST的任務以及使用CIFAR-100資料庫製作的更加困難的連續任務進行實驗，效果成功超越現行最佳的其他研究。甚至只需使用極小的記憶容量，慢想者可以廣泛性的應用到所有的任務上、並且沒有災耐性遺忘的問題。整體而言，快思慢想者模型不只在運行時適應(runtime-adaptation)的效率比其他模型高，且只需使用極少數的記憶容量，就能在機器的終生學習任務中擁有極佳的表現、且在少樣本學習中也有不錯的能力。

Deep Learning has achieved breakthroughs in many specialized domains; however, it is still limited when it comes to learning like humans, i.e. effective learning throughout lifetime without catastrophic forgetting and rapid learning even from little information. To contribute in bridging this gap, we propose the Fast-and-Slow Thinker (FST) model for lifelong and few-shot learning, which is inspired by how the brain works. It is believed that for consolidating memory and making life decisions, the brain is acted upon by two systems: a quick, intuitive thinker and a slow, more effortful decision-maker. FST's key components are 1) a per-task memory-augmented network, simply referred to as Fast Thinker (FT), and 2) a slow, adaptive network called Slow Thinker (ST) that is trained to be easily fine-tuned during inference time. The two systems work together through a form of feedback from the slow thinker that guides the fast thinker in representation learning. Furthermore, FT and ST are jointly alternately trained to encourage more interaction during learning. Using the permuted MNIST task and more difficult sequential tasks created using CIFAR-100, FST outperforms the state-of-the-art for lifelong learning task. Furthermore, even with very small memory size (saving at least 10 examples per task), ST can generalize across tasks and does not suffer from forgetting. Overall, FST demonstrated high accuracy in lifelong problem, decent performance in few-shot tasks, and with lower memory storage consumption and more effective runtime adaptation compared to baseline models.

摘要 ---------------------------------------------------- i
Abstract -------------------------------------------------- iii
Acknowledgement -------------------------------------- v
List of Tables ------- -------------------------------------- x
List of Figures -------------------------------------------- xi
1 Introduction ------------------------------------------ 1
2 Related Work ---------------------------------------- 9
2.1 Few-shot Learning --------------------------------- 9
2.1.1 Memory-augmented Neural Networks --------- 10
2.1.2 Meta-learning ----------------------------------- 11
2.2 Lifelong Learning ----------------------------------- 11
2.3 Lifelong Few-shot Learning ------------------------ 13
3 Fast-and-Slow Thinker (FST) ------------------------ 15
3.1 Predicting ----------------------------------------16
3.2 FST Framework -------------------------------------16
3.3 FST Optimization ------------------------------------ 17
3.4 Technical Details ----------------------------------- 23
3.4.1 Alternate training --------------------------------- 23
3.4.2 Approximation of higher-order derivatives -------23
3.4.3 Slow thinker’s hyperparameters ---------------- 24
4 Experimental Evaluation ------------------------------ 25
4.1 Lifelong (Continual) Learning ----------------------- 25
4.1.1 Permuted MNIST ----------------------------------27
4.1.2 CIFAR-100 Sequential Task -------------------------30
4.2 Few-shot Learning --------------------------------- 32
4.3 More Insights on FST ---------------------------------35
4.3.1 Different Alternate Training Step Ratio (FT:ST) -----36
4.3.2 Different Number of Runtime Adaptation Steps ---38
4.3.3 Different Combination of the Number of k-Nearest Neighbors and Runtime Adaptation Steps that ST used --------------- 39
5 Conclusion and Future Work ------------------------- 40
5.1 Conclusion ------------------------------------------- 40
5.2 Future work -------------------------------------------41
References -------------------------------------------------43

[1] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. In In Advances in neural information processing systems, 2012.
[2] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep resid- ual learning for image recognition. In In IEEE Conference on Com- puter Vision and Pattern Recognition, 2015.
[3] Karen Simonyan and Andrew Zisserman. Very deep convolutional net- works for large-scale image recognition. In ICLR, 2015.
[4] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for com- puter vision. In CVPR, 2016.
[5] Soroush Mehri, Kundan Kumar, Ishaan Gulrajani, Rithesh Kumar,


Shubham Jaina, Jose Sotelo, Aaron Courville, and Yoshua Bengio. Samplernn: An unconditional end-to-end neural audio generation model. 2016.
[6] Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew W. Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. In CoRR, 2016.
[7] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, An- dreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, and Shane Legg & Demis Hassabis. Human-level con- trol through deep reinforcement learning. In Nature, 2015.
[8] Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learn- ing. In Proceedings of the 33nd International Conference on Machine Learning, 2016.


[9] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepe, and Demis Hassabis. Mastering the game of go without human knowledge. In Nature, 2017.
[10] Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaob- ing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. Google’s neural machine translation system: Bridging the gap between human and machine translation. 2016.
[11] James McClelland, Bruce McNaughton, and Randall O’Reilly. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist


models of learning and memory. Psychological Review, 1995.

[12] Michael McCloskey and Neal J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. The Psy- chology of Learning and Motivation, 1989.
[13] Brenden Lake, Ruslan Salakhutdinov, Jason Gross, and Joshua B Tenenbaum. One shot learning of simple visual concepts. In In Pro- ceedings of the 33rd Annual Conference of the Cognitive Science So- ciety, 2011.
[14] Fei-Fei Li, Rob Fergus, and Pietro Perona. One-shot learning of object categories. In PAMI, 2006.
[15] Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta-learning with memory-augmented neural networks. In 33nd International Conference on Machine Learning, 2016.
[16] Chelsea Finn, Pieter Abbeel, and Sergy Levine. Model-agnostic meta- learning for fast adaptation of deep networks. In International Confer- ence on Machine Learning, 2017.


[17] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In NIPS, 2016.
[18] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Ve- ness Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcom- ing catastrophic forgetting in neural networks. In Proceedings of the National Academy of Sciences, 2017.
[19] Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Life- long learning with dynamically expandable networks. In International Conference on Learning Representations, 2018.
[20] Ronald Kemker and Christopher Kanan. Fearnet: Brain-inspired model for incremental learning. international conference on learning representations. In International Conference on Learning Representa- tions, 2018.
[21] Alexander Gepperth and Cem Karaoguz. A Bio-inspired Incremen-

tal Learning Architecture for Applied Perceptual Problems. Cognitive


Computation, Springer, 2016.

[22] Ronald Kemker, Marc McClure, Angelina Abitino, Tyler Hayes, and Christopher Kanan. Measuring catastrophic forgetting in neural net- works. In AAAI, 2018.
[23] Lukasz Kaiser, Ofir Nachum, Aurko Roy, and Samy Bengio. Learning to remember rare events. In International Conference on Learning Representations, 2017.
[24] German Parisi, Ronald Kemker, Jose Part, Christopher Kanan, and Ste- fan Wermter. Continual lifelong learning with neural networks: A re- view. 2018.
[25] Daniel Kahneman. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011.
[26] Dharshan Kumaran, Demis Hassabis, and James L. McClelland. What learning systems do intelligent agents need? complementary learning systems theory updated. Trends Cogn Sci, 2016.
[27] Pablo Sprechmann, Siddhant Jayakumar, Jack Rae, Alexander Pritzel,


Adria Puigdomenech Badia, Benigno Uri, Oriol Vinyals, Demis Hass- abis, Razvan Pascanu, and Charles Blundell. Memory-based parameter adaptation. In International Conference on Learning Representations, 2018.
[28] Sachin Ravi and Hugo Larochelle. Optimization as a model for few- shot learning. In ICLR, 2017.
[29] Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung-Woo Ha, and Byoung-Tak Zhang. Overcoming catastrophic forgetting by incremen- tal moment matching. In NIPS, 2017.
[30] David Lopez-Paz and Marc’ Aurelio Ranzato. Gradient episodic mem- ory for continual learning. In NIPS,, 2017.
[31] Zhizhong Li and Derek Hoiem. Learning without forgetting. In Euro- pean Conference on Computer Vision - ECCV,, 2016.
[32] Xu He and Herbert Jaeger. Overcoming catastrophic interference us- ing conceptor-aided backpropagation. In International Conference on Learning Representations, 2018.


[33] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In ICML, 2017.
[34] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Lifelong few-shot learning. ICML, 2017. URL http://people.
eecs.berkeley.edu/˜cbfinn/_files/icml2017_ llworkshop.pdf.