近年來，類神經網路隨著深度學習的發展而受到矚目，成為了最熱門的話題之一，相較於現代電腦在運算功能上有突出的表現，人腦則是在圖像辨識、感知、功率消耗上遠勝於電腦，如果要建構一個擁有相同優勢的電腦，模仿大腦的架構是最直接的方法，因此突波式神經網路則被提出來，其主要傳遞的訊號為動作電位(突波)，除此之外，透過生物上的量測結果發現，雜訊會使得神經元產生隨機性的突波行為，而更有文獻顯示雜訊有助於突波式神經網路的學習，因此提供了一個新穎的想法對於設計單一神經元電路。
本論文的研究重點是設計一個具有隨機性突波行為的神經元電路並使其成為大型神經網路的運算單元，而此神經元電路的雜訊源來自一個可調變雜訊能力的電晶體，由於此雜訊電晶體與生物的雜訊擁有相同的頻譜特徵，因此我們可以透過此元件自身的雜訊影響神經元電路的突波頻率以達到隨機行為的實現，其中我們所使用的突波神經元模型為 Izhikevich 模型，其最大的特色是可以用兩個微分方程式以及四個參數完成多樣的神經元突波波型。因此本篇論文提出一個具有多樣性波型且具隨機行為的神經元電路用以建構大型神經網路及演算法所使用。

With the development of deep-learning Artificial Neural Network(ANN) attracts attention and becomes one of the hottest topics recently. Compared to the good performance of modern computer on computation, the human brain has advantages on pattern recognition, sense, and power consumption. If we want to build a computer which has the same characteristics as the brain, the direct way is to mimic the architecture of the brain. The Spiking Neural Network(SNN) was therefore proposed and the main signals transmitting between neurons are spikes. Besides, according to the experimental results of a biological neuron, scientists found that noise induces the stochastic behavior of spiking pattern. Moreover, in some literature, the noise is also helpful for SNN during the learning progress. Thus, It gives a new thought to design a neuron circuit.
The main purpose of this thesis is to design a neuron circuit with stochastic behavior and make it as a cell of a large-scale neural network. The noise source of neuron circuit comes from the noise-adaptable transistor whose frequency spectrum is similar to the noise in the real neuron. Therefore, we can implement the stochastic behavior of spiking frequency through the internal noise of noise-adaptable transistor. In particular, Izhikevich model is employed to our proposed silicon neuron. The feature of Izhikevich model is that it can reproduce different kinds of spiking behavior with two first-order differential equations and four parameters. Thus, this thesis is to propose a silicon neuron which has rich neuronal dynamics and stochastic behavior and use it to build a large-scale neural network.

Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Design Flow and Chapter Layout 2
Chapter 2 Theory Description and Literature Review 3
2.1 Characteristics of biological neuron 3
2.1.1 The ion channel and noise of neuron 3
2.1.2 Spike-Time-Dependent-Plasticity 5
2.2 The introduction of some widely used neuron models 6
2.2.1 Leaky Integrate-and-Fire Model 6
2.2.2 Integrate-and-Fire Model 7
2.2.3 Izhikevich Model 8
2.3 The hardware architecture of some widely used neuron models 9
2.3.1 Current mode filter 10
2.3.1.1 Translinear principle 10
2.3.1.2 DPI 11
2.3.1.3 Tau-Cell 12
2.3.2 I&F model 13
2.3.2.1 Pesudo-conductance silicon neuron 13
2.3.2.3 The conductance-based silicon neuron 14
2.3.3 Izhikevich model 16
2.3.3.1 Current mode 16
2.3.3.1 Voltage mode 19
2.4 Noise-adaptable transistor 20
Chapter 3 The mathematical analysis of Izhikevich model 22
3.1 The threshold of Izhikevich model 22
3.2 The crucial factor affecting the spiking frequency most 23
3.3 The specification of noise source 24
Chapter 4 An Izhikevich silicon neuron 26
4.1 Current mode filter 26
4.1.1 Tau-Cell 26
4.1.2 DPI(Differential Pair Integrator) 27
4.2 Silicon neuron 28
4.2.1 Circuit Description 28
4.2.2 Operation modes 30
4.2.3 The Kappa Factor 31
4.3 Bias circuit 32
4.3.1 Bootstrapped current reference 32
4.3.2 Current Splitter 33
4.3.3 Tunable circuit 35
4.4 Simulation Results and measurement considerations 35
Chapter 5 Silicon neuron with noise-adaptable transistors 45
5.1 The I&F silicon neuron with noise-adaptable transistors 45
5.2 The noise specification of Izhikevich silicon neuron 46
5.3 Noisy circuit design 48
5.3.1 The estimated model of RPOFET in TN28HPM 48
5.3.2 Design description 51
5.3.3 Simulation results 52
Chapter 6 The spiking neuron chip with multiple neurons 54
6.1 The interface of silicon neuron 54
6.2 The interface circuit design of Izhikevich neuron 55
6.3 The detailed information of whole chip 56
6.4 Conclusion 62
6.5 Future work 63
References 64

[1] Neuron. (2009, August 22). In Wikipedia, the free encyclopedia. Retrieved from
https://en.wikipedia.org/wiki/Neuron
[2] Membrane Potential. (2011, October 26). In Wikipedia, the free encyclopedia. Retrieved from https://en.wikipedia.org/wiki/Membrane_potential
[3] Action Potential. (2011, October 26). In Wikipedia, the free encyclopedia. Retrieved from https://en.wikipedia.org/wiki/Action_potential
[4] White JA, Rubinstein JT, Kay AR, “Channel noise in neurons”, Elsevier Science, TINS vol. 23, No. 3,2000
[5] Wiesenfeld, K. and Moss, F. "Stochastic resonance and the benefits of noise: from ice ages to crayfish and SQUIDs," Nature, vol. 373, pp. 33-36, 1995.
[6] Casado, J. M., "Synchronization of two Hodgkin-Huxley neurons due to internal noise", Physics Letters A, vol. 310, no. 5-6, pp. 400-406, 2003.
[7] Dan.Y, Mu-ming Poo, “ Spike Timing-Dependent Plasticity of Neural Circuits”, Neuron, Vol.44,23-30,2004.
[8] Ma, W. J., Beck, J. M., Latham, P. E., and Pouget, A., "Bayesian inference with probabilistic population codes", Nature Neuroscience, vol. 9, no. 11, pp. 1432-1438, Nov. 2006.
[9] Gerstner, W., M. Kistler, W., Naud, R., Paninski, L. “ Neuronal Dynamics”.
[10] Izhikevich, E.M. “ Hybrid spiking models”,Phil. Trans R. Soc A, 368, 5061-5070, 2010.
[11] Izhikevich, E.M. “ Which Model to Use for Cortical Spiking Neurons ?”, IEEE TRANSACTION ON NEURAL NETWORKS,Vol.15, No. 5, Sep, 2004.
[12] Izhikevich, E.M. “ Simple Model of Spiking Neurons”, IEEE TRANSACTION ON NEURAL NETWORKS,Vol.14, No. 6, Nov, 2003.
[13] Gilbert.B “ Translinear circuits: Aproposed classification” IEEE ELECTRONICS Letters,Vol.11,14-16, Jan,1975.
[14] Mead, C.A. “ Analog VLSI: Circuits and Principles”
[15] Indiveri,G., Stefanini,F, Chicca, E. “ Spike-based learning with a generalized integrate and fire silicon neuron” IEEE Circuits and Systems(ISCAS),2010.
[16] Chicca, E., Stefanini,F,Bartolozzi, C. “ Neuromorphic Electronic Circuits for building Autonomous Cognitive Systems”.Proc. IEEE102,1367-1388.doi:10.1109, 2014.
[17] Schaik, A.V., Jin.C, McEwan, A.” A log-domain implementation of the Izhikevich neuron model.”, IEEE International Symposium on Circuits and Systems,2010.
[18] Rangan, V., Ghosh, A., Aparin,V., Cauwenberghs,G. “ A subthreshold aVLSI Implementation of Izhikevich Simple Neuron Model”, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. 4164-4167,2010.
[19] Demirkol, A.S., Ozoguz, S. “ A Low Power VLSI Implementation of Izhikevich Neuron Model.” 2011 IEEE 9th International New Circuits and systems conference.
[20] Yu, T., Park, J., Joshi,S., Maier,C., Cauwenberghs, G. “65k-Neuron I&F Array Transceiver with Address-event Reconfigurable Synaptic Routing” 2012 IEEE Biomedical Circuits and Systems Conference(Biocas).
[21] Livi,P., Indiveri,G. “A current-mode conductance-based silicon neuron for the Address-Event neuromorphic system “ 2009 IEEE International Symposium on Circuits and Systems.
[22] Wijekoon, J.H.B., Dudek, P. “ Spiking and Bursting Firing Patterns of a Compact VLSI Cortical Neuron Circuit” 2007 International Joint Conference on Neural Networks. 1332-1337.
[23] Wijekoon, J.H.B., Dudek, P.” Compact silicon neuron circuit spiking and bursting behavior”, Elsevier Science, Neural Networks, Vol.21, 524-534,(2007).
[24] Yu, T., Park, J., Joshi,S., Neftci,E., Cauwenberghs, G.” A65k-Neuron 73-Mevents/s 22-pJ/event Asynchronous Micro-Pipelined I&F Array transceiver”,2014 IEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings, 675-678(2014).
[25] Bezrukov,S.M,Winterhalter, M. , “Examining Noise Sources at the Single-Molecule Level 1f Noise of an Open Maltoporin Channel”, PHYSICAL REVIEW LETTERS, Vol. 85, No. 1,Sep, 2000.
[26] T.J. Chiu, Y.C. King, J. Gong, Y.H. Tsai, and H. Chen* “A Resistive-Protective-Oxide Transistor with Adaptable Low-frequency Noise for Stochastic Neuromorphic Computation in VLSI” the IEEE Electron Device Letters, vol.32, no.9, pp.1293-1295, 2011
[27] Y.D.Wu and H.Chen, “The Diffusion Network in Analog VLSI Exploiting Noise-Induced Stochastic Dynamics to Regenerate Various Continuous Paths”, IEEE Transactions on Circuits and Systems-I, Vol.62, No.6, 2015.
[28] Delbruck, T., Schaik, A.V., “Bias current generators with wide dynamic range”, IEEE International Symposium on Circuits and Systems,Vol.1, 337-340, 2004.
[29] Jason Huang,” Noise Controlled Transistor Modeling and Application Circuit ”, 2016.
[30] K. A. Boahen, "Point-to-Point Connectivity Between Neuromorphic Chips Using Address Events," IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS-II: ANALOG AND DIGITAL SIGNAL PROCESSING, vol. 47, pp. 416-434, 2000.