隨著科技的進步，在我們日常生活中帶有雙鏡頭的智慧型手機變得逐漸普
及，而且有了深度的資訊，許多應用與演算法都可以達到更好的效能與準確度，
為了提升深度圖的品質，我們參考五個視點的光場資訊來計算成本函數，這樣
的成本函數會比只使用兩個視點的更加穩固可靠。另外，由於搭配高解析度相
機的嵌入式系統有變多的趨勢，導致深度廣泛的景象也越來越常見，因此成本
函數計算所支援的視差範圍需要進一步提升，但越多視點的使用和對於支援廣
泛深度範圍的需求，會導致使用的記憶體資源劇烈增加，因此在這篇論文中，
我們提出了一個高記憶體使用效率的成本函數計算設計來解決這些問題。

我們的目標是實作一個支援高畫質每秒三十幀的成本函數計算引擎，且其所
支援的視差範圍為0 像素到128 像素，由於這樣的規格會導致記憶體需求量遽
增。我們採用鬆散光場金字塔以及搭配必要讀取法的二元標籤來節省動態隨機
存取記憶體的頻寬和存儲垂直視點的晶片內部記憶體；而個別計算順序的提出
是為了最大化下採樣像素的重複使用效率進而節省晶片內部記憶體的頻寬與面
積；另外，我們還開發出一個高面積使用效率的下採樣模組——先進先出收縮
暫存器，其能夠減少95% 的下採樣模組的邏輯閘數。

最後，應用於即時高畫質廣泛成本函數計算的超大型積體電路設計是透過台
積電40 奈米製程所合成，總共使用了212.2k 個邏輯閘以及25.5-KB 的晶片內
部記憶體，而在使用五個視點光場資訊的情況底下，晶片外部記憶體的能源消
耗為156.6mW。

The dual cameras on the cellphones are more and more common in our daily life. With the depth information, many algorithms and applications can achieve better performance and accuracy. In order to raise the quality of depth map, we use five-view light fields for cost volume generation and its matching costs are more robust than that in the two-view stereo matching system. Moreover, since the high disparity range scenes are becoming popular as the trend of the high-resolution cameras on the embedded systems arises, the disparity range for cost volume generation needs to be raised. When more views are used for disparity estimation and the high-range depth estimation is required, the requirement to memory resources is increasing dramatically. Therefore, in this thesis, we present a memory-efficient cost volume generation design to solve these problems.

We aim to implement a Full-HD cost volume generation engine which disparity range is from zero- to 128-pixel at 30 fps. Due to the requirement of memories is extremely high, we adopt dyadic labeling with essential accessing in sparse light field pyramid to save DRAM bandwidth and the size of vertical views on-chip memories. Respective order is proposed to maximizing down-sampled pixels reuse efficiency to save the bandwidth and area of on-chip memories. The area-efficient down-sample module, shrinking FIFO, is developed to reduce the logic gates of the down-sampled modules by 95%.

The synthesized VLSI design using TSMC 40nm technology process for real-time Full-HD (1080p) high-range cost volume generation has 212.2k gates and 25.5-KB on-chip memory, and the off-chip memory consumes 156.6 mW in five-view light fields configuration.

Abstract v
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 Active Sensors . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.2 Stereo Matching with Two Views . . . . . . . . . . . . . . . 3
1.2.3 Stereo Matching with Multiple Views . . . . . . . . . . . . . 6
1.3 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 Architecture of Cost Volume Generation 9
2.1 Algorithm of Cost Volume Generation . . . . . . . . . . . . . . . . 10
2.2 Area-Efficient Cost Volume PE Design . . . . . . . . . . . . . . . . 10
3 DRAM Resource Analysis for Sparse Light Field Pyramid 15
3.1 Dyadic Labeling and Sparse Light Field Pyramid . . . . . . . . . . 15
3.2 Optimization for DRAM Power Consumption . . . . . . . . . . . . 19
3.2.1 Essential Accessing for Reducing DRAM Bandwidth . . . . 19
3.2.2 Optimized DRAM Mapping for Tile-Based Architecture . . 21
4 Data Reuse Optimization for Area-Efficient SRAM 25
4.1 Inter-Tile and Intra-Tile Data Reuse . . . . . . . . . . . . . . . . . 25
4.2 Analysis of Data Reuse Efficiency in Prior Art . . . . . . . . . . . . 26
4.2.1 Analysis of Inter-Tile Data Reuse . . . . . . . . . . . . . . . 28
4.2.2 Analysis of Intra-Tile Data Reuse . . . . . . . . . . . . . . . 28
4.3 Respective Order for Enhancing Data Reuse Efficiency to Reduce
SRAM Area . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.4 Respective Order with Dyadic Labeling and Sparse Light Field
Pyramid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.4.1 Maximizing Down-Sampled Pixels Reuse Efficiency . . . . . 35
4.4.2 Down-Sample Engine Optimization . . . . . . . . . . . . . . 36
4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5 VLSI Implementation 43
5.1 Truncated Cost Volume for Area-Efficient On-Chip Memory . . . . 43
5.2 Synthesized Result . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6 Conclusion and Future Work 49

[1] X. Mei, X. Sun, M. Zhou, S. Jiao, H. Wang, and Xiaopeng Zhang, “On building an accurate stereo matching system on graphics hardware,” in 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Nov 2011, pp. 467–474.
[2] H. H. Chen, C. T. Huang, S. S. Wu, C. L. Hung, T. C. Ma, and L. G. Chen, “23.2 a 1920×1080 30fps 611 mw five-view depth-estimation processor for light-field applications,” in 2015 IEEE International Solid-State Circuits Conference - (ISSCC) Digest of Technical Papers, Feb 2015, pp. 1–3.
[3] Hong-Hui Chen, Light-field Processing System for Depth-based Real-time Video Applications, Ph.D. thesis, National Taiwan University, 2015.
[4] M. Levoy, “Light fields and computational imaging,” Computer, vol. 39, no. 8, pp. 46–55, Aug 2006.
[5] S. B. Gokturk, H. Yalcin, and C. Bamji, “A time-of-flight depth sensor - system description, issues and solutions,” in 2004 Conference on Computer Vision and Pattern Recognition Workshop, June 2004, pp. 35–35.
[6] H. Hirschmuller and D. Scharstein, “Evaluation of stereo matching costs on images with radiometric differences,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 9, pp. 1582–1599, Sept 2009.
[7] S. Birchfield and C. Tomasi, “A pixel dissimilarity measure that is insensitive to image sampling,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 401–406, Apr 1998.
[8] N. Y. C. Chang, T. H. Tsai, B. H. Hsu, Y. C. Chen, and T. S. Chang, “Algorithm and architecture of disparity estimation with mini-census adaptive support weight,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 6, pp. 792–805, June 2010.
[9] J. Ko and Y. S. Ho, “Stereo matching using census transform of adaptive window sizes with gradient images,” in 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), Dec 2016, pp. 1–4.
[10] A. Hosni, C. Rhemann, M. Bleyer, C. Rother, and M. Gelautz, “Fast costvolume filtering for visual correspondence and beyond,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 2, pp. 504–511, Feb 2013.
[11] C. Ttofis and T. Theocharides, “High-quality real-time hardware stereo matching based on guided image filtering,” in 2014 Design, Automation Test in Europe Conference Exhibition (DATE), March 2014, pp. 1–6.
[12] D. W. Yang, L. C. Chu, C. W. Chen, J. Wang, and M. D. Shieh, “Depth reliability-based stereo-matching algorithm and its vlsi architecture design,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 6, pp. 1038–1050, June 2015.
[13] Stan Z Li, Markov random field modeling in image analysis, Springer Science & Business Media, 2009.
[14] Y. Y. Boykov and M. P. Jolly, “Interactive graph cuts for optimal boundary amp; region segmentation of objects in n-d images,” in Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, 2001, vol. 1, pp. 105–112 vol.1.
[15] Jian Sun, Nan-Ning Zheng, and Heung-Yeung Shum, “Stereo matching using belief propagation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 787–800, July 2003.
[16] S. Wanner and B. Goldluecke, “Globally consistent depth labeling of 4d light fields,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, June 2012, pp. 41–48.
[17] L. Si, Q. Wang, and Z. Xiao, “Matching cost fusion in dense depth recovery for camera-array via global optimization,” in 2014 International Conference on Virtual Reality and Visualization, Aug 2014, pp. 180–185.
[18] S. Wanner and B. Goldluecke, “Variational light field analysis for disparity estimation and super-resolution,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 3, pp. 606–619, March 2014.
[19] C. Chen, H. Lin, Z. Yu, S. B. Kang, and J. Yu, “Light field stereo matching using bilateral statistics of surface cameras,” in 2014 IEEE Conference on Computer Vision and Pattern Recognition, June 2014, pp. 1518–1525.
[20] Changil Kim, Henning Zimmer, Yael Pritch, Alexander Sorkine-Hornung, and Markus Gross, “Scene reconstruction from high spatio-angular resolution light fields,” ACM Transactions on Graphics, vol. 32, no. 4, pp. 1, jul 2013.
[21] Q. Yang, “A non-local cost aggregation method for stereo matching,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, June 2012, pp. 1402–1409.
[22] X. Mei, X. Sun, W. Dong, H. Wang, and X. Zhang, “Segment-tree based cost aggregation for stereo matching,” in 2013 IEEE Conference on Computer Vision and Pattern Recognition, June 2013, pp. 313–320.
[23] “Micron, DDR3 SDRAM System-Power Calculator,” https://www.micron.com/support/tools-and-utilities/power-calc, Accessed: 2017-06-10.
[24] “The heidelberg collaboratory for image processing (hci),” https://hci.iwr.uni-heidelberg.de/benchmarks, Accessed: 2017-07-05.
[25] C. C. Cheng, C. T. Li, C. K. Liang, Y. C. Lai, and L. G. Chen, “Architecture design of stereo matching using belief propagation,” in Proceedings of 2010 IEEE International Symposium on Circuits and Systems, May 2010, pp. 4109–4112.