隨著計算機視覺的蓬勃發展，高品質且高解析度的深度圖變得越來越加重要。為了即時產生高解析度的深度圖，計算非常密集並且需要大量的硬體資源。
因此在本論文中，我們設計一個基於區塊的信度傳播引擎，此引擎採用大區塊和條件隨機場來用於產生高品質的深度圖。而為了能夠即時估測深度，我們實作基於區塊的信度傳播的超大型積體電路。

有兩種提高深度品質的方法------大區塊和條件隨機場。
使用大區塊的挑戰是大量的晶片內部的記憶體面積的需求。為了解決此困難，我們提出單一個區塊訊息記憶體的改進架構，與最先進的架構相比面積節省37\%。
另一方面，為了增進深度圖的物體邊界附近與細節部分，我們提出基於條件隨機場的自適應斜率方法。根據兩個相鄰像素的強度差自適應地改變斜率，讓視差的平均均方差減少12\%。並且我們為自適應斜率方法設計出代價很少的架構。

我們基於台積電40奈米製程實作深度估測電路，此電路使用了667千位元組的晶片內部的記憶體以及84.6萬的邏輯閘。運作在200 MHz時，它可以每秒提供六千兩百萬視差點的吞吐量來支持每秒30幀的高畫質深度圖。

High-quality and high-resolution depth maps become essential in emerging computer vision applications. Generating high-resolution depth maps is computation-intensive and requires heavy hardware resources for real-time applications. Therefore, in this thesis, we design a tile-based belief propagation engine with large tiles and Conditional random field for high-quality depth maps, and we present a VLSI circuit that applies tile-based belief propagation to support real-time depth estimation.

There are two methods to improve depth quality, large tiles and Conditional random field. The design challenge for using large tiles is the demand of large memory area. To address the difficulty, we propose the improved architecture with only one tile message memory, and the area saving is 37\% compared to the state-of-the-art architecture.
On the other hand, the adaptive slope method based on Conditional random field is proposed to enhance depth map near object boundaries and detailed parts. We adaptively change the slope according to the difference in intensity between two neighboring pixels, and average mean squared error of disparity is reduced by 12\%. And we design architectures with little overhead for the adaptive slope method.

We implement a VLSI circuit for depth estimation using TSMC 40 nm technology process with 667 KB on-chip memory and 846 K gate counts. When synthesized at 200 MHz, it delivers 62 M disparity/s to support full-HD depth at 30 fps.

Contents
Abstract iii
1 Introduction 1
1.1 Motivation 1
1.2 Related Work 2
1.2.1 Markov Random Field and Conditional Random Field 2
1.2.2 Approximate Inference for Markov Random Field 3
1.2.3 Hardware-Friendly Belief Propagation 4
1.3 Thesis Organization 6
2 Algorithm Analysis of Tile-Based Belief Propagation 9
2.1 Belief Propagation 9
2.2 Analysis of Tile-Based Belief Propagation 11
2.2.1 Block-Based and Tile-Based Belief Propagation 11
2.2.2 Tile Size 13
2.2.3 Inner Iteration Times 14
2.3 Conditional Random Field for Belief Propagation 17
2.3.1 Adaptive Slope Method for Belief Propagation 17
2.3.2 Different Kernels for CRF 17
2.3.3 Analysis of CRF with Different Kernels 19
3 System Architecture of Tile-Based Belief Propagation 23
3.1 Analysis of Parallelism 25
3.2 Schedule for Four Parallelism and Fast Message Update Unit 26
3.3 Four-Bank Interleaving Scheme for On-Chip Memory 26
4 Architecture Design of Large Tile Belief Propagation 29
4.1 Four Messages Architecture 29
4.2 Single Message Architecture 30
4.3 Fully-Utilized Single Message Architecture 33
4.4 Comparison 35
5 Architecture Design of Adaptive Slope Method 39
5.1 Fixed-Point Adaptive Slope Method 39
5.2 Pre-Calculated Architecture 40
5.3 On-The-Fly Architecture 42
5.4 Comparison 45
6 Implementation of Tile-Based Belief Propagation 49
6.1 Analysis Bandwidth of Off-Chip Memory for Side Message with
Different Tile Size 49
6.2 Synthesized Result 51
7 Conclusion and Future Work 55

[1] Y. Boykov, O. Veksler, and R. Zabih, "Fast approximate energy minimization via graph cuts," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222–1239, Nov 2001.
[2] Jian Sun, Nan-Ning Zheng, and Heung-Yeung Shum, "Stereo matching using belief propagation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 787–800, July 2003.
[3] O. Veksler, "Stereo correspondence by dynamic programming on a tree," in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), June 2005, vol. 2, pp. 384–390 vol. 2.
[4] H. Hirschmuller, "Accurate and efficient stereo processing by semi-global matching and mutual information," in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), June 2005, vol. 2, pp. 807–814 vol. 2.
[5] V. Kolmogorov, "Convergent tree-reweighted message passing for energy minimization," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 10, pp. 1568–1583, Oct 2006.
[6] John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira, "Conditional random fields: Probabilistic models for segmenting and labeling sequence data," in Proceedings of the Eighteenth International Conference on Machine Learning, San Francisco, CA, USA, 2001, ICML ’01, pp. 282–289, Morgan Kaufmann Publishers Inc.
[7] D. Scharstein and C. Pal, "Learning conditional random fields for stereo," in 2007 IEEE Conference on Computer Vision and Pattern Recognition, June 2007, pp. 1–8.
[8] K. R. Kim and C. S. Kim, "Adaptive smoothness constraints for efficient stereo matching using texture and edge information," in 2016 IEEE International Conference on Image Processing (ICIP), Sept 2016, pp. 3429–3433.
[9] Y. C. Tseng, N. Chang, and T. S. Chang, "Low memory cost block-based belief propagation for stereo correspondence," in 2007 IEEE International Conference on Multimedia and Expo, July 2007, pp. 1415–1418.
[10] C. K. Liang, C. C. Cheng, Y. C. Lai, L. G. Chen, and H. H. Chen, "Hardware-efficient belief propagation," IEEE Transactions on Circuits and Systems for Video Technology, vol. 21, no. 5, pp. 525–537, May 2011.
[11] Y. C. Tseng and T. S. Chang, "Architecture design of belief propagation for real-time disparity estimation," IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 11, pp. 1555–1564, Nov 2010.
[12] S. S. Wu, H. H. Chen, C. H. Tsai, and L. G. Chen, "Memory efficient architecture for belief propagation based disparity estimation," in 2015 IEEE International Symposium on Circuits and Systems (ISCAS), May 2015, pp. 2521–2524.
[13] D. Scharstein, R. Szeliski, and R. Zabih, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," in Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001), 2001, pp. 131–140.
[14] Gabriele Facciolo, Carlo De Franchis, and Enric Meinhardt, "MGM: A significantly more global matching for stereovision," in BMVC 2015, 2015.
[15] C. C. Cheng, C. T. Li, C. K. Liang, Y. C. Lai, and L. G. Chen, "Architecture design of stereo matching using belief propagation," in Proceedings of 2010 IEEE International Symposium on Circuits and Systems, May 2010, pp. 4109– 4112.
[16] H. H. Chen, C. T. Huang, S. S. Wu, C. L. Hung, T. C. Ma, and L. G. Chen, "23.2 A 1920x1080 30fps 611 mw five-view depth-estimation processor for light-field applications," in 2015 IEEE International Solid-State Circuits Conference (ISSCC) Digest of Technical Papers, Feb 2015, pp. 1–3.
[17] T. Yu, R. S. Lin, B. Super, and B. Tang, "Efficient message representations for belief propagation," in 2007 IEEE 11th International Conference on Computer Vision, Oct 2007, pp. 1–8.
[18] Y. C. Lai, C. C. Cheng, C. K. Liang, and L. G. Chen, "Efficient message reduction algorithm for stereo matching using belief propagation," in 2010 IEEE International Conference on Image Processing, Sept 2010, pp. 2977– 2980.
[19] P. F. Felzenszwalb and D. R. Huttenlocher, "Efficient belief propagation for early vision," in Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., June 2004, vol. 1, pp. I–261–I–268 Vol.1.
[20] S. F. Hsiao, J. M. Huang, and P. S. Wu, "VLSI implementation of belief-propagation-based stereo matching with linear-model message update," in 2014 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS), Nov 2014, pp. 73–76.
[21] C. C. Cheng, C. K. Liang, Y. C. Lai, H. H. Chen, and L. G. Chen, "Fast belief propagation process element for high-quality stereo estimation," in 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, April 2009, pp. 745–748.
[22] S. S. Wu, C. H. Tsai, and L. G. Chen, "Efficient hardware architecture for large disparity range stereo matching based on belief propagation," in 2016 IEEE International Workshop on Signal Processing Systems (SiPS), Oct 2016, pp. 236–241.
[23] S. Wanner, S. Meister, and B. Goldlücke, "Datasets and benchmarks for densely sampled 4d light fields," in Vision, Modeling & Visualization, 2013, pp. 225–226.
[24] H. H. Chen, Light-field Processing System for Depth-based Real-time Video Applications, Ph.D. thesis, National Taiwan University, 2015.