本研究選取H.264/AVC為實驗平台，H.264錯誤遮隱的方法係採用平均像素內插法(pixel mean interpolation)來改善收到錯誤資訊畫面的品質，但此平台的內建演算法有時無法得到較佳的視訊品質(例如會有邊緣模糊或方塊的效應產生)為了解決此問題，本論文提出了一個新型的空間時域錯誤遮隱方法，此方法可同時適用於I-frame及P-frame，有別於以往的錯誤遮隱方法只能適用其一或者偏重於時間域或空間域的演算法，造成相對高複雜度運算的問題。此新方法運用邊緣匹配與整體螺旋以及小區塊(sub-blocks)自體螺旋，配合邊緣垂直平分軸對稱重建法找出邊緣的相關性並於畫面內做錯誤遮隱，改善了當損壞的巨方塊有邊緣的資訊時，先前方法不能將它重建回來的缺失，亦可處理損壞的巨方塊內有一個以上邊緣的情況。
本文所提的演算法所得之結果較邊界匹配演算法(Boundary Matching Algorithm, BMA)及空間時域邊界匹配演算法(Spatio-temporal Boundary Matching Algorithm, STBMA)為精準，且計算上也不會增添額外的運算複雜度。此外，本論文同時提出一個邊緣比例門檻值分析的新機制，以達到更有效節省演算時所花費的時間。此處的門檻值分析，是指利用本論文所提之方法，在執行邊緣偵測時紀錄並統計損壞的巨方塊內各邊緣像素點於巨方塊內所佔之比重，再經由統整數據歸納後，依據邊緣數量適時選擇所採納之方案。實驗結果顯示，當Hall影像序列的第二張P畫面發生5%的巨方塊錯誤率時，使用本論文提出的方法後，其PSNR值最高可以達到58.99 dB，比BMA和STBMA的方法分別高出1.46 dB及0.72 dB；在10%錯誤率發生時，大部分視頻影像的平均PSNR值也較BMA及STBMA方法為優；在20%錯誤率發生時，對於內容和緩變動的視訊能夠得到良好的錯誤遮隱效果。

This research chooses H.264/AVC as the experimental platform. This paper uses the pixel-mean interpolation (adopted by the error concealment method for the H.264 standard in JM 18.4 reference software) to improve the video quality when the decoded image blocks are partially in error. However, this method may fail to achieve better video quality and produce unwanted effects such as edge blurring effect or blocking artifact. To solve this problem, this study proposes a new spatio-temporal error concealment method which can be applied for I-frame and P-frame. Unlike the previous approaches that can be applied for I-frame or P-frame only. The proposed method applies global spiral, sub-block autologous spiral method and exploits the edge matching information. Based on the principle of edge's perpendicular bisector symmetry reconstruction, system evaluates the potential edges in a corrupted MB and performs the error concealment algorithm for each single frame. The proposed method enables to recovery the fidelity of edge in the corrupted MB, which is brilliant than the previous methods. Based on our observation, it can also recover a corrupted MB containing multi-edges.
The proposed method can provide more accurate result than that of the Boundary Matching Algorithm(BMA), and Spatio-temporal Boundary Matching Algorithm(STBMA) costs a great of computational complexity, resulting in longer processing time. To reduce the computation time, this study also proposes an edge ratio analysis mechanism, where corrupted MBs are either concealed by the proposed method or the pixel-mean interpolation depending on the percentage of the number of the edge pixels in an MB. With this edge threshold analysis mechanism, the edge blurring and blocking effects can be strongly reduced with a slight increasing in computation time. The experimental results show that when Hall video sequence in P frame’s block error rate (BER) is 5%, the peak signal to noise ratio (PSNR) reaches as high as 58.99 dB, higher than BMA and STBMA with 1.46 dB and 0.72dB, respectively. In 10% BER, the proposed method outperforms BMA and STBMA both visually and in the objective measurement. In 20% BER, good error concealment performance can be achieved for the video with whatever complicated contents.

摘要………………………………………………………………………I
Abstract………………………………………………………II
致謝…………………………………………………………………III
英文縮寫對照表………………………………………………IV
目錄………………………………………………………………………V
圖目錄………………………………………………………………VII
表目錄…………………………………………………………………IX
第一章 緒論…………………………………………………………………1

1-1 研究背景……………………………………………………1
1-1-1 錯誤遮隱原理…………………………2
1-2 研究目的……………………………………………………3

第二章 文獻回顧與討論………………………………………………4

2-1 視訊傳輸錯誤……………………………………………4
2-2 錯誤更正機制……………………………………………5
2-2-1 自動回覆請求(Automatic Repeat Request, ARQ)………5
2-2-2 前向糾錯(Forward Error Correction, FEC)………………5
2-2-3 錯誤遮隱(Error Concealment, EC)……………………………………5
2-3 錯誤遮隱法的種類……………………………………………………………………………………………………6
2-3-1 時間域的錯誤遮隱(Temporal Error Concealment, TEC)………………6
2-3-2 空間域的錯誤遮隱(Spatial Error Concealment, SEC)…………………………7
2-4 邊緣資訊……………………………………………………………………………………………………………………7
2-4-1 Sobel邊緣偵測……………………………………………………………………………………8
2-4-2 Canny 邊緣偵測…………………………………………………………………………………9

第三章 新型的螺旋重建方法…………………………………………………………………………………………………11

3-1 邊緣匹配(edge matching, EM)………………………………………………………………11
3-2 邊緣強度(edge strength, ES)的計算方式…………………………………………13
3-3 邊緣垂直對稱重建法……………………………………………………………………………………………14
3-4 螺旋式像素重建排序法………………………………………………………………………………………14
3-5 邊緣比例門檻值……………………………………………………………………………………………………16
3-5-1 邊緣比例門檻值計算…………………………………………………………………………17
3-6 螺旋方法之演進……………………………………………………………………………………………………17
3-6-1 方法一：整體(global)螺旋空間域錯誤遮隱……………………………………………17
3-6-2 方法二：整體螺旋空間時域錯誤遮隱……………………………………………………………18
3-6-3 方法三：小區塊(sub-blocks)自體螺旋空間時域錯誤遮隱…………………………………………………………………………………………………………………………………………………18
3-7 螺旋重建方法之流程圖………………………………………………………………………………………21

第四章 實驗結果與討論…………………………………………………………………………………………………………24

4-1 編碼端的參數設定………………………………………………………………………………………………25
4-2 實驗一：螺旋重建方法的錯誤遮隱效果………………………………………………………26
4-2-1 空間域的錯誤遮隱……………………………………………………………………………26
4-2-2 空間時域的錯誤遮隱…………………………………………………………………………30
4-2-3 不同錯誤率下的PSNR………………………………………………………………………37
4-3 實驗二：I畫面與P畫面的PSNR曲線圖…………………………………………………………39
4-4 實驗三：不同的量化參數(Quantization parameter, QP)比較…………………………………………………………………………………………………………………………………………………41
4-5 實驗四：高畫質(1920 x 1080)影像實作………………………………………………43
4-6 實驗五：邊緣比例門檻值分析…………………………………………………………………………44
4-7 實驗六：複雜度計算…………………………………………………………………………………………45

第五章 結論與未來展望…………………………………………………………………………………………………………48

參考文獻………………………………………………………………………………………………………………………………………49
附錄………………………………………………………………………………………………………………………………………………52

[1] A. Luthra, G.J. Sullivan, and T. Wiegand, “Special Issue on the H.264/AVC Video Coding Standard,” IEEE Trans. Circuits Syst. Video Tech., vol. 13, Jul. 2003.
[2] G.J. Sullivan, P. Topiwala, and A. Luthra, “The H.264/AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions,” SPIE Conference on Applications of Digital Image Processing, Aug. 2004.
[3] Y. K. Wang, M. M. Hannuksela, V. Varsa, A. Hourunranta, and M. Gabbouj, “The error concealment feature in the H.26L test model,” in Proc. Int. Conf. Image Processing (ICIP), pp.729-732, Sep. 2003.
[4] S. Yue, Z. Xuhui, X. Jun, and H. Yin, “A Fast and Efficient Spatial Error Concealment for Intra-coded Frames, ” IEEE Congress on Image and Signal Processing (CISP), vol. 1, pp. 264-267, May 2008.
[5] J. Zhou, B. Yan, and H. Gharavi, “Efficient Motion Vector Interpolation for Error Concealment of H.264/AVC, ” IEEE Trans. on Broadcasting, vol. 57, no. 1, Mar. 2011.
[6] W. M. Lam, A. R. Reibman, and B. Liu, “Recovery of lost or erroneously received motion vectors,” Proceeding of ICASSP, vol.5, vol. 3, pp 417–420, 1993.
[7] M. Mengyao, C. Oscar, S.-H. Gary Chan, and M.-T. Sun, “Edge-Directed Error Concealment, ” IEEE Trans. on Circuits and Systems for Video Technology, vol. 20, no. 3, Mar. 2010.
[8] A. Midya, and S. Sengupta, “Hybrid Temporal/Spatial Error Concealment Strategy Robust to Scene Transitions, ” IEEE Pacific Rim Conference on Computers and Signal Processing (PacRim), pp. 416-421, Aug. 2011.
[9] Y. Zhang, X. Xiang, D. Zhao, S. Ma, and W. Gao, “Packet Video Error Concealment With Auto Regressive Model, ” IEEE Trans. on Circuits and Systems for Video Technology, vol. 22, no. 1, Jan. 2012.
[10] Y. Liu, J. Wang, and H. Zhang, “Depth Image-Based Temporal Error Concealment for 3-D Video Transmission, ” IEEE Trans. on Circuits and Systems for Video Technology, vol. 20, no. 4, Apr. 2010.
[11] M. Yang, X. Lan, N. Zheng, and P. Cosman, “Depth-Assisted Temporal Error Concealment for Intra Frame Slices in 3-D Video, ” IEEE Trans. on Broadcasting, vol. 60, no. 2, Jun. 2014.
[12] C. Yan, A. Oscar, H. Chiwang, and Z. Jiantao, “Spatio-Temporal Boundary Matching Algorithm for Temporal Error Concealment,” IEEE International Symposium on Circuits Syst.(ISCAS), May 2006.
[13] X. Youjun, F. Liangmou, X. Shengli, and Z. Zhiheng, “An efficient spatio-temporal boundary matching algorithm for video error concealment,” Multimedia Tools and Applications ,vol. 52, pp. 91-103, Mar. 2011.
[14] H. O. Burton and D. D. Sullivan, “Errors and error control,” Proc. of IEEE, vol. 60, pp. 1293-1301, Nov. 1972.
[15] U. DEMIR and O. AKTAS, “Raptor versus Reed Solomon Forward Error Correction Codes,”IEEE International Symposium on Computer Networks, pp.264-269, Oct. 2006.
[16] M. J. Chen, C. S. Chen, and M. C. Chi, “Temporal error concealment algorithm by recursive block-matching principle,” IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 11, pp. 1385-1393, Nov. 2005.
[17] P. J. Lee and M. L. Lin, “ Fuzzy logic based error concealment algorithm for H.264 video transmission,” in Proc. of TENCON 2005, Australia, pp. 1-4, Nov. 2005.
[18] Y. Hongtao, and Y. Bo, “A Novel Spatial Error Concealment Method for Wireless Video Transmission,” in Proc. Int. Conf. Wireless Communications, Networking and Mobile Computing (WCNMC), pp.24-26, Sep. 2009.
[19] L. Wen-Nung, L. Chang-Ming, Y. Chung-Hua, and G. Zhi-Wei, “Motion Recovery for Video Error Concealment Using Iterative Dynamic-programming Optimization,” IEEE Trans. Multimedia, vol. 16, pp. 216-227, Jan. 2014.
[20] S. Mojtaba, H. Ghanei, and S. Kasaei, “A Novel Boundary Matching Algorithm forVideo Temporal Error Concealment,” in Proc. Graphics and Signal Processing, MECS, May 2014.
[21] M. Kim, H. Lee, and S. Sull, “Spatial error concealment for H.264 Using Sequential Directional Interpolation,” IEEE Trans. Consumer Electron., vol. 54, issue 4, pp 1811-1818, Nov. 2008.
[22] H. Asheri, H.R. Rabiee, N. Pourdamghani, and M. Ghanbari, “Multi-directional spatial error concealment using adaptive edge thresholding,” IEEE Trans. Consumer Electron., vol. 58, issue 3, pp 880-885, Aug. 2012.
[23] 繆紹綱編譯，數位影像處理，普林斯頓國際有限公司，2003。
[24] J. Canny, “A computational approach to edge detection,” IEEE Trans. on PAMI. vol. PAMI-8. no. 6, pp. 679-698, Nov. 1986.
[25] K. Wei-Ying, K. Chang-Su, and C. Jay, “Spatial and Temporal Error Concealment Techniques for Video Transmission Over Noisy Channels,” IEEE Trans. Circuits Syst. Video Tech., vol. 16, Jul. 2006.
[26] http://iphome.hhi.de/suehring/tml/download/H.264/AVC reference software _ JM