當自然災害發生時，能夠有效和準確地評估受災地區至關重要，以便能夠分配適當的資源並執行相應的應對機制。最近，機器學習和遙測被發現是可以自動化該過程的強大工具。在此類模型的工程中常遇見的挑戰是需要太多數據並且無法很好地處理看不見的區域。我們基於現實將數據或時間限制納入考量並探索孿生神經網絡和小樣本圖像識別的有效性，孿生神經網路可以基於距離指標學習具有判別性的特徵，而小樣本圖像識別可以僅從單個例子中學習相關的分類訊息並且在沒看過的數據上表現良好。此外，我們還探索了使用數據擴增來處理類失衡和物件檢測。總體而言，我們的孿生與小樣本識別網絡能夠學習具有判別性的特徵，使其在僅有6% 的訓練資料的情況下顯著優於基準模型。我們了解到，模型應該優先考慮災難情況的召回指標，並且在評估建築物損壞時，還應該考慮建築物的周圍環境，因為它可以作為損壞評估的良好指標。

When a natural disaster occurs, it is crucial to be able to assess damaged regions efficiently and accurately so that the proper resources can be allocated and response mechanisms carried out accordingly. Recently, machine learning and remote sensing have been found to be powerful tools for automating the process. A common challenge encountered in the engineering of such models is a need for too much data and an inability to generalize well to unseen regions. We consider a realistic setting where there are data or time constraints and explore the effectiveness of siamese neural networks, which can learn discriminative features based on a distance metric, and one-shot image recognition, which can learn relevant categorical information from only a single example and has been known to perform well on unseen data, in such a setting. Additionally, we also explored the use of data augmentation to deal with class imbalance and object detection. Overall, our siamese one shot network was able to learn discriminative features that allowed it to significantly outperform the baseline model with only six percent of the training data. We learn that a model should show some preference to the recall metric for disaster situations and that when assessing building damage, we should also take the building's surrounding environment into account, as it can serve as a good indicator for damage assessment.

摘要 i
Abstract ii
Acknowledgements iii
Contents v
List of Tables vi
List of Figures vii
1 Introduction 1
1.1 Motivation.... 1
1.2 Research Questions... 4
1.3 Research Outline ... 4
1.4 Contributions.... 5
2 Literature Review 6
2.1 Disaster Damage Detection in Satellite Imagery... 6
2.2 Convolutional Neural Networks ... 9
2.2.1 DataAugmentation ... 10
2.2.2 Image Segmentation with MaskRCNN . . . 10
2.3 Siamese Neural Networks with One-Shot Image Recognition ... 12
3 Methodology 13
3.1 Dataset ... 13
3.2 Baseline Architecture... 16
3.3 System Overview ... 16
3.4 Localization Model ... 17
3.5 Classification Model ... 20
3.6 One Shot Image Recognition.... 23
3.7 Evaluation Metrics ... 23
4 Results and Discussion 26
4.1 Building Localization... 26
4.2 Damage Classification ... 30
4.3 Localization and Damage Classification Combined... 36
5 Conclusion and Future Work 37
5.0.1 Future Work ...38
References 40

[1] R. Gupta, R. Hosfelt, S. Sajeev, N. Patel, B. Goodman, J. Doshi, E. Heim, H. Choset, and M. Gaston, “xbd: A dataset for assessing building damage from satellite imagery,” 2019.
[2] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-cnn,” 2018.
[3] H. Ritchie and M. Roser, “Natural disasters,” Our World in Data, 2014. https://ourworldindata.org/natural-disasters. Accessed: 2021-06-10.
[4] NOAA National Centers for Environmental Information (NCEI), “U.S. billion-dollar weather and climate disasters,” 2021. https://www.ncdc.noaa.gov/billions/, DOI: 10.25921/stkw-7w73. Accessed: 2021-06-10.
[5] M. Coronese, F. Lamperti, K. Keller, F. Chiaromonte, and A. Roventini, “Evi- dence for sharp increase in the economic damages of extreme natural disasters,” Proceedings of the National Academy of Sciences, vol. 116, no. 43, pp. 21450– 21455, 2019.
[6] R. Eguchi, C. Huyck, S. Ghosh, B. Adams, and A. McMillan, Utilizing New Technologies in Managing Hazards and Disasters, pp. 295–323. 01 2010.
[7] J. Z. Xu, W. Lu, Z. Li, P. Khaitan, and V. Zaytseva, “Building damage detection in satellite imagery using convolutional neural networks,” CoRR, vol. abs/1910.06444, 2019.
[8] F. Nex, D. Duarte, F. G. Tonolo, and N. Kerle, “Structural building damage detection with deep learning: Assessment of a state-of-the-art cnn in operational conditions,” Remote Sensing, vol. 11, no. 23, 2019.
[9] A. Vetrivel, M. Gerke, N. Kerle, F. Nex, and G. Vosselman, “Disaster damage detection through synergistic use of deep learning and 3d point cloud features derived from very high resolution oblique aerial images, and multiple-kernel- learning,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 140, pp. 45–59, 2018. Geospatial Computer Vision.
[10] J. Z. Xu, W. Lu, Z. Li, P. Khaitan, and V. Zaytseva, “Building damage detection in satellite imagery using convolutional neural networks,” 2019.
[11] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org.
[12] L. Fei-Fei, R. Fergus, and P. Perona, “One-shot learning of object categories,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 594–611, 2006.
[13] G. R. Koch, “Siamese neural networks for one-shot image recognition,” 2015.
[14] L. Fan, H. Liu, and Y. Hou, “An improved siamese network for face sketch recognition,” in 2019 International Conference on Machine Learning and Cybernetics (ICMLC), pp. 1–7, 2019.
[15] N. Kerle, F. Nex, M. Gerke, D. Duarte, and A. Vetrivel, “Uav-based structural damage mapping: A review,” ISPRS International Journal of Geo-Information, vol. 9, no. 1, 2020.
[16] L. Dong and J. Shan, “A comprehensive review of earthquake-induced building damage detection with remote sensing techniques,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 84, p. 85–99, 10 2013.
[17] S. Koshimura, L. Moya, E. Mas, and Y. Bai, “Tsunami damage detection with remote sensing: A review,” Geosciences, vol. 10, no. 5, 2020.
[18] M. Hoque, S. Phinn, and C. Roelfsema, “A systematic review of tropical cyclone disaster management research using remote sensing and spatial analysis,” Ocean |& Coastal Management, vol. 146, pp. 109–120, 09 2017.
[19] R. Vatsavai, M. Tuttle, B. Bhaduri, E. Bright, A. Cheriyadat, V. Chandola, and J. Graesser, “Rapid damage assessment using high-resolution remote sensing imagery: Tools and techniques,” in 2011 IEEE International Geoscience and Remote Sensing Symposium, pp. 1445–1448, 2011.
[20] W. Shi, M. Zhang, R. Zhang, S. Chen, and Z. Zhan, “Change detection based on artificial intelligence: State-of-the-art and challenges,” Remote Sensing, vol. 12, no. 10, 2020.
[21] J. Su, Y. Bai, X. Wang, D. Lu, B. Zhao, H. Yang, E. Mas, and S. Koshimura, “Technical solution discussion for key challenges of operational convolutional neural network-based building-damage assessment from satellite imagery: Per- spective from benchmark xbd dataset,” Remote Sensing, vol. 12, no. 22, 2020.
[22] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” 2015.
[23] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recog- nition,” 2015.
[24] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255, Ieee, 2009.
[25] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” 2016.
[26] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Pro- cessing Systems (F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, eds.), vol. 25, Curran Associates, Inc., 2012.
[27] T. G. J. Rudner, M. Rußwurm, J. Fil, R. Pelich, B. Bischke, V. Kopackova, and P. Bilinski, “Multi3net: Segmenting flooded buildings via fusion of multiresolution, multisensor, and multitemporal satellite imagery,” 2018.
[28] R. Gupta and M. Shah, “Rescuenet: Joint building segmentation and damage assessment from satellite imagery,” 2020.
[29] J. Lee, J. Z. Xu, K. Sohn, W. Lu, D. Berthelot, I. Gur, P. Khaitan, Ke-Wei, Huang, K. Koupparis, and B. Kowatsch, “Assessing post-disaster damage from satellite imagery using semi-supervised learning techniques,” 2020.
[30] S. Tilon, F. Nex, N. Kerle, and G. Vosselman, “Post-disaster building damage detection from earth observation imagery using unsupervised and transferable anomaly detecting generative adversarial networks,” Remote. Sens., vol. 12, p. 4193, 2020.
[31] S. Dotel, A. Shrestha, A. Bhusal, R. Pathak, A. Shakya, and S. Panday, “Disas- ter assessment from satellite imagery by analysing topographical features using deep learning,” pp. 86–92, 03 2020.
[32] J. Doshi, S. Basu, and G. Pang, “From satellite imagery to disaster insights,” 2018.
[33] M. Bosch, C. Conroy, B. Ortiz, and P. Bogden, “Improving emergency response during hurricane season using computer vision,” 2020.
[34] S. Albawi, T. A. Mohammed, and S. Al-Zawi, “Understanding of a convolutional neural network,” in 2017 International Conference on Engineering and Technology (ICET), pp. 1–6, 2017.
[35] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” 2015.
[36] A. Mikołajczyk and M. Grochowski, “Data augmentation for improving deep learning in image classification problem,” in 2018 International Interdisci- plinary PhD Workshop (IIPhDW), pp. 117–122, 2018.
[37] S. Minaee, Y. Y. Boykov, F. Porikli, A. J. Plaza, N. Kehtarnavaz, and D. Ter- zopoulos, “Image segmentation using deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1, 2021.
[38] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” 2017.
[39] Y. Wu, A. Kirillov, F. Massa, W.-Y. Lo, and R. Girshick, “Detectron2.” https: //github.com/facebookresearch/detectron2, 2019.
[40] J. Dai, H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu, and Y. Wei, “Deformable convolutional networks,” 2017.
[41] M. D. Li, K. Chang, B. Bearce, C. Y. Chang, A. J. Huang, J. P. Campbell, J. M. Brown, P. Singh, K. V. Hoebel, D. Erdoğmuş, S. Ioannidis, W. E. Palmer, M. F. Chiang, and J. Kalpathy-Cramer, “Siamese neural networks for contin- uous disease severity evaluation and change detection in medical imaging,” npj Digital Medicine, vol. 3, p. 48, Mar 2020.
[42] Y. Guo and L. Zhang, “One-shot face recognition by promoting underrepre- sented classes,” 2018.
[43] N. O’Mahony, S. Campbell, A. Carvalho, L. Krpalkova, G. V. Hernandez, S. Harapanahalli, D. Riordan, and J. Walsh, “One-shot learning for custom identification tasks; a review,” Procedia Manufacturing, vol. 38, pp. 186–193, 2019. 29th International Conference on Flexible Automation and Intelligent Manufacturing ( FAIM 2019), June 24-28, 2019, Limerick, Ireland, Beyond In- dustry 4.0: Industrial Advances, Engineering Education and Intelligent Manu- facturing.
[44] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. De- Vito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning li- brary,” in Advances in Neural Information Processing Systems 32 (H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, eds.), pp. 8024–8035, Curran Associates, Inc., 2019.
[45] T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Dollár, “Microsoft coco: Common objects in context,” 2015.
[46] T. DeVries and G. W. Taylor, “Improved regularization of convolutional neural networks with cutout,” 2017.
[47]
[48] E. K. V. I. I. A. Buslaev, A. Parinov and A. A. Kalinin, “Albumentations: fast
and flexible image augmentations,” ArXiv e-prints, 2018.
[49] P. Indyk and R. Motwani, “Approximate nearest neighbors: Towards removing the curse of dimensionality,” in Proceedings of the Thirtieth Annual ACM Sym- posium on Theory of Computing, STOC ’98, (New York, NY, USA), p. 604– 613, Association for Computing Machinery, 1998.
[50] J. Johnson, M. Douze, and H. Jégou, “Billion-scale similarity search with gpus,” arXiv preprint arXiv:1702.08734, 2017.
[51] N. Kokhlikyan, V. Miglani, M. Martin, E. Wang, B. Alsallakh, J. Reynolds, A. Melnikov, N. Kliushkina, C. Araya, S. Yan, and O. Reblitz-Richardson, “Captum: A unified and generic model interpretability library for pytorch,” 2020.
[52] S. Lundberg and S.-I. Lee, “A unified approach to interpreting model predic- tions,” 2017.