此篇論文提出以數位快照的方式收集物聯網資料來作為元數據的內容儲存在相片或是影 音中。尤其我們專注在使用藍芽低功耗技術的物聯網裝置上,這些裝置不只用來做資料的傳 輸也可以支持近距離傳感而且在穿戴式、智慧型手機、私人的裝置、市內導航感應器以及家 庭裡自動化的裝置都無所不在。物聯網元數據可以涵蓋很多不同的層次,像是無線電的信號 強度和媒體存取控制位址到裝置的名稱、室內位置、溫濕度、空氣品質到廣播的資料。這些 物聯網資料可以很輕易地在元數據新增像是時間戳記、GPS位置和原先拍攝時相機原有的設 置資訊等,並將這些數據存在數位相片檔案內。有了這些物聯網元數據,使用者將可以有更 多的方式去了解所拍攝的相片場景或其他媒體中的物品和環境的資料,像是物品的詳細資 訊、廣播的事件或是在場景中的私人標籤。這些元數據提供了一個新穎的方式來搜尋、訪問 並組織照片和相關的物品。 此外,我們提出的這個新的機制是有能力去與遠端的裝置去做配 對,前提是這些裝置的身份和路由訊息當初有被當作物聯網元數據收集起來,所以經認證過 的合法使用者可以與遠端的裝置配對並獲取新的資訊。我們認為收集物聯網元數據可以為使 用者帶來顯著且有意義的好處。

This thesis proposes the capturing of IoT data and their embedding as metadata in digital snap- shots, including photos, audios, and videos. Specifically, we focus on IoT devices based on Bluetooth Low Energy (BLE) Technology, which is used for not only data communication but also proximity sensing and is ubiquitous in wearables, smartphones, personal tags, indoor-navigation beacons, and home-automation devices. The IoT metadata can cover different levels, from the radio signal strength and the media-access controller (MAC) address to device name, indoor location, temperature, hu- midity, air quality, and advertising messages. These IoT data can readily augment metadata such as timestamp, GPS location, and camera settings already captured by today’s cameras and saved in digital photo files. With such IoT metadata, the user will have much richer ways to understand the subject and environment of the scene being captured by the photo or media, such as the subject’s fitness condition, the advertised events, and the tagged personal items at the scene. These metadata will enable new ways of searching, querying, and organizing the photos and the associated objects. Moreover, a novel mechanism we propose is the ability to pair with a remote device whose identity and routing information is captured as IoT metadata, so that an authorized user can then pair with it at a later time remotely. We contend that IoT metadata capturing can bring significant benefits to the users.

1 Introduction 1
1.1 Motivation........................................ 1
1.1.1 EnvironmentalClimateData.......................... 2
1.1.2 HealthandFitnessData ............................ 2
1.1.3 ObjectData................................... 2
1.1.4 Control..................................... 3
1.2 Contributions ...................................... 3
2 Background 4
2.1 Metadata ........................................ 4 2.1.1 GeneralMetadata................................ 4 2.1.2 MetadataforDigitalImages .......................... 5
2.2 BluetoothLowEnergy(LE) .............................. 6
2.2.1 LinkLayer ................................... 6
2.2.2 Logical Link Control Adaptation Protocol (L2CAP)
2.2.3 GenericAccessProfile................
2.2.4 SecurityManager(SM)................
2.2.5 GenericAttributeProfile(GATT) ....................... 7
3 Related Work 9
3.1 EmbeddedMetadatainMultimedia..................... 9 3.1.1 MetadataCreation .............................. 9
3.1.2 Management ................................... 10
3.1.3 EnablingServiceCreation ...................... 10
3.2 PhotoAnnotation ................................ 10
3.3 RemoteAccessandControl.......................... 11
4 System Overview 12
4.1 SystemContext................................... 12 4.1.1 IoTMetadata................................... 12 4.1.2 IoTMetadataCreationProcess.................... 14
4.2 Server/ClientCommunication...................... 16
4.3 AdaptiveWindowSizeandLimitation................. 17
4.4 UseCase ........................................ 18
4.4.1 SearchImprovements............................ 18
4.4.2 RemoteControlofDeviceswithBLEviaPhoto..........19
4.4.3 ReducingTimeofAnnotation ..................... 19
4.4.4 ANovelWayofAdvertisement...................... 19
5 System Implementation 20
5.1 SmartMobileTerminal............................. 20 5.1.1 Camera........................................ 20 5.1.2 BLEManager.................................... 21 5.1.3 UITouch ...................................... 21 5.1.4 LocationManager .............................. 22 5.1.5 MetadataHandler............................... 23 5.1.6 FileManager .................................. 23 5.1.7 SocketClient.................................. 23
5.2 Gateway......................................... 24 5.2.1 Bluepy ....................................... 24
5.3 BLEDevices ..................................... 25 5.3.1 Hardware...................................... 25 5.3.2 BLEPeripheral................................. 25
5.3.3 Beacon ....................................... 26
6 Evaluation 27
6.1 Latency ........................................ 27
6.2 Scalability..................................... 28
6.3 CaseStudies..................................... 29
6.3.1 SearchPhotos.................................. 29 6.3.2 RemoteControl................................. 31 6.3.3 BeaconAdvertisement .......................... 32
7 Conclusions and Future Work 34
7.1 Conclusions..................................... 34 7.2 FutureWork...................................... 34

[1] Bluepy. https://github.com/IanHarvey/bluepy, .
[2] bluez. http://www.bluez.org/about/history/, .
[3] Flickr. http://www.flickr.com.
[4] Plus. http://ns.useplus.org/go.ashx.
[5] Bruce L Davis, Geoffrey B Rhoads, and William Y Conwell. Associating metadata with media signals, and searching for media signals using metadata, February 21 2012. US Patent 8,121,342.
[6] Hyunmo Kang and Ben Shneiderman. Exploring personal media: A spatial interface support- ing user-defined semantic regions. Journal of Visual Languages & Computing, 17(3):254–283, 2006.
[7] Allan Kuchinsky, Celine Pering, Michael L Creech, Dennis Freeze, Bill Serra, and Jacek Gwiz- dka. FotoFile: a consumer multimedia organization and retrieval system. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pages 496–503. ACM, 1999.
[8] Michael Lanza, Naicu Octavian, and Michael Rutchik. Embedded metadata in a media presen- tation, March 27 2007. US Patent App. 11/691,936.
[9] KwangYeolLeeandJaeWeonChoi.Remote-controlledhomeautomationsystemviaBluetooth home network. In SICE 2003 Annual Conference, volume 3, pages 2824–2829. IEEE, 2003.
[10] Simon Mayer, Nadine Inhelder, Ruben Verborgh, Rik Van de Walle, and Friedemann Mattern. Configuration of smart environments made simple: Combining visual modeling with semantic metadata and reasoning. In Internet of Things (IOT), 2014 International Conference on the, pages 61–66. IEEE, 2014.
[11] Mor Naaman, Susumu Harada, QianYing Wang, Hector Garcia-Molina, and Andreas Paepcke. Context data in geo-referenced digital photo collections. In Proceedings of the 12th annual ACM international conference on Multimedia, pages 196–203. ACM, 2004.
[12] Rajeev Piyare and M Tazil. Bluetooth based home automation system using cell phone. In Consumer Electronics (ISCE), 2011 IEEE 15th International Symposium on, pages 192–195. IEEE, 2011.
[13] Risto Sarvas, Erick Herrarte, Anita Wilhelm, and Marc Davis. Metadata creation system for mobile images. In Proceedings of the 2nd international conference on Mobile systems, applica- tions, and services, pages 36–48. ACM, 2004.
[14] Ben Shneiderman and Hyunmo Kang. Direct annotation: A drag-and-drop strategy for labeling photos. In Information Visualization, 2000. Proceedings. IEEE International Conference on, pages 88–95. IEEE, 2000.
[15] Guangming Song, Fei Ding, Weijuan Zhang, and Aiguo Song. A wireless power outlet system for smart homes. IEEE Transactions on Consumer Electronics, 54(4):1688–1691, 2008.
[16] N Sriskanthan, F Tan, and A Karande. Bluetooth based home automation system. Microproces- sors and Microsystems, 26(6):281–289, 2002.
[17] AnitaWilhelm,YuriTakhteyev,RistoSarvas,NancyVanHouse,andMarcDavis.Photoannota- tion on a camera phone. In CHI’04 extended abstracts on Human factors in computing systems, pages 1403–1406. ACM, 2004.
[18] Ka-Ping Yee, Kirsten Swearingen, Kevin Li, and Marti Hearst. Faceted metadata for image search and browsing. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 401–408. ACM, 2003.