未來在低功耗的嵌入式系統及IoT裝置設計上，次世代的可位元存取非揮發性記憶體 (NVRAM) 具有高度的潛力來同時取代未來DRAM及快閃記憶體，成為這類系統上的主憶體兼存儲體。因為NVRAM不需要動態能耗且支援位元存取，因此可望被用在建立未來之運算平台之上。未來之運算平台可以利用次世代 NVRAM為「記憶-儲存系統」之底層單一之記憶體架構核心(One-memory Architecture)，本研究之主軸著重於發展新的整合頁面快取 (union page cache) 機制，利用 NVRAM的特性來提升資料存取效能並且降低系統存取資料的能耗，同時利用整合頁面管理的特性來最小化檔案操作的負擔並消除不必要的資料搬動。我們更進一步研究新的檔案系統，也就是發展新的整合管理機制來發展新的檔案系統，使得此檔案系統能整合利用主記憶體及儲存體的 NVRAM空間。此整合管理機制的主要目標為破除固定分割記憶體及儲存體空間的傳統設計，以解決 NVRAM的耐用度問題，並且避免造成資料存取過程中的不必要搬動及複製。

Due to the fast access performance, byte-addressability, and non-volatility, non-volatile random access memory (NVRAM) has emerged as a popular candidate for the design of memory/storage systems of mobile computing systems. When NVRAM is considered as both main memory and storage in an embedded system, existing page cache mechanisms, which were designed to hide the performance gap between main memory and secondary storage, introduce too many unnecessary data movements between main memory and storage. To resolve this issue, we propose the concept of ``union page cache,'' which jointly manages data of the page cache in both main memory and storage. On the other hand, due to the fact that both main memory and storage are in the same NVRAM device and can both be accessed by the processor directly, we further boost the file system access performance via proposing a union storage file system (UnistorFS), which aims to jointly manage the NVRAM resource for main memory and storage. The proposed UnistorFS realizes the concept of using the NVRAM resource as memory and storage interchangeably, so as to achieve resource sharing while main memory and storage coexist on the same NVRAM device with no partition or logical boundary.

Abstract . . . . . . . . . . . . iii
Acknowledgments . . . . . . . . v
Contents . . . . . . . . . . . vii
List of Figures . . . . . . . . xii
List of Tables . . . . . . . . . xiii
1 Introduction . . . . . . . . . 1
1.1 Overview . . . . . . . . . . 1
1.2 Related Work . . . . . . . . 5
1.2.1 Traditional Page Cache Mechanism . . . . . . . . . 5
1.2.2 Existing NVRAM-based File Systems . . . . . . . . . 8
1.3 Organization . . . . . . . . . . . . . . . . . . . . 11
2 Union Page Cache for NVRAM-Based Storage Devices . . . 12
2.1 Motivation . . . . . . . . . . . . . . . . . . . . . 12
2.2 Union Page Cache – Partial Page Cache Strategy . . . 14
2.2.1 Overview . . . . . . . . . . . . . . . . . . . . . 14
2.2.2 Components of Partial Page Cache Strategy . . . . . 17
2.2.3 File Operations . . . . . . . . . . . . . . . . . 22
2.2.4 Implementation Remark . . . . . . . . . . . . . . 25
2.3 Analysis . . . . . . . . . . . . . . . . . . . . . . 27
2.3.1 Performance Enhancement . . . . . . . . . . . . . . 27
2.3.2 Overhead on Address Re-translation . . . . . . . . 31
2.4 Performance Evaluation . . . . . . . . . . . . . . . 32
2.4.1 Experiment Setup . . . . . . . . . . . . . . . . . 32
2.4.2 Experimental Results . . . . . . . . . . . . . . . 34
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . 40
3 A Union Storage File system for NVRAM-based Computer Systems 42
3.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . 42
3.2 Design Philosophy of UnistorFS . . . . . . . . . . . 45
3.3 UnistorFS . . . . . . . . . . . . . . . . . . . . . . 47
3.3.1 Overview . . . . . . . . . . . . . . . . . . . . . 47
3.3.2 Page Tracking Methods . . . . . . . . . . . . . . . 49
3.3.3 Interchangeable Page Management between Memory and Storage52
3.3.4 UnistorFS: Memory-based File System . . . . . . . . 53
3.4 Implementation Remarks for UnistorFS on Linux . . . . 55
3.5 Performance Evaluation . . . . . . . . . . . . . . . 60
3.5.1 Evaluation Methodology . . . . . . . . . . . . . . 60
3.5.2 Experiments on Real Platform . . . . . . . . . . . 61
3.5.3 Experiments on QEMU Emulator . . . . . . . . . . . 64
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . 68
4 Conclusion . . . . . . . . . . . . . . . . . . . . . . 69
References . . . . . . . . . . . . . . . . . . . . . . . 71
Publication List . . . . . . . . . . . . . . . . . . . . 83

[1] N. Aggarwal, J.F. Cantin, M.H. Lipasti, and J.E. Smith. Power-efficient dram speculation. In High Performance Computer Architecture, 2008. HPCA 2008. IEEE 14th International Symposium on, pages 317–328, Feb 2008.
[2] N. Agrawal, W. J. Bolosky, J. R. Douceur, and J. R. Lorch. A five-year study of
file-system metadata. Trans. Storage, 3(3), October 2007.
[3] Joy Arulraj, Andrew Pavlo, and Subramanya R. Dulloor. Let’s talk about storage and recovery methods for non-volatile memory database systems. Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, pages 707–722, May 2015.
[4] S. Baek, C. Hyun, J. Choi, D. Lee, and S.H. Noh. Design and analysis of a space conscious nonvolatile-ram file system. TENCON 2006. 2006 IEEE Region 10 Conference, pages 1–4, November 2006.
[5] William H. Butler and Arunava Gupta. Magnetic memory: A signal boost is in order. Nature Materials, 3:845–847, 2004.
[6] Daniel Campello, Hector Lopez, Luis Useche, Ricardo Koller, and Raju Rangaswami. Non-blocking writes to files. In FAST’15 Proceedings of the 13th USENIX Conference on File and Storage Technologies, pages 151–165, 2015.
[7] Adrian M. Caulfield, Joel Coburn, Todor Mollov, Arup De, Ameen Akel, Jiahua He, Arun Jagatheesan, Rajesh K. Gupta, Allan Snavely, and Steven Swanson. Understanding the impact of emerging non-volatile memories on high-performance, io-intensive computing. In Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’10, pages 1–11, 2010.
[8] Adrian M. Caulfield, Arup De, Joel Coburn, Todor I. Mollow, Rajesh K. Gupta, and Steven Swanson. Moneta: A high-performance storage array architecture for next-generation, non-volatile memories. In Proceedings of the 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture, MICRO ’43, pages 385–395, 2010.
[9] ChaN. Fatfs - generic fat file system module @ONLINE, http://elm-chan. org/fsw/ff/00index\_e.html, 2015.
[10] M. T. Chang, P. Rosenfeld, S. L. Lu, and B. Jacob. Technology comparison for large last-level caches (l3cs): Low-leakage sram, low write-energy stt-ram, and refresh optimized edram. In High Performance Computer Architecture (HPCA2013), 2013
IEEE 19th International Symposium on, pages 143–154, Feb 2013.
[11] Shimin Chen, Phillip B. Gibbons, and Suman Nath. Rethinking database algorithms for phase change memory. 5th biennial Conference on Innovative Data Systems Research, pages 21–31, 2011.
[12] Tseng-Yi Chen, Yuan-Hao Chang, Shuo-Han Chen, Chih-Ching Kuo, Ming-Chang Yang, Hsin-Wen Wei, and Wei-Kuan Shih. Enabling write-reduction strategy for journaling file systems over byte-addressable nvram. In Proceedings of the 54th
Annual Design Automation Conference 2017, DAC ’17, 2017.
[13] Tseng-Yi Chen, Yuan-Hao Chang, Shuo-Han Chen, Chih-Ching Kuo, Ming-Chang Yang, Hsin-Wen Wei, and Wei-Kuan Shih. Enabling write-reduction strategy for journaling file systems over byte-addressable nvram. In Proceedings of the 54th Annual Design Automation Conference 2017, pages 44:1–44:6, 2017.
[14] S.W. Cheng, Y.F. Chang, Y.H. Chang, H.W. Wei, and W.K. Shih. Warranty aware page management for pcm-based embedded systems. In Proceedings of the 2014 IEEE/ACM International Conference on Computer-Aided Design, ICCAD ’14, pages 734–741, 2014.
[15] S. Cho and H. Lee. Flip-n-write: A simple deterministic technique to improve pram
write performance, energy and endurance. In Microarchitecture, 2009. MICRO-42. 42nd Annual IEEE/ACM International Symposium on, pages 347–357, Dec 2009.
[16] J. Condit, E. B. Nightingale, C. Frost, E. Ipek, B. Lee, D. Burger, and D. Coetzee. Better i/o through byte-addressable, persistent memory. Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles, pages 133–146, October 2009.
[17] J. Condit, E. B. Nightingale, C. Frost, E. Ipek, B. Lee, D. Burger, and D. Coetzee. Better i/o through byte-addressable, persistent memory. Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles, pages 133–146, October 2009.
[18] G. Dhiman, R. Ayoub, and T. Rosing. Pdram: A hybrid pram and dram main memory system. In Design Automation Conference, 2009. DAC ’09. 46th ACM/IEEE,
pages 664–669, July 2009.
[19] Subramanya R. Dulloor, Sanjay Kumar, Anil Keshavamurthy, Philip Lantz, Dheeraj Reddy, Rajesh Sankaran, and Jeff Jackson. System software for persistent memory. In Proceedings of the Ninth European Conference on Computer Systems, EuroSys’14, 2014.
[20] D. Economou, S. Rivoire, and C. Kozyrakis. Full-system power analysis and modeling for server environments. Workshop on Modeling Benchmarking and Simulation (MOBS), page 8, 2006.
[21] Filebench. Filebench. https://github.com/filebench/filebench, 2016. Accessed: 2017-09-03.
[22] Fio. Flexible IO. https://github.com/axboe/fio, 2005. Accessed: 2017-09-03.
[23] J. Hu, Xue C.J., W.-C. Tseng, Y. He, M. Qiu, and E.H.M. Sha. Reducing write activities on non-volatile memories in embedded cmps via data migration and recomputation. In Design Automation Conference (DAC), 2010 47th ACM/IEEE, pages 350–355, June 2010.
[24] I. Hur and C. Lin. A comprehensive approach to dram power management. In High Performance Computer Architecture, 2008. HPCA 2008. IEEE 14th International Symposium on, pages 305–316, Feb 2008.
[25] Intel. Introducing breakthrough memory technology @ONLINE, http://www.intel.com/content/www/us/en/architecture-and-technology/non-volatile-memory.html?wapkw=3d+xpoint/, 2015.
[26] IOzone. Iozone filesystem benchmark @ONLINE, http://www.iozone.org/, 2006.
[27] E. Ipek, J. Condit, E. B. Nightingale, D. Burger, and T. Moscibroda. Dynamically replicated memory: Building reliable systems from nanoscale resistive memories. In Proceedings of the Fifteenth Edition of ASPLOS on Architectural Support for Programming Languages and Operating Systems, ASPLOS XV, pages 3–14, 2010.
[28] Engin Ipek, Jeremy Condit, Edmund B. Nightingale, Doug Burger, and Thomas Moscibroda. Dynamically replicated memory: Building reliable systems from nanoscale resistive memories. In Proceedings of the Fifteenth Edition of ASPLOS on Architectural Support for Programming Languages and Operating Systems, ASPLOS XV, pages 3–14, 2010.
[29] ITRS. ITRS Roadmap 2014. http://www.itrs.net/reports.html, 2014. Accessed: 2017-09-03.
[30] Lei Jiang, Youtao Zhang, and Jun Yang. Enhancing phase change memory lifetime through fine-grained current regulation and voltage upscaling. In Proceedings of the 17th IEEE/ACM international symposium on Low-power electronics and design, pages 127–132, 2011.
[31] Y. Joo, D. Niu, X. Dong, G. Sun, N. Chang, and Y. Xie. Energy- and endurance aware design of phase change memory caches. In Design, Automation Test in Europe Conference Exhibition (DATE), 2010, pages 136–141, March 2010.
[32] A. Joshi, V. Nagarajan, S. Viglas, and M. Cintra. Atom: Atomic durability in nonvolatile memory through hardware logging. In 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA), pages 361–372, Feb 2017.
[33] J.-Y. Jung and S. Cho. Memorage: emerging persistent ram based malleable main emory and storage architecture. Proceedings of the 27th international ACM conference on International conference on supercomputing, pages 115–126, 2013.
[34] Jeffrey Katcher. Postmark: A new file system benchmark. Technical report, Technical Report TR3022, Network Appliance, 1997. www.netapp.com/tech library/3022. html, 1997.
[35] B. C. Lee, E. Ipek, O. Mutlu, and D. Burger. Architecting phase change memory as a scalable dram alternative. In Proceedings of the 36th Annual International Symposium on Computer Architecture, ISCA ’09, pages 2–13, 2009.
[36] B. C. Lee, E. Ipek, O. Mutlu, and D. Burger. Architecting phase change memory as a scalable dram alternative. Proceedings of the 36th Annual International Symposium on Computer Architecture, pages 2–13, June 2009.
[37] B.C. Lee, Ping Zhou, Jun Yang, Youtao Zhang, Bo Zhao, E. Ipek, O. Mutlu, and
D. Burger. Phase-change technology and the future of main memory. Micro, IEEE, 30(1):143–143, Janurary 2010.
[38] E. Lee, S. Yoo J. Jang, and H. Bahn. Wips: a write-in-place snapshot file system for storage-class memory. Electronics Letters, pages 1053–1054, Auguest 2012.
[39] E. Lee, S. Yoo, J. Jang, and H. Bahn. Wips: a write-in-place snapshot file system for storage-class memory. Electronics Letters, 8(17):1053 – 1054, August 2012.
[40] Eunji Lee, Hyokyung Bahn, and Sam H. Noh. Unioning of the buffer cache and journaling layers with non-volatile memory. 11th USENIX Conference on File and Storage Technologies, 2013.
[41] H. G. Lee, S. Baek, C. Nicopoulos, and Kim J. An energy- and performance-aware
dram cache architecture for hybrid dram/pcm main memory systems. Computer Design (ICCD), 2011 IEEE 29th International Conference on, pages 381 – 387, October 2011.
[42] H. G. Lee, S. Baek, C. Nicopoulos, and J. Kim. An energy- and performance-aware
dram cache architecture for hybrid dram/pcm main memory systems. In Computer Design (ICCD), 2011 IEEE 29th International Conference on, pages 381–387, Oct 2011.
[43] S. Lee, H. Bahn, and S. H. Noh. Characterizing memory write references for efficient management of hybrid pcm and dram memory. In Proceedings of the 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation
of Computer and Telecommunication Systems, MASCOTS ’11, pages 168–175, 2011.
[44] S. Lee, H. Bahn, and S.H. Noh. Characterizing memory write references for efficient management of hybrid pcm and dram memory. In Modeling, Analysis Simulation of Computer and Telecommunication Systems (MASCOTS), 2011 IEEE 19th International Symposium on, pages 168–175, July 2011.
[45] S. Lee, H. Bahn, and S.H. Noh. Clock-dwf: A write-history-aware page replacement algorithm for hybrid pcm and dram memory architectures. Computers, IEEE Transactions on, 63(9):2187–2200, Sept 2014.
[46] Y. Li, X. Li, L. Ju, and Z. Jia. A three-stage-write scheme with flip-bit for pcm main memory. In Design Automation Conference (ASP-DAC), 2015 20th Asia and South Pacific, pages 328–333, Jan 2015.
[47] D. Liu, T. Wang, Y. Wang, Z. Qin, and Z. Shao. A block-level flash memory management scheme for reducing write activities in pcm-based embedded systems. In Proceedings of the Conference on Design, Automation and Test in Europe, DATE’12, pages 1447–1450, 2012.
[48] Robert Love. Linux System Programming, 2nd Edition. O’Reilly Media, 2013.
[49] W. Mauerer. Chapter 8: The virtual filesystem. In Professional Linux Kernel Architecture, pages 519–582, 2008.
[50] Micron. Breakthrough nonvolatile memory technology @ONLINE, http://www.micron.com/about/innovations/3d-xpoint-technology, 2015.
[51] Microsoft. Microsoft research @ONLINE, http://research.microsoft. com/apps/pubs/default.aspx?id=246499, 2015.
[52] L. Minas and B. Ellison. The problem of power consumption in servers @ONLINE, https://software.intel.com/en-us/articles/the-problem-of-power-consumption-in-servers, 2012.
[53] MongoDB. MongoDB. https://www.mongodb.com/, 2014. Accessed:2017-09-10.
[54] Iulian Moraru, David G. Andersen, Michael Kaminsky, Niraj Tolia, Parthasarathy Ranganathan, and Nathan Binkert. Consistent, durable, and safe memory management for byte-addressable non volatile main memory. Proceedings of the First ACM SIGOPS Conference on Timely Results in Operating Systems, November 2013.
[55] P.J. Nair, Chiachen Chou, B. Rajendran, and M.K. Qureshi. Reducing read latency of phase change memory via early read and turbo read. In High Performance Computer Architecture (HPCA), 2015 IEEE 21st International Symposium on, pages 309–319, Feb 2015.
[56] H. Nakamura, T. Nakada, and S. Miwa. Normally-off computing project : Challenges and opportunities. Design Automation Conference (ASP-DAC), pages 1–5, January 2014.
[57] NOVA. https://github.com/Andiry/nova, 2016. Accessed: 2017-09-03.
[58] PMFS. Persistent Memory File System. https://github.com/linux-pmfs/pmfs, 2015. Accessed: 2017-09-03.
[59] PRAMFS. Pramfs @ONLINE, http://pramfs.sourceforge.net/, 2013.
[60] Qemu. Qemu. https://www.qemu.org/, 2016. Accessed: 2017-09-03.
[61] T. Quan, D. Yeo, and Y. Won. Cmfs: Compressed metadata file system for hybrid storage. Network Infrastructure and Digital Content, 2010 2nd IEEE International Conference on, pages 1030 – 1034, September 2011.
[62] T. Quan, D. Yeo, and Y.Won. Cmfs: Compressed metadata file system for hybrid storage. network infrastructure and digital content. Network Infrastructure and Digital Content, 2010 2nd IEEE International Conference on, pages 1030–1034, September 2011.
[63] M. K. Qureshi, J. Karidis, M. Franceschini, V. Srinivasan, L. Lastras, and B. Abali. Enhancing lifetime and security of pcm-based main memory with start-gap wear leveling. In Proceedings of the 42Nd Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 42, pages 14–23, 2009.
[64] M. K. Qureshi, J. Karidis, M. Franceschini, V. Srinivasan, L. Lastras, and B. Abali. Enhancing lifetime and security of pcm-based main memory with start-gap wear leveling. In Proceedings of the 42Nd Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 42, pages 14–23, 2009.
[65] M. K. Qureshi, V. Srinivasan, and J. A. Rivers. Scalable high performance main memory system using phase-change memory technology. Proceedings of the 36th annual international symposium on Computer architecture, pages 24–33, June 2009.
[66] Freeman Rawson. Mempower: A simple memory power analysis tool set. In IBM Austin Research Laboratory, pages 1–6, Janurary 2004.
[67] Stuart Schechter, Gabriel H. Loh, Karin Straus, and Doug Burger. Use ecp, not ecc, for hard failures in resistive memories. In Proceedings of the 37th Annual International Symposium on Computer Architecture, ISCA ’10, pages 141–152, 2010.
[68] Vivek Seshadri, Gennady Pekhimenko, Olatunji Ruwase, Onur Mutlu, Phillip B. Gibbons, Michael A. Kozuch, Todd C. Mowry, and Trishul Chilimbi. Page overlays: an enhanced virtual memory framework to enable fine-grained memory management. Proceedings of the 42nd Annual International Symposium on Computer Architecture, pages 79–91, 2015.
[69] Z. Shao, Y. Liu, Y. Chen, and T. Li. Utilizing pcm for energy optimization in embedded systems. In VLSI (ISVLSI), 2012 IEEE Computer Society Annual Symposium
on, pages 398–403, Aug 2012.
[70] A. J. Smith. Line (block) size choice for cpu cache memories. IEEE Transactions on Computers, 36(9):1063–1076, 1987.
[71] Meyer D. T. and Bolosky W. J. A study of practical deduplication. In Proceedings of the 9th USENIX Conference on File and Stroage Technologies, FAST’11, 2011.
[72] Masao Uebayash. execute-in-place (xip) support for netbsd @ONLINE, https://www.bsdcan.org/2010/schedule/attachments/127_xip.pdf,2010.
[73] Hans Vandierendonck and Ahmad Hassan. On the energy-efficiency of byte addressablenon-volatile memory. Computer Architecture Letters, PP:1, September
2014.
[74] T. Vogelsang. Understanding the energy consumption of dynamic random access memories. In Microarchitecture (MICRO), 2010 43rd Annual IEEE/ACM International Symposium on, pages 363–374, Dec 2010.
[75] K. Vtt, I. Cutress, and R. Smith. Analyzing intel-micron 3d xpoint: The next generation non-volatile memory @ONLINE, http://goo.gl/xhjPRr, 2015.
[76] EricWWeisstein. Gamma Distribution. http://mathworld.wolfram.com/ GammaDistribution.html, 2017. Accessed: 2017-09-10.
[77] X. Wu and A. L. N. Reddy. Scmfs: A file system for storage class memory. In Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’11, pages 39:1–39:11, 2011.
[78] X.Wu and A. L. N. Reddy. Scmfs: A file system for storage class memory. High Performance Computing, Networking, Storage and Analysis (SC), 2011 International Conference for, pages 1–11, November 2011.
[79] C. Xu, X. Dong, N. P. Jouppi, and Y. Xie. Design implications of memristor-based rram cross-point structures. In 2011 Design, Automation Test in Europe, pages 1–6, March 2011.
[80] Jian Xu and Steven Swanson. Nova: A log-structured file system for hybrid volatile/non-volatile main memories. 14th USENIX Conference on File and Storage Technologies, 2016.
[81] Yahoo! Yahoo! Cloud Serving Benchmark (YCSB). https://github.com/brianfrankcooper/YCSB, 2010. Accessed: 2017-09-10.
[82] M.-C. Yang, M. Kuo, C.-W. Tsao, and Y.-H. Chang. A fifty-percent rule to minimize the energy consumption of pcm-based storage systems. In Embedded and Real-Time Computing Systems and Applications (RTCSA), 2013 IEEE 19th International Conference on, pages 139–144, Aug 2013.
[83] M.-C. Yang, M. Kuo, C.-W. Tsao, and Y.-H. Chang. A fifty-percent rule to minimize the energy consumption of pcm-based storage systems. Embedded and Real-Time Computing Systems and Applications (RTCSA), 2013 IEEE 19th International Conference on, pages 139 – 144, Auguest 2013.
[84] J. Yue and Y. Zhu. Exploiting subarrays inside a bank to improve phase change memory performance. In Design, Automation Test in Europe Conference Exhibition (DATE), 2013, pages 386–391, March 2013.
[85] Yiying Zhang, Jian Yang, Amirsaman Memaripour, and Steven Swanson. Mojim: A reliable and highly-available non-volatile memory system. Proceedings of the
Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems, pages 3–18, March 2015.
[86] P. Zhou, B. Zhao, J. Yang, and Y. Zhang. A durable and energy efficient main memory using phase change memory technology. In Proceedings of the 36th Annual International Symposium on Computer Architecture, ISCA ’09, pages 14–23, 2009.
[87] J.-G. Zhu, Y. Zheng, and G. A. Prinz. Ultrahigh density vertical magnetoresistive random access memory (invited). Journal of Applied Physics, 87(9), 2000.