本論文應用多鬆弛時間晶格波茲曼法及圖形顯示卡叢集模擬周期性山坡紊流。為了加速模擬，使用訊息傳遞介面對三維流場進行二維切割及圖形顯示卡叢集做平行處理，因此模擬所需時間可大幅縮短。為了確認程式的正確性，透過雷諾數Re = 25,50,75,100的層流，且使用D3Q19單鬆弛時間及多鬆弛時間晶格波茲曼法模擬，兩者方法與benchmark比較後，皆得到相近的結果。紊流方面，則針對Re = 700做研究，使用的模型為D3Q19多鬆弛時間晶格波茲曼法模擬，模擬結果與benchmark作比較後，得到相當吻合的結果。本論文最後利用多張圖形顯示卡對程式平行效率作測試。

In the thesis, the laminar and turbulent channel flows over periodic hills are simulated with single-relaxation-time and multiple-relaxation-time lattice Boltzmann method. To speed up the simulation, the computation is conducted on multi-GPU cluster with two-dimensional decomposition by message passing interface(MPI). The laminar flow is simulated at Reynolds number Re=25,50,75,100 and compared with Chang et al. for validation. For turbulent flow simulations, the Reynolds number is set to be Re=700 and the results are in comparison with Breuer et al. Both results are compared and are in good agreement. In addition, the parallel performance is tested by the strong scaling test on the different GPU cluster.

1 Introduction 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . 1
1.2 Literature survey . . . . . . . . . . . . . . . .. . 4
1.2.1 Theory of Lattice Boltzmann methods . . . .. . . 4
1.2.2 Boundary conditions . . . . . . . . . . . . . . . 5
1.2.3 Flow over periodic hills . . . . . . . .. .. . . 6
1.2.4 GPU implementation . . . . . . . . . . . . .. . . 7
1.3 Motivations . . . . . . . . . . . . . . . . . .. . . 8
2 Methodology 9
2.1 The Boltzmann equation . . . . . . . . . . . . . . . 9
2.2 The BGK approximation . . . . . . . . . . . . . . . 11
2.3 The low-Mach-number approximation . . . . . . . . 13
2.4 Discretization of the Boltzmann equation . . . . . 13
2.4.1 Discretization of time . . . . . . . . . . . . . 13
2.4.2 Discretization of phase space . . . . . . . . .. 15
2.5 The Chapman-Enskog expansion . . . . .. . . . . . . 17
2.6 The multi-relaxation-time lattice Boltzmann model . 18
3 Numerical algorithm 22
3.1 Simulation procedure . . . . . . . . . . . . . . . 22
3.2 Boundary conditions . . . . . . . . . . . . . . . . 23
3.2.1 Computational domain . . .. . . . . . . . . . . . 23
3.2.2 Solid-fluid boundary technique . . .. . . . . . . 24
3.3 External force. . . . . . . . . . . . . . . . . . . 26
3.4 GPU implementation. . . . . . . . . . . . . . . . . 26
3.5 Two dimensional domain decomposition. . . . . . . . 28
3.6 Memory allocation improvement . . . . . . . . . . 29
4 Numerical results and discussion 35
4.1 Laminar flows over periodic hills . .. . . . . .. . 36
4.1.1 In 6h inter hill distance . . . . . . . . . . . . 36
4.1.2 In 9h inter hill distance . . . . . . . . . . . . 37
4.2 Turbulent
ows over periodic hills . . . . . . . . . . . . . . . . 37
4.3 Parallel performance . . . . . . . . . . . . . . . 39
5 Conclusions 57

[1] P.H. Chang, C.C Liao, H.W. Hsu, S.H. Liu, C.A. Lin, ”Simulations of laminar and turbulent flows over periodic hills with immersed boundary method”, Computers and Fluids, 92, 233-243, (2014).
[2] M. Breuer, N. Peller, Ch. Rapp, M. Manhart, ”Flow over periodic hillsnumerical and experimental study in a wide range of Reynolds numbers”, Computers and Fluids, 38, 433-457, (2009).
[3] Y. Zhu and R. A. Antonia, “Temperature dissipation measurements in a fully developed turbulent channel flow,” Exp. Fluids 15, 191-199, (1993).
[4] N. Kasagi, Y. Tomita and A. Kuroda, “Direct numerical simulation of passive scalar field in a turbulent channel flow,” J. Heat Transfer 114, 598-606, (1992).
[5] H. Kawamura, H. Abe and H. Shingai, “DNS of turbulence and heat transport in channel flow with respect to Reynolds and Prandtl number effects,” Int. J. Fluid Flow 20, 196-207, (1999).
[6] U. Frisch, B. Hasslacher, and Y. Pomeau, “Lattice-gas automata for the Navier- Stokes equation”, Physics Review Letters 56, 1505, (1986).
[7] S. Wolfram, “Cellular automaton fluids 1: Basic theory”, Journal Statistic Physics 45, 471, (1986).
[8] S. Succi, E. Foti and F. J. HIGUERA, “Three-Dimensional Flows in Complex Geometries with the Lattice Boltzmann Method”, Europhysics Letters 9, 345, (1989).
[9] F. J. Higuera, and J. Jem´enez, “Boltzmann approach to lattice gas simulations”, Europhysics Letters 9, 663, (1989).
[10] P. L. Bhatnagar, E. P. Gross, and M. Grook, “A model for collision processes in gases. I. small amplitude processes in charged and neutral one-component systems”, Physics Reviews E 94, 511, (1954).
[11] S. Harris, “An introduction to the throry of the Boltzmann equation”, Holt, Rinehart and Winston, New York, (1971)
[12] S. Chen, H. Chen, D. O. Martinez, and W. H. Matthaeus, “Lattice Boltzmann model for simulation of magnetohydrodynamics,” Physics Review Letters 67, 3776, (1991).
[13] Y. H. Qian, D. d’Humi`eres, and P. Lallemand, “ Lattice BGK model for Navier- Stokes equation,” Europhysics Letters 17, 479, (1992).
[14] S. Chen and G. D. Doolen, “Lattice Boltzmann method for fluid flow,” Annual Reviews of Fluid Mechanics 30, 329, (1998).
[15] Y. H. Qian, D. d’Humi´eres, and P. Lallemand, “Lattice BGK models for Navier- Stokes equation”, Europhysics Letters 17, 479, (1992).
[16] H. Chen, S. Chen, and W. H. Matthaeus, “Recovery of the Navier-Stokes quations using a lattice gas Boltzmann method”, Physics Reviews A 45, R5339, (1992).
[17] A. Dieter, Wolf-Gladrow, “Lattice gas cellular automata and lattice Boltzmann models”, Springer: Berlin, (2000).
[18] P. Lallemand, and L. S. Luo, “Theory of the lattice Boltzmann method: dispersion, dissipation, isotropy, Galilean invariance, and stability,” Phys. Rev. E 61, 6546, (2000).
[19] J. Latt and B. Chopard,“Lattice Boltzmann method with regularized pre- collision distribution functions“, Mathematics and Computers in Simulation 72, 165, (2006)
[20] S. Ansumali,and I. V. Karlin, “Entropy function approach to Lattice Boltzmann Method“, Journal of Statical Physicas 107, Nos.1/2, (2002).
[21] L. S. Lin, H. W. Chang, C. A. Lin, Multi relaxation time lattice Boltzmann simulations of transition in deep 2D lid driven cavity using GPU, Computers and Fluids 80, 381, (2013).
[22] C. Obrecht, F. Kuznik, B. Tourancheau, and J. J. Roux, Multi-GPU implementation of a hybrid thermal lattice Boltzmann solver using the TheLMA framework, Comput. Fluids 80, 269, (2013).
[23] C. Obrecht, F. Kuznik, B. Tourancheau, and J. J. Roux, Multi-GPU implementation of the lattice Boltzmann method, Comput. Math. Appl 65, 252, (2013).
[24] X. Wang, T. Aoki, Multi-GPU performance of incompressible flow computation by lattice Boltzmann method on GPU cluster, Parallel. Computing 37, 521, (2011).
[25] C. Obrecht, F. Kuznik, B. Tourancheau, and J. J. Roux, Scalable lattice Boltzmann solvers for CUDA GPU clusters, Comput. Fluids 39, 259, (2013).
[26] F. Kuznik, C. Obrecht, G. Rusaouen, J. J. Roux, LBM based flow simulation using GPU computing processor, Computers and Mathematics with Applications 59, 2380, (2010).
[27] C. Obrecht, F. Kuznik, B. Tourancheau, J. J. Roux, A new approach to the lattice Boltzmann method for graphics processing units, Computers and Mathematics with Applications 61, 3628, (2011).
[28] G. R. McNamara, and G. Zanetti, “Use of the Boltzmann equation to simulate lattice-gas automata,” Phys. Rev. Lett. 61, 2332, (1988).
[29] F. J. Higuera, and J. Jim´enez, “Boltzmann approach to lattice gas simulations,” Europhys. Lett. 9, 663, (1989).
[30] S. Chen, H. Chen, D. O. Martinez, and W. H. Matthaeus, “Lattice Boltzmann model for simulation of magnethydrodynamics,” Phys. Rev. Lett. 67, 3776, (1991).
[31] H. Chen, S. Chen, and W. H. Matthaeus, “Recovery of the Navier-Stokes equations using a lattice-gas Boltzmann method,” Phys. Rev. A. 45, 5339, (1992).
[32] Y. H. Qian, D. d’Humi`eres, and P. Lallemand, “Lattice BGK models for Navier- Stokes equation,” Europhys. Lett. 17, 479, (1992).
[33] X. He, and L. S. Luo, “Theory of the lattice Boltzmann method: From the Boltzmann equation to the lattice Boltzmann equation,” Phys. Rev. E 56, 6811, (1997).
[34] D. d’Humi`eres, I. Ginzburg, M. Krafczyk, P. Lallemand, and L. S. Luo, “Multi- relaxation-time lattice Boltzmann models in three dimensions,” Phil. Trans. R. Soc. Lond. A 360, 437, (2002).
[35] J. S. Wu, and Y. L. Shao, “Simulation of lid-driven cavity flows by parallel lattice Boltzmann method using multi-relaxation-time scheme,” Int. J. Numer. Meth. Fluids 46, 921, (2004).
[36] X. He, and L. S. Luo, “A priori derivation of the lattice Boltzmann equation,” Phys. Rev. E 55, 6333, (1997).
[37] K. Kono, T. Ishizuka, H. Tsuda, and A. Kurosawa, “Application of lattice Boltzmann model to multiphase flows with phase transition,” Comput. Phys. Commun. 129, 110, (2000).
[38] S. Hou, X. Shan, Q. Zou, G. D. Doolen, and W. E. Soll, “Evaluation of two lattice Boltzmann models for multiphase flows,” J. Comput. Phys. 138, 695, (1997).
[39] X. He, S. Chen, and R. Zhang, “A lattice Boltzmann scheme for incompressible multiphase flow and its application in simulation of Rayleigh- Taylor instability,” J. Comput. Phys. 152, 642, (1999).
[40] C. H. Shih, C. L. Wu, L. C. Chang, and C. A. Lin, “Lattice Boltzmann simulations of incompressible liguid-gas system on partial wetting surface,” Phil. Trans. R. Soc. A 369, 2510, (2011).
[41] M. Krafczyk, M. Schulz, and E. Rank, “Lattice-gas simulations of two-phase flow in porous media,” Commun. Numer. Meth. Engng 14, 709, (1998).
[42] J. Bernsdorf, G. Brenner, and F. Durst, “Numerical analysis of the pressure drop in porous media flow with lattice Boltzmann (BGK) automata,” Comput. Phys. Commun. 129, 247, (2000).
[43] D. M. Freed, “Lattice-Boltzmann method for macroscopic porous media modeling,” Int. J. Mod. Phys. C 9, 1491, (1998).
[44] Y. Hashimoto, and H. Ohashi, “Droplet dynamics using the lattice-gas method,” Int. J. Mod. Phys. 8, 977, (1997).
[45] H. Xi, and C. Duncan, “Lattice Boltzmann simulations of three-dimensional single droplet deformation and breakup under simple shear flow,” Phys. Rev. E 59, 3022, (1999).
[46] S. Hou, “Lattice Boltzmann Method for Incompressible, Viscous Flow“, Ph.D. Thesis, Department of Mechanical Engineering, Kansas State University, (1995).
[47] D. d’Humi`eres, “Generalized lattice Boltzmann equation“, In Rarefied Gas Dynamics: Theory and Simulations, Progress in Astronautics and Aeronautics, 159, Shizgal BD, Weaver DP (eds).AIAA: Washington, DC, 45, (1992).
[48] R. A. Brownlee, A.N Gorban, J.Levesley,”Stabilisation of the lattice- Boltzmann method using the Ehrenfests’ coarse-graining”, (2008)
[49] M. Spasov, D. Rempfer, and P. Mokhasi,“Simulation of a turbulent channel flow with entropic Lattice Boltzmann method“, International Journal For Numerical Methods In Fluids60, 1241, (2008)
[50] F. Tosi, S. Ubertini, S. Succi, and I. V. Karlin,“Optimization strategies for the Entropic lattice Boltzmann method“, Journal of Scientific Computing 30, No. 3, (2006)
[51] D. d’Humi`eres, “Generalized lattice Boltzmann equation,” In Rarefied gas dymanics - Theory and simulations, Progress in Astronautics and Aeronautics, vol. 159, Shizgal BD, Weaver DP(eds). AIAA: Washington, DC, 45-458, (1992).
[52] G. McNamara, A. L. Garcia, and B. J. Alder, “Stabilization of thermal lattice Boltzmann models,” Journal Statics Physics 81, 395, (1995).
[53] G. McNamara and B. Alder, “Analysis of the lattice Boltzmann treatment of hydrodynamics,” Physica A 194, 218, (1993)
[54] X. Shan, “Simulation of Rayleigh-Benard convection using a lattice Boltzmann method,” Physical Review E 55 , 2780, (1997).
[55] Y. Peng, C. Shu and Y. T. Chew, “Simplified thermal model for the lattice Boltzmann model for incompressible thermal flows,” Physics Reviews E 68, 026701, (2003).
[56] Q. Li, Y. L. He, Y. Wang and G. H. Tang, “An improved thermal lattice Boltzmann model for flows without viscous heat dissipation and compression work,” Journal of Modern Physics C Vol. 19, No. 1 (2008) ,125-150.
[57] Y. Shi, T. S. Zhao and Z. L. Guo, “Thermal lattice Bhatnagar-Gross-Krook model for flows with viscous heat dissipation in the incompressible limit,” Physics Review E70, 066310, (2004).
[58] Z. W. Tian, C. Zuo, H. J. Liu, Z. H. Liu, Z. L. Guo and C. G. Zheng, ”Thermal lattice Boltzmann model with viscous heat dissipation in the incompressible limit,” Journal of Modern Physics C Vol. 17, No. 8 (2006) ,1131-1139.
[59] A. J. C. Ladd, “Numerical simulations of particulate suspensions via a discretized Boltzmann equation. Part 1. Theoretical foundation, J. Fluid Mech, 271, 285, (1994).
[60] A. J. C. Ladd, “Numerical simulations of particulate suspensions via a discretized Boltzmann equation. Part 2. Numerical results, J. Fluid Mech, 271, 311, (1994).
[61] I. Ginzbourg and P. M. Adler, “Boundary condition analysis for the three- dimensional lattice Boltzmann model“, J. Phys. II France 4, 191-214, (1994)
[62] P. A. Skordos, “Initial and boundary conditions for the lattice Boltzmann method,” Phys. Rev. E 48, 4823, (1993).
[63] D. R. Noble, S. Chen, J. G. Georgiadis and R. O. Buckius, “A consistent hydrodynamic boundary condition for the lattice Boltzmann method,” Phys. Fluid 7, 2928, (1995).
[64] T. Inamuro, M. Yoshino, and F. Ogino, “A non-slip boundary condition for lattice Boltzmann simulations,” Phys. Fluids 7, 2928, (1995).
[65] S. Chen, D. Martinez, and R. Mei, “On boundary conditions in lattice Boltzmann methods,” Phys. Fluids 8, 2527, (1996).
[66] Q. Zou, and X. He, “On pressure and velocity boundary conditions for the lattice Boltzmann BGK model,” Phys. Fluids 9, 1591, (1997).
[67] C. F. Ho, C. Chang, K. H. Lin, and C. A. Lin, “Consistent boundary conditions for 2D and 3D lattice Boltzmann simulations,” CMES 44, 137, (2009).
[68] G. H. Tang, W. Q. Tao and Y. L. He,“Thermal boundary condition for thermal lattice Boltzmann equation,” Physics Review E 72, 016703, (2005).
[69] A. D’Orazio, S. Succi and C. Arrighettic, “Lattice Boltzmann simulation of open flows with heat transfer,” Physics Fluids 15, 2778, (2003).
[70] Z. Guo, B. Shi and C. Zheng, “A coupled lattice BGK model for the Boussinesq equations,” Int. J. Numer. Methods Fluids 39, 325, (2002)
[71] C. H. Liu, K. H. Lin, H. C. Mai and C. A. Lin, “Thermal boundary conditions for thermal lattice Boltzmann simulations,” Computers and Mathematics with applications, (2009).
[72] C. P. Mellen, J. Frohlich, W. Rodi, Large eddy simulation of the flow over periodic hills, 16th IMACS World Congress(2000)
[73] L. Temmerman, M. A. Leschziner, C. P. Mellen, Investigation of wall-function approximations and subgrid-scale models in large eddy simulation of separated flow in a channel with streamwise periodic constrictions, J. Heat and Fluid Flow, 24(2003) 157-180.
[74] M. Breuer, N. Peller, Flow over periodic hills Numerical and experimental study in a wide range of Reynolds numbers, J. Computer and Fluid, 38(2009) 433-457.
[75] J. Bolz, I. Farmer, E. Grinspun, and P. Schro¨der, “Sparse matrix solvers on the GPU: Conjugate gradients and multigrid,” ACM Trans. Graph. (SIGGRAPH) 22, 917, (2003).
[76] F. A. Kuo, M. R. Smith, C. W. Hsieh, C. Y. Chou, and J. S. Wu, “GPU acceleration for general conservation equations and its application to several engineering problems,” Comput. Fluids 45, 147, (2011).
[77] J. T¨olke, “Implementation of a lattice Boltzmann kernel using the compute unified device architecture developed by nVIDIA,” Comput. Visual Sci. 13, 29, (2008).
[78] J. T¨olke, and M. Krafczyk, “TeraFLOP computing on a desktop PC with GPUs for 3D CFD,” Int. J. Comput. Fluid D. 22, 443, (2008).
[79] E. Riegel, T. Indinger, and N. A. Adams, “Implementation of a Lattice- Boltzmann method for numerical fluid mechanics using the nVIDIA CUDA technology,” CSRD 23, 241, (2009).
[80] C. Obrecht, F. Kuznik, B. Tourancheau, and J. J. Roux, “A new approach to the lattice Boltzmann method for graphics processing units,” Comput. Math. Appl. 61, 3628, (2011).
[81] C. Obrecht, F. Kuznik, B. Tourancheau, and J. J. Roux, “Scalable lattice Boltzmann solvers for CUDA GPU clusters,” Comput. Fluids. 39, 259, (2013).
[82] H. W. Chang, P.Y. Hong, L.S. Lin and C.A. Lin, “Simulations of three- dimensional cavity flows with multi relaxation time lattice Boltzmann method and graphic processing units,” Procedia Engineering. 61, 94, (2013)
[83] P. Y. Hong“Lattice Boltzmann model with multi-GPU implementation,” Master’s thesis, National Tsing Hua University, (2013)
[84] L. M. Huang“Implementation of lattice Boltzmann method on multi-GPU cluster using two-dimensional decomposition,” Master’s thesis, National Tsing Hua University, (2014)
[85] Y. H. Lee,“Turbulent duct flow simulations with single-relaxation tim and entropic lattice boltzmann method on multi-GPU cluster,” Master’s thesis, National Tsing Hua University, (2015).
[86] T. I. Gombosi, “Gas kinetic theory,” Cambridge University Press, (1994).
[87] J. W. Deardorff, “A numerical study of three-dimensional turbulent channel flow at large Reynolds numbers.“, Journal of Fluid Mechanics 41.2, 453, (1970).
[88] S. Hou, J. Sterling, S. Chen and G. D. Doolen, “A lattice Boltzmann subgrid model for high Reynolds number flows.”, Pattern formation and lattice gas automata, vol.6, 151, (1996).
[89] H. Yu, L. S. Luo, and S. S. Girimaji, “LES of turbulent square jet flow using an MRT lattice Boltzmann model,” Comput. Fluids 35, 957, (2006).
[90] M. J. Pattison, K. N. Premnath and S.Banerjee, “Computation of turbulent flow and secondary motions in a square duct using a forced generalized lattice Boltzmann equation.“, Physical of Review , Vol. 79, 026704, (2009).
[91] H. W. Hsu. “Investigation of turbulent Couette-Poiseuille and Couette flows inside a square duct“, Ph.D. Thesis, Department of Mechanical Engineering, National Tsing Hua University, (2012).
[92] L. S. Luo, “Theory of the lattice Boltzmann method: Lattice Boltzmann models for nonideal gases.”, Physical Review E 62.4, 4982, (2000).
[93] R. Cornubert, D. d’Humi`eres, and D Levermore, “A Knudsen layer theory for lattice gases,” Physica D 47, 241, (1991)
[94] I. C. Kim “Second Order Bounce Back Boundary Condition for the Latice Boltzmann Fluid Simulation,” KSME 14, 84, (2000)
[95] S. Gokaltun and G. S. Dulikravich , “Simulation of channel flows using thermal lattice Boltzamnn method”, Compt. Math. Application 24, (2008).
[96] B. Debusschere and C. J. Rutland, “Turbulent scalar transport mechanisms in plane channel and Couette flows”, Int. J. Heat and Mass Transfer 47, 1771- 1781, (2004).