連線池是一種常見的技術，允許多個客戶端共享和重複使用連線。資料庫連 線池可以減少開啟和關閉連線的成本，特別是對於傳統的資料庫系統，如 PostgreSQL。傳統的連線池作為用戶空間應用程式實現，利用 Linux 核心提 供的高相容性、安全性和隔離的網路堆疊。然而，由於過多的用戶核心交互 和核心網路堆疊遍歷，它們的性能較低。
我們提出了 kpgpool，一個基於 eBPF 的 PostgreSQL 連線池，在資料包進 入用戶空間之前代理客戶端和 PostgreSQL 伺服器之間的資料包。我們的實 驗表明，kpgpool 在與其他用戶空間連線池相比，能提高 19% 的吞吐量並降 低 10% 的延遲，同時顯著降低 CPU 使用率。

Connection pooling is a common technique that allows multiple clients to share and reuse connections. Database connection pooling can reduce the cost of opening and closing connections, especially for traditional database systems, such as Post- greSQL. Traditional connection pools are implemented as user-space applications, which take advantage of high compatibility, security, and isolation networking stack provided by the Linux kernel. However, they suffer from low performance due to excessive user-kernel crossings and kernel networking stack traversing.
We present kpgpool, an eBPF-based connection pool for PostgreSQL that proxies packets between clients and the PostgreSQL server before the packets enter user-space. Our experiments show that kpgpool improve throughput by 19% and lower latency by 10% with a significantly lower CPU usage comparing to other user-space pools.

1 Introduction ................................................. 1
2 Background .................................................. 5
2.1 eBPF .................................................. 5
2.1.1 eBPF Architecture ............................... 5
2.1.2 eBPF Program Type and Hooks ..................... 7
2.1.3 eBPF Map ........................................ 8
2.1.4 Kernel Packet Flow .............................. 9
2.1.5 XDP, TC and the sk_skb Hooks ................... 10
2.2 PostgreSQL Message Flow .............................. 15
2.2.1 Startup Message ................................ 15
2.2.2 Simple Query Protocol .......................... 16
2.2.3 Extended Query Protocol ........................ 17
3 Related works .............................................. 20
4 Motivation ................................................. 23
4.1 Naive TCP Proxy Performance Gain through eBPF ......... 23
4.1.1 The eBPF TCP Proxy ............................. 24
4.1.2 The User-Space Proxy ........................... 25
4.1.3 Evaluation ..................................... 25
5 Implementation ............................................. 28
5.1 Session Mode .......................................... 28
5.1.1 Registering the sockets ........................ 30
5.1.2 Maintaining the sessions ....................... 31
5.2 Transaction Mode ...................................... 32
5.3 Supporting Prepared Statement ......................... 33
5.3.1 User-space supported prepared statement ........ 34
5.3.2 Bypass prepared statement with eBPF ............ 36
6 Evaluation ................................................. 38
6.1 Simple Query Throughput Under Transaction Mode ........ 39
6.2 Simple Query Latency Under Transaction Mode ........... 40
6.3 Session Mode Performance ............................. 41
6.4 Extended Protocol Performance ........................ 42
6.5 CPU Usage ............................................ 44
7 Conclusion ................................................. 46
8 Reference .................................................. 47

[1] BCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more. https://github.com/iovisor/bcc.
[2] Bertin,G.Xdp in practice: integrating xdp into our ddos mitigation pipeline. In Technical Conference on Linux Networking, Netdev (2017), vol. 2, The NetDev Society, pp. 1–5.
[3] Borkmann, D. Virtual Ethernet Device Optimization with eBPF. https://cilium.io/blog/2020/11/10/cilium-19/veth.
[4] Borkmann, D., and Pumputis, M. Kubernetes service load-balancing at scale with bpf xdp. In Linux Plumber Conference (2020).
[5] Linux source code of BPF queue/stack maps. https://elixir.bootlin.com/linux/v6.1/source/kernel/bpf/queue_stack_maps.c.
[6] Linux kernel patch: bpf: Add redirect_peer helper. https://github.com/tor valds/linux/commit/9aa1206e8f482.
[7] Linux kerenl patch: bpf: Adding support for sock_ops. https://lwn.net/Articles/727189/.
[8] Butrovich, M., Ramanathan, K., Rollinson, J., Lim, W. S., Zhang, W., Sherry, J., and Pavlo, A. Tigger: A database proxy that bounces with user-bypass. Proc. VLDB Endow. 16, 11 (jul 2023), 3335–3348.
[9] cilium/ebpf: ebpf-go is a pure-Go library to read, modify and load eBPF programs and attach them to various hooks in the Linux kernel. https://github.com/cilium/ebpf.
[10] Custer, C. What is connection pooling, and why should you care. https://www.cockroachlabs.com/blog/what-is-connection-pooling/.
[11] Isovalent: eBPF Docs: Verifier. https://ebpf-docs.dylanreimerink.nl/linux/concepts/verifier/.
[12] Fowler–Noll–Vo hash function. https://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo_hash_function#FNV-1a_hash.
[13] Google Cloud Platform: feedback General-purpose machine family for Compute Engine. https://cloud.google.com/compute/docs/general-purpose-machines.
[14] Ghigoff, Y., Sopena, J., Lazri, K., Blin, A., and Muller, G. BMC: Accelerating memcached using safe in-kernel caching and pre-stack processing. In 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21) (Apr. 2021), USENIX Association, pp. 487–501.
[15] Go: An open-source programming language supported by Google. https://go.dev/.
[16] Effective Go: goroutines. https://go.dev/doc/effective_go#goroutines.
[17] Gupta, K., and Mathuria, M. Improving performance of web application approaches using connection pooling. In 2017 International conference of Electronics, Communication and Aerospace Technology (ICECA) (2017), vol. 2, pp. 355–358.
[18] Katran: A high performance layer 4 load balancer. https://github.com/facebookincubator/katran.
[19] devconf.cz 2018. Linux packet journey, napi, hardware queue, skb. https: //www.youtube.com/watch?v=6Fl1rsxk4JQ.
[20] Liu, Z., Ben-Basat, R., Einziger, G., Kassner, Y., Braverman, V., Friedman, R., and Sekar, V. Nitrosketch: robust and general sketch-based monitoring in software switches. In Proceedings of the ACM Special Interest Group on Data Communication (New York, NY, USA, 2019), SIGCOMM ’19, Association for Computing Machinery, p. 334–350.
[21] Miano,S.,Chen,X.,Basat,R.B.,and Antichi,G. Fast in-kernel traffic sketching in ebpf. SIGCOMM Comput. Commun. Rev. 53, 1 (apr 2023), 3–13.
[22] Monnet, Q. Understanding tc “direct action” mode for BPF. https://qmonnet.github.io/whirl-offload/2020/04/11/tc-bpf-direct-action/.
[23] Mullane, G. S. CrunchyData: Prepared Statements in Transaction Mode for PgBouncer. https://www.crunchydata.com/blog/prepared-statements-in-transaction-mode-for-pgbouncer#why-prepared-statements-can-be-a-problem-in-transaction-mode.
[24] Flow of network packets through Netfilter with legacy iptables packet filtering. https://en.wikipedia.org/wiki/Netfilter#/media/File:Netfilter-packet-flow.svg.
[25] PgBouncer: Lightweight connection pooler for PostgreSQL. https://www. pgbouncer.org/.
[26] PgBouncer: Lightweight connection pooler for PostgreSQL: Configuration. https://www.pgbouncer.org/config.html#pool_mode.
[27] PgBouncer: Support of prepared statements. https://github.com/pgbouncer/pgbouncer/pull/845.
[28] PostgreSQL Message Flow: Extended Query. https://www.postgresql.org /docs/13/protocol-flow.html#PROTOCOL-FLOW-EXT-QUERY.
[29] PostgreSQL: The World’s Most Advanced Open Source Relational Database. https://www.postgresql.org/.
[30] DigitalOcean: How to Manage Connection Pools for PostgreSQL Database Clusters. https://docs.digitalocean.com/products/databases/postgresql/how-to/manage-connection-pools/.
[31] PostgreSQL Limits. https://www.postgresql.org/docs/current/limits.html.
[32] PostgreSQL: MessageFlow: Start-up. https://www.postgresql.org/docs/13/protocol-flow.htmlid-1.10.5.7.3.
[33] PostgreSQL: Resource Consumption: temp_buffers. https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-TEMP-BUF FERS.
[34] PostgreSQL: Resource Consumption: work_mem. https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-WORK-MEM.
[35] ScyllaDB. https://www.scylladb.com/.
[36] CMU Database Group: ScyllaDB: No-Compromise Performance(AviKivity). https://www.youtube.com/watch?t=2586&v=0S6i9BmuF8U.
[37] SELTZER, G. BPF Map Concurrency Techniques. https://www.grant.pizza/blog/bpf-concurrency/.
[38] Sheng, S., Puyang, H., Huang, Q., Tang, L., and Lee, P. P. C. FarReach: Write-back caching in programmable switches. In 2023 USENIX Annual Technical Conference (USENIX ATC 23) (Boston, MA, July 2023), USENIX Association, pp. 571–584.
[39] sockperf: Network Benchmarking Utility. https://github.com/Mellanox/sockperf.
[40] Linux kernel patch: net, sched: add clsact qdisc. https://lwn.net/Articles/6 71458/.
[41] Tu, W., Wei, Y.-H., Antichi, G., and Pfaff, B. revisiting the open vswitch dataplane ten years later. In Proceedings of the 2021 ACM SIGCOMM 2021 Conference (New York, NY, USA, 2021), SIGCOMM ’21, Association for Computing Machinery, p. 245–257.
[42] Vieira, M. A. M., Castanho, M. S., Pacífico, R. D. G., Santos, E. R. S., Júnior, E. P. M. C., and Vieira, L. F. M. Fast packet processing with ebpf and xdp: Concepts, code, challenges, and applications. ACM Comput. Surv. 53, 1 (feb 2020).
[43] Wang, F., Zhao, G., Zhang, Q., Xu, H., Yue, W., and Xie, L. Oxdp: Offloading xdp to smartnic for accelerating packet processing. In 2022 IEEE 28th Inter- national Conference on Parallel and Distributed Systems (ICPADS) (2023), pp. 754–761.
[44] Zhong, Y., Li, H., Wu, Y. J., Zarkadas, I., Tao, J., Mesterhazy, E., Makris, M., Yang, J., Tai, A., Stutsman, R., and Cidon, A. XRP: In-Kernel storage func- tions with eBPF. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22) (Carlsbad, CA, July 2022), USENIX Associa- tion, pp. 375–393.
[45] Zhou, Y., Wang, Z., Dharanipragada, S., and Yu, M. Electrode: Acceler- ating distributed protocols with eBPF. In 20th USENIX Symposium on Net- worked Systems Design and Implementation (NSDI 23) (Boston, MA, Apr. 2023), USENIX Association, pp. 1391–1407.
[46] Zhou, Y., Xiang, X., Kiley, M., Dharanipragada, S., and Yu, M. DINT: Fast In-Kernel distributed transactions with eBPF. In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24) (Santa Clara, CA, Apr. 2024), USENIX Association, pp. 401–417.