Connecting to MySQL/Postgres through Lua filter

[Thiva Haran]

I am trying out some custom API rate limiting algorithms in Envoy by implementing them through Lua Scripts. For this purpose, I want to connect to a database(MySQL or Postgres), to store counters, logs etc. within the Lua script. 
The issue is, MySQL/Postgres Lua client libraries does not seem to work within Envoy's Lua scripts. 

It gives 'Caught Segmentation fault, suspect faulting address ... ' error while trying to make a connection with a Database instance.
The Database connection using the same Lua client libraries seems to work correctly outside Envoy, with the same Lua version and dependencies. Surprisingly, Redis client libraries seem to work within Envoy Lua filters.
Why am I not being able to connect to MySQL/ Postgres through Envoy Lua filter and is there any alternative way to make this connection ?

Here are the related envoy.yaml files in GitHub and the error logs.

Postgres
envoy.yaml using a postgres lua library pgmoon :

https://github.com/thiva-k/mysql-lua-test/blob/posgres/envoy.yaml

2024-07-16 19:44:17 [2024-07-16 14:14:17.688][1][info][upstream] [source/common/upstream/cluster_manager_impl.cc:240] cm init: all clusters initialized
2024-07-16 19:44:17 [2024-07-16 14:14:17.688][1][info][main] [source/server/server.cc:950] all clusters initialized. initializing init manager
2024-07-16 19:44:17 [2024-07-16 14:14:17.688][1][info][config] [source/common/listener_manager/listener_manager_impl.cc:930] all dependencies initialized. starting workers
2024-07-16 19:45:15 [2024-07-16 14:15:15.065][30][critical][backtrace] [./source/server/backtrace.h:127] Caught Segmentation fault, suspect faulting address 0x4c0d0
2024-07-16 19:45:15 [2024-07-16 14:15:15.065][30][critical][backtrace] [./source/server/backtrace.h:111] Backtrace (use tools/stack_decode.py to get line numbers):
2024-07-16 19:45:15 [2024-07-16 14:15:15.065][30][critical][backtrace] [./source/server/backtrace.h:112] Envoy version: d79f6e8d453ee260e9094093b8dd31af0056e67b/1.30.2/Clean/RELEASE/BoringSSL
2024-07-16 19:45:15 [2024-07-16 14:15:15.065][30][critical][backtrace] [./source/server/backtrace.h:114] Address mapping: 55e7d05c7000-55e7d2f6f000 /envoy
2024-07-16 19:45:15 [2024-07-16 14:15:15.065][30][critical][backtrace] [./source/server/backtrace.h:121] #0: [0x7fd5d149f320]
2024-07-16 19:45:15 ActiveStream 0x5657ec94000, stream_id_: 17666413969032772578&filter_manager_:
2024-07-16 19:45:15   FilterManager 0x5657ec940b0, state_.has_1xx_headers_: 0
2024-07-16 19:45:15   filter_manager_callbacks_.requestHeaders():
2024-07-16 19:45:15     ':authority', 'localhost:8000'
2024-07-16 19:45:15     ':path', '/'
2024-07-16 19:45:15     ':method', 'GET'
2024-07-16 19:45:15     ':scheme', 'http'
2024-07-16 19:45:15     'sec-ch-ua', '"Not/A)Brand";v="8", "Chromium";v="126", "Google Chrome";v="126"'
2024-07-16 19:45:15     'sec-ch-ua-mobile', '?0'
2024-07-16 19:45:15     'sec-ch-ua-platform', '"Windows"'
2024-07-16 19:45:15     'upgrade-insecure-requests', '1'
2024-07-16 19:45:15     'user-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36'
2024-07-16 19:45:15     'accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7'
2024-07-16 19:45:15     'sec-fetch-site', 'none'
2024-07-16 19:45:15     'sec-fetch-mode', 'navigate'
2024-07-16 19:45:15     'sec-fetch-user', '?1'
2024-07-16 19:45:15     'sec-fetch-dest', 'document'
2024-07-16 19:45:15     'accept-encoding', 'gzip, deflate, br, zstd'
2024-07-16 19:45:15     'accept-language', 'en-US,en;q=0.9'
2024-07-16 19:45:15     'x-forwarded-proto', 'http'
2024-07-16 19:45:15     'x-request-id', '85cc83d7-80ee-4c84-82a8-ca98db81e8d7'
2024-07-16 19:45:15   filter_manager_callbacks_.requestTrailers():   null
2024-07-16 19:45:15   filter_manager_callbacks_.responseHeaders():   null
2024-07-16 19:45:15   filter_manager_callbacks_.responseTrailers():   null
2024-07-16 19:45:15   &streamInfo():
2024-07-16 19:45:15     StreamInfoImpl 0x5657ec941f0, protocol_: 1, response_code_: null, response_code_details_: null, attempt_count_: null, health_check_request_: 0, getRouteName():     upstream_info_:     null
2024-07-16 19:45:15     OverridableRemoteConnectionInfoSetterStreamInfo 0x5657ec941f0, remoteAddress(): 172.18.0.1:51618, directRemoteAddress(): 172.18.0.1:51618, localAddress(): 172.18.0.3:8000
2024-07-16 19:45:15 Http1::ConnectionImpl 0x5657ec97810, dispatching_: 1, dispatching_slice_already_drained_: 0, reset_stream_called_: 0, handling_upgrade_: 0, deferred_end_stream_headers_: 1, processing_trailers_: 0, buffered_body_.length(): 0, header_parsing_state_: Done, current_header_field_: , current_header_value_:
2024-07-16 19:45:15 active_request_:
2024-07-16 19:45:15 , request_url_: null, response_encoder_.local_end_stream_: 0
2024-07-16 19:45:15 absl::get<RequestHeaderMapPtr>(headers_or_trailers_): null
2024-07-16 19:45:15 current_dispatching_buffer_ front_slice length: 663 contents: "GET / HTTP/1.1\r\nHost: localhost:8000\r\nConnection: keep-alive\r\nsec-ch-ua: \"Not/A)Brand\";v=\"8\", \"Chromium\";v=\"126\", \"Google Chrome\";v=\"126\"\r\nsec-ch-ua-mobile: ?0\r\nsec-ch-ua-platform: \"Windows\"\r\nUpgrade-Insecure-Requests: 1\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7\r\nSec-Fetch-Site: none\r\nSec-Fetch-Mode: navigate\r\nSec-Fetch-User: ?1\r\nSec-Fetch-Dest: document\r\nAccept-Encoding: gzip, deflate, br, zstd\r\nAccept-Language: en-US,en;q=0.9\r\n\r\n"
2024-07-16 19:45:15 ConnectionImpl 0x5657ec5ddb0, connecting_: 0, bind_error_: 0, state(): Open, read_buffer_limit_: 1048576
2024-07-16 19:45:15 socket_:
2024-07-16 19:45:15   ListenSocketImpl 0x5657ec52360, transport_protocol_: raw_buffer
2024-07-16 19:45:15   connection_info_provider_:
2024-07-16 19:45:15     ConnectionInfoSetterImpl 0x5657ec54e18, remote_address_: 172.18.0.1:51618, direct_remote_address_: 172.18.0.1:51618, local_address_: 172.18.0.3:8000, server_name_: 

MySQL

envoy.yaml using luasql Mysql library : 

https://github.com/thiva-k/mysql-lua-test/blob/main/envoy.yaml

In this case, just importing the library itself  as in the code above is giving the below error. Note that the library path is being correctly recognized, the issue comes when importing it into the Lua envoy filter section. This same library works perfectly works in a Lua environment outside envoy.

2024-07-16 20:53:36 [2024-07-16 15:23:36.570][1][critical][backtrace] [./source/server/backtrace.h:127] Caught Segmentation fault, suspect faulting address 0x4c0d0
2024-07-16 20:53:36 [2024-07-16 15:23:36.570][1][critical][backtrace] [./source/server/backtrace.h:111] Backtrace (use tools/stack_decode.py to get line numbers):
2024-07-16 20:53:36 [2024-07-16 15:23:36.570][1][critical][backtrace] [./source/server/backtrace.h:112] Envoy version: 816188b86a0a52095b116b107f576324082c7c02/1.30.1/Clean/RELEASE/BoringSSL
2024-07-16 20:53:36 [2024-07-16 15:23:36.570][1][critical][backtrace] [./source/server/backtrace.h:114] Address mapping: 5607f017c000-5607f3b5b000 /envoy
2024-07-16 20:53:36 [2024-07-16 15:23:36.570][1][critical][backtrace] [./source/server/backtrace.h:121] #0: [0x7f8ab1f7d320]

Please help me to identify the issue and suggest any potential ways to connect to a MySQL instance directly through Lua script within Envoy.

Thank you/

[yavl...@google.com]

You can see if anyone on the #envoy-dev Slack channel can help you.