ConnectorX

[Francois Loubser]

Hi

This might be of interest to the data scientists with data in databases:

ConnectorX uses a Rust library at its core. This allows for optimizations like being able to load from a data source in parallel with partitioning. Data in PostgreSQL, for instance, can be loaded this way by specifying a partition column.

Aside from PostgreSQL, ConnectorX also supports reading from MySQL/MariaDB, SQLite, Amazon Redshift, Microsoft SQL Server and Azure SQL, and Oracle. The results can be funneled into a Pandas or PyArrow DataFrame, or into Modin, Dask, or Polars by way of PyArrow.

https://towardsdatascience.com/connectorx-the-fastest-way-to-load-data-from-databases-a65d4d4062d5

https://github.com/sfu-db/connector-x

Francois

[Tobias Brandt]

Thanks Francois. ConnectorX does look great and the Python API is nice and simple.

I tried to use it from Rust and unfortunately the Rust API is not nearly as nice as the Python one. It's an unfortunate trait I've found with a few Python libraries written in Rust - they expose a great Python API but most of that is pulled together in the Python interface code and they're not nearly as easy to use from Rust itself.

--
You received this message because you are subscribed to the Google Groups "Cape Town Python Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ctpug+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ctpug/CAMFrngpoOrEPXETkGU8QAZm3d-VHsjG-FVOwU%3DSF7Hft9pRFrQ%40mail.gmail.com.