Go to file
David Peterson 7beb44ac4d Add pyarrow dependency and optimize DataFrame serialization in load_sas.py
Included pyarrow as a new dependency in requirements.txt for improved CSV serialization performance. Refactored the _prepare_for_copy function to utilize vectorized operations for date and timestamp conversions, reducing CPU overhead. Introduced a new _serialize_chunk_csv function leveraging pyarrow for faster CSV writing, enhancing efficiency during data copying to Postgres.
2026-04-20 21:32:56 -05:00
generic_loader Add pyarrow dependency and optimize DataFrame serialization in load_sas.py 2026-04-20 21:32:56 -05:00
utils Refine partition name patterns in sas_profiler.py 2026-04-20 19:27:01 -05:00
.gitignore adding explorer 2026-04-20 16:27:54 -05:00
requirements.txt Add pyarrow dependency and optimize DataFrame serialization in load_sas.py 2026-04-20 21:32:56 -05:00