Commit Graph

5 Commits

Author SHA1 Message Date
David Peterson
7beb44ac4d Add pyarrow dependency and optimize DataFrame serialization in load_sas.py
Included pyarrow as a new dependency in requirements.txt for improved CSV serialization performance. Refactored the _prepare_for_copy function to utilize vectorized operations for date and timestamp conversions, reducing CPU overhead. Introduced a new _serialize_chunk_csv function leveraging pyarrow for faster CSV writing, enhancing efficiency during data copying to Postgres.
2026-04-20 21:32:56 -05:00
David Peterson
b3b968edf2 Add openpyxl dependency to requirements.txt for Excel file handling 2026-04-20 18:38:24 -05:00
David Peterson
384103f489 Update pyreadstat version constraint in requirements.txt to allow for version 2.0 2026-04-20 14:10:08 -05:00
michael-corey
f101eacffd Merging main 2026-04-18 13:39:37 -05:00
michael-corey
edb9146682 moving files 2026-04-18 13:35:32 -05:00