foxtrot/generic_loader
David Peterson 96f2d6fe79 Update requirements and enhance SAS file processing with progress tracking
Updated the pyarrow version in requirements.txt to improve compatibility. Enhanced the _infer_cluster_schema and _stream_file functions in load_folder.py and load_sas.py to return total row counts for better progress tracking during data streaming. Integrated tqdm for visual feedback on row processing, improving user experience during large data loads.
2026-04-20 21:44:49 -05:00
..
samples adding explorer 2026-04-20 16:27:54 -05:00
.env.example Adding generic loader 2026-04-18 09:34:48 -05:00
generate_sample_folder.py Add generate_sample_folder.py and load_folder.py for clustered SAS file generation and loading 2026-04-18 11:25:04 -05:00
generate_sample_sas.py Adding generic loader 2026-04-18 09:34:48 -05:00
load_folder.py Update requirements and enhance SAS file processing with progress tracking 2026-04-20 21:44:49 -05:00
load_sas.py Update requirements and enhance SAS file processing with progress tracking 2026-04-20 21:44:49 -05:00
sample_config.yaml adding index field 2026-04-20 10:18:09 -05:00
sample_folder_config.yaml Enhance file clustering by implementing numeric sorting for last digit groups in stems and updating documentation for embedded-digit handling in auto-detection. 2026-04-20 11:48:22 -05:00