Go to file
David Peterson eac75cbb26 Refactor load_cluster function in load_folder.py for improved parallel file loading
Updated the load_cluster function to enhance parallel processing by committing the table creation before dispatching all files to worker processes. This change allows for more efficient handling of large datasets by reducing the serial workload and ensuring schema compatibility checks can access the committed table. The logic for streaming files has been clarified, maintaining progress tracking throughout the loading process.
2026-04-21 08:31:48 -05:00
generic_loader Refactor load_cluster function in load_folder.py for improved parallel file loading 2026-04-21 08:31:48 -05:00
utils Refine partition name patterns in sas_profiler.py 2026-04-20 19:27:01 -05:00
.gitignore adding explorer 2026-04-20 16:27:54 -05:00
requirements.txt Update requirements and enhance SAS file processing with progress tracking 2026-04-20 21:44:49 -05:00