David Peterson eac75cbb26 Refactor load_cluster function in load_folder.py for improved parallel file loading ... Updated the load_cluster function to enhance parallel processing by committing the table creation before dispatching all files to worker processes. This change allows for more efficient handling of large datasets by reducing the serial workload and ensuring schema compatibility checks can access the committed table. The logic for streaming files has been clarified, maintaining progress tracking throughout the loading process.		2026-04-21 08:31:48 -05:00
..
samples	adding explorer	2026-04-20 16:27:54 -05:00
.env.example	Adding generic loader	2026-04-18 09:34:48 -05:00
generate_sample_folder.py	Add generate_sample_folder.py and load_folder.py for clustered SAS file generation and loading	2026-04-18 11:25:04 -05:00
generate_sample_sas.py	Adding generic loader	2026-04-18 09:34:48 -05:00
load_folder.py	Refactor load_cluster function in load_folder.py for improved parallel file loading	2026-04-21 08:31:48 -05:00
load_sas.py	Enhance date and timestamp handling in _prepare_for_copy function in load_sas.py	2026-04-21 08:16:17 -05:00
sample_config.yaml	adding index field	2026-04-20 10:18:09 -05:00
sample_folder_config.yaml	Enhance file clustering by implementing numeric sorting for last digit groups in stems and updating documentation for embedded-digit handling in auto-detection.	2026-04-20 11:48:22 -05:00