David Peterson 9afb52aecb Add --chunk-rows option to load_folder.py for customizable memory management ... Introduced a new command-line argument, --chunk-rows, allowing users to specify the number of rows per chunk for pyreadstat streaming and COPY operations. This option overrides the GENERIC_LOADER_CHUNK_ROWS environment variable and auto-scaling behavior when using multiple workers. Enhanced memory management by providing detailed information on peak memory usage based on the specified chunk size, improving performance and usability during large dataset processing.		2026-04-21 10:05:21 -05:00
..
samples	adding explorer	2026-04-20 16:27:54 -05:00
.env.example	Adding generic loader	2026-04-18 09:34:48 -05:00
generate_sample_folder.py	Add generate_sample_folder.py and load_folder.py for clustered SAS file generation and loading	2026-04-18 11:25:04 -05:00
generate_sample_sas.py	Adding generic loader	2026-04-18 09:34:48 -05:00
load_folder.py	Add --chunk-rows option to load_folder.py for customizable memory management	2026-04-21 10:05:21 -05:00
load_sas.py	Add --chunk-rows option to load_folder.py for customizable memory management	2026-04-21 10:05:21 -05:00
sample_config.yaml	adding index field	2026-04-20 10:18:09 -05:00
sample_folder_config.yaml	Enhance file clustering by implementing numeric sorting for last digit groups in stems and updating documentation for embedded-digit handling in auto-detection.	2026-04-20 11:48:22 -05:00