foxtrot/generic_loader
David Peterson f84e127796 Update type inference behavior in load_sas.py to scan entire files by default
Changed the default setting for TYPE_INFERENCE_SAMPLE_ROWS to None, allowing type and nullability inference to consider all rows in a SAS file. This adjustment ensures accurate handling of null values and integer ranges, addressing issues observed in production with large datasets. Updated documentation to reflect the implications of this change and the risks associated with using an integer cap for sampling.
2026-04-20 20:43:27 -05:00
..
samples adding explorer 2026-04-20 16:27:54 -05:00
.env.example Adding generic loader 2026-04-18 09:34:48 -05:00
generate_sample_folder.py Add generate_sample_folder.py and load_folder.py for clustered SAS file generation and loading 2026-04-18 11:25:04 -05:00
generate_sample_sas.py Adding generic loader 2026-04-18 09:34:48 -05:00
load_folder.py Enhance file clustering by implementing numeric sorting for last digit groups in stems and updating documentation for embedded-digit handling in auto-detection. 2026-04-20 11:48:22 -05:00
load_sas.py Update type inference behavior in load_sas.py to scan entire files by default 2026-04-20 20:43:27 -05:00
sample_config.yaml adding index field 2026-04-20 10:18:09 -05:00
sample_folder_config.yaml Enhance file clustering by implementing numeric sorting for last digit groups in stems and updating documentation for embedded-digit handling in auto-detection. 2026-04-20 11:48:22 -05:00