foxtrot/generic_loader
David Peterson 5b48872dd7 Add generate_sample_folder.py and load_folder.py for clustered SAS file generation and loading
Introduce generate_sample_folder.py to create a test folder with clustered SAS XPORT files, including configurations for schema compatibility checks. Implement load_folder.py to facilitate loading entire directories of SAS files into Postgres, supporting explicit and auto-detect clustering. Update sample_folder_config.yaml for usage examples and configuration structure. Enhance load_sas.py with a public schema compatibility check function for orchestrators.
2026-04-18 11:25:04 -05:00
..
.env.example Adding generic loader 2026-04-18 09:34:48 -05:00
.gitignore Update load_sas.py to support streaming data loads with iter_sas_chunks and copy_dataframes. Enhance documentation for schema inference and type detection, clarifying the use of read_sas_preview and the implications of sampling. Add __pycache__ to .gitignore. 2026-04-18 10:44:32 -05:00
generate_sample_folder.py Add generate_sample_folder.py and load_folder.py for clustered SAS file generation and loading 2026-04-18 11:25:04 -05:00
generate_sample_sas.py Adding generic loader 2026-04-18 09:34:48 -05:00
load_folder.py Add generate_sample_folder.py and load_folder.py for clustered SAS file generation and loading 2026-04-18 11:25:04 -05:00
load_sas.py Add generate_sample_folder.py and load_folder.py for clustered SAS file generation and loading 2026-04-18 11:25:04 -05:00
requirements.txt Adding generic loader 2026-04-18 09:34:48 -05:00
sample_config.yaml Adding generic loader 2026-04-18 09:34:48 -05:00
sample_folder_config.yaml Add generate_sample_folder.py and load_folder.py for clustered SAS file generation and loading 2026-04-18 11:25:04 -05:00