dp
0 Followers · 1 Following
  • Joined on 2026-04-18
Repositories Projects Packages Public Activity Starred Repositories
dp approved OFRA/foxtrot#2 2026-04-20 00:06:23 +00:00
file_viewer
dp pushed to main at OFRA/foxtrot 2026-04-18 18:23:53 +00:00
1bbe0d4cd6 removing latin encoding, adding usage notes
c1e1fec10b Update requirements.txt to support new package versions and add boto3 dependency
3b913b2ca6 adding user prompt for db creds
5b48872dd7 Add generate_sample_folder.py and load_folder.py for clustered SAS file generation and loading
5645ff5597 Update load_sas.py to support streaming data loads with iter_sas_chunks and copy_dataframes. Enhance documentation for schema inference and type detection, clarifying the use of read_sas_preview and the implications of sampling. Add __pycache__ to .gitignore.
Compare 7 commits »
dp merged pull request OFRA/foxtrot#1 2026-04-18 18:23:52 +00:00
batch_folder_processing
dp pushed to batch_folder_processing at OFRA/foxtrot 2026-04-18 17:41:10 +00:00
c1e1fec10b Update requirements.txt to support new package versions and add boto3 dependency
dp created pull request OFRA/foxtrot#1 2026-04-18 16:26:28 +00:00
batch_folder_processing
dp pushed to batch_folder_processing at OFRA/foxtrot 2026-04-18 16:25:06 +00:00
5b48872dd7 Add generate_sample_folder.py and load_folder.py for clustered SAS file generation and loading
dp pushed to batch_folder_processing at OFRA/foxtrot 2026-04-18 15:44:34 +00:00
5645ff5597 Update load_sas.py to support streaming data loads with iter_sas_chunks and copy_dataframes. Enhance documentation for schema inference and type detection, clarifying the use of read_sas_preview and the implications of sampling. Add __pycache__ to .gitignore.
dp created branch batch_folder_processing in OFRA/foxtrot 2026-04-18 15:29:19 +00:00
dp pushed to batch_folder_processing at OFRA/foxtrot 2026-04-18 15:29:19 +00:00
3a0537270c Implement type inference sampling in load_sas.py to improve performance on large SAS files. Introduce TYPE_INFERENCE_SAMPLE_ROWS to limit the number of rows scanned for type detection while ensuring nullability checks cover the entire column. Update documentation to reflect these changes.
4f7ded09c6 Enhance load_sas.py with detailed usage instructions, YAML config structure, and command-line interface documentation for loading SAS files.
Compare 2 commits »