3 Commits

Author	SHA1	Message	Date
David Peterson	4fc85081c8	Enhance SAS profiling performance in sas_profiler.py ... Added a new constant for profiling chunk size to optimize memory usage during profiling operations. Refactored the update method in the _ColumnStats class to improve efficiency in handling missing values and calculating statistics for numeric and string data types. This update includes vectorized operations for better performance and clarity in the implementation.	2026-04-20 19:03:40 -05:00
David Peterson	5449a25b44	Refactor partition candidate logic in sas_profiler.py ... Updated the partition candidate selection process to restrict candidates to columns matching specific name patterns, improving accuracy and reducing noise. Removed outdated distinct value constraints and clarified documentation for partitioning behavior. Enhanced handling of pre-sharded columns and refined the classification logic for better performance.	2026-04-20 18:49:23 -05:00
David Peterson	f1af1136dc	Add standalone SAS profiling utility ... Introduced a new script `sas_profiler.py` that profiles local SAS files and generates an Excel report with recommendations for drops, partitions, and indexes, along with type-inference warnings. The utility supports command-line overrides for configuration and is compatible with Python 3.10+. This addition enhances the existing tools for SAS file management.	2026-04-20 18:38:01 -05:00