OFRA/foxtrot
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

removing latin encoding, adding usage notes

Browse Source
This commit is contained in:
michael-corey 2026-04-18 12:54:29 -05:00
parent c1e1fec10b
commit 1bbe0d4cd6
2 changed files with 16 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
generic_loader/load_folder.py
View File

@ -47,6 +47,7 @@ USAGE
:: ::
python load_folder.py --config folder_config.yaml [--dry-run] [--fail-fast] python load_folder.py --config folder_config.yaml [--dry-run] [--fail-fast]
[--dbcreds]
Flags: Flags:
--config PATH Required. Path to the YAML config above. --config PATH Required. Path to the YAML config above.
@ -56,6 +57,11 @@ Flags:
--fail-fast Abort the whole run on the first cluster failure. --fail-fast Abort the whole run on the first cluster failure.
Default is to log the failure, roll that cluster back, Default is to log the failure, roll that cluster back,
and keep going. and keep going.
--dbcreds Prompt interactively for the database username and
password instead of reading ``PGUSER`` / ``PGPASSWORD``
from the environment or ``.env`` file. The password
prompt does not echo. Has no effect with ``--dry-run``
(no connection is opened).
Exit codes: Exit codes:
0 - every cluster loaded successfully (or dry-run completed) 0 - every cluster loaded successfully (or dry-run completed)

11
generic_loader/load_sas.py
View File

@ -57,6 +57,7 @@ the vars) before calling :func:`connect`.
:: ::
python load_sas.py --config path/to/config.yaml [--validate] [--dry-run] python load_sas.py --config path/to/config.yaml [--validate] [--dry-run]
[--dbcreds]
Flags: Flags:
--config PATH Required. Path to the YAML config above. --config PATH Required. Path to the YAML config above.
@ -66,6 +67,11 @@ Flags:
``--dry-run``. ``--dry-run``.
--dry-run Print the inferred ``CREATE TABLE`` SQL and stop. The --dry-run Print the inferred ``CREATE TABLE`` SQL and stop. The
database is never touched (no connection is opened). database is never touched (no connection is opened).
--dbcreds Prompt interactively for the database username and
password instead of reading ``PGUSER`` / ``PGPASSWORD``
from the environment or ``.env`` file. The password
prompt does not echo. Has no effect with ``--dry-run``
(no connection is opened).
Exit codes: Exit codes:
0 - success (load completed, or dry-run/validate passed) 0 - success (load completed, or dry-run/validate passed)
@ -85,6 +91,9 @@ Typical invocations::
# Actually load the data. # Actually load the data.
python load_sas.py --config sample_config.yaml python load_sas.py --config sample_config.yaml
# Load the data, prompting for credentials instead of using .env.
python load_sas.py --config sample_config.yaml --dbcreds
4. Expected-types manifest (``--validate``) 4. Expected-types manifest (``--validate``)
------------------------------------------- -------------------------------------------
``--validate`` looks for a JSON file named ``<sas-stem>.expected.json`` next ``--validate`` looks for a JSON file named ``<sas-stem>.expected.json`` next
@ -403,7 +412,7 @@ def _sas_reader(path: Path) -> Tuple[Any, Dict[str, Any]]:
if suffix in (".xpt", ".xport"): if suffix in (".xpt", ".xport"):
return pyreadstat.read_xport, {} return pyreadstat.read_xport, {}
if suffix == ".sas7bdat": if suffix == ".sas7bdat":
return pyreadstat.read_sas7bdat, {"encoding": "latin-1"} return pyreadstat.read_sas7bdat, {}
raise ValueError(f"Unsupported SAS file extension: {suffix}") raise ValueError(f"Unsupported SAS file extension: {suffix}")