persona_db/scripts/bulk_insert.py
Stream JSON table rows into PostgreSQL using bounded insert batches.
This loader accepts unqualified table identifiers and uses ijson to avoid
loading the whole dataset (or even one complete table) into memory. Apply the
schema first. Install dependencies with pip install -e persona_db.
:::info Fonte
Módulo persona_db.scripts.bulk_insert — 106 linhas · 0 classe(s) · 2 função(ões) pública(s). Ver no GitHub
:::
Funções
load_dataset()
def load_dataset(database_url: str, source: Path, page_size: int=1000) -> int
Insert table rows from a JSON snapshot in bounded-memory batches.
persona_db/scripts/bulk_insert.py:52
main()
def main() -> int