Skip to content

Add an option to load ABP locally, or make it clearer in the docs #35

Description

@dodgydave67

Hi there, some departments/orgs prohibit use of APIs but still have ABP stored on locally/cloud/databricks.

There isn't a clear option (unless I'm missing something) to load ABP data without an API in the docs.

This is the way I did it, by creating a config.yaml file and editing the ukam_os_builder steps:

config.yaml:

source:
  type: abp

paths:
  work_dir: /dbfs/mnt/your_storage_folder/addressbase_working

os_downloads:
  package_id: "0"
  version_id: "0"

processing:
  num_chunks: 10
  duckdb_memory_limit: "16GB"

Code to run:

from ukam_os_builder.api.settings import load_settings
from ukam_os_builder.data_sources.abp.transform.runner import run_flatfile_step

settings = load_settings("config.yaml")
output_files = run_flatfile_step(settings, force=True)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions