Code examples and Python notebook

Python and Javascript examples and a ready-to-run Python notebook

The HDX metadata endpoints can be used with many different coding languages and processes in the humanitarian sector. See specific code examples below.

Note that you can access the endpoints directly in your browser as well.

See Python specific examples below.

While these endpoints are designed for lightweight, token-free access, two Python libraries are available if you prefer a higher-level interface: ckanapi, a low-level library from CKAN, and the HDX Python API, which builds on it and is primarily intended for data contributors publishing data to HDX.

Search HDX for the most recently updated datasets of South Sudan for returnees data

import requests
response = requests.get(
    "https://data.humdata.org/api/action/package_search",
    params={
        "q": "returnees",
        "fq": "groups:ssd",
        "sort": "metadata_modified desc",
        "rows": 5
    }
)

data = response.json()
for dataset in data["result"]["results"]:
    print(dataset["title"])
    print(f"  Updated: {dataset['metadata_modified'][:10]}")

Search for datasets from UNFPA for population

import requests
response = requests.get(
    "https://data.humdata.org/api/action/package_search",
    params={
        "q": "population",
        "fq": "organization:unfpa",
        "rows": 10
    }
)

data = response.json()
print(f"Found {data['result']['count']} datasets")

for dataset in data["result"]["results"]:
    print(f"• {dataset['title']}")

Get all downloadable resources for a specific dataset

import requests
dataset_id = "ukraine-border-crossings"

response = requests.get(
    "https://data.humdata.org/api/action/package_show",
    params={"id": dataset_id}
)

data = response.json()
dataset = data["result"]

print(f"Dataset: {dataset['title']}")
print(f"Resources: {len(dataset['resources'])}\n")

for resource in dataset["resources"]:
    print(f" {resource['name']}")
    print(f"   Format: {resource.get('format', 'N/A')}")
    if 'download_url' in resource:
        print(f"   Download: {resource['download_url']}")
    print()

See Javascript specific examples below.

For JavaScript, there are also CKAN client libraries available, including a newer option from Datopian and a legacy version.

Search HDX for the most recently updated datasets of South Sudan for returnees data

const params = new URLSearchParams({
    q: "returnees",
    fq: "groups:ssd",
    sort: "metadata_modified desc",
    rows: 5
});

fetch(`https://data.humdata.org/api/action/package_search?${params}`)
    .then(response => response.json())
    .then(data => {
        data.result.results.forEach(dataset => {
            console.log(dataset.title);
            console.log(`  Updated: ${dataset.metadata_modified.substring(0, 10)}`);
        });
    });

Search for datasets from UNFPA for population

const params = new URLSearchParams({
    q: "population",
    fq: "organization:unfpa",
    rows: 10
});

fetch(`https://data.humdata.org/api/action/package_search?${params}`)
    .then(response => response.json())
    .then(data => {
        console.log(`Found ${data.result.count} datasets`);
        data.result.results.forEach(dataset => {
            console.log(`• ${dataset.title}`);
        });
    });

Get all downloadable resources for a specific dataset

const datasetId = "ukraine-border-crossings";
const params = new URLSearchParams({ id: datasetId });

fetch(`https://data.humdata.org/api/action/package_show?${params}`)
    .then(response => response.json())
    .then(data => {
        const dataset = data.result;
        console.log(`Dataset: ${dataset.title}`);
        console.log(`Resources: ${dataset.resources.length}\n`);
        
        dataset.resources.forEach(resource => {
            console.log(`${resource.name}`);
            console.log(`   Format: ${resource.format || 'N/A'}`);
            if (resource.download_url) {
                console.log(`   Download: ${resource.download_url}`);
            }
        });
    });

Python notebook

We provide a ready-to-run Python ipynb notebook showing how to query the metadata endpoints (package_search and package_show) for Python scripts and pipelines. You can open it directly in your browser via GitHub and test queries without needing to install anything locally.

Open the Python notebook here.

Important:

For the advanced section, you must provide your own HDX API token in the notebook.
Never commit your token or share it publicly! Treat it like a password.

The notebook covers:

Searching the HDX catalogue using package_search with q and fq filters
Aggregating catalogue stats using facet queries with rows=0
Extracting full dataset metadata via package_show
Automating shapefile downloads and loading into a GeoDataFrame
Querying tabular data via TDE (datastore_info, datastore_search)

PreviousOpenAPI specification NextTroubleshooting and error handling

Last updated 2 months ago

Was this helpful?