Scout: A Cloud-Native Geospatial Pipeline Powered by DuckDB, GeoParquet, PMTiles, STAC and AI

By Sarah GamalAugust 3, 20263 min read
Scout: A Cloud-Native Geospatial Pipeline Powered by DuckDB, GeoParquet, PMTiles, STAC and AI

The Problem with Traditional Tile Workflow

If you've ever tried to publish a vector dataset as web tiles, you know the pain. You typically need a tile server running somewhere, a PostGIS database loaded with your data, a tiling tool like Tippecanoe, and then a CDN in front of all of it. That's four moving parts before anyone can see your data on a map.

I built Scout to collapse that entire stack into a single pipeline that runs in your browser and publishes to cloud storage, no servers, no databases, no DevOps.

What Scout Does

Paste any vector data URL: GeoJSON, GeoPackage, Shapefile, or GeoParquet, and Scout does the rest:

Data URL → GeoParquet → PMTiles → XYZ endpoints → STAC catalog

Within seconds you have:

- A cloud-optimized GeoParquet file ready for analytics

- A single-file PMTiles archive serving vector tiles

- Live XYZ tile endpoints you can plug directly into QGIS, ArcGIS, or MapLibre GL

- A STAC 1.0.0-compliant catalog with all assets bundled and discoverable

- Everything optionally published to Cloudflare R2 with one click

The Architecture

DuckDB Spatial: The Engine Room

The entire ingestion pipeline runs inside DuckDB with its spatial extension. DuckDB's ability to read remote files over HTTP — including GeoParquet, GeoJSON, and even Shapefiles; means the pipeline never needs to write intermediate files to disk unnecessarily.

The pipeline uses DuckDB to:

1. Read the source data from the URL

2. Reproject to WGS84 if needed

3. Serialize geometry as WKB

4. Write out cloud-optimized GeoParquet

DuckDB handles this in-process inside a Next.js API route, which means the whole thing runs serverlessly — no external database required.

GeoParquet: The Analytics Layer

GeoParquet is the cloud-native successor to Shapefile for analytics. Scout stores every ingested dataset as a GeoParquet file alongside its PMTiles archive. This gives you two things at once: a tile-optimized format for rendering and a columnar format for fast analytical queries.

The STAC item for each layer carries a link to both files, so downstream tools can pick whichever format they need.

PMTiles v3: Serverless Tiles

PMTiles is a single-file archive format for map tiles. Instead of a tile server generating tiles on demand, a PMTiles file is hosted on any object storage (R2, S3, GCS) and clients fetch only the tiles they need using HTTP range requests.

Scout writes a fully spec-compliant PMTiles v3 archive directly in TypeScript — including the Hilbert curve spatial index that makes range requests efficient. The result: vector tiles served from Cloudflare R2 with no tile server, no egress fees beyond R2's generous free tier, and global CDN performance.

STAC: Discoverability Built In

Every publish generates a STAC 1.0.0-compliant catalog hierarchy:

catalog.json

└── collection.json

└── item.json (per layer)

├── PMTiles asset

├── GeoParquet asset

└── style.json asset

Clicking "Browse STAC Catalog" after publishing opens the catalog directly in STAC Browser v5. This means every dataset Scout publishes is immediately discoverable by any STAC-compatible tool — SpatioTemporal Asset Catalog clients, QGIS STAC plugins, or custom pipelines.

AI-Powered SQL Explorer

Scout includes a built-in SQL Explorer powered by Claude. You can type a natural language prompt like:

▎ "Show me all buildings taller than 50 metres in the downtown area" and Claude translates it to DuckDB spatial SQL, runs it against your GeoParquet layer, and renders the results on the map.

The Monaco editor lets you inspect and edit the generated SQL before running it, so you stay in control. This is particularly useful for GIS professionals who know what they want spatially but don't want to hand-craft ST_Within and ST_DWithin predicates every time.

Cloud Publishing in One Click

The Publish to Cloud button in Scout's Export Bar:

1. Uploads PMTiles and GeoParquet for every layer to Cloudflare R2 (or any cloud storage)

2. Builds the STAC catalog hierarchy and writes catalog.json, collection.json, and item.json

3. Generates a style.json with PMTiles sources ready for MapLibre GL

4. Returns a direct link to browse the published catalog in STAC Browser

From that point, the XYZ tile URLs (backed by the PMTiles archive on R2) work in any standard GIS client — no authentication, no tile server, just HTTP range requests to object storage.

Who Scout Is For

Scout is designed for GIS professionals who need to:

- Rapidly prototype a web map from any vector source without standing up infrastructure

- Publish datasets in a standards-compliant, discoverable format (STAC + GeoParquet)

- Share tile endpoints with colleagues using QGIS, ArcGIS, or Mapbox GL

- Query spatial data with natural language without leaving the map

Related Articles