Collect 3d city model open datasets

[HideBa]

Task: Collect Open 3D City Model Datasets

Objective: Create configuration files for open 3D city model datasets to build a comprehensive global STAC catalog.

Deliverable: Multiple YAML configuration files (one per dataset) in the collections/ directory.

Target Deadline: 📅 March 6th, 2026 (negotiable - see Section 2.1)

---

Table of Contents

1. Background
2. Prerequisites
3. Timeline & Deadline
4. Head Start: Existing Draft Configs
5. Understanding the Config File
6. Data Sources
7. Programmatic Data Collection
8. Step-by-Step Workflow
9. Common Scenarios
10. Quality Checklist
11. Naming Conventions
12. Submitting Your Work
13. Troubleshooting
14. Working with Uncertainty
15. Quick Reference

---

1. Background

What is STAC?

STAC (SpatioTemporal Asset Catalog) is a specification for describing geospatial data. Think of it as a standardized way to catalog and search for spatial datasets.

What are we building?

We're creating a global catalog of open 3D city models. These are digital representations of cities in 3D, including buildings, roads, vegetation, and more. Common formats include:

• CityJSON - A JSON format for 3D city models
• CityGML - An XML/GML format for 3D city models

Why this matters

Currently, finding open 3D city model datasets is difficult - they're scattered across various websites, portals, and repositories. By creating a standardized STAC catalog, we make it easy for:

• Researchers to discover and access 3D city data
• Developers to build applications using this data
• Cities to share their 3D models with the world

Your role

You'll be finding open 3D city model datasets and creating configuration files that describe each dataset. These configs will be used to automatically generate STAC metadata.

---

2. Prerequisites

What you need

• Text editor - Any code editor (VS Code, Sublime, etc.) or even a simple text editor
• Web browser - To access data sources and verify URLs
• Basic YAML understanding - YAML is a simple configuration format (see examples below)

Helpful but not required

• Familiarity with geospatial data concepts
• Understanding of coordinate reference systems (CRS)
• Knowledge of open data licenses

Setup

1. Clone or access this repository
2. Navigate to the project directory
3. Create a collections/ directory if it doesn't exist:

   mkdir -p collections

---

2.1 Timeline & Deadline

Target Deadline: March 6th, 2026 📅

This is a soft target - the deadline is negotiable based on what you discover.

Suggested Timeline

Week	Focus Area	Target Output
Week 1	Explore sources, collect URLs	15-20 dataset configs
Week 2	Continue collection, fill metadata	15-20 more dataset configs
Week 3	Finalize configs, validate, open PR	Complete PR with all configs

Scope is Flexible

The exact number of datasets is not fixed. What matters more:

• ✅ Quality over quantity - 20 well-documented configs > 50 poorly documented ones
• ✅ Geographic diversity - Datasets from different countries/regions
• ✅ Format variety - Mix of CityJSON and CityGML datasets
• ✅ URLs are real - Links point to actual data files, not just portals

💡 Work Smart: Use AI Tools!

This is not an assignment or exam - it's a real task where the goal is to get it done efficiently. You're encouraged to:

• ✅ Use AI assistants (ChatGPT, Claude, etc.) to help with:

  • Writing scraping scripts
  • Extracting metadata from web pages
  • Generating YAML configs from template
  • Debugging automation code
  • Researching licenses and data sources

• ✅ Leverage existing tools:

  • Browser extensions for web scraping
  • GitHub Copilot for code generation
  • AI-powered research tools

• ✅ Focus on outcomes, not process:

  • The sooner you finish, the better
  • No one will audit your workflow
  • Results matter more than how you got them

Example AI Prompts:

"Write a Python script to scrape all .city.json download links from https://3d.bk.tudelft.nl/opendata/opencities/"
"Extract license information from this webpage text and convert to SPDX identifier: [paste text]"
"Generate a YAML config for a CityJSON dataset with these properties: title=..., url=..., license=..."

Bottom line: If AI can help you finish this task faster and easier, use it. We care about the catalog, not how you build it.

Negotiable Aspects

If you're running into challenges, we can adjust:

Issue	Adjustment Option
Can't find enough datasets	Reduce target count, focus on quality
Datasets require complex access	Document in comments, mark for follow-up
License information unclear	Use proprietary + comments, move forward
Technical scraping difficulties	Switch to manual collection for remaining
Need more time	Extend deadline by 1-2 weeks

Communication

If you anticipate missing the deadline:

1. Don't wait until the last day - communicate early
2. Share what you have - open PR with partial collection
3. Explain blockers - what's slowing you down?
4. Propose adjustment - new deadline, reduced scope, or different approach

Remember: This task is about building a useful catalog, not hitting an arbitrary number. A PR with 30 well-documented datasets opened on March 8th is better than 50 rushed configs opened on March 6th.

---

2.2 Head Start: Existing Draft Configs 🚀

Good news! Draft configuration files already exist in the opendata/ directory that you can build upon.

What's Already There

The opendata/ directory contains 27 draft collection configs covering datasets from:

Region	Config Files	Count
Netherlands	netherlands-3d-bag, rotterdam, the-hague	3
Germany	berlin, dresden, hamburg, potsdam, ingolstadt, north-rhine-westphalia	6
Finland	helsinki, espoo, vantaa	3
Austria	vienna, linz, luxembourg	3
France	lyon	1
Belgium	brussels, namur	2
North America	new-york-doitt, new-york-tum, montreal, american-cities	4
Asia	singapore, japan-plateau	2
Other	estonia, various-cityjson	2

Plus: catalog-config.yaml for aggregating all collections

Your Task: Verify & Complete

Instead of starting from scratch, your job is to:

1. Review existing configs - Check each file in opendata/
2. Verify accuracy - Are URLs correct? Is license info accurate?
3. Fill gaps - Add missing metadata, fix incorrect information
4. Test URLs - Verify download links actually work
5. Add new datasets - Extend the collection with additional sources beyond these 27

Quick Start with Existing Configs

# List all existing configs
ls opendata/*.yaml

# Review a specific config
cat opendata/vienna-config.yaml

# Validate an existing config
cityjson-stac collection --config opendata/vienna-config.yaml --dry-run

# Validate all existing configs at once
for file in opendata/*-config.yaml; do
    echo "Validating $file..."
    cityjson-stac collection --config "$file" --dry-run
done

What to Check in Each Config

For each existing config, verify:

Field	What to Check
inputs URLs	Do the download links work?
license	Is the SPDX identifier correct?
providers	Are organization names and URLs accurate?
description	Is it detailed and accurate?
extent	Is the bounding box reasonable? (may be empty for auto-detection)
keywords	Are they relevant and useful?

Common Issues to Fix

Look for and correct:

• 🔗 Broken URLs - Links that redirect or return 404
• 📜 Wrong licenses - Generic proprietary when actual license is known
• 🏢 Missing providers - Incomplete organization information
• 📝 Vague descriptions - "3D city model" without details
• 🌍 Empty metadata - Missing geographic or temporal info

Example: Review Process

# 1. Pick a config to review
vi opendata/vienna-config.yaml

# 2. Check the URL works
curl -I https://data.data.gv.at/cat/vienna.city.json

# 3. Verify license on source page
# Visit the provider's website and confirm

# 4. Test validation
cityjson-stac collection --config opendata/vienna-config.yaml --dry-run

# 5. Fix any issues found
# Edit the config file

# 6. Re-validate
cityjson-stac collection --config opendata/vienna-config.yaml --dry-run

Bonus: Extend the Collection

Once you've verified the existing configs, add more datasets from:

• Additional cities in the TU Delft portal
• Other open data portals (municipal, national)
• Research repositories
• CityGML/CityJSON community sites

Summary

You're not starting from zero! You have:

• ✅ 27 draft configs to verify and improve
• ✅ A working template structure
• ✅ Coverage of major datasets already identified

Your job is quality assurance + expansion, not blank-slate creation.

---

3. Understanding the Config File

Each dataset needs a YAML configuration file. Here's the complete structure:

Required Fields

Every config file MUST have:

id: unique-identifier
title: Human Readable Title
description: Detailed description
license: SPDX-LICENSE-IDENTIFIER
providers:
  - name: Organization Name
    roles:
      - producer
inputs:
  - https://example.com/data.city.json

Complete Field Reference

Field	Required?	Description	Example
id	Yes	Unique identifier for the collection	3dbag-delft-2023
title	Yes	Human-readable name	3DBAG Delft
description	Yes	Detailed description (use | for multiline)	See below
license	Yes	SPDX license identifier	CC-BY-4.0
keywords	No	List of tags for categorization	["3d city model", "buildings"]
providers	Yes	List of organizations involved	See below
inputs	Yes	Data source URLs or API endpoints	See below
extent.spatial	No	Bounding box and CRS	See below
extent.temporal	No	Time range of data	See below
links	No	Related URLs (license, about, etc.)	See below
summaries	No	Custom metadata fields	Various

Provider Roles

Each provider should have at least one role:

Role	When to Use
producer	Organization that created the data
licensor	Organization that manages the license
processor	Organization that processed/transformed the data
host	Organization that hosts the data
curator	Organization that maintains the data catalog

Common License Identifiers

License	SPDX ID	When to Use
Creative Commons Attribution 4.0	CC-BY-4.0	Free to use with attribution
Creative Commons Zero	CC0-1.0	Public domain, no attribution required
Open Data Commons Attribution	ODC-BY-1.0	Open data with attribution
MIT License	MIT	Permissive software license
Not specified	proprietary	Unknown or proprietary license

Find more: https://spdx.org/licenses/

---

5. Data Sources

Primary Sources to Explore

1. Awesome CityGML (GitHub)

🔗 https://github.com/OloOcki/awesome-citygml#World

A curated list of open 3D city model datasets organized by country. This is your main source.

How to use:

• Navigate to different country sections
• Click through to dataset pages
• Extract metadata from those pages

2. TU Delft Open Cities

🔗 https://3d.bk.tudelft.nl/opendata/opencities/

Academic repository of open 3D city models maintained by TU Delft.

How to use:

• Browse by city or country
• Check for download links or API documentation
• Note any special access requirements

Additional Discovery

Look for datasets in:

• Municipal open data portals - Search "[city name] open data 3d city model"
• National geospatial portals - Many countries have national clearinghouses
• Research repositories - University datasets often available
• CityGML/CityJSON communities - Forums and community sites

What to Include

✅ Include datasets that are:

• Openly accessible (no payment required)
• In CityJSON or CityGML format
• Documented with a license
• From verifiable sources

❌ Don't include:

• Commercial datasets requiring payment
• Unpublished or internal datasets
• Broken or inaccessible links
• Datasets with unclear licensing

---

5.1 Programmatic Data Collection

⚠️ Important: Manually collecting URLs for dozens of datasets is time-consuming and error-prone. Consider using programmatic approaches to automate data discovery and URL extraction.

Why Automate?

• Scale: You'll likely process 50+ datasets
• Accuracy: Prevents copy-paste errors
• Maintainability: Easy to update when sources change
• Reproducibility: Can verify and re-run collection process

Recommended Approaches

1. Web Scraping

For datasets with predictable page structures:

# Example: Scrape TU Delft Open Cities portal
import requests
from bs4 import BeautifulSoup

url = "https://3d.bk.tudelft.nl/opendata/opencities/"
response = requests.get(url)
soup = BeautifulSoup(response.content, 'html.parser')

# Extract download links
for link in soup.select('a[href$=".city.json"]'):
    title = link.text.strip()
    href = link['href']
    print(f"{title}: {href}")

Tools:

• Python: requests, beautifulsoup4, scrapy
• JavaScript: playwright, puppeteer

2. API Queries

For portals with APIs:

# Example: Query a CKAN-based open data portal
import requests

base_url = "https://data.example.com/api/3/action/package_search"
params = {
    'q': 'citygml OR cityjson',
    'rows': 100,
    'start': 0
}

response = requests.get(base_url, params=params)
data = response.json()

for result in data['result']['results']:
    name = result['name']
    title = result['title']
    download_url = result['resources'][0]['url']
    # Generate YAML config...

3. GitHub Scraping

For GitHub-hosted datasets:

# Example: Find CityJSON files in a repository
import requests
import re

# Search GitHub code for .city.json files
api_url = "https://api.github.com/search/code"
params = {
    'q': 'extension:city.json language:JSON',
    'per_page': 100
}

headers = {
    'Accept': 'application/vnd.github.v3+json',
    # Add your token: 'Authorization': 'token YOUR_TOKEN'
}

response = requests.get(api_url, params=params, headers=headers)
results = response.json()

for item in results['items']:
    print(f"{item['name']}: {item['html_url']}")

GitHub Search Query:

extension:city.json OR extension:gml stars:>10 pushed:>2020

4. Directory Listing Parsing

For web-accessible directories:

# Example: Parse Apache directory listing
import requests
from lxml import html

url = "https://example.com/data/cityjson/"
response = requests.get(url)
tree = html.fromstring(response.content)

# Extract all .city.json links
for link in tree.xpath('//a[contains(@href, ".city.json")]'):
    filename = link.text
    full_url = url + filename
    print(f"{filename}: {full_url}")

Automation Workflow

1. Discovery Script

   python scripts/discover_datasets.py > sources/raw_urls.txt

2. Metadata Extraction

   # Parse source pages for metadata (license, provider, etc.)
   python scripts/extract_metadata.py sources/raw_urls.txt > configs/

3. Config Generation

   # Auto-generate YAML configs
   python scripts/generate_configs.py sources/metadata.json --output collections/

4. Validation

   # Validate all generated configs
   for file in collections/*.yaml; do
       cityjson-stac collection --config "$file" --dry-run
   done

Example: End-to-End Script

#!/usr/bin/env python3
"""
Collect 3D city model datasets from Awesome CityGML list
"""
import requests
from bs4 import BeautifulSoup
import yaml

SOURCES = {
    'awesome_citygml': 'https://github.com/OloOcki/awesome-citygml',
    'tudelft_opencities': 'https://3d.bk.tudelft.nl/opendata/opencities/',
}

def scrape_github_readme(url):
    """Extract dataset links from GitHub README"""
    # Implementation...
    pass

def scrape_tudelft_portal(url):
    """Extract datasets from TU Delft portal"""
    # Implementation...
    pass

def generate_config(dataset):
    """Generate YAML config from dataset info"""
    config = {
        'id': dataset['id'],
        'title': dataset['title'],
        'description': dataset['description'],
        'license': dataset.get('license', 'proprietary'),
        'providers': dataset['providers'],
        'inputs': dataset['urls'],
        'keywords': dataset.get('keywords', [])
    }
    return config

def main():
    all_datasets = []

    for source_name, source_url in SOURCES.items():
        print(f"Scraping {source_name}...")
        if 'github' in source_url:
            datasets = scrape_github_readme(source_url)
        else:
            datasets = scrape_tudelft_portal(source_url)
        all_datasets.extend(datasets)

    # Generate YAML configs
    for dataset in all_datasets:
        config = generate_config(dataset)
        filename = f"collections/{dataset['id']}.yaml"
        with open(filename, 'w') as f:
            yaml.dump(config, f, default_flow_style=False)
            print(f"Generated {filename}")

if __name__ == '__main__':
    main()

Best Practices

1. Respect rate limits - Add delays between requests
2. Cache responses - Save scraped data to avoid re-fetching
3. Handle errors gracefully - Log failures, don't crash on single errors
4. Validate inputs - Check URLs are accessible before generating configs
5. Document sources - Keep track of where data came from
6. Version control - Commit scraping scripts alongside configs

Tools & Libraries

Task	Python	JavaScript
HTTP requests	requests, httpx	axios, node-fetch
HTML parsing	beautifulsoup4, lxml	cheerio, jsdom
Web scraping	scrapy	puppeteer, playwright
YAML handling	pyyaml	js-yaml
GitHub API	PyGithub	@octokit/rest

---

7. Step-by-Step Workflow

Follow these steps for each dataset you discover:

Step 1: Identify the Dataset

1. Visit the dataset page
2. Verify it's a 3D city model (CityJSON/CityGML)
3. Check the file format and size
4. Determine how to access the data:
   • Direct download - Simple URL to a file
   • API endpoint - Requires API calls
   • Data portal - Web interface or multiple files

Step 2: Extract Metadata

Collect the following information from the dataset page:

Basic Information:

• Title (official name of the dataset)
• Description (what's included, coverage, completeness)
• License (look for "license", "terms of use", or "data rights")

Provider Information:

• Organization name (who created/published the data)
• Organization URL
• Contact information (if available)

Data Access:

• Download URL(s)
• API documentation (if applicable)
• Authentication requirements (API keys, etc.)

Spatial/Temporal Coverage:

• Geographic area (city name, country)
• Bounding box coordinates (if provided)
• Year or date range of the data

Step 3: Create the Config File

1. Create a new YAML file in the collections/ directory
2. Use the appropriate template from Section 8
3. Fill in all required fields
4. Add comments for any special access requirements
5. Save with the appropriate filename (see Section 10)

Step 4: Validate the Config

Use the --dry-run flag to validate your config:

cityjson-stac collection --config collections/your-file.yaml --dry-run

Expected output:

✓ Config file is valid YAML
✓ All required fields present
✓ URLs are well-formed
✓ License is valid SPDX identifier
✓ Provider information complete

If there are errors, fix them and re-run the validation.

Step 5: Test the URL (Optional but Recommended)

Verify the data URL is accessible:

# For direct downloads
curl -I https://example.com/data.city.json

# For API endpoints
curl https://api.example.com/endpoint

Step 6: Submit the File

Once validated:

1. Place the YAML file in the collections/ directory
2. Ensure it follows naming conventions
3. Submit all files for review

---

8. Common Scenarios

Scenario 1: Simple Direct Download

Use when: A single downloadable file (e.g., .city.json, .zip)

Example:

id: 3dbag-delft-2023
title: 3DBAG Delft
description: |
  Complete 3D building model of Delft, Netherlands.
  Contains all buildings with LoD2.2 geometry including
  building parts and semantic surfaces.
license: CC-BY-4.0
keywords:
  - 3d city model
  - buildings
  - netherlands
  - delft
  - lod2
providers:
  - name: 3DBAG
    url: https://3dbag.nl
    roles:
      - producer
      - licensor
    description: 3D Basisregistratie Adressen en Gebouwen
inputs:
  - https://example.com/delft.city.json

---

Scenario 2: API Endpoint

Use when: Data is accessed via an API (not direct download)

Example:

id: citylab-karlsruhe-2022
title: CityLab Karlsruhe 3D City Model
description: |
  3D city model of Karlsruhe accessible via API.
  Includes buildings, vegetation, and transportation
  features at multiple Levels of Detail.
license: CC-BY-4.0
keywords:
  - 3d city model
  - germany
  - karlsruhe
  - api
providers:
  - name: CityLab Karlsruhe
    url: https://citylab.example.de
    roles:
      - producer
      - host
    description: Municipal 3D geoinformation lab
inputs:
  - https://api.example.com/v1/cityjson endpoint

# Documentation:
# API Docs: https://api.example.com/docs
# Example call: GET /v1/cityjson?bbox=8.3,48.9,8.5,49.1
# Returns: CityJSON FeatureCollection
# Authentication: None required
# Rate limit: 100 requests/minute

links:
  - rel: about
    href: https://api.example.com/docs
    type: text/html
    title: API Documentation

---

Scenario 3: Multiple Files

Use when: Dataset split across multiple files (districts, tiles, etc.)

Example:

id: opendata-zurich-2023
title: Open Data Zurich 3D
description: |
  Complete 3D city model of Zurich organized by district.
  Each district is provided as a separate CityJSON file.
  Includes buildings (LoD2), vegetation, and transportation.
license: CC0-1.0
keywords:
  - 3d city model
  - switzerland
  - zurich
  - lod2
providers:
  - name: City of Zurich
    url: https://www.stadt-zuerich.ch
    roles:
      - producer
      - licensor
    description: City of Zurich open data portal
inputs:
  - https://example.com/citydata/district-01.city.json
  - https://example.com/citydata/district-02.city.json
  - https://example.com/citydata/district-03.city.json
  - https://example.com/citydata/district-04.city.json
  - https://example.com/citydata/district-05.city.json
  - https://example.com/citydata/district-06.city.json
  - https://example.com/citydata/district-07.city.json
  - https://example.com/citydata/district-08.city.json
  - https://example.com/citydata/district-09.city.json
  - https://example.com/citydata/district-10.city.json
  - https://example.com/citydata/district-11.city.json
  - https://example.com/citydata/district-12.city.json

---

Scenario 4: CityGML Format

Use when: Dataset is in CityGML format (not CityJSON)

Example:

id: opendata-nyc-2021
title: New York City 3D Building Model
description: |
  3D building model of New York City in CityGML format.
  Contains approximately 1 million building footprints
  with height information and building attributes.
license: CC-BY-4.0
keywords:
  - 3d city model
  - citygml
  - usa
  - new york
  - buildings
providers:
  - name: NYC Planning
    url: https://www1.nyc.gov/site/planning
    roles:
      - producer
      - licensor
    description: NYC Department of City Planning
inputs:
  - https://example.com/newyork.gml.zip

# Additional Notes:
# - Format: CityGML 2.0
# - Compression: ZIP containing GML files
# - Projection: EPSG:2263 (NAD83 / New York Long Island)
# - Size: ~500 MB compressed

---

Scenario 5: Unknown/Missing Metadata

Use when: Some information is not available

Example:

id: unknown-montreal-2020
title: Montreal 3D City Model
description: |
  3D city model of Montreal, Canada. Exact temporal coverage
  and coordinate reference system not specified in source.
  Includes buildings and transportation networks.
license: CC-BY-4.0
keywords:
  - 3d city model
  - canada
  - montreal
providers:
  - name: City of Montreal
    url: https://ville.montreal.qc.ca
    roles:
      - producer
    description: Municipal open data portal
inputs:
  - https://data.example.com/montreal.city.json

# Note: CRS will be auto-detected from data during processing
# Note: Year estimated from metadata (actual year may vary)

---

9. Quality Checklist

Before submitting each config file, verify:

Content Quality

• All required fields are present (id, title, description, license, providers, inputs)
• Description is detailed and informative (not just "3D city model")
• License is a valid SPDX identifier
• At least one provider with name and role(s)
• Keywords are relevant and useful for searching

Data Access

• URL is accessible (test in browser or with curl)
• URL format is correct (https://...)
• API endpoints are documented in comments
• Special access requirements (API keys, auth) are noted

Formatting

• YAML syntax is valid (indentation with spaces, not tabs)
• Lists use proper YAML syntax (- for items)
• Multiline strings use \| or > correctly
• Comments document any special cases

Validation

• Config passes --dry-run validation
• No validation errors or warnings
• File follows naming convention (Section 10)

Validation Command

# Validate a single file
cityjson-stac collection --config collections/your-file.yaml --dry-run

# Validate all files at once (bash loop)
for file in collections/*.yaml; do
  echo "Validating $file..."
  cityjson-stac collection --config "$file" --dry-run
done

---

10. Naming Conventions

Config File Names

Format: <city-or-region>-<provider>.yaml

Rules:

• Use lowercase letters
• Separate words with hyphens (-)
• Include geographic identifier (city or region)
• Include source/provider identifier
• Use .yaml extension

Examples:

Dataset	File Name
Delft from 3DBAG	delft-3dbag.yaml
Rotterdam from Open Data	rotterdam-opendata.yaml
Singapore from URA	singapore-ura.yaml
Zurich from Statistics Office	zurich-statistics.yaml
New York City from Planning Dept	nyc-planning.yaml

Collection IDs

Format: <provider>-<city>-<year> (when year is known)

Rules:

• Use lowercase
• Include provider/source name
• Include city name
• Include year if known
• Use hyphens to separate parts

Examples:

Dataset	Collection ID
3DBAG Delft 2023	3dbag-delft-2023
Open Data Rotterdam 2022	opendata-rotterdam-2022
URA Singapore 2024	ura-singapore-2024
Unknown year	provider-city (omit year)

---

11. Submitting Your Work

Deliverables

Submit:

• Multiple YAML configuration files
• One file per dataset
• All files in collections/ directory
• All files validated with --dry-run

Optional but helpful:

• Summary document listing collected datasets
• Notes on any issues encountered
• Suggestions for additional sources

How to Submit

Option 1: Direct Hand-off

• Place files in collections/ directory
• Notify that files are ready for review
• Wait for feedback

Option 2: Pull Request

• Create a new git branch
• Commit your config files
• Create a pull request
• Address review feedback

What Happens Next

1. Review - Your configs will be reviewed for completeness and accuracy
2. Testing - Each config will be tested to ensure it works with cityjson-stac
3. Integration - Valid configs will be added to the catalog
4. Processing - STAC metadata will be generated from your configs

Definition of Done ✅

You have successfully completed this task when:

1. YAML files created - Multiple config files in collections/ directory
2. URLs populated - As many actual data file URLs as possible are filled in
3. Basic validation - Configs pass --dry-run validation (exit code 0)
4. Pull request opened - PR submitted to https://github.com/HideBa/city3d-stac

Repository Access:

• You'll be invited as a collaborator to the repository
• Create a new branch from main: git checkout -b collect-datasets
• Commit your configs: git add collections/ && git commit -m "feat: add dataset configs"
• Push and create PR: git push origin collect-datasets

Acceptance Criteria:

• ✅ Config files follow the naming convention (Section 10)
• ✅ All required fields present (id, title, description, license, providers, inputs)
• ✅ URLs point to actual data files (not just portal homepages)
• ✅ --dry-run validation passes for each config
• ✅ Pull request description summarizes the datasets added

Remember: Done is better than perfect! See Section 12.1 for guidance.

Example Pull Request Template

When you open your PR, use a template like this:

PR Title: feat: add 15 open 3D city model datasets

PR Description Body:

• Summary of datasets added (number, geographic distribution)
• Table listing each dataset with: city, country, provider, format, license, status
• Notes about any issues or uncertainties
• Confirmation that --dry-run validation passed

Example Summary Table:

City	Country	Provider	Format	License	Status
Delft	NL	3DBAG	CityJSON	CC-BY-4.0	✅ Verified
Rotterdam	NL	Open Data	CityJSON	CC0-1.0	✅ Verified
Zurich	CH	City Zurich	CityGML	CC-BY-4.0	⚠️ URL untested

---

• Configs follow naming convention
• All required fields present
• URLs point to actual data files
• --dry-run validation passes
• Uncertain information documented in comments

---

## 12. Troubleshooting

### Common Issues

**Issue:** `--dry-run` validation fails

- **Solution:** Check YAML syntax, ensure all required fields are present

**Issue:** URL is not accessible

- **Solution:** Verify the URL is correct, check if it requires authentication

**Issue:** Can't find license information

- **Solution:** Use `proprietary` and document in comments that license is unclear

**Issue:** API endpoint requires authentication

- **Solution:** Document authentication requirements in comments, include API docs link

**Issue:** Multiple conflicting sources for same city

- **Solution:** Create separate configs for each, include version/year in description

### Getting Help

If you encounter issues not covered here:

1. Check the project README and documentation
2. Review example config files in `examples/`
3. Ask a team member for clarification
4. Document the issue for future reference

---

## 12.1 Working with Uncertainty

**⚠️ Important Principle:** Done is better than perfect.

### Don't Spend Too Much Time

- **5-minute rule:** If you can't find information within 5 minutes, move on
- **Good enough is acceptable:** Partial information is better than no config at all
- **Iterate later:** Configs can be updated and improved in future PRs
- **Focus on scale:** It's better to have 50 decent configs than 5 perfect ones

### Using Comments for Uncertain Information

YAML supports comments with `#`. Use them liberally to document uncertainty:

```yaml
id: montreal-3d-2020
title: Montreal 3D City Model
description: |
  3D city model of Montreal, Canada.
  NOTE: Exact coverage area unclear from source page.
license: CC-BY-4.0  # NOTE: License mentioned on page but not explicitly linked
providers:
  - name: City of Montreal
    url: https://ville.montreal.qc.ca
    roles:
      - producer
    # NOTE: Contact information not available
inputs:
  - https://example.com/montreal.city.json  # NOTE: URL not tested, may require authentication

# Additional notes:
# - Year estimated from metadata (actual year may vary)
# - CRS not specified in source documentation
# - File size unknown
# - Portal may require registration for download

Common Uncertainty Scenarios

Scenario	What to Do
License unclear	Use proprietary + comment: # License not specified, assumed restrictive
Year unknown	Estimate from metadata + comment: # Year estimated from page copyright
URL not accessible	Include URL anyway + comment: # NOTE: URL not verified, may be broken
CRS not specified	Leave blank + comment: # CRS will be auto-detected from data
Provider contact missing	Omit + comment: # Contact information not available
Multiple format versions available	Choose one + comment: # NOTE: Other formats available on source page
API authentication unclear	Document what you found + comment: # NOTE: Auth requirements unclear

Comment Guidelines

DO use comments for:

• Information you're uncertain about
• Assumptions you're making
• Things you couldn't verify
• Alternative URLs or formats you found
• Notes about data quality or completeness
• Authentication or access requirements

DON'T stress about:

• Finding every single data URL for multi-file datasets (get the main ones)
• Verifying every URL works (sample check is fine)
• Perfect descriptions (good enough is okay)
• Complete metadata (fill what you can, comment the rest)
• Formatting inconsistencies (focus on content)

Example: Good Enough Config

id: rotterdam-opendata-2022
title: Rotterdam 3D City Model
description: |
  3D city model of Rotterdam, Netherlands.
  NOTE: Description translated from Dutch, may not be exact.
  Contains buildings and terrain data.
license: CC-BY-4.0  # NOTE: Confirming exact license version
providers:
  - name: City of Rotterdam
    url: https://www.rotterdam.nl
    roles:
      - producer
  # NOTE: Additional providers may exist
inputs:
  - https://example.com/rotterdam.city.json  # Main dataset
  # NOTE: Additional district files available but not collected

# TODO:
# - Verify license with official source
# - Add district file URLs if needed
# - Confirm data year (metadata unclear)

Review Process

During review, unclear information will be:

1. Identified - Reviewers will check comments and notes
2. Clarified - Team will investigate uncertain items
3. Resolved - Configs will be updated or marked for future improvement

Your comments help reviewers understand:

• What you tried to find but couldn't
• What assumptions you made
• What needs verification

Remember: A config with notes about uncertainty is more valuable than no config at all!

---

13. Quick Reference

Essential Fields

id: unique-id
title: Dataset Title
description: Detailed description
license: CC-BY-4.0
providers:
  - name: Organization
    roles: [producer]
inputs:
  - https://example.com/data.json

Common Licenses

• CC-BY-4.0 - Creative Commons Attribution
• CC0-1.0 - Public Domain
• ODC-BY-1.0 - Open Data Commons
• proprietary - Unknown/restricted

Provider Roles

• producer - Created the data
• licensor - Manages the license
• processor - Transformed the data
• host - Hosts the data

Validation Command

cityjson-stac collection --config collections/file.yaml --dry-run

---

Appendix A: Field Types Reference

Strings

Simple text values:

title: Delft 3D City Model
license: CC-BY-4.0

Multiline Strings

Use | for literal multiline:

description: |
  This is a long description
  that spans multiple lines.
  Formatting is preserved.

Lists

Use - for list items:

keywords:
  - 3d city model
  - buildings
  - netherlands

Nested Objects

Indent with 2 spaces:

providers:
  - name: Organization
    url: https://example.com
    roles:
      - producer
      - licensor

Comments

Use # for comments:

inputs:
  - https://example.com/data.json # Main dataset
  # API endpoint: https://api.example.com/v1/cityjson

---

Appendix B: Resources

STAC Specification

• https://stacspec.org/ - Core STAC standard
• https://stac-extensions.github.io/ - STAC extensions

CityJSON / CityGML

• https://www.cityjson.org/ - CityJSON specification

Licenses

• https://spdx.org/licenses/ - SPDX license list
• https://choosealicense.com/ - License guide

Coordinate Reference Systems

• https://epsg.io/ - CRS registry
• https://spatialreference.org/ - CRS reference

---

Good luck with your data collection! 🏙️

If you have questions, don't hesitate to ask. Your contribution will help make 3D city data more accessible to everyone.