Building, running, and changing BirdView
BirdView is one Go module: an ingest that compiles the dataset from published sources, a server
that embeds and serves the web UI, and five generators that build the committed tables under
data/. There is no database, no data fetch at run time, and no JavaScript framework.
Prerequisites
- Go, at the version
go.modrequires. - Chrome or Chromium, optional. Browser tests skip without it; set
BIRDVIEW_CHROMEif it is not in a standard place. A skip is not a pass, so check the test output for skip lines when your change affects rendered pages. - Python 3, Node, and Google Chrome, only to re-render the pipeline diagram (see below).
The only Go dependency is goldmark, which renders these guides. The source workbooks are read by
internal/xlsx, using only the standard library.
The data and its licenses
Everything the build needs is in the repository.
| Source | License | What is in this repository | Used for |
|---|---|---|---|
| AviList v2025b | CC BY 4.0 | the workbook, in third_party/avilist/ |
names, families, taxonomic sequence, species codes |
| AVONET (Tobias et al. 2022) | CC BY 4.0 | the workbook, in third_party/avonet/ |
bill, wing, tail, mass, habitat, diet |
| Fox et al. 2026 wingspan database | CC BY 4.0 | the workbook, in third_party/foxwingspan/ |
measured wingspans |
| GBIF: the eBird Observation Dataset and the iNaturalist research-grade observations | CC BY 4.0 (eBird); CC BY-NC 4.0 (iNaturalist), usable because BirdView is non-commercial | only tables BirdView derives, under data/; never a record |
where each bird recurs, the regions, how common it is |
| iNaturalist terms, retired | © iNaturalist | the terms, in third_party/inatterms/ and third_party/inatapi/ |
nothing since 2026-11-01; kept as the record of an earlier link |
| Natural Earth 1:10m states and provinces | public domain | the derived data/region-boundaries.json; the source is fetched by hash only to regenerate it |
the Here button |
| WCAG 2.2 | W3C Document License | the Recommendation, in third_party/wcag/ |
the accessibility rules BirdView meets, quoted |
Nothing under third_party/ is covered by BirdView's Apache 2.0 license. Each directory has
its own license and a README naming the source and how to cite it. The iNaturalist terms are not
openly licensed; they and the WCAG text are kept so every quotation of them in
river/ can be checked against the exact words. Retrieval details and hashes are in
river/requirements/imported/, and the
source register records every source considered,
including the rejected ones and why.
BirdView links to eBird and copies nothing from it or from iNaturalist: no photograph is embedded, cached, or hotlinked, so no contributor's license is engaged.
Every CC BY source is credited on the page that uses it, as its license asks; the home page's footer names each with its citation, and every other page links to it.
Build and run
From the repository root:
go run ./cmd/birdview-ingest # compiles web/data/species.json, and counties.gob.gz beside it
go run ./cmd/birdview # serves http://localhost:8080
The ingest prints a coverage summary: species in scope, and how many traits and wingspans are measured, near-match, or modeled. Expect 1082 species in 99 families across 27 orders.
The server reads species.json once, at startup, and with -counties the counties the ingest
wrote apart beside it. Restart it after re-running the ingest.
Use -dev web while editing the UI. Templates and static files are embedded with go:embed,
so a normal server keeps serving the versions it was built with, with no sign they are stale.
-dev web reads them from web/static/ and re-parses the templates on every request.
go run ./cmd/birdview -dev web
It is off by default, which TestEmbedIsTheDefault checks. Go code is still compiled in, so
restart after changing a Go file (for example, adding a field a template uses).
| Command | Flag | Default |
|---|---|---|
birdview-ingest |
-status |
data/status.tsv |
-avilist |
third_party/avilist/AviList-v2025b-10Jun2026-extended.xlsx |
|
-avonet |
third_party/avonet/AVONET Supplementary dataset 1.xlsx |
|
-wingspan |
third_party/foxwingspan/wingspan_fox2026_AppendixS1.xlsx |
|
-regions |
data/region-presence.tsv |
|
-habitat-review |
data/habitat-review.tsv |
|
-proportions-review |
data/proportions-review.tsv |
|
-counties |
data/county-presence |
|
-out |
web/data/species.json |
|
-counties-out |
counties.gob.gz beside -out |
|
birdview |
-addr |
localhost:8080 |
-data |
web/data/species.json |
|
-counties |
off; serve counties as regions, until every state and province has them | |
-county-data |
counties.gob.gz beside -data |
|
-dev |
off | |
-feedback |
none; the GitHub issue list (https://github.com/<owner>/<repo>/issues) whose filled-in new issues every report link opens; no links when unset, and the server refuses any other kind of URL |
|
-canonical |
none; the one host to answer at, e.g. birdview.site; requests for any other host are redirected there over HTTPS |
The defaults are relative paths, so run both commands from the repository root.
The generators
Four commands build committed tables under data/. None runs as part of the ingest: together they
make thousands of requests to other people's servers, tests make no network requests, and the
build must work offline. Run a generator by hand when its source changes, review the diff, and
commit it. How BirdView's data is built diagrams them and the ingest they feed.
| Command | Writes | Cost |
|---|---|---|
birdview-gbifmap |
data/gbif-taxa.tsv: GBIF backbone name → AviList name |
~5 min, ~2,800 requests |
birdview-regions |
data/region-presence.tsv: which regions each species occurs in; with -counties <download.zip> -state <GADM code> (or -state all), data/county-presence/<state>.tsv.gz for each state from a recorded GBIF download, gzipped (decisions.countyTablesCompressed) |
one facet query per region, resumable; none for counties |
birdview-status |
data/status.tsv: BirdView's own occurrence status per species |
two passes, resumable |
birdview-boundaries |
data/region-boundaries.json: region outlines for the Here button, from Natural Earth |
one ~15 MB download, hash-checked |
birdview-sea |
data/sea-distance.tsv: each species' eBird records on water by distance from shore, from the offshore-cells GBIF download and Natural Earth's land |
~2 min; two ~3.5 MB downloads, hash-checked and cached; needs the download under fetched/gbif/ and a built dataset |
birdview-names and birdview-regions read the built dataset, so run the ingest first.
The GBIF rate is BirdView's own, and the client slows further whenever GBIF answers 429.
birdview-status caches raw fetches under /tmp and skips what it already has, so an interrupted
run resumes. It refuses to write a partial table, because a species whose dates never arrived would
be classified as a vagrant on no evidence.
data/habitat-review.tsv is edited by hand: one row per species BirdView draws, holding the
habitats BirdView adds to AVONET's and the review's status. When species are added, or AVONET is
updated, the ingest's tests fail and name the rows to add or review again.
birdview-gbifdownload requests a GBIF SQL download from a query committed under
data/gbif-queries/, waits for it, writes it under fetched/gbif/ (never committed), and records
its key, DOI, license, and hash in data/gbif-downloads.tsv, the DOI being what the credits cite.
It needs a GBIF account, read from GBIF_USER and GBIF_PASSWORD; given -key, it fetches and
records a download already requested from GBIF's website, with no credentials. -validate only asks
GBIF whether it would accept the query. birdview-regions -counties reads the county download.
A sixth command, birdview-names, writes data/source-names.tsv: how many records eBird and
iNaturalist each hold under each species' name, used to find names the two disagree about. Only a
test reads it.
When a command's inputs or outputs change, update the diagram in docs/pipeline.md. GitHub
draws its Mermaid directly; BirdView serves an SVG rendered from it, because drawing Mermaid in the
browser would need a script from the network. After editing, run python3 docs/render-diagrams.py
and commit the SVG it writes beside the page. TestDiagramsAreCurrent fails until you do, and
TestPipelineDiagramNamesRealColumns fails if the diagram names a source column no reader asks
for.
Standing checks
Run all four, in this order, before calling a change done:
go build ./...
go vet ./...
gofmt -l . # must print nothing
go test ./...
Every spec check is a Go test in river/check/, run by go test ./...: River's checker and its
shared cases, BirdView's stricter goal rule, the vendored-file hashes, every quotation against its
vendored source, and every reference, anchor, and relative link into the bundle. They cover the
bundle's structure, not its substance: a green run means it is internally consistent, not that
it is right.
CI runs the same four on every push to main and every pull request
(.github/workflows/checks.yml), with the dataset built and
Chrome installed, so every test can run there. A skipped test fails the run.
Repository layout
cmd/birdview-ingest/ builds species.json from the sources and the data/ tables
cmd/birdview/ the server; embeds web/ and docs/ and serves them
cmd/birdview-gbifmap/ generator: GBIF backbone -> AviList name bridge
cmd/birdview-regions/ generator: per-region presence
cmd/birdview-status/ generator: BirdView's own occurrence status
cmd/birdview-boundaries/ generator: region boundaries for the Here button
cmd/birdview-sea/ generator: records on water by distance from shore
cmd/birdview-gbifdownload/ requests, fetches, and records GBIF SQL downloads
cmd/birdview-names/ measures how eBird and iNaturalist name each species
cmd/birdview-smoke/ checks a running deployment from outside, after each deploy
internal/avilist/ the AviList workbook reader: names, families, codes, sequence
internal/avonet/ the trait workbook reader, including habitat and trophic niche
internal/wingspan/ the Fox dataset and the fitted wingspan regression
internal/taxonomy/ the synonym tables, in both directions, and the order labels
internal/trait/ the habitat and diet vocabularies and BirdView's additions
internal/morphology/ the segment models (head, body, neck) and rounding
internal/region/ the regions, their zones on the map, and their IDs
internal/boundary/ the region boundary file and the reference lookup over it
internal/shapefile/ the Natural Earth shapefile readers: polygons and attribute tables
internal/geo/ on land or not, and distance to the nearest shore
internal/month/ the twelve months and the rule for presence in one
internal/gbif/ the paced GBIF client and the name bridge
internal/status/ recurrence: episodes, and the status they imply
internal/view/ ordering, sectioning, diagram geometry; pure, no HTML
internal/dataset/ the serialized types and the provenance kinds
internal/palette/ family and order colors, with contrast guarantees
internal/smoke/ the post-deploy checks birdview-smoke runs
data/ the committed generated tables
web/static/ templates, CSS, the Here script, and the favicon, embedded
web/data/ the built species.json and counties.gob.gz (gitignored)
docs/ this documentation, also served at /docs
third_party/ vendored sources, each under its own license
river/ the requirements bundle; normative, not served
river/check/ the spec checks, as Go tests
.github/workflows/ CI: the standing checks on every push and pull request
internal/view has no HTML or HTTP so the layout rules can be property-tested directly; code that
formats strings belongs in another package.
The spec-driven workflow
The requirements are normative and live in river/. Code implements them; where
this guide and river/ disagree, river/ wins.
river/README.md declares the River 3.0 module; the method copy lives at
.agents/skills/river/. To make a change you need only the loop
below.
The bundle
| File | Contents |
|---|---|
README.md |
the River 3.0 module declaration and directory index |
bindings/ |
how BirdView applies the method, and where it departs from it |
goals/ |
why the project exists: sixteen goals, goals.* |
requirements/local/product/ |
what it must do, one file per group: product.* |
requirements/local/tech/ |
technical constraints, one file per group: tech.* |
requirements/composed/internal/app/ |
every requirement the web application must meet, and its criteria (app.*) |
requirements/composed/internal/dataset/ |
every requirement the data pipeline and dataset must meet, and its criteria (dataset.*) |
requirements/composed/internal/repo/ |
every requirement repository layout, build, and release must meet, and its criteria (repo.*) |
requirements/README.md |
what each requirements file and directory holds |
requirements/imports/ |
every body of rules or data the project is subject to |
requirements/precedence/ |
which tier of source outranks which |
requirements/decisions/ |
how each conflict or scope was resolved: decisions.* |
requirements/sums.md |
content sums of every input to composition (river sum -w); each composed package's composed.sum records what it was composed from (river compose) |
conformance/ |
which guardrails check each criterion, and Watched failing: observations |
todo.md (repository root) |
everything still open: work, questions, unverified criteria, stale text |
arch.md |
how the code is organized |
check/ |
BirdView's repository-specific spec checks |
Each requirement has an ID such as product.tileContents; its first paragraph is the rule and the
rest is rationale. Cite IDs in code comments as the literal token, so one search finds every use:
// Verifies: product.tileContents, tech.badgeContrast. Criterion: app.tileCarriesHeaderAndMetrics.
The loop
A change goes requirement, then acceptance criterion, then code. An agent or contributor may draft
requirements and criteria; the project owner approves them by reviewing the pull request, whose
"Requirements impact" section river diff --base main --format pr river drafts. Not every change
needs all three:
| Change | What it needs |
|---|---|
| New feature or changed behavior | Edit the requirement, then the criterion, then the code |
| Bug fix | If a requirement already forbids the bug, add a criterion under it, then fix. If none does, that is a gap in the spec: add the requirement first |
| Refactor | No spec change, and the suite stays green. If you must change a check's assertions, it is not a refactor |
| Adopting a new source | An entry in river/requirements/imports/, then a pass to compose its requirements and resolve conflicts with the existing ones. Expect conflicts |
| Docs only | No spec change, unless writing the doc shows the spec is wrong |
Open work is tracked in todo.md, not in TODO comments. Read it before
starting something new, and add an item for anything you find or leave unfinished.
Three rules
Never resolve a conflict in code. If two requirements disagree, or a change reverses something
the spec states, stop and record a new entry in
river/requirements/decisions/. Silently choosing one is the failure this
process exists to prevent.
Never edit a requirement to match code that violates it without saying so and getting agreement. That turns a bug into a feature without anyone deciding it should be one.
A criterion you have not watched fail is not a criterion. Break the code on purpose, confirm the check fails, and read the message to confirm it fails for the right reason. Record how you broke it in the criterion's Status line. Make the deliberate break fail loudly if it does not apply, or a no-op edit will look like a passing check. Checks that test nothing are common and look fine in review: a whole-page string search that matches the help text instead of the control, a fixture too small to exercise the behavior, a test that still passes after its requirement is reversed.
Hard rules
Most of these are licensing rules, not style.
- Never commit the built dataset,
web/data/species.jsonandcounties.gob.gz. It is reproducible from the committed inputs in one command, and a stale copy in the tree is worse than none. - Never read or process Wiley Online Library material, including Ricklefs (2017). Its terms prohibit AI ingestion and automated access, and the source is rejected. Do not revive it as a length source.
- Never fetch, download, or programmatically process anything from BirdLife's DataZone
(
datazone.birdlife.org). Its terms forbid automated access and text and data mining; see the register entry. AVONET's BirdLife sheet is part of AVONET, under CC BY 4.0, and is not affected. - Only AVONET, the Fox wingspan data, and AviList may be committed as source data. All three
are CC BY 4.0. Before adding a fourth, read
imports.md: most candidates evaluated could not be redistributed. The generated tables underdata/are not source data but the project's own reductions, with their provenance in their headers. The vendored terms and WCAG text are kept for quotation, not as data. - Every third-party file goes under
third_party/<name>/, with its own license and README, and its provenance underriver/requirements/imported/<origin>/<name>/. - Never present a modeled value as a measurement. Every length and wingspan is marked measured or modeled, and the page shows which; see this decision.
AGENTS.md carries the same rules for AI coding agents.
Deployment
Deploying needs only the container image below, or the server binary and a built dataset.
Container image
The Dockerfile at the repository root builds a minimal static image, based on
gcr.io/distroless/static-debian12:nonroot, in four stages:
ingestbuilds and runsbirdview-ingest, producing/out/species.jsonand/out/counties.gob.gz.testruns every test except the spec checks against that dataset. If one fails, the build stops, so a failing change cannot be deployed. Tests that need Chrome skip here; CI runs them.builderbuilds the staticbirdviewserver binary, and only aftertesthas passed.- The final image copies in only the binary and the dataset, as
/data/species.jsonand/data/counties.gob.gz, which the server reads only with-counties.
To deploy despite failing tests, for an urgent fix past an unrelated failure, put an empty
file named SKIP_TESTS in the directory you build from, and delete it afterwards. Never commit
it. The build log then says the tests were skipped.
docker build -t birdview .
docker run --rm -p 8080:8080 birdview
By default the container runs birdview -addr 0.0.0.0:8080 -data /data/species.json. Arguments
after the image name replace those defaults entirely, so to add a flag such as -feedback, repeat
-addr and -data:
docker run --rm -p 8080:8080 birdview \
-addr 0.0.0.0:8080 \
-data /data/species.json \
-feedback <issue-tracker-url>
Recording the revision
Before deploying, write the commit you are deploying to REVISION in the directory you build from:
git rev-parse HEAD > REVISION
The image builds it into the server, which names it at startup and in the issues readers open from
its pages. git ignores the file, and .gcloudignore uploads it anyway. Without it the build still
succeeds, but the server says its revision was not recorded and the smoke check below fails.
After deploying
Check the deployment from outside. The first argument is its address; any others are addresses that should redirect to it:
go run ./cmd/birdview-smoke https://birdview.example https://www.birdview.example
It prints one line per check and exits non-zero if any fails. One check reads the revision from the
home page's issue link, so it needs -feedback set and a REVISION written before the deploy. It fetches nothing larger than a
page, so running it after every deploy costs little egress.
Rolling back
On Cloud Run, every deploy is kept as a revision, and sending traffic back to the previous one is one command:
gcloud run revisions list --service <service> --region <region>
gcloud run services update-traffic <service> --region <region> --to-revisions <revision>=100
Traffic stays pinned to that revision until you send it back with --to-latest.
Running without Docker
The server binds localhost:8080 by default; on a host, point -addr at the interface and port
the host expects:
go build -o birdview ./cmd/birdview
go run ./cmd/birdview-ingest
./birdview -addr 0.0.0.0:$PORT -data web/data/species.json
Templates, static files, and these guides are embedded in the binary; the dataset is the only file it reads at run time.
When a regenerated dataset drops a species, /species/<code> answers 404 with a page naming the
bird and why it is no longer drawn.
Contact
Through the project's issue tracker.