Earth Support Corp

Ground Truth · Methodology

How the numbers are made — and what they can’t tell you.

How Ground Truth collects, cleans and publishes its numbers — the rules we hold ourselves to, every dataset with its sources and freshness, and how to cite or correct us.

  • 9 datasets live
  • 527,826 source records
  • 31 sources
  • Refreshed Sep 11, 2026
All 31 of our sources are open

Distinct sources cited across live datasets, by licence · n = 31

  • Public domain15
  • Public record14
  • Open, attribution1
  • CC BY-SA1

01 · Principles

Five rules every page follows

  1. 01

    The pipeline computes every number

    Pages never type or re-derive a published figure. Each value comes out of a builder script that stamps it with its sources, record count and data-as-of date.

  2. 02

    Finding first, n always

    Headlines state the takeaway in plain English, and every chart says how many records it stands on — so a reader can judge how much weight it bears.

  3. 03

    Say where it came from and how old it is

    Every dataset lists its sources with a licence label, the date the data is current to, and whether it is re-pulled live or a frozen snapshot.

  4. 04

    Label geography truthfully

    Modules built on bid results cover Massachusetts only and say so. Public federal inventories cover all six New England states.

  5. 05

    Show the caveats, not just the chart

    Known gaps, cleaning rules and biases sit in each page’s method box, next to the numbers they qualify.

02 · Data-handling rules

What we never publish, and how we treat thin data

No personal data
No person’s name, email, phone number or homeowner address is published. Every file passes an automated guard that rejects contact fields and anything shaped like an email or phone number. Firms and public agencies may be named.
Bid data: aggregates, not listings
Bid results appear as totals, medians and rates. This is an analysis of the market, not a plan room: no project listings, contact details or solicitation documents.
Suppression
Any group built from fewer than 10 records is dropped from bid-market aggregates, so no single project or firm can be singled out or reverse-engineered.
Small numbers
A rate from 8 records is not a rate from 800. Tables show each group’s n; groups too thin to say anything are marked or left out rather than ranked.
Shrinkage when ranking
Where a page ranks rates across groups of very different size, it pulls small groups toward the overall rate — (k + p·m) ÷ (n + m), empirical-Bayes style — so three lucky records can’t top a league table. Pages say when they do this.
Snapshots aren’t trends
The bid-record extract was compiled in late June 2026 and is heaviest in 2021–22. Year-by-year counts in it reflect what was captured, not the market’s size, and are never presented as a trend without that caveat.
Not engineering advice
These are regional statistics for planning and curiosity. They are not a geotechnical report, a bid recommendation or a statement about any particular site, firm or project. Borings and an engineer decide what’s under your site.

03 · Every dataset

What’s in the hub, where it comes from, how fresh it is

Read straight from each dataset’s provenance envelope at build time. Record counts are the rows each dataset’s aggregates were built from.
  1. Bid spreads

    Massachusetts public bid openings, 2018–2026

    Records
    1,533
    Data as of
    Jun 29, 2026
    Refresh
    Frozen snapshot
    Geography
    Massachusetts

    1 source

    • Massachusetts public bid results and project records [Public record]
  2. Bid calendar

    Massachusetts bid dates, 2019–2026

    Records
    3,058
    Data as of
    Jun 29, 2026
    Refresh
    Frozen snapshot
    Geography
    Massachusetts

    1 source

    • Massachusetts public bid results and project records [Public record]
  3. Contractors

    Massachusetts public bid lists, mostly 2020–2026

    Records
    8,711
    Data as of
    Jun 29, 2026
    Refresh
    Frozen snapshotrebuilt when the leads database is re-pulled
    Geography
    Massachusetts

    2 sources

  4. Designers & owners

    Massachusetts construction projects

    Records
    3,873
    Data as of
    Jun 29, 2026
    Refresh
    Frozen snapshot
    Geography
    Massachusetts

    1 source

    • Massachusetts public bid results and project records [Public record]
  5. Bedrock & soils

    ME · NH · VT · MA logs · NE groundwater & frost

    Records
    343,355
    Data as of
    Sep 11, 2026
    Refresh
    Live pullweekly (groundwater levels daily at source)
    Geography
    New England

    8 sources

  6. Bridges

    Every NBI bridge in CT ME MA NH RI VT

    Records
    18,418
    Data as of
    Mar 31, 2025
    Refresh
    Live pullannual (FHWA NBI) + monthly state feeds
    Geography
    New England

    4 sources

  7. Dams & disasters

    Six New England states · 2005–2026

    Records
    28,665
    Data as of
    Sep 11, 2026
    Refresh
    Live pullweekly
    Geography
    New England

    5 sources

  8. Permit speed

    4 cities timed + Boston, Sep 2024–Aug 2026

    Records
    120,213
    Data as of
    Sep 11, 2026
    Refresh
    Live pullweekly
    Geography
    New England

    5 sources

  9. Town explorer

    1,500+ New England towns · CT ME MA NH RI VT

    Records
    367,748
    Data as of
    Sep 11, 2026
    Refresh
    Live pullweekly
    Geography
    New England

    9 sources

04 · How to cite

Use it freely — just say where it came from

Cite as Earth Support Corp, Ground Truth, <dataset>, accessed <date>. Where a figure comes from a public source, cite that source too — it’s listed with every dataset above.

Earth Support Corp, Ground Truth, “What public jobs actually bid for,” https://earthsupportcorp.com/data/bids, accessed <date>.

05 · Corrections

Found an error? We want to hear it.

Tell us the page, the number and what you believe it should be. We check it against the source, fix the pipeline rather than the page, and note material corrections in that page’s method box.
Report a correction

06 · How it refreshes

One command rebuilds every dataset

Each module has a small Python builder. Live modules re-pull their public API or file on every run; snapshot modules re-derive from a frozen extract. A builder that fails leaves its last good file in place, so the site never shows a half-built dataset.
# rebuild everything (pulls are cached)
pnpm data:refresh

# one module, or offline from cache
pnpm data:refresh --only bridges
pnpm data:refresh --offline

# then rebuild the search index
pnpm data:refresh --only search
  1. Step 1

    Pull

    Public APIs and files are fetched through an on-disk cache, so a flaky source never blanks a page.

  2. Step 2

    Check

    Every output passes the personal-data guard and a size budget; bid-market aggregates suppress groups under 10.

  3. Step 3

    Publish

    Datasets land as JSON with their provenance envelope; the site reads them at build time and stamps each page with its as-of date.

Back to the Ground Truth index.

CallStart Quote