Top-Level Domains

Top-level domains (abbrev. “TLD”/”TLDs”) are a significant indicator for the representativeness of the data, whether the data set or particular crawl is biased towards certain countries, regions or languages.

Metrics about top-level domains are show on the following pages:

• Groups of TLDs: generic vs. country-code TLDs (e.g., .com vs. .jp)
• TLDs of latest monthly crawl
• Estimation of representativeness: a comparison of a recent crawl with an external data source
• TLDs of selected crawls: lists percentage of TLDs for selected crawls from 2008–2022

Note, that top-level domain is defined here as the left-most element of a host name (com in www.example.com). Country-code second-level domains (“ccSLD”) and public suffixes are not covered by this metrics.