Number of pages, distribution of top-level domains, crawl overlaps, etc. - basic metrics about Common Crawl Monthly Crawl Archives
Latest crawl: CC-MAIN-2024-42
Detailed numbers and percentage of top-level domains (groups) in the latest monthly crawl (CC-MAIN-2024-42). Note that Internationalized country code TLDs (IDN ccTLDs) are mapped to their ASCII equivalents before counting TLDs. E.g., the counts of .ru
contain also the occurrences of .рф.
pages | urls | hosts | domains | %urls | %hosts | %domains | |
---|---|---|---|---|---|---|---|
type | |||||||
gTLD | 1,361,802,018 | 1,354,443,685 | 27,795,064 | 26,612,592 | 54.57 | 58.46 | 58.77 |
ccTLD | 1,077,971,520 | 1,072,574,953 | 19,309,814 | 18,245,143 | 43.22 | 40.62 | 40.29 |
sTLD | 44,912,487 | 44,587,655 | 213,339 | 205,735 | 1.80 | 0.45 | 0.45 |
grTLD | 6,017,403 | 5,984,570 | 144,493 | 139,346 | 0.24 | 0.30 | 0.31 |
IDN ccTLD | 4,178,068 | 4,167,921 | 74,851 | 73,289 | 0.17 | 0.16 | 0.16 |
IDN gTLD | 168,815 | 168,507 | 4,161 | 4,093 | 0.01 | 0.01 | 0.01 |
pages | urls | hosts | domains | %urls | %hosts | %domains | |
---|---|---|---|---|---|---|---|
tld | |||||||
com | 1,060,958,604 | 1,055,106,611 | 22,683,163 | 21,724,670 | 42.51 | 47.71 | 47.98 |
org | 139,506,981 | 138,790,123 | 1,844,920 | 1,737,415 | 5.59 | 3.88 | 3.84 |
ru | 114,560,248 | 114,188,446 | 1,328,852 | 1,268,567 | 4.60 | 2.80 | 2.80 |
de | 98,740,252 | 98,128,446 | 2,880,140 | 2,692,737 | 3.95 | 6.06 | 5.95 |
net | 85,162,485 | 84,789,193 | 1,394,430 | 1,323,267 | 3.42 | 2.93 | 2.92 |
jp | 55,300,351 | 55,075,500 | 847,894 | 799,313 | 2.22 | 1.78 | 1.77 |
uk | 53,499,517 | 53,223,141 | 1,054,100 | 1,005,895 | 2.14 | 2.22 | 2.22 |
it | 47,964,493 | 47,746,683 | 757,339 | 718,185 | 1.92 | 1.59 | 1.59 |
fr | 45,378,809 | 45,168,507 | 872,410 | 837,843 | 1.82 | 1.84 | 1.85 |
pl | 43,053,797 | 42,866,487 | 697,700 | 665,766 | 1.73 | 1.47 | 1.47 |
br | 39,577,206 | 39,421,304 | 550,683 | 516,157 | 1.59 | 1.16 | 1.14 |
nl | 37,214,867 | 37,005,821 | 966,799 | 916,836 | 1.49 | 2.03 | 2.02 |
edu | 28,140,046 | 27,997,745 | 122,729 | 119,967 | 1.13 | 0.26 | 0.26 |
au | 27,148,634 | 27,024,957 | 500,265 | 483,899 | 1.09 | 1.05 | 1.07 |
cz | 26,280,111 | 26,095,382 | 416,419 | 390,960 | 1.05 | 0.88 | 0.86 |
es | 23,711,577 | 23,598,774 | 340,865 | 326,340 | 0.95 | 0.72 | 0.72 |
ca | 22,895,975 | 22,779,798 | 421,988 | 405,689 | 0.92 | 0.89 | 0.90 |
cn | 20,846,580 | 20,801,518 | 753,799 | 677,549 | 0.84 | 1.59 | 1.50 |
eu | 17,435,351 | 17,345,786 | 360,587 | 341,886 | 0.70 | 0.76 | 0.76 |
ua | 16,748,612 | 16,413,456 | 116,725 | 112,629 | 0.66 | 0.25 | 0.25 |
info | 15,744,584 | 15,689,597 | 263,718 | 250,364 | 0.63 | 0.55 | 0.55 |
in | 15,742,023 | 15,676,152 | 270,521 | 257,513 | 0.63 | 0.57 | 0.57 |
ro | 15,215,596 | 15,166,553 | 184,256 | 173,714 | 0.61 | 0.39 | 0.38 |
ch | 15,121,788 | 15,038,326 | 422,849 | 394,994 | 0.61 | 0.89 | 0.87 |
se | 14,150,908 | 14,065,167 | 294,566 | 280,633 | 0.57 | 0.62 | 0.62 |
gr | 13,707,253 | 13,661,929 | 161,743 | 151,790 | 0.55 | 0.34 | 0.34 |
vn | 13,369,416 | 13,308,555 | 146,440 | 140,569 | 0.54 | 0.31 | 0.31 |
hu | 13,203,849 | 13,146,424 | 191,765 | 179,987 | 0.53 | 0.40 | 0.40 |
be | 12,858,042 | 12,785,572 | 319,531 | 299,524 | 0.52 | 0.67 | 0.66 |
tr | 11,363,544 | 11,336,258 | 134,629 | 123,502 | 0.46 | 0.28 | 0.27 |
dk | 11,439,103 | 11,334,387 | 246,797 | 235,963 | 0.46 | 0.52 | 0.52 |
co | 11,099,705 | 10,981,151 | 208,497 | 203,558 | 0.44 | 0.44 | 0.45 |
id | 10,703,155 | 10,658,581 | 149,440 | 145,043 | 0.43 | 0.31 | 0.32 |
at | 10,525,278 | 10,454,221 | 280,090 | 264,519 | 0.42 | 0.59 | 0.58 |
gov | 10,469,080 | 10,387,876 | 27,180 | 25,643 | 0.42 | 0.06 | 0.06 |
kr | 9,582,600 | 9,552,036 | 193,289 | 165,671 | 0.38 | 0.41 | 0.37 |
ir | 9,213,889 | 9,193,247 | 106,007 | 102,534 | 0.37 | 0.22 | 0.23 |
tw | 9,231,050 | 9,184,629 | 113,447 | 103,168 | 0.37 | 0.24 | 0.23 |
sk | 8,890,325 | 8,848,515 | 137,900 | 128,922 | 0.36 | 0.29 | 0.28 |
fi | 8,162,579 | 8,113,633 | 151,646 | 144,598 | 0.33 | 0.32 | 0.32 |
ar | 8,135,763 | 8,112,498 | 130,296 | 116,127 | 0.33 | 0.27 | 0.26 |
mx | 7,842,106 | 7,809,956 | 104,990 | 99,821 | 0.31 | 0.22 | 0.22 |
no | 7,699,423 | 7,656,980 | 142,721 | 136,055 | 0.31 | 0.30 | 0.30 |
pt | 7,612,150 | 7,577,796 | 107,697 | 100,599 | 0.31 | 0.23 | 0.22 |
io | 6,454,777 | 6,403,601 | 222,672 | 220,357 | 0.26 | 0.47 | 0.49 |
za | 6,116,574 | 6,087,970 | 129,048 | 123,962 | 0.25 | 0.27 | 0.27 |
cl | 5,917,407 | 5,897,289 | 97,341 | 92,919 | 0.24 | 0.20 | 0.21 |
app | 5,729,834 | 5,721,206 | 105,183 | 104,443 | 0.23 | 0.22 | 0.23 |
nz | 5,747,372 | 5,718,163 | 111,901 | 108,065 | 0.23 | 0.24 | 0.24 |
il | 5,343,385 | 5,322,523 | 82,254 | 79,230 | 0.21 | 0.17 | 0.17 |
by | 4,907,585 | 4,880,748 | 48,032 | 46,152 | 0.20 | 0.10 | 0.10 |
tv | 4,844,084 | 4,814,677 | 40,836 | 38,766 | 0.19 | 0.09 | 0.09 |
us | 4,615,926 | 4,592,650 | 101,204 | 97,496 | 0.19 | 0.21 | 0.22 |
me | 4,335,328 | 4,310,803 | 95,329 | 93,474 | 0.17 | 0.20 | 0.21 |
ie | 4,316,869 | 4,293,333 | 65,773 | 63,338 | 0.17 | 0.14 | 0.14 |
lt | 4,256,925 | 4,234,878 | 66,179 | 61,973 | 0.17 | 0.14 | 0.14 |
shop | 4,228,867 | 4,211,550 | 78,201 | 76,655 | 0.17 | 0.16 | 0.17 |
hr | 4,101,557 | 4,088,124 | 52,525 | 48,203 | 0.16 | 0.11 | 0.11 |
xyz | 3,990,030 | 3,966,331 | 125,197 | 123,582 | 0.16 | 0.26 | 0.27 |
ee | 3,784,248 | 3,759,375 | 50,584 | 45,532 | 0.15 | 0.11 | 0.10 |
online | 3,777,203 | 3,758,916 | 89,371 | 88,022 | 0.15 | 0.19 | 0.19 |
bg | 3,710,283 | 3,697,369 | 34,011 | 32,017 | 0.15 | 0.07 | 0.07 |
rs | 3,688,324 | 3,674,549 | 45,564 | 41,991 | 0.15 | 0.10 | 0.09 |
si | 3,658,038 | 3,634,731 | 56,975 | 52,401 | 0.15 | 0.12 | 0.12 |
biz | 3,525,111 | 3,507,128 | 88,040 | 84,236 | 0.14 | 0.19 | 0.19 |
cc | 3,437,236 | 3,408,584 | 83,672 | 80,136 | 0.14 | 0.18 | 0.18 |
kz | 3,319,768 | 3,308,521 | 39,218 | 37,608 | 0.13 | 0.08 | 0.08 |
cat | 3,250,065 | 3,227,823 | 27,741 | 26,004 | 0.13 | 0.06 | 0.06 |
th | 2,982,396 | 2,973,961 | 35,706 | 32,286 | 0.12 | 0.08 | 0.07 |
hk | 2,961,745 | 2,944,061 | 38,362 | 34,285 | 0.12 | 0.08 | 0.08 |
lv | 2,890,801 | 2,877,971 | 35,401 | 32,397 | 0.12 | 0.07 | 0.07 |
my | 2,853,681 | 2,842,421 | 48,581 | 46,202 | 0.11 | 0.10 | 0.10 |
pro | 2,166,866 | 2,154,540 | 50,589 | 49,527 | 0.09 | 0.11 | 0.11 |
pe | 2,096,494 | 2,086,405 | 25,867 | 24,673 | 0.08 | 0.05 | 0.05 |
pk | 1,992,950 | 1,983,808 | 23,311 | 22,304 | 0.08 | 0.05 | 0.05 |
news | 1,967,128 | 1,960,999 | 8,734 | 8,018 | 0.08 | 0.02 | 0.02 |
sg | 1,955,905 | 1,947,250 | 38,011 | 36,153 | 0.08 | 0.08 | 0.08 |
top | 1,877,178 | 1,864,312 | 119,650 | 112,408 | 0.08 | 0.25 | 0.25 |
store | 1,838,246 | 1,830,433 | 37,306 | 36,698 | 0.07 | 0.08 | 0.08 |
su | 1,822,618 | 1,817,201 | 22,917 | 21,983 | 0.07 | 0.05 | 0.05 |
club | 1,656,576 | 1,648,680 | 36,276 | 35,544 | 0.07 | 0.08 | 0.08 |
ng | 1,461,047 | 1,454,135 | 17,871 | 17,182 | 0.06 | 0.04 | 0.04 |
ai | 1,458,167 | 1,447,113 | 46,217 | 45,300 | 0.06 | 0.10 | 0.10 |
ae | 1,453,966 | 1,445,960 | 23,954 | 22,957 | 0.06 | 0.05 | 0.05 |
is | 1,421,978 | 1,414,420 | 18,952 | 17,811 | 0.06 | 0.04 | 0.04 |
ph | 1,394,578 | 1,386,515 | 18,617 | 17,836 | 0.06 | 0.04 | 0.04 |
fm | 1,380,953 | 1,376,323 | 8,786 | 8,563 | 0.06 | 0.02 | 0.02 |
az | 1,350,237 | 1,344,956 | 9,908 | 8,881 | 0.05 | 0.02 | 0.02 |
nu | 1,287,977 | 1,282,088 | 30,110 | 28,755 | 0.05 | 0.06 | 0.06 |