Statistics of Common Crawl Monthly Archives

Number of pages, distribution of top-level domains, crawl overlaps, etc. - basic metrics about Common Crawl Monthly Crawl Archives
Latest crawl: CC-MAIN-2024-26

View the Project on GitHub

Top-Level Domains of the Latest Monthly Crawl

Detailed numbers and percentage of top-level domains (groups) in the latest monthly crawl (CC-MAIN-2024-26). Note that Internationalized country code TLDs (IDN ccTLDs) are mapped to their ASCII equivalents before counting TLDs. E.g., the counts of .ru contain also the occurrences of .рф.

pages urls hosts domains %urls %hosts %domains
type
gTLD 1,566,930,159 1,557,119,698 30,360,507 29,084,818 56.00 57.61 57.94
ccTLD 1,168,753,731 1,161,527,940 21,891,977 20,677,811 41.77 41.54 41.20
sTLD 51,444,105 51,066,107 218,069 209,684 1.84 0.41 0.42
grTLD 6,512,470 6,476,580 149,456 143,845 0.23 0.28 0.29
IDN ccTLD 4,079,581 4,067,960 74,968 73,265 0.15 0.14 0.15
IDN gTLD 183,231 182,845 4,557 4,474 0.01 0.01 0.01
ARPA 2 2 2 2 0.00 0.00 0.00

pages urls hosts domains %urls %hosts %domains
tld
com 1,243,494,224 1,235,567,726 25,075,331 24,045,270 44.44 47.58 47.90
org 153,777,885 152,871,990 1,932,694 1,810,622 5.50 3.67 3.61
ru 118,563,512 118,103,946 1,290,887 1,225,184 4.25 2.45 2.44
de 118,180,707 117,238,618 4,013,498 3,770,242 4.22 7.62 7.51
net 90,569,380 90,064,333 1,491,678 1,413,147 3.24 2.83 2.82
it 56,041,308 55,726,897 1,069,601 1,019,908 2.00 2.03 2.03
uk 55,333,293 55,003,371 1,065,901 1,013,002 1.98 2.02 2.02
jp 55,179,427 54,936,559 831,888 782,224 1.98 1.58 1.56
fr 47,447,891 47,162,142 873,888 834,276 1.70 1.66 1.66
pl 45,838,062 45,586,778 890,394 856,423 1.64 1.69 1.71
nl 42,950,261 42,637,992 1,038,044 977,566 1.53 1.97 1.95
br 36,555,665 36,402,493 496,283 464,457 1.31 0.94 0.93
edu 33,456,425 33,261,678 126,555 123,629 1.20 0.24 0.25
cz 30,379,265 30,158,761 441,856 410,842 1.08 0.84 0.82
es 29,081,288 28,870,359 376,326 356,173 1.04 0.71 0.71
au 27,999,894 27,858,322 494,756 478,064 1.00 0.94 0.95
ca 24,745,509 24,584,059 444,786 424,933 0.88 0.84 0.85
cn 22,003,024 21,942,966 623,868 571,632 0.79 1.18 1.14
eu 21,804,306 21,570,460 1,225,828 1,201,438 0.78 2.33 2.39
info 16,955,558 16,882,469 287,150 272,434 0.61 0.54 0.54
ro 16,680,842 16,624,034 202,501 186,548 0.60 0.38 0.37
ua 16,814,050 16,342,857 110,471 106,362 0.59 0.21 0.21
ch 16,320,457 16,208,442 435,889 403,414 0.58 0.83 0.80
se 16,078,161 15,953,084 307,718 290,630 0.57 0.58 0.58
in 15,790,287 15,683,497 274,758 260,643 0.56 0.52 0.52
gr 15,383,826 15,326,737 172,611 160,004 0.55 0.33 0.32
vn 14,800,917 14,697,439 150,427 143,993 0.53 0.29 0.29
hu 14,707,591 14,637,012 199,918 186,848 0.53 0.38 0.37
be 13,853,714 13,765,215 328,803 306,126 0.50 0.62 0.61
dk 12,911,169 12,803,377 269,004 255,064 0.46 0.51 0.51
co 12,302,633 12,150,607 212,941 207,260 0.44 0.40 0.41
kr 11,593,017 11,555,510 220,775 187,774 0.42 0.42 0.37
tr 11,307,359 11,270,460 119,773 109,714 0.41 0.23 0.22
at 11,268,138 11,179,385 290,206 272,381 0.40 0.55 0.54
gov 11,078,320 10,983,560 27,956 26,306 0.40 0.05 0.05
id 10,852,877 10,799,652 144,444 139,515 0.39 0.27 0.28
sk 9,930,186 9,870,757 141,923 131,637 0.36 0.27 0.26
fi 9,578,079 9,515,482 165,872 155,170 0.34 0.31 0.31
tw 9,396,696 9,356,943 111,214 101,801 0.34 0.21 0.20
ar 9,190,214 9,155,662 137,757 120,993 0.33 0.26 0.24
ir 9,094,992 9,064,194 107,062 103,248 0.33 0.20 0.21
no 8,622,393 8,560,713 148,400 140,103 0.31 0.28 0.28
pt 7,932,952 7,892,562 109,352 100,645 0.28 0.21 0.20
mx 6,952,650 6,915,098 101,593 95,955 0.25 0.19 0.19
io 6,846,730 6,791,544 210,342 207,935 0.24 0.40 0.41
cl 6,163,531 6,136,723 99,869 94,094 0.22 0.19 0.19
za 5,838,481 5,817,035 117,984 112,197 0.21 0.22 0.22
nz 5,841,799 5,808,130 111,478 107,389 0.21 0.21 0.21
tv 5,514,377 5,479,610 45,621 43,151 0.20 0.09 0.09
us 5,189,733 5,161,709 108,855 104,744 0.19 0.21 0.21
il 5,181,689 5,154,819 76,923 73,908 0.19 0.15 0.15
app 5,010,793 5,000,673 87,024 86,324 0.18 0.17 0.17
ie 4,998,980 4,964,528 77,491 74,060 0.18 0.15 0.15
me 4,913,332 4,885,296 95,684 93,616 0.18 0.18 0.19
by 4,845,200 4,831,364 49,865 47,637 0.17 0.09 0.09
xyz 4,790,468 4,741,974 105,729 104,283 0.17 0.20 0.21
hr 4,624,803 4,604,049 55,058 49,587 0.17 0.10 0.10
lt 4,521,907 4,491,391 67,777 62,864 0.16 0.13 0.13
rs 4,450,351 4,434,312 50,405 45,806 0.16 0.10 0.09
ee 4,458,358 4,430,203 57,012 50,831 0.16 0.11 0.10
si 4,234,656 4,214,614 60,054 54,365 0.15 0.11 0.11
shop 4,015,191 3,995,620 67,569 66,022 0.14 0.13 0.13
online 3,987,868 3,968,373 78,644 77,324 0.14 0.15 0.15
bg 3,937,544 3,922,120 33,950 31,724 0.14 0.06 0.06
cc 3,911,124 3,882,978 58,511 55,263 0.14 0.11 0.11
biz 3,789,319 3,770,916 96,573 92,365 0.14 0.18 0.18
kz 3,450,082 3,430,667 37,761 36,172 0.12 0.07 0.07
cat 3,262,907 3,238,834 29,254 27,139 0.12 0.06 0.05
hk 3,194,658 3,170,228 39,669 35,427 0.11 0.08 0.07
my 3,097,195 3,083,603 49,814 47,143 0.11 0.09 0.09
lv 3,006,261 2,988,254 39,064 35,116 0.11 0.07 0.07
th 2,971,044 2,960,817 34,907 31,290 0.11 0.07 0.06
pro 2,374,528 2,358,585 46,655 45,562 0.08 0.09 0.09
pe 2,298,885 2,288,113 27,330 25,809 0.08 0.05 0.05
pk 2,241,852 2,230,269 23,964 22,793 0.08 0.05 0.05
sg 2,218,578 2,206,222 40,221 37,883 0.08 0.08 0.08
top 2,154,407 2,137,545 104,092 99,268 0.08 0.20 0.20
news 2,060,219 2,053,230 8,639 7,938 0.07 0.02 0.02
su 1,933,852 1,926,400 22,642 21,539 0.07 0.04 0.04
store 1,772,056 1,763,534 31,605 31,044 0.06 0.06 0.06
fm 1,751,189 1,742,646 8,232 7,984 0.06 0.02 0.02
club 1,702,516 1,694,201 36,527 35,714 0.06 0.07 0.07
ng 1,621,152 1,614,870 19,100 18,270 0.06 0.04 0.04
ph 1,654,459 1,597,304 20,487 19,485 0.06 0.04 0.04
is 1,591,197 1,582,526 19,787 18,398 0.06 0.04 0.04
ae 1,495,567 1,486,277 22,609 21,539 0.05 0.04 0.04
nu 1,459,181 1,448,343 32,885 30,988 0.05 0.06 0.06
az 1,428,511 1,424,700 10,082 8,945 0.05 0.02 0.02