Number of pages, distribution of top-level domains, crawl overlaps, etc. - basic metrics about Common Crawl Monthly Crawl Archives
Latest crawl: CC-MAIN-2024-10
Detailed numbers and percentage of top-level domains (groups) in the latest monthly crawl (CC-MAIN-2024-10). Note that Internationalized country code TLDs (IDN ccTLDs) are mapped to their ASCII equivalents before counting TLDs. E.g., the counts of .ru
contain also the occurrences of .рф.
pages | urls | hosts | domains | %urls | %hosts | %domains | |
---|---|---|---|---|---|---|---|
type | |||||||
gTLD | 1,730,534,374 | 1,716,434,482 | 26,492,490 | 25,182,694 | 55.71 | 57.11 | 57.51 |
ccTLD | 1,308,108,107 | 1,297,499,274 | 19,439,479 | 18,166,637 | 42.11 | 41.91 | 41.49 |
sTLD | 55,645,285 | 55,117,023 | 225,379 | 215,724 | 1.79 | 0.49 | 0.49 |
grTLD | 7,549,900 | 7,504,696 | 150,434 | 144,312 | 0.24 | 0.32 | 0.33 |
IDN ccTLD | 4,343,847 | 4,326,140 | 73,937 | 72,294 | 0.14 | 0.16 | 0.17 |
IDN gTLD | 210,903 | 210,340 | 4,810 | 4,733 | 0.01 | 0.01 | 0.01 |
ARPA | 2 | 2 | 2 | 2 | 0.00 | 0.00 | 0.00 |
pages | urls | hosts | domains | %urls | %hosts | %domains | |
---|---|---|---|---|---|---|---|
tld | |||||||
com | 1,363,820,843 | 1,352,662,424 | 21,415,753 | 20,371,863 | 43.90 | 46.17 | 46.53 |
org | 176,628,223 | 175,305,520 | 1,949,544 | 1,817,146 | 5.69 | 4.20 | 4.15 |
ru | 135,649,457 | 135,002,702 | 1,303,469 | 1,235,591 | 4.38 | 2.81 | 2.82 |
de | 125,699,122 | 123,447,651 | 2,878,421 | 2,657,304 | 4.01 | 6.21 | 6.07 |
net | 103,650,360 | 102,855,312 | 1,503,637 | 1,417,824 | 3.34 | 3.24 | 3.24 |
uk | 65,285,855 | 64,746,322 | 1,017,129 | 947,759 | 2.10 | 2.19 | 2.16 |
jp | 62,997,966 | 62,640,108 | 853,262 | 791,356 | 2.03 | 1.84 | 1.81 |
it | 61,850,597 | 61,471,929 | 851,056 | 801,299 | 2.00 | 1.83 | 1.83 |
fr | 56,185,698 | 55,781,368 | 880,808 | 838,937 | 1.81 | 1.90 | 1.92 |
pl | 53,042,862 | 52,718,987 | 690,570 | 655,307 | 1.71 | 1.49 | 1.50 |
nl | 52,187,517 | 51,794,001 | 1,066,748 | 1,002,660 | 1.68 | 2.30 | 2.29 |
br | 39,208,882 | 38,991,693 | 452,750 | 416,543 | 1.27 | 0.98 | 0.95 |
edu | 35,097,822 | 34,813,445 | 129,819 | 126,628 | 1.13 | 0.28 | 0.29 |
au | 32,470,283 | 32,252,158 | 453,444 | 432,573 | 1.05 | 0.98 | 0.99 |
es | 31,652,617 | 31,441,024 | 365,541 | 345,987 | 1.02 | 0.79 | 0.79 |
cz | 30,361,180 | 30,126,013 | 418,012 | 389,861 | 0.98 | 0.90 | 0.89 |
ca | 29,719,598 | 29,484,406 | 442,501 | 420,546 | 0.96 | 0.95 | 0.96 |
cn | 22,974,413 | 22,917,928 | 757,129 | 684,534 | 0.74 | 1.63 | 1.56 |
eu | 21,400,739 | 21,252,165 | 349,086 | 326,650 | 0.69 | 0.75 | 0.75 |
info | 20,309,656 | 20,216,824 | 301,429 | 285,243 | 0.66 | 0.65 | 0.65 |
ro | 19,537,781 | 19,458,304 | 198,587 | 181,771 | 0.63 | 0.43 | 0.42 |
se | 19,227,530 | 19,073,628 | 313,869 | 295,962 | 0.62 | 0.68 | 0.68 |
ch | 18,943,325 | 18,777,087 | 439,401 | 405,457 | 0.61 | 0.95 | 0.93 |
in | 18,521,850 | 18,414,707 | 272,249 | 256,919 | 0.60 | 0.59 | 0.59 |
gr | 17,479,103 | 17,405,154 | 169,236 | 156,058 | 0.56 | 0.36 | 0.36 |
hu | 16,682,847 | 16,574,699 | 205,133 | 191,413 | 0.54 | 0.44 | 0.44 |
vn | 16,532,462 | 16,447,016 | 150,627 | 143,970 | 0.53 | 0.32 | 0.33 |
be | 16,518,254 | 16,382,307 | 337,665 | 313,795 | 0.53 | 0.73 | 0.72 |
ua | 16,018,353 | 15,887,928 | 104,473 | 99,991 | 0.52 | 0.23 | 0.23 |
at | 14,450,515 | 14,320,482 | 332,369 | 312,672 | 0.46 | 0.72 | 0.71 |
co | 14,029,639 | 13,648,175 | 206,671 | 200,630 | 0.44 | 0.45 | 0.46 |
gov | 13,070,596 | 12,940,904 | 30,285 | 27,759 | 0.42 | 0.07 | 0.06 |
dk | 12,922,729 | 12,807,203 | 234,311 | 221,760 | 0.42 | 0.51 | 0.51 |
id | 11,985,115 | 11,938,369 | 137,340 | 131,972 | 0.39 | 0.30 | 0.30 |
fi | 11,237,335 | 11,143,601 | 164,947 | 154,777 | 0.36 | 0.36 | 0.35 |
sk | 10,987,818 | 10,907,084 | 142,336 | 132,080 | 0.35 | 0.31 | 0.30 |
ir | 10,722,697 | 10,678,230 | 109,245 | 105,308 | 0.35 | 0.24 | 0.24 |
ar | 10,556,508 | 10,507,214 | 137,274 | 120,785 | 0.34 | 0.30 | 0.28 |
kr | 10,508,148 | 10,468,062 | 220,953 | 187,861 | 0.34 | 0.48 | 0.43 |
no | 10,432,840 | 10,336,580 | 150,761 | 142,074 | 0.34 | 0.33 | 0.32 |
pt | 9,246,744 | 9,185,696 | 109,797 | 100,705 | 0.30 | 0.24 | 0.23 |
tw | 9,151,600 | 9,095,974 | 107,162 | 96,678 | 0.30 | 0.23 | 0.22 |
tr | 8,536,008 | 8,499,523 | 101,993 | 92,210 | 0.28 | 0.22 | 0.21 |
mx | 7,879,203 | 7,832,237 | 99,478 | 93,044 | 0.25 | 0.21 | 0.21 |
io | 7,428,003 | 7,356,123 | 211,956 | 209,256 | 0.24 | 0.46 | 0.48 |
cl | 7,017,166 | 6,978,074 | 97,148 | 91,277 | 0.23 | 0.21 | 0.21 |
nz | 6,994,610 | 6,938,880 | 105,020 | 99,888 | 0.23 | 0.23 | 0.23 |
za | 6,441,308 | 6,401,735 | 106,175 | 99,866 | 0.21 | 0.23 | 0.23 |
us | 6,357,615 | 6,318,824 | 114,375 | 109,393 | 0.21 | 0.25 | 0.25 |
ie | 6,025,112 | 5,976,305 | 78,975 | 75,150 | 0.19 | 0.17 | 0.17 |
il | 5,804,909 | 5,750,699 | 68,764 | 65,344 | 0.19 | 0.15 | 0.15 |
tv | 5,760,864 | 5,713,222 | 46,059 | 43,385 | 0.19 | 0.10 | 0.10 |
by | 5,490,086 | 5,467,570 | 49,349 | 47,048 | 0.18 | 0.11 | 0.11 |
app | 5,277,507 | 5,260,702 | 79,211 | 78,482 | 0.17 | 0.17 | 0.18 |
me | 5,279,123 | 5,234,825 | 101,499 | 99,309 | 0.17 | 0.22 | 0.23 |
hr | 5,192,809 | 5,165,346 | 54,810 | 49,191 | 0.17 | 0.12 | 0.11 |
lt | 5,036,216 | 4,994,062 | 67,927 | 62,970 | 0.16 | 0.15 | 0.14 |
si | 4,805,582 | 4,775,679 | 59,183 | 53,480 | 0.15 | 0.13 | 0.12 |
rs | 4,773,917 | 4,746,893 | 49,319 | 44,691 | 0.15 | 0.11 | 0.10 |
ee | 4,634,287 | 4,602,294 | 52,573 | 46,732 | 0.15 | 0.11 | 0.11 |
bg | 4,485,676 | 4,462,419 | 33,955 | 31,685 | 0.14 | 0.07 | 0.07 |
biz | 4,421,247 | 4,399,399 | 99,681 | 95,005 | 0.14 | 0.21 | 0.22 |
shop | 4,435,153 | 4,399,205 | 55,635 | 54,284 | 0.14 | 0.12 | 0.12 |
online | 4,232,118 | 4,202,972 | 72,984 | 71,665 | 0.14 | 0.16 | 0.16 |
cat | 3,935,571 | 3,896,621 | 30,424 | 28,161 | 0.13 | 0.07 | 0.06 |
kz | 3,773,981 | 3,752,566 | 37,597 | 36,027 | 0.12 | 0.08 | 0.08 |
my | 3,670,759 | 3,646,438 | 51,527 | 48,117 | 0.12 | 0.11 | 0.11 |
xyz | 3,666,079 | 3,645,540 | 94,390 | 92,726 | 0.12 | 0.20 | 0.21 |
cc | 3,608,804 | 3,587,784 | 55,396 | 52,099 | 0.12 | 0.12 | 0.12 |
hk | 3,433,047 | 3,404,169 | 40,478 | 35,992 | 0.11 | 0.09 | 0.08 |
lv | 3,341,545 | 3,310,643 | 38,863 | 34,950 | 0.11 | 0.08 | 0.08 |
th | 3,302,880 | 3,287,609 | 36,254 | 32,471 | 0.11 | 0.08 | 0.07 |
sg | 2,769,125 | 2,746,818 | 40,106 | 37,585 | 0.09 | 0.09 | 0.09 |
pro | 2,727,338 | 2,706,549 | 44,400 | 43,283 | 0.09 | 0.10 | 0.10 |
pe | 2,611,695 | 2,593,839 | 26,710 | 25,115 | 0.08 | 0.06 | 0.06 |
pk | 2,551,908 | 2,536,548 | 23,365 | 22,013 | 0.08 | 0.05 | 0.05 |
top | 2,377,962 | 2,365,864 | 110,506 | 104,401 | 0.08 | 0.24 | 0.24 |
su | 2,357,120 | 2,346,743 | 23,439 | 22,295 | 0.08 | 0.05 | 0.05 |
news | 2,291,502 | 2,283,644 | 8,557 | 7,849 | 0.07 | 0.02 | 0.02 |
club | 2,092,617 | 2,080,210 | 35,720 | 34,908 | 0.07 | 0.08 | 0.08 |
store | 1,960,626 | 1,943,312 | 27,876 | 27,320 | 0.06 | 0.06 | 0.06 |
ng | 1,820,300 | 1,809,576 | 17,912 | 17,045 | 0.06 | 0.04 | 0.04 |
is | 1,821,030 | 1,806,521 | 19,925 | 18,649 | 0.06 | 0.04 | 0.04 |
ph | 1,808,211 | 1,795,125 | 19,195 | 18,246 | 0.06 | 0.04 | 0.04 |
nu | 1,751,273 | 1,738,848 | 34,124 | 32,149 | 0.06 | 0.07 | 0.07 |
ae | 1,741,684 | 1,725,413 | 21,441 | 20,248 | 0.06 | 0.05 | 0.05 |
fm | 1,656,415 | 1,643,548 | 8,827 | 8,556 | 0.05 | 0.02 | 0.02 |