Number of pages, distribution of top-level domains, crawl overlaps, etc. - basic metrics about Common Crawl Monthly Crawl Archives
Latest crawl: CC-MAIN-2025-21
Just the distribution (percentage / relative frequency) of URLs on top-level domains for few selected crawls, include the latest monthly crawl… Only TLDs which reach at least 0.05% in one of the selected crawls are shown.
crawl | CC-MAIN-2008-2009 | CC-MAIN-2012 | CC-MAIN-2014-10 | CC-MAIN-2016-30 | CC-MAIN-2019-09 | CC-MAIN-2022-49 | CC-MAIN-2025-21 |
---|---|---|---|---|---|---|---|
tld | |||||||
ae | 0.0010 | 0.0102 | 0.0123 | 0.0099 | 0.0266 | 0.0452 | 0.0746 |
ai | 0.0000 | 0.0004 | 0.0000 | 0.0000 | 0.0049 | 0.0257 | 0.0789 |
app | NaN | NaN | NaN | NaN | 0.0054 | 0.0992 | 0.1128 |
ar | 0.0818 | 0.2380 | 0.0991 | 0.0782 | 0.2123 | 0.3146 | 0.3481 |
at | 0.0025 | 0.3797 | 0.0632 | 0.1556 | 0.4178 | 0.4894 | 0.4454 |
au | 0.6008 | 0.9933 | 0.4579 | 0.5644 | 0.9376 | 0.2563 | 1.1715 |
az | 0.0002 | 0.0060 | 0.0012 | 0.0031 | 0.0393 | 0.0408 | 0.0560 |
ba | 0.0014 | 0.0101 | 0.0020 | 0.0028 | 0.0244 | 0.0377 | 0.0528 |
be | 0.0024 | 0.3972 | 0.0564 | 0.1547 | 0.4501 | 0.6211 | 0.5131 |
bg | 0.0010 | 0.0379 | 0.0078 | 0.0161 | 0.1109 | 0.1414 | 0.1331 |
biz | 0.5549 | 0.3746 | 0.0384 | 0.0532 | 0.2751 | 0.1652 | 0.1455 |
blog | NaN | NaN | NaN | NaN | 0.0886 | 0.1752 | 0.0460 |
br | 0.3366 | 0.7641 | 0.2175 | 0.2958 | 1.1828 | 0.3427 | 1.5931 |
by | 0.0023 | 0.0353 | 0.0009 | 0.0138 | 0.2156 | 0.1634 | 0.1954 |
ca | 0.0141 | 0.6899 | 0.4690 | 0.5206 | 0.7436 | 0.9774 | 0.9699 |
cat | 0.0004 | 0.0495 | 0.0103 | 0.0150 | 0.0927 | 0.1414 | 0.1163 |
cc | 0.0035 | 0.0957 | 0.0404 | 0.1105 | 0.1024 | 0.1144 | 0.1485 |
ch | 0.0033 | 0.5265 | 0.1018 | 0.2150 | 0.5642 | 0.6399 | 0.5884 |
cl | 0.0010 | 0.0834 | 0.0660 | 0.0633 | 0.1189 | 0.2034 | 0.2490 |
club | NaN | NaN | 0.0000 | 0.0004 | 0.0990 | 0.0750 | 0.0670 |
cn | 1.0883 | 0.4208 | 0.0157 | 0.3096 | 0.8917 | 0.7137 | 0.9411 |
co | 0.0165 | 0.0454 | 0.0328 | 0.0638 | 0.4113 | 0.4348 | 0.4673 |
com | 71.2855 | 55.7965 | 74.6931 | 69.1837 | 46.5228 | 48.6201 | 45.0165 |
cz | 0.0065 | 0.7826 | 0.0480 | 0.2279 | 1.0371 | 1.1055 | 0.9955 |
de | 0.0523 | 4.8439 | 0.7005 | 1.5107 | 4.0391 | 4.5011 | 4.2731 |
dk | 0.0033 | 0.3705 | 0.0803 | 0.1525 | 0.3846 | 0.4511 | 0.4684 |
edu | 1.2783 | 0.6181 | 3.0894 | 2.9478 | 0.6389 | 1.5073 | 1.0666 |
ee | 0.0003 | 0.0481 | 0.0058 | 0.0134 | 0.1143 | 0.1541 | 0.1566 |
es | 0.0339 | 0.6738 | 0.1764 | 0.2350 | 0.8848 | 1.0323 | 0.9506 |
eu | 0.0045 | 0.4589 | 0.1181 | 0.1842 | 0.7732 | 0.7495 | 0.6931 |
fi | 0.0009 | 0.1756 | 0.0262 | 0.0514 | 0.2876 | 0.3944 | 0.3492 |
fm | 0.0001 | 0.0631 | 0.0582 | 0.0879 | 0.0401 | 0.0666 | 0.0809 |
fr | 0.0197 | 1.3202 | 0.2698 | 0.4410 | 1.6463 | 1.9863 | 1.6968 |
gov | 0.3797 | 0.2692 | 1.6174 | 1.5330 | 0.2411 | 0.4903 | 0.4313 |
gr | 0.0014 | 0.1163 | 0.0574 | 0.0765 | 0.4000 | 0.5677 | 0.5389 |
hk | 0.0355 | 0.0431 | 0.0170 | 0.0253 | 0.0784 | 0.1075 | 0.1198 |
hr | 0.0034 | 0.0683 | 0.0136 | 0.0279 | 0.0950 | 0.1505 | 0.1793 |
hu | 0.0025 | 0.3732 | 0.0466 | 0.0971 | 0.4634 | 0.5877 | 0.4727 |
id | 0.0006 | 0.0648 | 0.0556 | 0.0675 | 0.1580 | 0.2772 | 0.4276 |
ie | 0.0017 | 0.1183 | 0.0746 | 0.0920 | 0.1360 | 0.2021 | 0.1756 |
il | 0.0136 | 0.0864 | 0.0216 | 0.0395 | 0.1695 | 0.0431 | 0.2362 |
in | 0.0100 | 0.2150 | 0.1058 | 0.1037 | 0.4128 | 0.5471 | 0.6760 |
info | 1.8065 | 1.3293 | 0.1828 | 0.2468 | 1.4370 | 0.7181 | 0.5948 |
int | 0.0000 | 0.0136 | 0.0739 | 0.0816 | 0.0166 | 0.0328 | 0.0248 |
io | NaN | 0.0049 | 0.0128 | 0.0189 | 0.1391 | 0.2231 | 0.2880 |
ir | 0.0007 | 0.0553 | 0.0141 | 0.0144 | 0.3398 | 0.3411 | 0.4007 |
is | 0.0001 | 0.0345 | 0.0379 | 0.0362 | 0.0396 | 0.0564 | 0.0589 |
it | 0.0439 | 1.3311 | 0.2255 | 0.4406 | 1.6574 | 2.0844 | 1.8125 |
jp | 0.0518 | 1.1748 | 0.1047 | 0.6076 | 2.1464 | 1.5369 | 2.1822 |
kr | 0.0009 | 0.0781 | 0.0451 | 0.0846 | 0.3449 | 0.3297 | 0.3965 |
kz | 0.0003 | 0.0215 | 0.0009 | 0.0096 | 0.1192 | 0.1161 | 0.1539 |
link | NaN | NaN | NaN | 0.0002 | 0.0199 | 0.0281 | 0.0589 |
lt | 0.0003 | 0.0999 | 0.0063 | 0.0218 | 0.1429 | 0.1667 | 0.1792 |
lv | 0.0013 | 0.0609 | 0.0048 | 0.0140 | 0.0987 | 0.1008 | 0.1075 |
me | 0.0000 | 0.0586 | 0.0367 | 0.0557 | 0.3160 | 0.2104 | 0.1812 |
mx | 0.0786 | 0.1461 | 0.0752 | 0.0898 | 0.1992 | 0.1444 | 0.3235 |
my | 0.0302 | 0.0465 | 0.0285 | 0.0414 | 0.0788 | 0.1083 | 0.1266 |
net | 9.2006 | 5.4523 | 3.2464 | 3.4275 | 4.3670 | 3.7199 | 3.3103 |
news | NaN | NaN | NaN | 0.0017 | 0.0349 | 0.0821 | 0.0862 |
ng | 0.0008 | 0.0013 | 0.0008 | 0.0010 | 0.0359 | 0.0578 | 0.0658 |
nl | 0.0073 | 1.4584 | 0.1612 | 0.4426 | 1.4058 | 1.8702 | 1.5330 |
no | 0.0063 | 0.2376 | 0.0612 | 0.0717 | 0.2699 | 0.3576 | 0.3184 |
nu | 0.0001 | 0.0662 | 0.0271 | 0.0208 | 0.0747 | 0.0572 | 0.0495 |
nz | 0.0322 | 0.2275 | 0.0710 | 0.0815 | 0.2162 | 0.0694 | 0.2430 |
online | NaN | NaN | NaN | 0.0013 | 0.0662 | 0.1163 | 0.1610 |
org | 8.1017 | 6.0326 | 9.8271 | 10.5485 | 5.4686 | 6.5296 | 5.2896 |
pe | 0.0155 | 0.0288 | 0.0077 | 0.0153 | 0.0463 | 0.0722 | 0.0928 |
ph | 0.0157 | 0.0316 | 0.0106 | 0.0096 | 0.0389 | 0.0559 | 0.0641 |
pk | 0.0074 | 0.0141 | 0.0109 | 0.0137 | 0.0411 | 0.0695 | 0.0942 |
pl | 0.1857 | 1.8384 | 0.0851 | 0.2666 | 1.5376 | 1.6942 | 1.7395 |
pro | 0.0000 | 0.0082 | 0.0011 | 0.0019 | 0.0706 | 0.0863 | 0.0958 |
pt | 0.0178 | 0.1075 | 0.0414 | 0.0590 | 0.2080 | 0.2818 | 0.3083 |
ro | 0.0191 | 0.5857 | 0.0252 | 0.0918 | 0.4791 | 0.5707 | 0.6644 |
rs | 0.0002 | 0.0389 | 0.0074 | 0.0186 | 0.0935 | 0.1486 | 0.1551 |
ru | 0.0533 | 1.7335 | 0.0891 | 0.6051 | 6.1134 | 4.3371 | 4.4093 |
se | 0.0019 | 0.5330 | 0.1349 | 0.1919 | 0.7082 | 0.6311 | 0.5942 |
sg | 0.0229 | 0.0486 | 0.0190 | 0.0219 | 0.0636 | 0.0839 | 0.0842 |
shop | NaN | NaN | NaN | 0.0000 | 0.0243 | 0.1147 | 0.2442 |
si | 0.0027 | 0.0569 | 0.0071 | 0.0105 | 0.0896 | 0.1521 | 0.1596 |
sk | 0.0010 | 0.1601 | 0.0107 | 0.0502 | 0.2659 | 0.3714 | 0.3674 |
store | NaN | NaN | NaN | 0.0000 | 0.0090 | 0.0504 | 0.0936 |
su | 0.0001 | 0.0304 | 0.0025 | 0.0052 | 0.1153 | 0.0839 | 0.0779 |
th | 0.0003 | 0.0217 | 0.0206 | 0.0316 | 0.0680 | 0.0807 | 0.1165 |
tk | 0.0000 | 0.0244 | 0.0015 | 0.0021 | 0.0981 | 0.0442 | 0.0034 |
top | NaN | NaN | NaN | 0.0001 | 0.0404 | 0.0696 | 0.0881 |
tr | 0.0623 | 0.1156 | 0.0503 | 0.0769 | 0.1944 | 0.0919 | 0.3828 |
tv | 0.0013 | 0.2584 | 0.1455 | 0.1339 | 0.2394 | 0.2084 | 0.2075 |
tw | 0.0923 | 0.1092 | 0.0505 | 0.0985 | 0.3123 | 0.1672 | 0.3693 |
ua | 0.2190 | 0.4143 | 0.0099 | 0.0538 | 0.9984 | 0.3666 | 0.7142 |
uk | 4.0407 | 3.5031 | 1.5932 | 1.6472 | 2.5186 | 0.4007 | 2.2828 |
us | 0.0046 | 0.4176 | 0.3289 | 0.2699 | 0.3470 | 0.2399 | 0.1926 |
vn | 0.0136 | 0.0797 | 0.0500 | 0.0667 | 0.3598 | 0.5071 | 0.5272 |
ws | 0.0006 | 0.0841 | 0.0109 | 0.0081 | 0.0287 | 0.0174 | 0.0183 |
xyz | NaN | NaN | NaN | 0.0008 | 0.1077 | 0.1506 | 0.1261 |
za | 0.0126 | 0.1613 | 0.0737 | 0.1456 | 0.2099 | 0.0355 | 0.2621 |