Statistics of Common Crawl Monthly Archives

Number of pages, distribution of top-level domains, crawl overlaps, etc. - basic metrics about Common Crawl Monthly Crawl Archives
Latest crawl: CC-MAIN-2026-43

View the Project on GitHub

Distribution of Languages

The language of a document is identified by Compact Language Detector 2 (CLD2). It is able to identify 160 different languages and up to 3 languages per document. The table lists the percentage covered by the primary language of a document (returned first by CLD2). So far, only HTML pages are passed to the language detector. The underlying data including page counts is provided in languages.csv.

crawl CC-MAIN-2026-34 CC-MAIN-2026-39 CC-MAIN-2026-43
language
<unknown> 3.2480 3.0577 3.0552
aar 0.0001 0.0001 0.0001
abk 0.0007 0.0007 0.0006
afr 0.0082 0.0079 0.0080
aka 0.0003 0.0003 0.0003
amh 0.0046 0.0046 0.0042
ara 0.6677 0.6679 0.6675
asm 0.0031 0.0031 0.0031
aym 0.0001 0.0001 0.0001
aze 0.0617 0.0609 0.0612
bak 0.0034 0.0032 0.0031
bel 0.0172 0.0168 0.0157
ben 0.1084 0.1100 0.1142
bih 0.0006 0.0006 0.0008
bis 0.0001 0.0002 0.0002
bod 0.0012 0.0011 0.0011
bos 0.0641 0.0578 0.0585
bre 0.0024 0.0020 0.0020
bul 0.2903 0.3119 0.2446
cat 0.2049 0.1976 0.2089
ceb 0.0016 0.0016 0.0019
ces 1.1458 1.1277 1.1472
chr 0.0000 0.0000 0.0000
cos 0.0029 0.0029 0.0030
crs 0.0001 0.0001 0.0001
cym 0.0111 0.0116 0.0116
dan 0.5240 0.5233 0.5156
deu 5.9083 5.7252 5.7673
div 0.0025 0.0024 0.0024
dzo 0.0001 0.0001 0.0001
ell 0.5490 0.5190 0.5107
eng 40.4526 41.8620 42.5647
epo 0.0082 0.0088 0.0071
est 0.1466 0.1447 0.1457
eus 0.0374 0.0383 0.0372
fao 0.0053 0.0058 0.0057
fas 0.7603 0.6934 0.6164
fij 0.0001 0.0001 0.0001
fin 0.3823 0.3723 0.3766
fra 4.8476 4.6769 4.7605
fry 0.0020 0.0023 0.0020
gla 0.0019 0.0018 0.0020
gle 0.0069 0.0071 0.0077
glg 0.0339 0.0315 0.0333
glv 0.0008 0.0007 0.0008
got 0.0000 0.0000 0.0000
grn 0.0010 0.0010 0.0009
guj 0.0127 0.0120 0.0128
hat 0.0028 0.0027 0.0031
hau 0.0034 0.0030 0.0040
haw 0.0008 0.0009 0.0011
heb 0.2666 0.2567 0.2639
hin 0.2144 0.2147 0.2111
hmn 0.0010 0.0009 0.0010
hrv 0.2219 0.2124 0.2173
hun 0.5341 0.5253 0.5320
hye 0.0320 0.0315 0.0304
ibo 0.0014 0.0013 0.0016
iku 0.0004 0.0004 0.0004
ile 0.0003 0.0003 0.0003
ina 0.0007 0.0007 0.0006
ind 1.1182 1.0945 1.0636
ipk 0.0001 0.0001 0.0001
isl 0.0346 0.0351 0.0366
ita 2.4624 2.4622 2.5792
jav 0.0031 0.0032 0.0031
jpn 5.2484 5.1668 4.8876
kal 0.0009 0.0009 0.0009
kan 0.0151 0.0145 0.0150
kas 0.0000 0.0000 0.0000
kat 0.0464 0.0457 0.0471
kaz 0.0458 0.0428 0.0385
kha 0.0004 0.0004 0.0005
khm 0.0110 0.0107 0.0107
kin 0.0029 0.0029 0.0033
kir 0.0116 0.0099 0.0123
kor 0.8500 0.8396 0.8816
kur 0.0071 0.0069 0.0065
lao 0.0061 0.0055 0.0056
lat 0.1040 0.1052 0.0961
lav 0.0911 0.0886 0.0918
lif 0.0000 0.0000 0.0000
lin 0.0005 0.0006 0.0006
lit 0.1756 0.1700 0.1723
ltz 0.0038 0.0037 0.0038
lug 0.0004 0.0004 0.0004
mal 0.0238 0.0220 0.0224
mar 0.0267 0.0264 0.0274
mfe 0.0002 0.0002 0.0001
mkd 0.0396 0.0394 0.0409
mlg 0.0041 0.0036 0.0042
mlt 0.0032 0.0032 0.0034
mon 0.0162 0.0159 0.0159
mri 0.0014 0.0013 0.0019
msa 0.0860 0.0855 0.0917
mya 0.0164 0.0165 0.0157
nau 0.0000 0.0000 0.0000
nep 0.0596 0.0534 0.0503
nld 1.7762 1.7381 1.7510
nno 0.0142 0.0141 0.0138
nor 0.3313 0.3358 0.3386
nso 0.0001 0.0001 0.0001
nya 0.0009 0.0009 0.0008
oci 0.0033 0.0033 0.0031
ori 0.0071 0.0066 0.0064
orm 0.0007 0.0007 0.0007
pan 0.0089 0.0091 0.0098
pol 2.0638 2.0329 1.8900
por 2.5263 2.4899 2.4697
pus 0.0043 0.0045 0.0045
que 0.0009 0.0009 0.0008
roh 0.0014 0.0013 0.0013
ron 0.5926 0.5730 0.5758
run 0.0002 0.0001 0.0002
rus 6.8659 6.6954 6.3018
sag 0.0005 0.0005 0.0005
san 0.0034 0.0033 0.0039
sco 0.0006 0.0007 0.0007
sin 0.0089 0.0085 0.0091
slk 0.4131 0.3951 0.4043
slv 0.1553 0.1523 0.1569
smo 0.0008 0.0007 0.0009
sna 0.0011 0.0011 0.0011
snd 0.0023 0.0021 0.0020
som 0.0056 0.0056 0.0060
sot 0.0007 0.0007 0.0009
spa 4.6257 4.7448 4.7390
sqi 0.0502 0.0481 0.0469
srp 0.2671 0.2577 0.2635
ssw 0.0000 0.0000 0.0001
sun 0.0014 0.0016 0.0015
sux 0.0000 0.0000 0.0000
swa 0.0117 0.0109 0.0099
swe 0.7245 0.7196 0.7077
syr 0.0004 0.0003 0.0003
tam 0.0449 0.0449 0.0434
tat 0.0075 0.0077 0.0076
tel 0.0199 0.0191 0.0184
tgk 0.0072 0.0065 0.0068
tgl 0.0121 0.0121 0.0128
tha 0.3740 0.3577 0.3480
tir 0.0011 0.0007 0.0006
ton 0.0001 0.0001 0.0001
tsn 0.0003 0.0004 0.0003
tso 0.0001 0.0001 0.0001
tuk 0.0033 0.0030 0.0030
tur 1.3598 1.2777 1.1445
uig 0.0018 0.0016 0.0017
ukr 0.8371 0.8138 0.7870
urd 0.0322 0.0308 0.0315
uzb 0.0313 0.0311 0.0297
ven 0.0000 0.0000 0.0001
vie 1.0750 0.9927 0.9800
vol 0.0012 0.0008 0.0009
war 0.0016 0.0014 0.0014
wol 0.0002 0.0002 0.0002
xho 0.0010 0.0011 0.0010
yid 0.0017 0.0017 0.0022
yor 0.0017 0.0017 0.0017
zha 0.0001 0.0001 0.0001
zho 4.3829 4.2255 4.3706
zul 0.0017 0.0018 0.0017