Statistics of Common Crawl Monthly Archives

Number of pages, distribution of top-level domains, crawl overlaps, etc. - basic metrics about Common Crawl Monthly Crawl Archives
Latest crawl: CC-MAIN-2026-39

View the Project on GitHub

Distribution of Languages

The language of a document is identified by Compact Language Detector 2 (CLD2). It is able to identify 160 different languages and up to 3 languages per document. The table lists the percentage covered by the primary language of a document (returned first by CLD2). So far, only HTML pages are passed to the language detector. The underlying data including page counts is provided in languages.csv.

crawl CC-MAIN-2026-30 CC-MAIN-2026-34 CC-MAIN-2026-39
language
<unknown> 3.2020 3.2480 3.0577
aar 0.0001 0.0001 0.0001
abk 0.0006 0.0007 0.0007
afr 0.0078 0.0082 0.0079
aka 0.0004 0.0003 0.0003
amh 0.0037 0.0046 0.0046
ara 0.6548 0.6677 0.6679
asm 0.0046 0.0031 0.0031
aym 0.0001 0.0001 0.0001
aze 0.0620 0.0617 0.0609
bak 0.0031 0.0034 0.0032
bel 0.0158 0.0172 0.0168
ben 0.1096 0.1084 0.1100
bih 0.0006 0.0006 0.0006
bis 0.0002 0.0001 0.0002
bod 0.0012 0.0012 0.0011
bos 0.0595 0.0641 0.0578
bre 0.0021 0.0024 0.0020
bul 0.2808 0.2903 0.3119
cat 0.2065 0.2049 0.1976
ceb 0.0016 0.0016 0.0016
ces 1.1646 1.1458 1.1277
chr 0.0000 0.0000 0.0000
cos 0.0029 0.0029 0.0029
crs 0.0001 0.0001 0.0001
cym 0.0107 0.0111 0.0116
dan 0.5273 0.5240 0.5233
deu 5.9862 5.9083 5.7252
div 0.0026 0.0025 0.0024
dzo 0.0001 0.0001 0.0001
ell 0.5595 0.5490 0.5190
eng 40.5782 40.4526 41.8620
epo 0.0073 0.0082 0.0088
est 0.1395 0.1466 0.1447
eus 0.0378 0.0374 0.0383
fao 0.0055 0.0053 0.0058
fas 0.7039 0.7603 0.6934
fij 0.0001 0.0001 0.0001
fin 0.3841 0.3823 0.3723
fra 4.8032 4.8476 4.6769
fry 0.0023 0.0020 0.0023
gla 0.0017 0.0019 0.0018
gle 0.0063 0.0069 0.0071
glg 0.0323 0.0339 0.0315
glv 0.0008 0.0008 0.0007
got 0.0000 0.0000 0.0000
grn 0.0008 0.0010 0.0010
guj 0.0142 0.0127 0.0120
hat 0.0028 0.0028 0.0027
hau 0.0026 0.0034 0.0030
haw 0.0007 0.0008 0.0009
heb 0.2515 0.2666 0.2567
hin 0.2223 0.2144 0.2147
hmn 0.0007 0.0010 0.0009
hrv 0.2230 0.2219 0.2124
hun 0.5271 0.5341 0.5253
hye 0.0339 0.0320 0.0315
ibo 0.0012 0.0014 0.0013
iku 0.0005 0.0004 0.0004
ile 0.0003 0.0003 0.0003
ina 0.0007 0.0007 0.0007
ind 1.0851 1.1182 1.0945
ipk 0.0001 0.0001 0.0001
isl 0.0338 0.0346 0.0351
ita 2.4867 2.4624 2.4622
jav 0.0029 0.0031 0.0032
jpn 5.3160 5.2484 5.1668
kal 0.0009 0.0009 0.0009
kan 0.0160 0.0151 0.0145
kas 0.0000 0.0000 0.0000
kat 0.0448 0.0464 0.0457
kaz 0.0469 0.0458 0.0428
kha 0.0004 0.0004 0.0004
khm 0.0119 0.0110 0.0107
kin 0.0028 0.0029 0.0029
kir 0.0127 0.0116 0.0099
kor 0.8429 0.8500 0.8396
kur 0.0064 0.0071 0.0069
lao 0.0044 0.0061 0.0055
lat 0.1035 0.1040 0.1052
lav 0.0902 0.0911 0.0886
lif 0.0000 0.0000 0.0000
lin 0.0006 0.0005 0.0006
lit 0.1732 0.1756 0.1700
ltz 0.0038 0.0038 0.0037
lug 0.0004 0.0004 0.0004
mal 0.0236 0.0238 0.0220
mar 0.0300 0.0267 0.0264
mfe 0.0002 0.0002 0.0002
mkd 0.0376 0.0396 0.0394
mlg 0.0034 0.0041 0.0036
mlt 0.0032 0.0032 0.0032
mon 0.0155 0.0162 0.0159
mri 0.0018 0.0014 0.0013
msa 0.0897 0.0860 0.0855
mya 0.0167 0.0164 0.0165
nau 0.0000 0.0000 0.0000
nep 0.0611 0.0596 0.0534
nld 1.8179 1.7762 1.7381
nno 0.0140 0.0142 0.0141
nor 0.3372 0.3313 0.3358
nso 0.0001 0.0001 0.0001
nya 0.0008 0.0009 0.0009
oci 0.0035 0.0033 0.0033
ori 0.0073 0.0071 0.0066
orm 0.0006 0.0007 0.0007
pan 0.0085 0.0089 0.0091
pol 2.0439 2.0638 2.0329
por 2.5152 2.5263 2.4899
pus 0.0039 0.0043 0.0045
que 0.0009 0.0009 0.0009
roh 0.0014 0.0014 0.0013
ron 0.5695 0.5926 0.5730
run 0.0001 0.0002 0.0001
rus 6.8217 6.8659 6.6954
sag 0.0004 0.0005 0.0005
san 0.0034 0.0034 0.0033
sco 0.0007 0.0006 0.0007
sin 0.0094 0.0089 0.0085
slk 0.4074 0.4131 0.3951
slv 0.1431 0.1553 0.1523
smo 0.0007 0.0008 0.0007
sna 0.0011 0.0011 0.0011
snd 0.0023 0.0023 0.0021
som 0.0055 0.0056 0.0056
sot 0.0007 0.0007 0.0007
spa 4.6418 4.6257 4.7448
sqi 0.0498 0.0502 0.0481
srp 0.2396 0.2671 0.2577
ssw 0.0001 0.0000 0.0000
sun 0.0018 0.0014 0.0016
sux 0.0000 0.0000 0.0000
swa 0.0117 0.0117 0.0109
swe 0.7361 0.7245 0.7196
syr 0.0004 0.0004 0.0003
tam 0.0488 0.0449 0.0449
tat 0.0075 0.0075 0.0077
tel 0.0200 0.0199 0.0191
tgk 0.0060 0.0072 0.0065
tgl 0.0114 0.0121 0.0121
tha 0.3697 0.3740 0.3577
tir 0.0005 0.0011 0.0007
ton 0.0001 0.0001 0.0001
tsn 0.0002 0.0003 0.0004
tso 0.0001 0.0001 0.0001
tuk 0.0032 0.0033 0.0030
tur 1.3455 1.3598 1.2777
uig 0.0029 0.0018 0.0016
ukr 0.7955 0.8371 0.8138
urd 0.0330 0.0322 0.0308
uzb 0.0312 0.0313 0.0311
ven 0.0000 0.0000 0.0000
vie 1.0586 1.0750 0.9927
vol 0.0004 0.0012 0.0008
war 0.0017 0.0016 0.0014
wol 0.0002 0.0002 0.0002
xho 0.0009 0.0010 0.0011
yid 0.0016 0.0017 0.0017
yor 0.0016 0.0017 0.0017
zha 0.0001 0.0001 0.0001
zho 4.4292 4.3829 4.2255
zul 0.0015 0.0017 0.0018