Character Encoding of HTML Pages

The character set or encoding of HTML pages only is identified by Tika’s AutoDetectReader. The table shows the percentage how character sets have been used to encode HTML pages crawled by the latest monthly crawls. The underlying data including page counts is provided in charsets.csv.

crawl	CC-MAIN-2026-30	CC-MAIN-2026-34	CC-MAIN-2026-39
charset
<other>	0.0001	0.0001	0.0000
<unknown>	2.0062	2.1044	2.0219
Big5	0.0202	0.0257	0.0256
Big5-HKSCS	0.0000	0.0000	0.0000
EUC-JP	0.1231	0.1220	0.1184
EUC-KR	0.0731	0.0754	0.0709
GB18030	0.0089	0.0093	0.0085
GB2312	0.1643	0.1684	0.1603
GBK	0.0853	0.0870	0.0823
IBM420	0.0046	0.0050	0.0047
IBM424	0.0012	0.0013	0.0013
IBM500	0.0005	0.0006	0.0006
IBM855	0.0000	0.0000	0.0000
IBM866	0.0001	0.0001	0.0001
ISO-2022-JP	0.0012	0.0011	0.0010
ISO-8859-1	2.6022	2.4618	2.4286
ISO-8859-13	0.0000	0.0001	0.0000
ISO-8859-15	0.0402	0.0411	0.0350
ISO-8859-16	0.0003	0.0003	0.0001
ISO-8859-2	0.0747	0.0769	0.0694
ISO-8859-3	0.0003	0.0003	0.0000
ISO-8859-4	0.0006	0.0007	0.0007
ISO-8859-5	0.0006	0.0005	0.0005
ISO-8859-6	0.0000	0.0000	0.0000
ISO-8859-7	0.0039	0.0040	0.0037
ISO-8859-8	0.0007	0.0007	0.0006
ISO-8859-9	0.0216	0.0233	0.0214
KOI8-R	0.0075	0.0081	0.0077
KOI8-U	0.0001	0.0001	0.0001
Shift_JIS	0.1478	0.1440	0.1325
TIS-620	0.0031	0.0029	0.0026
US-ASCII	0.0264	0.0241	0.0223
UTF-16	0.0016	0.0016	0.0015
UTF-16BE	0.0003	0.0002	0.0002
UTF-16LE	0.0054	0.0040	0.0028
UTF-32	0.0001	0.0001	0.0001
UTF-32LE	0.0002	0.0002	0.0002
UTF-8	93.8283	93.8314	94.0548
windows-1250	0.0663	0.0651	0.0600
windows-1251	0.4793	0.4998	0.4641
windows-1252	0.1502	0.1605	0.1485
windows-1253	0.0019	0.0020	0.0020
windows-1254	0.0111	0.0107	0.0109
windows-1255	0.0053	0.0051	0.0048
windows-1256	0.0215	0.0203	0.0202
windows-1257	0.0035	0.0034	0.0034
windows-31j	0.0003	0.0002	0.0002
x-iso-8859-11	0.0000	0.0000	0.0000
x-windows-874	0.0061	0.0057	0.0055
x-windows-949	0.0000	0.0000	0.0000