Character Encoding of HTML Pages

The character set or encoding of HTML pages only is identified by Tika’s AutoDetectReader. The table shows the percentage how character sets have been used to encode HTML pages crawled by the latest monthly crawls. The underlying data including page counts is provided in charsets.csv.

crawl	CC-MAIN-2026-34	CC-MAIN-2026-39	CC-MAIN-2026-43
charset
<other>	0.0001	0.0000	0.0000
<unknown>	2.1044	2.0219	2.0119
Big5	0.0257	0.0256	0.0245
Big5-HKSCS	0.0000	0.0000	0.0000
EUC-JP	0.1220	0.1184	0.1210
EUC-KR	0.0754	0.0709	0.0733
GB18030	0.0093	0.0085	0.0091
GB2312	0.1684	0.1603	0.1673
GBK	0.0870	0.0823	0.0851
IBM420	0.0050	0.0047	0.0050
IBM424	0.0013	0.0013	0.0007
IBM500	0.0006	0.0006	0.0007
IBM855	0.0000	0.0000	0.0000
IBM866	0.0001	0.0001	0.0001
ISO-2022-JP	0.0011	0.0010	0.0010
ISO-8859-1	2.4618	2.4286	2.4774
ISO-8859-13	0.0001	0.0000	0.0000
ISO-8859-15	0.0411	0.0350	0.0365
ISO-8859-16	0.0003	0.0001	0.0001
ISO-8859-2	0.0769	0.0694	0.0631
ISO-8859-3	0.0003	0.0000	0.0002
ISO-8859-4	0.0007	0.0007	0.0006
ISO-8859-5	0.0005	0.0005	0.0005
ISO-8859-6	0.0000	0.0000	0.0000
ISO-8859-7	0.0040	0.0037	0.0032
ISO-8859-8	0.0007	0.0006	0.0006
ISO-8859-9	0.0233	0.0214	0.0203
KOI8-R	0.0081	0.0077	0.0065
KOI8-U	0.0001	0.0001	0.0002
Shift_JIS	0.1440	0.1325	0.1311
TIS-620	0.0029	0.0026	0.0029
US-ASCII	0.0241	0.0223	0.0219
UTF-16	0.0016	0.0015	0.0015
UTF-16BE	0.0002	0.0002	0.0002
UTF-16LE	0.0040	0.0028	0.0024
UTF-32	0.0001	0.0001	0.0001
UTF-32LE	0.0002	0.0002	0.0001
UTF-8	93.8314	94.0548	94.0547
windows-1250	0.0651	0.0600	0.0617
windows-1251	0.4998	0.4641	0.4245
windows-1252	0.1605	0.1485	0.1441
windows-1253	0.0020	0.0020	0.0020
windows-1254	0.0107	0.0109	0.0096
windows-1255	0.0051	0.0048	0.0045
windows-1256	0.0203	0.0202	0.0194
windows-1257	0.0034	0.0034	0.0044
windows-31j	0.0002	0.0002	0.0003
x-iso-8859-11	0.0000	0.0000	0.0000
x-windows-874	0.0057	0.0055	0.0055
x-windows-949	0.0000	0.0000	0.0000