Character Encoding of HTML Pages

The character set or encoding of HTML pages only is identified by Tika’s AutoDetectReader. The table shows the percentage how character sets have been used to encode HTML pages crawled by the latest monthly crawls. The underlying data including page counts is provided in charsets.csv.

crawl	CC-MAIN-2026-17	CC-MAIN-2026-21	CC-MAIN-2026-25
charset
<other>	0.0001	0.0001	0.0000
<unknown>	1.8918	2.0799	1.9748
Big5	0.0298	0.0221	0.0209
Big5-HKSCS	0.0000	0.0000	0.0001
EUC-JP	0.1266	0.1283	0.1248
EUC-KR	0.0771	0.0757	0.0746
GB18030	0.0150	0.0120	0.0104
GB2312	0.2011	0.1915	0.1750
GBK	0.0934	0.0905	0.0867
IBM420	0.0051	0.0052	0.0053
IBM424	0.0011	0.0009	0.0009
IBM500	0.0008	0.0008	0.0007
IBM855	NaN	0.0000	0.0000
IBM866	0.0001	0.0001	0.0001
ISO-2022-JP	0.0010	0.0011	0.0012
ISO-8859-1	2.7125	2.8610	2.6514
ISO-8859-13	0.0000	0.0000	0.0000
ISO-8859-15	0.0444	0.0446	0.0377
ISO-8859-16	0.0003	0.0003	0.0004
ISO-8859-2	0.0817	0.0794	0.0766
ISO-8859-3	0.0003	0.0003	0.0003
ISO-8859-4	0.0007	0.0007	0.0007
ISO-8859-5	0.0008	0.0007	0.0006
ISO-8859-6	0.0000	0.0000	0.0000
ISO-8859-7	0.0040	0.0038	0.0041
ISO-8859-8	0.0007	0.0006	0.0008
ISO-8859-9	0.0235	0.0220	0.0211
KOI8-R	0.0076	0.0076	0.0075
KOI8-U	0.0001	0.0001	0.0001
Shift_JIS	0.1479	0.1567	0.1502
TIS-620	0.0036	0.0031	0.0031
US-ASCII	0.0203	0.0241	0.0257
UTF-16	0.0022	0.0020	0.0017
UTF-16BE	0.0003	0.0003	0.0003
UTF-16LE	0.0039	0.0036	0.0035
UTF-32	0.0000	0.0001	0.0001
UTF-32LE	0.0002	0.0001	0.0002
UTF-8	93.7434	93.3986	93.7782
windows-1250	0.0666	0.0698	0.0677
windows-1251	0.4838	0.4885	0.4864
windows-1252	0.1526	0.1684	0.1554
windows-1253	0.0024	0.0025	0.0021
windows-1254	0.0123	0.0117	0.0111
windows-1255	0.0053	0.0059	0.0058
windows-1256	0.0240	0.0241	0.0223
windows-1257	0.0053	0.0051	0.0034
windows-31j	0.0004	0.0002	0.0002
x-iso-8859-11	0.0000	0.0000	0.0000
x-windows-874	0.0057	0.0059	0.0057
x-windows-949	0.0000	0.0000	0.0000