{"id":37690,"date":"2007-12-20T08:36:21","date_gmt":"2007-12-20T08:36:21","guid":{"rendered":"https:\/\/www.viafirma.com\/detect-at-a-low-level-if-a-file-is-utf-8-from-java\/"},"modified":"2026-01-23T12:05:31","modified_gmt":"2026-01-23T12:05:31","slug":"detect-at-a-low-level-if-a-file-is-utf-8-from-java","status":"publish","type":"post","link":"https:\/\/www.viafirma.com\/en\/detect-at-a-low-level-if-a-file-is-utf-8-from-java\/","title":{"rendered":"Detect at a low level if a file is UTF-8 from Java"},"content":{"rendered":"

Although initially it seems a very simple task, those who have had to detect if a file is UTF-8 will have realized that the subject is not as obvious as it seems.<\/p>\n

First a theoretical introduction<\/strong><\/p>\n

The files are stored as arrays of bytes that are later associated to characters, to make this association different encodings are used (ASCII, ISO-8859-1, UTF-8, etc.).<\/p>\n

In order to establish a relationship between its code and any character used by any language in the world, Unicode was created, which is nothing more than a gigantic numeric-graphic code association to allow its computer representation.<\/p>\n

In this context, UTF-8 is simply a way of encoding Unicode text to allow it to be serialized in files or data streams.<\/p>\n

Since Unicode tries to associate codes to all essential characters, we need more than one byte to encode them, so UTF-8 uses a variable structure of 1 to 4 bytes to encode the different characters.<\/p>\n

This variable size is the reason why sometimes files saved in one format are displayed with strange characters when retrieved using the wrong encoding.<\/p>\n

Algorithm Approach<\/strong><\/p>\n

The process is very simple, just read the file byte by byte and check that all bytes comply with the UTF-8 standard.<\/p>\n