Character Encoding in XML and JSON Conversion

Character encoding is handled differently in XML and JSON, and mismatches can cause data corruption during conversion.

XML encoding flexibility:

XML supports multiple character encodings, declared in the XML declaration:

<?xml version="1.0" encoding="UTF-8"?>
<?xml version="1.0" encoding="UTF-16"?>
<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml version="1.0" encoding="Shift_JIS"?>

JSON encoding is always UTF-8. Per RFC 8259, JSON text exchanged between systems must be encoded in UTF-8. No encoding declaration is needed or possible.

XML character references:

XML can represent characters using numeric references:

<text>Caf&#233; &#x2014; a nice place</text>
<!-- &#233; = e with acute (decimal) -->
<!-- &#x2014; = em dash (hexadecimal) -->

In JSON, these become their actual Unicode characters:

{ "text": "Cafe\u0301 \u2014 a nice place" }

Or, since JSON supports UTF-8 directly:

{ "text": "Caf\u00e9 \u2014 a nice place" }

XML predefined entities:

XML has five predefined entity references that must be escaped in element content and attribute values:

Entity	Character	JSON equivalent
&lt;	<	< (direct)
&gt;	>	> (direct)
&amp;	&	& (direct)
&quot;	"	\" (escaped)
&apos;	'	' (direct)

Conversion considerations:

1. XML to JSON: Decode all entity references and character references to their actual characters. The JSON output should contain the real characters, not the XML escape sequences.
2. JSON to XML: Escape special characters using either entity references or CDATA sections. At minimum, <, >, and & must be escaped in text content.
3. Non-BMP characters (emoji, CJK extensions) are valid in both formats but may need surrogate pairs in JSON (\uD83D\uDE00 for a smiley) or numeric references in XML (😀).

Best practice: Always normalize to UTF-8 during conversion. If the source XML uses a non-UTF-8 encoding, transcode the bytes to UTF-8 before parsing.