Clean Hidden Characters from Copy-Pasted Text
Remove zero-width spaces, non-breaking spaces, BOMs, and other hidden characters that get inserted when copying text from web pages and documents.
Detailed Explanation
The Copy-Paste Problem
Copying text from web pages, PDFs, word processors, and messaging apps frequently introduces invisible characters that are not present in the original visible text. These hidden characters can cause subtle bugs that are extremely difficult to diagnose because the text looks correct in every editor and terminal.
Common Hidden Characters in Pasted Text
| Character | Source | Effect |
|---|---|---|
| NBSP (U+00A0) | Web pages, Word docs | Breaks string comparisons and parsers |
| ZWS (U+200B) | Web pages, rich text | Breaks URLs, variable names, regexes |
| ZWJ (U+200D) | Emoji sequences, Arabic text | Unexpected string length |
| ZWNJ (U+200C) | Persian/Arabic text | Breaks token matching |
| SHY (U+00AD) | Word processors | Invisible hyphens in wrong places |
| BOM (U+FEFF) | Notepad (Windows), some editors | Breaks shell scripts, JSON |
Step-by-Step Cleaning Process
- Paste your suspicious text into the Whitespace Visualizer input panel.
- Review the visualization output — hidden characters appear as color-coded markers like [ZWS], °, [BOM], [SHY].
- Check statistics to see exactly how many of each type are present.
- Select which character types to remove in the Clean section.
- Click Clean to strip them, then Copy Cleaned to get the sanitized text.
Prevention Tips
When working with text from external sources, always run it through the visualizer before using it in code or configuration. The String Escape/Unescape tool can also help you see the exact character sequences in a string, while the Word & Character Counter can reveal unexpected character count differences.
Use Case
A data engineer pastes API documentation into a YAML config file and the parser fails with a cryptic error. The Whitespace Visualizer reveals three zero-width spaces and a BOM character that were copied from the Confluence wiki page. After cleaning, the YAML parses correctly.