How does an analyzer prepare text for search?

Question

IT Interview Practice · Accepted Answer

An analyzer prepares text for searching by turning it into tokens: units, often words, that can be matched later. ## Watch each transformation For a deliberately configured HTML-stripping, word-splitting, lowercasing pipeline: ```text Blue Shoes -> Blue Shoes -> [Blue, Shoes] -> [blue, shoes] ``` - A character filter removes the HTML before splitting. - A tokenizer splits the text into words. - A token filter lowercases those words. The final tokens explain why lowercase `shoes` can match the original input. This is a custom example; not every analyzer removes HTML. ## Inspect instead of guessing In Kibana Dev Tools, this request shows the tokens for that pipeline: ```http POST /_analyze { "char_filter": ["html_strip"], "tokenizer": "standard", "filter": ["lowercase"], "text": "Blue Shoes" } ``` Look for `blue` and `shoes` in the returned token entries. Then test real accents, languages and product codes. Index-time analysis prepares stored terms; search-time analysis prepares user queries. They need compatible terms, even if the two pipelines differ. [Official documentation](https://www.elastic.co/docs/manage-data/data-store/text-analysis).

Watch each transformation

Inspect instead of guessing

Related questions