What Is Latent Semantic Indexing?

Have you ever wondered how Search Engines (SE) know which pages to give you when you do a search? For example when you search for "Apple" the computer, how does the SE know not to give you pages about Apple juice or Apple farming?

The solution that handles this is what is often referred to here as a SE algorithm. An algorithm is mathematical formula for solving complex problems. SE algo is the formula used by SEs to rank the pages contained in its index depending on the query. Years back when SE algos were still rudimentary they used, what in retrospect are very crude means to provide most relevant results to a query. Among them was the use of keyword density. This used frequency of a particular word or term to decide if it should rank higher or lower.

But unfortunately this does not represent complete reality. For example it was the assumption of the algo that if a page is about dieting, the word should be the most frequently used word apart from other grammatical necessities like me, I, and, the, or etc. But this is not always true. This is because in most writings, to avoid monotony there is a mix of words of similar meaning - synonyms. So a page about dieting may also include lose weight or weight loss.

Enter LSI; LSI or Latest Semantic Indexing is an informational retrieval system that depends on a technique used to process natural language into a mathematical form that the computer can understand and process.

The use of semantics is really central in LSI for SE. They are used to try to emulate what a human being would rank highly. Semantics refers to the aspect of meaning in a language. Its here we get terms like synonyms and polysems (i.e. the same word with different meanings e.g. Apple for the Macintosh computers and Apple for the fruit)

The question doing rounds in SE forums around the internet is whether or not Google is using LSI in their ranking. What is known for sure is that Google Inc. did buy Applied Semantics, a company that creates software applications based on LSI technology. We also know that LSI is in their systems. How do we know you ask? Just use the toogle key before your search query. The toogle key is the key often to the extreme left of the computer keyboard just below the escape key with the "~" sign. You will need to hold down the shift key.

If you type it before the term "apple" you will notice that in the resulting SERP are about computers with words like Mac, Macintosh, G4, Apple Computer bolded. If you toggle the plural i.e. "apples" you will notice that now you have pages on apple fruits and the word fruit is also bolded. The bolded words are what the SE also considers as synonyms to the search queries "apple" and apples" respectively.