SEO History

Web masters (usually website owners) and content providers began optimizing sites for search engines in the mid-1990s, at first this merely meant you built a website, checked it for spelling errors and then submit it to search engines to be cataloged. This was the early Web. Once a web master had submitted a page, or URL, to the various engines they would send a spider to "crawl" that page, extract links to other pages from it, and return information found on the page to be indexed. The process has evolved so that now a search engine spider downloads a target page and stores it on the search engine's own server, where a second program, known as an indexer, breaks down the information, extracts various elements that make up the page, such as the words it contains, the coding information, META tags, description, title, keywords, link text, and where these are located, as well as any weight for specific words and all links the page contains, which are then placed into a scheduler for crawling at a later date. It then gives each element a value according to the search engine algorithm to determine the overall relevance of the page in regard to certain search terms.

Site owners started to recognize the value of having their sites highly ranked and visible in search engine results. As more users began to search for goods and services online, it was possible to track how the search results were used. It was easy to see that searchers would not often go through many pages of results to find what they wanted.

Early versions of search algorithms relied on web master-provided information such as the keyword meta tag, or index files in. Meta-tags provided a guide to each page's content. But using meta data to index pages was found to be less than reliable because the web master's account of keywords in the meta tag were not always relevant to the site's actual keywords. In short using the information provided by the web master’s to determine the search result was like using the fox to guard the hen house. Inaccurate, incomplete, and inconsistent or manipulated data in meta tags caused pages to rank for irrelevant searches. Irrelevant search results began to frustrate searchers. They began to look to other search engines to provide more relevant searches. The search engines that used an algorithm based on information that involved elements actually in the content on the page and how it related to the search.

While they were graduate students at Stanford University, Larry Page and Sergey Brin developed "backrub", a search engine that relied on a mathematical algorithm to rate the prominence of web pages. The number calculated by the algorithm, PageRank, is a function of the quantity and strength of inbound links. This means that some links are stronger than others, or give more weight to the PageRank because the contents of the page is more likely to be relevant to the searcher.

In 1998 the grad students founded Google. Google attracted a loyal following among Internet users, who liked the results their system produced. Off-page factors such as PageRank and hyper link analysis were considered, as well as on-page factors, to enable Google to avoid the kind of manipulation seen in search engines that only considered on-page factors for their rankings. Although PageRank was more complex, web masters went to work using link building schemes to manipulate search results to their favor. Web masters focused on exchanging, buying, and selling links, often on a massive scale. Some of these schemes, or link farms, involved the creation of thousands of sites for the sole purpose of link spamming.

To counter the adverse impact of link schemes, as of 2007, search engines again had to evolve to consider a wider range of undisclosed factors for their ranking algorithms. Google has since disclosed that it now uses more than 200 different elements to rank pages. The three leading search engines, Google, Yahoo and Microsoft's Live Search, closely guard the algorithms they use to rank pages.

What Does The Future Hold?