Search Engine 101: What They Are and How They Work

Knowing a bit about what Search Engines are and how they work is important to any internet user. However, this information is especially important to any website owner/manager. Having at least a basic understanding will help you develop and/or improve your overall webmarketing strategy for maximum effectiveness and to help you maximize your return on your website development and maintenance investment.

Search Engine History & Overview:

WebCrawler, which was born in 1994, is considered to be the first true search engine. It was an automated ''crawler based'' full text search engine. In 2000, Google was released and rapidly gained popularity because it delivered very relevant results to queries (search keyword or keyphrase). Google is by far the most popular search engine, and has been for the past several years. Yahoo! and Microsoft's Bing are the others that make up the "Big 3", although in 2009 There are also hundreds of other "major" search engines, like "Ask.com", and thousands of ''minor'' ones. Many search engines are "powered" by others (in 2009 Yahoo! began to use Bing technology), so the results can be similar, sometimes almost identical.

Search Engine vs Directory:
There are also online Website Directories (think internet phone book). The terms are often incorrectly used interchangeably, but ''human run'' directories (such as www.Dmoz.org) are very different from search engines, which are "robot run".

How Search Engines Work:
There are three primary pieces of software involved: the spider software, the index software and the query software.
SPIDER: Search engines automatically create web site listings by using "spider robots" (software programs) that "crawl" web pages, index their information, Spiders, like GoogleBot, usually return to already-crawled sites on a regular basis in order to check for updates or changes.
INDEX: Everything that they find goes into the search engine database. The engines use complex (and secret) algorithms (mathematical formulas), to help determine what goes where in the database. Think of the file cabinet in your office. The better you evaluate, label and sort (alpha, numeric, etc.) the faster you can find stored information you need when you need it later.
QUERY: When a user searches (submits a query), the search engines evaluates the query, checks it's database, and then using it's algorithms, provides the user with what it considers a list of "best match" results that it ranks by relevancy to the word or phrase in the query. An analogy would be you going to a library, telling the librarian you need a book on "cars", and the librarian retrieves and presents you with a some books. Since your question was vague and general, you might get a cart full of books! The more specific your question ("history of BMW" for example), the better the librarian can help you quickly find exactly what you need - i.e., "we have 1,251 books on cars, but here are 3 that seem best matched to your specific question".


SERPS (Search Engine Results Pages) and Different Types of Listings:After a query, a user is presented a two different types of results - Organic & Paid.
Organic/Natural Search Results, are listings on SERPS that appear because of their relevance to the search terms, as opposed to paid advertisements. Most search engines will present "pay per click" ads on the column on the right, often with a few at the top center (above the natural results). Those in the premium position (top center) get there based on highest cost per click paid and frequency of clicks. Much research has been done regarding this topic, but it's my professional opinion that most users, most of the time will first click organic search results before they click on the paid listings.


Why Should You Know This AND Why Should You Care?
As a website owner or manager, you know how important it is for prospective customers to be able to easily find your website when they search for your products and services.  Now that you know what Search Engines are and how they work, you can help create or modify the organization of your website content, and create or modify specific page text to best the needs of the search engines.  The spider robots need a "search engine friendly" website (easy to crawl & easy to index) that's filled with keyword rich text.

LEARN MORE: To learn more about Search Engines and SEO, use the "blog search box" (upper left) to find other posts in this Cole WebMarketing blog, using keywords/keyphrases such as Search Engine, Online Directory, Google, Yahoo, Bing, SEO, Search Engine Optimization, etc.. 

For more information about this blog post topic, please visit CarolinaWebsiteDesign.com and then call Cole WebMarketing at 704-503-7069.