Searching the Web

Lesson 1: Tool Type Primer

Search Engines and Metasearch Engines

What is a Search Engine?

A search engine permits you to search its (generally enormous) index of web pages by specifying one or more keywords that relate to your topic. More often than not, a search engine home page contains nothing more than an empty edit box into which you type your query.

• See Figure 4.6 on page 71 of The Extreme Searcher’s Internet Handbook for an example of the AllTheWeb search engine home page.
• See Figure 4.11 on page 87 of The Extreme Searcher’s Internet Handbook for an example of the Google search engine home page.
• See Figure 4.15 on page 99 of The Extreme Searcher’s Internet Handbook for an example of the HotBot search engine home page.

Unlike directories, which are usually organized by people, search engines gather their huge collection of web pages using computers. These computers are commonly referred to as spiders, ‘bots (short for robots), crawlers, or indexers. A spider goes about its job methodically seeking out web pages and collecting the information it wants. What the spider looks for varies from tool to tool and can include everything from just the title, description, and/or keywords associated with a page to every word the page contains.

Spiders are much more democratic in their collection of data than a person would be, because they are completely non-judgmental. While this machine-based approach results in equal access to all of the information within its reach, it should also be noted that a spider would not be able to tell poetry from pornography, or the difference between a spoof and a sincere report on a news event. It’s also important to note that a spider can only recognize text when visiting a web page. So if you were searching for a schematic drawing of a Piper J-3 Cub jet engine, you would only be able to find it if the page(s) containing the drawing also included text labeling it as such. Without the text, a spider would otherwise ignore that page.

How Does a Search Engine Differ from a Directory?

While a directory is designed for browsing and meandering through a hierarchy of topics, a search engine is designed for quickly cutting to the chase when it comes to finding very specific information.

When Should I Use a Search Engine?

Use a search engine when

• You have a fairly specific topic in mind, such as
  • Historical information on Piper J-3 Cub
  airplanes
  • Pictures of the “Bridge of Sighs” in Venice, Italy
• You want to use special search techniques, such as
• Phrase searches (searching on a connected group of words)
• Boolean logic searches (using AND, OR, and NOT operators to narrow, broaden, or limit a search)
• Field searches (constraining a search to specific HTML fields)
• Natural language searches (where you can ask a simple question such as “Who won the Academy Award for Best Actor in 1974?”)

NOTE: In Lesson 3 you will learn how to use each of the special search techniques mentioned above.

What is a Metasearch Engine?

Metasearch engine tools do not harvest and maintain their own index of web pages; instead they allow you to use their interface to perform keyword searches at a variety of other search tools simultaneously.

• Visit http://www.dogpile.com to see the Dogpile metasearch engine home page.
• Visit http://www.metacrawler.com to see the Metacrawler metasearch engine home page.

When Should I Use a Metasearch Engine?

A metasearch engine is useful when you want to get a quick overview of what the web has to offer on a fairly specific topic. Since no single search tool includes every single resource available on the Internet, you usually will need to do your research at multiple sites to get a full measure of the information available on your topic. A metasearch engine lets you do that from a single page instead of having to visit multiple tools.

Gimme More!

Lesson 3 introduces you to a variety of search engines and metasearch engines.

  1  2  3  4  5