Below are the Top few reasons we've compiled over the years as to why you may not be finding your Web site in one or more search engines:
1. INDEX TIME: First, make sure you've allowed enough time to become indexed. The amount of time to allow is sometimes listed on the search engine's submission page. Average index times often range from one to eight weeks depending on the engine. Some engines like AltaVista and Inktomi offer paid options if you wish to be indexed more quickly.
2. ALREADY INDEXED: Be sure you're not already indexed but just don't know it. Unfortunately, none of the major engines are kind enough to e-mail or notify you as to if and when you've been indexed.
In addition, you cannot simply do a search on a keyword that applies to your Web site and expect it to pop up at the top. In fact, you must take pro-active steps to optimize your pages for each search engine. If you don't, it's very unlikely you'll find your Web site except on the most specific of searches.
3. MISSING PAGE: Make sure you have uploaded the pages to your site before submitting them. This one will seem obvious to many people, but submitting a page that does not exist or submitting with a subtle typo in the URL is a goof we might all make at one time or another. This is important since not all search engines will notify you if the URL does not exist when you submit.
4. ROADMAP FROM HOME PAGE: Some engines have been known to drop pages that cannot be traveled to from the home page. Think of your Web site as a series of roads (i.e., links) from one page to another. If there's no road from your home page to the page you want indexed, a search engine may decide the page is unimportant or of low-quality. You could submit the page directly, but the engine may reject it or may drop it at a later date when it finds no "road."
5. EXTERNAL LINKS: Some search engines such as Google and HotBot have been known to reject to index Web sites that do not have any other Web sites linking to them. Or, they may index your home page but reject to index any other pages awaiting you achieve at least one or more links from another domain. Or, they may index you for awhile but then "prune" their database later of all Web sites that did not achieve any external links within a certain period of time. However, do not worry! You simply need to establish some links and when that's done, resubmit both your pages and the pages that link to you. Once you have links to your Web site, it becomes much easier to get indexed, stay indexed, and to achieve top rankings.
6. FRAMES: Content inside HTML frames, this can cause problems with submissions. For example, the search engine may index the main content of the page, but not the surrounding menu frame. Visitors to your site will then find some information but may not see the associated menu! It's generally best if you can create non-framed versions of your pages.
7. SPIDER BLOCKS: Search engine spiders cannot index sites that require any kind of registration or password. A spider cannot fill out a form of any kind.
8. SUBMISSION LIMITS: Make sure you're submitting within the recommended limits. Some engines do not like more than a certain number of submissions per day for the same domain.
9. DYNAMIC PAGES: Dynamic pages are often ignored by the search engine spiders. In fact, any URL containing special symbols like a question mark (?) or an ampersand (&) will be ignored by many engines. Pages generated on the fly from a database often contain these symbols. In this situation, it's important to generate "static" versions of each page you wish to be indexed. In regard to the search engines, the simpler the page is, the better. Does this mean, for example, having a javascript to count visits to the page will prevent you from being indexed, or lower your rankings? No. It simply means that the search engine will most likely ignore the javascript and index the remaining areas of the page.
10. NON-INDEXABLE CONTENT: It's important to know the types of content that the average search engine cannot index. Most engines cannot index text that is embedded in images. Text that appears in multimedia files (audio and video) will not be indexed. Most engines cannot index information that is generated by Java applets, Flash files or in XML coding.
LARGE PAGES: If your site has a slow connection or the pages are very complex and take a long time to load,
DEEP LINKS: If you submit just your home page, don't expect a search engine to travel more than one or two links away from the home page or from the page that you submitted.
UNRELIABLE HOSTS: If your Web site fails to respond when the search engine spider pays a visit, you will not be indexed. Even worse, if you are indexed and they pay a visit when your site is down, you could be removed from their database! Consequently, it pays to have a reliable hosting service that is up 99.5% of the time.
SPAM: If you have ever used any questionable techniques that might be considered an overt attempt at spamming
REDIRECTS: If your site contains redirects or meta refresh tags these things can sometimes cause the engines to have trouble indexing your site
PROPER DIRECTORY SUBMISSIONS: If you're submitting to a directory site like Yahoo, Open Directory, Looksmart, or others, then a human being will review your site. They must decide if the site is of sufficient "quality" before they will list it.
PAGE LIMITS: If you have many pages indexed but are having a hard time getting new ones recognized, be aware that there are limits. Each search engine will only spider so many pages of your Web site. This may range from a few dozen or three or four hundred depending on the engine. Google is one engine that tends to crawl deeper into your site. However, how deep they go may depend on factors like your link popularity.
