Web Development

  Homes arrow Web Development arrow Submitting to the Spidering Search Engines
 Webmaster Tools
Base64 Encoding 
Browser Settings 
CSS Coder 
CSS Navigation Menu 
Datetime Converter 
DHTML Tooltip 
Dig Utility 
DNS Utility 
Dropdown Menu 
Fetch Content 
Fetch Header 
Floating Layer 
htaccess Generator 
HTML Encoder 
HTML Entities 
IP Convert 
Meta Tags 
Password Encryption
Password Strength
Pattern Extractor 
Ping Utility 
Pop-Up Window 
Regex Extractor 
Regex Match 
Scrollbar Color 
Source Viewer 
Syntax Highlighting 
URL Encoding 
Web Safe Colors 
Forums Sitemap 
Weekly Newsletter
Developer Updates  
Free Website Content 
 RSS  Articles
 RSS  Forums
 RSS  All Feeds
Write For Us 
Contact Us 
Site Map 
Privacy Policy 
  >>> SIGN UP!  
  Lost Password? 

Submitting to the Spidering Search Engines
By: Developer Shed
  • Search For More Articles!
  • Disclaimer
  • Author Terms
  • Rating:  stars stars stars stars stars / 0

    Table of Contents:

    Rate this Article: Poor Best 
      Del.ici.ous Digg
      Blink Simpy
      Google Spurl
      Y! MyWeb Furl
    Email Me Similar Content When Posted
    Add Developer Shed Article Feed To Your Site
    Email Article To Friend
    Print Version Of Article
    PDF Version Of Article




    Submitting to the Spidering Search Engines

    From: Metamorphozis.com
    By: Jill Whalen

    I'm often asked how often one should submit or resubmit their site to the spidering search engines. Many people believe they should submit their site on a periodic basis to keep their site in the search engines' "faces." Submitting URLs that are already in a search engine's database, which hasn't had any significant content changes, is not a good idea.

    A general rule of thumb for submitting and resubmitting pages is to first check each engine to see if the new page has been indexed. There is no reason to resubmit a page to an engine if the updated page is already in the database.

    Submitting to Google

    Google is the best engine for spidering through sites. It's only necessary to add your main URL and Google will eventually spider the rest of your site. You can check if Google has indexed your site by typing your URL into their search box. They currently take about one-to-two months to index and reindex pages. Representatives from Google have warned that they prefer to find sites and pages on their own, rather than you submitting them through their form. However, they also state that there is no penalty for submitting to them through their form. Google has no immediate plans to start a pay-for-inclusion program. Google's Add URL form can be found here.

    Submitting to Hotbot/Inktomi

    At HotBot, you can type www.yoursite.com into their search box and see if it shows up. Since HotBot uses the Inktomi database for its search results, you can simply wait for the Inktomi spider to find your site. If you've been listed in the LookSmart directory, the Inktomi spider should easily find you in no time. You can also submit to them through any Inktomi partner site. As with most of the search engines, HotBot/Inktomi does not guarantee that they will list all sites that are submitted to them for free. For guaranteed inclusion in their database, you might want to use the Inktomi Paid Inclusion Program. Basically, Inktomi has partnered with some submission companies that, for a fee, guarantee the Inktomi spider will crawl your submitted pages every 48 hours. This will also keep your URL in their database for a year. You won't get any boost in rankings, but you'll have peace of mind knowing your site is listed.Submitting to AltaVista

    AltaVista used to be one the quickest to index new pages. However, those days are long gone, unless you use their paid inclusion program. In order to keep the amount of spam submittals in check, AV has instituted an interesting submittal procedure. When you submit a URL to AV's free "Add-URL form," you'll see a graphical image of some strange- looking letters. You are instructed to type those letters into a box in order to continue the submission. Once you do that, you can then submit up to five URLs. I think it's a neat idea, to thwart automatic submission programs. Since I hand submit anyway, it's not a problem for me. As long as it means that my submissions get added in a relatively short time period, I'm all for it!

    To see if your pages are in AltaVista, type in "URL:www.yoursite.com" into their search box (substituting your own domain name for yoursite.com). Your indexed pages will show up. Compare the title and description AV lists, with the Title tag and Meta description tag on your actual pages. If AV shows your old title and/or description, then they haven't yet indexed your new page. If they show your new stuff, then you're all set! Links to AltaVista's various submission programs can be viewed here.

    Submitting to Lycos/FAST

    Sites have been getting added fairly quickly and flawlessly as of late to Lycos. You can usually type your URL into their search box to see if it's in their database. Lycos has partnered with FAST, whose spider crawls through pages on a regular basis. Very often FAST will find your inner pages without you needing to submit them at all. Lycos/FAST also recently instituted a new pay-for-inclusion program called, InSite Select which will guarantee your pages regular 48 hour spidering. Lycos' add URL programs can be reached here.

    Excite, Webcrawler and GO are Dead

    Excite and its sister engine, Webcrawler, along with Disney's Go (formerly Infoseek) are all basically dead. They currently use Overture pay-per-click ad links, and have no database of their own.

    Patience is a Virtue

    Once you've submitted your URL to the engines, the next thing for you to do is practice the fine art of P-A-T-I-E-N-C-E!!! Unless you've paid for inclusion, your best bet is to submit and then forget about it for awhile. Wait for three months, then do a reality check of all the engines as noted above. I know that three months is probably too long for most of you to wait, but if you don't like to be driven crazy, it's a good strategy. I admit that I do check at least once a month after submission, but for the average Web site owner, waiting three months is a smart idea. After three months, you'll probably be pleasantly surprised to find that your site has been indexed by all of the search engines. And better yet…if you optimized them correctly they should be bringing in tons of highly targeted traffic!

    Contact Jill Whalen by e-mail at stickysauce@highrankings.com.

    Jill Whalen of High Rankings is an internationally recognized search engine optimization consultant and host of the free weekly High Rankings Advisor search engine marketing newsletter.

    She specializes in search engine optimization, SEO consultations and seminars. Jill's handbook, "The Nitty-gritty of Writing for the Search Engines" teaches business owners how and where to place relevant keyword phrases on their Web sites so that they make sense to users and gain high rankings in the major search engines.

    DISCLAIMER: The content provided in this article is not warranted or guaranteed by Developer Shed, Inc. The content provided is intended for entertainment and/or educational purposes in order to introduce to the reader key ideas, concepts, and/or product reviews. As such it is incumbent upon the reader to employ real-world tactics for security and implementation of best practices. We are not liable for any negative consequences that may result from implementing any information covered in our articles or tutorials. If this is a hardware review, it is not recommended to open and/or modify your hardware.

    More Web Development Articles
    More By Developer Shed



    - On Page SEO for New Domains
    - Improve Your Site`s Speed
    - Safari Books Online Review
    - Creating an Estore From the Ground Up
    - Most Common SEO Mistakes Developers Make
    - Making the Most of Your Titles and Meta Desc...
    - Five Ways Using Flash Can Damage Your Site
    - A Web Designer`s Guide to Colors
    - Use Webstarts to Create a Free Site
    - More Than Just Looks. How Your Web Design C...
    - How to Design Content Pages
    - Mint Review
    - Make Your WordPress Website Look Professional
    - How to Create a Mobile Web Site
    - Meta Tags: Still Useful?

    Developer Shed Affiliates


    © 2003-2018 by Developer Shed. All rights reserved. DS Cluster - Follow our Sitemap