Paid Search Engine Inclusion Programs - The Details
By
Dirk Brockhausen
Following our overview of current paid search engine inclusion (PFI) programs, we will now take a closer look at the technical procedures involved, spidering activity, quality of service, etc. This first instalment deals with Inktomi and AltaVista, more will follow in a later article.
Inktomi
Inktomi's current submission partners are:ineedhits.com, Outrider, Position Technologies and
VeriSign.
Outrider is PositionTech's London based partner. Personnel connections exist between PositionTech and Outrider via Detlef Johnson, the former Supervisor Search Engine Optimization Services with Outrider, now functioning as VP of Technology with PositionTech.
We registered new domains with all four partners and tracked the ensuing spider visits.
ineedhits.com
The Inktomi spider arrived the day following page submission:
#UA Mozilla/3.0 (Slurp/si; slurp@inktomi.com;
http://www.inktomi.com/slurp.html)
j6002.inktomi.com
209.185.141.249
Regularly, this spider will crawl your pages every second day. However, we detected exceptions from the rule. E.g. following the turn of the month, the spider took 4 days to come back.
Beyond the pages registered for paid inclusion, the domain URL (e.g. http://www.domain.com/ ) was spidered.
This will normally occur every 5 days, with the site's robots.txt file being crawled as well.
However, the domain URL hasn't to date been included in the Inktomi index.
About two weeks following paid submission, this Inktomi spider started to fetch the pages in irregular intervals:
#UA Mozilla/5.0 (Slurp/cat; slurp@inktomi.com;
http://www.inktomi.com/slurp.html)
wm3018.inktomi.com
216.35.116.103
Beyond Spidering
Spider visits apart, what's important for the submitting webmaster is, of course, the PFI partners' Management Area or control center.
In ineedhits.com's "Subscription Management Area" you will see your URLs's current status.
The "Click-thru hits report" displays the number of hits and the search terms which generated them. You can click on the links to either AOL, HotBot or MSN to see the pertinent search results page. According to ineedhits.com: "Click-Thru reports currently only reflect a small percentage of the entire click-thrus Inktomi Search Partners refer to your URL."
This is no exaggeration: in our case, only about 75% of hits and keywords actually generated were displayed in this report. Obviously, this calls for some serious improvement.
Outrider
About 1 to 2 hours after submission, our site was visited by the PositionTech spider:
#UA Mozilla/4.0 (compatible; MSIE 4.01; Windows NT)
169.207.238.180
It was a one-hit affair only: we haven't seen this spider visit our site again since.
Another two hours later, the Inktomi crawling cycle commended with a visit from this spider:
#UA Mozilla/3.0 (Slurp/si; slurp@inktomi.com;
http://www.inktomi.com/slurp.html)
j6002.inktomi.com
209.185.141.249
Following this, the spider started visiting the site in the same rhythm as with ineedhits.com.
Beyond Spidering
In Outrider's "Account Service" section you will see your URLs's current status and the number of generated clicks. According to Outrider: "Click-Thru reports reflect less then half of all traffic generated by Inktomi." This can be confirmed on our part: on average, less
than 50% of generated clicks were reported.
PositionTech
The PositionTech spider arrived one hour after submission:
#UA Mozilla/4.0 (compatible; MSIE 4.01; Windows NT)
169.207.238.180
Another hour later, the Inktomi spider cropped up:
#UA Mozilla/3.0 (Slurp/si; slurp@inktomi.com;
http://www.inktomi.com/slurp.html)
j6002.inktomi.com
209.185.141.249
Ensuing spider behavior was identical to submissions via ineedhits.com and Outrider.
Beyond Spidering
In PositionTech's subscribers area you will see your URLs's current status and the number of generated clicks as well as the keywords entered by visitors. In our case, only 40% of actually generated hits and keywords were displayed. Here, there is no pointer to the fact that not all clicks will be correctly displayed. This calls for improvement. Click-thru data can also be checkd in a graphical view.
A very nifty feature exclusively offered by PositionTech is their "Pure Web Search" function. Under URL < http://169.207.238.189/search/ > a search form for pure Inktomi is offered. Search results are uncluttered by "enhancements" such as Overture PPC links, ODP stuff, ads, etc. This URL is accessible to the general public (at least, currently).
VeriSign
One day after submission, the Inktomi spider arrived:
#UA Mozilla/3.0 (Slurp/si; slurp@inktomi.com;
http://www.inktomi.com/slurp.html)
j6002.inktomi.com
209.185.141.249
Ensuing spider behavior was identical to submissions via the other partners.
Beyond Spidering
In VeriSign's subscribers area you will see your URLs's current status and the number of generated clicks as well as the keywords entered by visitors. However, this did not work in our case. Although our access log files registered several hits, VeriSign did not display so much as a single one of them.
Inktomi PFI Summary
* Pricing is identical, regardless of submission partner.
* Submitting your pages to any of the partners will trigger the Inktomi spider, which will then crawl your pages following its own rhythm, independent of chosen submission partner. Partners do seem to hold a certain influence on commencement of the spidering cycle, though.
* With PositionTech and Outrider it took a mere 1-2 hours for the first Inktomi spider to arrive. They are also the only submission partners checking sites with their own (shared) spider.
* Inktomi seems to have devoted a single spider for crawling pages under the PFI program.
* Crawled pages will normally show up in the Inktomi index about one day after spidering.
* PositionTech's subscribers area control center offers the most comprehensive functionality, making it the superior service.
However, seeing that no submission partner will currently process 100% of generated hits, a systematic analysis of your own server traffic logs cannot be discarded.
AltaVista's infoSpider
Paid inclusion for AltaVista is handled by their independent partner infoSpider.
Shortly after submission, a spider will check the submitted URL:
#UA infoSpider URL Checker
208.185.243.149.proboost.com
208.185.243.149
Four days after, the first AltaVista spider arrived on
site:
#UA Scooter-3.0.3
scooter2.sv.av.com
209.73.162.172
Pages were included in AltaVista's index the following day.
The spidering cycle is determined by AltaVista's index runs for paid inclusions, which is about once per week.
The AltaVista spider(s) will visit submitted pages one, two or three days prior to this index run. There is currently no regular, predictable rhythm discernible as with Inktomi.
AltaVista has devoted at least one other spider to
crawling PFI pages:
#UA Scooter-3.2
scooter3.sv.av.com
209.73.162.143
Beyond Spidering
In infoSpider's Service section, you can check on your URLs' current status. No hits and keywords data is being offered.
Compared to Inktomi's PFI program, AltaVista's refresh cycles are considerably longer, and the service area offered features only very basic functionality.
This text may be freely republished or distributed provided all product links and the following resource box are included intact (box may be included below or above article) and a complimentary copy or notice (link) is sent to the author at the address specified below:
Dirk Brockhausen is the co-founder and principal of fantomaster.com GmbH (Belgium), http://fantomaster.com/ , a company specializing in webmasters software development,
industrial-strength cloaking and search engine positioning services. He holds a doctorate in physics and has worked as an SAP consultant and software developer since 1994. He is
also Technical Editor of fantomNews, a free newsletter focusing on search engine optimization, available at:http://fantomaster.com/fantomnews-sub.html
You can contact him at mailto:fntecheditor@fantomaster.com
(c) copyright 2002 by fantomaster.com All rights reserved. Downloaded at: http://fantomaster.com/
| DISCLAIMER: The content provided in this article is not warranted or guaranteed by Developer Shed, Inc. The content provided is intended for entertainment and/or educational purposes in order to introduce to the reader key ideas, concepts, and/or product reviews. As such it is incumbent upon the reader to employ real-world tactics for security and implementation of best practices. We are not liable for any negative consequences that may result from implementing any information covered in our articles or tutorials. If this is a hardware review, it is not recommended to open and/or modify your hardware. |
More Search Engine Tricks Articles
More By Developer Shed
 developerWorks - FREE Tools!
|
Hear how IBM Rational Project and Portfolio Management integrated solutions help teams put the right tools and processes in place to maximize the effectiveness and efficiency of project teams and ensure that the business vision is being executed correctly. Learn how to automate and integrate requirements prioritization, top-down project planning, communications and controls, and methodology deployment to keep your scope, costs, and schedules under control. Tackle with an end-to-end approach the management of scope and scope changes, usage of methodology to control and empower project teams, and optimization of resources to align activity costs with the overall project plan.
FREE! Go There Now!
|
|
|
|
Poor Requirements Management capabilities in an Enterprise have been linked to excessive project failures, escalating IT costs, and failure to deliver competitive advantage into the marketplace. Join Brianna M Smith from IBM Rational and learn about how successful organizations align IT and Business stakeholders through collaborative processes and tools for effective requirements management, and how an integrated approach across the IT lifecycle can provide unparalleled visibility and traceability to ensure that project teams are delivering on the business vision by "doing the right things" and "doing things right."
FREE! Go There Now!
|
|
|
|
CakePHP is a stable production-ready, rapid-development aid for building Web sites in PHP. This "Cook up Web sites fast with CakePHP" series shows you how to build an online product catalog using CakePHP.
FREE! Go There Now!
|
|
|
|
Download the Rational Application Developer (RAD) v7.5 open beta code and start developing applications for the JEE5 standard which features EJB3.0, JPA, JSF 1.2, JSP 2.1 and Servlet 2.5 standards. When you use this beta you will see how you can increase developer productivity for already existing applications with improved support for refactoring, as well as adding new features to existing applications. In addition, the beta provides tooling for JD Edwards, Oracle, SAP, Siebel and PeopleSoft to improve the developer productivity with these enterprise systems.
FREE! Go There Now!
|
|
|
|
Discover how IBM Rational AppScan Standard Edition can help you detext vulnerabilities in your web applications in the Web Application Security eKit. IBM Rational AppScan is a leading suite of automated web application security solutions that scan and test for common Web application vulnerabilities. The new Web Application Security eKit provides you with valuable resources, including white papers, demos, and additional information on the benefits of testing your Web applications.
FREE! Go There Now!
|
|
|
|
Download a free trial version of IBM Rational Developer for System i V7.1, which provides a complete development environment for traditional i5/OS application development. IBM Rational Developer for System i is a new eclipse-based workstation offering for i5/OS application development that provides a comprehensive Integrated Development Environment for edit/compile/debug of traditional RPG/COBOL/C/C++ i5/OS applications.
FREE! Go There Now!
|
|
|
|
Visit IBM developerWorks to download a free trial version of WebSphere Extended Deployment Compute Grid, which lets you schedule, execute, and monitor batch jobs. Because online transaction processing and batch jobs execute simultaneously on the same server resources, you can avoid costly duplication of resources. Compute Grid supports job types of Java transactional batch, compute-intensive and a new type called "native execution", which enables non-Java workloads to run on distributed end points.
FREE! Go There Now!
|
|
|
|
Join this webcast to see how IBM Data Studio Developer and pureQuery can take the pain out of Java data access. uApplications developed using both Java and SQL have become a common requirement. Database connectivity using Java Database Connectivity (JDBC) to create an application is a multi-step tedious process, and tooling that covers both SQL and Java has been unavailable, until now. IBM Data Studio introduces the pureQuery platform: a high-performance, Java data access platform focused on simplifying the tasks of developing, managing, and optimizing database applications and services.
FREE! Go There Now!
|
|
|
|
Rational Modeling Extension for Microsoft .NET enhances usability for code generation supporting a more intelligent refactoring. The latest enhancements enable organizations with Java and .NET systems and software development maintain architectural integrity across heterogeneous platforms.
FREE! Go There Now!
|
|
|
|
Join this Rational Talks to You teleconference on December 6 at 1:00 pm ET to participate in an agile application development discussion and get your questions answered on using IBM Rational Method Composer in a distributed environment.Get your questions answered!
FREE! Go There Now!
|
|
|
|
All FREE IBM® developerWorks Tools!
|
