Karen Scharf is an Indianapolis marketing consultant who works with small business owners and entrepreneurs. She offers several whitepapers, free reports and checklists, including her FREE Can-Spam checklist and FREE email pre-flight checklist to ensure your emails get delivered, get opened and get read. Download your copies at http://www.ModernImage.com.
In order to analyze the content on your web site and figure out which pages to display (and in which order) for certain search terms, the search engines (SEs) use a program called a web crawler. You might have also heard these referred to as spiders or web-bots.
Every SE has its own web crawler program and its own formula for ranking pages. However, while page factors might be ranked differently by different engines, the idea behind the web crawlers is basically the same.
Getting your site crawled often is a good indicator that the SE finds your site important. And since important sites get better rankings, getting your site crawled frequently is definitely a good thing.
While there is no way you can make the SE crawl your site, there are a few things you can to do encourage the spiders to visit your site more often:
1. First and foremost, keep your content up to date. Add unique content often and on a regular basis. And be sure to ping Google each time you update your site.
2. Get as many backlinks as possible from other relevant sites that are crawled regularly. There are several tools available that will track and measure the crawl rate of other websites.
3. Use a unique title tag for each page. Also, while not as important as the title, each page should also have its own keywords and description tag. This will help the spiders "know" that each page is a unique and individual entity.
4. Keep your pages as small as possible. The spider will not spend an unlimited amount of time at your web site. If it's held up by your huge images, monstrous PDFs or even excessive text, it will simply abandon the page and move on.
5. Pay close attention to your internal links and avoid any duplicate content issues. If you have multiple pages with the same content (often used for testing and/or tracking purposes) be sure to use your robots text file to keep the SEs away.
6. Create a sitemap for the content you want crawled and make sure your content is compliant.
7. Monitor your web server. Obviously, your server needs to be up and running when the spiders come a-crawling. There are many commercial providers who will monitor your server and notify you when there is a problem.
In addition, ensure that your server handles all of your error pages correctly. Refer to the Google Webmaster Tools report of unreached pages for an idea of what errors are being returned.
8. And while we're discussing Google Webmaster Tools, don't forget to use it to monitor your crawl rate, adjust it if necessary, test and track and see what works best for your site.
Time to implement: As with all SE optimization, working toward an optimal crawl rate is not a one-time job. You will need to add content approximately three times a week and update your site map as needed. Monitoring your web server is an ongoing task.
- Related Videos
- Related Articles
- Ask / Related Q&A
- 4PSEO Search Engine Optimization (SEO) Company from India:
- Search Marketing: Turning your Website Outside-in to Increase Sales
- A Turn-Key SEO Reseller Program - Customized for your business
- SEO Reseller Plans – Referral vs. Private Label Why is it so hard to find a good SEO Reseller Plan! Help!
- Endless Rise’s White Label SEO program offers support, campaign management, and sales training
- Making Money with White Label SEO !!!
- Using Video Marketing to Drive SEO Traffic
- SEO Turning Visitors Into Sales
SEO vs PPC
By: creative | 28/12/2009You Need Them Both We hope you’ve heard about “Paid Search” or PPC (pay per click) if your business relies on Web traffic. It’s one of the key ways to draw traffic to your site, and it works wonders when applied with SEO (search engine optimization). In other words, if you’re still using SEO as your only marketing avenue, you need to balance your marketing approach because your competitors may already have.
Freelance content writers Vs Professional content writing firm
By: Kamesh Goud | 28/12/2009One should be careful while hiring a freelance content writer or a professional content writing firm as the content written by a freelancer may be duplicate of a copyrighted content. A professional firm always makes sure that content developed is unique and does not violate any copyright law.
Make the holidays more fun with a hot tub for the whole family!
By: johnmak | 28/12/2009This holiday season; why not give your family a cozy and relaxing present under the tree. Hot tubs are a great addition to any home and are the perfect gift for the entire family. Hot tubs can offer your family a wide range of health benefits, muscle relaxation, stress relief and social activity.
Link Building Service- Its Vital Aspect
By: Bhargav Bavarva | 28/12/2009For anyone who has considered going through the process of search engine optimization, it is more likely than not, that you have heard the word “backlink”. However, if you are new to the idea of SEO, the question which probably popped into your mind is- what is a backlink and why is a link building service needed?
SEO Techniques Every Website Owner Needs to Know
By: Alice Ault | 28/12/2009It is certainly no secret that in order for an internet marketing venture to succeed much less thrive it will need a great deal of traffic flowing to its website.
The Top Three Reasons Why Visitors Leave Your Site without Buying a Product or Filling out a Form
By: Aaron Garcia | 28/12/2009It is the frustration of every website owner. You look at your analytics at the end of the week and are delighted to see that your traffic has increased, but you have not seen the same increase in sales or leads.
Pay Per Click versus Search Engine Optimization: Behind All the Hype
By: Peter Dulay | 28/12/2009Each online personality in the vast online world seeks the achievement of the similar dream: A boundless stream of preferred visitors on their websites,admiring their works, reading their contents and putting their advertisements and offerings into account.
Think smart, only pay if people actually click on your links!!
By: johnmak | 28/12/2009Pay per Click or just PPC, is a part of Search engine optimization. As the name suggests, advertisers pay their host if and only if their ad is clicked. It is a very effective internet advertising model used on many websites.
Why Won't My Site Show Up in Google?
By: Karen Scharf | 07/10/2009 | SEOWe often get inquiries from prospective clients who would like us to build them a new website. They believe that their current website must be “bad” because it's not listed in the search engines. They're convinced that a new website will automatically ensure a top ten Google rank.
There Is No Such Thing as a Duplicate Content Penalty
By: Karen Scharf | 07/10/2009 | SEOFirst, I want you to understand that I am NOT giving you license to steal other people’s content. There ARE repercussions to using duplicate content, but it is not a penalty per se.
Email Marketing Tests to Get You Started
By: Karen Scharf | 07/10/2009 | Internet MarketingIf you’re utilizing email marketing, you should be utilizing email testing. But it can seem complex and costly, and many small business owners aren’t quite sure where to start. If you’d like to use email testing, but aren’t sure where to start, here are 4 of the easiest tests to get you going:
Save Money With Geo-Targeting
By: Karen Scharf | 22/09/2009 | Internet MarketingIf you're a local business on a tight budget, you can save money on your advertising budget by targeting only specific locations with your AdWords campaigns.
Landing Page Optimization Tips
By: Karen Scharf | 22/09/2009 | Internet MarketingWhen we hear the words Landing Page we generally think of PPC campaigns or some other form of internet advertising. But I like to use landing pages for all of my off-line advertising also. However, you’re using it, a landing page is a web page, typically hidden from the site’s menu bar and other navigation – meaning the only way you can get to the page is to type the URL directly into the browser.
Boring Subject Lines Outpull Hype
By: Karen Scharf | 18/09/2009 | Small BusinessIn email marketing, the subject line is often seen as the single most important element. Afterall, if your subject line isn't persuasive, no one will open your email and your message won't get read. The subject line is generally equated with the headline of a display ad.
Measuring Your Social Media ROI
By: Karen Scharf | 16/09/2009 | Small BusinessIf you’ve added social media to your marketing mix, you’re probably asking yourself, “Is it worth all the time and effort involved?” That might seem like a difficult question to answer at first, but there are definite steps you can take to measure the return on your social media investments.