Kamg Logo Png

Optimizing Your Site

»

Allowing Google to Index Your Pages
Allowing Google To Index Your Pages

Your content cannot rank on search engine results pages (SERPs) if it is not included in Google’s index.

Think of the internet as a library and Google is the librarian. When you aren’t sure where to find an answer to your question, you normally would ask the librarian for help.

But before the librarian can help people with answers, they need to find books, understand them, and index or categorise them in terms of what they are about.

So when Google indexes your web pages it is to gather data about them and categorise them. This helps Google to define what queries or searches your webpages could potentially answer.

The following is how indexing works according to Google.

A page is indexed by Google if it has been visited by the Google crawler (“Googlebot”), analysed for content and meaning, and stored in the Google index.
Indexed pages can be shown in Google Search results (if they follow Google’s webmaster guidelines).
While most pages are crawled before indexing, Google may also index pages without access to their content (for example, if a page is blocked by a robots.txt directive).

And of course, with almost 227 million Google searches happening in an hour and about 5.4 billion per day, you don’t want to miss the opportunity of your web content being indexed and shown to people.

Especially if you’re selling something.

So as an eCommerce owner or marketer, you need to ensure that Google is indexing your content. If not, you’re missing out on potential revenue driven by organic or search traffic.

And yes, there are instances where Google isn’t able to index a page. But you can fix that when it happens to you.

In this section of the KAMG SEO content hub, we will guide you through what you can do to ensure that Google effectively indexes your eCommerce site content.

Allowing Google To Index Your Pages

Crucial Terms You Need to Know

Before we dive in, there are a few words that you have to properly define.

Crawling

Crawling is Google’s process of finding new content. It is mostly done by following hyperlinks.

Web Spider

Googlebot is Google’s branded web spider or crawler. This software carries out the crawling process.

Indexing

Indexing is Google’s process of storing web page contents into a vast categorised database.

One vital fact to remember is that indexing is not ranking. 

Indexing is showing up in the competition while ranking is winning it. 

When Google indexes your site, it knows what your pages are about and can show your content to answer relevant queries. But Google will ensure to show first content that is highly relevant and authoritative, whether it is from you or someone else.

So if your content is poor, it won’t rank highly on SERPs. 

But of course, you can’t win if you don’t show up. So you need to ensure indexing happens.

How to Know if Your Website is Indexed

Google, in essence, should be able to index all internet content. So you probably don’t have to worry about not showing up on SERPs.

But it never hurts to double check.

Go to Google and type in and search site:yourwebsite.com.

Allowing Google To Index Your Pages

The results number shows an approximation of how many of your pages have been indexed by Google.

You can use the same step to check for the indexing of a certain web page as shown in the example image below.

Allowing Google To Index Your Pages

If there’s at least one result, then your page is indexed.

You can also get a more accurate insight using the Google Search Console’s Coverage report.

Simply go to Google Search Console > Index > Coverage.

Allowing Google To Index Your Pages

Put your attention to the number of valid pages, with warnings and without.

If the total of these two numbers is anything greater than zero, then Google has indexed at least some of your website pages.

Otherwise, you can definitely say that you have a severe problem because not one of your site pages have been indexed.

You can also use the URL Inspection tool of the Search Console to check if a certain web  page is indexed.

If the page you’re checking is indexed, the tool will display “URL is on Google” like in the image below.

Allowing Google To Index Your Pages

How to Ensure That Your Pages are Indexed

If you have found out that one or more of your web pages aren’t indexed by Google, you may try fixing the problem by doing the following:

  1. Ensure that your page follows Google’s general guidelines.
  2. Go to Google Search Console
  3. Open the URL inspection tool
  4. Paste the URL of the page you’d like to index into the search bar.
  5. Wait for Google to finish checking the URL
  6. Click the “Request indexing” button

When you have a newly published post or page, you should follow the above steps. By doing so, you effectively tell Google that you’ve added something new to your site and that they should check it out.

Still, there might be underlying issues that disallow Google from indexing your page, most especially old ones. 

Also Google can take anywhere from a few days to a few weeks to index a website.

Fortunately, there are steps you can take to improve indexing efficiency. 

To assess and solve the problem, simply go through the following steps.

Other Things You Can do to Ensure Indexing

Optimise Your robot.txt File

Robots.txt files are recognised by Googlebot as an indication that a web page should not be crawled and thus not indexed. Robots.txt is also recognized by Bing and Yahoo! search engine spiders.

You would use robots.txt files to assist crawlers in prioritising more important pages so that your own site is not overburdened with requests.

Although this may appear to be a bit technical, it all boils down to making sure your page is crawlable, which you can check with SemRush’s On Page SEO Checker. It provides optimisation feedback, including technical edits such as whether a page is crawlable as shown in the image below.

Allowing Google To Index Your Pages

The tool is intuitive and will give you information on how to fix issues when they arise.

Ensure That Your SEO Tags Are Free of Errors

SEO tags are another method for directing search engine spiders such as Googlebot.

There are two types of SEO tags that should be optimised.

  • Rogue Noindex Tags

These tags instruct search engines not to index pages. If certain pages aren’t being indexed, it’s possible that they have noindex tags. 

Look for the following two types:

  • Meta Tags

Meta elements are HTML and XHTML tags that provide structured metadata about a Web page.

Look for noindex page warnings to see which pages on your website may have noindex meta tags.

To index a page that has been marked as noindex, simply remove the meta tag.

  • X-Robots Tag

The X-Robots-Tag can be used as an HTTP header response element for a given URL. Any directive that can be specified as an X-Robots-Tag can also be specified as a robots meta tag.

To find out which pages have an X-Robots-Tag in their HTML header, use Google’s Search Console’s URL Inspection Tool.

Look for the response to “Indexing allowed?” after entering a page.

If you see the words No: ‘noindex’ detected in ‘XRobots-Tag’ http header, you know you need to remove an X-Robots-Tag.

  • Canonical Tags

Canonical tags inform crawlers whether a particular version of a page is favoured.

If a page lacks a canonical tag, Googlebot acknowledges it as the preferred and only variant of that page, and will index it.

If a page has a canonical tag, Googlebot supposes there is another preferred version of that page and will not index it, even if that other variant does not exist.

To inspect for canonical tags, use Google’s URL Inspection Tool. You’ll see a warning that says Alternate page with canonical tag in this case.

Optimise Your Entire Site’s Architecture

To see to it that Google crawls and indexes your website and its pages more efficiently, you have to ensure seamless internal linking and excellent backlinking.

Internal linking assists crawlers in finding your web pages. Non-linked pages are referred to as “orphan pages,” and they are rarely indexed. 

Proper internal linking is ensured by proper site architecture, as laid out in a sitemap.

Your XML sitemap organises all of the content on your website, helping to identify non-linked pages.

Here are a few more best practices for internal linking:

  • Remove nofollow internal links.

When Googlebot encounters nofollow tags, it alerts Google that the tagged target link should be removed from its index.

Nofollow tags should be removed from links.

  • Include high-ranking internal links.

Spiders discover fresh content by crawling your website, as previously stated. And internal links help to speed up the process. 

Streamline indexing by internally linking to new pages from high-ranking pages.

  • Produce high-quality backlinks.

Google recognises that pages are relevant and important if authority sites consistently link to them. 

Backlinks alert Google to the fact that a page should be indexed.

Make High-Quality Content a Priority

All indexing and ranking rely heavily on high-quality content. Consider removing poor and underperforming pages from your website to ensure its content is high-performing.

This enables Googlebot to focus on more important pages on your website, allowing you to make better use of your “crawl budget.”

Additionally, you want each page on your site to be valuable to users. Your content should be one-of-a-kind as Google Analytics may otherwise flag it as duplicate content.

You need to get indexed if you are to rank.

So clearly, you need to ensure that your website and its pages are indexed by Google if it is going to rank at all on SERPs.

Ranking higher means that more people will see your content and probably go to your eCommerce site to do business with you. This can be good for your revenue.

And so, there’s a lot more to do after ensuring indexing. To rank higher on SERPs, you need to optimise your content and web pages. 

And that’s what you will learn in the next sections of this KAMG SEO content hub.

Privacy Policy

Last updated: June 08, 2022

This Privacy Policy describes Our policies and procedures on the collection, use and disclosure of Your information when You use the Service and tells You about Your privacy rights and how the law protects You.

We use Your Personal data to provide and improve the Service. By using the Service, You agree to the collection and use of information in accordance with this Privacy Policy.

Interpretation and Definitions

Interpretation

The words of which the initial letter is capitalized have meanings defined under the following conditions. The following definitions shall have the same meaning regardless of whether they appear in singular or in plural.

Definitions

For the purposes of this Privacy Policy:

  • Account means a unique account created for You to access our Service or parts of our Service.

  • Company (referred to as either “the Company”, “We”, “Us” or “Our” in this Agreement) refers to Kas Andz Marketing Group, International House, 24 Holborn Viaduct, London EC1A 2BN.

  • Cookies are small files that are placed on Your computer, mobile device or any other device by a website, containing the details of Your browsing history on that website among its many uses.

  • Country refers to: United Kingdom

  • Device means any device that can access the Service such as a computer, a cellphone or a digital tablet.

  • Personal Data is any information that relates to an identified or identifiable individual.

  • Service refers to the Website.

  • Service Provider means any natural or legal person who processes the data on behalf of the Company. It refers to third-party companies or individuals employed by the Company to facilitate the Service, to provide the Service on behalf of the Company, to perform services related to the Service or to assist the Company in analyzing how the Service is used.

  • Usage Data refers to data collected automatically, either generated by the use of the Service or from the Service infrastructure itself (for example, the duration of a page visit).

  • Website refers to Kas Andz Marketing Group, accessible from https://www.kasandz.com/

  • You means the individual accessing or using the Service, or the company, or other legal entity on behalf of which such individual is accessing or using the Service, as applicable.

Collecting and Using Your Personal Data

Types of Data Collected

Personal Data

While using Our Service, We may ask You to provide Us with certain personally identifiable information that can be used to contact or identify You. Personally identifiable information may include, but is not limited to:

  • Email address

  • First name and last name

  • Phone number

  • Address, State, Province, ZIP/Postal code, City

  • Usage Data

Usage Data

Usage Data is collected automatically when using the Service.

Usage Data may include information such as Your Device’s Internet Protocol address (e.g. IP address), browser type, browser version, the pages of our Service that You visit, the time and date of Your visit, the time spent on those pages, unique device identifiers and other diagnostic data.

When You access the Service by or through a mobile device, We may collect certain information automatically, including, but not limited to, the type of mobile device You use, Your mobile device unique ID, the IP address of Your mobile device, Your mobile operating system, the type of mobile Internet browser You use, unique device identifiers and other diagnostic data.

We may also collect information that Your browser sends whenever You visit our Service or when You access the Service by or through a mobile device.

Tracking Technologies and Cookies

We use Cookies and similar tracking technologies to track the activity on Our Service and store certain information. Tracking technologies used are beacons, tags, and scripts to collect and track information and to improve and analyze Our Service. The technologies We use may include:

  • Cookies or Browser Cookies. A cookie is a small file placed on Your Device. You can instruct Your browser to refuse all Cookies or to indicate when a Cookie is being sent. However, if You do not accept Cookies, You may not be able to use some parts of our Service. Unless you have adjusted Your browser setting so that it will refuse Cookies, our Service may use Cookies.
  • Flash Cookies. Certain features of our Service may use local stored objects (or Flash Cookies) to collect and store information about Your preferences or Your activity on our Service. Flash Cookies are not managed by the same browser settings as those used for Browser Cookies. For more information on how You can delete Flash Cookies, please read “Where can I change the settings for disabling, or deleting local shared objects?” available at https://helpx.adobe.com/flash-player/kb/disable-local-shared-objects-flash.html#main_Where_can_I_change_the_settings_for_disabling__or_deleting_local_shared_objects_
  • Web Beacons. Certain sections of our Service and our emails may contain small electronic files known as web beacons (also referred to as clear gifs, pixel tags, and single-pixel gifs) that permit the Company, for example, to count users who have visited those pages or opened an email and for other related website statistics (for example, recording the popularity of a certain section and verifying system and server integrity).

Cookies can be “Persistent” or “Session” Cookies. Persistent Cookies remain on Your personal computer or mobile device when You go offline, while Session Cookies are deleted as soon as You close Your web browser. Learn more about cookies on the Free Privacy Policy website article.

We use both Session and Persistent Cookies for the purposes set out below:

  • Necessary / Essential Cookies

    Type: Session Cookies

    Administered by: Us

    Purpose: These Cookies are essential to provide You with services available through the Website and to enable You to use some of its features. They help to authenticate users and prevent fraudulent use of user accounts. Without these Cookies, the services that You have asked for cannot be provided, and We only use these Cookies to provide You with those services.

  • Cookies Policy / Notice Acceptance Cookies

    Type: Persistent Cookies

    Administered by: Us

    Purpose: These Cookies identify if users have accepted the use of cookies on the Website.

  • Functionality Cookies

    Type: Persistent Cookies

    Administered by: Us

    Purpose: These Cookies allow us to remember choices You make when You use the Website, such as remembering your login details or language preference. The purpose of these Cookies is to provide You with a more personal experience and to avoid You having to re-enter your preferences every time You use the Website.

For more information about the cookies we use and your choices regarding cookies, please visit our Cookies Policy or the Cookies section of our Privacy Policy.

Use of Your Personal Data

The Company may use Personal Data for the following purposes:

  • To provide and maintain our Service, including to monitor the usage of our Service.

  • To manage Your Account: to manage Your registration as a user of the Service. The Personal Data You provide can give You access to different functionalities of the Service that are available to You as a registered user.

  • For the performance of a contract: the development, compliance and undertaking of the purchase contract for the products, items or services You have purchased or of any other contract with Us through the Service.

  • To contact You: To contact You by email, telephone calls, SMS, or other equivalent forms of electronic communication, such as a mobile application’s push notifications regarding updates or informative communications related to the functionalities, products or contracted services, including the security updates, when necessary or reasonable for their implementation.

  • To provide You with news, special offers and general information about other goods, services and events which we offer that are similar to those that you have already purchased or enquired about unless You have opted not to receive such information.

  • To manage Your requests: To attend and manage Your requests to Us.

  • For business transfers: We may use Your information to evaluate or conduct a merger, divestiture, restructuring, reorganization, dissolution, or other sale or transfer of some or all of Our assets, whether as a going concern or as part of bankruptcy, liquidation, or similar proceeding, in which Personal Data held by Us about our Service users is among the assets transferred.

  • For other purposes: We may use Your information for other purposes, such as data analysis, identifying usage trends, determining the effectiveness of our promotional campaigns and to evaluate and improve our Service, products, services, marketing and your experience.

We may share Your personal information in the following situations:

  • With Service Providers: We may share Your personal information with Service Providers to monitor and analyze the use of our Service, to contact You.
  • For business transfers: We may share or transfer Your personal information in connection with, or during negotiations of, any merger, sale of Company assets, financing, or acquisition of all or a portion of Our business to another company.
  • With Affiliates: We may share Your information with Our affiliates, in which case we will require those affiliates to honor this Privacy Policy. Affiliates include Our parent company and any other subsidiaries, joint venture partners or other companies that We control or that are under common control with Us.
  • With business partners: We may share Your information with Our business partners to offer You certain products, services or promotions.
  • With other users: when You share personal information or otherwise interact in the public areas with other users, such information may be viewed by all users and may be publicly distributed outside.
  • With Your consent: We may disclose Your personal information for any other purpose with Your consent.

Retention of Your Personal Data

The Company will retain Your Personal Data only for as long as is necessary for the purposes set out in this Privacy Policy. We will retain and use Your Personal Data to the extent necessary to comply with our legal obligations (for example, if we are required to retain your data to comply with applicable laws), resolve disputes, and enforce our legal agreements and policies.

The Company will also retain Usage Data for internal analysis purposes. Usage Data is generally retained for a shorter period of time, except when this data is used to strengthen the security or to improve the functionality of Our Service, or We are legally obligated to retain this data for longer time periods.

Transfer of Your Personal Data

Your information, including Personal Data, is processed at the Company’s operating offices and in any other places where the parties involved in the processing are located. It means that this information may be transferred to — and maintained on — computers located outside of Your state, province, country or other governmental jurisdiction where the data protection laws may differ than those from Your jurisdiction.

Your consent to this Privacy Policy followed by Your submission of such information represents Your agreement to that transfer.

The Company will take all steps reasonably necessary to ensure that Your data is treated securely and in accordance with this Privacy Policy and no transfer of Your Personal Data will take place to an organization or a country unless there are adequate controls in place including the security of Your data and other personal information.

Disclosure of Your Personal Data

Business Transactions

If the Company is involved in a merger, acquisition or asset sale, Your Personal Data may be transferred. We will provide notice before Your Personal Data is transferred and becomes subject to a different Privacy Policy.

Law enforcement

Under certain circumstances, the Company may be required to disclose Your Personal Data if required to do so by law or in response to valid requests by public authorities (e.g. a court or a government agency).

Other legal requirements

The Company may disclose Your Personal Data in the good faith belief that such action is necessary to:

  • Comply with a legal obligation
  • Protect and defend the rights or property of the Company
  • Prevent or investigate possible wrongdoing in connection with the Service
  • Protect the personal safety of Users of the Service or the public
  • Protect against legal liability

Security of Your Personal Data

The security of Your Personal Data is important to Us, but remember that no method of transmission over the Internet, or method of electronic storage is 100% secure. While We strive to use commercially acceptable means to protect Your Personal Data, We cannot guarantee its absolute security.

Children’s Privacy

Our Service does not address anyone under the age of 13. We do not knowingly collect personally identifiable information from anyone under the age of 13. If You are a parent or guardian and You are aware that Your child has provided Us with Personal Data, please contact Us. If We become aware that We have collected Personal Data from anyone under the age of 13 without verification of parental consent, We take steps to remove that information from Our servers.

If We need to rely on consent as a legal basis for processing Your information and Your country requires consent from a parent, We may require Your parent’s consent before We collect and use that information.

Links to Other Websites

Our Service may contain links to other websites that are not operated by Us. If You click on a third party link, You will be directed to that third party’s site. We strongly advise You to review the Privacy Policy of every site You visit.

We have no control over and assume no responsibility for the content, privacy policies or practices of any third party sites or services.

Changes to this Privacy Policy

We may update Our Privacy Policy from time to time. We will notify You of any changes by posting the new Privacy Policy on this page.

We will let You know via email and/or a prominent notice on Our Service, prior to the change becoming effective and update the “Last updated” date at the top of this Privacy Policy.

You are advised to review this Privacy Policy periodically for any changes. Changes to this Privacy Policy are effective when they are posted on this page.

Contact Us

If you have any questions about this Privacy Policy, You can contact us: