Home
How To

How to fix Sitemap Disaster in Google’s Indexing Pipeline?

How to fix Sitemap Disaster in Google’s Indexing Pipeline
Google search console

There is biggest disaster in Google’s sitemap Indexing Pipeline. If you have already submitted your Blog or website sitemap to Google Webmaster tools then you should check now. Because after 25th September 2015 most of the indexed link dropped rapidly.

If your Blog or website has more than 500 posts and Google has indexed all but just check it now and see that your indexed links decreased to 151. Because there are major problem causing in Google’s indexing pipeline. According to Google they are working on that. But in the meantime you will be loser. Because you would see most of the link removed from Google’s search Engine.


Though Google is working on it but if we depend on Google then we may lose significant number of traffic. In this circumstances I have found a solution, which is page based indexing. This means we will index our sitemap page basis.

sitemap index problem

You can have a look on former indexing system. But this is not working right now.
  • For up to 500 posts  atom.xml?redirect=false&start-index=1&max-results=500
  • For more than 500 posts atom.xml?redirect=false&start-index=501&max-results=500
  • For more than 1000 posts atom.xml?redirect=false&start-index=1001&max-results=500

Because Google will index only 148 or 151 link under per submit. So the solution is indexing sitemap page based. Because per page Google can detect now 150 links and they index it on Google search console. You can see from below image that the number of content Google has indexed under sitemap.

indexing dropped

For re-indexing your Blog sitemap Please follow the below steps-

Step 1 Go to https://www.google.com/webmasters/tools/home?hl=en and sign in with Gmail account

Step 2 Now click on your blog to enter it to the webmaster Tools 

Step 3 From left side click on Crawl to expend and select Sitemaps 

test and add sitemap


Step 4 At the top right side click on red color Add/Test Sitemap button.

Step 5 Now add your sitemap at once like below code
  • http://www.bloggerspice.com/sitemap.xml  (Recommended Method)

If you are having problem with above method then simply add your sitemap page by page. And click on blue color Submit Sitemap button. For example

  • sitemap.xml?page=1 (This will Index 150 posts from your first page)
  • sitemap.xml?page=2 (This will Index another 150 posts from your first page)
  • sitemap.xml?page=3 (This will Index another 150 posts from your first page)
  • sitemap.xml?page=4 (This will Index another 150 posts from your first page)
  • sitemap.xml?page=5 (This will Index another 150 posts from your first page)

Note that If your Blog has 500 posts then you have to submit upto page 4. Because up to page 4 Google will able to index 600 post from a Blog.

sitemap submitted
In case of my Blog you will see my total post number 803. And I have submitted post up to Page 6. From Page 1 to Page 5 = 750 posts indexed by Google. And after submitting page 6 (e.g sitemap.xml?page=6) Google has indexed rest of the 53 posts. So now If I make any new post in my Blog then all post will be index under page 6 and when my post will reach 900 then I have to submit page 7 (e.g sitemap.xml?page=7).

I hope this will help you until a better solution of Google sitemap index pipeline. However still Google’s engineers are working hard to fix this issue.  

18 comments

  1. Nice work Bro
    Helped a lot
    Thanks for Sharing
    • Welcome Abid...glade to hear that. However Google will overcome from this problem soon.
  2. Helldo dear admin ... I am Nidhi From Mumbai .. my blogger post is almost 200 but google webmaster showing submitted URL are 150 & index URL are 150 too.. whats problem with me ?? really do not know .. please help me .. on more thing .. same problem happening for my second domain too.. after over 150 post google webmaster showing 150 only .. Note: i am using custome domain on blogger ..
    • Hi Shikha.. I have already shared the solution in this tutorial. However I think you have submitted sitemap before by using below code atom feed.

      atom.xml?redirect=false&start-index=1&max-results=500

      the above site map used for indexing up to 500 posts. But there was some problem occurred while Google engineer trying to update their server.

      For solving this issue now first remove all the previous sitemap link. and after that use below 2 sitemap link. Google will index 150 post per page submit. But you have 200 post so submit 2, this will help you to index 300 up to posts.

      sitemap.xml?page=1
      sitemap.xml?page=2

      when you will publish over 300 post then submit

      sitemap.xml?page=3

      this will allow you to index upto 450 post. I hope you would understand. If you need any further help feel free to leave comment.

      Thank You.

      :)
  3. tnx admin, I've also faced this prblm.
    • Welcome. I hope you have solve your content indexing problem. :-d
  4. আমার সাইটে কি সমস্যা বুঝতেছি না, Meta Description, Robots.txt add করলাম কিন্তু Blank দেখাচ্ছে কেন??
    SreenShot : http://i.imgur.com/CZtGfDs.jpg
    • Apnar site a meta description disable kora.. ooita enable koron. guideline er jonno nicer URL visit koron-

      http://www.bloggerspice.com/2015/12/add-search-description-in-Blog-Post-for-SEO-optimization-and-better-search-visibility.html

      http://www.bloggerspice.com/2014/12/blog-meta-title-description-for-search-engine.html

      Best of luck |o|
  5. enable করছিলাম, but auto disabled হয়ে গেল কিভাবে?
    এই সমস্যাটা ২বার হইছে।
    • After enable the feature you must click on "Save settings" button from top right corner of the screen.
  6. Hello Fazle..Salam..Brother before my sitemap is running smoothly ,but don't know from yesterday it's start showing this error "When we tested a sample of URLs from your Sitemap, we found that some URLs redirect to other locations. We recommend that your Sitemap contain URLs that point to the final destination (the redirect target) instead of redirecting to another URL." can you tell me why it's coming ? this error occurs in sitemap xml page 2 and in sitemap xml page 1 i'm using default blogger sitemap..please help me .
    • Hi Anuj. This is bit confusing. Because this error displaying in all major Blog's sitemap. But i use 1 custom redirect URL in my Blog. And also using auto refresh if my blog page open for 30 minutes at a time then it will automatically refresh the page.

      However I do not understand how adding the redirected links from site B to the sitemap in site A will remove the old links. Obviously Google can see the 301 redirect and the canonical tag but this isn't defined in the sitemap as a direct correlation between site A and B.

      But it is not affecting sitemap. Because Google has indexed all of my content. I need time to figure it out. :re
  7. Thanks Rabbi

    Its a great post and very helpful. All my post indexed with in 5 minutes. But i want to know about some thing. My main sitemap is sitemap.xml which is showing some 10 warnings and show some post in warning. When i check the link, link open fine. So what is the error and where is the error. Will you please help?

    Thanks
    • Hi Sumit,
      This problem arise from HTTP(Less secured) and HTTPS(More secured). Google custom domain doesn't support HTTPS but google hosted domain support HTTPS. So here conflicts has arises. Your Problem will be show like below on Google web master tools-


      "When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot due to network timeouts. If this problem persists, please check the network availability of your DNS and web servers. All accessible URLs will still be submitted."

      This means Googlebot become confused and can't access to those 10 files. After re-submitting your sitemap this problem me overcome soon. And this problem will be solve automatically.
      Thank you :)
  8. i submit 9 post index show 5 and this error show on sitemap give me some solution plz (/atom.xml?redirect=false&start-index=1&max-results=500)
    • Hi There please use recommended method like below

      http://www.bloggerspice.com/sitemap.xml

      I hope your problem will solve.
  9. nice blog bro. keep it up
    • Thank you :up
Go Up