Warning: XML SiteMap Generator Plugin for WordPress May Report Fewer URL’s

Arne Brachold’s popular XML Sitemap generator plugin for WordPress may report fewer URL than that exists on your website. This in turn means less URL’s which you are submitting to search engines for indexing.

I found out that in my case 25% of the actual number of url’s (posts) were being reported. I haven’t investigated the cause. However when I changed the option to use Standard SQL, it is now reporting the URL’s fine.

Another big problem with XML Sitemap generator plugin is that it write to the file directly and incrementally, so if Google or other search engines access the file while it is still writing, well you are in trouble. It is will read a smaller and improperly formatted file. A better approach would be to write the file elsewhere and swap it as an atomic operation.

How can you find out the number of post url’s reported and the actual number of posts?
Run a simple SQL query against your MySQL database to get the number of published posts in your database:
select count(*) from st_posts where post_status = 'publish' and post_type='post';

Now grep sitemap.xml to find the post url’s reported. For example for this blog, each post has the word ‘/acrhive/’ prefixed in the url. So my Linux command was:
grep '/archive/' sitemap.xml|wc -l

Note: wc -l give the number of lines.

You can similarly verify the categories and tags report too. However I don’t think they are as important.