We were having a problem with google webmaster verification of a website; it constantly said:

“Our system has experienced a temporary problem”

It turned out that a misconfiguration of wordpress on that site in the main .htaccess file was producing error 500 codes instead of error 404 codes for http requests for non-existent pages.  You can see if you have this problem just by trying to load “/nonexistent.html” for your website.  If that reports error 500, that is the problem you need to fix in order to make google happy, and the place to look is the top level .htaccess page.  In our case, a fantastico installation of wordpress had added the following:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

which wasn’t very helpful given that the wordpress installation was not in that directory at all…

I’ve a tendency to use wget for everything. Here is one of many ways to check if a page contains dead links:

wget -k http://your.site/page.html
wget –spider –force-html -i page.html

(of course there are plenty of online services for this too).