Alex King is responsible for a lot of great WordPress plugins but one of my favorites is his simple 404 Notifier.

404 Notifier is an admin-side plugin that logs any 404 errors your visitors encounter while visiting your blog.

The neat part of 404 Notifier is that you can subscribe to a special RSS feed that will automatically notify you of any new 404 errors.