Running multiple large Drupal sites on one machine has created some performance anxiety for us. Certain high traffic periods combined with aggressive web crawlers have ground our servers to a halt on a few occasions.

My latest investigation went into cleaning up popular 404 requests on our larger sites. This noticeably reduced disk I/O. I wanted to take 404 a step further and change the way Drupal 404s were handled on our machines. I turned to the Fast 404 module.

Fast 404 works by intercepting the Drupal bootstrap process and performing its own path and file checking. Fast 404 configuration and execution is performed in the site’s settings.php file.

Version 6.x-1.1 did not work immediately out of the box and required a few changes and bug fixes. I have submitted them to the module author.

The whitelisting did not seem to work so I altered line 38 from:

if (in_array($_SERVER['QUERY_STRING'],$allowed)) {
  return TRUE; // URL is whitelisted. Assumed good.
}

To:

foreach ($allowed as $needle) {
  if (stristr($_SERVER['QUERY_STRING'],$needle)) {
    return TRUE; // URL is whitelisted. Assumed good.
  } 
}

Line 111:

$sql = "SELECT path FROM menu_router WHERE '%s' LIKE CONCAT(path,'%')";
$sql2 = "SELECT pid FROM url_alias WHERE '%s' LIKE CONCAT(dst,'%')";
$sql3 = "SELECT rid FROM path_redirect WHERE '%s' LIKE source";

Change to:

$sql = "SELECT path FROM " . $db_prefix['default'] . "menu_router WHERE path LIKE '%%s%'";
$sql2 = "SELECT pid FROM " . $db_prefix['default'] . "url_alias WHERE dst LIKE '%%s%' ";
$sql3 = "SELECT rid FROM " . $db_prefix['default'] . "path_redirect WHERE '%s' LIKE source";

I also changed line 48 to remove duplicate code:

header('HTTP/1.0 404 Not Found');
$fast_404_html = variable_get('fast_404_html','<html xmlns="http://www.w3.org/1999/xhtml"><head><title>404 Not Found</title></head><body><h1>Not Found</h1><p>The requested URL "@path" was not found on this server.</p></body></html>');
print strtr($fast_404_html, array('@path' => check_plain(request_uri())));
exit();

Change to the function that calls the same code:

fast_404_error_return();

One limitation of the Fast 404 module are that it will print a generic “Page Not Found” message on a white page. We would prefer to have a themed landing page similar to Drupal’s normal 404 page. My solution was to create a static 404.html file for the site and tell Fast 404 to print the contents of the file instead of its generic message.

$conf['fast_404_html'] = file_get_contents('./sites/pathtosite/404.html');

I also wanted to keep track of the requests Fast 404 was catching. I added a direct syslog() call. If your Drupal site is not currently configured to use Syslog see my post on setup and configuration. The log format I chose fit our needs, you could format it however you like.

Around line 186, inside fast_404_error_return():

syslog(1, $_SERVER['SERVER_NAME'].' Fast_404 '.$_SERVER['REMOTE_ADDR'].' '.$_SERVER['SCRIPT_URI'].' '.$_SERVER['HTTP_USER_AGENT'].' '.$_SERVER['HTTP_REFERER']);

I think monitored my Drupal log file for false positives and added them to the white list.

Take a look at your Drupal site and see if you have a lot of 404s being served. If so, try this module out and see if load is reduced.