For the second time in two weeks we've had a Drupal problem related to user access and the anonymous user. Both sites are running Drupal 4.6.11. It goes something like this:

Suddenly we have errors like this one:

    Fatal error: Duplicate entry '6e00c2d43650a9a6a00a23a73d5dee11'
    for key 1 query: INSERT INTO sessions (sid, uid, hostname, timestamp)
    VALUES ('6e00c2d43650a9a6a00a23a73d5dee11', 0, '69.131.184.249',
    1180063799) in
    /usr/local/share/drupal-4.6.11/includes/database.mysql.inc on line 66

Or - regular pages (like the home page) suddenly display access denied messages like:

    Access denied
    You are not authorized to access this page.

Upon looking in the database we discover that there is no longer a user with UID 0. That's the uid reserved for the anonymous user and Drupal sites will crash and burn in all kinds of weird ways without it.

Equally distrubing we discover that there is a new user (recently inserted according to the time stamp field) with a different UID, but without a name or email value. The anon user typically has no name and the name field is unique: you can't have two users with no name.

So, we get rid of the weird user and add the anonymous user with uid 0 to the users table. Then we're faced with an Access Denied message.

How the Anon user was removed and the user without a name or address was inserted is my biggest concern to prevent this from happening again. However, my immediate task is to get the site operational.

So... these are the steps I took to debug the Access Denied message.

We have a central drupal installation that all of our members use. In order to trace where the access denied message is coming from, I have to tweak the central code. I don't want to do that for all members, so my first step was to backup this site's web directory (which contains symlinks to our central Drupal installation) and then re-create their web directory with a copy of the Drupal code. This way I can tweak the drupal code without affecting other members.

Next, I added a function to the settings.php file:

    function mf_output($output,$die = false)
    {
      if($_SERVER['REMOTE_ADDR'] == '1.2.3.4') print_r($output);
      if($die) die();
    }

This function will let me output debugging info, but just for me (or anybody else at my IP address). Replace 1.2.3.4 with your IP if you want to do this.

Then, I grepped the code for "access denied":

    grep -ir "access denied" *

I found a solid reference in include/common.php. There's a function called drupal_access_denied that seems to control that feature.

Next I looked for references to that function:

    grep -ir "drupal_access_denied" *

Lots of modules use this function. I first checked out modules/node.module. Before each call of that function I added:

    mf_output(1);

I then reloaded the browser - no luck. The message wasn't coming from the node module.

Then, I discovered that index.php calls it. I used the same technique and discovered that the function menu_execute_active_handler is returning "MENU_ACCESS_DENIED". Wonder why that is happening. The function menu_execute_action_handler is in the includes/menu.inc file.

The relevant code is:

    if (!_menu_item_is_accessible(menu_get_active_item())) {
      return MENU_ACCESS_DENIED;
    }

So I continued looking for the _menu_item_is_accessible function.

The first thing it does is build a menu array with:

    $menu = menu_get_menu();

Dang. This is a complex function that builds the menu from all the modules.

I started by inserting this in the function:

    mf_output($menu,true);

That produced an insanely huge array. The _menu_item_is_accessible function seems to look for a key in this array called 'access.' The insanely huge array has the key set to either 1 or it's unset (in most cases). Where is that supposed to be set?

At this point, I decided to check out the user.module, which is the module that defines what users have access to what permissions. In the course of looking at the user_access function, I noticed it pulling from the users_roles table. When I looked more closely at this table, I realized that there was no entry linking the anonymous user (uid 0) to the anonymous role (rid 1). So I added a record to the users_roles table with uid 0 and rid 1. Then, I emptied the cache table and now it seems to be working again.

This seems to support the theory that the anonymous user was in fact deleted by drupal code - because the users_roles entry was elegantly removed as well, as opposed to the new weird user be created via an update of the existing anonymous user.