Martian Thoughts on Design

I’ve been working on understanding the Drupal CMS for the past couple of weeks, and it can be a little bit confusing.  As usual with an open source project most of the documentation focuses on Unix/Linux or Windows.  There is some good information for Mac users, but for me, it seemed a little disjointed and was hard to bring together for a coherent setup.  This is my attempt to explain a complete setup, from start to finish, for doing web development on my local machine running OS X 10.4.  I’m completely new to Drupal, so if I could have done something better here, please let me know.

Let me start by saying that all of this information was gathered from the Drupal documentation.  I’ll try to give references to those sources as I go along, and point out where I made slightly different choices than the authors of those sections. Let’s look at what we’re trying to accomplish.  I really like the Multi-Site capability of Drupal.  It’s awesome that you can work with several websites, while only having to install the Drupal code once on your machine.  So:

• We’re going to use Drupal’s Multi-Site capability.
• We’re going to have a separate database for each site that we use.
• We’re going to host Drupal in our local user’s Sites/ folder, instead of in the master web folder.
• We’re going to set up Virtual Hosting on our webserver to help ourselves out.

Assumptions: You have MySQL 5.0+ and PHP 5.2+ already set up on your machine. That should be all you need.  Some of this could definitely be done easier with additional tools, but I’m trying to make sure that everyone can follow along with this, whether they have tools like PHPMyAdmin or MySQL Workbench installed or not.  So I’m going to be working a lot on the command line in the Terminal.  Don’t let this intimidate you, I try to take it step by step. Ok, so let’s get started.

Step 1: Setting up the database

We’ll start by setting up the database for our first installation of Drupal.  You can find some additional information about installing the database for Drupal here.  We’ll skip some of the unnecessary details like flushing privileges, because MySQL handles that for us with the GRANT statement. Open a Terminal window, and create a new database.  In this case we’ll call our database “drupal_sandbox”, but you can call it whatever you like.

$ mysql -u yourrootaccount -p
mysql> CREATE SCHEMA drupal_sandbox;

Notice that I put yourrootaccount as the username when launching the mysql client.  This is an account which has privileges to CREATE databases and GRANT privileges.  In most case this user will be root, but when I install MySQL on my systems, I always change the root user to a different username, as a simple security measure.  This follows with the OS X methodology of not enabling the root account for Unix.  In the future I’ll post an article on my MySQL installation procedures.

Now we’ll create the application user for the database.  You want to create a separate username for the application, to prevent any hacker who cracks your application from having total access to the database.  In SQL users are created by using the GRANT statement, which has the form GRANT privileges ON dbname.* TO ‘username‘@’location‘ IDENTIFIED BY ‘password‘.  We need to use the same database name we created above, our username will be “drupal_app“, and the password will be “fuzzy“.

mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER
    ->   ON drupal_sandbox.*
    ->   TO 'drupal_app'@'localhost'
    ->   IDENTIFIED BY 'fuzzy';

That’s it.  Drupal will install all the database tables during installation. Exit MySQL by typing the following:

mysql> quit;

Step 2: Add a new host entry to the local machine

Next, let’s set up the Apache webserver that comes with OS X 10.4.  For OS X 10.5+ these instructions will be slightly different, since it uses Apache 2.2.  The basics for this virtual host setup can be found in the Drupal documentation here, which does a good job of describing the Apache 2.2 setup.

We’re going to add an entry to /etc/hosts, so that a request to DOCUMENT_ROOT will point to our Drupal installation in ~/Sites. Drupal would normally expect our installation to be in the /Library/Webserver/Documents folder, but I prefer not to clutter that up with all my development work.  Open a Terminal window, and type the following:

$ sudo nano /etc/hosts

Add the following lines to the end of the file:

##################################
# Virtual Hosts section
##################################
127.0.0.1    drupal6.local

Save the file. This is similar to what was done in the documentation, but we’re changing the hostname to drupal6 to allow for greater flexibility in the future, when we install Drupal 7.x.  In that case, we’ll probably want a drupal7.local path.

Step 3: Edit the Apache configuration files

Begin by editing httpd.conf located in /etc/httpd.  This file will be in /etc/apache2 on OS X 10.5+.

$ cd /etc/httpd
$ sudo nano httpd.conf

Verify that the Rewrite module is enabled.  Make sure the following line is uncommented:

LoadModule rewrite_module        libexec/httpd/mod_rewrite.so

Again, this will be somewhat different for Apache 2 which is on OS X 10.5+.  We just want to make sure that we are able to use .htaccess files. Enable the following line by removing the # in front of it:

NameVirtualHost *:80

Scroll to the end of the file and verify that the following directive is enabled:

Include /private/etc/httpd/users/*.conf

This will enable us to add our virtual host directives in a separate file in the users directory.  On OS X 10.5+ you’ll be looking for something in the httpd/extra/ folder.  Close the httpd.conf file.  We’re going to add a file called httpd-vhosts.conf to the users directory.

$ cd /etc/httpd/users
$ sudo touch httpd-vhosts.conf
$ sudo nano httpd-vhosts.conf

Add the following lines to this configuration file (obviously changing yourusername to your user name on your local system):

#
# Virtual hosts
#
 
<VirtualHost *:80>
    DocumentRoot "/Library/WebServer/Documents"
    ErrorLog "/private/var/log/httpd/error_log"
    CustomLog "/private/var/log/httpd/access_log" common
</VirtualHost>
 
<VirtualHost *:80>
    ServerAdmin webmaster@drupal6.local
    DocumentRoot "/Users/yourusername/Sites/drupal/6"
    ServerName drupal6.local
    ServerAlias *.drupal6.local
    ErrorLog "/private/var/log/httpd/drupal6.local-error_log"
    CustomLog "/private/var/log/httpd/drupal6.local-access_log" common
    <Directory "/Users/yourusername/Sites/drupal/6/">
        Options Indexes FollowSymLinks
        AllowOverride All
        Order allow,deny
        Allow from all
    </Directory>
</VirtualHost>

As stated in the article listed above, the first section needs to match what is in your httpd.conf file.  The second section sets up our Drupal virtual host.  In addition to what was described in that article, we’ve added the <Directory> section.  This change is important because we want to make sure that SymLinks are followed, and we need the AllowOverride directive to make sure that our .htaccess file is read. Open your System Preferences, select Sharing, stop and restart Personal Web Sharing.  Ok, so now we’re ready to move on.

Step 4: Create the directory for your local copy of Drupal

Navigate to your local Sites/ directory and create a folder for Drupal.  You can do this through Finder or on the command line:

$ cd ~/Sites
$ mkdir drupal
$ mkdir drupal/6

Our installation of the Drupal files will go inside the 6/ sub-folder. When Drupal 7.x is released, we’ll add a drupal/7/ folder for the installation of it.

Step 5: Test our installation so far

Navigate to your Sites and open up the drupal/6/ folder with Finder.  Create a new file, and name it index.php.  Open this file with the text editor of your choice, and insert the following code:

<?php
    echo "<h1>You are in the Drupal 6 folder</h1>";
?>

Save the file and in your web browser enter http://drupal6.local/ and hit return.  If you see a page with “You are in the Drupal 6 folder“, then your virtual host configuration is working correctly.

Step 6: Install the Drupal files

We’ll be using Finder for the first part of this step, but keep the Terminal handy for the second part.

• Open your Sites/ folder
• Open the drupal/ folder and move the 6/ folder we created in Step 4 to the Trash
• Download the latest copy of Drupal (6.13 at the time of this writing)
• Find the file you downloaded, and move it into your Sites/ folder
• Double click the compressed file you downloaded to extract the contents
• You should now have a folder like drupal-6.13/ in your Sites/ folder
• Drag drupal-6.13/ onto your drupal/ folder
• Open the drupal/ folder and rename drupal-6.13/ to 6/

Now switch back to the Terminal so we can verify that we have all the files we need.  Type the following:

$ cd ~/Sites/drupal/6
$ ls -la

Verify that the list contains the .htaccess file.  This file is important to have.

Step 7: Setting up the sites/ directory and configuration script

Now we’re ready to set up our configuration script.  All we’d really need to do now, would be to copy the configuration script in the sites/default/ directory in our drupal installation, but we’re going to change things up a bit to help ourselves out with setting up the MultiSite cron jobs later on. In the Terminal, navigate to our installation folder and then into the sites/ sub-folder as follows:

$ cd ~/Sites/drupal/6/sites
$ mkdir drupal6.local

While we’re at it, let’s go ahead and make the directories for site specific themes and modules.

$ mkdir all/modules
$ mkdir all/themes
$ mkdir drupal6.local/modules
$ mkdir drupal6.local/themes

We need to copy the default.settings.php file, in the sites/default/ directory, into the drupal6.local/ directory we just created.  We also need to rename the copy settings.php.  We can take care of both of those items with the following command.

$ cp default/default.settings.php drupal6.local/settings.php

The Drupal documentation tells you that you need to set permissions for the settings file.  We won’t be doing that explicitly, because our next step will take care of it.

Step 8: Giving ownership to the webserver

The web server will need access to the files in these directories, so we’re going modify the access privileges for the ~/Sites/drupal/6/ directory.  We’re also going to change the group of this directory to www, which is the web server.  You can do this through the Finder by selecting the 6/ folder and pressing Command-I, and editing the Ownership & Permissions info, but it’s simple to do from the command line.  So type in the following:

$ cd ~/Sites/drupal/6/sites
$ chmod g+w drupal6.local
$ chmod g+w drupal6.local/settings.php
$ sudo chgrp -R www *

Step 9: Installing Drupal

We’re finally ready to set up our installation of Drupal.  Open your web browser of choice, and enter the following in the address bar:

http://drupal6.local/

This should start the installation process.  If you don’t see the Drupal logo and “Choose a Language” on the webpage, go back and check your settings.

• Select Install Drupal in English
• Leave the Database type at mysqli
• Enter the database information you used in Step 1 above
• Leave the Advanced database options alone unless you need to use them
• Hit save and continue, and the site will install

The final process of the setup  asks you to Configure the site.  You’ll enter a Site name, username for your administrator, and some email addresses for automated notifications.

• Site name: Drupal Sandbox
• Site e-mail address: youremail@yourdomain.com
• Administrator Username: whatever you like (admin)
• Admin email address: youremail@yourdomain.com
• Password: yourpassword
• Set your timezone
• Enable Clean URLs if possible
• Check for updates

Now change the permissions of the drupal6.local/ folder and the settings.php file back to not writable

$ cd ~/Sites/drupal/6/sites
$ chmod g-w drupal6.local
$ chmod a-w drupal6.local/settings.php

That should be it.  Click on “your new site” to see that the webpage is functioning properly.  If you look under “Administer” you should see a warning that “Cron has not run”. We’ll take care of that next.

Step 10: Configuring cron

Running a cron process is extremely important for Drupal.  There are multiple functions for which Drupal uses the cron process, including: indexing content, pruning logs, checking for updates, and others.  The simple way to get this to work would be to just update the systems crontab to update the site, as described in Configuring cron jobs.  However, since we’re installing a multi-site system, and we don’t want to have to remember to update the crontab each time we add a site, we’re going to use a script to automate this process. 

Again, the Drupal documentation has many examples of scripts, using bash, php, python and more.  I encourage you to take a look at them. I made my own here, because I either didn’t understand what the ones on the Drupal site were trying to do, or they didn’t seem flexible enough, but most of them looked like they might not cover all the likely possibilities.

There was a pretty good one posted by Carl van Denzen, that had a similar methodology to this script, but it had external dependencies.  So I just dove in and tried to come up with my own solution. (As an option, I’ve posted a copy of the file for you to download, so you don’t have to type it all in).

Open a new file in your text editor and save it as cronall.php in the drupal/6/ folder.  Add the following to this file:

<?php
    /**
    *   This script will call a cron job for each valid Drupal site
    *   in your installation.  Whether you have one site or many in a
    *   Multi-Site configuration.  It will find all folders contained
    *   in the "sites" folder of your code base which contain a
    *   settings.php file.  These are assumed to be valid sites.
    *   This script should be put in the top level of your code base.
    *
    *   Note: It is assumed that you have properly configured your
    *   webserver to map each top level directory in "sites" to an
    *   equivalent VirtualHost.
    *
    *   This script will take these folders and transfer them to URLs.
    *   For example:
    *
    *       sites/drupal6.local        ==>   http://drupal6.local/cron.php
    *       sites/drupal6.local/site1  ==>   http://drupal6.local/site1/cron.php
    *       sites/bunnies.local        ==>   http://bunnies.local/cron.php
    *
    *   It is a "Best Practice" to use a directory for your master
    *   domain, instead of using the 'default' folder.  However, if
    *   you use the 'default' folder, the value in $masterDomainName
    *   will be inserted to replace 'default' in the URL.
    *
    *       sites/default          ==>   http://drupal6.local/cron.php
    *       sites/default/site1    ==>   http://drupal6.local/site1/cron.php
    *
    *   For any sites not picked up by this script automatically, you can
    *   manually specify the path in the $addedSites array.
    *
    *   Notice:  This script is provided "As Is".  No warranty is expressed
    *            or implied.  You may freely modify/use it as you like.
    */
 
    $sitesDir = "./sites";
    $cronURL  = 'http://%s/cron.php';
 
    //  any additional sites not picked up by this script may
    //  be manually inserted here, just uncomment $addedSites
    //$addedSites = array('insertDomainPathHere');
 
    $masterDomainName = 'drupal6.local';
 
    $sites = array();
 
    //  populate the $sites array by calling find_sites()
    find_sites( $sites, '' );
 
    //  If $addedSites is not empty and it is an array,
    //  merge it with $sites
    if ( !empty( $addedSites ) && is_array( $addedSites ) )
    {
        $sites = array_merge( $sites, $addedSites );
    }
 
    //  remove any duplicate entries in $sites
    //  this is in case the 'default' directory
    //  and a $masterDomainName directory both exist
    $sites = array_unique( $sites );
 
    //  for each site in $sites, execute a cron command
    foreach ( $sites as $site )
    {
        //  Use ONE of the following command lines
        //$cmd = '/usr/bin/lnyx -source ' . sprintf( $cronURL, $site );
        //$cmd = 'wget -O - -q -t 1 ' . sprintf( $cronURL, $site );
        $cmd = '/usr/bin/curl --silent --compressed ' . sprintf( $cronURL, $site );
 
        echo "Executing command: $cmd<br/>\n";
        exec( $cmd );
    }
 
    /**
     *   Recursively find all valid Drupal sites
     *
     *   This method recursively looks in the $sitesDir directory
     *   and will find all folders that include a settings.php
     *   file.  These are assumed to be valid/active sites in
     *   the Drupal installation.  Any folder containing the
     *   settings.php file will be added to the $sites[] array,
     *   which is passed into the function by reference.
     *
     *   If the 'default' folder is used, the value in $masterDomainName
     *   will be substituted into the path in the $sites[] array.
     *
     *   @param  string[]  &$sites  a string array containing all the sites.
     *                              This parameter is passed by reference, so
     *                              changes will persist outsite the function.
     *   @param  string   $dirPath  a string containing the directory path
     *                              to traverse
     *   @return void
     */
    function find_sites( &$sites, $dirPath )
    {
        global $sitesDir;
        global $masterDomainName;
 
        if ( $dh = opendir( "$sitesDir/$dirPath" ) )
        {
            while ( ( $filename = readdir( $dh ) ) !== false )
            {
                //  must account for the empty $dirPath the
                //  first time through the function
                if ( !empty( $dirPath ) )
                {
                    $subDirPath = "$dirPath/$filename";
                }
                else
                {  //  This is not the first time through,
                   //  just add the filename
                   $subDirPath = $filename;
                }
 
                //  ignore the 'all' directory, the '.' and '..' directories
                //  also ignore regular files
                if ( ( $filename != 'all' ) && ( $filename != '.' ) &&
                     ( $filename != '..' ) && is_dir( "$sitesDir/$subDirPath" ) )
                {
                    //  if the directory contains a settings.php file, it is
                    //  a valid site.  we need to add it to the $sites[] array
                    if ( file_exists( "$sitesDir/$subDirPath/settings.php" ) )
                    {
                        //  if the path contains 'default', replace 'default'
                        //  with $masterDomainName and insert it into $sites[]
                        //  need to keep 'default' in $subDirPath
                        //  so we can navigate the tree
                        if ( strstr( $subDirPath, 'default' ) )
                        {
                            $sites[] = substr_replace( $subDirPath, 
                                                       $masterDomainName,
                                                       0, strlen( 'default' ) );
                        }
                        else
                        {
                            $sites[] = $subDirPath;
                        }  //end if else
 
                        //  enter the next loop of recursion
                        //  pass find_sites the current dir path
                        find_sites( $sites, $subDirPath );
 
                    }  // end if
 
                }  // end if
 
            }  // end while
 
        }
        else
        {
            die( "Unable to find directory $sitesDir/$dirPath" );
        }  // end if else
 
        closedir( $dh );
 
        return;
 
    }  // end function find_sites()
 
?>

Save the file. Now we need to make sure that our cronall script is called automatically by the system. You have 2 options for making this happen:

• Set up cron to call cronall once per hour, as described in the rest of this Step
• Use OS X’s launchd to automate the task, as I described in my subsequent post Automating Drupal cron with OS X’s launchd

If you’ve chosen the more Unix like approach, using cron, then we’re first going to set the default editor to nano. We’re also going to edit the cron script and run it as the user www, for security reasons. So, switch back to the Terminal and do the following:

$ export EDITOR=nano
$ sudo crontab -u www -e

Add the following line to the end this file:

0 * * * * /usr/bin/curl --silent --compressed http://drupal6.local/cronall.php

That first position is what minute of each hour you want the script to run (0-59).  So if you prefer to go at 30 min after the hour, you would subsitute 30 for 0.  You can also do a little bit of testing by changing the first number to */1, or */2.  This will set the cronall script to run every minute or every other minute respectively.  Navigate in your Drupal website to Administer > Reports > Status report and update every couple of minutes to verify that cron is running.  Just remember to set it back to a whole number when you are done.

Step 11: Decide how to organize additional sites

Ok, so now we’re ready to add additional sites to our system.  The first thing to do is decide what structure you want to use.  You have basically three options.  You can:

1. Host the site as a subdirectory to your existing site (eg. http://drupal6.local/site1/)
2. Create a subdomain to the existing site (eg http://sub1.drupal6.local/)
3. Or you can create an entirely different domain (eg http://bunnies.local/)

We’re going to cover the 2nd option.  But if you want to do the 1st option, you basically create a new database with appropriate user privileges, add a folder named ‘site1’ in the drupal6.local/ folder, copy the default.settings.php to the new directory as before, and point your browser at http://drupal6.local/site1/ For more details about the 3rd option, you should be able to look at Setting up multi-site Drupal 5 on Windows or Linux, and figure out how to mash it up with these instructions.

In my case, I’m going to use the new site to work through the Drupal Essentials training by Tom Geller over at Lynda.com, so I’ll use that when choosing a naming scheme for this additional site.

Step 12: Adding a database for your Multi-Site

Now we’ll add our new database.  We follow the same process as in Step 1, except we’ll just use the same application user from above.  If you want to create a new user, just follow the GRANT procedure from before.  Switch to the Terminal, and enter the following:

$ mysql -u yourrootaccount -p
mysql> CREATE SCHEMA drupal_essentials;
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER
    ->   ON drupal_essentials.*
    ->   TO 'drupal_app'@'localhost';
mysql> quit;

Step 13: Update the hosts file

We’re going to modify /etc/hosts again so that our machine recognizes a call to http://essentials.drupal6.local/ is really just pointing to the local machine.  Go to your Terminal window and type the following:

$ sudo nano /etc/hosts

Add the following entry at the end of the file:

127.0.0.1     essentials.drupal6.local

Save the file.  We’ve already taken care of configuring the Virtual Hosts, so this should be all we need.

Step 14: Setting up the sites/ folder

Now we need to set up the sites/ folder like we did before.  We’ll breeze through it, but this is almost identical to process covered in Steps 7-8 above, if you want to look back at any of it.  In your Terminal do the following:

$ cd ~/Sites/drupal/6/sites
$ mkdir essentials.drupal6.local
$ mkdir essentials.drupal6.local/modules
$ mkdir essentials.drupal6.local/themes
$ cp default/default.settings.php essentials.drupal6.local/settings.php
$ chmod g+w essentials.drupal6.local
$ chmod g+w essentials.drupal6.local/settings.php
$ sudo chgrp -R www essentials.drupal6.local

Step 15: Installing the new Drupal site

To install the new site you should open your web browser of choice and type in: http://essentials.drupal6.local/ Follow the same procedures as listed in Step 9.  Again, when we are done, we should remove the write permissions from the essentials.drupal6.local/ folder and the setings.php file.

$ cd ~/Sites/drupal/6/sites
$ chmod g-w essentials.drupal6.local
$ chmod a-w essentials.drupal6.local/settings.php

Step 16: Repeat

Congratulations! You’re done. Really, that’s all there is to it.  Turn off your computer!

You can repeat Steps 11-15 as many times as you like, and set up as many sites as you need to work on.  Each one can have it’s own unique modules and themes. I hope you’ve gotten something useful out of this tutorial.  I know it’s rather long.  There are obviously some simpler ways to do some of this stuff, but I wanted a system that was pretty flexible, robust and easy to expand.  Hopefully I’ve accomplished that. Enjoy!

[Update] When you get ready to update your Drupal installation, you can check back here for my other tutorial, Updating Drupal Multi-Site on OS X.