"Linux Gazette...making Linux just a little more fun!"

Setting Up a Web-based Archive for a Mailing List

By Lawrence Teo

1. Introduction

In this article, I'll describe how to set up a web-based archive for a mailing list. This article picks up from where I left off in my previous article, "A Quick and Easy Way to Set Up a Mailing List", also in this issue of Linux Gazette. If you haven't already done so, I highly recommend that you read that article first before reading this one. A lot of the context and content in here are derived from the methods described in that article.

1.1 The mailing list setup

First, I'll briefly describe the mailing list setup that we'll be using in this article. If you have read the previous article, this setup will be familiar to you.

Let's say that we have a mailing list called theproject@mybox.example.com, running on a Linux system called mybox.example.com. The e-mail addresses subscribed to this list are linus@mybox.example.com, alan@example.net, and esr@example.org. We set this up using our Mail Transfer Agent's aliases file (usually /etc/mail/aliases or /etc/aliases, depending on your distribution). Our mailing list has been set up like this in the aliases file:

theproject:
  linus,
  alan@example.net,
  esr@example.org

So when e-mails are sent to theproject@mybox.example.com, those e-mails will be automatically propagated to linus@mybox.example.com, alan@example.net, and esr@example.org.

2. Setting up the web-based archive

Before we get started with the instructions for setting up the web-based archive, I'll first explain what we plan to achieve. What we want to do is allow your workgroup members to access a website that will host your mailing list archive. For example, you can host your mailing list archive on http://mybox.example.com/theproject/ and make it accessible to all your workgroup members.

Having a web-based archive makes it easier and more convenient to check out what has been discussed. Also, it can act as a central location to store documents and other attachments. You can also use it as a backup in the unfortunate event that you lose your e-mails (which I hope will never happen!).

If you want to set up a web-based archive for your mailing list, you'll need:

The Apache webserver. Of course, you can use some other webserver. However, Apache is the most common webserver installed by default on almost all Linux distributions (to my knowledge), and it's also the one that I'm most familiar with. In this article, I'll use Apache 1.3.20 as the example.
Hypermail. Hypermail is a nifty program that you can use to generate web-based mail archives from a UNIX mailbox file. You can grab it from http://www.hypermail.org/. I'll use hypermail 2.1.2 for the purposes of this article.
cron. cron is a program that you can use to run tasks at scheduled times. It should be installed by default in all Linux distributions (I'll be shocked if I find a distribution that doesn't include cron!). In this article, we'll use cron to update our web-based archive periodically.

You'll first need to check if the webserver is running. Again, you can use the netstat command to do this:

lteo@mybox:~$ netstat -a | grep www
tcp        0      0 *:www                   *:*                     LISTEN

If the system responds with that line, it is likely that your webserver is already running. If it is not, you can start it by issuing the command "/etc/init.d/apache start" on Debian, the same or "/etc/rc.d/init.d/httpd start" on Red Hat, etc. In Slackware, issue the command "/etc/rc.d/rc.httpd start".

2.1 Installing hypermail

After you've downloaded hypermail, proceed to install it using the instructions in its README file. The installation steps should be pretty standard. If you're in a hurry, the following commands should work for you (they're meant for hypermail 2.1.2; substitute the version number for the hypermail version you downloaded):

root@mybox:~# tar zxf hypermail-2.1.2.tgz
root@mybox:~# cd hypermail-2.1.2
root@mybox:~/hypermail-2.1.2# ./configure
root@mybox:~/hypermail-2.1.2# make
root@mybox:~/hypermail-2.1.2# make install

2.2 Creating a dummy account

The next thing you need to do is to set up a dummy user account on your system. We will register this account on the mailing list, and use it exclusively for collecting all mails sent to the mailing list. We will then generate the mailing list archive using this dummy user account's mailbox.

Let's call our dummy account "projarc". You can create it in the same way you create a normal user account on your Linux distribution. I personally use the adduser command on my Debian GNU/Linux system:

root@mybox:~# adduser
Enter a username to add: projarc
Adding user projarc...
Adding new group projarc (1004).
Adding new user projarc (1004) with group projarc.
Creating home directory /home/projarc.
Copying files from /etc/skel
Enter new UNIX password: <password>
Retype new UNIX password: <password>
passwd: password updated successfully
Changing the user information for projarc
Enter the new value, or press return for the default
        Full Name []: Dummy user
        Room Number []:
        Work Phone []:
        Home Phone []:
        Other []:
Is the information correct? [y/n] y

You will need to add this user to your the aliases file (/etc/mail/aliases or /etc/aliases) to get it registered as a member of the mailing list. To do this, just edit the file and add the username to the list. The section that describes your mailing list members in the aliases file should look like this:

# The Project mailing list
theproject:
  projarc,
  linus,
  alan@example.net,
  esr@example.org

Remember to run that command to inform your MTA about the changes after you've done this.

We will be using projarc's public webspace to host our mailing list archive. To do that, create a public_html directory in the projarc's home directory:

lteo@mybox:~$ su - projarc
Password: <password>
projarc@mybox:~$ mkdir public_html

Note that the user's public webspace may be represented by another name instead of public_html. It depends on your webserver setup. You also need to ensure that your webserver allows users to host public webspaces in this manner. I'll explain how to enable this in the next section.

2.3 Setting up Apache

The next step is to set up Apache so that it allows users on the machine to have their own public web directories. The Apache configuration file you need to edit is /etc/apache/httpd.conf. Again, this may differ depending on your distribution. If it's not there, issue the command "locate httpd.conf" or "find / -name httpd.conf" to find it. Once you've found it, open it up with your text editor and make sure that the following lines are uncommented (meaning they don't have the # symbol in front of them):

<IfModule mod_userdir.c>
    UserDir public_html
</IfModule>

It is likely that your UserDir value is not public_html, and may be something else like www. You can use whatever directory name you wish to represent the user's webspace.

Now if you want your workgroup members to access your archive using an address such as http://mybox.example.com/theproject/ you'll need to set up a symbolic link from your Apache's root webspace to point to projarc's webspace. To find out what your Apache's root webspace is, check out the DocumentRoot value in /etc/apache/httpd.conf:

root@mybox:~# grep ^DocumentRoot /etc/apache/httpd.conf
DocumentRoot /var/www

In the above example, Apache's root webspace is at /var/www. To create a symbolic link to point to projarc's public webspace, issue the following commands:

root@mybox:~# cd /var/www
root@mybox:/var/www# ln -s /home/projarc/public_html theproject

2.4 Testing hypermail

When a user receives e-mails, the e-mails will be stored in a file called /var/mail/username. So in the case of projarc, the file will be called /var/mail/projarc (Note: in some distributions, this would be /var/spool/mail/projarc).

We can use hypermail to read that mail file to generate the web-based archive. However, when the projarc account is newly created, that file won't exist yet. So you'll need to send an e-mail to theproject@mybox.example.com first just to get that file created.

After sending out that test mail, run the following command as projarc:

projarc@mybox:~$ hypermail -m /var/mail/projarc -l "The Project" -d /home/projarc/public_html

Now open up http://mybox.example.com/theproject/ in your web browser and you should see your mailing list archive there. It should look something like this:

The Project
By Thread

Most recent messages
1 messages sorted by: [ author ] [ date ] [ subject ] [ attachment ]

Starting: Sat Oct 20 2001 - 01:45:23 EDT
Ending: Sat Oct 20 2001 - 01:45:23 EDT

This is the first message Lawrence Teo (Sat Oct 20 2001 - 01:45:23 EDT)

Last message date: Sat Oct 20 2001 - 01:45:23 EDT
Archived on: Sun Oct 21 2001 - 01:50:56 EDT

1 messages sorted by: [ author ] [ date ] [ subject ] [ attachment ]

It would be more convenient to use a hypermail configuration file to run hypermail instead of typing in all those command-line parameters all the time. To do this, create a file called /home/projarc/projarc-hmrc and fill it with the following lines:

mbox = /var/mail/projarc
label = The Project
dir = /home/projarc/public_html

You can now generate the mailing archive by running the following command:

projarc@mybox:~$ hypermail -c /home/projarc/projarc-hmrc

2.5 Setting up cron

Now, we would definitely want our mailing list archive to be automatically updated whenever somebody sends mail to the mailing list. We will use cron to do this. It won't be updated in real time, but we can set cron to run hypermail every 5 minutes, which should be frequent enough for a simple mailing list. Of course, you can always use a shorter interval such as 2 minutes; it's entirely up to you. Just remember that the shorter the interval, the more load the machine will have to handle. That may not be good if you have mail files that are really big with a lot of attachments, and you're hosting several mailing lists on a slow machine.

So, let's set up cron. Issue the following command to edit your cron table:

projarc@mybox:~$ crontab -e

You should now be in an editor with your crontab file open. If you want hypermail to run every 5 minutes, enter the following lines:

# Update The Project mailing list archive every 5 minutes
*/5 * * * * /usr/bin/hypermail -c /home/projarc/projarc-hmrc

When you're done, just save and exit. To test it, just wait for five minutes and refresh the http://mybox.example.com/theproject/ web page on your browser. You should see the updated "Archived on" timestamp after hypermail runs.

2.6 Summary

Here's a summary of the steps we used to set up our web-based mailing list archive:

Make sure that your Apache webserver is up and running. You can use the command "netstat -a | grep www" to check this.
Install hypermail on your system.
Create a dummy account to host your mailing list archive, e.g. projarc.
Add the dummy account's username to the aliases file (/etc/aliases or /etc/mail/aliases). Run the "newaliases" (for sendmail) or "postalias /etc/aliases" (for Postfix) command to inform the MTA about the changes. You don't need to run any command for exim.
Make the public webspace directory for the projarc user, e.g. "mkdir public_html"
Set up Apache to allow public webspaces for users in the /etc/apache/httpd.conf file.
Set up a symbolic link to point to projarc's webspace, e.g. "ln -s /home/projarc/public_html theproject"
Test hypermail by running a command such as 'hypermail -m /var/mail/projarc -l "The Project" -d /home/projarc/public_html'. Then check "http://mybox.example.com/theproject/" to see if your mailing list archive is there.
Set up cron to run hypermail every 5 minutes (or at any interval you wish).

I hope that you'll find the web-based mailing list archive to be as useful as I have. If you have any comments or suggestions, please feel free to drop me some e-mail.

Lawrence Teo

Lawrence Teo is a Ph.D. student at the University of North Carolina at Charlotte. He researches on intrusion detection and critical infrastructure protection technologies with his research unit, the Laboratory of Information Integration, Security, and Privacy (LIISP). Lawrence has previously worked as a contract software engineer at Lycos, Singapore and as a research assistant at DSTC in Melbourne, Australia. He holds an Honors Degree in Bachelor of Computing from Monash University in Melbourne, Australia. You can send him e-mail at lawrenceteo<SPAM>@lycos.com.