Using 404 errors to serve up virtual pages and virtual categories

About this document

This page documents how FreshPorts can use the 404 error process to serve up virtual pages.

It is assumed that the reader is familiar with the FreeBSD ports tree and how it works.

Background

The FreeBSD ports tree contains several thousand port skeletons. These skeletons are used to download, compile, and installed software from source. For ease of use, these ports are divided into different directories called categories.

FreshPorts uses the PostgreSQL RDBMS.

FreshPorts allows a user to broswe through the website in much the same way as they might move around in a directory tree. In effect, the website mirrors what the user might find on disk but with enhanced information and presentation.

For example, on disk the user would find the following:

[dan@m20:/usr/ports] $ ls
INDEX           distfiles       news
INDEX-5         editors         palm
INDEX.db        emulators       picobsd
LEGAL           finance         portuguese
MOVED           french          print
Makefile        ftp             russian
Mk              games           science
README          german          security
Templates       graphics        shells
Tools           hebrew          sysutils
archivers       hungarian       textproc
astro           irc             ukrainian
audio           japanese        vietnamese
benchmarks      java            www
biology         korean          x11
cad             lang            x11-clocks
chinese         mail            x11-fm
comms           math            x11-fonts
converters      mbone           x11-servers
databases       misc            x11-toolkits
deskutils       multimedia      x11-wm
devel           net
[dan@m20:/usr/ports] $

Most of the above directories are categories. Exceptions include INDEX, Mk, and README. For this dicussion, we will ignore the non-categories which appear in the /usr/ports directory.

For example, the shells directory contains:

[dan@m20:/usr/ports/shells] $ ls
44bsd-csh       mudsh           scponly
Makefile        nologinmsg      tcsh
bash1           osh             vshnu
bash2           pash            wapsh
es              pdksh           zsh
esh             perlsh          zsh+euc_hack
fd              pkg             zsh-devel
flash           rc
ksh93           sash
[dan@m20:/usr/ports/shells] $
None of the above directories actually appear on the webserver. The website takes advantage of the ErrorDocument directive in the Apache webserver. This is the entry found within the FreshPorts website definition:
ErrorDocument   404 /missing.php
Should an page be requested which is not found, missing.php is invoked. The code within this page determines which category and/or port is being requested. This is accomplished using $_SERVER['REQUEST_URI'] and database queries to establish which category and/or port is being requested.

Processing the REQUEST_URI

The process used to determine the category and/or port needs to be improved and simplified. The process can be express by this pseudo-code:

read list of categories
IF category exists THEN
   IF port name is supplied THEN
      IF port exists THEN
         display details for port
      ELSE
        display 'port not found' page
      END IF
   ELSE
      display ports within this category
   END IF
ELSE
   display 'category not found' page
END IF

Virtual categories

So far we have dealt only with categories which correspond to a physical subdirectory within the ports tree. There are some categories, referred to as virtual categories that do not have a corresponding subdirectory in the ports tree. For more information on virtual categories, please see Current list of categories in the FreeBSD Porter's Handbook.

Here is entry from the Makefile for databases/zpygresqlda which specifies the categories for that port:

CATEGORIES=     databases www zope

The first category listed is the subdirectory which contains the files for this port. The other categories may be either real or virtual, but the first category must always be the real or physical directory.

The physical directory for each port is stored within the ports table. The table also contains a categories field which contains the value extracted from the Makefile (e.g. for this port, the field would contain databases www zope). Have a look at databases/zpygresqlda to see how this information is used. Look for the entry titled Also listed in.

The proposed solution for

Existing implementation of categories

The FreshPorts database contains a ports table and a categories table. The system caters for a port which resides within one physical category but does not cater for virtual categories. In brief, these tables look like this:

Categories table

Field nametype
idint
nametext
is_primaryboolean

Ports table

Field nametype
idint
category_idint

The category_id field in the ports table contains a foreign key reference to the categories table.

Proposed implementation of virtual categories

It is proposed that we introduce a new table, ports_category which will look similar to this:

ports_category

Field nametypereferential integrity
port_idintlinked to ports table
category_ididlinked to categories table

For databases/zpygresqlda, there will be three entries in this table, one for each category.

This table will be populated via a rule (as opposed to a trigger) on the ports table. For each update, insert, and delete on the ports table, the rule will amend the ports_category table accordingly.

The same rule could be used to maintain the categories table. This would ensure that new virtual categories are added/removed as/when necessary.

This solution should make it fairly each to serve up virtual categories. More investigation is needed to consider the possible implications for other parts of the system.

Where to from here

Feedback please, preferably by posting in the Website Feedback or via email to the webmaster of this domain.


Servers and bandwidth provided by
New York Internet, iXsystems, and RootBSD
Valid HTML, CSS, and RSS.
Copyright © 2000-2021 Dan Langille. All rights reserved.