MeTA1 README

Claus Aßmann


Contents

Introduction to MeTA1

MeTA1 is a Message Transfer Agent

This distribution contains the source code for MeTA1 which implements a message transfer agent (MTA). It supports the Simple Mail Transfer Protocol (SMTP) as specified by RFC 2821 [Kle01] and various extensions, e.g., STARTTLS [Hof99], AUTH [Mye99], PIPELINING [Fre00], as well as other protocols, e.g., LMTP [Mye96].

MeTA1 is intended to be used as a secure and efficient mail gateway. It does not provide any mail content modification capabilities, e.g., masquerading of addresses or changing (addition, removal) of headers. Later versions will probably add such capabilities.


Main Components of MeTA1

MeTA1 is a modularized message transfer agent consisting of five (or more) persistent processes, four of which are multi-threaded. A central queue manager (QMGR) controls SMTP servers (SMTPS) and SMTP clients (SMTPC) to receive and send e-mails, an address resolver (SMAR) provides lookups in various maps including DNS for mail routing, and a main control program (MCP) starts the others processes and watches over their execution. The queue manager organizes the flow of messages through the system and provides measures to avoid overloading the local or remote systems by implementing a central control instance.

More information about each component will be given in the appropriate sections. Complete documentation and background information can be found in [Aßmb]. Section 7.1 describes the data flow in MeTA1, the following is a brief summary. Figure 1.1 shows the interaction of the various components and databases1.1. Incoming messages are accepted by an SMTP server (SMTPS) which stores the messages in the content database (CDB). The envelope information, i.e., sender and recipients, is stored by the queue manager in an incoming queue (IQDB) and written to disk to the incoming queue backup database (IBDB). For a delivery, the envelope information must be transferred into the active queue (AQ). The scheduler in QMGR takes recipient envelopes from AQ and creates transactions which are given to an SMTP client (SMTPC) for delivery. An SMTP client takes the transaction information and tries to send a message whose content is read from CDB. After a successful delivery attempt a record is written to IBDB that logs this information. The deferred envelope database (DEFEDB) is only used if a message cannot be delivered during the first attempt.

Figure 1.1: MeTA1: Overall Structure

\begin{picture}(120, 120)\epsfxsize 120mm
\leavevmode
\epsffile{overview1.eps}
\end{picture}

This version of MeTA1 does not come with a local delivery agent nor a mail submission program. See Sections 4.3.1 and 4.2 which programs can be used to achieve the desired functionality.


Documentation

The document ``sendmail X: Requirements, Architecture, and Functional Specification'' [Aßmb] provides the background about the MeTA1 design, its architecture, as well as the functional specification, and details about the implementation.

Typographical Conventions

In this documentation, a command written as

$ command

should be executed as an unprivileged user. Only a command written as

# command

should be executed as the superuser.

If a command contains components that need to be replaced by values that depend on the environment or the local configuration, then it is usually written as a macro, e.g., $LOGFILE.

A number in parentheses behind a command or function refers to the manual section, e.g., syslog(3), as it is usual for Unix documents.

Version

This document has been written for MeTA1-1.0.PreAlpha24.0, see also the greeting of the SMTP server and the version output of the main components. See Section 6.8 for information about version naming.

Current State

There are still some error conditions which may not be handled gracefully, i.e., in case of some resource problems (e.g., out of memory or out of disk space) the system may abort; however, this is common for most open source MTAs which simply abort if they are running out of memory. See Section 4.10.1 how to deal with those conditions. The software is running since 2004-01-01 as MTA on the main machine of the author without any significant problem, i.e., it never lost any mail.


Providing Feedback

Please report bugs and provide feedback either to the developers list[Aßma] (if you are subscribed) or directly to1.2:

< MeTA1 + feedback (at) MeTA1 . org >

Feedback about the code, the documentation (including typographical, syntactical, and grammatical errors, pointing out parts that are not well enough explained, etc.), as well as patches and enhancements are highly appreciated.

For the Impatient

For those who do not want to read the entire documentation, it is adviced to read at least sections 2.2 and 2.4, and the appropriate section of Chapter 4.

Building, Testing, and Installing MeTA1


Verifying the Source Code Distribution

The source code is distributed as a (compressed) tar file and is accompanied by a PGP signature file which has the same name as the tar file plus the ending .sig. To verify the integrity of the source code PGP [PGP] or GPG [Gnu] are required as well as the MeTA1 PGP signing key [MeT]:

$ gpg -verify meta1-$VERSION.tar.gz.sig
or:
$ pgp meta1-$VERSION.tar.gz.sig meta1-$VERSION.tar.gz

Further information, especially about warnings or possible errors, can be found in the documentation for PGP or GPG.


Building MeTA1

MeTA1 uses a configure file generated by GNU autoconf for configuration. Hence you can build it (after verifying and unpacking the distribution) as follows:

$ mkdir obj.$OS && cd obj.$OS && $PATHTO/meta1-$VERSION/configure $OPTIONS \
  && make && make check

Obviously you have to replace $OS, $VERSION, $OPTIONS, as well as $PATHTO. It is also possible to build MeTA1 in the source tree, however, this is discouraged:

$ ./configure && make && make check

Notes: do not run this as root; this is not just a basic security measure (only use a privileged account if it is really required), but most of the programs refuse to run with root privileges. It might be useful to save the output of these commands2.1for later inspection.


Compile-Time Configuration Options

Beside the usual configure options like --prefix a few MeTA1 specific configuration options are available:

--enable-TLS Enable check for STARTTLS support. The default is yes, i.e., configure tries to determine whether OpenSSL is available on the machine. Requires OpenSSL 0.9.6 or newer [Ope]. Note: check the OpenSSL website [Ope] for security announcement and be aware that due to the complexity of the software it may cause (security) problems.

--enable-SASL Enable check for AUTH support. The default is yes, i.e., configure tries to determine whether Cyrus SASL v2 is available on the machine. Requires Cyrus SASL version 2.1.18 or newer [Cyr]. Notes:
  1. check http://asg.web.cmu.edu/cyrus/ and http://asg.web.cmu.edu/sasl/ for security announcement and be aware that due to the complexity of the software it may cause (security) problems.
  2. If Cyrus SASL uses Berkeley DB then it is necessary that the version which has been used during compilation matches the version that it is linked against.

--with-sasl-libdir=path Path to directory containing Cyrus SASL v2 library.

--with-sasl-incdir=path Path to directory containing Cyrus SASL v2 include files.

--disable-included-bdb MeTA1 ships with a modified version of Berkeley DB 4.3.28 which is built and used by default. To use a different version of Berkeley DB (it must be 4.3, 4.2, or 4.1), e.g., one that is part of the host OS, specify --disable-included-bdb.

Notes:

  1. If you do not use the Berkeley DB version that comes with MeTA1, make sure you run all the tests. For example, with Berkeley DB 4.2.50 on OpenBSD 3.2/i386 at least one of the test programs fails and hence this combination must not be used. Moreover, if you encounter a problem using some other BDB version then you must try to reproduce the problem with the shipped version before reporting a possible bug.
  2. Do not use Berkeley DB 4.3.27/28 in 64 bit mode on Solaris 5.8/9 as it crashes at least in those configurations2.2. This bug is fixed in the version that is distributed with MeTA1.

--with-bdb-libdir=path Path to directory containing Berkeley DB library. This option is only needed if --disable-included-bdb is used and Berkeley DB is not installed in a location that the compiler or linker use by default. Note: configure currently checks only for a static library.

--with-bdb-incdir=path Path to directory containing Berkeley DB include files. This option is only needed if --disable-included-bdb is used and Berkeley DB is not installed in a location that the compiler uses by default. Example:
$ B=/usr/local/BerkeleyDB.4.3
$ $PATHTO/meta1-$VERSION/configure --with-bdb-libdir=$B/lib \
  --with-bdb-incdir=$B/include --disable-included-bdb

--enable-pmilter Enable policy milter protocol, see Chapter 5.

--enable-msp Enable a simple mail submission program (MSP) that is currently not supported (located in contrib/). This is just a helper program for those who do not want to install a different MSP but need only some basic functionality (which does not include a queueing mechanism). Note: this will install the MSP as sendmail thus overriding any existing program of that name (as well as a man page).

--enable-tinycdb Enable support for cdb map type, based on tinycdb 0.75 [Tok].

To get the current list of configuration options, use configure --help.


Test Programs

$ make check

will run all test programs; currently those tests take about eighty minutes to run on a standard workstation. For each of the test programs one line is printed to denote whether the test succeeded, i.e., the output consists of lines with the marker PASS: or FAIL: and the name of the test program program. Additional output might be generated by the test programs themselves, e.g.,

2 of 2 tests completed successfully,

or some debug output. The debug output may even indicate an error, but only a final FAIL: indicates a test failure. Some tests depend on compilation options and are only conditionally enabled; others may depend on environment variables, see 2.3.1. For disabled tests SKIP is shown.

Since some of the tests may fail (see Section 2.3.2) and make will usually stop after encountering an error, it might be required to use

$ make -i check >check.out 2>&1

to perform all tests.


Environment Variables used by Test Programs

Environment variables can be used to disable some test programs if required or change the behavior of some test programs. These environment variables and their effects are:

In this example the DNS timeout is set to 60 seconds and tests that take a very long time are disabled:

$ MTA_DNS_TIMEOUT=60
$ MTA_NO_SLOW_TEST=1
$ export MTA_DNS_TIMEOUT MTA_NO_SLOW_TEST
$ make -i check


Known Test Program Problems

Note: DNS related test programs may fail if the first nameserver entry in /etc/resolv.conf does not respond properly (and reasonably fast) to DNS queries. See Section 12.3.1 how to override the default nameserver selection: MTA_NAMESERVER.

For more information about possible test program problems see Section 12.3.2. For problems with programs in the contrib/ directory, see contrib/README.


Installing MeTA1

MeTA1 needs several users to provide separation of privileges and to enhance security. Currently there are four required accounts (the numbers for uid and gid are examples only); the last one listed below (meta1) is not really required:

meta1s:*:260:260:meta1 SMTPS:/nonexistent:/sbin/nologin
meta1q:*:261:261:meta1 QMGR:/nonexistent:/sbin/nologin
meta1c:*:262:262:meta1 SMTPC:/nonexistent:/sbin/nologin
meta1m:*:263:263:meta1 misc:/nonexistent:/sbin/nologin
meta1:*:264:264:meta1 other:/nonexistent:/sbin/nologin

with the corresponding groups:

meta1s:*:260:
meta1q:*:261:
meta1c:*:262:meta1s
meta1m:*:263:meta1s,meta1q
meta1:*:264:

Note: on some operating systems the star character is not a valid value for the password field. Check passwd(5)2.3to determine which value to use to disable the password.

To check whether the required users and groups exist, run

$ ./misc/sm.check.sh -p

(in the build directory); see below how to override the default values for the user and group names.

A shell script to setup the directories, files, etc. as described below is available in misc/sm.setup.sh.in. This script is modified by configure to create misc/sm.setup.sh (in the build directory) which is invoked when

# make install

is called. Most defaults in the installation script misc/sm.setup.sh can be overridden with environment variables (default is listed in square brackets):

Important Notes:

  1. The users and groups must be created before make install is invoked.

  2. misc/sm.setup.sh will not overwrite existing files or directories, hence it does not work for upgrading a system if configuration files or directory/file owners need to changed.


Directories, Files, and Permissions

make install (i.e., misc/sm.setup.sh) will create all the required directories and files with the correct permissions provided the users and groups have been set up properly. This section explains what the created structure looks like.

The CDB directories (0-9, A-F) must be owned by meta1s and have group meta1q with the permissions 0771:

drwxrwx--x  2 meta1s  meta1q        0/

Note: this means that everyone with access to the machine can guess the name of content files (see Section 10.2 for the format; the names can also be read from the logfiles if those are world-readable) and list (ls(1)) them, however, they cannot access the content files as those are owned by meta1s with mode 0640 and group meta1c, e.g.,

-rw-r-----  1 meta1s  meta1c  1993 Jul  9 21:19 2/S000000000006B1D200

The main (DEFEDB) and incoming queues (IBDB) must belong to meta1q and should not accessible by anyone else:

drwx------  2 meta1q  meta1q        defedb/
drwx------  2 meta1q  meta1q        ibdb/
drwx------  2 meta1q  meta1q        ibdb/ibdb/

Mailertable, aliases map, and other maps for SMAR (see Section 3.9.3) should belong to meta1m and can be readable as local conventions require:

-rw-r--r--  1 meta1m  meta1m         mt
-rw-r--r--  1 meta1m  meta1m         aliases.db

In general, maps should be owned by the user id of the program that uses them, e.g., meta1q owns the QMGR configuration map qmgr_conf.db (see Section 3.8.1).

The meta1 configuration file can either belong to root or the generic meta1 user:

-rw-r--r--  1 meta1   meta1          meta1.conf

The directories in which the communication sockets between QMGR and the other programs are located must belong to meta1q and be group accessible for the corresponding program:

drwxrws---  2 meta1q  meta1m        qmsmar/
drwxrws---  2 meta1q  meta1c        qmsmtpc/
drwxrws---  2 meta1q  meta1s        qmsmtps/

The directory in which the communication socket between MCP and SMTPS is located must belong to meta1s:

drwxr-x---  2 meta1s  meta1s        smtps/

The logfiles must be owned by the corresponding user and may have relaxed group (or even world) read permissions:

-rw-r-----  1 meta1q  operator   qmgr.log
-rw-r-----  1 meta1m  operator   smar.log
-rw-r-----  1 meta1c  operator   smtpc.log
-rw-r-----  1 meta1s  operator   smtps.log

To check whether an installation was successful, run

# ./misc/sm.check.sh -P

(in the build directory). Note: this script uses the same environment variables as the installation script.

Upgrading from earlier MeTA1 Versions

Currently there is no support for automated upgrades. If you have an earlier version of MeTA1 installed and want to upgrade, here are some tips (note: all programs should be run from the build directory unless mentioned otherwise):

Run-Time Configuration of MeTA1

Overview

Configuration of MeTA1 can be done via command line parameters or via a configuration file (the latter is preferred, the former offers only a small subset of the available configuration options). If a configuration file and command line options are specified, then the options are currently processed in order, i.e., later settings override earlier ones for the same options. Information about the former is available by invoking a program with the option -h (MCP currently uses syslog(3) instead of stderr), it will show the usage as well as the default values. The syntax of the configuration files is specified in the following sections. To actually use a configuration file, the option -f $CONFIGFILE must be used, otherwise the programs use only the built-in default values, but not a configuration file. Option '-V' can be used to show version information, specifying '-V' multiple times shows more detail, e.g., '-VVVVV' will show the configuration data including the default value for (almost) every option, and '-VVVVVV' will also show all available flags.

Some configuration options can be set via maps, these maps are: qmgr_conf for QMGR (see Section 3.8.1) and access for SMTPS (indirectly via the address resolver, see Section 3.9.3).


Configuration File Syntax

The grammar for a MeTA1 configuration file is very simple:

conf ::= entries
entries ::= entry *
entry ::= option $\vert$ section
section ::= keyword [name ] "{" entries "}" [";"]
option ::= option-name "=" rhs
rhs ::= value ";" $\vert$ "{" value-list "}" [";"]

A configuration file consists of entries, each entry is either an option or a section. An option has a name, an equal sign, and a value terminated by a semicolon or a (bracketed) list of values separated by comma3.1. A section consists of a keyword, an optional name, and a (bracketed) sequence of entries. Keywords and options are not case sensitive. The layout of a configuration file does not matter, i.e., indentation and line breaks are irrelevant (in general, but see below for strings).


Configuration File Values

Values in a configuration file are usually strings or numbers. If a string is used, then it should be quoted, unless it contains no special characters which are treated specially by the grammar. If a string is very long it can be broken into substrings spread out over several lines (just like strings in ANSI C), e.g.,

  somemessage = "this is a very long string which is spread "
     "out over several lines because otherwise it is too "
     "hard too read.";

Numeric values can have the usual prefixes (known from the programming language C) of 0x for hexadecimal (with digits 0 to 9, A to F, and a to f) and 0 for octal (with digits 0 to 7). Valid boolean values are 0, false, off for false, and 1, true, on for true (case insensitive).

In some cases it is possible to have units for values. Currently time and size values make use of this feature. Valid time units are w for weeks, d for days, h for hours, m for minutes, and s for seconds. Valid units for size are B for bytes, KB for kilo bytes, MB for mega bytes, and GB for giga bytes. It is allowed to specify a sequence of numbers and units, e.g., 1h 5m 12s. Unless otherwise specified, the default units for times and sizes in a configuration file are s and B, respectively; for those values these units can be used.


Example Configuration File

The installation script creates the file meta1.conf in the configuration directory (/etc/meta1, see Section 2.4). Check the comments in the file and edit it if required. A configuration file for meta1 contains several sections: a global section which specifies the locations of sockets and directories that are used by multiple components, and one section each for QMGR, SMAR, SMTP server, and SMTP client. Other sections may define services that are started by MCP, e.g., a local mailer.

CDB_base_directory = "/var/spool/meta1/";

qmgr {
  AQ_max_entries = 8192;
  smtpc { initial_connections = 19; max_connections = 101; }
  smtps { max_connections = 5; max_connection_rate=160; }
  max_errors_per_DSN=16;
  wait_for_server = 4; wait_for_client = 4;
  start_action = wait; user = meta1q;
  restart_dependencies = { smtps, smtpc, smar };
  path = "/usr/libexec/qmgr"; arguments = "qmgr -f /etc/meta1/meta1.conf";
}

smtps { flags = {8bitmime}; CDB_gid = 261; IO_timeout = 5m3s;
  listen_socket { type = inet; port = 25; }
  start_action = pass; pass_fd_socket = smtps/smtpsfd;
  user = meta1s; path = /usr/libexec/smtps;
  arguments = "smtps -f /etc/meta1/meta1.conf"; }

smtpc {
  Log_Level = 12; IO_timeout = 6m; wait_for_server = 4;
  start_action = wait; user = meta1c; path = "/usr/libexec/smtpc";
  arguments = "smtpc -f /etc/meta1/meta1.conf"; }

smar {
  Log_Level = 12;
  nameserver = {10.10.10.9, 127.0.0.1};
  DNS_timeout = 6;
  start_action = wait; user = meta1m; restart_dependencies = { smtps, qmgr };
  path = "/usr/libexec/smar"; arguments = "smar -f /etc/meta1/meta1.conf";
}


Common Global Configuration

All of the following options have defaults and should only be changed if necessary.

  1. hostname: set the hostname to use for the various components. This can be set if gethostbyname(3) does not return a valid (fully qualified) hostname (format: string).
  2. CDB_base_directory: base directory of CDB (format: string); this should either be empty (which is the default) or a path to a directory including a trailing slash; the CDB library currently simply appends the directory names (see Section 2.4.1) to it. It might be useful to move some subdirectories to different disks (by creating (symbolic) links (ln(1))) to spread the I/O load.
  3. SMAR_socket: socket created by the address resolver over which clients (SMTPS, QMGR) can send requests (format: string).
  4. SMTPC_socket: communication socket between SMTPC and QMGR (format: string).
  5. SMTPS_socket: communication socket between SMTPS and QMGR (format: string).

The sockets are currently Unix domain sockets only, hence the value is simply the pathname of the socket.


Common Configuration Options

There is currently one configuration option which is the same across all modules but is not specified in the global section because it is specific to the individual modules.

  1. log: this is a section with the following options:
    1. facility: see syslog(3) for valid facilities, here are some valid options provided the OS offers them: daemon, mail, auth, local0, etc.
    2. ident: identification string for openlog(3), defaults to name of the modules. It might be useful to chose other identifiers, e.g., MeTA1 or MeTA1QMGR.
    3. options: options for openlog(3) (without the leading LOG_) as provided by the OS, e.g., pid or ndelay.

    Example:

    qmgr { log { facility = daemon; ident=meta1-qmgr; } }
    smtps { log { facility = mail; ident=meta1-MTA; } }
    

    Note: debug output is currently sent to stdout; syslog(3) is not used for debugging.

All modules have an option to set the amount of logging (log_level) that should be done. The larger the value the more information is logged. For normal operation a value of 9 is recommended. During testing values of 12 to 14 are useful.


Pathnames for Files, Directories, and Maps

Most names of files (including maps) and directories in the configuration file have a default name (compiled into the binary) without an absolute path, e.g., aliases.db. If a pathname is not explicitly set in the configuration file or does not use a absolute path (i.e., begins with a slash), then the default is relative to either

  1. the configuration directory: maps and configuration files, e.g., aliases.db and cert_file.
  2. the main queue directory: pathnames of sockets, and databases to store envelope information (IBDB, DEFEDB) or message contents (CDB).

The paths for files mentioned in case 1 are taken relative to the path of the configuration file which is passed via the -f option to the various modules. For example: if SMAR is started as

/usr/libexec/smar -f /etc/meta1/meta1.conf

then the pathname used for the aliases map is /etc/meta1/aliases.db. This applies to the SMAR maps aliases, mailertable, and access (3.9.2), the QMGR qmgr_conf map (3.8.1), and the STARTTLS related files and directories used by the SMTP server (3.10) and client (3.11).

The paths for files mentioned in case 2 are taken relative to the execution directory. All MeTA1 modules should be started (via MCP) in the main queue directory (default: /var/spool/meta1, see Section 2.4).

See the various configuration options explained below how to override the defaults. Note: relative pathnames specified in the configuration file are (currently) always relative to the main queue directory.


Configuration for MCP

Every section in a MeTA1 configuration file that refers to one of its four main components (QMGR, SMTPS, SMTPC, and SMAR; see Section 1.1.1) has some options that are relevant for MCP. These MCP options are:

  1. start_action: one of nostartaccept, accept, pass, wait (required).

  2. listen_socket: this is a subsection that specifies the socket on which a process should listen. It must be specified for any start_action except wait. There are two different socket types available:

    1. type = inet
      1. port: port number on which process should listen (format: numeric).
      2. address: IP address on which process should listen, if none is specified the process listens on all local (IPv4) addresses (format: IPv4 address).

    2. type = unix
      1. path: pathname of Unix Domain socket on which process should listen (format: string).
      2. umask: umask for socket (format: numeric).
      3. user: owner of socket (format: string).
      4. group: group of socket (format: string).

  3. pass_fd_socket: pathname of Unix Domain socket to pass a file descriptor to the process.
  4. user: user name to run process.
  5. group: group name to run process.
  6. restart_dependencies: list of other MeTA1 components that need to be restarted when this one is restarted (or crashes).
  7. path: path to program to execute (required).
  8. arguments: arguments (argv), must start with name of program, see execv(2) (required).
  9. pass_id: option to use to pass a unique, numeric identifier to the spawned process via the command line. The option will be inserted as first argument. Example:
    smtpc { pass_id = "-i"; min_processes = 4; max_processes = 4;
      path = /usr/libexec/smtpc; arguments = "smtpc -f meta1.conf"; }
    
    will cause MCP to start four smtpc processes, each with the options -i $ID$ -f meta1.conf where $ID$ is replaced with a unique identifier.

Notes about start_action:

MCP is currently a generic control program that does not have any builtin knowledge about the various MeTA1 modules. Hence the MCP options for each MeTA1 component must be specified properly, there are no builtin defaults that could be associated with the functionality of the various MeTA1 modules. The default configuration file created by the installation program contains the correct defaults. These should only be changed if really necessary.


Configuration for QMGR

The following configuration options are valid for QMGR:

  1. AQ_max_entries: maximum number of entries in AQ (active queue) (unit: entries). Note: this value must be larger than the largest number of recipients accepted by a single transaction.
  2. conf: name of configuration map (including extension), see Section 3.8.1 for details. See also Section 3.6 about relative pathnames.
  3. control_socket: specify pathname of ``control'' socket (for querying and making requests). This socket can be used by the query/control program qmgrctl, see Section 4.6.3.

  4. subsection DEFEDB:
    1. base_directory: home directory for DEFEDB.
    2. log_directory: log directory for DEFEDB. For better performance, this directory can be set to point to a different disk than the base directory of DEFEDB.

  5. subsection DSN_handling:
    1. merge_delay_max: maximum time to wait for merging multiple DSNs into one (unit: s).
    2. flags: configuration flags:
      1. header_only: include only the headers in a DSN; by default the first bounce includes the entire message and subsequent ones include only the headers.
      2. MIME_Format: use MIME to structure a DSN. Note: this is not (yet) a DSN in the format specified by RFC 3464 [MV03].
    3. max_errors_per_DSN: maximum number of error messages (failed recipients) in a bounce (DSN) (unit: entries).

  6. double_bounce_address: RFC 2821 address for double bounces; defaults to <postmaster@hostname>.
  7. subsection IBDB:
    1. max_commit_delay: maximum time between commits to IBDB (unit: $\mu$s)
    2. size: maximum size of each IBDB file (unit: B).
    3. max_open_TAs: maximum number of open transactions in IBDB before a commit is performed (unit: entries).

    Note: the configuration file offers no way to specify a base directory for IBDB, however, the directory can be easily moved elsewhere and a (symbolic) link (ln(1)) can be added.

  8. subsection IQDB:
    1. max_cache_entries: maximum number of entries in IQDB cache (unit: entries). This must be larger than the sum of all recipients in open transactions.
    2. hash_table_entries: size of hash table for IQDB (unit: entries). This must be larger than max_cache_entries.

  9. log_level: logging level.
  10. min_disk_space: minimum amount of free disk space (unit: KB). This value should be significantly larger than the maximum size of a message to be accepted by the SMTP server, it should be as large as the maximum message size multiplied by the maximum number of incoming connections.
  11. OCC_max_entries: size of outgoing (SMTPC) connection cache (unit: entries). This should be large enough to keep track of outgoing connections over a time span that is at least as long as the maximum retry time.
  12. ok_disk_space: amount of free disk space at which normal operation continues (unit: KB). Must be larger than min_disk_space.
  13. queue_return_timeout: maximum time in queue (unit: s).
  14. queue_delay_timeout: send delay warning (``delayed DSN'') if the mail is still in the queue after at least this duration (unit: s). To turn off delayed DSNs set this to a value bigger than queue_return_timeout. Note: based on the retry schedule the delayed DSN might be sent later than the option specifies.
  15. retry_max_delay: maximum time for retrying a delivery (unit: s).
  16. retry_min_delay: minimum time for retrying a delivery (unit: s).

  17. subsection smtpc:
    1. initial_connections: maximum initial number of outgoing connections to a single host (unit: entries). The sliding window for the slow start algorithm (see Section 3.8.1) is initialized with this value.
    2. max_connections: maximum number of outgoing connections to a single host (unit: entries).
    3. lmtp_max_rcpts_per_transaction: maximum number of recipients per transaction for mail sent via LMTP (unit: entries).
    4. smtp_max_rcpts_per_transaction: maximum number of recipients per transaction for mail sent via (E)SMTP (unit: entries).

    5. flags: configuration flags:
      1. lookup_rcpt_conf: Look up recipient configuration data (see Section 3.11.1) in the access map (see Section 3.9.3)

      2. lookup_session_conf: Look up session configuration data (see Section 3.11.1) in the configuration map (see item 2).

    6. rcpt_conf_lookup_flags: If recipient configuration data (see item 17(e)i) is looked up in the access map, then these flags determine which kind of lookups should be performed.

      1. full_adress: use the full address as key.
      2. detail_plus: lookup also ``user++@subdomain''.
      3. detail_star: lookup also ``user+*@subdomain''.
      4. star: lookup also ``user*@subdomain''.
      5. domain: lookup domain part.
      6. dotsubdomain: iterate through subdomains.
      7. dot: lookup also ``.''.

      The default is to perform all lookups.

  18. subsection smtps:
    1. max_connection_rate: maximum incoming connection rate from a single host (unit: connections/60s).
    2. max_connections: maximum number of open incoming connection from a single host (unit: entries).

  19. wait_for_client: maximum amount of time to wait for a client to become available (unit: s)
  20. wait_for_server: maximum amount of time to wait for a server to become available (unit: s)


Configuration Map for QMGR

QMGR implements a ``slow start'' algorithm to control the number of concurrent connections to one IP address. Initially, it will at most create a (small) number of open connections up to a specified initial limit. For each successful delivery, the allowed number is increased up to specified maximum limit.

For incoming connections, QMGR establishes two limits: the connection rate and the number of open connections.

The Berkeley DB hash map qmgr_conf.db (the file should be owned by meta1q) can have the following entries:

  1. oci: this key specifies the initial number of concurrent outgoing connection to an IP address.
  2. ocm: this key specifies the maximum number of concurrent outgoing connection to an IP address.
  3. octo: specify the timeout for an entry in the outgoing connection cache.
  4. icr: this key specifies the maximum rate for incoming connections (per 60s).
  5. icm: this key specifies the maximum number of concurrently open incoming sessions.
  6. smtpc_session_conf: see Section 3.11.1

oci:, ocm:, icr:, icm:, and smtpc_session_conf: take an IP address/net as parameter such that the limits can be imposed per IP address/net. For example:

oci:127.0.0.1      5
ocm:127.0.0.1     10
oci:10            10
ocm:10            50
oci:               1
ocm:               4
icr:10             5
icr:127.0.0.1    100
icm:127.0.0.1    120

Note, however, that the limits apply only to single IP addresses, they are not aggregated for nets. That is, for the example every single host in the IP net 10.x.y.z can have a maximum incoming connection rate of 5 messages per minute.

The default values for these configuration options are set in the binary and can be changed via command line options or the configuration file (see Section 3.8):

  1. -C n maximum number of concurrent connections to one IP address [default: 100]
  2. -c n initial number of concurrent connections to one IP address [default: 10]
  3. -O R=n maximum connection rate per 60s (SMTPS) [default: 100]
  4. -O O=n maximum number of open connections (SMTPS) [default: 100]


Configuration for SMAR


Declaring Maps for SMAR

In general, maps must be declared before they can get used. Each map declaration in a configuration file is a named subsection - the name is used for later references - map in the smar section with the following options:

  1. type: type of the map; currently one of hash (Berkeley DB hash), cdb (tinycdb), sequence, socket, and passwd.
  2. file: the filename of the db file (including the extension) (for type hash, cdb).
  3. mapname: name of the map used in the protocol (type socket only).
  4. address: IPv4 address of inet socket. (type socket only).
  5. path: the pathname of the Unix domain socket (for type socket).
  6. port: port for inet socket (type socket only).
  7. maps: list of map names to use in the map (type sequence only).

Note: for socket maps either a Unix domain socket (path) or an inet socket (address and port) must be specified.

Example:

map localusers { type = hash; file = "/etc/meta1/localusr.db"; }
map otherusers { type = cdb; file = "/etc/meta1/otherusr.cdb"; }
map password { type = passwd; }
map seq1 { type = sequence; maps = { localusers, otherusers }; }
map seq2 { type = sequence; maps = { password, otherusers }; }


Configuration Options for SMAR

The following configuration options are valid for SMAR:

  1. access_map: this is a subsection that specifies the access control map. See Section 3.9.3 for details.

    Note: only one of file (1a) and name (1b) must be specified.

    1. file: filename of access map (including extension) [default: access.db].
    2. name: name of access map . This can be used if a different map type should be used, in which case the map must be declared as explained in Section 3.9.1.

  2. address_delimiter: list of delimiters (specified as string) for address extensions in local part, [default: "+"]. Note: if address_delimiter has more than one character, the first one that is found in the local part of an address is used as delimiter in map lookups (see Section 3.12). For example: if the following option is used in the configuration file:

    address_delimiter = "/_-";

    then for the address ``$<$user/ext-list@dom.ain$>$'', the delimiter for map lookups is ``/'' and the address detail is ``ext-list'', while for the address ``$<$user-ext_list@dom.ain$>$'', the delimiter for map lookups is ``-'' and the address detail is ``ext_list''.

  3. aliases: this is a subsection that specifies the parameters for aliases.

    Note: only one of file (3a) and name (3b) must be specified.

    1. file: filename of aliases map (including extension) [default: aliases.db].
    2. name: name of aliases map . This can be used if a different map type should be used, in which case the map must be declared as explained in Section 3.9.1.
    3. flags:
      1. localpart: the aliases map contains only localparts of addresses and those are only looked up for local addresses.
      2. local_domains: the aliases map contains fully qualified addresses which are only looked up for local addresses. This can be used similar to virtual users in sendmail 8, e.g.,
        vuser1@virt1.tld: user1
        vuser2@virt1.tld: user2
        vuser3@virt2.tld: user3

      3. all_domains: the aliases map contains fully qualified addresses which are looked up for all domains.
      4. implicitly_match_detail: the items are looked up according to the algorithm specified in Section 3.12.1. and additionally +detail is implicitly matched when the pattern is ``user@hostname''. That is, it overrides the default matching explained in case 1e in Section 3.12.1.
      5. replace_macros: replace macros in the RHS of the map entries by the appropriate value, see Section 3.12.3.
      6. preserve_domain: if the RHS of an entry is an unqualified address, do not append the local hostname to it but the domain of the original address, i.e., preserve the original domain.
  4. DNS: this subsection contains DNS related options.
    1. nameservers: list of up to four IPv4 addresses3.2of nameservers. Note: it is important that all of these nameserves work properly. Currently they are used ``round robin'' without excluding nameservers that do not answer3.3.
    2. retries: maximum number of retries. A value of 0 means one query only, i.e., no retry.
    3. timeout: the default timeout for a single DNS query (unit: s). Notes:
      • the timeout for a DNS request is the product of the number of tries and the individual timeout, i.e., (retries + 1) * timeout.
      • this value is only the default timeout which can be overridden by an application. For example, QMGR dynamically increases the timeout for addresses which did not resolve in earlier tries.
    4. flags: The flag use_resolvconf causes the list of nameservers (see 4a) to be read from /etc/resolv.conf. This flag is set by default unless the nameservers option is used. Note: the list of nameservers is not updated when /etc/resolv.conf is changed, smar needs to be restarted to achieve that.

      More flags can be found in Section 8.4.

  5. dnsbl: specify a DNS based blacklist3.4. This section can be specified multiple times3.5; it has the following required options: The client IPv4 address A.B.C.D is looked up via DNS as D.C.B.A.domain querying for an A record. If an A record W.X.Y.Z is found, then it is looked up in the access map as tag:W.X.Y.Z. for temporary and permanent DNS lookup failures the entries that will be checked in the access map are tag:temp and tag:perm, respectively.

    Notes:

    The access map entry should have one of the usual rejection RHSs as explained in 3.9.3. Example: configuration file:

    smar { dnsbl { domain = dnsbl.tld; tag = dnsbltld; } }
    

    access map:

    dnsbltld:127.0.0.1  error:550 5.7.1 listed at dnsbl.tld as open relay
    dnsbltld:127.0.0.2  error:550 5.7.1 listed at dnsbl.tld as spam source
    dnsbltld:127.0.0.9  error:451 4.7.1 listed at dnsbl.tld as suspicious
    dnsbltld:temp       error:451 4.7.1 temporary lookup failure at dnsbl.tld
    

    If multiple DNS based blacklists are specified, the DNS queries are made concurrently but the lookups in the access map are performed in the order in which the blacklists are given; the first successful lookup is used as result, no further priorization is performed.

  6. greylisting: specify greylisting options, see Section 3.9.4 for details.
    1. grey_wait: how long before greylisted can be confirmed.
    2. grey_expire: timeout for greylisted entries (did not confirm within that time).
    3. white_expire: expire whitelisted entries after this time if necessary.
    4. white_timeout: force whitelisted entries to reconfirm after this time.
    5. main_DB_name: name of main database (including .db extension).
    6. secondary_DB_name: name of secondary database (including .db extension).
    7. expire_limit: try to expire entries when this limit is reached.
    8. netmask: by default the entire IPv4 address is used as a key, however, by specifying a netmask, e.g., 0xFFFFFF00, the least significant bits can be cut off. This can be used to deal with server farms, see Section 3.9.4, e.g., if those are in the same class C subnet.

  7. local_user_map: this is a subsection that specifies a map of valid local addresses.
    1. name: Name of the map of valid local addresses; the map must have been declared as explained in Section 3.9.1.
    2. flags:
      1. implicitly_match_detail: +detail is implicitly matched when the pattern is ``user@hostname''. That is, it overrides the default matching explained in case 1e in Section 3.12.1.

  8. log_level: logging level.

  9. mailertable: this is a subsection that specifies a mailertable, currently you can specify exactly one of the following two options:
    1. file: filename of mailertable [default: mt]. In this case a plain text file is read during startup and placed in an internal hash table.
    2. name: name of a mailertable map that has been declared before (see Section 3.9.1).

    3. flags: these flags can be used to select a subset of the matching described in Section 3.12.1.
      1. full_adress: use the full address as key.
      2. detail_plus: lookup also ``user++@subdomain''.
      3. detail_star: lookup also ``user+*@subdomain''.
      4. star: lookup also ``user*@subdomain''.
      5. domain: lookup domain part.
      6. dotsubdomain: iterate through subdomains.
      7. dot: lookup also ``.''.

      The default is domain, dotsubdomain, dot.

    The format of entries in the map is explained in Section 3.9.3. Note: reloading mailertable (Section 4.7) while SMAR is running can be done only if it is declared as Berkeley DB (case 9b with the proper map)


Configuration Maps for SMAR

SMAR requires a mailertable, and it can make use of an alias map as well as an access map, all of which are described in the subsequent sections.


Access Map

To activate the access map the flag access (see Section 3.10, item 3b) (or the option -a) must be given to the SMTP servers. All entries consist of a left hand side (LHS, key) which in turn has a tag and a (partial) address and a right hand side (RHS, value). Valid tags are:

Tag refers to
from: envelope sender address (MAIL)
to: envelope recipient address (RCPT)
cltaddr: client IPv4 address
cltname: client host name
cltresolve: result of forward and reverse client lookup
mxbadip: IPv4 addresses that are not allowed for MX - A records
certissuer: DN of CA cert that signed that presented cert
certsubject: DN of presented cert
protectedrcpt: restrictions for recipient address (see Section 3.10.3)
smtps_session_conf: configuration options for a session in the SMTP server (see Section 3.10.1)
smtpc_rcpt_conf: configuration options for recipient in the SMTP client (see Section 3.11.1)

Valid addresses for from: and to: are RFC 2821 addresses without the angle backets (localpart@domain) as well as partial addresses in the form localpart and @domain, i.e., domains must be preceeded with an at (@) sign. Valid addresses for cltaddr: and mxbadip: are IPv4 addresses and (sub)nets, and for cltname: host names. The client host name is determined by performing a reverse lookup (PTR record) for its IP address. The resulting names are looked up as A records. Only if one of the A records matches the client IP address, the host name is set. Note: the host name has a trailing dot after DNS resolution, this dot must be included in the entry. The result of these lookups can be used for cltresolve: where the following keys are valid:

ok reverse and forward lookup match
no reverse and forward lookup do not match
tempptr reverse lookup (PTR) caused a temporary error
tempa forward lookup (A) caused a temporary error

Valid values for RHS are

relay allow relaying; currently only for to:, cltaddr:,
  cltname:, certissuer:, and certsubject:
ok accept command
error:XYZ A.B.C.D text return an error consisting of SMTP reply code XYZ,
  enhanced status code A.B.C.D, and text,
  i.e., the part after error: is returned to the client.
reject same as error:550 5.7.0 Rejected.
discard accept command but silently discard its effects.
cont stop current check (e.g., map lookup), but continue others.

Some tags may allow for other RHS values, these are explained when those tags are discussed in more detail.

Optionally a RHS can be preceeded by the modifier quick:. For an error: entry it causes an immediate rejection when the entry matches. Otherwise rejections can be delayed to the RCPT stage - if SMTPS is configured appropriately, see Section 3.10, item 3c - and can be overridden using the modifier quick: together with ok or relay in the access map for the recipient address with the to: tag. Using the modifier quick: together with relay for an entry with the cltaddr: tag causes it to override all other access map checks. quick:ok for an entry with the cltaddr: tag causes it to override other access map checks unless they are necessary to allow relaying.

Domain names (@domain) must have an exact match, subdomain matching can be specified with a leading dot, i.e., @.domain, see Section 3.12.1.

Examples:

cltresolve:tempptr error:451 4.7.1 reverse lookup failed
mxbadip:127.0.0.1 error:551 5.7.1 Bad IP address 127.0.0.1 in MX/A list
mxbadip:192.168.255.255 error:551 5.7.1 Bad IP address 192.168.255.255 in MX/A list
from:@spammer.domain error:551 5.7.1 No spammers
from:@.spammer.domain error:551 5.7.1 No spammers in subdomains either
to:root error:551 5.7.1 No mail to root
to:abuse quick:ok
cltaddr:10 error:551 5.7.1 No direct mail from 10.x.y.z
cltname:spammer.domain. quick:error:551 5.7.1 No mail from spammers
to:@primary.domain relay
cltaddr:10 relay
cltaddr:127.0.0.1 quick:relay


Discard

The effect of discard depends on the protocol stage in which it is returned. If it is returned for a session, e.g., when a client connects, all transactions in the session are discarded. If it is returned for MAIL only that transaction is discarded. If it is returned for RCPT only that recipient is discarded; however, if no valid recipients are left, the entire transaction is discarded. Moreover, if quick:discard is returned for one recipient the entire transaction is discarded too.


Mailertable

The address resolver implements an asynchronous DNS resolver and by default it uses a file called mt (mailertable) (see Section 3.9.2, item 9) which consists of domain parts of e-mail addresses and corresponding IP addresses (in square brackets) or domain/host names. An entry consists (as usual in a map) of a LHS and a RHS; in the case of a flat text file, i.e., case 9a of Section 3.9.2, those are separated by one or more whitespace characters.

LHS ::= [ local "@" ] [ "." ] hostname $\vert$ "."
RHS ::= [[ port "^" ] ["esmtp:"]] hosts $\vert$ "lmtp:" $\vert$ port "^"lmtp:" hosts
port ::= integer
hosts ::= hostname $\vert$ iplist
iplist ::= "[" IPv4-address "]" [ " " iplist ]

The key (LHS) is an address (without angle brackets), a hostname, or a dot (denoting the default entry), the value (RHS) consists of an optional port number, an optional (esmtp) mailer and a hostname or a list of IPv4 addreses (in square brackets) which are separated by spaces. If LMTP should be used, then the lmtp mailer must be selected. There are two cases: just lmtp: by itself means the delivery agent will use the Unix domain socket specified in the configuration file (see Section 3.11, item 2), if an inet socket should be used then a port and a host must be specified. A hostname is subject to MX lookups.

Example:

localhost lmtp:
SPAM.FILTER.DOMAIN 2525^esmtp:[127.0.0.1]
LMTPHOST.MY.DOMAIN 525^lmtp:[10.11.12.13]
MY.DOMAIN esmtp:[10.1.2.3]
ANOTHER.DOMAIN esmtp:MTA.SERVER
.TLD esmtp:GATE.WAY
. esmtp:SMART.HOST

Note: currently this file must exist, even if there are no entries (it is created during installation).


Aliases

To specify aliases for local addresses the map aliases.db (Section 3.9.2, item 3a) is used. The key in the map must be

based on the flags of the aliases option (see Section 3.9.2, 3c). The value (RHS) for an alias entry is a list of one or more RFC 2821 addresses (including the angle brackets) separated by spaces (not commas). If the RHS has only a single address which does not start with an angle bracket, then it is converted into an RFC 2821 address by SMAR, i.e., SMAR will append the hostname of the machine and put angle brackets around the string. Example:

myalias: localuser
mylist: <user1@my.dom> <user2@my.dom> <localuser@local.host>
owner-mylist: someuser

For mailing lists, the owner- notation is supported, i.e., if there are aliases list and owner-list then mail sent to list will use owner-list as envelope sender address; the original domain will be preserved.

Example for the flag local_domains (see 3.9.2, 3(c)ii). Let two domains be local, i.e., in mailertable:

first.dom lmtp:
second.dom lmtp:

and these entries be in aliases:

myalias@first.dom: user1
another@second.dom: user2

Then mail to <myalias@second.dom> and <another@first.dom> would be rejected while mail to <myalias@first.dom> or <another@second.dom> would be accepted.

Aliases can be nested (currently up to 5 levels, see smar/rcpts.c).


Greylisting

MeTA1 supports a very simple form of greylisting [Hara] which only uses the client IP address as key [Posa] instead of a tuple consisting of client IP address, envelope sender, and envelope recipient. The idea behind greylisting is simple: do not accept mail from an unknown source on the first connection, but reject it with a temporary error. Any MTA that conforms to RFC 2821 [Kle01] will try to send the mail later on, however, spamming systems often do not do that. An IP address can be in three different states: unknown: the client has not connected before or the entry is expired from the database, greylisted: the client has connected before but it did not yet connect again within the configured time interval, whitelisted: the client has connected before and it connected again within the configured time interval. The time interval is specified by its lower limit grey_wait and its upper limit grey_expire. A lower limit is used to prevent system from getting accepted that just send a single message within a few seconds again and again. The upper limit is used to avoid filling up the database. If an entry has made it to the whitelisted state, it will stay there for (at least) up to the timeout specified by white_expire. The greylisting algorithm implemented in MeTA1 uses another timeout white_timeout after which a whitelisted entry is considered stale and must go through the greylisting stages again, i.e., it is considered to be in state unknown. Each time a mail is sent from a whitelisted host, the entry is updated, to avoid that systems which regularly sent mail become greylisted again.

Greylisting is performed at the RCPT stage of the SMTP dialogue. It is only done when a valid recipient is specified, i.e., all other checks must have been successful. Hence clients that do not try to send mail or just try invalid recipient addresses will not be added to the greylisting database. If a transaction is subject to greylisting then the session is aborted with an 421 error. If a server uses callbacks to verify the sender address, then the option delay_greylisting_error_until_DATA (see Section 3.10, item 3d) is useful to avoid unnecessary delays. Here is an example: host A is the main MX server for domain example.com and it uses greylisting, host B is the main MX server for domain example.net and it uses sender callbacks. If a mail is sent to host B for <rcpt@example.net> with the sender address <user@example.com> then host B will connect to host A to test whether <user@example.com> is a valid recipient. However, if host A does not have host B in its whitelist, it will return a 421 error after the RCPT To:<user@example.com> command, which (depending on the implementation of the sender callback) will cause host B to temporarily reject the mail for <rcpt@example.net>. By specifying the option delay_greylisting_error_until_DATA on host A the RCPT command will succeed and the original mail to <rcpt@example.net> will go through without delay.

The greylisting implementation uses two persistent databases (specified by main_DB_name and secondary_DB_name), where the second DB is just a secondary index (by expiration time) for the main DB. These databases should be on a filesystem with sufficient free disk space depending on how many connections from different clients the MTA receives. Entries are only removed from the DB if there are more than expire_limit elements. However, if none of the entries are expired yet, then the number of elements can exceed that limit.


Greylisting: Whitelisting

Greylisting can be disabled for selected hosts by adding them to the access map (see Section 3.9.3), e.g.,

cltaddr:10 relay
cltaddr:127.0.0.1 quick:relay


Possible Problems with Greylisting

Some legitimate mailers do not behave properly and will not retry a mail that had a temporary error. This can cause mail loss in various situations, e.g., because the receiving system is currently out of some resources. However, to minimize the impact of greylisting on these misbehaving mailers it might be useful to explicitly whitelist them as:

cltaddr:12.107.209.244 ok
cltaddr:64.12.137 ok

A list of such broken mailers can be found at http://cvs.puremagic.com/viewcvs/greylisting/schema/whitelist_ip.txt [Harb]. A related problem are server farms where a mail might be resent from a different IP address. These should probably be whitelisted too; some of these can be found at the URL given before. However, entries in that file which have the comment ``unique sender per attempt'' do not need to be whitelisted as this implementation does not use the sender address.

Note: if a client authenticates via STARTTLS or AUTH such that relaying is allowed then greylisting is disabled for that client.


Configuration for SMTP Server

The following configuration options are valid for SMTPS:

  1. auth: this is a subsection that specifies the parameters for AUTH support. It is only available if the system has been configured with the option --enable-SASL, see Section 2.2.1.
    1. flags: flags for SMTP AUTH

      See the Cyrus SASL documentation for the meaning of these flags: noplaintext, noactive, nodictionary, forward_secrecy, noanonymous, pass_credentials, mutual_auth.

    2. trusted_mechanisms: list of SASL mechanisms for which relaying is allowed if a client successfully authenticated using one of those

    Note: the name for the Cyrus-SASL configuration file is currently meta1.conf. That file can be used to adjust the list of mechanisms (option mech_list) that should be advertised (besides many other things), hence this option is not in the SMTP server itself.

  2. CDB_gid: (numeric) group id for CDB files, i.e., the group id of meta1c, see Section 2.4.1.
  3. flags:
    1. 8bitmime: offer 8BITMIME: MeTA1 is 8 bit transparent, but it does not perform any conversion, so this option should only be used if all communication partners can deal with 8 bit data.
    2. access: use access map (in SMAR). Note: currently this flag is required to perform a reverse lookup for a client IP address to get the hostname of the client which then can be used for logging and the Received: header.
    3. delay_checks: delay acceptance check until RCPT stage (unless explicitly overridden, see Section 3.9.3).
    4. delay_greylisting_error_until_DATA: if greylisting (3e) is enabled then wait until the DATA command to return an error; see Section 3.9.4 for details.
    5. greylisting: enable greylisting (which must also be enabled in SMAR, see Section 3.9.2 item 6), see Section 3.9.4 for details.
    6. lmtp_does_not_imply_relaying: even if a domain in the mailertable has lmtp: as RHS do not implicitly allow relaying to it, i.e., do not consider the domain as ``local'' with respect to relaying. This is useful for an MSA to avoid external mail to local domains without authentication.
    7. soft_bounce: change permanent (5xy) SMTP error replies into temporary (4xy) errors. This is a useful feature for testing to avoid bounces due to misconfigurations.
    8. require_EHLO_before_MAIL: require EHLO (or HELO) before a MAIL command.
    9. strict_EHLO_checks: perform a strict syntax check on the argument for EHLO (or HELO).

    10. xverp: offer XVERP extension to turn on VERP [Ber97] support for a transaction. This is a parameter for the MAIL command, e.g.

      MAIL From:<sender@some.domain> XVERP

    11. allow_data_before_greeting: allow a client to send data before the initial 220 greeting.

    12. prdr: PRDR is an SMTP extension [Hal07] to return individual RCPT status after the end of data (similar to LMTP). This feature is turned on if a policy milter is configured (see item 17).

    13. lookup_session_conf: Look up session configuration data (see Section 3.10.1) in the access map (which must be activated, see 3b).

  4. id: unique identifier for SMTP server (0); see Section 3.10.2.
  5. io_timeout: timeout for SMTP operations.
  6. max_threads: maximum number of threads.
  7. max_bad_commands_per_session: maximum number of bad, i.e., unknown, SMTP commands per session accepted by server. After this limit is reached the connection is terminated with an 421 error.
  8. max_invalid_addresses_per_session maximum number of invalid, e.g., unknown, RCPT addresses per session accepted by server. After this limit is reached the connection is terminated with an 421 error.
  9. max_nop_commands_between_transactions: maximum number of NOOP, RSET, and related SMTP commands between two successful transactions accepted by server. After this limit is reached the connection is terminated with an 421 error.
  10. max_bad_commands_per_transaction: maximum number of bad, i.e., unknown, SMTP commands per transaction accepted by server. After this limit is reached the connection is terminated with an 421 error.
  11. max_nop_commands_in_transaction: maximum number of NOOP and related SMTP commands in a single transaction accepted by server. After this limit is reached the connection is terminated with an 421 error.
  12. max_invalid_addresses_per_transaction maximum number of invalid, e.g., unknown, RCPT addresses per transaction accepted by server. After this limit is reached the connection is terminated with an 421 error.
  13. max_recipients_per_session: maximum number of recipients per session.
  14. max_recipients_per_transaction: maximum number of recipients per transaction.
  15. max_hops: maximum number of hops (Received: headers). If this value is exceeded the incoming mail is rejected because it is considered a possible mail loop.
  16. max_message_size: maximum message size (unit: KB).
  17. policy_milter: this is a subsection that specifies the parameters for pmilter support (see Section 5). It is only available if it has been enabled during configure (--enable-pmilter, see Section 2.2.1).
    1. socket: this is a subsection that specifies the socket to communicate with policy milter. The type (option type) of the socket must be either inet or unix.
      1. type = inet
        1. port: port number for connection.
        2. address: IP address for connection.

      2. type = unix
        1. path: pathname of Unix Domain socket.

    2. timeout: maximum amount of time to wait for a reply from a policy milter.
    3. flags: policy milter flags. If the connection to pmilter fails then SMTPS will ignore pmilter by default. This behavior can be changed by setting one of following two flags:
      1. abort: if the connection to pmilter fails then abort the current session with a 421 error.
      2. accept_but_reconnect: if the connection to pmilter fails then continue the current session but try to reconnect for the next session.

  18. processes: number of processes to start.
  19. protected_recipients: this is a subsection which provides a few simple options to protect recipients by restricting who can send mail to them.
    1. allow_by: this is a required subsection which has two possible flags (at least one must be specified).
      1. sender: allow sending mail based on the envelope sender (MAIL) address. Even though this address can be forged it provides some basic protection.
      2. client_ip: allow sending mail based on the client IP address.

    2. match_type: this specifies what type of matching should be done. By default, exact matches are required. Alternatively, one of the following two options can be selected:
      1. generic_lookup: the items are looked up according to the algorithm specified in Section 3.12.1.
      2. implicitly_match_detail: the items are looked up according to the algorithm specified in Section 3.12.1. and additionally +detail is implicitly matched when the pattern is ``user@hostname''. That is, it overrides the default matching explained in case 1e in Section 3.12.1.

    See Section 3.10.3 for details.

  20. max_transactions: maximum number of transactions per session.

  21. tls: this is a subsection that specifies the parameters for STARTTLS support. It is only available if the system been configured with the option --enable-TLS, see Section 2.2.1. See Section 11.1 for some background information about these options.

    1. cache_size: size of TLS session session cache (0: disable cache).
    2. cache_timeout: timeout for entries in TLS session session cache.
    3. cert_file: file with certificate in PEM format.
    4. key_file: file with private key for certificate in PEM format.
    5. CAcert_file: file with CA certificate in PEM format.
    6. CAcert_directory: directory with (symbolic links for) CA certificates in PEM format.

    7. flags: some flags are available to influence the behavior of the SMTP server with respect to STARTTLS.
      1. allow_relaying_if_verified: if the client presented a certificate that can be verified by the CA certificates that are available to the server (see above: CAcert_file and CAcert_directory), then relaying is allowed for the SMTP session.

      2. check_access_map_for_relaying: if this flag is set then the access map (which must be activated, see 3b) is checked to see whether relaying should be allowed for a client which presented a certificate that has been verified (see above). For this purpose, the DN of the cert issuer is looked up in the access map using the tag certissuer:. If the resulting value is relay, relaying is allowed. If it is cont, the DN of the cert subject is looked up next in the access map using the tag certsubject:. If the value is relay, relaying is allowed; every other value is currently ignored.

        To avoid problems with the DN names in map lookups, they are modified as follows: each non-printable character and the characters '<', '>', '(', ')', '"', '+', ' ' are replaced by their hexadecimal ASCII value with a leading '+'. For example:

        /C=US/ST=California/O=endmail.org/OU=private/CN=
        Darth Mail (Cert)/emailAddress=darth+cert@endmail.org

        is encoded as:

        /C=US/ST=California/O=endmail.org/OU=private/CN=
        Darth+20Mail+20+28Cert+29/emailAddress=darth+2Bcert@endmail.org

        Examples:

        To allow relaying for everyone who can present a cert signed by

        /C=US/ST=California/O=endmail.org/OU=private/CN=
        Darth+20Mail+20+28Cert+29/emailAddress=darth+2Bcert@endmail.org

        simply use:

        certissuer:/C=US/ST=California/O=endmail.org/OU=private/CN=
        Darth+20Mail+20+28Cert+29/emailAddress=darth+2Bcert@endmail.org relay

        To allow relaying only for a subset of machines that have a cert signed by

        /C=US/ST=California/O=endmail.org/OU=private/CN=
        Darth+20Mail+20+28Cert+29/emailAddress=darth+2Bcert@endmail.org

        use:

        certissuer:/C=US/ST=California/O=endmail.org/OU=private/CN=
        Darth+20Mail+20+28Cert+29/emailAddress=darth+2Bcert@endmail.org cont
        CertSubject:/C=US/ST=California/O=endmail.org/OU=private/CN=
        DeathStar/emailAddress=deathstar@endmail.org relay

        Notes:

        • line breaks have been inserted after CN= for readability, each tagged entry must be one (long) line in the access map.
        • if OpenSSL 0.9.6 is used then the emailAddress= part of a DN is replaced by Email=.

    8. requirements_violation: This option decides how to (globally) handle TLS requirement violations (see Section 3.10.1). Possible values are: permfail: generate permanent failure, tempfail: generate temporary failure, abort: abort session. These option take effect when a MAIL command is issued by the client.

  22. session_features: This section can be used to define classes of features for an SMTP server session which then can be referenced via the access map. Available flags are a subset of those listed in item 3: starttls, auth, delay_checks, allow_data_before_greeting, require_EHLO_before_MAIL, strict_EHLO_checks, check_EHLO.

    Example: to turn off STARTTLS for some clients, declare a notls class in the configuration file:

    session_features  notls { flags = { -starttls }}
    

    and reference it in the access map for those clients:

    smtps_session_conf:217.126.135.148	session_feature=notls;
    


SMTP Server Session Configuration

Some options can be set via the access map (see Section 3.9.3, tag smtps_session_conf) because they apply to a session, not globally. Currently available are: STARTTLS requirements (tls_requirements) (see Section 3.13) and session_features (see Section 3.10, item 22).


Multiple SMTP Servers with different Configurations

The normal way to run multiple SMTP servers is to let MCP start several SMTP servers. Each SMTP server must given a unique identifier (see Section 3.10, item 4) and each SMTP server section in meta1.conf must have a unique name (e.g., MTA and MSA), which is passed via the option -N name to smtps. Example: meta1.conf:

smtps MTA {
  id = 0;
  listen_socket { type=inet; port = 25; }
  start_action = pass; pass_fd_socket = smtps/mtafd;
  user = meta1s;
  path = /usr/libexec/smtps;
  arguments = "smtps -N MTA -f /etc/meta1/meta1.conf";
  log { facility = mail; ident=meta1-MTA; }
}

smtps MSA {
  id = 1;
  listen_socket { type=inet; port = 587; }
  start_action = pass; pass_fd_socket = smtps/msafd;
  user = meta1s;
  path = /usr/libexec/smtps;
  arguments = "smtps -N MSA -f /etc/meta1/meta1.conf";
  log { facility = mail; ident=meta1-MSA; }
  auth { trusted_mechanisms = "CRAM-MD5 DIGEST-MD5";
         flags = { noplaintext }; } }

For tests it is also possible to let MCP start only one SMTP server which creates several copies of itself if multiple daemon addresses are specified (see Section 3.10, item 1). Note: this only works for unprivileged ports because the SMTP server does not run as root.


Protecting Recipients

A few simple features are available to protect recipients by restricting who can send mail to them. To do this the configuration section protected_recipients must be turned on and at least one of the two flags sender and client_ip must be selected (see Section 3.10, item 19a). If this is done, then every recipient is looked up in the access map (which must be activated, see Section 3.10, item 3b), using the tag protectedrcpt:. If a matching entry is found, it must have a list of (one or more) restrictions, each of which must be one of the following:

restriction required allow_by flag
from:sender sender
cltaddr:IPv4-address client_ip
list:alias sender

The meaning of the first two restriction types should be obvious, the third one is interesting: it refers to an alias (in the aliases map, see Section 3.9.3) and requires that the sender address matches one of the entries to which the alias expands. This can be used to allow only subscribed members of a mailing list to send mail to it.

The restrictions are evaluated sequentially, if there is a match, the recipient is accepted (sequential OR). If none of them matches, the recipient is rejected.

By default exact matches are required. However, if the flag generic_lookup is set (see Section 3.10, item 19b), the items are looked up as specified in Section 3.12.1. The flag implicitly_match_detail is useful for the list: restriction if a sender uses +detail without having that specified during subscription.

Examples: consider the following aliases map:

list1: <user1-1@l1-1.dom> <user2-1@l1-1.dom> <list2@local.dom>
list2: <user1-2@l2-1.dom> <user2-2@l2-2.dom>
list3: <user1-3@l3-1.dom> <user2-3@l3-2.dom>

together with this access map:

protectedrcpt:list1@local.dom list:<list1@local.dom>
protectedrcpt:list3 from:<moderator3@local.dom> cltaddr:1.2.3.4 cltaddr:10

The mails to <list1@local.dom> are only accepted from <user1-1@l1-1.dom> and <user2-1@l1-1.dom>. Note: the list is not recursively expanded, i.e., members of list2 are not allowed, that restriction must be listed in the access map. Mails to <list3@local.dom> are only accepted from <moderator3@local.dom>, the client with the IPv4 address 1.2.3.4, or clients in the IPv4 net 10. The latter requires that the flag generic_lookup is turned on too.


Configuration for SMTP Client

The following configuration options are valid for SMTPC:

  1. io_timeout: timeout for SMTP operations (unit: s).
  2. LMTP_socket: Unix domain socket to use for LMTP [default: lmtpsock].
  3. log_level: logging level.

  4. tls: this is a subsection that specifies the parameters for STARTTLS support. It is only available if the system been configured with the option --enable-TLS, see Section 2.2.1. See Section 11.1 for some background information about these options.
    1. cert_file: file with certificate in PEM format.
    2. key_file: file with private key for certificate in PEM format.
    3. CAcert_file: file with CA certificate in PEM format.
    4. CAcert_directory: directory with (symbolic links for) CA certificates in PEM format.

  5. wait_for_server: maximum amount of time to wait for a server (QMGR) to become available (unit: s).


SMTP Client Session/Recipient Configuration

The following options can only be set via the access map (see Section 3.9.3, tag smtpc_rcpt_conf) or the configuration map for QMGR (see 3.8.1, tag smtpc_session_conf) not in the SMTP client configuration section itself, as they apply to a session or recipient, not globally. Currently only STARTTLS requirements are available which are documented in Section 3.13, more option might be added later on. Note: as more options might be added, the structure of this may change a bit.


Lookup Orders


Lookup Orders in Maps

In many cases an item is not just looked up verbatim in a map, but it may be split into logical parts and then less significant parts are iteratively removed and the remaining data is looked up until either a match is found or the data is empty; in the latter case a default key may be looked up depending on the map. These steps can be controlled by flags that are specified in the configuration file for the map. These flags are listed below for the various lookup steps.

For domain names of the form ``sub2.sub1.tld'' the lookup order is ``sub2.sub1.tld'', ``.sub1.tld'', ``.tld'', and ``.'' (without the quotes). The subdomains are tried if the flag dotsubdomain is set. The last lookup (``.'') is only done if the flag dot is set, as it is the default for mailertable. Obviously this schema is extended if more components are specified. As the sequence shows there is no implicit ``match all subdomains'' lookup, instead entries in a map must have a leading dot for subdomains matches. To reiterate: ``sub2.sub1.tld'' does neither match the entry ``sub1.tld'' nor ``tld''.

For IPv4 addresses of the form ``A.B.C.D'', the lookup order is ``A.B.C.D'', ``A.B.C'', ``A.B'', and ``A'' (without the quotes). In contrast to domain lookups, no trailing dots are required (nor checked) to denote subnet matches, because the number of components of an IPv4 address is fixed (and known) in contrast to the number of components in a host name or domain name.

For RFC 2821 addresses of the form ``$<$user+detail@domain