Database Formats

Database Formats

Introduction

This is an attempt to document the cyrus database formats. It should not be considered authoritative and is subject to change.

No external tools should make use of this information. The only supported method of access to the databases is via the API specific to the database and the underlying cyrusdb interface.

Mailbox List (mailboxes.db)

This database contains the master list of all mailboxes on the system. The database is indexed by mailbox name and each data record contains the mailbox type, the partition on which the mailbox resides and the ACL on the mailbox. The format of each record is as follows:

Key: <Mailbox Name>

Data: <Type Number>SP<Partition>SP<ACL (space-separated userid/rights pairs)>

Annotations (annotations.db)

This database contains mailbox and server annotations. The database is indexed by mailbox name (empty for server annotations) + annotation name + userid (empty for shared annotations) and each data record contains the value size, value data, content-type of the data and timestamp of the record. The format is each record is as follows:

Key: <Mailbox Name>\0<Annotation Name>\0<Userid>\0

Data: <Value Size (4 bytes)><Value>\0<Content-Type>\0<Timestamp (4 bytes)>

Quotas (quotas.db)

This database contains the master list of quotaroots on the system. The database is indexed by quota root and each data record contains the current usage of all mailboxes under the quota root and the limit of the quota root. The format of each record is as follows:

Key: <Quota Root>

Data: <Usage (in bytes)>SP<Limit (in Kbytes)>

Legacy Quotas

The legacy quota database uses a distributed system in which each quota root is stored in a separate file named by quota root and the contents has the following format:

<Usage (in bytes)>\n
<Limit (in Kbytes)>\n

The translation to/from this data record format is handled by the quota_legacy cyrusdb backend.

Duplicate Delivery (deliver.db)

This database is used for duplicate delivery suppression, retrieving usenet articles by message-id, and tracking Sieve redirects and vacation responses. The database is indexed by message-id + recipient (either mailbox or email address) and each data record contains the timestamp of the record and the UID of the message within the mailbox (if delivered locally). The format of each record is as follows:

Key: <Message-ID>\0<Recipient>\0

Data: <Timestamp (4 bytes)><Message UID (4 bytes)>

TLS cache (tls_sessions.db)

This database caches SSL/TLS sessions so that subsequent connections using the same session-id can bypass the SSL/TLS handshaking, resulting is shorter connection times. The database is indexed by session-id and each data record contains the timestamp of the record and the ASN1 representation of the session data. The format of each record is as follows:

Key: <Session-ID (multi-byte)>

Data: <Timestamp (4 bytes)><Session Data (multi-byte)>

PTS cache (ptscache.db)

This database caches authentication state records, resulting in shorter authentication/canonicalization times. The database is indexed by userid and each data record contains an authentication state for the userid. The format of each record is as follows:

Key: <Userid>

Data: <Auth State (multi-byte)>

STATUS cache (statuscache.db)

This database caches IMAP STATUS information resulting in less I/O when the STATUS information hasn't changed (mailbox and \Seen state unchanged). The database is indexed by mailbox name + userid and each data record contains the database version number, a bitmask of the stored status items, the mtime, inode, and size of the cyrus.index file at the time the record was written, the total number of messages in the mailbox, the number of recent messages, the next UID value, the mailbox UID validity value, the number of unseen messages, and the highest modification sequence in the mailbox. The format of each record is as follows:

Key: <Mailbox Name>\0<Userid>\0

Data: <Version>SP<Bitmask of Items>SP<Mtime of Index>SP<Inode of Index>SP<Size of Index>SP<# of Messages>SP<# of Recent Messages>SP<Next UID>SP<UID Validity>SP<# of Unseen Messages>SP<Highest Mod Sequence>

User Access (user_deny.db)

This database contains a list of users that are denied access to Cyrus services. The database is indexed by userid and each data record contains the database version number (currently 2), a list of wildmat patterns specifying Cyrus services to be denied, and a text message to be displayed to the user upon denial. The service names to be matched are those as used in cyrus.conf(5). The format of each record is as follows:

Key: <Userid>

Data: <Version>TAB<Deny List (comma-separated wildmat patterns)>TAB<Deny Message>

Timezone Info (zoneinfo.db)

This database is used for the timezone service and contains records relating to timezones and their aliases. The database is indexed by timezone ID and each data record contains the database version number, a record type, a timestamp, and an optional list of strings (either aliases for a timezone or the reference timezone for an alias). The format of each record is as follows:

Key: <TZID>

Data: <Version>SP<Record Type>SP<Timestamp>SP<Data Strings (TAB-separated)>

Seen State (<userid>.seen)

This database is a per-user database and maintains the list of messages that the user has read in each mailbox. The database is indexed by mailbox unique-id and each data record contains the database version number, the timestamp of when a message was last read, the message unique-id of the last read message, the timestamp of the last record change and a list of message unique-ids which have been read. The format of each record is as follows:

Key: <Mailbox UID>

Data: <Version>SP<Last Read Time>SP<Last Read UID>SP<Last Change Time>SP<List of Read UIDs>

Subscriptions (<userid>.sub)

This database is per-user and contains the list of mailboxes to which the user has subscribed. The database is indexed by mailbox name and each data record contains no data. The format of each record is follows:

Key: <Mailbox Name>

Data: None

Mailbox Keys (<userid>.mboxkey)

This database is per-user and contains the list of mailbox access keys which are used for generating URLAUTH-authorized URLs. The database is indexed by mailbox name and each data record contains the database version number and the associated access key. The format of each record is follows:

Key: <Mailbox Name>

Data: <Version (2 bytes)><Access Key (multi-byte)>

DAV Index (<userid>.dav)

This SQLite database is per-user and primarily maintains a mapping from DAV resource names (URLs) to the corresponding Cyrus mailboxes and IMAP message UIDs. The database is designed to have one table per resource type (iCalendar, vCard, etc) with each table containing metadata specific to that resource type.

CalDAV

The format of the iCalendar table used by CalDAV is as follows:

CREATE TABLE ical_objs (
 rowid INTEGER PRIMARY KEY,
 creationdate INTEGER,
 mailbox TEXT NOT NULL,
 resource TEXT NOT NULL,
 imap_uid INTEGER,
 lock_token TEXT,
 lock_owner TEXT,
 lock_ownerid TEXT,
 lock_expire INTEGER,
 comp_type INTEGER,
 ical_uid TEXT,
 organizer TEXT,
 dtstart TEXT,
 dtend TEXT,
 comp_flags INTEGER,
 sched_tag TEXT,
 UNIQUE( mailbox, resource )
);

Because CalDAV Scheduling requires the server to locate a resource by iCalendar UID regardless of which calendar collection (mailbox) it resides in, the iCalendar table has an additional index as follows:

CREATE INDEX idx_ical_uid ON ical_objs ( ical_uid );

CardDAV

The format of the vCard table used by CardDAV is as follows (work in progress):

CREATE TABLE vcard_objs (
 rowid INTEGER PRIMARY KEY,
 creationdate INTEGER,
 mailbox TEXT NOT NULL,
 resource TEXT NOT NULL,
 imap_uid INTEGER,
 lock_token TEXT,
 lock_owner TEXT,
 lock_ownerid TEXT,
 lock_expire INTEGER,
 version INTEGER,
 vcard_uid TEXT,
 kind INTEGER,
 fullname TEXT,
 name TEXT,
 nickname TEXT,
 email TEXT,
 UNIQUE( mailbox, resource )
);