Cyrus IMAP Questions
- Questions about features
|What is Duplicate Delivery Supression / How does it work?|
Duplicate delivery supression is a feature of the Cyrus IMAP server that allows multiple copies of an identical messages delivered to one user to be supressed, so that the user only recieves one copy. This can be conveinent if, say, one user is on multiple mailing lists that commonly recieve the same message.
The way that a message is determined to be a duplicate is a lookup is done in the duplicate delivery database for a message-id/mailbox pair. If a match is found, the message is supressed. If a match is not found, the pair is added to the database and the message is delivered.
Duplicate Delivery Supression can also affect sieve redirects. In this case, supression is done on a message-id/redirect-target basis.
Duplicate Delivery Supression can also affect vacation messages. In this case, supression is done based on a hash of the sender's address and the vacation message.
|What database backend should I use for which databases?|
Here's a brief summary of the three major database types:
The default database backend for each database is the type currently recommended by the Cyrus developers. Please reference the imapd.conf(5) manpage for your version of Cyrus imapd to see what the defaults are.
- Installation and setup trouble
|I'm seeing errors about kssl.h and krb5.h when compiling Cyrus IMAPd on my Red Hat Linux server|
When trying to compile Cyrus IMAPd on Red Hat Linux or Fedora, you may be encountering errors like these:
In file included from /usr/include/openssl/ssl.h:179, from prot.h:56, from prot.c:72: /usr/include/openssl/kssl.h:72:18: krb5.h: No such file or directory In file included from /usr/include/openssl/ssl.h:179, from prot.h:56, from prot.c:72: /usr/include/openssl/kssl.h:134: syntax error before "krb5_enctype" /usr/include/openssl/kssl.h:136: syntax error before '*' token /usr/include/openssl/kssl.h:137: syntax error before '}' token /usr/include/openssl/kssl.h:149: syntax error before "kssl_ctx_setstring" /usr/include/openssl/kssl.h:149: syntax error before '*' token /usr/include/openssl/kssl.h:150: syntax error before '*' token /usr/include/openssl/kssl.h:151: syntax error before '*' token /usr/include/openssl/kssl.h:151: syntax error before '*' token /usr/include/openssl/kssl.h:152: syntax error before '*' token /usr/include/openssl/kssl.h:153: syntax error before "kssl_ctx_setprinc" /usr/include/openssl/kssl.h:153: syntax error before '*' token /usr/include/openssl/kssl.h:155: syntax error before "kssl_cget_tkt" /usr/include/openssl/kssl.h:155: syntax error before '*' token /usr/include/openssl/kssl.h:157: syntax error before "kssl_sget_tkt" /usr/include/openssl/kssl.h:157: syntax error before '*' token /usr/include/openssl/kssl.h:159: syntax error before "kssl_ctx_setkey" /usr/include/openssl/kssl.h:159: syntax error before '*' token /usr/include/openssl/kssl.h:161: syntax error before "context" /usr/include/openssl/kssl.h:162: syntax error before "kssl_build_principal_2" /usr/include/openssl/kssl.h:162: syntax error before "context" /usr/include/openssl/kssl.h:165: syntax error before "kssl_validate_times" /usr/include/openssl/kssl.h:165: syntax error before "atime" /usr/include/openssl/kssl.h:167: syntax error before "kssl_check_authent" /usr/include/openssl/kssl.h:167: syntax error before '*' token /usr/include/openssl/kssl.h:169: syntax error before "enctype" In file included from prot.h:56, from prot.c:72: /usr/include/openssl/ssl.h:909: syntax error before "KSSL_CTX" /usr/include/openssl/ssl.h:931: syntax error before '}' token make: *** [prot.o] Error 1 make: Leaving directory `/opt/cyrus-imapd-2.2.3/lib' make: *** [all] Error 1
The key bit of this error is:
/usr/include/openssl/kssl.h:72:18: krb5.h: No such file or directory
Essentially, what is happening is that Cyrus is trying to use OpenSSL, and OpenSSL's headers include some Kerberos header files. Red Hat, in their infinite wisdom, chose to place the Kerberos headers outside the normal header search path, despite having built OpenSSL to require the Kerberos headers. As a result, even if your program is not using Kerberos, it may fail to compile if gcc can't find the Kerberos headers.
The solution is to either add the Kerberos headers to gcc's header search path, or prevent OpenSSL from trying to use the Kerberos includes in the first place. To tell OpenSSL you really don't want kerberos, just run
(as suggested by Ken Murchison on info-cyrus) before you run ./configure. Alternately, you can tell gcc where to find the Kerberos includes so that it'll stop complaining:
If neither of these work, make sure you have the Kerberos development libraries installed ( you should have if you have openssl-devel, but one never does know ...). If you run rpm -q openssl-devel krb5-devel you should get a result like:
|I'm having problems detecting Berkley DB on my Red Hat Linux machine|
Modern Redhat Linux ships with a Berkeley DB compiled --with-pthreads, which cyrus is unable to directly link against.
You can either recompile Berkeley DB without pthread support, or you can add "-lpthread" to your LDFLAGS enviornment variable before you configure/build.
|Help! There must be an easier way to get all this going...|
Currently, the Cyrus project only distributes Cyrus IMAPd and Cyrus SASL as source code. If you prefer not to work directly with the source distribution - you don't have the time to learn it, find it too difficult, or it's otherwise unsuitable - there are a few other options.
There are a number of projects outside the main Cyrus project that provide easy-to-install packages of the Cyrus software. Additionally, there are a fair few commercial software packages that include the Cyrus software as a component. A partial list of these can be found on the ExternalLinks page.
Most people active on the info-cyrus mailing list appear to use the source distribution, so more people are likely to be familiar with that than any vendor-specific custom versions. Additionally, it can be easier to upgrade to the latest version if you're using the source packages. Finally - you learn more! In the end, the best way to get Cyrus depends on what your specific needs are.
|Cyrus delivers claims that the mailbox does not exist when it contains a '@' e.g,|
Given that you have a username firstname.lastname@example.org, unless you escape the '@' as explained below, cyrus deliver will claim that the mailbox does not exist.
you should be using
and instead of
you should be using
As far as the '@' is concerned, I believe the same applies while not using the unixhierarchysep.
keywords: cyrdeliver master.cf
|I have multiple imapd-SERVICES configured and experience occasional freezes when I try to log in!|
If you have more than one imapd-service configured in cyrus.conf then they must have distinct names (the first argument in each line). Otherweise a deadlock can occur because the services will try to use the same lockfile, whose filename is derived from the name you gave to the service. Certain characters, while not strictly forbidden, should not be used in these names, because they will cause the name to be truncated. For example, "imap" and "imap_remote" will result in the same lockfile (/var/imapd/socket/imap-0.lock). It is best to only use letters and digits.
- Common feature requests
|How can I make CyrusSieve work with public shared folders?|
|Does the CyrusMurder support High Availability configurations?|
The Cyrus Murder clustering solution drasticly increases system-wide reliability as compared to a traditional standalone IMAP server. However, this configuration is not a true High Availability/Redundancy solution (nor is such a beast trivial to implement given the constraints on traditional Email systems).
The frontends of the CyrusMurder are, in every sense, 100% redundant. No server-specific data is maintained on them, and one is as good as any other from the point of view of a user.
The mupdate server is indeed a single point of failure, however a failure at this point only prohibits message delivery (via the LMTP proxies that are querying it for up-to-date mailbox data) and certain mailbox operations (create, delete, setacl, and so on). Mail can still be accessed by all users of the system, and incoming mail will recieve temporary failures, so the sending MTA should continue to queue the mail.
The failure of a backend will result in a partial systemwide failure -- the mailboxes that were stored on that backend will no longer be available for use by any users. Delivery of new mail will again be tempfailed, and queued by the sending MTA.
Some people have suggested shared storage and the use of hot spare backends to combat this solution. The Cyrus authors do not generally recommend the use of shared storage in the Cyrus enviornment. However people have reported success in various configurations. The shared filesystem still needs to support file locking, and it is highly recommended that it have an efficient mmap implementation.
|Can I configure pop3d to log amount and size of messages fetched by user|
The short answer is no, there isn't any logging based on the size of message retrieved or when a specific message is retrieved. Patches are welcome.
|Can I have subfolders not appear under INBOX?|
The altnamespace setting in /etc/imapd.conf changes the way the mailbox list is presented to users (including fileinto directives in Sieve scripts). Instead of appearing as subfolders of INBOX, altnamespace makes subfolders appear at the same level as INBOX. It does not affect the internal storage nor the way the folders are presented through cyradm. See the man page for imapd.conf(5) for more information.
- Common operational questions
|How do I view ACLs on a mailbox|
|Why is deliver.db so large?|
Having a large deliver.db (on the order of tens of megabytes) is not uncommon, since a record is kept of every message that goes through lmtpd (so that duplicate supression can do its work), so there's a lot of data there. Note also that it is common for databases to have unused space that may not be immediately recovered for performance reasons (this is the case with Berkeley DB and CyrusSkiplist) You can control the size by increasing the frequency at which it gets pruned.
|Why does ctl_cyrusdb -r take so long with Berkeley DB?|
Berkeley DB maintains a log of all the transactions since the last checkpoint of the database. In order to ensure the database is in a consistent state, you must recover the log after any outage (thus the recommendation to run these processes when cyrus starts). They can take a long time for a few reasons.
The most common one is that you need to checkpoint the cyrusdb more often. This can be done with a simple ctl_cyrusdb -c If you do this very often, the amount of log that needs to be recovered will be significantly shorter. We recommend doing this at least once every half hour, and more often on busy sites.
The other reason is that your deliver.db may be very large. This is solvable by increasing the pruning interval (the -E parameter to ctl_deliver, which you should run on a regular basis), or (in a pinch) by just removing the database (since the effects of losing it do not prevent operation, they just cause vacation messages to be resent, and duplicate delivery supression to possibly deliver duplicates).
|What does the "unable to join environment" error mean?|
This error is caused if you build Berkley DB 4 with threading support, but do not then link the pthreads library to Cyrus. This is commonly caused by a Linux distribution which supplies the pre-threaded library version.
One workaround is to rebuild the db4 library without thread support. Another is provided by Scott Adkins (email@example.com):
Starting with Cyrus 2.2.1, it is possible to build without linking Berkeley DB at all, which may be a viable option for some sites.
|Why does Cyrus set the MAIL FROM address of the sender of vacation responses to '<>'?|
Because many of the Sieve features are autoresponders, Cyrus tries to be a good network citizen by adhering to the appropriate RFCs:
From RFC 2821:
This notification message must be from the SMTP server at the relay host or the host that first determines that delivery cannot be accomplished. Of course, SMTP servers MUST NOT send notification messages about problems transporting notification messages. One way to prevent loops in error reporting is to specify a null reverse-path in the MAIL command of a notification message. When such a message is transmitted the reverse-path MUST be set to null (see section 1.5.5 for additional discussion).
A MAIL command with a null reverse-path appears as follows:
Also from draft-moore-auto-email-response-02.txt:
The primary purpose of the MAIL FROM address is to serve as the destination for delivery status messages and other automatic responses. Since in most cases it is not appropriate to respond to an automatic response, and the responder is not interested in delivery status messages, a MAIL FROM address of <> MAY be used for this purpose.
This is of course slightly more applicable to reject than it is to vacation, but there isn't a very strong argument why they should be treated differently.
Note that draft-moore-auto-email-response-02.txt also states:
A MAIL FROM address which is specifically chosen for the purpose of sending automatic responses, and which will not automatically respond to any message sent to it, MAY be used instead of <>.
Therefore, it may be reasonable to have this be a configurable fixed address in the future.
|Is the "db3: x Lockers" log message harmful?|
This message is generally not harmful unless it is increasing at an alarming rate or you are seeing other performance problems. It indicates contention within a Berkeley DB database page, and especially on small databases will occur quite frequently.
If there are performance problems, often times the best solution is to convert your mailbox list to a skiplist format.
Paul M Fleming (firstname.lastname@example.org) also notes:
Andreas (email@example.com) writes:
Update, 2007/07/09: They've moved since sleepycat got bought by Oracle (did they really I wonder, I heard something about this, and their site seems to have moved....) Anyway, I found the pages listed above:
The links above talk about the C api, but don't get alarmed, these paremeters can be set with a DB_CONFIG configuration file located in the DB environment home.
I also suppose the db_stat utility can be used to diagnose this.
|Why can I not delete a mailbox as an admin user?|
By default, the permissions on newly created mailboxes do not allow access to any other user. Therefore even root user can not delete mailbox.
To delete a mailbox, you need to give appropriate rights with the command "sam user.mailbox username all"
after that you can delete the mailbox. Even just giving the user the "c" right (Create and Delete mailbox) is enough.
|Why does reconstruct -m not work?|
From the reconstruct man page:
−m NOTE: CURRENTLY UNAVAILABLE Rebuild the mailboxes file. Use whatever data in the existing mailboxes file it can scavenge, then scans all partitions listed in the imapd.conf(5) file for additional mailboxes.
Reconstruct is currently unable to rebuild the mailboxes db, and comments on the mailing list indicate this ability will not be added - at least, any time soon.
The solution here is to make sure that you back up your mailboxes db, preferably including a plain-text copy of it. Please see Backups for more information.
If you do find yourself with a corrupted mailboxes.db, there are a few things you can try. The first is to see if db_recover can recover your database. If that doesnt' work, there should be backups in $CONFIGDIRECTORY/db.backup1 and $CONFIGDIRECTORY/db.backup2 that may be OK.
If you're using Simon Matter's RPMs, plain-text copies of the mailboxes database should be being generated and saved in /var/lib/imap/backup . Try rebuilding the db from one of those using ctl_mboxlist. Alternately, try asking the mailing list for help.
|Can I use (insert shared file system here) to make a highly available Cyrus?|
In short, we don't recommend it. If you want to do it, it may possibly work but you may also lose your email or have corrupted cyrus.* files. You can look at the mailing list archives for more information.
There are several critical things that must be supported for this to work: * file locking * mmap() * writing to an mmap()'d file with the write() syscall In general, this is bad news. Don't do it. Really. About the closest you can currently get is having a "warm spare" that takes over the IP of the affected backend, and have a SAN between the two systems.
Cyrus Murder may be a "good enough" solution in some enviornments, as a way to partition failures and spread load across many machines. Combined with a warm-spare approach this can be a good way to achieve highly available systems (though not ones which are fully redundant)
|Can I use MySQL? or another SQL database as the primary mail store?|
Using a database as the main message store for Cyrus is not currently supported. According to discussions on the mailing list, there is little benefit in adding such support (given typical IMAP access patterns, optimizations in the current mail store that make it fast, and the amount of effort involved to retrofit a different mailstore into the backend), so it's unlikely to ever be written.
The Cyrus mail store is a normal directory tree, with mailboxes stored as directories and messages stored as individual files. Some additional information, eg index data, is kept in the "cyrus.*" files in the mailboxes.
Cyrus DOES have the option of using databases of various types for storing some other information, such as authentication data, mailbox lists, etc. There may be reasons to add a SQL backend to these databases in the future (in addition to flat, skiplist, and berkeley).
|Why is mail being rejected with No Mailbox found due to MiXed CaSe incoming e-mail?|
If you are using a newer and standard compliant MTA to deliver mail to Cyrus IMAP's LMTP agent you may run into a problem where mail will be bounced and not delivered with errors like this in your maillog..
Feb 29 10:01:02 myhost lmtpunix: append_check() of 'MyDomain.tld!user.MyUser' failed (Mailbox does not exist)
This is due to LMTP adhering to the RFC standards and the MTA adhering to the RFC standards of preserving the case in the e-mail address. so User@MyDomain.tld is not the same as firstname.lastname@example.org. And this causes issues if with users who alter the case of their e-mail address when signing up for mailing lists, or sending e-mail.
However, all is not lost there is a solution. Cyrus IMAP has an option that can be added into the imapd.conf..
Inserting that into your imapd.conf and restarting cyrus imap will cause cyrus to "downcase" the e-mail address up to the recipient delimiter of +. The result will be this..
Note that this option first appeared in Cyrus IMAP 2.1.14.
|Why do POP3 connections take so long, but once the connection is established all is well?|
In some configurations POP3 needs to generate random data for use by APOP authentications. This can lead to delays if /dev/random does not have enough entropy immediately available.
If you don't need or want support for APOP, you can either compile SASL with --disable-checkapop, or you can disable APOP support in pop3[proxy]d at runtime by setting allowapop:0 in imapd.conf.
Otherwise, since there isn't a strong need for high-quality random numbers with SASL, there are two options for dealing with this -- link /dev/urandom to /dev/random. Alternatively, recompile SASL with --with-devrandom=/dev/urandom. It's prefered to do the later, as it avoids affecting other applications.
|What annotations are available?|
Cyrus annotations are based on a draft (http://tools.ietf.org/html/draft-daboo-imap-annotatemore-08) version of RFC 5464.
/admin - Sets the administrator email address for the server. (See cyradm(1))
/check - Boolean value "true" or "false" that indicates whether this mailbox should be checked at regular intervals by the client. The interval in minutes is specified by the
/checkperiod entry. (Draft RFC)
/checkperiod - Numeric value indicating a period of minutes that the client uses to determine the interval of regular 'new mail' checks for the corresponding mailbox. (Draft RFC)
/comment - Sets a comment or description associated with the mailbox. (cyradm(1)) /motd - Sets a "message of the day". The message gets displayed as an ALERT after authentication.
/sort - Defines the default sort criteria [I-D.ietf-imapext-sort] to use when first displaying the mailbox contents to the user, or NIL if sorting is not required. (Draft RFC)
/thread - Defines the default thread criteria [I-D.ietf-imapext-sort] to use when first displaying the mailbox contents to the user, or NIL if threading is not required. If both sort and thread are not NIL, then threading should take precedence over sorting. (Draft RFC)
/vendor/cmu/cyrus-imapd/condstore - Enables the IMAP CONDSTORE extension (modification sequences) on the mailbox. (cyradm(1))
/vendor/cmu/cyrus-imapd/duplicatedeliver - Flag signalling that we're allowing duplicate delivery of messages to the mailbox, overriding system-wide duplicate suppression. (http://cyrusimap.web.cmu.edu/imapd/internal/mailbox-format.html)
/vendor/cmu/cyrus-imapd/expire - Sets the number of days after which messages will be expired from the mailbox. (cyradm(1))
/vendor/cmu/cyrus-imapd/freespace - Undocumented.
/vendor/cmu/cyrus-imapd/lastpop - (time_t) of the last pop3 login to this INBOX, used to enforce the "poptimeout" imapd.conf option. (http://cyrusimap.web.cmu.edu/imapd/internal/mailbox-format.html)
/vendor/cmu/cyrus-imapd/lastupdate - (time_t) of the last time a message was appended (http://cyrusimap.web.cmu.edu/imapd/internal/mailbox-format.html)
/vendor/cmu/cyrus-imapd/news2mail - Sets an email address to which messages injected into the server via NNTP will be sent. (cyradm(1))
/vendor/cmu/cyrus-imapd/partition - Undocumented.
/vendor/cmu/cyrus-imapd/pop3newuidl - Flag signalling that we're using "uidvalidity.uid" instead of just "uid" for the output of the POP3 UIDL command. (http://cyrusimap.web.cmu.edu/imapd/internal/mailbox-format.html)
/vendor/cmu/cyrus-imapd/server - Undocumented.
/vendor/cmu/cyrus-imapd/sharedseen - Enables the use of a shared \Seen flag on messages rather than a per-user \Seen flag. The ’s’ right in the mailbox ACL still controls whether a user can set the shared \Seen flag. (cyradm(1))
/vendor/cmu/cyrus-imapd/shutdown - Sets a shutdown message. The message gets displayed as an ALERT and all users are disconnected from the server (subsequent logins are disallowed). (cyradm(1))
/vendor/cmu/cyrus-imapd/sieve - Indicates the name of the global sieve script that should be run when a message is delivered to the shared mailbox (not used for personal mailboxes). (cyradm(1))
/vendor/cmu/cyrus-imapd/size - Undocumented.
/vendor/cmu/cyrus-imapd/squat - Indicates that the mailbox should have a squat index created for it. (squatter(8))
|Is it safe to put configdir/proc and configdir/lock on a tmpfs filesystem?|
It's safe to put both of these in tmpfs, and since they're fairly busy it's a good idea for performance reasons. Further, nothing ever removes files from <configdirectory>/lock/, so it's probably a good idea in general to mount that on tmpfs such that a reboot will purge it. The Cyrus code will create all the necessary directory structure under <configdirectory>/lock on-demand.
- Common interoperability problems with Cyrus (Cyrus Interoperability)
|Why does Cyrus reject 8 bit characters in the headers of my messages?|
8-bit characters are not allowed in the headers of an RFC 822 message.
We're not about to consider a patch to "fix" the problem of replacing 8-bit characters with 'X's that doesn't atleast supply a default character set and properly QP-encode the nonconforming header.
Another possiblity is suggested by Chris Newman:
|Why does Cyrus reject messages with "bare newlines"|
Please first read about Cyrus Interoperability.
Bare newlines is a nono in an RFC822 message. You should first try to fix the software that is causing the problem.
As it happens, the bare newlines-rejection has never been a problem for us.
-- RobSiemborski - 24 Jul 2003
I have encountered this issue when migrating mail from other mail systems, such as MDaemon for WinNT? , that may not be so fussy about RFC822 compliance. It hasn't been an issue for us post-migration however.
-- Craig Ringer, 5 Nov 2003
Well, the mail client of an American software vendor, infamous because of its operating system and its behaviour against open standards, seems quite often to use bare newlines, especially for binary attachments. Unfortunately they won't give me the sourcecode so I can't fix it. My question for you: Is there a patch, or even better a setting to suppress the bare newlines check? Any information here is very appreciated!
-- Karsten Gresch, 25 Oct 2005
|Why does Cyrus screw up messages sent by Eudora that have MIME attachments?|
Please first read about Cyrus Interoperability.
Note that this does not apply to Cyrus versions starting with 2.1.14 (the interoperability problem was solved). Note that due to the unique nature of this problem the later versions have a rfc2046_strict ImapdConf? option which will restore the old behavior.
RFC 2046 has some text which, if interpreted literally, does not allow outer mime boundrys to be substrings of inner mime boundries. Eudora, as it happens, creates messages which have just that format. The problem is that earlier versions of the MIME spec did not contain this restriction, so 2046 is not backwards compatible (in a literal interpretation).
After discussions with other implementors within the IETF, it was decided that Eudora's behavior was clearly broken, Cyrus's behavior was unfriendly to interoperability but technically correct, and RFC 2046 probably needs a clarification. Thus, we corrected the behavior in Cyrus with an option to revert to the strict behavior if a site so desires.
|How do I get CyrusSieve to play nice with Exim?|
This problem results from Cyrus using CRLF sequences to denote line breaks in its pipe to the sendmail process, which normal Sendmail handles just fine. However, exim does not handle these sequences ok. Exim 4.20 and later has a drop_cr option which you can use, the following wrapper script was suggested by Bernhard Erdmann (email@example.com):
- /etc/imapd.conf: sendmail: /opt/exim/exim_dropcr
- /opt/exim/exim_dropcr: #!/bin/sh /opt/exim/exim -dropcr $@
|Why does mail delivery go slow or hang sometimes?|
There are a number of things that can affect mail delivery. Here are some common causes. 1. Murder Master is overloaded. In a normal setup the murder master is queried for each incoming messages 2. Delivery.db corruption. A backend will/may reject mail if the delivery.db is corrupt 3. Slow dns. Some MTAs (like sendmail) talk to dns a lot. If dns is slow, mail delivery appears slow. We run named on each Cyrus machine 4. Lmtpproxyd is having trouble. Sometimes lmtpproxyd will die and dissapear. Sometimes it just hangs. Not sure why, but restarting may help.
If you have a large volume of mail then Cyrus Murder Mail Delivery has some tips.
We experienced some filesystem locking problems when allowing to many lmtpd-processes to be started. Performance was seriously degraded, resulting in timeouts and endless retries. We use a postfix instance between to limit parallel deliveries to lmtpd resulting in much higher throughput. With berkeley-nosync as duplicate suppression db, 50-100 Messages/sec are doable.
Cyrus SASL Questions
|Why do CRAM-MD5 and DIGEST-MD5 not work with CyrusSaslauthd?|
Saslauthd is only capabile of verifying plaintext passwords (it takes a plaintext password and a username and responds with "yes" or "no", essentially). Therefore, since the plaintext password isn't passed from client to server in DIGEST-MD5 and CRAM-MD5, Saslauthd can't verify the password.
Authentication in a CyrusSaslauthd? -only enviornment will not only fail with these mechanisms, it doesn't really make a lot of sense. You'll want to use an AuxpropPlugin? instead (for example, sasldb).
|Why does CyrusSasl store plaintext passwords in its databases?|
To operate with the CRAM-MD5 and DIGEST-MD5 mechanisms, Cyrus SASL stores plaintext versions of the passwords in its secret database (an AuxpropPlugin? ).
This is typically regarded as insecure practice, however the alternative is not much better. For CRAM-MD5 and DIGEST-MD5 to function, they must have a plaintext equivalent locally in order to confirm the hash that actually goes across a wire. This, if these equivalents were compromised, it is trivially easy for an attacker to have access to any account on the system.
Note that for DIGEST-MD5 this isn't strictly true: the hash that DIGEST can use limits the attack to only the realm for which the password applies, but this is a questionable security gain for the increased management hassles (you can't share them between mechanisms) that the plaintext equivalents cause.
|What is the difference between an Authorization ID and a Authentication ID?|
Authentication is the act of proving who you are. "Hello, I'm Dave. To prove it, here's my password: Foo." Authorization is the act of deciding whether to grant access to resources.
In the example, I'm trying to read my wife's mail. I supply my own username as the *"authentication identifier", my own password (Or biometric scan, or whatever else is required to prove I'm really me with whichever mechanism is in use), and my wife's username as the "authorization identifier".
At no point need I know my wife's password - instead, either Kellie or an administrator needs to explicitly state that I am allowed in "as Kellie". Once I've logged in, all the access checks are done against Kellie, not against Dave, because I'm acting for her. To all intents and purposes, after the authentication exchange itself, the server can simply forget about who authenticated - it's not important anymore - and concentrate on who needs to be authorized.
Another, more common example of the use of differing authentication identifiers and authorization identifiers is in the design of many proxy systems. You authenticate perfectly normally to the proxy, authorizing as yourself. The proxy then authenticates to the master as itself, but supplies you as the authorization identifier, thus getting all the right access checks done at source, but not having to have access to your authentication credentials. Finally, some mechanisms don't support passing a distinct authorization identifier, and for most its optional, and defaults to the case that most people are familiar with, where authorization and authentication identifiers are the same.
|Why am I having a problem running dbconverter-2 to upgrade from SASLv1 to SASLv2?|
The migration is documented in the official documentation.
When migrating the /etc/sasldb database using the utils/dbconverter-2 utility, you may encounter the error message "Error opening password file". This is usually due to the fact your SASL V1 library was compiled using a different version of Berkeley DB than the SASL V2 library. You can work around this by using Berkeley DB's db_upgrade utility (possibly chaining the DB3 and DB4 upgrade utilities) to upgrade a copy of sasldb prior to conversion using dbconverter-2.
Here is the script we use at our installation, where SASL has to coexist with SASL2:
#!/bin/sh cp /etc/sasldb /tmp/sasldb.$$ /usr/local/BerkeleyDB.4/bin/db_upgrade /etc/sasldb echo ""|/usr/local/sasl/sbin/dbconverter-2 cp /tmp/sasldb.$$ /etc/sasldb
|How do I configure OpenLDAP +SASL+GSSAPI?|
This article assumes that you have read and followed the SASL chapter of the OpenLDAP Administrator's Guide . You should have a Kerberos server installed (such as Heimdal or MIT), and created all the appropriate principals (client and service) necessary.
To verify that you have the Cyrus GSSAPI mechanism properly installed, use the pluginviewer command. For instance:
server:~# pluginviewer | grep -i gssapi CRAM-MD5 PLAIN NTLM GSSAPI OTP DIGEST-MD5 ANONYMOUS LOGIN EXTERNAL Plugin "gssapiv2" [loaded], API version: 4 SASL mechanism: GSSAPI, best SSF: 56, supports setpass: no CRAM-MD5 PLAIN NTLM GSSAPI OTP DIGEST-MD5 ANONYMOUS LOGIN EXTERNAL Plugin "gssapiv2" [loaded], API version: 4 SASL mechanism: GSSAPI, best SSF: 56
Both your server and client systems will need to have this mechanism installed. If not, you may find the mechanism located in a binary package that you do not yet have installed, or you may need to recompile your Cyrus SASL installation.
On your client system, search the Root DSE of the server to view advertised mechanisms:
client:~# ldapsearch -LLL -x -H ldap://ldap.example.org -s "base" -b "" supportedSASLMechanisms dn: supportedSASLMechanisms: DIGEST-MD5 supportedSASLMechanisms: NTLM supportedSASLMechanisms: GSSAPI supportedSASLMechanisms: OTP supportedSASLMechanisms: CRAM-MD5
If you received a No Such Object error, you may have an ACL misconfiguration on your server. See .
If you do not see GSSAPI listed, verify that the server can read the appropriate keytab. The default is /etc/krb5.keytab and is typically only readable by the root user. If your Kerberos library supports it, you can create a keytab in an alternate location, such as /etc/krb5.keytab-ldap, with only your LDAP service principal, and read permissions for your OpenLDAP user.
If your OpenLDAP server is looking for an unexpected principal within your keytab, use sasl-host and sasl-realm to influence which principal it will use (see the slapd.conf man page).
For more control over how the SASL library operates within the OpenLDAP? server, you can create a slapd.conf SASL configuration. This is not the same configuration file as the OpenLDAP configuration (slapd.conf). The location of this file is dependent how on Cyrus SASL was compiled (via the --with-plugindir option during configure). It is the directory where your plugins are installed.
For instance, if you create /usr/lib/sasl2/slapd.conf (assuming that is the correct location on your system) with the following contents:
keytab: /etc/krb5.keytab-ldap mech_list: CRAM-MD5 DIGEST-MD5 GSSAPI
then the server will search within /etc/krb5.keytab-ldap when initializing the GSSAPI plugin. The server will only offer the mechanisms listed in mech_list. If mech_list is not specified, the server will offer all the mechanisms available, and that it can initialize.
Once you have verified that the server is advertising GSSAPI support, then try:
client:~# ldapsearch -LLL -Y GSSAPI -H ldap://ldap.example.org -s "base" -b "" supportedSASLMechanisms SASL/GSSAPI authentication started SASL username: host/client.example.org@EXAMPLE.ORG SASL SSF: 56 SASL data security layer installed. dn: supportedSASLMechanisms: DIGEST-MD5 supportedSASLMechanisms: NTLM supportedSASLMechanisms: GSSAPI supportedSASLMechanisms: OTP supportedSASLMechanisms: CRAM-MD5
If you receive a list of mechanisms, then congratulations, you're done.
If instead you receive an error, then it's possible that your client is requesting a service principal that mismatches what your server is using.
Verify that you have received a TGT ticket from your kerberos server, and that you have also received a service principal ticket for the server (e.g. ldap/ldap.example.org@EXAMPLE.ORG). Use klist to verify:
client:~# klist Credentials cache: FILE:/tmp/krb5cc_0 Principal: host/client.example.org@EXAMPLE.ORG
Issued Expires Principal Feb 22 11:00:02 Feb 22 21:00:01 krbtgt/EXAMPLE.ORG@EXAMPLE.ORG Feb 22 11:24:39 Feb 22 21:00:01 ldap/ldap.example.org@EXAMPLE.ORG
If not, then a network capture utility, such as wireshark, offers a good way to show you which service principal your client is requesting, and what errors are being returned by the server.
If you wish to also view the interaction with the LDAP server with a capture utility, you will need to negotiate a SASL layer without encryption. You can specify '-O maxssf=1' in your client side command (ldapwhoami, ldapsearch etc.), e.g.:
client:~# ldapsearch -LLL -Y GSSAPI -H ldap://ldap.example.org -O maxssf=1 -s "base" -b "" supportedSASLMechanisms
You might find it easier to specify defaults on your client system, to keep your commands shorter. See the man page for ldap.conf for details. You can specify the default server in your ldap.conf file:
And you can specify the default mechanism in ~/.ldaprc (in your home directory):
How To Report Bugs
- Output of the cyradm(1) 'version' command. If you are having problems with Perl, then connect to the server with imtest(1) and use the command:
a ID NIL
This info will show us the timestamp of the source code, the server platform, as well as most (if not all) of the compile-time options.
- Contents of imapd.conf. This will show us the run-time configuration.
- Contents of cyrus.conf. This will show us how your services are configured.
- The configure lines used to configure Cyrus IMAPd and Cyrus SASL.
- It helps to call attention to the following parts of your configuration
- Are you using altnamespace, unixhierarcysep, or virtdomains?
- Are you in a murder configuration (please include the above information for all hosts, if so!)?