init commit
Raphael Sprenger
Fri, 28 Aug 2026 13:40:47 +0200
57 files changed,
7981 insertions(+),
0 deletions(-)
jump to
A
LICENSE
@@ -0,0 +1,661 @@
+ GNU AFFERO GENERAL PUBLIC LICENSE + Version 3, 19 November 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/> + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU Affero General Public License is a free, copyleft license for +software and other kinds of works, specifically designed to ensure +cooperation with the community in the case of network server software. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +our General Public Licenses are intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + Developers that use our General Public Licenses protect your rights +with two steps: (1) assert copyright on the software, and (2) offer +you this License which gives you legal permission to copy, distribute +and/or modify the software. + + A secondary benefit of defending all users' freedom is that +improvements made in alternate versions of the program, if they +receive widespread use, become available for other developers to +incorporate. Many developers of free software are heartened and +encouraged by the resulting cooperation. However, in the case of +software used on network servers, this result may fail to come about. +The GNU General Public License permits making a modified version and +letting the public access it on a server without ever releasing its +source code to the public. + + The GNU Affero General Public License is designed specifically to +ensure that, in such cases, the modified source code becomes available +to the community. It requires the operator of a network server to +provide the source code of the modified version running there to the +users of that server. Therefore, public use of a modified version, on +a publicly accessible server, gives the public access to the source +code of the modified version. + + An older license, called the Affero General Public License and +published by Affero, was designed to accomplish similar goals. This is +a different license, not a version of the Affero GPL, but Affero has +released a new version of the Affero GPL which permits relicensing under +this license. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU Affero General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Remote Network Interaction; Use with the GNU General Public License. + + Notwithstanding any other provision of this License, if you modify the +Program, your modified version must prominently offer all users +interacting with it remotely through a computer network (if your version +supports such interaction) an opportunity to receive the Corresponding +Source of your version by providing access to the Corresponding Source +from a network server at no charge, through some standard or customary +means of facilitating copying of software. This Corresponding Source +shall include the Corresponding Source for any work covered by version 3 +of the GNU General Public License that is incorporated pursuant to the +following paragraph. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the work with which it is combined will remain governed by version +3 of the GNU General Public License. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU Affero General Public License from time to time. Such new versions +will be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU Affero General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU Affero General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU Affero General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + <one line to give the program's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published + by the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see <https://www.gnu.org/licenses/>. + +Also add information on how to contact you by electronic and paper mail. + + If your software can interact with users remotely through a computer +network, you should also make sure that it provides a way for users to +get its source. For example, if your program is a web application, its +interface could display a "Source" link that leads users to an archive +of the code. There are many ways you could offer source, and different +solutions will be better for different programs; see section 13 for the +specific requirements. + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU AGPL, see +<https://www.gnu.org/licenses/>.
A
Readme.md
@@ -0,0 +1,104 @@
+Postern +======= + +> [!WARNING] +> **Alpha warning:** This software is in an experimental state. +> Do not use in production or for any important emails while we work towards v1.0. + +Postern is a modern all-in-one mail delivery agent. It combines IMAP (IMAP4rev2), +SMTP Submission and SMTP delivery. + +We have EU Sovereignity at our core values. We welcome users and contributions from around the world and will keep +service providers such as server hosting and DNS registrars EU based. + +Overview +-------- + +What makes Postern special is its policy engine. Imagine it as a modern Sieve based on +[Starlark](https://starlark-lang.org), which uses Python syntax. +The `policy.star` file has an example how new senders end up in a "Gatekeeper" mailbox. +Simply moving a message from one folder to another updates how the policy engine reacts to this sender +for future messages. + +Reacting on side effects such as database state or network calls is probably the biggest advantage of the policy engine +over Sieve when it comes to modern email management. + +Postern has secure defaults, stores emails encrypted at rest and runs as a single binary with an SQLite backend. + +You can subscribe to *announcements* by sending an email to `postern+subscribe@lists.p25.dev`. +The mailing list server runs on Postern and mlmmj. + +Architecture +------------ + +Postern was developed for easy email self-hosting while not fighting email deliverability or a multitude of DNS records. + +Postern is designed to sit behind an email relay and MX backup like [p25.dev](https://p25.dev). With just 3 DNS records +(DKIM, MX and domain verification) you are ready for sending and receiving emails. Postern can run behind other services +such as Postfix too, as it is fully standards compliant. The main use case will be the operation +behind [p25.dev](https://p25.dev) though just for its simplicity and to support more inexperienced admins in self-hosting +email. + +Your emails are stored encrypted in [Data At Rest Encryption (DARE) format](https://github.com/minio/sio). +Note that the SQLite database stores email _metadata_ such as To and From addresses and Subjects +in **plain text** to support fast IMAP search requests. + +All networking is secure by default. IMAP and SMTP Submission run on implicit TLS. SMTP Delivery requires +STARTTLS and authentication. Passwords are generated by Postern to avoid weak choices. + +DKIM key management is designed for easy key rotation. + +Quickstart +---------- + +Postern is in alpha state. You might experience some trial and error to arrive at a working solution. +This quickstart tutorial assumes you work on a modern Linux installation. +Because you read this Readme, it looks like you have access to the rest of the source code and can compile it with +`go build -o postern .`. + +1. Install the `postern.service` file under `/etc/systemd/system/postern.service`. +2. Place `policy.star` under `/etc/postern/policy.star` and adjust to your requirements. +3. For secure storage of the masterkey, + run `systemd-ask-password -n | systemd-creds encrypt - /etc/credstore/masterkey` +4. Retrieve a TLS certificate, for example with [acme.sh](https://github.com/acmesh-official/acme.sh). + Make sure the systemd service can access the certificates (`/etc/postern/`). +5. Enable and start Postern `systemctl enable postern` and `systemctl start postern` +6. Use the Postern CLI to add configuration. Based on the systemd integration, this is admittedly clunky + and will be improved soon. + +```Shell +# The masterkey file is only decrypted with a running postern service and only accessible by root. +# (this needs to be changed soon) +CONFIGURATION_DIRECTORY=/etc/postern POSTERN_PERSISTENCE_DIR=/var/lib/postern POSTERN_MASTERKEY_FILE=/run/credentials/postern.service/masterkey /usr/local/bin/postern +``` + +You will need to create a user, a backup-mx user for email delivery and a DKIM key. + +```Shell +postern user add testuser +postern user add-address testuser user@example.com +postern backup-mx add p25dev +postern dkim add selector example.com # Don't forget to enable the key with 'dkim enable selector example.com' +``` + +If you want a smooth start for sending and receiving emails, make an account at [p25.dev](https://p25.dev) to integrate +Postern with it. + +Contributing via Mailing List +----------------------------- + +We manage development via plain-text email patches rather than web-based pull requests. + +**Mailing list:** `postern-dev@lists.p25.dev` (subscribe with an email to `postern-dev+subscribe@lists.p25.dev`) + +Use `git send-email` to contribute your patch. For a tutorial see [https://git-send-email.io](https://git-send-email.io/). +It is okay to use the mailing list as a chat rather than a formalized procedure. Just say 'hi'! + +> [!TIP] +> You can integrate `git send-email` with Postern too! + +Security +-------- + +Please do not disclose security vulnerabilities via the public mailing list. +Instead, use `https://p25.dev/.well-known/security.txt`
A
go.mod
@@ -0,0 +1,31 @@
+module postern + +go 1.26.5 + +require ( + github.com/emersion/go-imap/v2 v2.0.0-beta.8 + github.com/emersion/go-msgauth v0.7.0 + github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6 + github.com/emersion/go-smtp v0.25.0 + github.com/google/uuid v1.6.0 + github.com/minio/sio v0.5.1 + github.com/spf13/cobra v1.10.2 + go.starlark.net v0.0.0-20260708150628-5395d018f003 + golang.org/x/crypto v0.55.0 + golang.org/x/term v0.45.0 + modernc.org/sqlite v1.56.0 +) + +require ( + github.com/dustin/go-humanize v1.0.1 // indirect + github.com/emersion/go-message v0.18.2 // indirect + github.com/inconshreveable/mousetrap v1.1.0 // indirect + github.com/mattn/go-isatty v0.0.24 // indirect + github.com/ncruces/go-strftime v1.0.0 // indirect + github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect + github.com/spf13/pflag v1.0.10 // indirect + golang.org/x/sys v0.47.0 // indirect + modernc.org/libc v1.75.3 // indirect + modernc.org/mathutil v1.7.1 // indirect + modernc.org/memory v1.12.0 // indirect +)
A
go.sum
@@ -0,0 +1,114 @@
+github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/emersion/go-imap/v2 v2.0.0-beta.8 h1:5IXZK1E33DyeP526320J3RS7eFlCYGFgtbrfapqDPug= +github.com/emersion/go-imap/v2 v2.0.0-beta.8/go.mod h1:dhoFe2Q0PwLrMD7oZw8ODuaD0vLYPe5uj2wcOMnvh48= +github.com/emersion/go-message v0.18.2 h1:rl55SQdjd9oJcIoQNhubD2Acs1E6IzlZISRTK7x/Lpg= +github.com/emersion/go-message v0.18.2/go.mod h1:XpJyL70LwRvq2a8rVbHXikPgKj8+aI0kGdHlg16ibYA= +github.com/emersion/go-msgauth v0.7.0 h1:vj2hMn6KhFtW41kshIBTXvp6KgYSqpA/ZN9Pv4g1INc= +github.com/emersion/go-msgauth v0.7.0/go.mod h1:mmS9I6HkSovrNgq0HNXTeu8l3sRAAuQ9RMvbM4KU7Ck= +github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6 h1:oP4q0fw+fOSWn3DfFi4EXdT+B+gTtzx8GC9xsc26Znk= +github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6/go.mod h1:iL2twTeMvZnrg54ZoPDNfJaJaqy0xIQFuBdrLsmspwQ= +github.com/emersion/go-smtp v0.25.0 h1:krfiHrme2JbJYDh0DGuSRbvPpbnQTH/v9CIfPincl1I= +github.com/emersion/go-smtp v0.25.0/go.mod h1:ZtRRkbTyp2XTHCA+BmyTFTrj8xY4I+b4McvHxCU2gsQ= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 h1:LMLX+LgTNWpfvCBdFebv6EsYotImrt/Ppc5cXIriCSo= +github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= +github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= +github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= +github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI= +github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A= +github.com/minio/sio v0.5.1 h1:sqtImrnCSHbDqO/lVy3tfbsctHzfelDv3NbXWEVcWT8= +github.com/minio/sio v0.5.1/go.mod h1:4ANoe4CCXqnt1FCiLM0+vlBUhhWZzVOhYCz0069KtFc= +github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w= +github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= +github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU= +github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4= +github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk= +github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= +go.starlark.net v0.0.0-20260708150628-5395d018f003 h1:cAxcqHgW8fnmT0cEBU3TzvVYHIFt8IIGDMWUF6rImk4= +go.starlark.net v0.0.0-20260708150628-5395d018f003/go.mod h1:Iue6g6iirlfLoVi/DYCi5/x0h/bAOuWF3dULTKpt2Vo= +go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= +golang.org/x/crypto v0.55.0 h1:+KWHjbgOaAQ66dh/YlkZKHlz9ZUlq61AFirAR9ntP8M= +golang.org/x/crypto v0.55.0/go.mod h1:uq0V9dE/fzQuJtbnL+2EhWOE63vo164FY8xqEnV9xis= +golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= +golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= +golang.org/x/mod v0.38.0 h1:MECBjubtXD7yj4HrhIUcywNaGeNVUdfVnxmPajOk4yk= +golang.org/x/mod v0.38.0/go.mod h1:V6Xz0pq8TQ3dGqVQ1FVHuelZpAL0uNhSkk9ogYP3c40= +golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= +golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= +golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek= +golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= +golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= +golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= +golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0= +golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= +golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= +golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= +golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= +golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU= +golang.org/x/tools v0.48.0 h1:3+hClM1aLL5mjMKm5ovokw9epgRXPuu2tILgismM6RE= +golang.org/x/tools v0.48.0/go.mod h1:08xX0orndb/F7jJxGDicx061tyd5pcMto75YMAXr6lk= +golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE= +google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +modernc.org/cc/v4 v4.29.2 h1:h6+9ciCnPKutf4I03CvheAvDLX7+IHlqR6Iy6J+cgd8= +modernc.org/cc/v4 v4.29.2/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI= +modernc.org/ccgo/v4 v4.35.0 h1:F+TUsmw09QxLzmi3aeYYGxjAXarmZaKgj3mKQHNaA8w= +modernc.org/ccgo/v4 v4.35.0/go.mod h1:qrVGs9S3Sr2Ztcg9ve+kTAYMp5a3YvWjo+SoN06kJ5I= +modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM= +modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU= +modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI= +modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito= +modernc.org/gc/v3 v3.1.5 h1:21ldfPfRYE31Tb7B3mwAK8gy1AxP4+dKjrOQPfqakoc= +modernc.org/gc/v3 v3.1.5/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY= +modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks= +modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI= +modernc.org/libc v1.75.3 h1:vCqT5+R0jPXMnvMkGo0T2zXvFNth+lYXVCx5X7CCX/g= +modernc.org/libc v1.75.3/go.mod h1:MjAX68G+0oufI+hNuh0QXcK+Ap+sL8bNPPcIC6EqOfo= +modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU= +modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg= +modernc.org/memory v1.12.0 h1:twkmYNkGXCvtYWzoux02jtK6eovjZbdI0uHFUYp6kuU= +modernc.org/memory v1.12.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw= +modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg= +modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns= +modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w= +modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE= +modernc.org/sqlite v1.56.0 h1:/D8e2RfFqoy/Zc6PuC76U28zFwmI/sYx1Kjm4yEn9e0= +modernc.org/sqlite v1.56.0/go.mod h1:yCJ2cmAaIkHQ25oXWrF8H4O1lIfPYPR26yCEDj2P3pQ= +modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0= +modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A= +modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y= +modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
A
internal/cli/admin.go
@@ -0,0 +1,376 @@
+package cli + +import ( + "crypto/rand" + "errors" + "fmt" + "log/slog" + "math/big" + "net" + "net/mail" + "os" + "strings" + "text/tabwriter" + + "postern/internal/db" + + "github.com/spf13/cobra" + "golang.org/x/crypto/bcrypt" +) + +func (c *Command) addUser(cmd *cobra.Command, args []string) error { + cmd.SilenceUsage = true // suppress runtime error help text + ctx := cmd.Context() + username := args[0] + + if exists, err := c.db.UserExists(ctx, username); err != nil || exists { + if err != nil { + return err + } + return fmt.Errorf("user %q already exists", username) + } + + password, err := generateSafeToken(16) + if err != nil { + return err + } + + hash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost) + if err != nil { + return err + } + + tx, err := c.db.GetWriteTx(ctx) + if err != nil { + return fmt.Errorf("beginning transaction: %w", err) + } + + userID, err := db.InsertUser(ctx, tx, username, hash[:]) + if err != nil { + txErr := tx.Rollback() + if txErr != nil { + slog.Error("rolling back transaction", "error", err) + } + return fmt.Errorf("inserting user %q: %w", username, err) + } + + if err := db.InsertDefaultMailboxes(ctx, tx, userID); err != nil { + txErr := tx.Rollback() + if txErr != nil { + slog.Error("rolling back transaction", "error", err) + } + return fmt.Errorf("inserting default mailboxes: %w", err) + } + + if err := tx.Commit(); err != nil { + return fmt.Errorf("committing transaction: %w", err) + } + + // subscribe user to all mailboxes + allMailboxes, err := c.db.GetUserMailboxes(ctx, userID) + if err != nil { + return err + } + + for _, mb := range allMailboxes { + err = c.db.Subscribe(ctx, userID, mb.ID, mb.Name) + if err != nil { + return fmt.Errorf("subscribing mailbox %q: %w", mb.Name, err) + } + slog.Debug("subscribed mailbox", "mailbox", mb.Name) + } + + _, _ = fmt.Fprintf(cmd.OutOrStdout(), "User %q created. Your password will only be shown once.\n", username) + _, _ = fmt.Fprintf(cmd.OutOrStdout(), "Password: %s\n", password) + return nil +} + +func (c *Command) removeUser(cmd *cobra.Command, args []string) error { + cmd.SilenceUsage = true // suppress runtime error help text + ctx := cmd.Context() + username := args[0] + + if exists, err := c.db.UserExists(ctx, username); err != nil || !exists { + if err != nil { + return err + } + return fmt.Errorf("user %q does not exist", username) + } + + err := c.db.RemoveUser(ctx, username) + if err != nil { + return fmt.Errorf("removing user %q: %w", username, err) + } + + _, _ = fmt.Fprintf(cmd.OutOrStdout(), "User %q removed.\n", username) + return nil +} + +func (c *Command) listUsers(cmd *cobra.Command, _ []string) error { + cmd.SilenceUsage = true + + users, err := c.db.ListUsers(cmd.Context()) + if err != nil { + fmt.Println("Error listing users") + return err + } + + w := tabwriter.NewWriter(os.Stdout, 1, 1, 1, ' ', 0) + _, _ = fmt.Fprintln(w, "Name") + for _, user := range users { + _, _ = fmt.Fprintf(w, "%s\n", user.Name) + } + _ = w.Flush() + + return nil +} + +func (c *Command) addUserAddress(cmd *cobra.Command, args []string) error { + cmd.SilenceUsage = true // suppress runtime error help text + ctx := cmd.Context() + username := args[0] + address := args[1] + + parsedAddress, err := mail.ParseAddress(address) + if err != nil { + fmt.Printf("Could not parse address: %s\n", address) + return err + } + + i := strings.LastIndex(parsedAddress.Address, "@") + if i == -1 { + fmt.Printf("invalid address does not have a domain part: %s\n", parsedAddress.Address) + return errors.New("invalid address") + } + + err = c.db.InsertUserAddress(ctx, username, parsedAddress.Address) + if err != nil { + return err + } + + fmt.Printf("address <%s> added for %q\n", address, username) + return nil +} + +func (c *Command) addDKIM(cmd *cobra.Command, args []string) error { + cmd.SilenceUsage = true + selector := args[0] + domain := args[1] + + record, err := c.dkim.Add(cmd.Context(), selector, domain) + if err != nil { + return err + } + + fmt.Println("Create a DNS TXT record for", domainkeyName(selector, domain)) + fmt.Println("Remember to enable this selector once it was made available through DNS. (dkim enable)") + fmt.Println(record) + return nil +} + +func (c *Command) listDKIM(cmd *cobra.Command, _ []string) error { + cmd.SilenceUsage = true // suppress runtime error help text + + dkims, err := c.dkim.List(cmd.Context()) + if err != nil { + fmt.Println("Error list DKIM") + return err + } + + if len(dkims) == 0 { + fmt.Println("No DKIM records found") + return nil + } + + w := tabwriter.NewWriter(os.Stdout, 1, 1, 1, ' ', 0) + _, _ = fmt.Fprintln(w, "Identifier\tKey Type\tEnabled") + for _, dkim := range dkims { + _, _ = fmt.Fprintf(w, "%s\t%s\t%v\n", domainkeyName(dkim.Selector, dkim.Domain), dkim.KeyType, dkim.Enabled) + } + _ = w.Flush() + + return nil +} + +func (c *Command) getDKIM(cmd *cobra.Command, args []string) error { + cmd.SilenceUsage = true + selector := args[0] + domain := args[1] + + record, err := c.dkim.PublicDNSRecord(cmd.Context(), selector, domain) + if err != nil { + return err + } + + fmt.Println("Create a DNS TXT record for", domainkeyName(selector, domain)) + fmt.Println("Remember to enable this selector once it was made available through DNS. (dkim enable)") + fmt.Println(record) + + return nil +} + +func (c *Command) setDKIMEnabled(cmd *cobra.Command, args []string, enable bool) error { + cmd.SilenceUsage = true + selector := args[0] + domain := args[1] + + if enable { + force, err := cmd.Flags().GetBool("force") + if err != nil { + return err + } + + if !force { + s := domainkeyName(selector, domain) + records, err := net.LookupTXT(s) + if err != nil { + fmt.Println("resolving TXT records: %w", err) + } + + expected, err := c.dkim.PublicDNSRecord(cmd.Context(), selector, domain) + if err != nil { + return err + } + + if expected != strings.Join(records, "") { + fmt.Println("The DKIM record in DNS does not match the expected record.") + fmt.Println("To enable this record anyway, use '--force'.") + fmt.Println("Expected:") + fmt.Println(expected) + fmt.Println("Actual:") + fmt.Println(strings.Join(records, "")) + return errors.New("DKIM record does not match the expected record") + } + } + + err = c.dkim.SetEnabled(cmd.Context(), selector, domain, true) + if err != nil { + return err + } + fmt.Printf("Enabling DKIM selector %q for domain %q\n", selector, domain) + } else { + if err := c.dkim.SetEnabled(cmd.Context(), selector, domain, false); err != nil { + return err + } + fmt.Printf("Disabling DKIM selector %q for domain %q\n", selector, domain) + } + return nil +} + +func (c *Command) removeDKIM(cmd *cobra.Command, args []string) error { + cmd.SilenceUsage = true + selector := args[0] + domain := args[1] + + return c.dkim.Remove(cmd.Context(), selector, domain) +} + +func (c *Command) listSMTPAuth(cmd *cobra.Command, _ []string) error { + cmd.SilenceUsage = true + + smtpAuths, err := c.db.ListSMTPAuthUsers(cmd.Context()) + if err != nil { + fmt.Println("Error listing SMTP auth users") + return err + } + + w := tabwriter.NewWriter(os.Stdout, 1, 1, 1, ' ', 0) + _, _ = fmt.Fprintln(w, "Name") + for _, auth := range smtpAuths { + _, _ = fmt.Fprintf(w, "%s\n", auth.Name) + } + _ = w.Flush() + + return nil +} + +func (c *Command) addSMTPAuth(cmd *cobra.Command, args []string) error { + cmd.SilenceUsage = true // suppress runtime error help text + ctx := cmd.Context() + name := args[0] + + if exists, err := c.db.SMTPAuthNameExists(ctx, name); err != nil || exists { + if err != nil { + return err + } + return fmt.Errorf("user %q already exists", name) + } + + password, err := generateSafeToken(16) + if err != nil { + return err + } + + hash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost) + if err != nil { + return err + } + + err = c.db.InsertSMTPAuthUser(ctx, name, hash[:]) + if err != nil { + return fmt.Errorf("inserting smtp auth %q: %w", name, err) + } + + _, _ = fmt.Fprintf(cmd.OutOrStdout(), "SMTP Auth %q created. The password will only be shown once.\n", name) + _, _ = fmt.Fprintf(cmd.OutOrStdout(), "Password: %s\n", password) + return nil +} + +func (c *Command) removeSMTPAuth(cmd *cobra.Command, args []string) error { + cmd.SilenceUsage = true // suppress runtime error help text + ctx := cmd.Context() + name := args[0] + + if exists, err := c.db.SMTPAuthNameExists(ctx, name); err != nil || !exists { + if err != nil { + return err + } + return fmt.Errorf("user %q does not exist", name) + } + + err := c.db.RemoveSMTPAuthUser(ctx, name) + if err != nil { + return err + } + + _, _ = fmt.Fprintf(cmd.OutOrStdout(), "User %q removed.\n", name) + return nil +} + +func domainkeyName(selector, domain string) string { + if !strings.HasSuffix(domain, ".") { + domain = domain + "." + } + return fmt.Sprintf("%s._domainkey.%s", selector, domain) +} + +// Define the character set: +// - Numbers: 2-9 (No 0, 1) +// - Uppercase: A-Z (No I, O) +// - Lowercase: a-z (No l, o) +const letterBytes = "23456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnpqrstuvwxyz" + +func generateSafeToken(length int) (string, error) { + if length <= 0 { + return "", fmt.Errorf("length must be greater than 0") + } + + ret := make([]byte, length) + + // We calculate the max index based on the charset length + charsetLen := big.NewInt(int64(len(letterBytes))) + + for i := range length { + // rand.Int returns a uniform random value in [0, max). + // It automatically handles modulo bias, ensuring an even distribution. + num, err := rand.Int(rand.Reader, charsetLen) + if err != nil { + return "", err + } + + // Use the random number as an index to pick a character + ret[i] = letterBytes[num.Int64()] + } + + return string(ret), nil +}
A
internal/cli/admin_test.go
@@ -0,0 +1,100 @@
+package cli + +import ( + "bytes" + "crypto/rand" + "log" + "os" + "path" + "strings" + "testing" + + "postern/internal/db" + "postern/internal/persistence" + + "github.com/spf13/cobra" +) + +// executeCommand helps invoke a cobra command with specific arguments and captures its stdout/stderr. +func executeCommand(cmd *cobra.Command, args ...string) (string, error) { + buf := new(bytes.Buffer) + cmd.SetOut(buf) + cmd.SetErr(buf) + cmd.SetArgs(args) + + err := cmd.Execute() + return buf.String(), err +} + +// TestAddUserCommand checks the loop of creating a new user, finding it in the db and checking that the password works. +func TestAddRemoveUserCommand(t *testing.T) { + f := t.TempDir() + + // create temporary masterkey + mkeyPath := path.Join(f, "masterkey") + secretKey := make([]byte, 32) + _, _ = rand.Read(secretKey) + err := os.WriteFile(mkeyPath, secretKey, 0666) + if err != nil { + t.Fatal(err) + } + + p, err := persistence.New(persistence.Config{ + BlobRoot: path.Join(f, "blobs"), + MasterkeyFile: mkeyPath, + }) + if err != nil { + log.Fatal(err) + } + + posternDB, err := db.OpenDB(&db.Config{ + DBPath: path.Join(f, "meta.db"), + }) + if err != nil { + log.Fatal(err) + } + + c, err := New(&Config{ + DB: posternDB, + Persistence: p, + }) + if err != nil { + t.Fatal(err) + } + + output, err := executeCommand(c.rootCmd(), "user", "add", "user1") + if err != nil { + t.Fatalf("expected no error, got: %v", err) + } + + _, after, found := strings.Cut(output, "Password: ") + password := strings.TrimSpace(after) + if !found { + t.Fatal("expected to find a password in output. Found none") + } + + u, err := c.db.GetUser(t.Context(), "user1") + if err != nil { + t.Fatalf("expected no error, got: %v", err) + } + + err = u.VerifyPassword([]byte(password)) + if err != nil { + t.Fatalf("password mismatch, got: %v", err) + } + + // Try removing user + output, err = executeCommand(c.rootCmd(), "user", "remove", "user1") + if err != nil { + t.Fatalf("expected no error, got: %v", err) + } + if !strings.Contains(output, "removed") { + t.Fatal("expected to find 'removed' in output. Found none") + } + + // Try removing user again (failure) + output, err = executeCommand(c.rootCmd(), "user", "remove", "user1") + if err == nil { + t.Errorf("expected error when deleting non-existing user, got none") + } +}
A
internal/cli/cli.go
@@ -0,0 +1,234 @@
+package cli + +import ( + "context" + "fmt" + "log/slog" + "os" + "os/signal" + "sync" + "syscall" + + "postern/internal/db" + "postern/internal/dkim" + "postern/internal/persistence" + + "github.com/spf13/cobra" +) + +type Service interface { + Start() error + Stop() error + Status() string +} + +type Config struct { + DB *db.DB + Persistence *persistence.Persistence + DKIM *dkim.DKIM + Services map[string]Service +} + +type Command struct { + db *db.DB + persistence *persistence.Persistence + dkim *dkim.DKIM + services map[string]Service +} + +func New(config *Config) (*Command, error) { + return &Command{ + db: config.DB, + persistence: config.Persistence, + dkim: config.DKIM, + services: config.Services, + }, nil +} + +func (c *Command) rootCmd() *cobra.Command { + rootCmd := &cobra.Command{ + Use: "postern", + Short: "Modern mail management", + } + + serveCmd := &cobra.Command{ + Use: "serve", + Short: "Start the mail server", + RunE: c.startServer, + } + + pruneCmd := &cobra.Command{ + Use: "prune", + Short: "Delete orphaned mails", + RunE: c.pruneMails, + } + + userCmd := &cobra.Command{ + Use: "user", + Short: "Manage local login accounts", + } + + userListCmd := &cobra.Command{ + Use: "list", + Short: "List all users", + RunE: c.listUsers, + } + + addUserCmd := &cobra.Command{ + Use: "add <username>", + Short: "Add a user", + Args: cobra.ExactArgs(1), + RunE: c.addUser, + } + + addUserAddressCmd := &cobra.Command{ + Use: "add-address <username> <address>", + Short: "Add an address for a user", + Args: cobra.ExactArgs(2), + RunE: c.addUserAddress, + } + + userRemoveCmd := &cobra.Command{ + Use: "remove <username>", + Short: "Remove a user", + Args: cobra.ExactArgs(1), + RunE: c.removeUser, + } + + userCmd.AddCommand(userListCmd) + userCmd.AddCommand(addUserCmd) + userCmd.AddCommand(addUserAddressCmd) + userCmd.AddCommand(userRemoveCmd) + + backupMxCmd := &cobra.Command{ + Use: "backup-mx", + Short: "Manage inbound SMTP authentication", + } + + backupMxListCmd := &cobra.Command{ + Use: "list", + Short: "List all SMTP auth users", + Args: cobra.NoArgs, + RunE: c.listSMTPAuth, + } + + backupMxAddCmd := &cobra.Command{ + Use: "add <name>", + Short: "Add SMTP auth user", + Args: cobra.ExactArgs(1), + RunE: c.addSMTPAuth, + } + + backupMxRemoveCmd := &cobra.Command{ + Use: "remove <name>", + Short: "Remove SMTP auth user", + Args: cobra.ExactArgs(1), + RunE: c.removeSMTPAuth, + } + + backupMxCmd.AddCommand(backupMxListCmd) + backupMxCmd.AddCommand(backupMxAddCmd) + backupMxCmd.AddCommand(backupMxRemoveCmd) + + dkimCmd := &cobra.Command{ + Use: "dkim", + Short: "Manage DKIM selectors", + } + + dkimListCmd := &cobra.Command{ + Use: "list", + Short: "List all available selectors", + Args: cobra.NoArgs, + RunE: c.listDKIM, + } + + addDKIMCmd := &cobra.Command{ + Use: "add [selector] [domain]", + Short: "Add DKIM selector for domain", + Args: cobra.ExactArgs(2), + RunE: c.addDKIM, + } + + getDKIMCmd := &cobra.Command{ + Use: "get [selector] [domain]", + Short: "Get public DKIM record for the given selector and domain", + Args: cobra.ExactArgs(2), + RunE: c.getDKIM, + } + + enableDKIMCmd := &cobra.Command{ + Use: "enable [selector] [domain]", + Short: "Enable DKIM selector", + Args: cobra.ExactArgs(2), + RunE: func(cmd *cobra.Command, args []string) error { + return c.setDKIMEnabled(cmd, args, true) + }, + } + enableDKIMCmd.Flags().BoolP("force", "f", false, "Force enable DKIM even if validation fails") + + disableDKIMCmd := &cobra.Command{ + Use: "disable [selector] [domain]", + Short: "Disable DKIM selector", + Args: cobra.ExactArgs(2), + RunE: func(cmd *cobra.Command, args []string) error { + return c.setDKIMEnabled(cmd, args, false) + }, + } + + dkimRemoveCmd := &cobra.Command{ + Use: "remove [selector] [domain]", + Short: "Remove DKIM selector", + Args: cobra.ExactArgs(2), + RunE: c.removeDKIM, + } + + dkimCmd.AddCommand(dkimListCmd) + dkimCmd.AddCommand(addDKIMCmd) + dkimCmd.AddCommand(getDKIMCmd) + dkimCmd.AddCommand(enableDKIMCmd) + dkimCmd.AddCommand(disableDKIMCmd) + dkimCmd.AddCommand(dkimRemoveCmd) + + rootCmd.AddCommand(serveCmd) + rootCmd.AddCommand(pruneCmd) + rootCmd.AddCommand(userCmd) + rootCmd.AddCommand(backupMxCmd) + rootCmd.AddCommand(dkimCmd) + + return rootCmd +} + +func (c *Command) Execute() error { + return c.rootCmd().Execute() +} + +func (c *Command) startServer(_ *cobra.Command, _ []string) error { + fmt.Println("Starting postern server...") + slog.SetLogLoggerLevel(slog.LevelDebug) + + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) + defer stop() + + var wg sync.WaitGroup + for name, svc := range c.services { + wg.Go(func() { + err := svc.Start() + if err != nil { + slog.Error("running service", "name", name, "error", err) + } + }) + } + + go func() { + <-ctx.Done() + slog.Info("shutdown signal received, stopping services") + for name, svc := range c.services { + if err := svc.Stop(); err != nil { + slog.Error("stopping service", "name", name, "error", err) + } + } + }() + + wg.Wait() + return nil +}
A
internal/cli/prune.go
@@ -0,0 +1,31 @@
+package cli + +import ( + "context" + "fmt" + + "github.com/spf13/cobra" +) + +func (c *Command) pruneMails(cmd *cobra.Command, _ []string) error { + fmt.Println("Pruning in progress...") + + targets, err := c.db.GetPrunableTargets(context.Background()) + if err != nil { + return err + } + + for _, target := range targets { + err = c.persistence.RemoveBlob(target.BlobHash) + if err != nil { + return err + } + err = c.db.DeleteMessage(context.Background(), target.MessageID) + } + + if err != nil { + return err + } + fmt.Println("Pruning complete.") + return nil +}
A
internal/db/address.go
@@ -0,0 +1,57 @@
+package db + +import ( + "context" + "database/sql" + "errors" + "time" + + "postern/internal/model" +) + +func (db *DB) InsertUserAddress(ctx context.Context, username, address string) error { + tx, err := db.write.BeginTx(ctx, nil) + if err != nil { + return err + } + defer txRollback(tx) + + // Get userID + var userID int + row := tx.QueryRowContext(ctx, `SELECT id FROM users WHERE name = ?`, username) + if err = row.Scan(&userID); err != nil { + if errors.Is(err, sql.ErrNoRows) { + return errors.New("user not found") + } + return err + } + + // Insert address + _, err = tx.ExecContext(ctx, `INSERT INTO addresses (user_id, name, created_at) VALUES (?, ?, ?)`, userID, address, time.Now().Unix()) + if err != nil { + return err + } + return tx.Commit() +} + +func (db *DB) GetUserForAddress(ctx context.Context, address string) (model.User, error) { + res := db.read.QueryRowContext(ctx, `SELECT users.id, users.name, users.password_hash FROM users INNER JOIN addresses ON users.id = addresses.user_id WHERE addresses.name = ?`, address) + + var u model.User + var pw []byte + err := res.Scan(&u.ID, &u.Name, &pw) + u.SetPassword(pw) + return u, err +} + +func (db *DB) GetGatekeepDecision(ctx context.Context, userID int, fromAddress string) (string, error) { + res := db.read.QueryRowContext(ctx, `SELECT destination FROM gatekeepers WHERE from_address = ? and user_id = ?`, fromAddress, userID) + + var mailbox string + err := res.Scan(&mailbox) + if err != nil { + return "", err + } + + return mailbox, nil +}
A
internal/db/db.go
@@ -0,0 +1,70 @@
+package db + +import ( + "context" + "database/sql" + "fmt" + "strings" + + _ "modernc.org/sqlite" +) + +type Config struct { + DBPath string +} +type DB struct { + write *sql.DB + read *sql.DB +} + +func OpenDB(config *Config) (*DB, error) { + ctx := context.Background() + dsn := config.DBPath + "?_pragma=foreign_keys(1)" + + writeDB, err := sql.Open("sqlite", dsn) + if err != nil { + return nil, fmt.Errorf("opening meta.db for writes: %w", err) + } + // SQLite only supports one writer at a time; enforce it at the pool level. + writeDB.SetMaxOpenConns(1) + + readDB, err := sql.Open("sqlite", dsn) + if err != nil { + _ = writeDB.Close() + return nil, fmt.Errorf("opening meta.db for reads: %w", err) + } + + db := &DB{write: writeDB, read: readDB} + + if err := db.applyPragmas(ctx); err != nil { + return nil, fmt.Errorf("applying pragmas: %w", err) + } + + if err := db.applySchema(ctx); err != nil { + return nil, fmt.Errorf("applying schema: %w", err) + } + + if err := writeDB.Ping(); err != nil { + _ = writeDB.Close() + return nil, fmt.Errorf("pinging meta.db write pool: %w", err) + } + + if err := readDB.Ping(); err != nil { + _ = writeDB.Close() + _ = readDB.Close() + return nil, fmt.Errorf("pinging meta.db read pool: %w", err) + } + + return db, nil +} + +func (db *DB) GetWriteTx(ctx context.Context) (*sql.Tx, error) { + return db.write.BeginTx(ctx, nil) +} + +func escapeLike(s string) string { + s = strings.ReplaceAll(s, `\`, `\\`) + s = strings.ReplaceAll(s, `%`, `\%`) + s = strings.ReplaceAll(s, `_`, `\_`) + return s +}
A
internal/db/dkim.go
@@ -0,0 +1,93 @@
+package db + +import ( + "context" + "database/sql" + "strings" + "time" + + "postern/internal/model" +) + +// GetDKIMForDomain deterministically returns the first enabled selector for the given domain +func (db *DB) GetDKIMForDomain(ctx context.Context, domain string) (model.DKIM, error) { + row := db.read.QueryRowContext( + ctx, + `SELECT id, selector, blob_address, key_type FROM dkim WHERE domain = ? AND enabled = 1 ORDER BY selector LIMIT 1`, domain) + var d model.DKIM + err := row.Scan(&d.ID, &d.Selector, &d.BlobAddress, &d.KeyType) + return d, err +} + +func (db *DB) GetDKIMForSelectorDomain(ctx context.Context, selector, domain string) (model.DKIM, error) { + domain = strings.TrimSuffix(domain, ".") + row := db.read.QueryRowContext( + ctx, + `SELECT id, selector, blob_address, key_type, enabled FROM dkim WHERE domain = ? AND selector = ?`, domain, selector) + if row.Err() != nil { + return model.DKIM{}, row.Err() + } + + var d model.DKIM + var enabledToggle int + err := row.Scan(&d.ID, &d.Selector, &d.BlobAddress, &d.KeyType, &enabledToggle) + if err != nil { + return model.DKIM{}, err + } + if enabledToggle == 1 { + d.Enabled = true + } + return d, nil +} + +func (db *DB) ListDKIM(ctx context.Context) ([]model.DKIM, error) { + rows, err := db.read.QueryContext( + ctx, + `SELECT id, domain, selector, blob_address, key_type, enabled FROM dkim ORDER BY domain, selector, key_type, enabled DESC`) + if err != nil { + return nil, err + } + defer func(rows *sql.Rows) { + _ = rows.Close() + }(rows) + + out := make([]model.DKIM, 0) + for rows.Next() { + var d model.DKIM + var enabledToggle int + err = rows.Scan(&d.ID, &d.Domain, &d.Selector, &d.BlobAddress, &d.KeyType, &enabledToggle) + if err != nil { + return nil, err + } + if enabledToggle == 1 { + d.Enabled = true + } + out = append(out, d) + } + return out, nil +} + +func (db *DB) SetDKIMEnabled(ctx context.Context, selector, domain string, enabled bool) error { + enableToggle := 0 + if enabled { + enableToggle = 1 + } + _, err := db.write.ExecContext( + ctx, + `UPDATE dkim SET enabled = ? WHERE domain = ? AND selector = ?`, enableToggle, domain, selector) + return err +} + +func (db *DB) InsertDKIMForDomain(ctx context.Context, d model.DKIM) error { + _, err := db.write.ExecContext( + ctx, ` +INSERT INTO dkim (domain, selector, blob_address, key_type, created_at, enabled) +VALUES (?, ?, ?, ?, ?, ?)`, d.Domain, d.Selector, d.BlobAddress, d.KeyType, time.Now().Unix(), d.Enabled) + return err +} + +func RemoveDKIM(ctx context.Context, tx *sql.Tx, selector, domain string) error { + domain = strings.TrimSuffix(domain, ".") + _, err := tx.ExecContext(ctx, `DELETE FROM dkim WHERE domain = ? AND selector = ?`, domain, selector) + return err +}
A
internal/db/errors.go
@@ -0,0 +1,5 @@
+package db + +import "errors" + +var ErrMailboxNotFound = errors.New("mailbox not found")
A
internal/db/flag.go
@@ -0,0 +1,31 @@
+package db + +import ( + "context" + "database/sql" + "log/slog" +) + +func (db *DB) GetMailboxFlags(ctx context.Context, mailboxID int) ([]string, error) { + rows, err := db.read.QueryContext(ctx, `SELECT DISTINCT flag +FROM message_flags +WHERE mailbox_id = ?`, mailboxID) + if err != nil { + return nil, err + } + defer func(rows *sql.Rows) { + err := rows.Close() + if err != nil { + slog.Error("closing rows", "error", err) + } + }(rows) + var flags []string + for rows.Next() { + var flag string + if err := rows.Scan(&flag); err != nil { + return nil, err + } + flags = append(flags, flag) + } + return flags, nil +}
A
internal/db/mailbox.go
@@ -0,0 +1,271 @@
+package db + +import ( + "context" + "database/sql" + "errors" + "fmt" + "log/slog" + + "postern/internal/model" +) + +func (db *DB) GetUserMailbox(ctx context.Context, userID int, mailboxName string) (model.Mailbox, error) { + mailboxes, err := db.GetUserMailboxes(ctx, userID) + if err != nil { + return model.Mailbox{}, err + } + + for _, mb := range mailboxes { + if mb.Name == mailboxName { + return mb, nil + } + } + return model.Mailbox{}, ErrMailboxNotFound +} + +// GetUserMailboxes returns all existing mailboxes with their calculated IMAP status values. +func (db *DB) GetUserMailboxes(ctx context.Context, userID int) ([]model.Mailbox, error) { + // This query safely aggregates counts using subqueries, preventing + // Cartesian products if a message has multiple flags. + query := ` + SELECT + mb.id, + mb.name, + mb.special_use, + mb.uidnext, + mb.uidvalidity, + mb.highest_modseq, + (s.id IS NOT NULL) AS is_subscribed, + (SELECT COUNT(*) FROM mailbox_messages mm WHERE mm.mailbox_id = mb.id AND mm.expunged_modseq IS NULL) AS num_messages, + (SELECT COALESCE(SUM(m.size), 0) FROM mailbox_messages mm JOIN messages m ON mm.message_id = m.id WHERE mm.mailbox_id = mb.id AND mm.expunged_modseq IS NULL) AS size, + (SELECT COUNT(*) FROM mailbox_messages mm WHERE mm.mailbox_id = mb.id AND mm.expunged_modseq IS NULL AND NOT EXISTS (SELECT 1 FROM message_flags mf WHERE mf.mailbox_id = mm.mailbox_id AND mf.uid = mm.uid AND mf.flag = '\Seen')) AS num_unseen, + (SELECT COUNT(*) FROM mailbox_messages mm WHERE mm.mailbox_id = mb.id AND mm.expunged_modseq IS NULL AND EXISTS (SELECT 1 FROM message_flags mf WHERE mf.mailbox_id = mm.mailbox_id AND mf.uid = mm.uid AND mf.flag = '\Deleted')) AS num_deleted + FROM mailboxes mb + LEFT JOIN subscriptions s ON mb.id = s.mailbox_id + WHERE mb.user_id = ? + ` + + rows, err := db.read.QueryContext(ctx, query, userID) + if err != nil { + return nil, err + } + defer func(rows *sql.Rows) { + _ = rows.Close() + }(rows) + + var mailboxes []model.Mailbox + for rows.Next() { + var mb model.Mailbox + var specialUse sql.NullString + + err := rows.Scan( + &mb.ID, + &mb.Name, + &specialUse, + &mb.UIDNext, + &mb.UIDValidity, + &mb.HighestModSeq, + &mb.IsSubscribed, + &mb.NumMessages, + &mb.Size, + &mb.NumUnseen, + &mb.NumDeleted, + ) + if err != nil { + return nil, err + } + + if specialUse.Valid { + mb.SpecialUse = new(specialUse.String) + } + mailboxes = append(mailboxes, mb) + } + + return mailboxes, rows.Err() +} + +func (db *DB) CreateMailbox(ctx context.Context, userID int, mailbox string) error { + tx, err := db.write.BeginTx(ctx, nil) + if err != nil { + return fmt.Errorf("beginning transaction: %w", err) + } + defer txRollback(tx) + + nextUID, err := nextUIDValidity(tx) + if err != nil { + return err + } + + _, err = tx.ExecContext(ctx, ` + INSERT INTO mailboxes + (user_id, name, uidvalidity, uidnext, highest_modseq) + VALUES + (?, ?, ?, 1, 0)`, + userID, mailbox, nextUID, + ) + if err != nil { + return fmt.Errorf("creating mailbox %q: %w", mailbox, err) + } + + return tx.Commit() +} + +func (db *DB) DeleteMailbox(ctx context.Context, userID int, mailbox string) error { + _, err := db.write.ExecContext(ctx, `DELETE FROM mailboxes WHERE user_id = ? AND name = ?`, userID, mailbox) + if err != nil { + return fmt.Errorf("deleting mailbox %q: %w", mailbox, err) + } + + return nil +} + +func (db *DB) RenameMailbox(ctx context.Context, userID int, oldName, newName string) error { + tx, err := db.write.BeginTx(ctx, nil) + if err != nil { + return fmt.Errorf("beginning transaction: %w", err) + } + defer txRollback(tx) + + _, err = tx.ExecContext(ctx, ` + UPDATE mailboxes + SET name = ? || SUBSTR(name, LENGTH(?)+1) + WHERE user_id = ? + AND (name = ? OR name LIKE ? ESCAPE '\') +`, newName, oldName, userID, oldName, escapeLike(oldName)+"/%") + if err != nil { + return fmt.Errorf("renaming mailbox %q: %v", oldName, err.Error()) + } + + return tx.Commit() +} + +// GetMailboxID returns the mailbox ID und UID Validity for the given mailbox name, scoped to the user ID +func (db *DB) GetMailboxID(ctx context.Context, userID int, mailbox string) (int, uint32, error) { + var id int + var uidValidity uint32 + err := db.read.QueryRowContext(ctx, "SELECT id, uidvalidity FROM mailboxes WHERE name = ? AND user_id = ?", mailbox, userID).Scan(&id, &uidValidity) + if errors.Is(err, sql.ErrNoRows) { + return 0, 0, ErrMailboxNotFound + } + return id, uidValidity, err +} + +// CreateAndMoveAllMessagesFromINBOXToMailbox is special handling for RENAME of mailbox INBOX +func (db *DB) CreateAndMoveAllMessagesFromINBOXToMailbox(ctx context.Context, userID int, toMailbox string) error { + tx, err := db.write.BeginTx(ctx, nil) + if err != nil { + return err + } + defer func(tx *sql.Tx) { + err := tx.Rollback() + if err != nil { + slog.Error("failed to rollback transaction", "error", err.Error()) + } + }(tx) + + // Rename INBOX -> toMailbox + _, err = tx.ExecContext(ctx, `UPDATE mailboxes SET name = ? WHERE user_id = ? AND name = 'INBOX'`, toMailbox, userID) + if err != nil { + return err + } + + // Create new INBOX mailbox + nextUID, err := nextUIDValidity(tx) + if err != nil { + return err + } + + _, err = tx.ExecContext(ctx, ` + INSERT INTO mailboxes + (user_id, name, uidvalidity, uidnext, highest_modseq) + VALUES + (?, 'INBOX', ?, 1, 0)`, + userID, nextUID, + ) + if err != nil { + return err + } + + return tx.Commit() +} + +// ListSubscribed fulfills IMAP LIST (SUBSCRIBED). +// It returns ONLY mailboxes that are subscribed to. The boolean map value +// indicates if the mailbox still exists (true) or was deleted (false). +func (db *DB) ListSubscribed(ctx context.Context, userID int) (map[model.Mailbox]bool, error) { + // INNER JOIN ensures we only get existing mailboxes the user is actually subscribed to. + // The UNION ALL handles mailboxes that were deleted (mailbox_id IS NULL) + // but still have a subscription record. + query := ` + SELECT mb.name, mb.special_use, 1 AS mb_exists + FROM subscriptions s + INNER JOIN mailboxes mb ON s.mailbox_id = mb.id + WHERE s.user_id = ? + + UNION ALL + + SELECT s.mailbox_name AS name, NULL AS special_use, 0 AS mb_exists + FROM subscriptions s + WHERE s.user_id = ? AND s.mailbox_id IS NULL + ` + + rows, err := db.read.QueryContext(ctx, query, userID, userID) + if err != nil { + return nil, err + } + defer func(rows *sql.Rows) { + _ = rows.Close() + }(rows) + + out := make(map[model.Mailbox]bool) + for rows.Next() { + var mb model.Mailbox + var exists bool + var specialUse sql.NullString + + if err := rows.Scan(&mb.Name, &specialUse, &exists); err != nil { + return nil, err + } + + if specialUse.Valid { + mb.SpecialUse = new(specialUse.String) + } + + mb.IsSubscribed = true + out[mb] = exists + } + + return out, rows.Err() +} + +func (db *DB) UIDToServerSeq(ctx context.Context, mailboxID int, uid uint32) (uint32, error) { + var seq uint32 + err := db.read.QueryRowContext(ctx, ` + SELECT COUNT(*) + FROM mailbox_messages + WHERE mailbox_id = ? AND expunged_modseq IS NULL AND uid <= ? + `, mailboxID, uid).Scan(&seq) + return seq, err // 0 means "no live message at or below this uid" +} + +func (db *DB) ServerSeqToUID(ctx context.Context, mailboxID int, seq uint32) (uint32, error) { + var uid uint32 + err := db.read.QueryRowContext(ctx, ` + SELECT uid FROM mailbox_messages + WHERE mailbox_id = ? AND expunged_modseq IS NULL + ORDER BY uid LIMIT 1 OFFSET ? + `, mailboxID, seq-1).Scan(&uid) // OFFSET is 0-based, seqnum 1-based + return uid, err +} + +// MailboxMessageCount returns the number of live (non-expunged) messages in the +// mailbox: the server-view EXISTS value, which is what RFC 9051 binds "*" to. +func (db *DB) MailboxMessageCount(ctx context.Context, mailboxID int) (uint32, error) { + var count uint32 + err := db.read.QueryRowContext(ctx, ` + SELECT COUNT(*) FROM mailbox_messages + WHERE mailbox_id = ? AND expunged_modseq IS NULL + `, mailboxID).Scan(&count) + return count, err +}
A
internal/db/message.go
@@ -0,0 +1,500 @@
+package db + +import ( + "context" + "database/sql" + "encoding/json" + "errors" + "fmt" + "log/slog" + "net/mail" + "time" + + "postern/internal/model" +) + +func (db *DB) GetAllMailboxMessages(ctx context.Context, userID, mailboxID int) ([]model.Message, error) { + query := ` +SELECT + mm.uid, + mm.modseq, + m.size, + m.blob_address, + mm.internal_date, + m.date, + m.subject, + m.message_id, + m.in_reply_to, + m.from_addr, + m.sender, + m.reply_to, + m.to_addr, + m.cc, + m.bcc, + coalesce(json_group_array(mf.flag), '[]') AS flags +FROM ( + SELECT + uid, + message_id, + modseq, + internal_date + FROM mailbox_messages + JOIN mailboxes ON mailboxes.id = mailbox_messages.mailbox_id + WHERE mailbox_messages.mailbox_id = ? + AND mailboxes.user_id = ? + AND mailbox_messages.expunged_modseq IS NULL +) numbered +JOIN mailbox_messages mm ON mm.mailbox_id = ? AND mm.uid = numbered.uid +JOIN messages m ON m.id = numbered.message_id +LEFT JOIN message_flags mf ON mf.mailbox_id = ? AND mf.uid = numbered.uid +GROUP BY mm.uid +ORDER BY mm.uid ASC +` + + rows, err := db.read.QueryContext(ctx, query, mailboxID, userID, mailboxID, mailboxID) + if err != nil { + return nil, err + } + defer func(rows *sql.Rows) { + err := rows.Close() + if err != nil { + slog.Error("closing rows", "error", err) + } + }(rows) + + out := make([]model.Message, 0) + for rows.Next() { + var m model.Message + var flagsJSON []byte + err := rows.Scan( + &m.UID, + &m.ModSeq, + &m.RFC822Size, + &m.BlobHash, + &m.InternalDate, + &m.EnvelopeDate, + &m.EnvelopeSubject, + &m.EnvelopeMessageID, + &m.EnvelopeInReplyTo, + &m.EnvelopeFrom, + &m.EnvelopeSender, + &m.EnvelopeReplyTo, + &m.EnvelopeTo, + &m.EnvelopeCc, + &m.EnvelopeBcc, + &flagsJSON, + ) + if err != nil { + return nil, fmt.Errorf("scanning message: %w", err) + } + m.ServerSeq = func(ctx context.Context) (uint32, error) { + serverSeq, err := db.UIDToServerSeq(ctx, mailboxID, m.UID) + if err != nil { + return 0, err + } + return serverSeq, nil + } + if len(flagsJSON) > 0 { + if err := json.Unmarshal(flagsJSON, &m.Flags); err != nil { + return nil, fmt.Errorf("unmarshaling flags: %w", err) + } + } else { + m.Flags = []string{} + } + out = append(out, m) + } + + return out, rows.Err() +} + +func (db *DB) GetMailboxMessagesByUID(ctx context.Context, userID, mailboxID int, uids []uint32) ([]model.Message, error) { + allMessages, err := db.GetAllMailboxMessages(ctx, userID, mailboxID) + if err != nil { + return nil, err + } + + uidSet := make(map[uint32]bool, len(uids)) + for _, uid := range uids { + uidSet[uid] = true + } + + out := make([]model.Message, 0, len(uids)) + for _, m := range allMessages { + if uidSet[m.UID] { + out = append(out, m) + } + } + + return out, nil +} + +func (db *DB) AppendMessage(ctx context.Context, mailboxID, userID int, + blobHash string, size int64, internalDate string, flags []string, parsedMsg *mail.Message) (uint32, error) { + + tx, err := db.write.BeginTx(ctx, nil) + if err != nil { + return 0, err + } + defer txRollback(tx) + + // 1. Allocate a UID and bump modseq, verifying mailbox ownership in one shot. + // RETURNING reflects post-update values, so uidnext-1 is the UID to assign. + var assignedUID uint32 + var newModSeq int64 + err = tx.QueryRowContext(ctx, ` + UPDATE mailboxes + SET uidnext = uidnext + 1, + highest_modseq = highest_modseq + 1 + WHERE id = ? AND user_id = ? + RETURNING uidnext - 1, highest_modseq + `, mailboxID, userID).Scan(&assignedUID, &newModSeq) + if errors.Is(err, sql.ErrNoRows) { + return 0, fmt.Errorf("mailbox %d not found for user %d", mailboxID, userID) + } + if err != nil { + return 0, fmt.Errorf("allocating uid: %w", err) + } + + // 2. Insert the message, or reuse the existing row for identical content. + // The no-op DO UPDATE forces RETURNING to yield the existing id on conflict. + var internalMessageID int64 + var subject, messageID, inReplyTo, fromAddr, sender, replyTo, toAddr, cc, bcc sql.NullString + var datestring sql.NullString + if parsedMsg != nil && parsedMsg.Header != nil { + date, err := parsedMsg.Header.Date() + if err == nil && !date.IsZero() { + datestring = sql.NullString{ + Valid: true, + String: date.UTC().Format(time.RFC3339), + } + } + + for headerKey, headerValue := range parsedMsg.Header { + switch headerKey { + case "Subject": + subject = sql.NullString{ + Valid: true, + String: headerValue[0], + } + case "Message-Id": + messageID = sql.NullString{ + Valid: true, + String: headerValue[0], + } + case "In-Reply-To": + inReplyTo = sql.NullString{ + Valid: true, + String: headerValue[0], + } + case "From": + fromAddr = sql.NullString{ + Valid: true, + String: headerValue[0], + } + case "Sender": + sender = sql.NullString{ + Valid: true, + String: headerValue[0], + } + case "Reply-To": + replyTo = sql.NullString{ + Valid: true, + String: headerValue[0], + } + case "To": + toAddr = sql.NullString{ + Valid: true, + String: headerValue[0], + } + case "Cc": + cc = sql.NullString{ + Valid: true, + String: headerValue[0], + } + case "Bcc": + bcc = sql.NullString{ + Valid: true, + String: headerValue[0], + } + } + } + } + + err = tx.QueryRowContext(ctx, ` + INSERT INTO messages (blob_address, size, subject, message_id, in_reply_to, date, from_addr, sender, reply_to, to_addr, cc, bcc) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + ON CONFLICT(blob_address) DO UPDATE SET blob_address = blob_address + RETURNING id + `, blobHash, size, subject, messageID, inReplyTo, datestring, fromAddr, sender, replyTo, toAddr, cc, bcc).Scan(&internalMessageID) + if err != nil { + return 0, fmt.Errorf("inserting message: %w", err) + } + + // 3. Link the message into the mailbox at the allocated UID. + _, err = tx.ExecContext(ctx, ` + INSERT INTO mailbox_messages (mailbox_id, uid, message_id, modseq, internal_date) + VALUES (?, ?, ?, ?, ?) + `, mailboxID, assignedUID, internalMessageID, newModSeq, internalDate) + if err != nil { + return 0, fmt.Errorf("linking message: %w", err) + } + + // 4. Store flags for this mailbox/uid. + for _, flag := range flags { + _, err = tx.ExecContext(ctx, ` + INSERT INTO message_flags (mailbox_id, uid, flag) + VALUES (?, ?, ?) + ON CONFLICT DO NOTHING + `, mailboxID, assignedUID, flag) + if err != nil { + return 0, fmt.Errorf("inserting flag %q: %w", flag, err) + } + } + + if err := tx.Commit(); err != nil { + return 0, fmt.Errorf("committing append: %w", err) + } + return assignedUID, nil +} + +// SetMessageFlags sets flags for a message and returns those flags. +// Returning flags is for consistency with addMessageFlags and deleteMessageFlags. +func (db *DB) SetMessageFlags(ctx context.Context, mailboxID int, messageID uint32, flags []string) ([]string, error) { + tx, err := db.write.BeginTx(ctx, nil) + if err != nil { + return nil, err + } + defer txRollback(tx) + + _, err = tx.ExecContext(ctx, `DELETE FROM message_flags WHERE mailbox_id = ? and uid = ?`, mailboxID, messageID) + if err != nil { + return nil, err + } + + for _, flag := range flags { + _, err = tx.ExecContext(ctx, `INSERT INTO message_flags (mailbox_id, uid, flag) VALUES (?, ?, ?)`, mailboxID, messageID, flag) + if err != nil { + return nil, err + } + } + return flags, tx.Commit() +} + +func (db *DB) AddMessageFlags(ctx context.Context, mailboxID int, messageID uint32, flags []string) ([]string, error) { + tx, err := db.write.BeginTx(ctx, nil) + if err != nil { + return nil, err + } + defer txRollback(tx) + + for _, flag := range flags { + _, err = tx.ExecContext(ctx, `INSERT INTO message_flags (mailbox_id, uid, flag) VALUES (?, ?, ?) ON CONFLICT DO NOTHING`, mailboxID, messageID, flag) + if err != nil { + return nil, err + } + } + + flagSet, err := getMessageFlags(ctx, tx, mailboxID, messageID) + if err != nil { + return nil, err + } + + return flagSet, tx.Commit() +} + +func (db *DB) DeleteMessageFlags(ctx context.Context, mailboxID int, messageID uint32, flags []string) ([]string, error) { + tx, err := db.write.BeginTx(ctx, nil) + if err != nil { + return nil, err + } + + for _, flag := range flags { + _, err = tx.ExecContext(ctx, `DELETE FROM message_flags WHERE mailbox_id = ? AND uid = ? AND flag = ?`, mailboxID, messageID, flag) + if err != nil { + return nil, errors.Join(err, tx.Rollback()) + } + } + + flagSet, err := getMessageFlags(ctx, tx, mailboxID, messageID) + if err != nil { + return nil, errors.Join(err, tx.Rollback()) + } + + return flagSet, tx.Commit() +} + +// getMessageFlags is a helper for addMessageFlags and deleteMessageFlags +func getMessageFlags(ctx context.Context, tx *sql.Tx, mailboxID int, messageID uint32) ([]string, error) { + var flags []string + rows, err := tx.QueryContext(ctx, `SELECT flag FROM message_flags WHERE mailbox_id = ? AND uid = ?`, mailboxID, messageID) + if err != nil { + return nil, err + } + defer func(rows *sql.Rows) { + _ = rows.Close() + }(rows) + + for rows.Next() { + var flag string + if err := rows.Scan(&flag); err != nil { + return nil, err + } + flags = append(flags, flag) + } + + return flags, rows.Err() +} + +func (db *DB) CopyMessagesToMailbox(ctx context.Context, fromMailboxID, toMailboxID int, uids []uint32) ([]uint32, error) { + tx, err := db.write.BeginTx(ctx, nil) + if err != nil { + return nil, err + } + defer txRollback(tx) + + destUIDs := make([]uint32, len(uids)) + for i, uid := range uids { + nextUID, err := getAndIncreaseNextUID(ctx, tx, toMailboxID) + if err != nil { + return nil, err + } + destUIDs[i] = nextUID + + _, err = tx.ExecContext( + ctx, + ` +INSERT INTO mailbox_messages (mailbox_id, uid, message_id, modseq, internal_date, expunged_modseq) +SELECT ?, ?, message_id, modseq, internal_date, expunged_modseq +FROM mailbox_messages +WHERE mailbox_id = ? AND uid = ?`, + toMailboxID, nextUID, fromMailboxID, uid) + if err != nil { + return nil, err + } + + _, err = tx.ExecContext( + ctx, + ` +INSERT INTO message_flags (mailbox_id, uid, flag) +SELECT ?, ?, flag +FROM message_flags +WHERE mailbox_id = ? AND uid = ?`, + toMailboxID, nextUID, fromMailboxID, uid) + if err != nil { + return nil, err + } + } + + return destUIDs, tx.Commit() +} + +func (db *DB) MoveMessagesToMailbox(ctx context.Context, fromMailboxID, toMailboxID int, uids []uint32) ([]uint32, error) { + tx, err := db.write.BeginTx(ctx, nil) + if err != nil { + return nil, err + } + defer txRollback(tx) + + destUIDs := make([]uint32, len(uids)) + for i, uid := range uids { + nextUID, err := getAndIncreaseNextUID(ctx, tx, toMailboxID) + if err != nil { + return nil, err + } + destUIDs[i] = nextUID + + _, err = tx.ExecContext( + ctx, + ` +UPDATE mailbox_messages +SET mailbox_id = ?, uid = ? +WHERE mailbox_id = ? AND uid = ?`, + toMailboxID, nextUID, fromMailboxID, uid) + if err != nil { + return nil, err + } + } + + return destUIDs, tx.Commit() +} + +func (db *DB) SetGatekeeperDecision(ctx context.Context, userID int, fromAddress, destination string) error { + _, err := db.write.ExecContext(ctx, + ` +INSERT INTO gatekeepers (user_id, from_address, destination, created_at) +VALUES (?, ?, ?, ?) ON CONFLICT DO UPDATE SET destination = ?`, + userID, fromAddress, destination, time.Now().Unix(), destination) + if err != nil { + return err + } + + slog.Info("set gatekeeper decision", "user_id", userID, "destination", destination, "from_address", fromAddress) + return nil +} + +// getAndIncreaseNextUID returns the next UID to be used. This function increases the mailbox' UID on each call. +// It should only be used as part of another transaction. +// There are certainly more efficient ways, but this is by far the most readable. +func getAndIncreaseNextUID(ctx context.Context, db *sql.Tx, mailboxID int) (uint32, error) { + res := db.QueryRowContext( + ctx, + `UPDATE mailboxes SET uidnext = uidnext + 1 WHERE id = ? RETURNING uidnext`, mailboxID) + if res.Err() != nil { + return 0, res.Err() + } + + var uidnext uint32 + err := res.Scan(&uidnext) + if err != nil { + return 0, err + } + return uidnext - 1, nil // The query returns the upcoming UID. We need to return the to-be-used UID +} + +func (db *DB) GetAllFlaggedDeletedMessages(ctx context.Context, mailboxID int) ([]uint32, error) { + rows, err := db.read.QueryContext(ctx, `SELECT uid FROM message_flags WHERE flag = '\Deleted' AND mailbox_id = ?`, mailboxID) + if err != nil { + return nil, err + } + + defer func(rows *sql.Rows) { + _ = rows.Close() + }(rows) + + uids := make([]uint32, 0) + for rows.Next() { + var uid uint32 + err := rows.Scan(&uid) + if err != nil { + return nil, err + } + uids = append(uids, uid) + } + return uids, rows.Err() +} + +func (db *DB) DeleteMessageFromMailbox(ctx context.Context, mailboxID int, uid uint32) error { + // message_flags are automatically deleted via CASCADE + _, err := db.write.ExecContext(ctx, `DELETE FROM mailbox_messages WHERE mailbox_id = ? AND uid = ?`, mailboxID, uid) + return err +} + +// UIDExists reports whether a user has a message with the given uid. +// RFC 9051: A non-existent unique identifier is ignored without any error message generated. Thus, it is possible for a +// UID FETCH command to return an OK without any data or a UID COPY, UID MOVE, or UID STORE to return an OK without +// performing any operations. +func (db *DB) UIDExists(ctx context.Context, userID int, uid uint32) (bool, error) { + query := ` + SELECT EXISTS ( + SELECT 1 FROM mailbox_messages mm + JOIN mailboxes m ON mm.mailbox_id = m.id + WHERE m.user_id = ? AND mm.uid = ? + )` + + var exists bool + err := db.read.QueryRowContext(ctx, query, userID, uid).Scan(&exists) + if err != nil { + return false, fmt.Errorf("error checking uid existence: %w", err) + } + return exists, nil +}
A
internal/db/meta.go
@@ -0,0 +1,209 @@
+package db + +import ( + "context" + "database/sql" + "errors" + "fmt" + "log/slog" + + "github.com/emersion/go-imap/v2" + _ "modernc.org/sqlite" +) + +const pragmas = ` +PRAGMA journal_mode = WAL; +PRAGMA synchronous = NORMAL; +PRAGMA foreign_keys = ON; +PRAGMA busy_timeout = 5000; +PRAGMA temp_store = MEMORY; +` + +const schema = ` + +CREATE TABLE IF NOT EXISTS users ( + id INTEGER PRIMARY KEY, + name TEXT NOT NULL UNIQUE, + password_hash BLOB NOT NULL, + created_at INTEGER NOT NULL +) STRICT; + +CREATE INDEX IF NOT EXISTS idx_users_name ON users(name); + +CREATE TABLE IF NOT EXISTS addresses ( + id INTEGER PRIMARY KEY, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + name TEXT NOT NULL UNIQUE, + created_at INTEGER NOT NULL +) STRICT; + +CREATE INDEX IF NOT EXISTS idx_addresses_user_id ON addresses(user_id); +CREATE INDEX IF NOT EXISTS idx_addresses_name ON addresses(name); + +CREATE TABLE IF NOT EXISTS gatekeepers ( + id INTEGER PRIMARY KEY, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + from_address TEXT NOT NULL, + destination TEXT NOT NULL, + created_at INTEGER NOT NULL, + UNIQUE (user_id, from_address) +) STRICT; + +CREATE TABLE IF NOT EXISTS mailboxes ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + name TEXT NOT NULL, + special_use TEXT, + uidvalidity INTEGER NOT NULL, + uidnext INTEGER NOT NULL DEFAULT 1, + highest_modseq INTEGER NOT NULL DEFAULT 0, + UNIQUE (user_id, name) +) STRICT; + +CREATE TABLE IF NOT EXISTS subscriptions ( + id INTEGER PRIMARY KEY, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + mailbox_id INTEGER REFERENCES mailboxes(id) ON DELETE SET NULL, + mailbox_name TEXT, -- mailboxes can be subscribed even if mailbox does not exist, hence duplicate name + UNIQUE (user_id, mailbox_id) +) STRICT; + +CREATE TABLE IF NOT EXISTS messages ( + id INTEGER PRIMARY KEY, + blob_address TEXT NOT NULL UNIQUE, + size INTEGER NOT NULL, + subject TEXT, + message_id TEXT, + in_reply_to TEXT, + + -- Envelope fields + date TEXT, + from_addr TEXT, -- JSON array of addresses + sender TEXT, -- JSON array of addresses + reply_to TEXT, -- JSON array of addresses + to_addr TEXT, -- JSON array of addresses + cc TEXT, -- JSON array of addresses + bcc TEXT -- JSON array of addresses +) STRICT; + +CREATE TABLE IF NOT EXISTS mailbox_messages ( + mailbox_id INTEGER NOT NULL REFERENCES mailboxes(id) ON DELETE CASCADE, + uid INTEGER NOT NULL, + message_id INTEGER NOT NULL REFERENCES messages(id), + modseq INTEGER NOT NULL, + internal_date TEXT NOT NULL, + expunged_modseq INTEGER, + PRIMARY KEY (mailbox_id, uid) +) STRICT; + +CREATE TABLE IF NOT EXISTS message_flags ( + mailbox_id INTEGER NOT NULL, + uid INTEGER NOT NULL, + flag TEXT NOT NULL, + PRIMARY KEY (mailbox_id, uid, flag), + FOREIGN KEY (mailbox_id) REFERENCES mailboxes(id) ON DELETE CASCADE, + FOREIGN KEY (mailbox_id, uid) REFERENCES mailbox_messages(mailbox_id, uid) ON DELETE CASCADE ON UPDATE CASCADE +) STRICT; + +CREATE TABLE IF NOT EXISTS dkim ( + id INTEGER PRIMARY KEY, + created_at INTEGER NOT NULL, + domain TEXT NOT NULL, + key_type TEXT NOT NULL, + blob_address TEXT NOT NULL, + selector TEXT NOT NULL, + enabled INTEGER NOT NULL, + UNIQUE (domain, selector) +) STRICT; + +CREATE TABLE IF NOT EXISTS smtp_auth ( + id INTEGER PRIMARY KEY, + name TEXT NOT NULL UNIQUE, + password_hash BLOB NOT NULL, + created_at INTEGER NOT NULL +) STRICT; + +CREATE INDEX IF NOT EXISTS idx_mailbox_messages_modseq + ON mailbox_messages(mailbox_id, modseq); + +CREATE INDEX IF NOT EXISTS idx_mailbox_messages_message + ON mailbox_messages(message_id); + +CREATE TABLE IF NOT EXISTS keywords ( + message_id INTEGER NOT NULL REFERENCES messages(id), + keyword TEXT NOT NULL, + PRIMARY KEY (message_id, keyword) +) STRICT; + +CREATE INDEX IF NOT EXISTS idx_keywords_keyword + ON keywords(keyword); + +CREATE TABLE IF NOT EXISTS uidvalidity_counter ( + id INTEGER PRIMARY KEY CHECK (id = 1), + counter INTEGER NOT NULL DEFAULT 0 +) STRICT; + +INSERT INTO uidvalidity_counter (id, counter) +VALUES (1, 0) +ON CONFLICT(id) DO NOTHING; +` + +func (db *DB) applyPragmas(ctx context.Context) error { + for _, conn := range []*sql.DB{db.write, db.read} { + if _, err := conn.ExecContext(ctx, pragmas); err != nil { + return fmt.Errorf("applying pragmas: %w", err) + } + } + return nil +} + +func (db *DB) applySchema(ctx context.Context) error { + if _, err := db.write.ExecContext(ctx, schema); err != nil { + return fmt.Errorf("applying schema: %w", err) + } + return nil +} + +func (db *DB) Close() error { + werr := db.write.Close() + rerr := db.read.Close() + if werr != nil { + return werr + } + return rerr +} + +func (db *DB) GetLastUID(ctx context.Context, userID, mailboxID int) (imap.UID, error) { + var uid imap.UID + err := db.read.QueryRowContext(ctx, ` + SELECT COALESCE(MAX(mm.uid), 0) + FROM mailbox_messages mm + JOIN mailboxes mb ON mb.id = mm.mailbox_id + WHERE mm.mailbox_id = ? + AND mb.user_id = ? + AND mm.expunged_modseq IS NULL + `, mailboxID, userID).Scan(&uid) + return uid, err +} + +// nextUIDValidity is db roll-back save. Even with a played-back backup, the counter will be increasing. +// It will also be increasing if it is called more than once per second. +func nextUIDValidity(tx *sql.Tx) (uint32, error) { + var v int64 + err := tx.QueryRow(` + UPDATE uidvalidity_counter + SET counter = MAX(counter + 1, CAST(strftime('%s','now') AS INTEGER)) + RETURNING counter + `).Scan(&v) + if err != nil { + return 0, err + } + return uint32(v), nil +} + +func txRollback(tx *sql.Tx) { + err := tx.Rollback() + if !errors.Is(err, sql.ErrTxDone) { + slog.Error("failed to roll back transaction", "error", err) + } +}
A
internal/db/prune.go
@@ -0,0 +1,45 @@
+package db + +import ( + "context" + "database/sql" +) + +// PrunableTarget holds the data needed to delete the file and clean the database. +type PrunableTarget struct { + MessageID int64 + BlobHash string +} + +// GetPrunableTargets reports whether it is safe to remove a file from the db perspective +func (db *DB) GetPrunableTargets(ctx context.Context) ([]PrunableTarget, error) { + rows, err := db.read.QueryContext( + ctx, `SELECT id, blob_address +FROM messages +WHERE id NOT IN (SELECT message_id FROM mailbox_messages)`) + + if err != nil { + return nil, err + } + + defer func(rows *sql.Rows) { + _ = rows.Close() + }(rows) + + out := make([]PrunableTarget, 0) + for rows.Next() { + var pt PrunableTarget + if err := rows.Scan(&pt.MessageID, &pt.BlobHash); err != nil { + return nil, err + } + out = append(out, pt) + } + + return out, nil +} + +// DeleteMessage deletes the message which is usually part of the pruning process +func (db *DB) DeleteMessage(ctx context.Context, id int64) error { + _, err := db.read.ExecContext(ctx, `DELETE FROM messages WHERE id = ?`, id) + return err +}
A
internal/db/smtpauth.go
@@ -0,0 +1,90 @@
+package db + +import ( + "context" + "database/sql" + "errors" + "fmt" + "time" + + "postern/internal/model" + + "modernc.org/sqlite" + sqlite3 "modernc.org/sqlite/lib" +) + +func (db *DB) SMTPAuthNameExists(ctx context.Context, name string) (bool, error) { + var count int + err := db.read.QueryRowContext(ctx, + `SELECT COUNT(*) FROM smtp_auth WHERE name = ?`, + name, + ).Scan(&count) + if err != nil { + return false, fmt.Errorf("checking name existence: %w", err) + } + return count > 0, nil +} + +func (db *DB) InsertSMTPAuthUser(ctx context.Context, name string, hash []byte) error { + _, err := db.write.ExecContext(ctx, + `INSERT INTO smtp_auth (name, password_hash, created_at) VALUES (?, ?, ?)`, + name, hash, time.Now().Unix(), + ) + if err != nil { + if insertErr, ok := errors.AsType[*sqlite.Error](err); ok { + if insertErr.Code() == sqlite3.SQLITE_CONSTRAINT_UNIQUE { + return fmt.Errorf("name %s already exists", name) + } + return fmt.Errorf("inserting smtp auth: %w", err) + } + } + + return err +} + +func (db *DB) GetSMTPAuthUser(ctx context.Context, name string) (model.SMTPAuth, error) { + res := db.read.QueryRowContext( + ctx, `SELECT id, name, password_hash FROM smtp_auth WHERE name = ?`, name) + + var u model.SMTPAuth + var pw []byte + err := res.Scan(&u.ID, &u.Name, &pw) + if err != nil { + return model.SMTPAuth{}, err + } + u.SetPassword(pw) + + return u, nil +} + +func (db *DB) RemoveSMTPAuthUser(ctx context.Context, name string) error { + _, err := db.write.ExecContext(ctx, `DELETE FROM smtp_auth WHERE name = ?`, name) + return err +} + +func (db *DB) ListSMTPAuthUsers(ctx context.Context) ([]model.SMTPAuth, error) { + rows, err := db.read.QueryContext(ctx, `SELECT id, name, password_hash FROM smtp_auth ORDER BY name`) + if err != nil { + return nil, err + } + defer func(rows *sql.Rows) { + _ = rows.Close() + }(rows) + + var auths []model.SMTPAuth + for rows.Next() { + var u model.SMTPAuth + var pw []byte + if err := rows.Scan(&u.ID, &u.Name, &pw); err != nil { + return nil, err + } + u.SetPassword(pw) + auths = append(auths, u) + } + + if err := rows.Err(); err != nil { + return nil, err + } + + return auths, nil +}
A
internal/db/subscription.go
@@ -0,0 +1,17 @@
+package db + +import ( + "context" +) + +func (db *DB) Subscribe(ctx context.Context, userID, mailboxID int, mailboxName string) error { + _, err := db.write.ExecContext(ctx, `INSERT INTO subscriptions (user_id, mailbox_id, mailbox_name) VALUES (?, ?, ?) ON CONFLICT DO NOTHING`, userID, mailboxID, mailboxName) + return err +} + +// Unsubscribe removes the given mailbox name from subscriptions. +// IMAP allows to unsubscribe from non-existing mailboxes. That's why we work with mailbox names instead of IDs. +func (db *DB) Unsubscribe(ctx context.Context, userID int, mailbox string) error { + _, err := db.write.ExecContext(ctx, `DELETE FROM subscriptions WHERE user_id = ? AND mailbox_name = ?`, userID, mailbox) + return err +}
A
internal/db/user.go
@@ -0,0 +1,157 @@
+package db + +import ( + "context" + "database/sql" + "errors" + "fmt" + "time" + + "postern/internal/model" + + "modernc.org/sqlite" + sqlite3 "modernc.org/sqlite/lib" +) + +func InsertUser(ctx context.Context, tx *sql.Tx, username string, hash []byte) (int, error) { + res, err := tx.ExecContext(ctx, + `INSERT INTO users (name, password_hash, created_at) VALUES (?, ?, ?)`, + username, hash, time.Now().Unix(), + ) + if err != nil { + if insertErr, ok := errors.AsType[*sqlite.Error](err); ok { + if insertErr.Code() == sqlite3.SQLITE_CONSTRAINT_UNIQUE { + return 0, fmt.Errorf("user %s already exists", username) + } + return 0, fmt.Errorf("inserting user: %w", err) + } + } + + userID, err := res.LastInsertId() + return int(userID), err +} + +func (db *DB) GetUser(ctx context.Context, username string) (model.User, error) { + rows, err := db.read.QueryContext( + ctx, `SELECT users.id, users.name, users.password_hash, addresses.name FROM users LEFT JOIN addresses ON users.id = addresses.user_id WHERE users.name = ?`, username) + if err != nil { + return model.User{}, err + } + defer func(rows *sql.Rows) { + _ = rows.Close() + }(rows) + + var u model.User + var pw []byte + var address sql.NullString + found := false + for rows.Next() { + err := rows.Scan(&u.ID, &u.Name, &pw, &address) + if err != nil { + return model.User{}, err + } + found = true + if address.Valid { + u.Addresses = append(u.Addresses, address.String) + } + } + if rows.Err() != nil { + return model.User{}, rows.Err() + } + + if !found { + return model.User{}, errors.New("user not found") + } + + u.SetPassword(pw) + return u, nil +} + +func (db *DB) UserExists(ctx context.Context, username string) (bool, error) { + var count int + err := db.read.QueryRowContext(ctx, + `SELECT COUNT(*) FROM users WHERE name = ?`, + username, + ).Scan(&count) + if err != nil { + return false, fmt.Errorf("checking user existence: %w", err) + } + return count > 0, nil +} + +func InsertDefaultMailboxes(ctx context.Context, tx *sql.Tx, userID int) error { + uidvalidity := uint32(time.Now().UnixMilli()) + + for _, mb := range model.DefaultMailboxes() { + _, err := tx.ExecContext(ctx, ` + INSERT INTO mailboxes + (user_id, name, special_use, uidvalidity, uidnext, highest_modseq) + VALUES + (?, ?, ?, ?, 1, 0)`, + userID, mb.Name, mb.SpecialUse, uidvalidity, + ) + if err != nil { + return fmt.Errorf("creating mailbox %q: %w", mb.Name, err) + } + } + + return nil +} + +func (db *DB) ListUsers(ctx context.Context) ([]model.User, error) { + query := ` + SELECT users.id, users.name, users.password_hash, addresses.name + FROM users + LEFT JOIN addresses ON users.id = addresses.user_id + ORDER BY users.name + ` + + rows, err := db.read.QueryContext(ctx, query) + if err != nil { + return nil, err + } + defer func(rows *sql.Rows) { + _ = rows.Close() + }(rows) + + var users []model.User + + for rows.Next() { + var id int + var name string + var passwordHash []byte + var addressName sql.NullString + + if err := rows.Scan(&id, &name, &passwordHash, &addressName); err != nil { + return nil, err + } + + // If the slice is empty, OR the last appended user has a different ID, + // we have encountered a new user. + if len(users) == 0 || users[len(users)-1].ID != id { + nextUser := model.User{ + ID: id, + Name: name, + Addresses: []string{}, + } + nextUser.SetPassword(passwordHash) + users = append(users, nextUser) + } + + if addressName.Valid { + lastIdx := len(users) - 1 + users[lastIdx].Addresses = append(users[lastIdx].Addresses, addressName.String) + } + } + + if err = rows.Err(); err != nil { + return nil, err + } + + return users, nil +} + +func (db *DB) RemoveUser(ctx context.Context, username string) error { + _, err := db.write.ExecContext(ctx, "DELETE FROM users WHERE name = ?", username) + return err +}
A
internal/db/user_test.go
@@ -0,0 +1,114 @@
+package db + +import ( + "log" + "path" + "slices" + "testing" +) + +// TestUsers tests db operations around user management +func TestUsers(t *testing.T) { + posternDB, err := OpenDB(&Config{ + DBPath: path.Join(t.TempDir(), "meta.db"), + }) + if err != nil { + log.Fatal(err) + } + + defer posternDB.Close() + + tx, err := posternDB.GetWriteTx(t.Context()) + if err != nil { + t.Fatal(err) + } + + _, err = InsertUser(t.Context(), tx, "testuser1", []byte{}) + if err != nil { + t.Fatal(err) + } + _, err = InsertUser(t.Context(), tx, "testuser2", []byte{}) + if err != nil { + t.Fatal(err) + } + err = tx.Commit() + if err != nil { + t.Fatal(err) + } + + err = posternDB.InsertUserAddress(t.Context(), "testuser1", "test1@example.com") + if err != nil { + t.Fatal(err) + } + + err = posternDB.InsertUserAddress(t.Context(), "testuser1", "test2@example.com") + if err != nil { + t.Fatal(err) + } + + // Test GetUser + u, err := posternDB.GetUser(t.Context(), "testuser1") + if err != nil { + t.Fatal(err) + } + if len(u.Addresses) != 2 { + t.Errorf("got %d addresses, want 2", len(u.Addresses)) + } + if !slices.Contains(u.Addresses, "test2@example.com") { + t.Errorf("did not find user address in addresses") + } + + // Test non-existing user + u, err = posternDB.GetUser(t.Context(), "non-existent") + if err == nil { + t.Errorf("got nil, want error for non-existent user") + } + + // Test for exists + exists, err := posternDB.UserExists(t.Context(), "testuser1") + if err != nil { + t.Fatal(err) + } + if !exists { + t.Errorf("expected user to exist") + } + exists, err = posternDB.UserExists(t.Context(), "non-existent") + if err != nil { + t.Fatal(err) + } + if exists { + t.Errorf("expected user to not exist") + } + + users, err := posternDB.ListUsers(t.Context()) + if err != nil { + t.Fatal(err) + } + if len(users) != 2 { + t.Errorf("got %d users, want 2", len(users)) + } + for _, u = range users { + if u.Name == "testuser1" { + if len(u.Addresses) != 2 { + t.Errorf("got %d addresses, want 2", len(u.Addresses)) + } + } + if u.Name == "testuser2" { + if len(u.Addresses) != 0 { + t.Errorf("got %d addresses, want 0", len(u.Addresses)) + } + } + } + + err = posternDB.RemoveUser(t.Context(), "testuser1") + if err != nil { + t.Fatal(err) + } + exists, err = posternDB.UserExists(t.Context(), "testuser1") + if err != nil { + t.Fatal(err) + } + if exists { + t.Errorf("expected user to not exist") + } +}
A
internal/dkim/dkim.go
@@ -0,0 +1,121 @@
+package dkim + +import ( + "bytes" + "context" + "database/sql" + "errors" + "fmt" + "strings" + + "postern/internal/db" + "postern/internal/model" + "postern/internal/persistence" +) + +type Config struct { + DB *db.DB + Persistence *persistence.Persistence +} + +type DKIM struct { + db *db.DB + persistence *persistence.Persistence +} + +func New(config *Config) *DKIM { + return &DKIM{ + db: config.DB, + persistence: config.Persistence, + } +} + +// Add generates a new RSA-2048 private key for the given selector/domain, stores it +// as a blob, inserts the corresponding row in the database and returns the expected +// DNS TXT record that should be published before enabling the selector. +func (d *DKIM) Add(ctx context.Context, selector, domain string) (string, error) { + if selector == "" { + return "", fmt.Errorf("invalid selector") + } + if strings.Contains(selector, ".") { + return "", fmt.Errorf("selector must not contain '.'") + } + if domain == "" { + return "", fmt.Errorf("invalid domain") + } + + privKey, keyType := genRSAPrivateKey() + encodedPrivKey := encodePrivateKey(privKey) + + blobName, _, err := d.persistence.WriteBlob(bytes.NewReader(encodedPrivKey)) + if err != nil { + return "", err + } + + err = d.db.InsertDKIMForDomain(ctx, model.DKIM{ + Domain: domain, + Selector: selector, + BlobAddress: blobName, + KeyType: keyType, + Enabled: false, + }) + if err != nil { + return "", err + } + + return pubKeyRecord(privKey.Public()), nil +} + +// List returns all configured DKIM selectors. +func (d *DKIM) List(ctx context.Context) ([]model.DKIM, error) { + return d.db.ListDKIM(ctx) +} + +// PublicDNSRecord returns the expected DNS TXT record for the given selector/domain +// as it should be published under selector._domainkey.domain. +func (d *DKIM) PublicDNSRecord(ctx context.Context, selector, domain string) (string, error) { + dbRecord, err := d.db.GetDKIMForSelectorDomain(ctx, selector, domain) + if err != nil { + return "", err + } + privKey, err := d.readPrivateKey(dbRecord.BlobAddress) + if err != nil { + return "", err + } + return pubKeyRecord(privKey.Public()), nil +} + +// SetEnabled enables or disables the given selector/domain selector. +func (d *DKIM) SetEnabled(ctx context.Context, selector, domain string, enabled bool) error { + return d.db.SetDKIMEnabled(ctx, selector, domain, enabled) +} + +// Remove deletes the private key and the database entry. +func (d *DKIM) Remove(ctx context.Context, selector, domain string) error { + dkimRow, err := d.db.GetDKIMForSelectorDomain(ctx, selector, domain) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + return errors.New("DKIM entry not found") + } + return err + } + + tx, err := d.db.GetWriteTx(ctx) + if err != nil { + return err + } + + err = db.RemoveDKIM(ctx, tx, selector, domain) + if err != nil { + _ = tx.Rollback() + return err + } + + err = d.persistence.RemoveBlob(dkimRow.BlobAddress) + if err != nil { + _ = tx.Rollback() + return err + } + + return tx.Commit() +}
A
internal/dkim/keys.go
@@ -0,0 +1,117 @@
+package dkim + +import ( + "crypto" + "crypto/ed25519" + "crypto/rand" + "crypto/rsa" + "crypto/x509" + "encoding/base64" + "encoding/pem" + "fmt" + "log" + "strings" + + "postern/internal/model" +) + +type privateKey interface { + Public() crypto.PublicKey +} + +func genRSAPrivateKey() (privateKey, model.DKIMAlgorithmType) { + nBits := 2048 + log.Printf("Generating a %v-bit RSA key", nBits) + privKey, err := rsa.GenerateKey(rand.Reader, nBits) + + if err != nil { + log.Fatalf("Failed to generate key: %v", err) + } + return privKey, model.DKIMAlgorithmRSA2048 +} + +func genEd25519PrivateKey() (privateKey, model.DKIMAlgorithmType) { + _, privKey, err := ed25519.GenerateKey(rand.Reader) + if err != nil { + log.Fatalf("Failed to generate key: %v", err) + } + return privKey, model.DKIMAlgorithmEd25519 +} + +func encodePrivateKey(privateKey privateKey) []byte { + privBytes, err := x509.MarshalPKCS8PrivateKey(privateKey) + if err != nil { + log.Fatalf("Failed to marshal private key: %v", err) + } + + privBlock := pem.Block{ + Type: "PRIVATE KEY", + Bytes: privBytes, + } + + encoded := pem.EncodeToMemory(&privBlock) + if encoded == nil { + log.Fatalf("Failed to write key PEM block: %v", err) + } + return encoded +} + +func decodePrivateKey(encoded string) (privateKey, error) { + // Decode PEM block from string + block, _ := pem.Decode([]byte(encoded)) + if block == nil { + return nil, fmt.Errorf("failed to decode PEM block: no valid PEM data found") + } + + // Validate PEM type + if block.Type != "PRIVATE KEY" { + return nil, fmt.Errorf("invalid PEM type: expected 'PRIVATE KEY', got '%s'", block.Type) + } + + // Parse PKCS#8 private key + key, err := x509.ParsePKCS8PrivateKey(block.Bytes) + if err != nil { + return nil, fmt.Errorf("failed to parse private key: %w", err) + } + + switch k := key.(type) { + case *rsa.PrivateKey: + return k, nil + case ed25519.PrivateKey: + return k, nil + default: + return nil, fmt.Errorf("unsupported private key type: %T", k) + } +} + +func pubKeyRecord(pubKey crypto.PublicKey) string { + var pubBytes []byte + var keyType string + switch p := pubKey.(type) { + case *rsa.PublicKey: + keyType = "rsa" + // RFC 6376 is inconsistent about whether RSA public keys should + // be formatted as RSAPublicKey or SubjectPublicKeyInfo. + // Erratum 3017 (https://www.rfc-editor.org/errata/eid3017) + // proposes allowing both. We use SubjectPublicKeyInfo for + // consistency with other implementations including opendkim, + // Gmail, and Fastmail. + var err error + pubBytes, err = x509.MarshalPKIXPublicKey(p) + if err != nil { + log.Fatalf("Failed to marshal public key: %v", err) + } + case ed25519.PublicKey: + keyType = "ed25519" + pubBytes = p + default: + panic("unreachable") + } + + params := []string{ + "v=DKIM1", + "k=" + keyType, + "p=" + base64.StdEncoding.EncodeToString(pubBytes), + } + return strings.Join(params, "; ") +}
A
internal/dkim/sign.go
@@ -0,0 +1,117 @@
+package dkim + +import ( + "bytes" + "context" + "crypto" + "fmt" + "io" + "log/slog" + "net/mail" + "strings" + "time" + + "github.com/emersion/go-msgauth/dkim" +) + +func rfc6376Section541() []string { + return []string{ + "From", + "Reply-To", + "Subject", + "Date", + "To", + "Cc", + "Resent-Date", + "Resent-From", + "Resent-To", + "Resent-Cc", + "In-Reply-To", + "References", + "List-Id", + "List-Help", + "List-Unsubscribe", + "List-Subscribe", + "List-Post", + "List-Owner", + "List-Archive", + } +} + +// Sign signs the message with the first enabled selector for the From: domain and returns the signed message. +func (d *DKIM) Sign(ctx context.Context, message []byte) ([]byte, error) { + // TODO check if we can make this a zero-copy operation with io.Reader instead of message []byte + // for DMARC alignment, we first need to find the From: domain + msg, err := mail.ReadMessage(bytes.NewReader(message)) + if err != nil { + return nil, fmt.Errorf("parsing message: %w", err) + } + + fromHeader := msg.Header.Get("From") + if fromHeader == "" { + return nil, fmt.Errorf("from header is empty") + } + parsedFromHeader, err := mail.ParseAddress(fromHeader) + if err != nil { + return nil, fmt.Errorf("parsing from header: %w", err) + } + i := strings.LastIndex(parsedFromHeader.Address, "@") + if i == -1 { + return nil, fmt.Errorf("from header does not contain @") + } + domain := parsedFromHeader.Address[i+1:] + + slog.Info("[DKIM] signing based on From: header", "domain", domain, "from", fromHeader) + + dRec, err := d.db.GetDKIMForDomain(ctx, domain) + if err != nil { + return nil, fmt.Errorf("get DKIM for domain: %w", err) + } + + signer, err := d.readPrivateKey(dRec.BlobAddress) + if err != nil { + return nil, fmt.Errorf("reading signing key: %w", err) + } + + var buffer bytes.Buffer + _, err = io.Copy(&buffer, bytes.NewReader(message)) + if err != nil { + return nil, fmt.Errorf("copy buffer: %w", err) + } + + err = dkim.Sign(&buffer, &buffer, &dkim.SignOptions{ + Domain: domain, + Selector: dRec.Selector, + Identifier: "", + Signer: signer.(crypto.Signer), + Hash: crypto.SHA256, + HeaderCanonicalization: dkim.CanonicalizationSimple, + BodyCanonicalization: dkim.CanonicalizationSimple, + HeaderKeys: rfc6376Section541(), + Expiration: time.Time{}, + QueryMethods: nil, + }) + if err != nil { + return nil, fmt.Errorf("creating signer: %w", err) + } + + return buffer.Bytes(), nil +} + +// readPrivateKey loads and decodes the private key blob referenced by nonceIdentifier. +func (d *DKIM) readPrivateKey(nonceIdentifier string) (privateKey, error) { + pk, err := d.persistence.BlobReader(nonceIdentifier) + if err != nil { + return nil, err + } + defer func(privKey io.ReadCloser) { + _ = privKey.Close() + }(pk) + + privKeyBytes, err := io.ReadAll(pk) + if err != nil { + return nil, err + } + + return decodePrivateKey(string(privKeyBytes)) +}
A
internal/imap/address.go
@@ -0,0 +1,37 @@
+package imap + +import ( + "fmt" + "net/mail" + "strings" + + "github.com/emersion/go-imap/v2" +) + +func parseAddressList(s string) ([]imap.Address, error) { + if s == "" { + return []imap.Address{}, nil + } + + addresses, err := mail.ParseAddressList(s) + if err != nil { + return nil, err + } + + result := make([]imap.Address, 0, len(addresses)) + for _, a := range addresses { + atSignPosition := strings.LastIndex(a.Address, "@") + if atSignPosition <= 0 { // need at least one character before the '@' sign + return nil, fmt.Errorf("invalid address: %s", a.Address) + } + + local, host := a.Address[:atSignPosition], a.Address[atSignPosition+1:] + + result = append(result, imap.Address{ + Name: a.Name, + Mailbox: local, + Host: host, + }) + } + return result, nil +}
A
internal/imap/errors.go
@@ -0,0 +1,47 @@
+package imap + +import ( + "errors" + + "github.com/emersion/go-imap/v2" +) + +var ( + errDBSequenceOutOfRange = errors.New("sequence out of range") + + errSequenceOutOfRange = &imap.Error{ + Type: imap.StatusResponseTypeBad, + Code: imap.ResponseCodeClientBug, + Text: "Sequence out of range", + } + + errMailboxNotExist = &imap.Error{ + Type: imap.StatusResponseTypeNo, + Code: imap.ResponseCodeNonExistent, + Text: "Mailbox does not exist", + } + + errInternalServerError = &imap.Error{ + Type: imap.StatusResponseTypeBad, + Code: imap.ResponseCodeServerBug, + Text: "internal server error", + } + + errBadFlag = &imap.Error{ + Type: imap.StatusResponseTypeBad, + Code: imap.ResponseCodeClientBug, + Text: "Unsupported flag", + } + + errNoMailboxSelected = &imap.Error{ + Type: imap.StatusResponseTypeNo, + Code: imap.ResponseCodeCannot, + Text: "No mailbox selected", + } + + errMailboxReadOnly = &imap.Error{ + Type: imap.StatusResponseTypeNo, + Code: imap.ResponseCodeCannot, + Text: "Mailbox was selected read-only (EXAMINE)", + } +)
A
internal/imap/flag.go
@@ -0,0 +1,51 @@
+package imap + +import ( + "strings" + + "github.com/emersion/go-imap/v2" +) + +func isAllowedFlag(flag imap.Flag) bool { + if len(flag) == 0 { // RFC 9051 requires one or more characters + return false + } + // system flags start with "\" and are defined per RFC 9051 + if strings.HasPrefix(string(flag), "\\") { + switch flag { + // \Recent was deprecated in IMAP 4 rev.2 + case imap.FlagSeen, imap.FlagAnswered, imap.FlagFlagged, imap.FlagDeleted, imap.FlagDraft: + return true + default: + return false + } + } + + // Clients are generally free in adding keywords + return true +} + +func toIMAPFlags(flags []string) []imap.Flag { + out := make([]imap.Flag, 0, len(flags)) + for _, flag := range flags { + if flag == "" { + continue + } + out = append(out, imap.Flag(flag)) + } + if len(out) == 0 { + return nil + } + return out +} + +func fromIMAPFlags(flags []imap.Flag) []string { + out := make([]string, 0, len(flags)) + for _, f := range flags { + if string(f) == "" { + continue + } + out = append(out, string(f)) + } + return out +}
A
internal/imap/handler.go
@@ -0,0 +1,1505 @@
+package imap + +import ( + "context" + "database/sql" + "errors" + "fmt" + "log/slog" + "math" + "slices" + "strings" + "time" + + "postern/internal/db" + "postern/internal/model" + "postern/internal/persistence" + "postern/internal/policy" + + "github.com/emersion/go-imap/v2" + "github.com/emersion/go-imap/v2/imapserver" + "github.com/google/uuid" +) + +type imapSession struct { + db *db.DB + conn *imapserver.Conn + mTracker *mailboxTrackerRegistry + persistence *persistence.Persistence + searchRes imap.NumSet // RFC 9051 6.4.4.1 + sessionTracker *imapserver.SessionTracker + selectedMailbox model.Mailbox + selectedMailboxReadOnly bool + user model.User + sessionID string + policyEngine *policy.Engine +} + +func (i *Server) handleIMAPConn(conn *imapserver.Conn) (imapserver.Session, *imapserver.GreetingData, error) { + sessionID := uuid.NewString() + slog.Info("handle new IMAP connection", "remote_ip", conn.NetConn().RemoteAddr().String(), "session_id", sessionID) + return &imapSession{ + db: i.db, + conn: conn, + selectedMailbox: model.Mailbox{}, + mTracker: i.mTracker, + persistence: i.persistence, + sessionTracker: nil, + sessionID: sessionID, + policyEngine: i.policyEngine, + }, &imapserver.GreetingData{}, nil +} + +func (m *imapSession) Close() error { + slog.Info("close IMAP session", "session_id", m.sessionID) + // Package calls EXPUNGE internally, which leads to a bug in which + // CLOSE emits an error instead of silently closing for read-only SELECTs + if m.sessionTracker != nil { + m.sessionTracker.Close() + } + m.selectedMailbox = model.Mailbox{} + return nil +} + +func (m *imapSession) Login(username, password string) error { + logger := slog.With("session_id", m.sessionID, "cmd", "LOGIN", "username", username) + u, err := m.db.GetUser(context.Background(), username) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + logger.Info("user not found") + } else { + logger.Error("getting user", "error", err) + } + return imapserver.ErrAuthFailed + } + + err = u.VerifyPassword([]byte(password)) + if err != nil { + slog.Info("invalid password", "username", username) + return imapserver.ErrAuthFailed + } + + logger.Info("logged in", "user_id", u.ID) + m.user = u + return nil + +} + +// Select and Examine are identical besides read-only (https://www.ietf.org/rfc/rfc9051.html#name-examine-command) +func (m *imapSession) Select(mailboxName string, selectOpt *imap.SelectOptions) (*imap.SelectData, error) { + logger := slog.With("session_id", m.sessionID, "cmd", "SELECT", "user_id", m.user.ID) + // INBOX is case-insensitive + if strings.EqualFold(mailboxName, string(model.MailboxINBOX)) { + mailboxName = string(model.MailboxINBOX) + } + + logger.Info("start SELECT", "mailbox", mailboxName, "read_only", selectOpt.ReadOnly) + + userMailbox, err := m.db.GetUserMailbox(context.Background(), m.user.ID, mailboxName) + if err != nil { + if errors.Is(err, db.ErrMailboxNotFound) { + return nil, errMailboxNotExist + } + logger.Error("getting mailbox", "error", err) + return nil, err + } + + if m.selectedMailbox.ID != userMailbox.ID && m.sessionTracker != nil { + m.sessionTracker.Close() + m.sessionTracker = nil + } + + m.sessionTracker = m.mTracker.getMailboxTracker(m.user.ID, userMailbox.ID, userMailbox.NumMessages).NewSession() + m.selectedMailbox = userMailbox + m.selectedMailboxReadOnly = selectOpt.ReadOnly + + activeFlags, err := m.db.GetMailboxFlags(context.Background(), m.selectedMailbox.ID) + if err != nil { + logger.Error("getting mailbox flags", "error", err) + return nil, err + } + + // RFC 9051: Upon successful completion of a SELECT or an EXAMINE command (after the tagged OK response), + // the current search result variable is reset to the empty sequence. + m.searchRes = imap.UIDSet{} + + logger.Info("end SELECT") + return &imap.SelectData{ + NumMessages: userMailbox.NumMessages, + NumRecent: 0, // (legacy, send 0) + UIDValidity: userMailbox.UIDValidity, + UIDNext: imap.UID(userMailbox.UIDNext), + PermanentFlags: []imap.Flag{ + imap.FlagWildcard, imap.FlagFlagged, imap.FlagSeen, + imap.FlagDeleted, imap.FlagAnswered, imap.FlagDraft, + }, + Flags: toIMAPFlags(activeFlags), + List: &imap.ListData{ + Attrs: getMailboxAttr(userMailbox), + Delim: '/', + Mailbox: mailboxName, + Status: &imap.StatusData{ + Mailbox: mailboxName, + NumMessages: new(userMailbox.NumMessages), + NumRecent: new(uint32), // obsolete + UIDNext: imap.UID(userMailbox.UIDNext), + UIDValidity: userMailbox.UIDValidity, + NumUnseen: new(uint32(userMailbox.NumUnseen)), + NumDeleted: new(uint32(userMailbox.NumDeleted)), + Size: new(userMailbox.Size), + AppendLimit: new(uint32), + DeletedStorage: new(int64), + HighestModSeq: uint64(userMailbox.HighestModSeq), + }, + }, + }, nil +} + +func (m *imapSession) Create(mailbox string, _ *imap.CreateOptions) error { + logger := slog.With("session_id", m.sessionID, "cmd", "CREATE", "user_id", m.user.ID) + logger.Info("start CREATE", "mailbox", mailbox) + if strings.EqualFold(mailbox, string(model.MailboxINBOX)) { + logger.Info("creating INBOX not allowed") + return &imap.Error{ + Type: imap.StatusResponseTypeNo, + Code: imap.ResponseCodeAlreadyExists, + Text: "Mailbox INBOX always exists", + } + } + + // per RFC 9051, 7.3.1, all mailbox attributes are optional. + // Furthermore, they are user-specific in case of a shared mailbox (for example \Sent has per-user meaning). + // Ignore special attributes now because we have pre-defined mailboxes already, which should be un-deletable. + + // mailbox should exist under the name without the trailing delimiter + mailbox, _ = strings.CutPrefix(mailbox, "/") + mailbox, _ = strings.CutSuffix(mailbox, "/") + + // This can be a toctou, but we have a unique constraint on mailbox names + mbID, _, err := m.db.GetMailboxID(context.Background(), m.user.ID, mailbox) + if err != nil && !errors.Is(err, db.ErrMailboxNotFound) { + logger.Error("getting mailbox", "error", err) + return errInternalServerError + } + if mbID != 0 { + logger.Info("cannot create already existing mailbox", "mailbox", mailbox) + return &imap.Error{ + Type: imap.StatusResponseTypeNo, + Code: imap.ResponseCodeNonExistent, + Text: "Mailbox already exists", + } + } + + err = m.db.CreateMailbox(context.Background(), m.user.ID, mailbox) + if err != nil { + logger.Error("creating mailbox", "error", err) + return errInternalServerError + } + logger.Info("end CREATE") + return nil +} + +func (m *imapSession) Delete(mailbox string) error { + logger := slog.With("session_id", m.sessionID, "cmd", "DELETE", "user_id", m.user.ID) + logger.Info("start DELETE", "mailbox", mailbox) + // We do not allow to delete default mailboxes because we need their special use present + for _, dmb := range model.DefaultMailboxes() { + if string(dmb.Name) == mailbox { + logger.Info("denied deleting default mailbox", "mailbox", mailbox) + return &imap.Error{ + Type: imap.StatusResponseTypeNo, + Code: imap.ResponseCodeCannot, + Text: "Deleting preset mailbox is not allowed", + } + } + } + + mailboxID, _, err := m.db.GetMailboxID(context.Background(), m.user.ID, mailbox) + if errors.Is(err, db.ErrMailboxNotFound) { + return errMailboxNotExist + } + if err != nil { + logger.Error("getting mailbox ID", "error", err) + return errInternalServerError + } + + // deleting messages is left to the "prune" CLI and not part of this operation + err = m.db.DeleteMailbox(context.Background(), m.user.ID, mailbox) + if err != nil { + logger.Error("deleting mailbox", "error", err) + return errInternalServerError + } + + m.mTracker.remove(m.user.ID, mailboxID) + logger.Info("end DELETE", "mailbox", mailbox) + return nil +} + +func (m *imapSession) Rename(mailbox, newName string, _ *imap.RenameOptions) error { + logger := slog.With("session_id", m.sessionID, "cmd", "RENAME", "user_id", m.user.ID) + logger.Info("start RENAME", "mailbox", mailbox, "new_mailbox_name", newName) + + if mailbox == string(model.MailboxINBOX) { + // Special case per RFC 6.3.6: + // Renaming INBOX is permitted and does not result in a tagged BAD response, and it has special behavior: + // It moves all messages in INBOX to a new mailbox with the given name, leaving INBOX empty. If the server + // implementation supports inferior hierarchical names of INBOX, these are unaffected by a rename of INBOX. + err := m.db.CreateAndMoveAllMessagesFromINBOXToMailbox(context.Background(), m.user.ID, newName) + if err != nil { + logger.Error("rename INBOX", "error", err) + return errInternalServerError + } + + logger.Info("end RENAME", "mailbox", mailbox, "new_mailbox_name", newName) + return nil + } + + // We do not allow to rename default mailboxes because we need their special use present + for _, dmb := range model.DefaultMailboxes() { + if string(dmb.Name) == mailbox { + logger.Info("denied renaming default mailbox", "mailbox", mailbox) + return &imap.Error{ + Type: imap.StatusResponseTypeNo, + Code: imap.ResponseCodeCannot, + Text: "Renaming preset mailbox is not allowed", + } + } + } + + // Check if mailbox exists + _, _, err := m.db.GetMailboxID(context.Background(), m.user.ID, mailbox) + if errors.Is(err, db.ErrMailboxNotFound) { + logger.Info("mailbox does not exist", "mailbox", mailbox) + return errMailboxNotExist + } + if err != nil { + logger.Error("getting mailbox ID", "error", err) + return errInternalServerError + } + + // Check if new mailbox name already exists + newMailboxID, _, err := m.db.GetMailboxID(context.Background(), m.user.ID, newName) + if newMailboxID != 0 { + slog.Info("new mailbox name already exists", "new_mailbox_name", newMailboxID) + return &imap.Error{ + Type: imap.StatusResponseTypeNo, + Code: imap.ResponseCodeAlreadyExists, + Text: "Mailbox with the same name already exists", + } + } + if err != nil && !errors.Is(err, db.ErrMailboxNotFound) { + logger.Error("getting mailbox ID", "error", err) + return errInternalServerError + } + + err = m.db.RenameMailbox(context.Background(), m.user.ID, mailbox, newName) + if err != nil { + logger.Error("renaming mailbox", "error", err) + return errInternalServerError + } + logger.Info("end RENAME", "mailbox", mailbox, "new_mailbox_name", newName) + return nil +} + +func (m *imapSession) Subscribe(mb string) error { + logger := slog.With("session_id", m.sessionID, "cmd", "SUBSCRIBE", "user_id", m.user.ID) + logger.Info("start SUBSCRIBE", "mailbox", mb) + mailboxID, _, err := m.db.GetMailboxID(context.Background(), m.user.ID, mb) + if err != nil { + if errors.Is(err, db.ErrMailboxNotFound) { + return errMailboxNotExist + } + logger.Error("getting mailbox ID", "error", err) + return errInternalServerError + } + err = m.db.Subscribe(context.Background(), m.user.ID, mailboxID, mb) + if err != nil { + logger.Error("subscribe mailbox", "error", err) + return errInternalServerError + } + logger.Info("end SUBSCRIBE", "mailbox", mb) + return nil +} + +func (m *imapSession) Unsubscribe(mb string) error { + logger := slog.With("session_id", m.sessionID, "cmd", "UNSUBSCRIBE", "user_id", m.user.ID) + logger.Info("start UNSUBSCRIBE", "mailbox", mb) + + // We do not allow to unsubscribe from default mailboxes because we need their special use present + for _, dmb := range model.DefaultMailboxes() { + if string(dmb.Name) == mb { + logger.Info("denied unsubscribing default mailbox", "mailbox", mb) + return &imap.Error{ + Type: imap.StatusResponseTypeNo, + Code: imap.ResponseCodeCannot, + Text: "Unsubscribing preset mailbox is not allowed", + } + } + } + + err := m.db.Unsubscribe(context.Background(), m.user.ID, mb) + if err != nil { + logger.Error("unsubscribe mailbox", "error", err) + return errInternalServerError + } + logger.Info("end UNSUBSCRIBE", "mailbox", mb) + return nil +} + +func (m *imapSession) List(w *imapserver.ListWriter, ref string, patterns []string, options *imap.ListOptions) error { + logger := slog.With("session_id", m.sessionID, "cmd", "LIST", "user_id", m.user.ID) + logger.Info("start LIST", "ref", ref, "patterns", patterns, "options", options) + + // Special case LIST "" "" - mainly for the client to get the folder hierarchy delimiter + if ref == "" && len(patterns) == 0 { + err := w.WriteList(&imap.ListData{ + Attrs: []imap.MailboxAttr{imap.MailboxAttrNoSelect}, + Delim: '/', + Mailbox: "", + }) + if err != nil { + logger.Error("listing mailboxes", "error", err) + return errInternalServerError + } + logger.Info("end LIST \"\" \"\"") + return nil + } + + // LIST (SUBSCRIBED) + if options.SelectSubscribed { + subscribed, err := m.db.ListSubscribed(context.Background(), m.user.ID) + if err != nil { + logger.Error("listSubscribed", "error", err) + return errInternalServerError + } + + for mb, exists := range subscribed { + for _, pattern := range patterns { + // We MUST pattern match subscribed mailboxes + if imapserver.MatchList(mb.Name, '/', ref, pattern) { + attr := getMailboxAttr(mb) + + if !exists { + attr = append(attr, imap.MailboxAttrNonExistent, imap.MailboxAttrNoSelect) + } + + err = w.WriteList(&imap.ListData{ + Attrs: attr, + Delim: '/', + Mailbox: mb.Name, + }) + if err != nil { + logger.Error("listing mailboxes", "error", err) + return errInternalServerError + } + // Matched this mailbox, don't write it again if it matches a 2nd pattern + break + } + } + } + logger.Info("end LIST subscribed") + return nil + } + + // Standard LIST "" "*" + mailboxes, err := m.db.GetUserMailboxes(context.Background(), m.user.ID) + if err != nil { + logger.Error("get user mailboxes", "error", err) + return errInternalServerError + } + + hasChildren := calcHasChildren(mailboxes) + + for _, mb := range mailboxes { + for _, pattern := range patterns { + match := imapserver.MatchList(mb.Name, '/', ref, pattern) + logger.Debug("LIST imapserver.MatchList", "mailbox", mb.Name, "ref", ref, "pattern", pattern, "match", match) + + if match { + attr := getMailboxAttr(mb) + + // RFC 9051: MAY return children attribute even if it was not requested + if hasChildren[mb.Name] { + attr = append(attr, imap.MailboxAttrHasChildren) + } else { + attr = append(attr, imap.MailboxAttrHasNoChildren) + } + + // 2. Pass the addresses (&) of those local variables + numMessages := mb.NumMessages + numUnseen := uint32(mb.NumUnseen) + numDeleted := uint32(mb.NumDeleted) + size := mb.Size + err := w.WriteList(&imap.ListData{ + Attrs: attr, + Delim: '/', + Mailbox: mb.Name, + ChildInfo: nil, // Should be returned on RECURSIVEMATCH + Status: &imap.StatusData{ + Mailbox: mb.Name, + UIDNext: imap.UID(mb.UIDNext), + UIDValidity: mb.UIDValidity, + HighestModSeq: uint64(mb.HighestModSeq), + NumMessages: &numMessages, + NumUnseen: &numUnseen, + NumDeleted: &numDeleted, + Size: &size, + NumRecent: new(uint32), + AppendLimit: new(uint32), + DeletedStorage: new(int64), + }, + }) + + if err != nil { + logger.Error("listing mailboxes", "error", err) + return errInternalServerError + } + + logger.Debug("WriteList", "mailbox", mb.Name, "attr", attr) + + // Matched this mailbox, don't write it again + break + } + } + } + + logger.Info("end LIST") + return nil +} + +func calcHasChildren(mailboxes []model.Mailbox) map[string]bool { + hasChildren := make(map[string]bool) + for _, mb := range mailboxes { + s := strings.Split(mb.Name, "/") + if len(s) > 1 { + hasChildren[strings.Join(s[:len(s)-1], "/")] = true + } + } + return hasChildren +} + +func (m *imapSession) Status(mailboxName string, _ *imap.StatusOptions) (*imap.StatusData, error) { + logger := slog.With("session_id", m.sessionID, "cmd", "STATUS", "user_id", m.user.ID) + logger.Info("start STATUS", "mailbox_name", mailboxName) + userMailboxes, err := m.db.GetUserMailboxes(context.Background(), m.user.ID) + if err != nil { + logger.Error("get user mailboxes", "error", err) + return &imap.StatusData{}, errInternalServerError + } + + var statusBox model.Mailbox + found := false + for _, mb := range userMailboxes { + if mb.Name == mailboxName { + statusBox = mb + found = true + break + } + } + + if !found { + logger.Info("mailbox not found", "name", mailboxName) + return &imap.StatusData{}, errMailboxNotExist + } + + logger.Info("end STATUS", "mailbox_name", mailboxName) + return &imap.StatusData{ + Mailbox: mailboxName, + NumMessages: new(statusBox.NumMessages), + NumRecent: new(uint32), // obsolete + UIDNext: imap.UID(statusBox.UIDNext), + UIDValidity: statusBox.UIDValidity, + NumUnseen: new(uint32(statusBox.NumUnseen)), + NumDeleted: new(uint32(statusBox.NumDeleted)), + Size: new(statusBox.Size), + AppendLimit: new(uint32), + DeletedStorage: new(int64), + HighestModSeq: uint64(statusBox.HighestModSeq), + }, nil +} + +func (m *imapSession) Append(mailbox string, r imap.LiteralReader, options *imap.AppendOptions) (*imap.AppendData, error) { + logger := slog.With("session_id", m.sessionID, "cmd", "APPEND", "user_id", m.user.ID) + logger.Info("start APPEND", "mailbox", mailbox) + // If the destination mailbox does not exist, a server MUST return an error and MUST NOT automatically create the + // mailbox. Unless it is certain that the destination mailbox cannot be created, the server MUST send the response + // code "[TRYCREATE]" as the prefix of the text of the tagged NO response. This gives a hint to the client that it + // can attempt a CREATE command and retry the APPEND if the CREATE is successful. + mailboxID, uidValidity, err := m.db.GetMailboxID(context.Background(), m.user.ID, mailbox) + if err != nil { + if errors.Is(err, db.ErrMailboxNotFound) { + logger.Info("mailbox not found", "mailbox", mailbox) + return nil, errMailboxNotExist + } + logger.Error("get mailbox", "error", err) + return nil, errInternalServerError + } + + for _, f := range options.Flags { + if !isAllowedFlag(f) { + logger.Info("flag not allowed", "flag", f) + return nil, errBadFlag + } + } + + internalTime := time.Now() + if !options.Time.IsZero() { + internalTime = options.Time + } + + fileName, size, parsedMsg, err := m.persistence.WriteBlobMessage(r) + if err != nil { + logger.Error("write blob", "error", err) + return nil, errInternalServerError + } + + logger.Info("wrote file", "filename", fileName, "size", size) + uid, err := m.db.AppendMessage(context.Background(), mailboxID, m.user.ID, fileName, size, internalTime.UTC().Format(time.RFC3339), fromIMAPFlags(options.Flags), parsedMsg) + if err != nil { + logger.Error("write message metadata", "error", err) + return nil, errInternalServerError + } + + // In the case of a mailbox that has permissions set so that the client can APPEND to the mailbox, but not SELECT or + // EXAMINE it, the server MUST NOT send an APPENDUID response code as it would disclose information about the mailbox. + + mb, err := m.db.GetUserMailbox(context.Background(), m.user.ID, mailbox) + if err != nil { + logger.Error("get user mailbox", "error", err) + return nil, errInternalServerError + } + + t := m.mTracker.get(m.user.ID, mailboxID) + if t != nil { + t.QueueNumMessages(mb.NumMessages) + } + + logger.Info("end APPEND", "mailbox", mailbox) + return &imap.AppendData{ + UID: imap.UID(uid), + UIDValidity: uidValidity, + }, nil +} + +func (m *imapSession) Poll(w *imapserver.UpdateWriter, allowExpunge bool) error { + logger := slog.With("session_id", m.sessionID, "cmd", "POLL", "user_id", m.user.ID) + logger.Debug("start POLL") + if m.sessionTracker == nil { + logger.Debug("session tracker not initialized") + return nil + } + err := m.sessionTracker.Poll(w, allowExpunge) + if err != nil { + logger.Error("poll session tracker", "error", err) + return errInternalServerError + } + logger.Debug("end POLL") + return nil +} + +func (m *imapSession) Idle(w *imapserver.UpdateWriter, stop <-chan struct{}) error { + logger := slog.With("session_id", m.sessionID, "cmd", "IDLE", "user_id", m.user.ID) + logger.Debug("start IDLE") + if m.sessionTracker == nil { + logger.Debug("session tracker not initialized") + return nil + } + err := m.sessionTracker.Idle(w, stop) + if err != nil { + logger.Error("idle session tracker", "error", err) + return errInternalServerError + } + logger.Debug("end IDLE") + return nil +} + +func (m *imapSession) Unselect() error { + logger := slog.With("session_id", m.sessionID, "cmd", "UNSELECT", "user_id", m.user.ID) + logger.Info("start UNSELECT") + m.selectedMailbox = model.Mailbox{} + m.sessionTracker.Close() + m.sessionTracker = nil + logger.Info("end UNSELECT") + return nil +} + +// removeFromSearchResult returns res with the given uid/seq removed. +// If res is not a set type we know about, it is returned unchanged. +func removeFromSearchResult(res imap.NumSet, uid, seq uint32) imap.NumSet { + switch s := res.(type) { + case imap.UIDSet: + if !s.Contains(imap.UID(uid)) { + return s + } + out := imap.UIDSetNum() + nums, _ := s.Nums() + for _, n := range nums { + if n != imap.UID(uid) { + out.AddNum(n) // FIX: was adding the deleted uid + } + } + return out + case imap.SeqSet: + if !s.Contains(seq) { + return s + } + out := imap.SeqSetNum() + nums, _ := s.Nums() + for _, n := range nums { + if n != seq { + out.AddNum(n) // FIX: was adding the deleted seq + } + } + return out + } + return res +} + +// Expunge SELECTED state +func (m *imapSession) Expunge(_ *imapserver.ExpungeWriter, uids *imap.UIDSet) error { + ctx := context.Background() + + logger := slog.With("session_id", m.sessionID, "cmd", "EXPUNGE", "user_id", m.user.ID) + logger.InfoContext(ctx, "start EXPUNGE", "uids", uids) + if m.selectedMailboxReadOnly { + logger.InfoContext(ctx, "selected mailbox is read-only") + // This will either break Close() or Expunge() because CLOSE expects a clean OK, EXPUNGE below error + return errMailboxReadOnly + } + + if m.selectedMailbox.ID == -1 { + logger.InfoContext(ctx, "no mailbox selected") + return errNoMailboxSelected + } + + // UID EXPUNGE restricts removal to the given set. A nil or empty set means + // "expunge every \Deleted message" (plain EXPUNGE semantics). + uidFilter := func(uid imap.UID) bool { return true } + if uids != nil { + if nums, _ := uids.Nums(); len(nums) > 0 { + uidFilter = func(uid imap.UID) bool { return uids.Contains(uid) } + } + } + + toBeDeleted, err := m.db.GetAllFlaggedDeletedMessages(ctx, m.selectedMailbox.ID) + if err != nil { + logger.ErrorContext(ctx, "get flagged deleted messages", "error", err) + return errInternalServerError + } + logger.InfoContext(ctx, "candidates", "mailbox_id", m.selectedMailbox.ID, "uids", toBeDeleted) + + // Resolve all seqnums BEFORE any deletion, then delete high-to-low. + // RFC 9051: The UID EXPUNGE command permanently removes all messages that have both the \Deleted flag set and a UID + // that is included in the specified sequence set from the currently selected mailbox. If a message either does not + // have the \Deleted flag set or has a UID that is not included in the specified sequence set, it is not affected. + type expunge struct{ uid, seq uint32 } + list := make([]expunge, 0, len(toBeDeleted)) + for _, uid := range toBeDeleted { + if !uidFilter(imap.UID(uid)) { + continue + } + seq, err := m.db.UIDToServerSeq(ctx, m.selectedMailbox.ID, uid) + if err != nil { + logger.ErrorContext(ctx, "UID to server seq", "error", err) + return errInternalServerError + } + list = append(list, expunge{uid, seq}) + } + slices.SortFunc(list, func(i, j expunge) int { return int(j.seq) - int(i.seq) }) // descending + + tracker := m.mTracker.get(m.user.ID, m.selectedMailbox.ID) + for _, seqNumUID := range list { + err = m.db.DeleteMessageFromMailbox(ctx, m.selectedMailbox.ID, seqNumUID.uid) + if err != nil { + logger.ErrorContext(ctx, "delete message from mailbox", "error", err) + return errInternalServerError + } + + if tracker != nil { + tracker.QueueExpunge(seqNumUID.seq) + } + + m.searchRes = removeFromSearchResult(m.searchRes, seqNumUID.uid, seqNumUID.seq) + } + + logger.InfoContext(ctx, "end EXPUNGE") + return nil +} + +// Search SELECTED State +func (m *imapSession) Search(kind imapserver.NumKind, criteria *imap.SearchCriteria, options *imap.SearchOptions) (*imap.SearchData, error) { + logger := slog.With("session_id", m.sessionID, "cmd", "SEARCH", "user_id", m.user.ID) + logger.Info("start SEARCH", "criteria", criteria, "options", options) + allMailboxMessages, err := m.db.GetAllMailboxMessages(context.Background(), m.user.ID, m.selectedMailbox.ID) + if err != nil { + logger.Error("Unable to retrieve messages for searching", "error", err) + return nil, errInternalServerError + } + + uidSet := make([]imap.UID, 0) + uidSetCount := 0 // leads to wrong results with len(uidSet) + for _, msg := range allMailboxMessages { + if m.evalSearchCriteria(criteria, msg, m.sessionTracker, m.persistence) { + uidSet = append(uidSet, imap.UID(msg.UID)) + uidSetCount++ + } + } + + slices.Sort(uidSet) + uidMin := 0 + if len(uidSet) > 0 { + uidMin = int(uidSet[0]) + } + uidMax := 0 + if len(uidSet) > 0 { + uidMax = int(uidSet[len(uidSet)-1]) + } + uidSetNum := imap.UIDSetNum(uidSet...) + + // Combination of Result Option "$" Marker Value + //SAVE MIN -> MIN + //SAVE MAX -> MAX + //SAVE MIN MAX -> MIN & MAX + //SAVE * [m] -> all found messages + if options.ReturnSave { + m.searchRes = uidSetNum + + if options.ReturnMin { + m.searchRes = imap.UIDSetNum(imap.UID(uint32(uidMin))) + } + + if options.ReturnMax { + m.searchRes = imap.UIDSetNum(imap.UID(uint32(uidMax))) + } + + if options.ReturnMin && options.ReturnMax { + n := imap.UIDSetNum(imap.UID(uint32(uidMin))) + n.AddNum(imap.UID(uint32(uidMax))) + m.searchRes = n + } + } + + if kind == imapserver.NumKindUID { + res := &imap.SearchData{ + All: uidSetNum, + Count: uint32(uidSetCount), + Min: uint32(uidMin), + Max: uint32(uidMax), + } + logger.Info("end SEARCH", "search_result", res) + return res, nil + } + + if kind == imapserver.NumKindSeq { + out := imap.SeqSetNum() + seqNumMin := uint32(math.MaxUint32) + seqNumMax := uint32(0) + seqNumCount := 0 + num, _ := uidSetNum.Nums() + for _, n := range num { + serverSeq, err := m.db.UIDToServerSeq(context.Background(), m.selectedMailbox.ID, uint32(n)) + if err != nil { + logger.Error("Unable to retrieve server seq", "error", err) + return nil, errInternalServerError + } + seqNumCount++ + out.AddNum(serverSeq) + if serverSeq < seqNumMin { + seqNumMin = serverSeq + } + if serverSeq > seqNumMax { + seqNumMax = serverSeq + } + } + res := &imap.SearchData{ + All: out, + Count: uint32(seqNumCount), + Min: seqNumMin, + Max: seqNumMax, + } + logger.Info("end SEARCH", "search_result", res) + return res, nil + } + + return nil, errInternalServerError +} + +// Fetch SELECTED State +func (m *imapSession) Fetch(w *imapserver.FetchWriter, numSet imap.NumSet, options *imap.FetchOptions) error { + logger := slog.With("session_id", m.sessionID, "cmd", "FETCH", "user_id", m.user.ID) + logger.Info("start FETCH", "num_set", numSet, "options", options) + var outMessages []model.Message + var err error + + sourceUIDs, err := m.resolveUIDs(context.Background(), numSet) + if err != nil { + if errors.Is(err, errDBSequenceOutOfRange) { + logger.Info("sequence out of range") + return errSequenceOutOfRange + } + logger.Error("unable to resolve UIDs", "error", err) + return errInternalServerError + } + + // getMailboxMessagesByUID inherently filters out non-existing UIDs + // because it queries by mailboxID and the resolved sourceUIDs. + outMessages, err = m.db.GetMailboxMessagesByUID(context.Background(), m.user.ID, m.selectedMailbox.ID, sourceUIDs) + if err != nil { + logger.Error("unable to fetch mailbox messages", "error", err) + return errInternalServerError + } + + // TODO but more on a global scale than just here: + // seq-number = nz-number / "*" + // ; message sequence number (COPY, FETCH, STORE + // ; commands) or unique identifier (UID COPY, + // ; UID FETCH, UID STORE commands). + // ; * represents the largest number in use. In + // ; the case of message sequence numbers, it is + // ; the number of messages in a non-empty mailbox. + // ; In the case of unique identifiers, it is the + // ; unique identifier of the last message in the + // ; mailbox or, if the mailbox is empty, the + // ; mailbox's current UIDNEXT value. + // ; The server should respond with a tagged BAD + // ; response to a command that uses a message + // ; sequence number greater than the number of + // ; messages in the selected mailbox. This + // ; includes "*" if the selected mailbox is empty. + for _, o := range outMessages { + seqNum, err := m.db.UIDToServerSeq(context.Background(), m.selectedMailbox.ID, o.UID) + if err != nil { + logger.Error("unable to retrieve server seq", "error", err) + return errInternalServerError + } + + clientSeqNum := m.sessionTracker.EncodeSeqNum(seqNum) + if clientSeqNum == 0 { // message does not exist from client pov + continue + } + + mw := w.CreateMessage(clientSeqNum) + if options.UID { + mw.WriteUID(imap.UID(o.UID)) + } + + if options.InternalDate { + t, _ := time.Parse(time.RFC3339, o.InternalDate) + mw.WriteInternalDate(t) + } + + if options.RFC822Size { + mw.WriteRFC822Size(o.RFC822Size) + } + + if options.Envelope { + fromAddresses, err := parseAddressList(o.EnvelopeFrom.String) + if err != nil { + logger.Error("unable to parse envelope FROM addresses", "error", err) + return errInternalServerError + } + senderAddresses, err := parseAddressList(o.EnvelopeSender.String) + if err != nil { + logger.Error("unable to parse envelope SENDER addresses", "error", err) + return errInternalServerError + } + + // RFC: If the Sender or Reply-To header fields are absent in the [RFC5322] header, or are present but empty, + // the server sets the corresponding member of the envelope to be the same value as the from member + // (the client is not expected to know how to do this). + if len(senderAddresses) == 0 { + senderAddresses = fromAddresses + } + + replyToAddresses, err := parseAddressList(o.EnvelopeReplyTo.String) + if err != nil { + logger.Error("unable to parse envelope REPLY TO addresses", "error", err) + return errInternalServerError + } + if len(replyToAddresses) == 0 { + replyToAddresses = fromAddresses + } + + toAddresses, err := parseAddressList(o.EnvelopeTo.String) + if err != nil { + logger.Error("unable to parse envelope TO addresses", "error", err) + return errInternalServerError + } + ccAddresses, err := parseAddressList(o.EnvelopeCc.String) + if err != nil { + logger.Error("unable to parse envelope CC addresses", "error", err) + return errInternalServerError + } + bccAddresses, err := parseAddressList(o.EnvelopeBcc.String) + if err != nil { + logger.Error("unable to parse envelope BCC addresses", "error", err) + return errInternalServerError + } + + var t time.Time + if o.EnvelopeDate.Valid { + t, _ = time.Parse(time.RFC3339, o.EnvelopeDate.String) + } + + // imap-go: The In-Reply-To and Message-ID values contain message identifiers without angle brackets. + inReplyToAddresses := make([]string, 0) + for a := range strings.SplitSeq(o.EnvelopeInReplyTo.String, ",") { + inReplyTo, _ := strings.CutPrefix(a, "<") + inReplyTo, _ = strings.CutSuffix(inReplyTo, ">") + inReplyToAddresses = append(inReplyToAddresses, inReplyTo) + } + + messageID, _ := strings.CutPrefix(o.EnvelopeMessageID.String, "<") + messageID, _ = strings.CutSuffix(messageID, ">") + + mw.WriteEnvelope(&imap.Envelope{ + Date: t, + Subject: o.EnvelopeSubject.String, + From: fromAddresses, + Sender: senderAddresses, + ReplyTo: replyToAddresses, + To: toAddresses, + Cc: ccAddresses, + Bcc: bccAddresses, + InReplyTo: inReplyToAddresses, + MessageID: messageID, + }) + } + + // RFC 9051: The \Seen flag is implicitly set; if this causes the flags to change, they SHOULD be included as + // part of the FETCH responses. + mustShowFlags := false + if options.BodyStructure != nil { + r, err := m.persistence.BlobReader(o.BlobHash) + if err != nil { + logger.Error("unable to read blob", "error", err) + return errInternalServerError + } + bs := imapserver.ExtractBodyStructure(r) + err = r.Close() + if err != nil { + logger.Error("unable to close body structure", "error", err) + } + mw.WriteBodyStructure(bs) + + // Honestly not sure if flagging messages as \\Seen here is correct, or only below for BodySection + if !slices.Contains(o.Flags, string(imap.FlagSeen)) { + logger.Info("flag message as \\Seen", "uid", o.UID) + _, err = m.db.AddMessageFlags(context.Background(), m.selectedMailbox.ID, o.UID, []string{string(imap.FlagSeen)}) + if err != nil { + logger.Error("unable to add message flags", "error", err) + return errInternalServerError + } + o.Flags = append(o.Flags, string(imap.FlagSeen)) + mustShowFlags = true + } + } + + for _, bs := range options.BodySection { + if !bs.Peek && !m.selectedMailboxReadOnly { + if !slices.Contains(o.Flags, string(imap.FlagSeen)) { + logger.Info("flag message as \\Seen", "uid", o.UID) + _, err = m.db.AddMessageFlags(context.Background(), m.selectedMailbox.ID, o.UID, []string{string(imap.FlagSeen)}) + if err != nil { + logger.Error("unable to add message flags", "error", err) + return errInternalServerError + } + o.Flags = append(o.Flags, string(imap.FlagSeen)) + mustShowFlags = true + } + } + r, err := m.persistence.BlobReader(o.BlobHash) + if err != nil { + logger.Error("unable to read blob", "error", err) + return errInternalServerError + } + ebs := imapserver.ExtractBodySection(r, bs) + err = r.Close() + if err != nil { + logger.Error("unable to close blob reader", "error", err) + } + + wc := mw.WriteBodySection(bs, int64(len(ebs))) + _, writeErr := wc.Write(ebs) + closeErr := wc.Close() + if writeErr != nil { + logger.Error("unable to write body section", "error", writeErr) + return errInternalServerError + } + if closeErr != nil { + logger.Error("unable to close body section", "error", closeErr) + return errInternalServerError + } + } + + for _, bs := range options.BinarySection { + if !bs.Peek && !m.selectedMailboxReadOnly { + _, err = m.db.AddMessageFlags(context.Background(), m.selectedMailbox.ID, o.UID, []string{string(imap.FlagSeen)}) + if err != nil { + logger.Error("unable to add message flags", "error", err) + return errInternalServerError + } + } + r, err := m.persistence.BlobReader(o.BlobHash) + if err != nil { + logger.Error("unable to read blob", "error", err) + return errInternalServerError + } + buf := imapserver.ExtractBinarySection(r, bs) + err = r.Close() + if err != nil { + logger.Error("unable to close blob reader", "error", err) + } + + wc := mw.WriteBinarySection(bs, int64(len(buf))) + _, writeErr := wc.Write(buf) + closeErr := wc.Close() + if writeErr != nil { + logger.Error("unable to write body section", "error", writeErr) + return errInternalServerError + } + if closeErr != nil { + logger.Error("unable to close body section", "error", closeErr) + return errInternalServerError + } + } + + for _, bss := range options.BinarySectionSize { + r, err := m.persistence.BlobReader(o.BlobHash) + if err != nil { + logger.Error("unable to read blob", "error", err) + return errInternalServerError + } + n := imapserver.ExtractBinarySectionSize(r, bss) + err = r.Close() + if err != nil { + logger.Error("unable to close blob reader", "error", err) + } + mw.WriteBinarySectionSize(bss, n) + } + + // Flags should come last so that the updated \Seen flag is properly communicated + if options.Flags || mustShowFlags { + mw.WriteFlags(toIMAPFlags(o.Flags)) + } + + if err = mw.Close(); err != nil { + logger.Error("unable to close imap writer", "error", err) + return errInternalServerError + } + } + + slog.Info("end FETCH") + return nil +} + +// Store SELECTED state +func (m *imapSession) Store(w *imapserver.FetchWriter, numSet imap.NumSet, flags *imap.StoreFlags, _ *imap.StoreOptions) error { + logger := slog.With("session_id", m.sessionID, "cmd", "STORE", "user_id", m.user.ID) + logger.Info("start STORE", "num_set", numSet, "flags", flags) + + if m.selectedMailbox.ID == -1 { + logger.Info("no mailbox selected") + return errNoMailboxSelected + } + + if m.selectedMailboxReadOnly { + logger.Info("mailbox selected as read-only") + return errMailboxReadOnly + } + + for _, f := range flags.Flags { + if !isAllowedFlag(f) { + logger.Info("flag not allowed", "flag", f) + return errBadFlag + } + } + + sourceUIDs, err := m.resolveUIDs(context.Background(), numSet) + if err != nil { + if errors.Is(err, errDBSequenceOutOfRange) { + logger.Info("sequence out of range") + return errSequenceOutOfRange + } + logger.Error("unable to resolve UIDs", "error", err) + return errInternalServerError + } + + for _, sourceUID := range sourceUIDs { + // Filter non-existing UIDs to prevent FK constraint errors.go and comply + // with IMAP semantics where STORE on non-existent messages is a no-op. + exists, err := m.db.UIDExists(context.Background(), m.user.ID, sourceUID) + if err != nil { + logger.Error("unable to retrieve UID", "error", err) + return errInternalServerError + } + if !exists { + continue + } + + var flagSet []string + switch flags.Op { + case imap.StoreFlagsAdd: + flagSet, err = m.db.AddMessageFlags(context.Background(), m.selectedMailbox.ID, sourceUID, fromIMAPFlags(flags.Flags)) + case imap.StoreFlagsDel: + flagSet, err = m.db.DeleteMessageFlags(context.Background(), m.selectedMailbox.ID, sourceUID, fromIMAPFlags(flags.Flags)) + case imap.StoreFlagsSet: + flagSet, err = m.db.SetMessageFlags(context.Background(), m.selectedMailbox.ID, sourceUID, fromIMAPFlags(flags.Flags)) + default: + logger.Error("unknown flag op", "flag", flags.Op) + return &imap.Error{ + Type: imap.StatusResponseTypeBad, + Code: imap.ResponseCodeClientBug, + Text: fmt.Sprintf("Unknown store flags: %v", flags.Op), + } + } + + if err != nil { + logger.Error("unable to store flags", "error", err) + return errInternalServerError + } + + seqNum, err := m.db.UIDToServerSeq(context.Background(), m.selectedMailbox.ID, sourceUID) + if err != nil { + logger.Error("unable to convert to server seq", "error", err) + return errInternalServerError + } + + imapFlags := toIMAPFlags(flagSet) + + if !flags.Silent { + fetchWriter := w.CreateMessage(m.sessionTracker.EncodeSeqNum(seqNum)) + fetchWriter.WriteFlags(imapFlags) + err = fetchWriter.Close() + if err != nil { + logger.Error("unable to close imap writer", "error", err) + return errInternalServerError + } + } + + m.mTracker.get(m.user.ID, m.selectedMailbox.ID).QueueMessageFlags(seqNum, imap.UID(sourceUID), imapFlags, m.sessionTracker) + } + logger.Info("end STORE") + return nil +} + +func (m *imapSession) Copy(numSet imap.NumSet, dest string) (*imap.CopyData, error) { + logger := slog.With("session_id", m.sessionID, "cmd", "COPY", "user_id", m.user.ID) + logger.Info("start COPY", "num_set", numSet, "dest", dest) + if m.selectedMailbox.ID == -1 { + logger.Info("no mailbox selected") + return nil, errNoMailboxSelected + } + + // must find destination mailbox, otherwise return failure (see RFC) + mailboxID, uidValidity, err := m.db.GetMailboxID(context.Background(), m.user.ID, dest) + if err != nil { + if errors.Is(err, db.ErrMailboxNotFound) { + logger.Info("destination mailbox not found") + return nil, errMailboxNotExist + } + logger.Error("unable to get mailbox ID", "error", err) + return nil, errInternalServerError + } + + sourceUIDs, err := m.resolveUIDs(context.Background(), numSet) + if err != nil { + if errors.Is(err, errDBSequenceOutOfRange) { + logger.Info("sequence out of range") + return nil, errSequenceOutOfRange + } + logger.Error("unable to resolve UIDs", "error", err) + return nil, errInternalServerError + } + + // Filter non-existing UIDs to prevent copyMessagesToMailbox from burning + // destination UIDs for messages that don't exist in the source mailbox. + filteredSourceUIDs := make([]uint32, 0, len(sourceUIDs)) + for _, uid := range sourceUIDs { + exists, err := m.db.UIDExists(context.Background(), m.user.ID, uid) + if err != nil { + logger.Error("unable to retrieve UID", "error", err) + return nil, errInternalServerError + } + if exists { + filteredSourceUIDs = append(filteredSourceUIDs, uid) + } + } + + destUIDs, err := m.db.CopyMessagesToMailbox(context.Background(), m.selectedMailbox.ID, mailboxID, filteredSourceUIDs) + if err != nil { + logger.Error("unable to copy messages to mailbox", "error", err) + return nil, errInternalServerError + } + + logger.Info("end COPY") + return &imap.CopyData{ + UIDValidity: uidValidity, + SourceUIDs: internalIDToUIDSet(filteredSourceUIDs), + DestUIDs: internalIDToUIDSet(destUIDs), + }, nil +} + +func (m *imapSession) Namespace() (*imap.NamespaceData, error) { + logger := slog.With("session_id", m.sessionID, "cmd", "NAMESPACE", "user_id", m.user.ID) + logger.Info("start NAMESPACE") + logger.Info("end NAMESPACE") + return &imap.NamespaceData{ + Personal: []imap.NamespaceDescriptor{ + { + Prefix: "", + Delim: '/', + }, + }, + Other: nil, + Shared: nil, + }, nil +} + +// Move SELECTED state +func (m *imapSession) Move(w *imapserver.MoveWriter, numSet imap.NumSet, dest string) error { + ctx := context.Background() + + logger := slog.With("session_id", m.sessionID, "cmd", "MOVE", "user_id", m.user.ID) + logger.InfoContext(ctx, "start MOVE", "num_set", numSet, "dest", dest) + + if m.selectedMailbox.ID == -1 { + logger.InfoContext(ctx, "no mailbox selected") + return errNoMailboxSelected + } + + if m.selectedMailboxReadOnly { + logger.InfoContext(ctx, "mailbox selected as read-only") + return errMailboxReadOnly + } + + // must find destination mailbox, otherwise return failure (see RFC) + destMailboxID, uidValidity, err := m.db.GetMailboxID(ctx, m.user.ID, dest) + if err != nil { + if errors.Is(err, db.ErrMailboxNotFound) { + logger.InfoContext(ctx, "destination mailbox not found") + return errMailboxNotExist + } + logger.ErrorContext(ctx, "unable to get mailbox ID", "error", err) + return errInternalServerError + } + + if m.selectedMailbox.ID == destMailboxID { + logger.InfoContext(ctx, "moving messages within the same mailbox is not supported") + return &imap.Error{ + Type: imap.StatusResponseTypeNo, + Code: imap.ResponseCodeCannot, + Text: "messages already moved", + } + } + + sourceUIDs, err := m.resolveUIDs(ctx, numSet) + if err != nil { + if errors.Is(err, errDBSequenceOutOfRange) { + logger.InfoContext(ctx, "sequence out of range") + return errSequenceOutOfRange + } + logger.ErrorContext(ctx, "unable to resolve UIDs", "error", err) + return errInternalServerError + } + + // Filter non-existing UIDs to prevent burning destination UIDs + filteredSourceUIDs := make([]uint32, 0, len(sourceUIDs)) + for _, uid := range sourceUIDs { + exists, err := m.db.UIDExists(ctx, m.user.ID, uid) + if err != nil { + logger.ErrorContext(ctx, "unable to retrieve UID", "error", err) + return errInternalServerError + } + if exists { + filteredSourceUIDs = append(filteredSourceUIDs, uid) + } + } + + if len(filteredSourceUIDs) == 0 { + logger.InfoContext(ctx, "no messages to move") + return nil + } + + // for expunging later, we need to calculate the serverSeq now + expungeServerSeq := make([]uint32, len(filteredSourceUIDs)) + for i, uid := range filteredSourceUIDs { + serverSeq, err := m.db.UIDToServerSeq(ctx, m.selectedMailbox.ID, uid) + if err != nil { + logger.ErrorContext(ctx, "unable to convert to server seq", "error", err) + return errInternalServerError + } + expungeServerSeq[i] = serverSeq + } + + // Fetch message envelopes BEFORE the move (after the move, dest UIDs are + // reassigned and source UIDs no longer resolve to live rows). + movedMessages, err := m.db.GetMailboxMessagesByUID(ctx, m.user.ID, m.selectedMailbox.ID, filteredSourceUIDs) + if err != nil { + logger.ErrorContext(ctx, "unable to fetch messages for policy hook", "error", err) + return errInternalServerError + } + + destUIDs, err := m.db.MoveMessagesToMailbox(ctx, m.selectedMailbox.ID, destMailboxID, filteredSourceUIDs) + if err != nil { + logger.ErrorContext(ctx, "unable to move messages to mailbox", "error", err) + return errInternalServerError + } + + // Fire the on_message_moved policy hook for each moved message. The move + // has already succeeded, so policy errors are logged but do not fail the + // MOVE command. + policyUser := policy.UserToStarlark(m.user, m.db) + for _, mv := range movedMessages { + msg := policy.MessageContextFromEnvelope(mv.EnvelopeFrom.String, mv.EnvelopeSubject.String, mv.RFC822Size) + if err := m.policyEngine.OnMessageMoved(policyUser, msg, m.selectedMailbox.Name, dest); err != nil { + logger.ErrorContext(ctx, "on_message_moved policy hook error", "error", err, "uid", mv.UID) + } + } + + // RFC 9051: Servers are also REQUIRED to send the COPYUID response code in an untagged OK before sending + // EXPUNGE or similar responses. + err = w.WriteCopyData(&imap.CopyData{ + UIDValidity: uidValidity, + SourceUIDs: internalIDToUIDSet(filteredSourceUIDs), + DestUIDs: internalIDToUIDSet(destUIDs), + }) + if err != nil { + logger.ErrorContext(ctx, "unable to write copy data", "error", err) + return errInternalServerError + } + + slices.Reverse(expungeServerSeq) + for _, serverSeq := range expungeServerSeq { + clientSeq := m.sessionTracker.EncodeSeqNum(serverSeq) + if clientSeq == 0 { + continue + } + m.mTracker.get(m.user.ID, m.selectedMailbox.ID).QueueExpunge(clientSeq) + } + + destMsgCount, err := m.db.MailboxMessageCount(ctx, destMailboxID) + if err != nil { + logger.ErrorContext(ctx, "unable to get mailbox message count", "error", err) + return errInternalServerError + } + destMailboxTracker := m.mTracker.getMailboxTracker(m.user.ID, destMailboxID, destMsgCount) + destMailboxTracker.QueueNumMessages(destMsgCount) + + logger.Info("end MOVE") + return nil +} + +func internalIDToUIDSet(internalID []uint32) imap.UIDSet { + out := imap.UIDSet{} + for _, i := range internalID { + out.AddNum(imap.UID(i)) + } + if len(out) == 0 { + return nil // empty UIDSets are not allowed + } + return out +} + +func resolveRange[T ~uint32](start, stop, last T) (T, T) { + if start == 0 { + start = last + } + if stop == 0 { + stop = last + } + return start, stop +} + +// resolveUIDs resolves a given imap.NumSet to the internal representation of imap UIDs which is uint32 +func (m *imapSession) resolveUIDs(ctx context.Context, numSet imap.NumSet) ([]uint32, error) { + staticSet, err := m.resolveDynamicSet(ctx, numSet) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + return nil, nil + } + return nil, err + } + + switch v := staticSet.(type) { + case imap.UIDSet: + nums, ok := v.Nums() + if !ok { + // This should theoretically not happen after dynamic resolution, + // but we handle it defensively. + return nil, fmt.Errorf("unresolvable UIDSet") + } + out := make([]uint32, len(nums)) + for i, num := range nums { + out[i] = uint32(num) + } + return out, nil + + case imap.SeqSet: + nums, ok := v.Nums() + if !ok { + return nil, fmt.Errorf("unresolvable SeqSet") + } + + uids := make([]uint32, 0, len(nums)) + for _, num := range nums { + uid, err := m.db.ServerSeqToUID(ctx, m.selectedMailbox.ID, m.sessionTracker.DecodeSeqNum(num)) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + return nil, errDBSequenceOutOfRange + } + return nil, err + } + uids = append(uids, uid) + } + return uids, nil + } + + return nil, fmt.Errorf("unsupported NumSet type %T", numSet) +} + +// resolveDynamicSet resolves the NumSet to the client view, so it can be converted later to the server view +func (m *imapSession) resolveDynamicSet(ctx context.Context, numSet imap.NumSet) (imap.NumSet, error) { + if !numSet.Dynamic() { + return numSet, nil + } + + if numSet.String() == "$" { + return m.searchRes, nil + } + + switch v := numSet.(type) { + case imap.UIDSet: + last, err := m.db.GetLastUID(ctx, m.user.ID, m.selectedMailbox.ID) + if err != nil { + return nil, err + } + if last == 0 { + return imap.UIDSet{}, nil // empty mailbox: UID commands match nothing + } + out := imap.UIDSet{} + for _, r := range v { + start, stop := resolveRange(r.Start, r.Stop, last) + out.AddRange(start, stop) + } + return out, nil + + case imap.SeqSet: + last, err := m.db.MailboxMessageCount(ctx, m.selectedMailbox.ID) + if err != nil { + return nil, err + } + if last == 0 { + return imap.SeqSet{}, nil // empty mailbox: "*" is 0, so any seq set matches nothing + } + out := imap.SeqSet{} + for _, r := range v { + start, stop := resolveRange(r.Start, r.Stop, m.sessionTracker.EncodeSeqNum(last)) + out.AddRange(start, stop) + } + return out, nil + + default: + return numSet, nil + } +}
A
internal/imap/handler_test.go
@@ -0,0 +1,27 @@
+package imap + +import ( + "testing" + + "postern/internal/model" +) + +func TestHasChildren(t *testing.T) { + mailboxes := []model.Mailbox{ + { + Name: "Trash", + }, + { + Name: "Trash/deleted", + }, + } + + hasChildren := calcHasChildren(mailboxes) + + if !hasChildren["Trash"] { + t.Error("expected trash to have a child") + } + if hasChildren["Trash/deleted"] { + t.Error("expected trashed folder to not have a child") + } +}
A
internal/imap/imap.go
@@ -0,0 +1,43 @@
+package imap + +import ( + "crypto/tls" + "sync" + + "postern/internal/db" + "postern/internal/persistence" + "postern/internal/policy" + + "github.com/emersion/go-imap/v2/imapserver" +) + +type Config struct { + DB *db.DB + Listen string + Persistence *persistence.Persistence + TLSConfig *tls.Config + PolicyEngine *policy.Engine +} +type Server struct { + db *db.DB + mTracker *mailboxTrackerRegistry + listen string + persistence *persistence.Persistence + tlsConfig *tls.Config + policyEngine *policy.Engine + + mu sync.Mutex + imapSrv *imapserver.Server + status string +} + +func NewServer(config *Config) (*Server, error) { + return &Server{ + mTracker: newMailboxTrackerRegistry(), + db: config.DB, + listen: config.Listen, + persistence: config.Persistence, + tlsConfig: config.TLSConfig, + policyEngine: config.PolicyEngine, + }, nil +}
A
internal/imap/mailbox.go
@@ -0,0 +1,32 @@
+package imap + +import ( + "postern/internal/model" + + "github.com/emersion/go-imap/v2" +) + +func getMailboxAttr(m model.Mailbox) []imap.MailboxAttr { + mbAttr := make([]imap.MailboxAttr, 0) + + var validSpecialUse = map[imap.MailboxAttr]bool{ + imap.MailboxAttrArchive: true, + imap.MailboxAttrDrafts: true, + imap.MailboxAttrJunk: true, + imap.MailboxAttrSent: true, + imap.MailboxAttrTrash: true, + } + + if m.SpecialUse != nil { + attr := imap.MailboxAttr(*m.SpecialUse) + if validSpecialUse[attr] { + mbAttr = append(mbAttr, attr) + } + } + + if m.IsSubscribed { + mbAttr = append(mbAttr, imap.MailboxAttrSubscribed) + } + + return mbAttr +}
A
internal/imap/search.go
@@ -0,0 +1,304 @@
+package imap + +import ( + "context" + "database/sql" + "io" + "log/slog" + "net/mail" + "strings" + "time" + + "postern/internal/model" + "postern/internal/persistence" + + "github.com/emersion/go-imap/v2" + "github.com/emersion/go-imap/v2/imapserver" +) + +func (m *imapSession) evalSearchCriteria(criteria *imap.SearchCriteria, msg model.Message, sessionTracker *imapserver.SessionTracker, persistence *persistence.Persistence) bool { + if criteria == nil { + return false + } + + for _, child := range criteria.Or { + if !m.evalSearchCriteria(&child[0], msg, sessionTracker, persistence) && !m.evalSearchCriteria(&child[1], msg, sessionTracker, persistence) { + return false + } + } + + for _, child := range criteria.Not { + if m.evalSearchCriteria(&child, msg, sessionTracker, persistence) { + return false + } + } + + if !evalSearchFlags(criteria, msg) { + return false + } + + if !evalSearchID(criteria, msg, sessionTracker) { + return false + } + + if !evalSearchDates(criteria, msg) { + return false + } + + if !evalSearchSizes(criteria, msg) { + return false + } + + if !evalSearchMessageContent(criteria, msg, persistence) { + return false + } + + return true +} + +func evalSearchID(criteria *imap.SearchCriteria, msg model.Message, sessionTracker *imapserver.SessionTracker) bool { + for _, uid := range criteria.UID { + if !uid.Contains(imap.UID(msg.UID)) { + return false + } + } + + for _, seq := range criteria.SeqNum { + serverSeq, err := msg.ServerSeq(context.Background()) + if err != nil { + slog.Error("could not get serverSeq", "error", err) + return false + } + + if !seq.Contains(sessionTracker.EncodeSeqNum(serverSeq)) { + return false + } + } + + return true +} + +func evalSearchFlags(criteria *imap.SearchCriteria, msg model.Message) bool { + for _, f := range criteria.Flag { + found := false + for _, hasFlags := range toIMAPFlags(msg.Flags) { + if strings.EqualFold(string(f), string(hasFlags)) { + found = true + break + } + } + if !found { + return false + } + } + + for _, f := range criteria.NotFlag { + for _, hasFlags := range toIMAPFlags(msg.Flags) { + if strings.EqualFold(string(f), string(hasFlags)) { + return false + } + } + } + + return true +} + +func evalSearchDates(criteria *imap.SearchCriteria, msg model.Message) bool { + // Internal Date (SINCE, BEFORE) + idate, _ := time.Parse(time.RFC3339, msg.InternalDate) + if !criteria.Since.IsZero() { + msgDay := idate.UTC().Truncate(24 * time.Hour) + refDay := criteria.Since.UTC().Truncate(24 * time.Hour) + if msgDay.Before(refDay) { + return false + } + } + if !criteria.Before.IsZero() { + msgDay := idate.UTC().Truncate(24 * time.Hour) + refDay := criteria.Before.UTC().Truncate(24 * time.Hour) + if !msgDay.Before(refDay) { + return false + } + } + + // Envelope Date (SENTSINCE, SENTBEFORE) + d, _ := time.Parse(time.RFC3339, msg.EnvelopeDate.String) + if !criteria.SentSince.IsZero() { + if msg.EnvelopeDate.String == "" { // searches for missing fields result in a failed search + return false + } + msgDay := d.UTC().Truncate(24 * time.Hour) + refDay := criteria.SentSince.UTC().Truncate(24 * time.Hour) + if msgDay.Before(refDay) { + return false + } + } + if !criteria.SentBefore.IsZero() { + if msg.EnvelopeDate.String == "" { // searches for missing fields result in a failed search + return false + } + msgDay := d.UTC().Truncate(24 * time.Hour) + refDay := criteria.SentBefore.UTC().Truncate(24 * time.Hour) + if !msgDay.Before(refDay) { + return false + } + } + + return true +} + +func evalSearchSizes(criteria *imap.SearchCriteria, msg model.Message) bool { + if criteria.Smaller != 0 { + if msg.RFC822Size > criteria.Smaller { + return false + } + } + + if criteria.Larger != 0 { + if msg.RFC822Size < criteria.Larger { + return false + } + } + + return true +} + +// evalSearchMessageContent compares lowercased strings where applicable. +// In the future, consider UTF-8 normalizing the strings first to find more of what was intended to be found. +func evalSearchMessageContent(criteria *imap.SearchCriteria, msg model.Message, persistence *persistence.Persistence) bool { + // if there's anything left over for full message analysis + nonEnvelopeHeader := make([]imap.SearchCriteriaHeaderField, 0, len(criteria.Header)) + + checkHeader := func(needle string, storedHeader sql.NullString) bool { + if needle == "" { + return storedHeader.Valid + } + if !storedHeader.Valid { + return false + } + return strings.Contains(strings.ToLower(storedHeader.String), needle) + } + + for _, h := range criteria.Header { + headerValueLower := strings.ToLower(h.Value) + switch strings.ToLower(h.Key) { + case "subject": + if !checkHeader(headerValueLower, msg.EnvelopeSubject) { + return false + } + case "message-id": + if !checkHeader(headerValueLower, msg.EnvelopeMessageID) { + return false + } + case "in-reply-to": + if !checkHeader(headerValueLower, msg.EnvelopeInReplyTo) { + return false + } + case "from": + if !checkHeader(headerValueLower, msg.EnvelopeFrom) { + return false + } + case "sender": + if !checkHeader(headerValueLower, msg.EnvelopeSender) { + return false + } + case "reply-to": + if !checkHeader(headerValueLower, msg.EnvelopeReplyTo) { + return false + } + case "to": + if !checkHeader(headerValueLower, msg.EnvelopeTo) { + return false + } + case "cc": + if !checkHeader(headerValueLower, msg.EnvelopeCc) { + return false + } + case "bcc": + if !checkHeader(headerValueLower, msg.EnvelopeBcc) { + return false + } + default: + nonEnvelopeHeader = append(nonEnvelopeHeader, h) + } + } + + // At this point, we need to load the full message from disk + if len(nonEnvelopeHeader) > 0 || len(criteria.Text) > 0 || len(criteria.Body) > 0 { + readMessage, err := persistence.BlobReader(msg.BlobHash) + if err != nil { + return false + } + + // TODO decode MIME + parsedMessage, err := mail.ReadMessage(readMessage) + if err != nil { + return false + } + err = readMessage.Close() + if err != nil { + return false + } + + for _, h := range nonEnvelopeHeader { + headerVal, ok := parsedMessage.Header[h.Key] + if !ok { // RFC special case to detect header existence + return false + } + foundInValues := false + for _, headerValue := range headerVal { + if strings.Contains(strings.ToLower(headerValue), strings.ToLower(h.Value)) { + foundInValues = true + break + } + } + if !foundInValues { + return false + } + } + + // need to read body + if len(criteria.Body) > 0 || len(criteria.Text) > 0 { + messageData, err := io.ReadAll(parsedMessage.Body) + if err != nil { + return false + } + // RFC expects case-insensitive search + messageDataLower := strings.ToLower(string(messageData)) + + for _, needle := range criteria.Body { + if !strings.Contains(messageDataLower, strings.ToLower(needle)) { + return false + } + } + + for _, needle := range criteria.Text { + needleLower := strings.ToLower(needle) + if strings.Contains(messageDataLower, needleLower) { + continue + } + found := false + for k, v := range parsedMessage.Header { + if strings.Contains(strings.ToLower(k), needleLower) { + found = true + break + } + for _, headerValue := range v { + if strings.Contains(strings.ToLower(headerValue), needleLower) { + found = true + break + } + } + if found { + break + } + } + if !found { + return false + } + } + } + } + + return true +}
A
internal/imap/search_test.go
@@ -0,0 +1,85 @@
+package imap + +import ( + "database/sql" + "testing" + "time" + + "postern/internal/model" + + "github.com/emersion/go-imap/v2" +) + +func TestSearchDate(t *testing.T) { + intDate := time.Now().UTC() + envDate := time.Now().UTC() + m := model.Message{ + UID: 123, + InternalDate: intDate.Format(time.RFC3339), + EnvelopeDate: sql.NullString{ + String: envDate.Format(time.RFC3339), + Valid: true, + }, + } + + // internal + criteria := &imap.SearchCriteria{ + Before: intDate.Add(-time.Hour * 24), + } + + if evalSearchDates(criteria, m) { + t.Error("expected BEFORE not to find given message") + } + + criteria.Before = criteria.Before.Add(time.Hour * 24 * 2) + if !evalSearchDates(criteria, m) { + t.Error("expected BEFORE to find given message") + } + + criteria = &imap.SearchCriteria{Since: time.Now().UTC()} + if !evalSearchDates(criteria, m) { + t.Error("expected SINCE to find given message") + } + + criteria.Since = criteria.Since.Add(time.Hour * 24) + if evalSearchDates(criteria, m) { + t.Error("expected SINCE not to find given message") + } + + // envelope date + criteria = &imap.SearchCriteria{ + SentBefore: intDate.Add(-time.Hour * 24), + } + if evalSearchDates(criteria, m) { + t.Error("expected SENTBEFORE not to find given message") + } + + criteria.SentBefore = criteria.SentBefore.Add(time.Hour * 24 * 2) + if !evalSearchDates(criteria, m) { + t.Error("expected SENTBEFORE to find given message") + } + + criteria = &imap.SearchCriteria{ + SentSince: time.Now().UTC(), + } + if !evalSearchDates(criteria, m) { + t.Error("expected SENTSINCE to find given message") + } + criteria.SentSince = time.Now().UTC().Add(time.Hour * 24) + if evalSearchDates(criteria, m) { + t.Error("expected SENTSINCE not to find given message") + } + + // missing envelope date will not result in any findings + m = model.Message{ + UID: 123, + InternalDate: intDate.Format(time.RFC3339), + EnvelopeDate: sql.NullString{}, + } + criteria = &imap.SearchCriteria{ + SentBefore: intDate.Add(time.Hour * 24), + } + if evalSearchDates(criteria, m) { + t.Error("expected SENTBEFORE not to find given message with missing envelope Date") + } +}
A
internal/imap/server.go
@@ -0,0 +1,50 @@
+package imap + +import ( + "github.com/emersion/go-imap/v2" + "github.com/emersion/go-imap/v2/imapserver" +) + +func (i *Server) Start() error { + op := &imapserver.Options{ + NewSession: i.handleIMAPConn, + Caps: imap.CapSet{ + imap.CapIMAP4rev1: {}, + imap.CapIMAP4rev2: {}, + }, + Logger: nil, + TLSConfig: i.tlsConfig, + InsecureAuth: false, + DebugWriter: nil, // os.Stdout, + } + + s := imapserver.New(op) + + i.mu.Lock() + i.imapSrv = s + i.status = "running" + i.mu.Unlock() + + err := s.ListenAndServeTLS(i.listen) + if err != nil { + return err + } + + return nil +} + +func (i *Server) Stop() error { + i.mu.Lock() + defer i.mu.Unlock() + i.status = "stopped" + if i.imapSrv != nil { + return i.imapSrv.Close() + } + return nil +} + +func (i *Server) Status() string { + i.mu.Lock() + defer i.mu.Unlock() + return i.status +}
A
internal/imap/submission.go
@@ -0,0 +1,98 @@
+package imap + +import ( + "context" + "errors" + "fmt" + "io" + "log/slog" + "net" + "time" + + "postern/internal/db" + "postern/internal/model" + "postern/internal/policy" +) + +func (i *Server) SubmitMessage(ctx context.Context, conn net.Conn, user model.User, r io.Reader) error { + logger := slog.With("component", "internal_submission", "user_id", user.ID) + internalTime := time.Now() + + fileName, size, parsedMsg, err := i.persistence.WriteBlobMessage(r) + if err != nil { + return fmt.Errorf("write blob: %w", err) + } + + if parsedMsg == nil { + return errors.New("parsed message is nil") + } + + policyUser := policy.UserToStarlark(user, i.db) + policyConn := policy.ConnToStarlark(conn) + + fromAddr, err := policy.ParseAddress(parsedMsg.Header.Get("From")) + if err != nil { + return model.ErrMissingFromHeader + } + toAddr, err := policy.ParseAddress(parsedMsg.Header.Get("To")) + if err != nil { + return errors.New("missing To header") + } + subject := parsedMsg.Header.Get("Subject") + msgSize := size + + msg := policy.MessageContext{ + HeaderFrom: fromAddr, + HeaderTo: toAddr, + Subject: subject, + Headers: parsedMsg.Header, + Size: msgSize, + } + + destinationMailbox, err := i.policyEngine.OnMessageDeliver(policyConn, policyUser, msg) + if destinationMailbox == "" || err != nil { + logger.Warn("Policy script error, falling back to INBOX", "error", err) + destinationMailbox = "INBOX" + } + + logger.InfoContext(ctx, "delivery", "mailbox", destinationMailbox) + + // Create mailbox if not exists + mailboxID, _, err := i.db.GetMailboxID(ctx, user.ID, destinationMailbox) + if err != nil { + if errors.Is(err, db.ErrMailboxNotFound) { + err = i.db.CreateMailbox(ctx, user.ID, destinationMailbox) + if err != nil { + return fmt.Errorf("create on-demand mailbox: %w", err) + } + mailboxID, _, err = i.db.GetMailboxID(ctx, user.ID, destinationMailbox) + if err != nil { + return fmt.Errorf("get on-demand mailbox ID: %w", err) + } + err = i.db.Subscribe(ctx, user.ID, mailboxID, destinationMailbox) + if err != nil { + return fmt.Errorf("subscribe on-demand mailbox: %w", err) + } + } else { + return fmt.Errorf("get mailbox ID: %w", err) + } + } + + _, err = i.db.AppendMessage(ctx, mailboxID, user.ID, fileName, size, internalTime.UTC().Format(time.RFC3339), nil, parsedMsg) + if err != nil { + return fmt.Errorf("append message: %w", err) + } + + // Notify user + t := i.mTracker.get(user.ID, mailboxID) + if t != nil { + mb, err := i.db.GetUserMailbox(ctx, user.ID, destinationMailbox) + if err != nil { + return fmt.Errorf("get user mailbox: %w", err) + } + t.QueueNumMessages(mb.NumMessages) + logger.Info("EXISTS message sent to user", "user", user.ID, "mailbox", mb.Name, "num_messages", mb.NumMessages) + } + + return nil +}
A
internal/imap/tracker.go
@@ -0,0 +1,48 @@
+package imap + +import ( + "sync" + + "github.com/emersion/go-imap/v2/imapserver" +) + +type mailboxTrackerKey struct { + userID, mailboxID int +} + +type mailboxTrackerRegistry struct { + mu sync.RWMutex + trackers map[mailboxTrackerKey]*imapserver.MailboxTracker +} + +func newMailboxTrackerRegistry() *mailboxTrackerRegistry { + return &mailboxTrackerRegistry{ + trackers: make(map[mailboxTrackerKey]*imapserver.MailboxTracker), + } +} + +func (m *mailboxTrackerRegistry) getMailboxTracker(userID, mailboxID int, numMessages uint32) *imapserver.MailboxTracker { + k := mailboxTrackerKey{userID, mailboxID} + + m.mu.Lock() + defer m.mu.Unlock() + + if t, ok := m.trackers[k]; ok { + return t + } + t := imapserver.NewMailboxTracker(numMessages) + m.trackers[k] = t + return t +} + +func (r *mailboxTrackerRegistry) get(userID, mailboxID int) *imapserver.MailboxTracker { + r.mu.Lock() + defer r.mu.Unlock() + return r.trackers[mailboxTrackerKey{userID, mailboxID}] +} + +func (r *mailboxTrackerRegistry) remove(userID, mailboxID int) { + r.mu.Lock() + defer r.mu.Unlock() + delete(r.trackers, mailboxTrackerKey{userID, mailboxID}) +}
A
internal/inbound/error.go
@@ -0,0 +1,19 @@
+package inbound + +import ( + smtpserver "github.com/emersion/go-smtp" +) + +var ( + errInternalServerError = &smtpserver.SMTPError{ + Code: 451, + EnhancedCode: smtpserver.EnhancedCode{4, 3, 0}, + Message: "Internal Server Error", + } + + errSTARTTLSrequired = &smtpserver.SMTPError{ + Code: 421, + EnhancedCode: smtpserver.EnhancedCode{4, 7, 0}, + Message: "STARTTLS required", + } +)
A
internal/inbound/inbound.go
@@ -0,0 +1,294 @@
+package inbound + +import ( + "bytes" + "context" + "crypto/tls" + "database/sql" + "errors" + "fmt" + "io" + "log/slog" + "net" + "net/mail" + "os" + "strings" + "sync" + "time" + + "postern/internal/db" + "postern/internal/model" + "postern/internal/policy" + + "github.com/emersion/go-imap/v2/imapserver" + "github.com/emersion/go-sasl" + smtpserver "github.com/emersion/go-smtp" + "github.com/google/uuid" +) + +type MessageDeliverer interface { + SubmitMessage(ctx context.Context, conn net.Conn, user model.User, r io.Reader) error +} + +type Config struct { + DB *db.DB + Deliverer MessageDeliverer + Listen string + TLS *tls.Config + PolicyEngine *policy.Engine +} +type Server struct { + db *db.DB + listen string + deliverer MessageDeliverer + tlsConfig *tls.Config + policyEngine *policy.Engine + + mu sync.Mutex + smtpSrv *smtpserver.Server + status string +} + +func NewServer(config *Config) (*Server, error) { + return &Server{ + db: config.DB, + listen: config.Listen, + deliverer: config.Deliverer, + tlsConfig: config.TLS, + policyEngine: config.PolicyEngine, + }, nil +} + +func (i *Server) Start() error { + be := &backend{ + db: i.db, + deliverer: i.deliverer, + policyEngine: i.policyEngine, + } + s := smtpserver.NewServer(be) + + s.Addr = i.listen + s.AllowInsecureAuth = false + s.TLSConfig = i.tlsConfig + s.MaxMessageBytes = 20_000_000 + + i.mu.Lock() + i.smtpSrv = s + i.status = "running" + i.mu.Unlock() + + err := s.ListenAndServe() + if err != nil { + return err + } + + return nil +} + +func (i *Server) Stop() error { + i.mu.Lock() + defer i.mu.Unlock() + i.status = "stopped" + if i.smtpSrv != nil { + return i.smtpSrv.Close() + } + return nil +} + +func (i *Server) Status() string { + i.mu.Lock() + defer i.mu.Unlock() + return i.status +} + +type smtpSession struct { + db *db.DB + conn *smtpserver.Conn + deliverer MessageDeliverer + authenticatedUser *model.SMTPAuth + toAddress string + fromAddress string + helo string + destinationUser model.User + sessionID string + policyEngine *policy.Engine + logger *slog.Logger +} + +// The backend implements SMTP server methods. +type backend struct { + db *db.DB + deliverer MessageDeliverer + policyEngine *policy.Engine +} + +// NewSession is called after client greeting (EHLO, HELO). +func (bkd *backend) NewSession(c *smtpserver.Conn) (smtpserver.Session, error) { + sessionID := uuid.NewString() + l := slog.With("component", "inbound", "remote_ip", c.Conn().RemoteAddr().String(), "session_id", sessionID) + l.Info("new session") + return &smtpSession{ + db: bkd.db, + conn: c, + deliverer: bkd.deliverer, + authenticatedUser: &model.SMTPAuth{}, + sessionID: sessionID, + logger: l, + policyEngine: bkd.policyEngine, + }, nil +} + +// AuthMechanisms returns a slice of available auth mechanisms +func (s *smtpSession) AuthMechanisms() []string { + return []string{sasl.Plain} +} + +// Auth is the handler for supported authenticators. +func (s *smtpSession) Auth(_ string) (sasl.Server, error) { + return sasl.NewPlainServer(func(identity, username, password string) error { + u, err := s.db.GetSMTPAuthUser(context.Background(), username) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + s.logger.Info("user not found") + } else { + s.logger.Error("getting user", "error", err) + } + return imapserver.ErrAuthFailed + } + + err = u.VerifyPassword([]byte(password)) + if err != nil { + slog.Info("invalid password", "username", username) + return imapserver.ErrAuthFailed + } + + s.authenticatedUser = &u + s.logger.Info("authenticated", "user_id", u.ID, "username", username) + return nil + }), nil +} + +func (s *smtpSession) Reset() { + s.toAddress = "" + s.fromAddress = "" +} + +func (s *smtpSession) Logout() error { + s.authenticatedUser = nil + return nil +} + +func (s *smtpSession) Mail(from string, _ *smtpserver.MailOptions) error { + if s.authenticatedUser == nil { + return smtpserver.ErrAuthFailed + } + + s.logger.Info("MAIL FROM", "from", from) + s.fromAddress = from + + return nil + +} + +func (s *smtpSession) Rcpt(to string, _ *smtpserver.RcptOptions) error { + if s.authenticatedUser == nil { + return smtpserver.ErrAuthFailed + } + ctx := context.Background() + s.logger.InfoContext(ctx, "RCPT TO", "to", to) + + addr, err := mail.ParseAddress(to) + if err != nil { + s.logger.InfoContext(ctx, "RCPT TO", "error", err) + return &smtpserver.SMTPError{ + Code: 501, + EnhancedCode: smtpserver.EnhancedCode{5, 1, 3}, + Message: "Bad recipient address syntax", + } + } + + atIndex := strings.LastIndex(addr.Address, "@") + if atIndex == -1 { + s.logger.InfoContext(ctx, "RCPT TO", "error", "malformed email: missing @ domain separator") + return &smtpserver.SMTPError{ + Code: 501, + EnhancedCode: smtpserver.EnhancedCode{5, 1, 3}, + Message: "Bad recipient address syntax", + } + } + + localPart := addr.Address[:atIndex] + domain := addr.Address[atIndex+1:] + baseName, tag, _ := strings.Cut(localPart, "+") + + s.logger.InfoContext(ctx, "parsed address", "baseName", baseName, "tag", tag, "domain", domain) + + u, err := s.db.GetUserForAddress(ctx, fmt.Sprintf("%s@%s", baseName, domain)) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + s.logger.InfoContext(ctx, "RCPT TO address not found") + return &smtpserver.SMTPError{ + Code: 550, + EnhancedCode: smtpserver.EnhancedCode{5, 1, 1}, + Message: "Mailbox unavailable", + } + } + s.logger.ErrorContext(ctx, "RCPT TO get address", "error", err) + return &smtpserver.SMTPError{ + Code: 451, + EnhancedCode: smtpserver.EnhancedCode{4, 3, 0}, + Message: "Temporary local problem, please try again later", + } + } + + s.destinationUser = u + s.toAddress = to + + s.logger.InfoContext(ctx, "RCPT TO", "destination_user", u.Name, "destination_user_id", u.ID) + + return nil +} + +func (s *smtpSession) Data(r io.Reader) error { + if s.authenticatedUser == nil { + return smtpserver.ErrAuthFailed + } + + ctx := context.Background() + s.logger.InfoContext(ctx, "DATA") + + var buffer bytes.Buffer + + // Prepend headers + buffer.WriteString(fmt.Sprintf("Return-Path: %s\r\n", s.fromAddress)) + myHostname, err := os.Hostname() + if err != nil { + myHostname = "postern.local" + } + with := "ESMTPSA" // we only allow encrypted (S) and authenticated (A) connections for inbound + receivedTimestampLayout := "Mon, 02 Jan 2006 15:04:05 -0700 (UTC)" + receivedTimestamp := time.Now().UTC().Format(receivedTimestampLayout) + buffer.WriteString(fmt.Sprintf("Received: from %s (%s) by %s (Postern p25.dev) with %s id %s for <%s>; %s\r\n", + s.conn.Hostname(), s.conn.Conn().RemoteAddr().String(), myHostname, with, s.sessionID, s.toAddress, receivedTimestamp)) + + _, err = io.Copy(&buffer, r) + if err != nil { + s.logger.ErrorContext(ctx, "Copy", "error", err) + return errInternalServerError + } + + message := buffer.Bytes() + err = s.deliverer.SubmitMessage(ctx, s.conn.Conn(), s.destinationUser, bytes.NewReader(message)) + if err != nil { + if errors.Is(err, model.ErrMissingFromHeader) { + return &smtpserver.SMTPError{ + Code: 550, + EnhancedCode: smtpserver.EnhancedCode{5, 7, 1}, + Message: "From header is required but missing", + } + } + s.logger.ErrorContext(ctx, "SubmitMessage", "error", err) + return errInternalServerError + } + return nil +}
A
internal/inbound/inbound_test.go
@@ -0,0 +1,64 @@
+package inbound + +import ( + "log/slog" + "os" + "testing" + + "postern/internal/db" + "postern/internal/model" +) + +func TestInboundRCPTTO(t *testing.T) { + pdb, err := os.CreateTemp(t.TempDir(), "postern.db") + if err != nil { + t.Fatal(err) + } + + d, err := db.OpenDB(&db.Config{DBPath: pdb.Name()}) + if err != nil { + t.Fatal(err) + } + + s := &smtpSession{ + db: d, + } + + rcptUser := "testuser" + + tx, err := d.GetWriteTx(t.Context()) + if err != nil { + t.Fatal(err) + } + _, err = db.InsertUser(t.Context(), tx, rcptUser, []byte{}) + if err != nil { + t.Fatal(err) + } + err = tx.Commit() + if err != nil { + t.Fatal(err) + } + + err = d.InsertUserAddress(t.Context(), rcptUser, "testuser@example.com") + if err != nil { + t.Fatal(err) + } + + s.authenticatedUser = &model.SMTPAuth{ + ID: 1, + Name: "smtpuser", + } + s.logger = slog.Default() + + // Test "+" addresses + err = s.Rcpt("testuser+tag+two@example.com", nil) + if err != nil { + t.Fatal(err) + } + + // provoke non existing user + err = s.Rcpt("notexist+tag+two@example.com", nil) + if err == nil { + t.Fatal("expected 550 error, got nil") + } +}
A
internal/model/dkim.go
@@ -0,0 +1,19 @@
+package model + +type DKIMAlgorithmType string + +var ( + // Verifiers MUST be able to validate signatures with keys ranging from 1024 bits to 4096 bits. + // https://datatracker.ietf.org/doc/html/rfc8301 + DKIMAlgorithmRSA2048 DKIMAlgorithmType = "rsa-2048" + DKIMAlgorithmRSA4096 DKIMAlgorithmType = "rsa-4096" + // https://datatracker.ietf.org/doc/html/rfc8463 + DKIMAlgorithmEd25519 DKIMAlgorithmType = "ed25519-sha256" +) + +type DKIM struct { + ID int + Domain, Selector, BlobAddress string + KeyType DKIMAlgorithmType + Enabled bool +}
A
internal/model/errors.go
@@ -0,0 +1,5 @@
+package model + +import "errors" + +var ErrMissingFromHeader = errors.New("missing From header")
A
internal/model/gatekeeper.go
@@ -0,0 +1,9 @@
+package model + +type GatekeeperDecision int + +const ( + GatekeeperUndecided GatekeeperDecision = iota + GatekeeperAllowed + GatekeeperDenied +)
A
internal/model/mailbox.go
@@ -0,0 +1,47 @@
+package model + +import "github.com/emersion/go-imap/v2" + +type DefaultMailbox struct { + Name MailboxName + SpecialUse imap.MailboxAttr +} + +type MailboxName string + +var ( + MailboxINBOX = MailboxName("INBOX") + MailboxGatekeeper = MailboxName("Gatekeeper") + MailboxArchive = MailboxName("Archive") + MailboxDrafts = MailboxName("Drafts") + MailboxJunk = MailboxName("Junk") + MailboxSent = MailboxName("Sent") + MailboxTrash = MailboxName("Trash") +) + +func DefaultMailboxes() []DefaultMailbox { + return []DefaultMailbox{ + {Name: MailboxINBOX}, + {Name: MailboxGatekeeper}, + {Name: MailboxArchive, SpecialUse: imap.MailboxAttrArchive}, + {Name: MailboxDrafts, SpecialUse: imap.MailboxAttrDrafts}, + {Name: MailboxJunk, SpecialUse: imap.MailboxAttrJunk}, + {Name: MailboxSent, SpecialUse: imap.MailboxAttrSent}, + {Name: MailboxTrash, SpecialUse: imap.MailboxAttrTrash}, + } +} + +type Mailbox struct { + ID int + Name string + SpecialUse *string + UIDValidity uint32 + UIDNext int + HighestModSeq int + + NumMessages uint32 + NumUnseen int + NumDeleted int + Size int64 + IsSubscribed bool +}
A
internal/model/message.go
@@ -0,0 +1,26 @@
+package model + +import ( + "context" + "database/sql" +) + +type Message struct { + UID uint32 + ModSeq uint32 + BlobHash string + InternalDate string + Flags []string + RFC822Size int64 + EnvelopeDate sql.NullString + EnvelopeSubject sql.NullString + EnvelopeFrom sql.NullString + EnvelopeSender sql.NullString + EnvelopeReplyTo sql.NullString + EnvelopeInReplyTo sql.NullString + EnvelopeTo sql.NullString + EnvelopeCc sql.NullString + EnvelopeBcc sql.NullString + EnvelopeMessageID sql.NullString + ServerSeq func(ctx context.Context) (uint32, error) // lazy evaluation +}
A
internal/model/user.go
@@ -0,0 +1,36 @@
+package model + +import ( + "golang.org/x/crypto/bcrypt" +) + +type User struct { + ID int + Name string + Addresses []string + password []byte +} + +func (u *User) SetPassword(pw []byte) { + u.password = pw +} + +// VerifyPassword checks a plaintext password against the stored sha256 hash. +func (u *User) VerifyPassword(password []byte) error { + return bcrypt.CompareHashAndPassword(u.password, password) +} + +type SMTPAuth struct { + ID int + Name string + password []byte +} + +func (u *SMTPAuth) SetPassword(pw []byte) { + u.password = pw +} + +// VerifyPassword checks a plaintext password against the stored sha256 hash. +func (u *SMTPAuth) VerifyPassword(password []byte) error { + return bcrypt.CompareHashAndPassword(u.password, password) +}
A
internal/persistence/blob.go
@@ -0,0 +1,207 @@
+package persistence + +import ( + "bufio" + "bytes" + "compress/gzip" + "crypto/rand" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "io" + "log/slog" + "net/mail" + "os" + "path/filepath" + + "github.com/minio/sio" + "golang.org/x/crypto/hkdf" +) + +// WriteBlob gzips and streams r into the nonce-addressed file and returns the hex-encoded nonce that identifies the blob. +// More information about encryption: https://github.com/minio/sio/blob/master/DARE.md +func (p *Persistence) WriteBlob(r io.Reader) (string, int64, error) { + // Generate a random nonce to derive an encryption key from the master key. + var nonce [32]byte + if _, err := io.ReadFull(rand.Reader, nonce[:]); err != nil { + return "", 0, fmt.Errorf("failed to read random data: %w", err) + } + + // Use the nonce as file name + nonceHex := hex.EncodeToString(nonce[:]) + dir := filepath.Join(p.blobRoot, nonceHex[:2]) + if err := os.MkdirAll(dir, 0755); err != nil { + return "", 0, fmt.Errorf("creating blob dir: %w", err) + } + finalPath := filepath.Join(dir, nonceHex) + + f, err := os.Create(finalPath) + if err != nil { + return "", 0, fmt.Errorf("creating file: %w", err) + } + + // Derive an encryption key from the master key and the nonce + var key [32]byte + kdf := hkdf.New(sha256.New, p.masterkey, nonce[:], nil) + if _, err = io.ReadFull(kdf, key[:]); err != nil { + return "", 0, fmt.Errorf("failed to derive encryption key: %w", err) + } + + // Create encryption writer + encrypted, err := sio.EncryptWriter(f, sio.Config{Key: key[:]}) + if err != nil { + return "", 0, fmt.Errorf("failed to create encrypted writer: %w", err) + } + + gzipWriter, err := gzip.NewWriterLevel(encrypted, gzip.BestCompression) + if err != nil { + return "", 0, fmt.Errorf("failed to create gzip writer: %w", err) + } + + size, err := io.Copy(gzipWriter, r) + if err != nil { + return "", 0, fmt.Errorf("copying data: %w", err) + } + + if err := gzipWriter.Close(); err != nil { + return "", 0, fmt.Errorf("closing gzip writer: %w", err) + } + + if err := encrypted.Close(); err != nil { + return "", 0, fmt.Errorf("closing encryption writer: %w", err) + } + + return nonceHex, size, nil +} + +// WriteBlobMessage parses the RFC 5322 headers from r, then gzips and streams the entire message to disk. +func (p *Persistence) WriteBlobMessage(r io.Reader) (string, int64, *mail.Message, error) { + msg, fullReader, err := parseMailHeader(r) + if err != nil { + return "", 0, nil, fmt.Errorf("parsing message header: %w", err) + } + + nonceHex, size, err := p.WriteBlob(fullReader) + if err != nil { + return "", 0, nil, err + } + + return nonceHex, size, msg, nil +} + +// parseMailHeader extracts RFC 5322 mail headers line-by-line without reading the whole body into memory. +func parseMailHeader(r io.Reader) (*mail.Message, io.Reader, error) { + var headerBuf bytes.Buffer + br := bufio.NewReader(r) + + for { + line, err := br.ReadBytes('\n') + headerBuf.Write(line) + + // Blank line (\r\n or \n) marks the end of headers + if bytes.Equal(line, []byte("\r\n")) || bytes.Equal(line, []byte("\n")) { + break + } + + if err != nil { + if errors.Is(err, io.EOF) { + break // End of message with headers only + } + return nil, nil, err + } + } + + // Parse header structure from the captured header bytes + msg, err := mail.ReadMessage(bytes.NewReader(headerBuf.Bytes())) + if err != nil { + return nil, nil, err + } + + // Reconstruct the exact stream: header bytes first, then remaining body bytes in br + fullReader := io.MultiReader(&headerBuf, br) + return msg, fullReader, nil +} + +// BlobReader decrypts and unzips the stored blob +func (p *Persistence) BlobReader(nonceIdentifier string) (io.ReadCloser, error) { + if len(nonceIdentifier) < 8 { + return nil, fmt.Errorf("invalid nonce identifier") + } + + dir := filepath.Join(p.blobRoot, nonceIdentifier[:2]) + finalPath := filepath.Join(dir, nonceIdentifier) + + nonce, err := hex.DecodeString(nonceIdentifier) + if err != nil { + return nil, fmt.Errorf("invalid nonce identifier: %w", err) + } + + // derive an encryption key from the master key and the nonce + var key [32]byte + kdf := hkdf.New(sha256.New, p.masterkey, nonce, nil) + if _, err = io.ReadFull(kdf, key[:]); err != nil { + return nil, fmt.Errorf("failed to derive encryption key: %w", err) + } + + f, err := os.Open(finalPath) // leave Close() to sio.DecryptReader + if err != nil { + return nil, fmt.Errorf("opening file: %w", err) + } + + decrypter, err := sio.DecryptReader(f, sio.Config{Key: key[:]}) + if err != nil { + return nil, fmt.Errorf("failed to create decrypt reader: %w", err) + } + + gzipReader, err := gzip.NewReader(decrypter) + if err != nil { + return nil, fmt.Errorf("failed to create gzip reader: %w", err) + } + + slog.Info("reading blob", "path", finalPath) + // Return a wrapper that closes both the gzip reader and the underlying file + return &blobReadCloser{ + Reader: gzipReader, + closeFunc: func() error { + gzErr := gzipReader.Close() + fErr := f.Close() + if gzErr != nil { + return gzErr + } + return fErr + }, + }, nil +} + +type blobReadCloser struct { + io.Reader + closeFunc func() error +} + +func (b *blobReadCloser) Close() error { + return b.closeFunc() +} + +func (p *Persistence) RemoveBlob(nonceIdentifier string) error { + if len(nonceIdentifier) < 8 { + return fmt.Errorf("invalid nonce identifier") + } + + dir := filepath.Join(p.blobRoot, nonceIdentifier[:2]) + finalPath := filepath.Join(dir, nonceIdentifier) + + err := os.Remove(finalPath) + if err != nil { + return fmt.Errorf("deleting blob: %w", err) + } + slog.Info("deleted blob", "path", finalPath) + + err = os.Remove(dir) + if err == nil { + // Only logs if the directory was actually empty and successfully deleted + slog.Info("deleted empty blob directory", "dir", dir) + } + + return nil +}
A
internal/persistence/persistence.go
@@ -0,0 +1,87 @@
+package persistence + +import ( + "errors" + "fmt" + "os" +) + +type Config struct { + BlobRoot string + MasterkeyFile string +} +type Persistence struct { + blobRoot string + masterkey []byte +} + +func New(config Config) (*Persistence, error) { + if config.MasterkeyFile == "" { + return nil, errors.New("no masterkey file specified") + } + + blobRoot := "./data/blobs" + if config.BlobRoot != "" { + blobRoot = config.BlobRoot + } + + if err := ensureBlobsDir(blobRoot); err != nil { + return nil, err + } + + // the master key used to derive encryption keys + masterkey, err := os.ReadFile(config.MasterkeyFile) + if err != nil { + return nil, fmt.Errorf("cannot read postern masterkey: %w", err) + } + + return &Persistence{ + blobRoot: blobRoot, + masterkey: masterkey, + }, nil +} + +func (p *Persistence) BlobRoot() string { + return p.blobRoot +} + +func ensureBlobsDir(path string) error { + if err := os.MkdirAll(path, 0700); err != nil { + return fmt.Errorf("creating blobs directory %s: %w", path, err) + } + return nil +} + +// CheckDirWritable ensures that the persistence layer can write to the given directory +func CheckDirWritable(dir string) error { + if dir == "" { + return errors.New("no path given") + } + info, err := os.Stat(dir) + if err != nil { + if os.IsNotExist(err) { + return fmt.Errorf("directory does not exist: %s", dir) + } + return fmt.Errorf("failed to stat directory: %w", err) + } + + // Ensure it's a directory + if !info.IsDir() { + return fmt.Errorf("path is not a directory: %s", dir) + } + + // Test writability by creating a temporary file + writeTest, err := os.CreateTemp(dir, ".write_test") + if err != nil { + return fmt.Errorf("directory is not writable: %w", err) + } + _ = writeTest.Close() + + // Clean up the test file + err = os.Remove(writeTest.Name()) + if err != nil { + return fmt.Errorf("directory is writable but not deletable: %w", err) + } + + return nil +}
A
internal/persistence/persistence_test.go
@@ -0,0 +1,10 @@
+package persistence + +import "testing" + +func TestCheckDirWritable(t *testing.T) { + err := CheckDirWritable("/tmp") + if err != nil { + t.Fatal(err) + } +}
A
internal/policy/parse.go
@@ -0,0 +1,391 @@
+package policy + +import ( + "fmt" + "log" + "net" + "net/mail" + "net/netip" + "os" + + "postern/internal/db" + + "go.starlark.net/starlark" + "go.starlark.net/starlarkstruct" + "go.starlark.net/syntax" +) + +type RelayConfig struct { + Host string + Port int + Username string + Password string +} + +type Engine struct { + Listeners []ListenerConfig + UsersModule *UsersModule + globals starlark.StringDict + thread *starlark.Thread +} + +type ListenerConfig struct { + Address netip.AddrPort + Role string + TLSCertFile string + TLSKeyFile string +} + +type Address struct { + Address, Name, Raw string +} + +func (a Address) String() string { return a.Raw } +func (a Address) Type() string { return "Address" } +func (a Address) Freeze() {} +func (a Address) Truth() starlark.Bool { return true } +func (a Address) Hash() (uint32, error) { return 0, nil } + +func (a Address) Attr(name string) (starlark.Value, error) { + switch name { + case "address": + return starlark.String(a.Address), nil + case "name": + return starlark.String(a.Name), nil + case "raw": + return starlark.String(a.Raw), nil + } + return nil, nil +} + +func (a Address) AttrNames() []string { + return []string{"address", "name", "raw"} +} + +type MessageContext struct { + HeaderFrom Address + HeaderTo Address + Subject string + Headers mail.Header + Size int64 +} + +func (m MessageContext) String() string { return "<MessageContext>" } +func (m MessageContext) Type() string { return "MessageContext" } +func (m MessageContext) Freeze() { + m.HeaderFrom.Freeze() + m.HeaderTo.Freeze() +} +func (m MessageContext) Truth() starlark.Bool { return true } +func (m MessageContext) Hash() (uint32, error) { return 0, nil } + +func (m MessageContext) Attr(name string) (starlark.Value, error) { + switch name { + case "header_from": + return m.HeaderFrom, nil + case "header_to": + return m.HeaderTo, nil + case "subject": + return starlark.String(m.Subject), nil + case "size": + return starlark.MakeInt64(m.Size), nil + } + return nil, nil +} + +func (m MessageContext) AttrNames() []string { + return []string{"header_from", "subject", "size"} +} + +// MDA implements starlark.Value and starlark.HasAttrs +// so it can be exposed as the `mda` global object in the script. +type MDA struct { + parsedListeners []ListenerConfig +} + +func (m *MDA) String() string { return "<module mda>" } +func (m *MDA) Type() string { return "module" } +func (m *MDA) Freeze() {} +func (m *MDA) Truth() starlark.Bool { return true } +func (m *MDA) Hash() (uint32, error) { return 0, nil } + +// Attr resolves mda.listen, mda.accept, mda.reject, mda.deliver_to, and mda.quarantine when called in Starlark +func (m *MDA) Attr(name string) (starlark.Value, error) { + switch name { + case "listen": + return starlark.NewBuiltin("listen", m.mdaListen), nil + } + return nil, nil +} +func (m *MDA) AttrNames() []string { + return []string{"listen"} +} + +type Conn struct { + RemoteAddr starlark.String + ClientIP starlark.String + Meta *starlarkstruct.Struct +} + +func (c Conn) String() string { return "<Conn>" } +func (c Conn) Type() string { return "Conn" } +func (c Conn) Freeze() { c.Meta.Freeze() } +func (c Conn) Truth() starlark.Bool { return true } +func (c Conn) Hash() (uint32, error) { return 0, nil } + +func (c Conn) Attr(name string) (starlark.Value, error) { + switch name { + case "remote_addr": + return c.RemoteAddr, nil + case "client_ip": + return c.ClientIP, nil + case "meta": + return c.Meta, nil + } + return nil, nil +} + +func (c Conn) AttrNames() []string { + return []string{"remote_addr", "client_ip", "meta"} +} + +func ConnToStarlark(conn net.Conn) Conn { + remoteAddr := conn.RemoteAddr().String() + var clientIP string + if tcpAddr, ok := conn.RemoteAddr().(*net.TCPAddr); ok { + clientIP = tcpAddr.IP.String() + } else { + clientIP = remoteAddr + } + + return Conn{ + RemoteAddr: starlark.String(remoteAddr), + ClientIP: starlark.String(clientIP), + Meta: starlarkstruct.FromStringDict(starlark.None, starlark.StringDict{}), + } +} + +// mdaListen handles mda.listen(address="...", role="...") +func (m *MDA) mdaListen(_ *starlark.Thread, b *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) { + var address, role, tlsCertFile, tlsKeyFile string + + // starlark.UnpackArgs maps Starlark arguments to Go variables safely + err := starlark.UnpackArgs( + b.Name(), + args, + kwargs, + "address", &address, + "role", &role, + "tls_cert_file", &tlsCertFile, + "tls_key_file", &tlsKeyFile) + if err != nil { + return nil, err + } + + addrPort, err := netip.ParseAddrPort(address) + if err != nil { + return nil, err + } + + // Save to our Go state + m.parsedListeners = append(m.parsedListeners, ListenerConfig{ + Address: addrPort, + Role: role, + TLSCertFile: tlsCertFile, + TLSKeyFile: tlsKeyFile, + }) + + return starlark.None, nil +} + +func NewEngine(policyFile string, database *db.DB) (Engine, error) { + script, err := os.ReadFile(policyFile) + if err != nil { + log.Fatal(err) + } + + // Create an 'os' module containing our getenv function + osModule := &starlarkstruct.Module{ + Name: "os", + Members: starlark.StringDict{ + "getenv": starlark.NewBuiltin("getenv", starlarkGetenv), + }, + } + + // Create the users module + usersModule := NewUsersModule(database) + + // Expose the "os", "mda", and "users" modules to the Starlark global environment + m := &MDA{ + parsedListeners: make([]ListenerConfig, 0), + } + predeclared := starlark.StringDict{ + "mda": m, + "os": osModule, + "users": usersModule, + } + + // Create a Starlark thread. We customize the Print function + // so the Starlark `print()` built-in logs cleanly in Go. + thread := &starlark.Thread{ + Name: "mda-main", + Print: func(_ *starlark.Thread, msg string) { + fmt.Printf("[init.star]: %s\n", msg) + }, + } + + // 1. Evaluate the script + globals, err := starlark.ExecFileOptions(&syntax.FileOptions{}, thread, "policy.star", script, predeclared) + if err != nil { + return Engine{}, err + } + + // 2. Call the init() function + initVal, ok := globals["init"] + if !ok { + return Engine{}, fmt.Errorf("init not found in policy") + } + + _, err = starlark.Call(thread, initVal, nil, nil) + if err != nil { + return Engine{}, err + } + + engine := Engine{ + Listeners: m.parsedListeners, + UsersModule: usersModule, + globals: globals, + thread: thread, + } + return engine, err +} + +func (e *Engine) OnMessageSubmit(user User, msg MessageContext) (RelayConfig, error) { + hookVal, ok := e.globals["on_message_submit"] + if !ok { + return RelayConfig{}, nil + } + + result, err := starlark.Call(e.thread, hookVal, starlark.Tuple{user, msg}, nil) + if err != nil { + return RelayConfig{}, err + } + + if result == starlark.None { + return RelayConfig{}, nil + } + + dict, ok := result.(*starlark.Dict) + if !ok { + return RelayConfig{}, fmt.Errorf("on_message_submit must return a dict, got %s", result.Type()) + } + + relay := RelayConfig{ + Host: dictString(dict, "host"), + Port: dictInt(dict, "port"), + Username: dictString(dict, "username"), + Password: dictString(dict, "password"), + } + return relay, nil +} + +func dictString(dict *starlark.Dict, key string) string { + v, _, err := dict.Get(starlark.String(key)) + if err != nil || v == starlark.None { + return "" + } + if s, ok := v.(starlark.String); ok { + return string(s) + } + return "" +} + +func dictInt(dict *starlark.Dict, key string) int { + v, _, err := dict.Get(starlark.String(key)) + if err != nil || v == starlark.None { + return 0 + } + if i, ok := v.(starlark.Int); ok { + if n, ok := i.Int64(); ok { + return int(n) + } + } + return 0 +} + +func (e *Engine) OnMessageDeliver(conn Conn, user User, msg MessageContext) (string, error) { + defaultMailbox := "INBOX" + hookVal, ok := e.globals["on_message_deliver"] + if !ok { + return defaultMailbox, nil + } + + result, err := starlark.Call(e.thread, hookVal, starlark.Tuple{conn, user, msg}, nil) + if err != nil { + return defaultMailbox, err + } + + mailbox := defaultMailbox + if result != starlark.None { + if s, ok := result.(starlark.String); ok { + mailbox = string(s) + } + } + + return mailbox, nil +} + +func (e *Engine) OnMessageMoved(user User, msg MessageContext, sourceFolder, destFolder string) error { + hookVal, ok := e.globals["on_message_moved"] + if !ok { + return nil + } + + _, err := starlark.Call(e.thread, hookVal, starlark.Tuple{user, msg, starlark.String(sourceFolder), starlark.String(destFolder)}, nil) + return err +} + +// MessageContextFromEnvelope builds a policy.MessageContext from the envelope +// fields of a stored message. The header_from address is parsed into an +// Address; if parsing fails the raw string is preserved. +func MessageContextFromEnvelope(from, subject string, size int64) MessageContext { + fromAddr, err := ParseAddress(from) + if err != nil || from == "" { + fromAddr = Address{Raw: from, Address: from} + } + return MessageContext{ + HeaderFrom: fromAddr, + Subject: subject, + Size: size, + } +} + +func ParseAddress(email string) (Address, error) { + a, err := mail.ParseAddress(email) + if err != nil { + return Address{}, err + } + + addr := Address{ + Raw: email, + Address: a.Address, + Name: a.Name, + } + + return addr, nil +} + +// starlarkGetenv implements Python's os.getenv("KEY", "default") +func starlarkGetenv(_ *starlark.Thread, b *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) { + var key string + var def starlark.Value = starlark.None + + if err := starlark.UnpackArgs(b.Name(), args, kwargs, "key", &key, "default?", &def); err != nil { + return nil, err + } + + if val, exists := os.LookupEnv(key); exists { + return starlark.String(val), nil + } + return def, nil +}
A
internal/policy/user.go
@@ -0,0 +1,201 @@
+package policy + +import ( + "context" + "database/sql" + "errors" + "log/slog" + + "postern/internal/db" + "postern/internal/model" + + "go.starlark.net/starlark" +) + +type UsersModule struct { + db *db.DB +} + +func NewUsersModule(db *db.DB) *UsersModule { + return &UsersModule{ + db: db, + } +} + +func (u *UsersModule) String() string { return "<module users>" } +func (u *UsersModule) Type() string { return "module" } +func (u *UsersModule) Freeze() {} +func (u *UsersModule) Truth() starlark.Bool { return true } +func (u *UsersModule) Hash() (uint32, error) { + return 0, nil +} + +func (u *UsersModule) Attr(name string) (starlark.Value, error) { + switch name { + case "get": + return starlark.NewBuiltin("get", u.usersGet), nil + case "get_by_address": + return starlark.NewBuiltin("get_by_address", u.usersGetByAddress), nil + } + return nil, nil +} + +func (u *UsersModule) AttrNames() []string { + return []string{"get", "get_by_address"} +} + +type User struct { + Name starlark.String + Disabled starlark.Bool + Addresses starlark.Tuple + gatekeeper *UserGatekeeper + user model.User +} + +func (u User) String() string { + return "<User " + u.Name.GoString() + ">" +} + +func (u User) Type() string { return "User" } +func (u User) Freeze() {} +func (u User) Truth() starlark.Bool { return true } +func (u User) Hash() (uint32, error) { + return 0, nil +} + +func (u User) Attr(name string) (starlark.Value, error) { + switch name { + case "name": + return u.Name, nil + case "is_disabled": + return u.Disabled, nil + case "addresses": + return u.Addresses, nil + case "gatekeeper": + if u.gatekeeper == nil { + return starlark.None, nil + } + return u.gatekeeper, nil + } + return nil, nil +} + +func (u User) AttrNames() []string { + return []string{"name", "is_disabled", "addresses", "gatekeeper"} +} + +func (u *UsersModule) usersGet(_ *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, _ []starlark.Tuple) (starlark.Value, error) { + var username starlark.String + if err := starlark.UnpackPositionalArgs("get", args, nil, 1, &username); err != nil { + return nil, err + } + + user, err := u.db.GetUser(context.Background(), username.GoString()) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + return starlark.None, nil + } + return nil, err + } + + return UserToStarlark(user, u.db), nil +} + +func (u *UsersModule) usersGetByAddress(_ *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, _ []starlark.Tuple) (starlark.Value, error) { + var address starlark.String + if err := starlark.UnpackPositionalArgs("get_by_address", args, nil, 1, &address); err != nil { + return nil, err + } + + user, err := u.db.GetUserForAddress(context.Background(), address.GoString()) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + return starlark.None, nil + } + return nil, err + } + + return UserToStarlark(user, u.db), nil +} + +func UserToStarlark(user model.User, database *db.DB) User { + addresses := make([]starlark.Value, len(user.Addresses)) + for i, addr := range user.Addresses { + addresses[i] = starlark.String(addr) + } + + return User{ + Name: starlark.String(user.Name), + Disabled: false, + Addresses: addresses, + gatekeeper: &UserGatekeeper{db: database, user: user}, + user: user, + } +} + +// UserGatekeeper is the per-User gatekeeper sub-module exposed as +// user.gatekeeper in Starlark. +type UserGatekeeper struct { + db *db.DB + user model.User +} + +func (g *UserGatekeeper) String() string { return "<UserGatekeeper>" } +func (g *UserGatekeeper) Type() string { return "UserGatekeeper" } +func (g *UserGatekeeper) Freeze() {} +func (g *UserGatekeeper) Truth() starlark.Bool { return true } +func (g *UserGatekeeper) Hash() (uint32, error) { + return 0, nil +} + +func (g *UserGatekeeper) Attr(name string) (starlark.Value, error) { + switch name { + case "get_decision": + return starlark.NewBuiltin("gatekeeper.get_decision", g.getDecision), nil + case "route_to": + return starlark.NewBuiltin("gatekeeper.route_to", g.routeTo), nil + } + return nil, nil +} + +func (g *UserGatekeeper) AttrNames() []string { + return []string{"get_decision", "route_to"} +} + +func (g *UserGatekeeper) getDecision(_ *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, _ []starlark.Tuple) (starlark.Value, error) { + var msg MessageContext + if err := starlark.UnpackPositionalArgs("get_decision", args, nil, 1, &msg); err != nil { + return nil, err + } + + fromAddress := msg.HeaderFrom.Address + slog.Info("retrieving gatekeeper decision", "module", "get_decision", "user", g.user.Name, "from_address", fromAddress) + destination, err := g.db.GetGatekeepDecision(context.Background(), g.user.ID, fromAddress) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + slog.Info("gatekeeper decision not found", "module", "get_decision") + return starlark.None, nil + } + return nil, err + } + + slog.Info("got gatekeeper decision", "module", "get_decision", "destination", destination) + return starlark.String(destination), nil +} + +func (g *UserGatekeeper) routeTo(_ *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, _ []starlark.Tuple) (starlark.Value, error) { + var from Address + var destination starlark.String + if err := starlark.UnpackPositionalArgs("route_to", args, nil, 2, &from, &destination); err != nil { + return nil, err + } + + err := g.db.SetGatekeeperDecision(context.Background(), g.user.ID, from.Address, destination.GoString()) + if err != nil { + return nil, err + } + + slog.Info("stored gatekeeper route", "user", g.user.ID, "from_address", from.Address, "destination", destination.GoString()) + + return starlark.None, nil +}
A
internal/submission/client.go
@@ -0,0 +1,41 @@
+package submission + +import ( + "crypto/tls" + "errors" + "io" + "net" + "strconv" + + "postern/internal/policy" + + "github.com/emersion/go-sasl" + "github.com/emersion/go-smtp" +) + +func shipMessage(from string, to []string, r io.Reader, relayConfig policy.RelayConfig) error { + if relayConfig.Host == "" { + return errors.New("no relay configured") + } + conn, err := net.Dial("tcp", net.JoinHostPort(relayConfig.Host, strconv.Itoa(relayConfig.Port))) + if err != nil { + return err + } + + c, err := smtp.NewClientStartTLS(conn, &tls.Config{ + ServerName: relayConfig.Host, + }) + if err != nil { + return err + } + defer func(c *smtp.Client) { + _ = c.Close() + }(c) + + err = c.Auth(sasl.NewPlainClient("", relayConfig.Username, relayConfig.Password)) + if err != nil { + return err + } + + return c.SendMail(from, to, r) +}
A
internal/submission/error.go
@@ -0,0 +1,13 @@
+package submission + +import ( + smtpserver "github.com/emersion/go-smtp" +) + +var ( + errInternalServerError = &smtpserver.SMTPError{ + Code: 451, + EnhancedCode: smtpserver.EnhancedCode{4, 3, 0}, + Message: "Internal Server Error", + } +)
A
internal/submission/submission.go
@@ -0,0 +1,249 @@
+package submission + +import ( + "bytes" + "context" + "crypto/tls" + "database/sql" + "errors" + "io" + "log/slog" + "sync" + + "postern/internal/db" + "postern/internal/dkim" + "postern/internal/model" + "postern/internal/policy" + + "github.com/emersion/go-sasl" + smtpserver "github.com/emersion/go-smtp" + "github.com/google/uuid" +) + +type Config struct { + DB *db.DB + DKIM *dkim.DKIM + Listen string + TLS *tls.Config + PolicyEngine *policy.Engine + IsLocalSubmission bool // If set, will disable TLS and Auth and accept all local mails for delivery. +} +type Server struct { + db *db.DB + dkim *dkim.DKIM + listen string + tlsConfig *tls.Config + policyEngine *policy.Engine + + mu sync.Mutex + smtpSrv *smtpserver.Server + status string + isLocalSubmission bool +} + +func NewServer(config *Config) (*Server, error) { + return &Server{ + db: config.DB, + dkim: config.DKIM, + listen: config.Listen, + tlsConfig: config.TLS, + policyEngine: config.PolicyEngine, + isLocalSubmission: config.IsLocalSubmission, + }, nil +} + +func (i *Server) Start() error { + be := &backend{ + db: i.db, + dkim: i.dkim, + isLocalSubmission: i.isLocalSubmission, + policyEngine: i.policyEngine, + } + s := smtpserver.NewServer(be) + + s.Addr = i.listen + s.MaxMessageBytes = 20_000_000 + + i.mu.Lock() + i.smtpSrv = s + i.status = "running" + i.mu.Unlock() + + if i.isLocalSubmission { + err := s.ListenAndServe() + if err != nil { + return err + } + } else { + s.TLSConfig = i.tlsConfig + err := s.ListenAndServeTLS() + if err != nil { + return err + } + } + + return nil +} + +func (i *Server) Stop() error { + i.mu.Lock() + defer i.mu.Unlock() + i.status = "stopped" + if i.smtpSrv != nil { + return i.smtpSrv.Close() + } + return nil +} + +func (i *Server) Status() string { + i.mu.Lock() + defer i.mu.Unlock() + return i.status +} + +type smtpSession struct { + db *db.DB + dkim *dkim.DKIM + conn *smtpserver.Conn + authenticatedUser model.User + sessionID string + mailFrom string + rcptTo []string + logger *slog.Logger + isLocalSubmission bool + policyEngine *policy.Engine +} + +// The backend implements SMTP server methods. +type backend struct { + db *db.DB + dkim *dkim.DKIM + isLocalSubmission bool + policyEngine *policy.Engine +} + +// NewSession is called after client greeting (EHLO, HELO). +func (bkd *backend) NewSession(c *smtpserver.Conn) (smtpserver.Session, error) { + sessionID := uuid.NewString() + l := slog.With("component", "submission", "remote_ip", c.Conn().RemoteAddr().String(), "session_id", sessionID) + l.Info("new session") + return &smtpSession{ + db: bkd.db, + dkim: bkd.dkim, + conn: c, + authenticatedUser: model.User{}, + sessionID: sessionID, + logger: l, + rcptTo: make([]string, 0), + isLocalSubmission: bkd.isLocalSubmission, + policyEngine: bkd.policyEngine, + }, nil +} + +// AuthMechanisms returns a slice of available auth mechanisms +func (s *smtpSession) AuthMechanisms() []string { + return []string{sasl.Plain} +} + +// Auth is the handler for supported authenticators. +func (s *smtpSession) Auth(_ string) (sasl.Server, error) { + return sasl.NewPlainServer(func(identity, username, password string) error { + u, err := s.db.GetUser(context.Background(), username) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + s.logger.Info("user not found") + } else { + s.logger.Error("getting user", "error", err) + } + return smtpserver.ErrAuthFailed + } + + err = u.VerifyPassword([]byte(password)) + if err != nil { + slog.Info("invalid password", "username", username) + return smtpserver.ErrAuthFailed + } + + s.authenticatedUser = u + s.logger.Info("authenticated", "user_id", u.ID, "username", username) + return nil + }), nil +} + +func (s *smtpSession) Reset() { + s.mailFrom = "" + s.rcptTo = make([]string, 0) +} + +func (s *smtpSession) Logout() error { + s.authenticatedUser = model.User{} + return nil +} + +func (s *smtpSession) Mail(from string, _ *smtpserver.MailOptions) error { + ctx := context.Background() + s.logger.InfoContext(ctx, "MAIL FROM", "from", from) + + if !s.isLocalSubmission && s.authenticatedUser.ID == 0 { + s.logger.InfoContext(ctx, "no authenticated user") + return smtpserver.ErrAuthFailed + } + + s.mailFrom = from + return nil +} + +func (s *smtpSession) Rcpt(to string, _ *smtpserver.RcptOptions) error { + ctx := context.Background() + s.logger.InfoContext(ctx, "RCPT TO", "to", to) + + if !s.isLocalSubmission && s.authenticatedUser.ID == 0 { + s.logger.InfoContext(ctx, "no authenticated user") + return smtpserver.ErrAuthFailed + } + + s.rcptTo = append(s.rcptTo, to) + return nil +} + +func (s *smtpSession) Data(r io.Reader) error { + ctx := context.Background() + s.logger.InfoContext(ctx, "DATA") + + if !s.isLocalSubmission && s.authenticatedUser.ID == 0 { + s.logger.InfoContext(ctx, "no authenticated user") + return smtpserver.ErrAuthFailed + } + + // TODO Check if this mail is for us or for the relay + + var buffer bytes.Buffer + _, err := io.Copy(&buffer, r) + if err != nil { + s.logger.ErrorContext(ctx, "Copy", "error", err) + return errInternalServerError + } + + message := buffer.Bytes() + + signed, err := s.dkim.Sign(ctx, message) + if err != nil { + s.logger.ErrorContext(ctx, "dkim sign", "error", err) + return errInternalServerError + } + + // Call policy hook to determine relay config + msgCtx := policy.MessageContextFromEnvelope(s.mailFrom, "", int64(len(signed))) + user := policy.UserToStarlark(s.authenticatedUser, s.db) + relayConfig, err := s.policyEngine.OnMessageSubmit(user, msgCtx) + if err != nil { + s.logger.ErrorContext(ctx, "policy on_message_submit", "error", err) + return errInternalServerError + } + + err = shipMessage(s.mailFrom, s.rcptTo, bytes.NewReader(signed), relayConfig) + if err != nil { + s.logger.ErrorContext(ctx, "ship message", "error", err) + } + return err +}
A
main.go
@@ -0,0 +1,175 @@
+package main + +import ( + "cmp" + "crypto/tls" + "log" + "log/slog" + "os" + "path" + + "postern/internal/cli" + "postern/internal/db" + "postern/internal/dkim" + "postern/internal/imap" + "postern/internal/inbound" + "postern/internal/persistence" + "postern/internal/policy" + "postern/internal/submission" +) + +func main() { + posternPersistenceDir := cmp.Or(os.Getenv("POSTERN_PERSISTENCE_DIR"), os.Getenv("STATE_DIRECTORY")) + err := persistence.CheckDirWritable(posternPersistenceDir) + if err != nil { + log.Fatal("POSTERN_PERSISTENCE_DIR: ", err) + } + slog.Info("using postern persistence", "dir", posternPersistenceDir) + + persistenceMasterkey := os.Getenv("POSTERN_MASTERKEY_FILE") + if persistenceMasterkey == "" { + persistenceMasterkey = path.Join(os.Getenv("CREDENTIALS_DIRECTORY"), "masterkey") + } + p, err := persistence.New(persistence.Config{ + BlobRoot: path.Join(posternPersistenceDir, "blobs"), + MasterkeyFile: persistenceMasterkey, + }) + if err != nil { + log.Fatal(err) + } + + posternDB, err := db.OpenDB(&db.Config{ + DBPath: path.Join(posternPersistenceDir, "meta.db"), + }) + if err != nil { + log.Fatal(err) + } + defer func(db *db.DB) { + if db == nil { + return + } + err := db.Close() + if err != nil { + log.Fatal(err) + } + }(posternDB) + + engine, err := policy.NewEngine(path.Join(os.Getenv("CONFIGURATION_DIRECTORY"), "policy.star"), posternDB) + if err != nil { + log.Fatal(err) + } + + dkimSvc := dkim.New(&dkim.Config{ + DB: posternDB, + Persistence: p, + }) + + listenAddrs := make(map[string]policy.ListenerConfig) + for _, l := range engine.Listeners { + listenAddrs[l.Role] = l + } + + services := make(map[string]cli.Service) + + if listener, ok := listenAddrs["imap"]; ok { + cert, err := tls.LoadX509KeyPair(listener.TLSCertFile, listener.TLSKeyFile) + if err != nil { + log.Fatal(err) + } + + tlsConfig := &tls.Config{ + Certificates: []tls.Certificate{cert}, + } + + imapServer, err := imap.NewServer(&imap.Config{ + DB: posternDB, + Listen: listener.Address.String(), + Persistence: p, + TLSConfig: tlsConfig, + PolicyEngine: &engine, + }) + if err != nil { + log.Fatal(err) + } + services["imap"] = imapServer + } + + if listener, ok := listenAddrs["smtp_inbound"]; ok { + cert, err := tls.LoadX509KeyPair(listener.TLSCertFile, listener.TLSKeyFile) + if err != nil { + log.Fatal(err) + } + + tlsConfig := &tls.Config{ + Certificates: []tls.Certificate{cert}, + } + + receptionServer, err := inbound.NewServer(&inbound.Config{ + DB: posternDB, + Deliverer: services["imap"].(inbound.MessageDeliverer), + Listen: listener.Address.String(), + TLS: tlsConfig, + PolicyEngine: &engine, + }) + if err != nil { + log.Fatal(err) + } + services["smtp_inbound"] = receptionServer + } + + if listener, ok := listenAddrs["smtp_submission"]; ok { + cert, err := tls.LoadX509KeyPair(listener.TLSCertFile, listener.TLSKeyFile) + if err != nil { + log.Fatal(err) + } + + tlsConfig := &tls.Config{ + Certificates: []tls.Certificate{cert}, + } + + submissionServer, err := submission.NewServer(&submission.Config{ + DB: posternDB, + DKIM: dkimSvc, + Listen: listener.Address.String(), + TLS: tlsConfig, + PolicyEngine: &engine, + }) + if err != nil { + log.Fatal(err) + } + services["smtp_submission"] = submissionServer + } + + if listener, ok := listenAddrs["smtp_local_submission"]; ok { + // Check that address is loopback. Other addresses are not allowed for security reasons. + if !listener.Address.Addr().IsLoopback() { + log.Fatal("smtp_local_submission address is not a loopback address") + } + + submissionServer, err := submission.NewServer(&submission.Config{ + DB: posternDB, + DKIM: dkimSvc, + Listen: listener.Address.String(), + PolicyEngine: &engine, + IsLocalSubmission: true, + }) + if err != nil { + log.Fatal(err) + } + services["smtp_local_submission"] = submissionServer + } + + posternCLI, err := cli.New(&cli.Config{ + DB: posternDB, + Persistence: p, + DKIM: dkimSvc, + Services: services, + }) + if err != nil { + log.Fatal(err) + } + + if err := posternCLI.Execute(); err != nil { + os.Exit(1) + } +}
A
policy.star
@@ -0,0 +1,62 @@
+def init(): + # Same TLS cert for all handlers + cert_file = "./localhost.pem" + key_file = "./localhost-key.pem" + + # SMTP Inbound + reception_addr = os.getenv("RECEPTION_ADDR", "0.0.0.0:10025") + mda.listen( + address = reception_addr, + role = "smtp_inbound", + tls_cert_file = cert_file, + tls_key_file = key_file, + ) + + # SMTP Submission + submission_addr = os.getenv("SUBMISSION_ADDR", "0.0.0.0:465") + mda.listen( + address = submission_addr, + role = "smtp_submission", + tls_cert_file = cert_file, + tls_key_file = key_file, + ) + + # SMTP Local Submission + mda.listen( + address = "127.0.0.1:12325", + role = "smtp_local_submission", + tls_cert_file = "", + tls_key_file = "", + ) + + # IMAP + imap_addr = os.getenv("IMAP_ADDR", "0.0.0.0:993") + mda.listen( + address = imap_addr, + role = "imap", + tls_cert_file = cert_file, + tls_key_file = key_file, + ) + +def on_message_deliver(conn, user, msg): + if msg.header_to.address.startswith("extra-address"): + return "extra" + dec = user.gatekeeper.get_decision(msg) + if not dec: + return "Gatekeeper" + return dec + +def on_message_moved(user, msg, source_folder, dest_folder): + if source_folder == "Gatekeeper" and dest_folder in ["Junk", "Trash"]: + user.gatekeeper.route_to(msg.header_from, "Junk") + return + if source_folder == "Gatekeeper": + user.gatekeeper.route_to(msg.header_from, "INBOX") + +def on_message_submit(user, msg): + return { + "host": "relay.p25.dev", + "port": 2587, + "username": os.getenv("RELAY_USERNAME", ""), + "password": os.getenv("RELAY_PASSWORD", ""), + }
A
postern.service
@@ -0,0 +1,32 @@
+[Unit] +Description=postern email server +After=network.target +Wants=network.target + +[Service] +Type=exec +DynamicUser=yes +StateDirectory=postern +ExecStart=/usr/local/bin/postern serve +LoadCredentialEncrypted=masterkey +ConfigurationDirectory=postern + +# Security hardening +NoNewPrivileges=yes +PrivateDevices=yes +ProtectKernelTunables=yes +ProtectKernelModules=yes +ProtectControlGroups=yes +RestrictAddressFamilies=AF_INET AF_INET6 +RestrictNamespaces=yes + +# Capabilities +AmbientCapabilities=CAP_NET_BIND_SERVICE +CapabilityBoundingSet=CAP_NET_BIND_SERVICE + +# Limit resources +MemoryHigh=150M +CPUQuota=50% + +[Install] +WantedBy=multi-user.target