Skip to content

feat(mail): event-driven SMTP send + IMAP recv client module - #884

Open
ithewei wants to merge 3 commits into
masterfrom
feat-mail-module
Open

ithewei wants to merge 3 commits into
masterfrom
feat-mail-module

Conversation

@ithewei

@ithewei ithewei commented Sep 19, 2026

Copy link
Copy Markdown
Owner

What

Closes #883 (send/receive email like curl). Adds a new optional mail/ module with asynchronous email clients built on the hloop event loop — replacing the synchronous demo in protocol/smtp.c (kept as-is for compatibility).

Module

  • mail/mime.{h,c} — MIME assembly + parsing, shared by both clients.
    • assembly: multipart/mixed + multipart/alternative, base64 attachments, RFC 2047 encoded-word for non-ASCII headers.
    • parsing: multipart split (nested), base64 / quoted-printable decode, extracts From/Subject/Date/text/html/attachments into mail_t.
    • mail_t supports multi-recipient To/Cc + attachments, with mail_set_* / mail_add_* / mail_clear helpers.
  • mail/smtp_client.{h,c} — async SMTP client (C core + SmtpClient C++ wrapper). State machine: EHLO → AUTH LOGIN → MAIL FROM → RCPT TO (multi) → DATA → QUIT. Result via callback.
  • mail/imap_client.{h,c} — async IMAP client (C core + ImapClient C++ wrapper). LOGIN → SELECT → SEARCH → FETCH (per id, literal {n} parsing) → LOGOUT, then MIME-parses each message and delivers via per-mail callback.
  • TLS: direct TLS (SMTPS 465 / IMAPS 993) via the existing hssl abstraction.

The API/structure mirrors the existing mqtt/ module (C core + C++ wrapper, _new/_run/_stop/_free, hloop state machine, hmutex for thread-safety).

Build

--with-mail / WITH_MAIL (default OFF), wired across configure / Makefile / CMake / Bazel, mirroring the mqtt module. Requires SSL for direct TLS. Linux CI now builds with --with-mail.

Examples + docs

  • examples/mail/sendmail_test.cpp, examples/mail/recvmail_test.cpp
  • docs/cn/SmtpClient.md, docs/cn/ImapClient.md
  • README / README-CN mention the mail client.

Scope / limitations (by design)

  • Direct TLS only (SMTPS 465 / IMAPS 993); no STARTTLS (587/143). Mainstream mailboxes (QQ/163/Gmail) all offer direct-SSL ports.
  • IMAP read-only minimal set (LOGIN/SELECT/SEARCH/FETCH/LOGOUT); no STORE/COPY/move/IDLE.
  • No POP3 (IMAP is the modern default; POP3 has little practical value).
  • MIME parsing does not transcode charset (returned as-is) to avoid an iconv dependency.

Testing

  • make libhv (--with-mail --with-openssl) ✅, make sendmail_test recvmail_test
  • CMake build with -DWITH_MAIL=ON
  • mail sources compile clean under -Wall -Wextra.
  • MIME build→parse round-trip verified with a standalone program (text + base64 attachment + RFC 2047 subject round-trip correctly).
  • SMTP send / IMAP fetch against real mailboxes are intended for manual verification (require live accounts); CI covers compilation.

ithewei and others added 2 commits September 19, 2026 13:45
Add a new optional `mail/` module providing asynchronous email clients
built on the hloop event loop, addressing issue #883 (send/recv email
like curl). Replaces the synchronous demo in protocol/smtp.c (kept as-is
for compatibility).

- mail/mime.{h,c}: MIME assembly (multipart/mixed + multipart/alternative,
  base64 attachments, RFC 2047 encoded-word for non-ASCII headers) and
  parsing (multipart split, base64 / quoted-printable decode). Shared by
  both clients. mail_t with multi-recipient To/Cc + attachments.
- mail/smtp_client.{h,c}: async SMTP client (C core + SmtpClient C++
  wrapper). State machine EHLO -> AUTH LOGIN -> MAIL FROM -> RCPT TO
  (multi) -> DATA -> QUIT. Direct TLS (SMTPS 465) via hssl.
- mail/imap_client.{h,c}: async IMAP client (C core + ImapClient C++
  wrapper). LOGIN -> SELECT -> SEARCH -> FETCH (per id, with literal {n}
  parsing) -> LOGOUT, then MIME-parses each message. Direct TLS (IMAPS 993).
- build wiring: --with-mail / WITH_MAIL (default OFF), mirroring the mqtt
  module across configure / Makefile / CMake / Bazel.
- examples/mail/{sendmail,recvmail}_test.cpp + docs/cn/{SmtpClient,ImapClient}.md.
- README: mention the mail client.

Scope: direct TLS only (no STARTTLS); IMAP read-only minimal set (no
STORE/COPY/IDLE); no POP3. Integration is manually verified against real
mailboxes; CI covers compilation.

Co-authored-by: TRAE CLI <traecli@bytedance.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
Copilot AI lite review requested due to automatic review settings September 19, 2026 05:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved critical security issues and multiple correctness and build findings block approval.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 5 High severity · 6 Medium severity

Open (11)
What changed in this PR

Adds an optional asynchronous SMTP/IMAP client module with shared MIME handling, C/C++ APIs, direct TLS support, documentation, examples, and build integration.

Changes:

  • Adds MIME assembly/parsing and SMTP/IMAP state machines.
  • Wires WITH_MAIL across Make, CMake, Bazel, configuration, and CI.
  • Adds mail examples and client documentation.
File Reviewed changes and final findings
README.md Documents mail client support; no final comments.
README-CN.md Documents mail client support in Chinese; no final comments.
Makefile.vars Adds mail headers; no final comments.
Makefile Builds mail sources and examples; no final comments.
mail/​smtp_client.h Defines SMTP C/C++ APIs; no final comments.
mail/​smtp_client.c Implements SMTP state machine. Moderate (3 votes): DATA payload requires dot-stuffing. Moderate (2 votes): accept valid 252 recipient responses. Critical (2 votes): set TLS hostname for SNI. Moderate (1 vote): release owned loops during teardown. Critical (2 votes): handle vsnprintf truncation to prevent out-of-bounds reads.
mail/​mime.h Defines MIME and mail APIs; no final comments.
mail/​mime.c Implements MIME assembly/parsing. Moderate (3 votes): decode RFC 2047 encoded-word subjects. Critical (3 votes): prevent CR/LF header injection. Moderate (2 votes): accept MIME base64 line breaks. Moderate (1 vote): reject or guard NULL attachment data. Moderate (1 vote): unfold continuation lines in headers.
mail/​imap_client.h Defines IMAP C/C++ APIs; no final comments.
mail/​imap_client.c Implements IMAP fetching. Critical (3 votes): safely encode credentials and reject control characters. Critical (2 votes): set TLS hostname for SNI. Moderate (1 vote): release owned loops during teardown. Moderate (3 votes): retain incomplete tagged responses. Moderate (1 vote): grow or reject SEARCH results beyond 4096 IDs. Moderate (1 vote): report failed LOGOUT responses. Moderate (1 vote): use BODY.PEEK[] to avoid marking messages read.
examples/​mail/​sendmail_test.cpp Adds SMTP usage example; no final comments.
examples/​mail/​recvmail_test.cpp Adds IMAP usage example; no final comments.
docs/​cn/​SmtpClient.md Documents SMTP usage; no final comments.
docs/​cn/​ImapClient.md Documents IMAP usage; no final comments.
configure Adds the mail configure option; no final comments.
config.ini Adds the mail feature flag; no final comments.
CMakeLists.txt Adds CMake mail integration. Moderate (2 votes): add CMake targets for both mail examples.
cmake/​vars.cmake Defines mail headers; no final comments.
BUILD.bazel Adds Bazel mail integration. Moderate (1 vote): add conditional mail example targets and filegroup entries.
.github/​workflows/​CI.yml Enables mail compilation in CI; no final comments.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread mail/imap_client.c
Comment thread mail/imap_client.c
Comment thread mail/mime.c Outdated
Comment thread mail/smtp_client.c
Comment thread mail/smtp_client.c
Comment thread mail/imap_client.c
Comment thread mail/mime.c Outdated
Comment thread mail/mime.c
Comment thread mail/smtp_client.c Outdated
Comment thread mail/smtp_client.c
MIME (mail/mime.c):
- header injection: encode header values containing CR/LF/control chars via
  RFC 2047 (was appended verbatim, allowing Bcc-style injection).
- base64 decode now tolerates MIME line breaks/whitespace (hv_base64_decode
  rejects CR/LF), so wrapped attachments no longer decode to 0 bytes.
- decode RFC 2047 encoded-words when parsing headers (Subject etc.), so a
  build->parse round-trip returns the original text.
- attachment decode is binary-safe now (track decoded length instead of
  strlen, which truncated data at the first embedded NUL).

SMTP (mail/smtp_client.c):
- clamp vsnprintf return to the actual buffer size to avoid reading past
  the stack buffer on very long host/address.
- dot-stuff the DATA payload so a body line "." cannot terminate DATA early.
- accept RCPT TO reply code 252 (accepted, cannot verify recipient).
- set TLS SNI hostname (hio_set_hostname) for SMTPS virtual hosts.

IMAP (mail/imap_client.c):
- quote LOGIN credentials and reject control characters (command injection).
- require a complete CRLF-terminated line before treating a tagged response
  as final, so responses split across reads are not discarded.
- set TLS SNI hostname for IMAPS virtual hosts.
- use BODY.PEEK[] so fetching does not mark messages as \Seen.

Build:
- examples/CMakeLists.txt: add WITH_MAIL targets (sendmail_test/recvmail_test).

Co-authored-by: TRAE CLI <traecli@bytedance.com>
Copilot AI review requested due to automatic review settings September 19, 2026 06:23
@ithewei

ithewei commented Sep 19, 2026

Copy link
Copy Markdown
Owner Author

Thanks for the thorough review — addressed in 0cd21e9. Summary of what changed:

High

  • SNI hostname (SMTP/IMAP): hio_create_socket only stores the peer address, so hio_set_hostname(io, host) is now called before hio_enable_ssl — name-based virtual SMTPS/IMAPS hosts now get proper SNI.
  • vsnprintf truncation (SMTP): return value (would-be length) is clamped to the actual buffer size before writing, so a long host/address can't cause an over-read.
  • Header CR/LF injection (MIME): header values with CR/LF/control chars are now forced through RFC 2047 encoding instead of the ASCII fast path.
  • IMAP credential injection: LOGIN username/password are quoted (with "/\ escaping) and control characters are rejected up front.

Medium

  • base64 with line breaks (MIME): hv_base64_decode rejects CR/LF, so a whitespace-stripping wrapper was added — previously any wrapped attachment decoded to 0 bytes. Verified a 5000-byte attachment now round-trips.
  • RFC 2047 decode on parse: Subject/etc. encoded-words are decoded when parsing, so build→parse round-trips to the original text.
  • IMAP partial tagged response: a tagged line is only treated as final once a full CRLF-terminated line has arrived, so responses split across reads aren't discarded.
  • RCPT TO 252: accepted as success.
  • DATA dot-stuffing: body lines starting with . are dot-stuffed so a lone . line can't terminate DATA early.
  • CMake example targets: examples/CMakeLists.txt now builds sendmail_test/recvmail_test under WITH_MAIL.
  • Also made attachment decoding binary-safe (track decoded length instead of strlen, which truncated at embedded NULs), and switched FETCH to BODY.PEEK[] so reading doesn't set \Seen.

Not changed (minor / out of scope for this module): SEARCH id cap >4096, reporting a failed LOGOUT, header continuation unfolding — can follow up if needed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Multiple unresolved critical security and correctness issues affect the SMTP, IMAP, MIME, and public API changes.

Review effort: Lite
Findings: 9 High severity · 2 Medium severity

Open (11)
Resolved since last review (10)
Previously missed (4)

In code that hasn't changed since last review

Medium severity Report failed FETCH responses

mail/​imap_client.c:183

ok is computed but ignored in this no-literal path. A tagged NO/BAD response to FETCH is consumed as a successful message ID, so the client advances and can eventually report done(0) despite the fetch failing. Report the error and close instead of returning success when ok is false.

This issue also appears on line 298 of the same file.

Medium severity Unfold continuation lines in RFC 5322 headers

mail/​mime.c:410

find_header intentionally returns only one physical line, so folded RFC 5322 headers are not unfolded. A folded Content-Type commonly puts the boundary parameter on the continuation line, causing multipart messages and attachments to be skipped; unfold continuations before parsing header values.

Medium severity Parse display names and addresses separately

mail/​mime.c:634

mail_addr_t.addr is documented as the mailbox address, but this stores the entire decoded From field. For From: Alice <alice@example.com>, callers receive Alice <alice@example.com> as from.addr and no from.name, producing an invalid address if the parsed mail is reused for SMTP. Split the display name and angle-address into the two fields.

Medium severity Buffer partial SMTP response lines

mail/​smtp_client.c:119

This parser only examines the bytes from one read callback. TCP/TLS may split a single 220 or 250 line across callbacks; the partial line is then discarded and the state machine waits forever for a response it already consumed. Keep an accumulated response buffer and parse only complete lines.

Comment thread mail/imap_client.c
Comment on lines +188 to +191
char* q = brace + 1;
long size = 0;
while (q < end && *q >= '0' && *q <= '9') { size = size * 10 + (*q - '0'); q++; }
if (q >= end || *q != '}') return 0; // incomplete literal header
Comment thread mail/imap_client.c
Comment on lines +486 to +489
// reject control characters in credentials (IMAP command injection guard)
if (imap_has_ctrl(cli->username) || imap_has_ctrl(cli->password)) return ERR_INVALID_PARAM;
if (mailbox) hv_strncpy(cli->mailbox, mailbox, sizeof(cli->mailbox));
if (criteria) hv_strncpy(cli->criteria, criteria, sizeof(cli->criteria));
Comment thread mail/imap_client.c
if (cli->ssl_ctx) hio_set_ssl_ctx(io, cli->ssl_ctx);
// set SNI hostname (see smtp_client for rationale)
hio_set_hostname(io, cli->host);
hio_enable_ssl(io);
Comment thread mail/mime.c
Comment on lines +97 to +100
int need = 12 + BASE64_ENCODE_OUT_SIZE(inlen) + 1;
if (outlen < need) return -1;
int n = snprintf(out, outlen, "=?utf-8?B?");
n += hv_base64_encode((const unsigned char*)in, inlen, out + n);
Comment thread mail/mime.c
if (close + 1 < end) {
int tlen = (int)(close - text);
if (enc == 'B' || enc == 'b') {
o += hv_base64_decode(text, tlen, (unsigned char*)out + o);
Comment thread mail/mime.c
if (addr->name && addr->name[0]) {
append_header_value(buf, addr->name);
membuf_puts(buf, " <");
membuf_puts(buf, addr->addr ? addr->addr : "");
Comment thread mail/mime.h
Comment on lines +30 to +34
typedef struct mail_s {
mail_addr_t from;
mail_addr_t* to; int to_count;
mail_addr_t* cc; int cc_count;
char* subject;
Comment thread mail/smtp_client.c
int smtp_client_send(smtp_client_t* cli, mail_t* mail) {
if (!cli || !mail) return -1;
if (!cli->host[0]) return ERR_INVALID_PARAM;
if (!mail->from.addr || (mail->to_count == 0 && mail->cc_count == 0)) return ERR_INVALID_PARAM;
Comment thread mail/smtp_client.c
// the TLS backend sends SNI from io->hostname. Without this, name-based
// virtual SMTPS hosts may reject the handshake or route incorrectly.
hio_set_hostname(io, cli->host);
hio_enable_ssl(io);
Comment thread BUILD.bazel
"with_mqtt": glob(["mqtt/*.h", "mqtt/*.c", "mqtt/*.cpp"], exclude = ["mqtt/*_test.c"]),
"//conditions:default": [],
}) + select({
"with_mail": glob(["mail/*.h", "mail/*.c", "mail/*.cpp"], exclude = ["mail/*_test.c"]),
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

能支持收发E-Mail 吗

2 participants