为双端群聊增加端到端加密附件

This commit is contained in:
xhgit committed 2026-11-14 09:05:40 +08:00
1 parent 5988ad6ed6
commit a95a007768
21 files changed
+405 -84

No files matched your search

+2 -2
View File
@@ -20,12 +20,12 @@
| 联系人 | 按用户名查找、处理好友申请、管理备注与分组,以及拉黑。变更可在双端实时同步。 |
| 加密文字聊天 | 在两个已核验设备之间发送 MLS 加密消息,支持离线接收。 |
| 加密群聊 | 创建群组、邀请联系人和管理成员。每个账号选一台已核验设备参与 MLS 加密文字聊天。 |
| 加密附件 | 两端可发送和下载最多 8 MiB 的文件,并在本机预览图片和视频。 |
| 加密附件 | 两端可在私聊和群聊发送、下载最多 8 MiB 的文件,并在本机预览图片和视频。 |
| 加密语音 | 两端可录制和播放最长 60 秒的语音消息。 |
| 消息管理 | 支持会话查找、可选加密已读回执、引用回复、编辑、撤回、转发、本机搜索和收藏。 |
| Chromium 浏览器 | Windows 端支持独立窗口、多标签、导航、搜索、书签、历史、下载和隐私窗口,网页运行在独立会话中。 |
当前为开发预览。大文件分片、群聊附件、音视频和多设备历史同步仍在开发,Windows 安装包尚未发布。 [开发进度与验收记录](docs/progress.md)
当前为开发预览。大文件分片、音视频和多设备历史同步仍在开发,Windows 安装包尚未发布。 [开发进度与验收记录](docs/progress.md)
## 本地预览
+1 -1
View File
@@ -21,7 +21,7 @@ server {
proxy_pass http://api:8080;
proxy_set_header Host $http_host;
}
location /api/v1/attachments/ {
location ~ ^/api/v1/(groups/[0-9a-fA-F-]+/)?attachments/ {
client_max_body_size 9m;
proxy_pass http://api:8080;
proxy_set_header Host $http_host;
+4
View File
@@ -32,6 +32,8 @@
| POST | /api/v1/groups/{id}/devices/{device}/claim | 群主一次性领取群成员设备的 KeyPackage |
| GET / POST | /api/v1/groups/{id}/epochs | 读取当前设备可访问的会话信息或由群主发布新世代 |
| GET / POST | /api/v1/groups/{id}/messages | 使用 after 游标补拉群密文或向当前世代提交群密文 |
| POST | /api/v1/groups/{id}/attachments/{attachment}?epoch= | 当前世代设备上传附件密文,使用 application/octet-stream,成功返回 204 |
| GET | /api/v1/groups/{id}/attachments/{attachment} | 本次群成员资格仍有效且属于原会话的设备下载已发布附件 |
| POST | /api/v1/contacts/{id}/devices/{device}/claim | 为当前设备原子领取一个联系人的未使用 KeyPackage |
用户名为 3–32 位 ASCII 字母、数字或下划线,忽略大小写。密码最少 12 个 Unicode 字符,最多 256 字节。platform 为 web 或 windows。公钥、签名和 challenge 使用无填充 Base64URL。设备使用 Ed25519 签名 UTF-8 字符串 `XHChat device authentication v1\n` 后紧接 challenge。该签名仅用于证明基础设备密钥持有权。
@@ -54,4 +56,6 @@ MLS 凭据由标准 Base64 的 publicKey 和 signature 组成,使用 UTF-8 前
成员变更后旧世代不能发消息。绑定设备被撤销后需以替代设备发布新世代。重新加入的成员只能补拉本次加入以来的世代。服务端校验外层格式和投递授权,MLS 内容验证由客户端执行。
群附件上传后尚不可下载。原上传设备通过同一世代的群消息提交 `attachmentId` 发布附件。每个附件最多绑定一条消息,接收端将投递字段与 MLS 密文中的附件标识核对。未发布的附件随保留期清理,已发布附件随关联消息的保留期清理。上传与下载限制与私聊附件共用。
下一阶段将增加 /conversations、/calls、/posts、/backups、/push 与独立的管理接口。
+4 -1
View File
@@ -10,9 +10,12 @@
| devices、device_key_packages | 公开设备凭据及签名的一次性密钥包,事务领取,撤销后停止发放 |
| device_envelopes | 二进制协议封包、投递游标和目标设备,按发送设备、接收设备和客户端消息 ID 幂等 |
| device_attachments | 预览版 8 MiB 附件密文与设备访问范围,按附件 ID 幂等,跟随消息保留期 |
| group_attachments | 按群和附件 ID 唯一。记录发送设备、会话和附件密文,群消息可引用同一会话下该设备上传的附件 |
| attachments、backups | 对象存储标识、密文尺寸、配额和版本,密钥由端侧传递 |
| calls、call_participants | 通话状态、参与设备和时间,不存媒体 |
| posts、post_audiences、reactions | 私人动态正文为密文,受众变更触发重新分发或换钥 |
| admin_audit | 管理员、行为、目标标识、时间和结果 |
迁移 001 创建身份相关表。迁移 002 添加联系人关系、黑名单和备注。迁移 003 添加设备封包队列。投递事务按固定顺序锁定账号和设备,使同一目标设备的游标按提交顺序可见。设备撤销和会话清除在同一个事务内处理。后台作业清理超出保留期的封包。后续的附件下载将按目标设备和对象关联重新检查授权。
迁移 001 创建身份相关表。迁移 002 添加联系人关系、黑名单和备注。迁移 003 添加设备封包队列。投递事务按固定顺序锁定账号和设备,使同一目标设备的游标按提交顺序可见。设备撤销和会话清除在同一个事务内处理。后台作业清理超出保留期的封包。
迁移 009 创建群附件表并为现有群消息追加可空附件引用。下载在同一次查询中验证附件已发布、设备属于原会话名单、当前群成员资格和设备撤销状态。
+2
View File
@@ -13,3 +13,5 @@ OpenMLS 0.9.2 兼容性用例已在 Linux 原生和 Windows Chrome WebAssembly
预览版附件使用 Web Crypto 的整文件 AES-256-GCM,限制 8 MiB。每份文件独立随机密钥和 96 位 IV,附件 ID、文件名和字节数纳入认证数据。下载在完整验证后才生成本地文件,服务端只持有密文及投递元数据。生产版仍需分块认证、断点续传和对象存储。
预览版群聊的每个投递世代使用独立的 OpenMLS 会话,最多 32 个账号,每个账号选一台设备。参与设备各自在可信渠道核验其他设备安全码,并核对 Welcome 的签名创建者、完整 MLS 叶节点名单和群标识。加密消息绑定群、世代、消息 ID 和发送设备。成员加入、离开或绑定设备撤销后,群主为有效设备重新建立会话。服务端拒绝旧世代新消息,已离开的设备拿不到新会话的 Welcome。旧会话的已解密消息保存在本机。正式版仍需同一 MLS 群中的密钥更新、入侵后恢复、多设备授权和群历史同步。
群附件复用整文件 AES-256-GCM。群、世代和附件标识用于服务端授权,文件名和解密材料由群 MLS 消息保护。收件设备校验 MLS 描述与投递附件 ID 一致,并验证文件认证标签后才读取字节。成员离开后失去服务器下载权限,已下载的副本仍由成员保有。
+6 -2
View File
@@ -19,7 +19,7 @@ Web 和 Electron 共用 Vue 客户端,独立接入同一 Go API。真实 Postg
## 尚待实现
- 阶段一还需邮箱和手机号注册、验证码服务、找回密码和双重验证。真实联调需要 SMTP 与短信供应商配置。
- 阶段二与三还需分片附件、群聊附件与消息操作、多设备可信授权、加密备份与跨设备历史同步。文字群聊及两设备文字和小附件私聊已进入开发预览。
- 阶段二与三还需分片附件、群聊消息操作、多设备可信授权、加密备份与跨设备历史同步。群聊和私聊的文字及小附件已进入开发预览。
- 阶段四还需 WebRTC 音视频、TURN、SFU、屏幕共享及媒体 E2EE。
- 阶段五还需反钓鱼及聊天联动。
- 阶段六到八还需社交、扩展生态、独立管理后台、推送、生产部署与备份演练、压力测试、安全评估、签名安装包和发布流程。
@@ -107,6 +107,10 @@ OpenMLS WASM 新增批量设备入组。创建端校验每个 KeyPackage 和预
## 双端加密群聊预览
Windows 和 Web 已接入群设备核验、MLS Welcome 校验、文字加解密和本机历史。群主为每名有效成员选定一台已核验设备,成员加入或离开后由群主重建加密会话。预览可补拉本机已加入会话的消息。跨世代补拉尚未在本机加入的旧会话、群附件、群成员多设备和自动世代更新待实现。
Windows 和 Web 已接入群设备核验、MLS Welcome 校验、文字加解密和本机历史。群主为每名有效成员选定一台已核验设备,成员加入或离开后由群主重建加密会话。预览可补拉本机已加入会话的消息。跨世代补拉尚未在本机加入的旧会话、群成员多设备和自动世代更新待实现。
验收使用 Windows Electron 和两个独立 Edge 会话,覆盖安全码校验、Windows 发消息给 Web、Web 离开群页面后补收消息、移除成员、新世代 Windows 接收和移除成员无法继续访问。服务端集成测试验证群设备目录授权和密钥包只能领取一次,Rust 验证错误的 MLS 群标识与移除设备不能解密。
## 群附件
Windows 和 Web 可发送和下载最多 8 MiB 的群附件,并在本机预览常见图片和视频。接口验收覆盖大小限制、上传幂等、未发布附件隔离、发送设备绑定、移除和重新入群后的历史下载隔离。双端验收覆盖 Windows 发送文件到 Edge、逐字节核对下载、拒绝篡改的密文,以及 Edge 发送 PNG 到 Windows 预览。群附件分片、断点续传和群语音仍待实现。
@@ -0,0 +1,17 @@
CREATE TABLE group_attachments (
group_id uuid NOT NULL,
id uuid NOT NULL,
epoch_id uuid NOT NULL,
sender_device uuid NOT NULL,
ciphertext bytea NOT NULL CHECK (octet_length(ciphertext) BETWEEN 16 AND 8388624),
created_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (group_id,id),
UNIQUE (group_id,epoch_id,sender_device,id),
FOREIGN KEY (group_id,epoch_id,sender_device) REFERENCES group_epoch_devices(group_id,epoch_id,device_id) ON DELETE CASCADE
);
ALTER TABLE group_messages ADD COLUMN attachment_id uuid;
ALTER TABLE group_messages ADD FOREIGN KEY (group_id,epoch_id,sender_device,attachment_id)
REFERENCES group_attachments(group_id,epoch_id,sender_device,id);
CREATE UNIQUE INDEX group_messages_attachment ON group_messages(group_id,attachment_id) WHERE attachment_id IS NOT NULL;
CREATE INDEX group_attachments_expiry ON group_attachments(created_at);
+7 -1
View File
@@ -31,6 +31,9 @@ var groupsMigration string
//go:embed 008_group_transport.sql
var groupTransportMigration string
//go:embed 009_group_attachments.sql
var groupAttachmentsMigration string
func Migrate(ctx context.Context, pool *pgxpool.Pool) error {
tx, err := pool.Begin(ctx)
if err != nil {
@@ -43,7 +46,10 @@ func Migrate(ctx context.Context, pool *pgxpool.Pool) error {
if _, err = tx.Exec(ctx, "CREATE TABLE IF NOT EXISTS schema_migrations (version integer PRIMARY KEY)"); err != nil {
return err
}
for index, migration := range []string{identityMigration, contactsMigration, deliveryMigration, keyPackagesMigration, mlsCredentialsMigration, attachmentsMigration, groupsMigration, groupTransportMigration} {
for index, migration := range []string{
identityMigration, contactsMigration, deliveryMigration, keyPackagesMigration, mlsCredentialsMigration,
attachmentsMigration, groupsMigration, groupTransportMigration, groupAttachmentsMigration,
} {
version := index + 1
var applied bool
if err = tx.QueryRow(ctx, "SELECT EXISTS (SELECT 1 FROM schema_migrations WHERE version=$1)", version).Scan(&applied); err != nil {
+58
View File
@@ -0,0 +1,58 @@
package groups
import (
"context"
"errors"
"github.com/jackc/pgx/v5"
)
var ErrAttachment = errors.New("group attachment unavailable or already used")
func (s *Store) UploadAttachment(ctx context.Context, group, user, device, epoch, id string, ciphertext []byte) error {
tx, err := s.db.Begin(ctx)
if err != nil {
return err
}
defer tx.Rollback(ctx)
if err = lockSendingEpoch(ctx, tx, group, user, device, epoch); err != nil {
return err
}
inserted, err := tx.Exec(ctx, `
INSERT INTO group_attachments(group_id,id,epoch_id,sender_device,ciphertext)
VALUES($1,$2,$3,$4,$5) ON CONFLICT(group_id,id) DO NOTHING
`, group, id, epoch, device, ciphertext)
if err != nil {
return err
}
if inserted.RowsAffected() == 0 {
var matches bool
err = tx.QueryRow(ctx, `
SELECT ciphertext=$5 FROM group_attachments WHERE group_id=$1 AND id=$2 AND epoch_id=$3 AND sender_device=$4
`, group, id, epoch, device, ciphertext).Scan(&matches)
if errors.Is(err, pgx.ErrNoRows) || err == nil && !matches {
return ErrAttachment
}
if err != nil {
return err
}
}
return tx.Commit(ctx)
}
func (s *Store) DownloadAttachment(ctx context.Context, group, user, device, id string) ([]byte, error) {
var ciphertext []byte
err := s.db.QueryRow(ctx, `
SELECT a.ciphertext FROM group_attachments a
JOIN group_messages x ON x.group_id=a.group_id AND x.attachment_id=a.id
JOIN group_epochs g ON g.group_id=a.group_id AND g.id=a.epoch_id
JOIN group_epoch_devices e ON e.group_id=g.group_id AND e.epoch_id=g.id AND e.device_id=$3
JOIN group_members m ON m.group_id=g.group_id AND m.user_id=$2 AND m.state='active'
JOIN devices d ON d.id=e.device_id AND d.user_id=m.user_id AND d.revoked_at IS NULL
WHERE a.group_id=$1 AND a.id=$4 AND g.membership_version>=m.joined_version
`, group, user, device, id).Scan(&ciphertext)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrUnavailable
}
return ciphertext, err
}
+61 -37
View File
@@ -32,12 +32,13 @@ type AvailableEpoch struct {
}
type Message struct {
Sequence string `json:"sequence"`
Epoch string `json:"epoch"`
ID string `json:"id"`
Sender string `json:"senderDevice"`
Wire []byte `json:"wire"`
CreatedAt time.Time `json:"createdAt"`
Sequence string `json:"sequence"`
Epoch string `json:"epoch"`
ID string `json:"id"`
Sender string `json:"senderDevice"`
Wire []byte `json:"wire"`
CreatedAt time.Time `json:"createdAt"`
AttachmentID *string `json:"attachmentId,omitempty"`
}
func lockTransport(ctx context.Context, tx pgx.Tx, group, user, device string) (string, bool, error) {
@@ -149,27 +150,22 @@ func retryEpoch(ctx context.Context, tx pgx.Tx, group, sender string, input Epoc
return tx.Commit(ctx)
}
func (s *Store) Send(ctx context.Context, group, user, device string, message Message) (string, error) {
tx, err := s.db.Begin(ctx)
if err != nil {
return "", err
}
defer tx.Rollback(ctx)
func lockSendingEpoch(ctx context.Context, tx pgx.Tx, group, user, device, epoch string) error {
version, _, err := lockTransport(ctx, tx, group, user, device)
if err != nil {
return "", err
return err
}
var enrolled bool
err = tx.QueryRow(ctx, `
SELECT EXISTS(SELECT 1 FROM group_epoch_devices e JOIN group_epochs g
ON g.group_id=e.group_id AND g.id=e.epoch_id
WHERE e.group_id=$1 AND e.epoch_id=$2 AND e.device_id=$3 AND g.membership_version=$4::bigint AND g.retired_at IS NULL)
`, group, message.Epoch, device, version).Scan(&enrolled)
`, group, epoch, device, version).Scan(&enrolled)
if err != nil {
return "", err
return err
}
if !enrolled {
return "", ErrVersion
return ErrVersion
}
// Match the device revocation transaction's row lock. Once a bound device is
@@ -177,36 +173,57 @@ func (s *Store) Send(ctx context.Context, group, user, device string, message Me
rows, err := tx.Query(ctx, `
SELECT d.revoked_at IS NULL FROM group_epoch_devices e JOIN devices d ON d.id=e.device_id
WHERE e.group_id=$1 AND e.epoch_id=$2 ORDER BY d.id FOR UPDATE OF d
`, group, message.Epoch)
`, group, epoch)
if err != nil {
return err
}
defer rows.Close()
for rows.Next() {
var active bool
if err = rows.Scan(&active); err != nil {
return err
}
if !active {
return ErrVersion
}
}
return rows.Err()
}
func (s *Store) Send(ctx context.Context, group, user, device string, message Message) (string, error) {
tx, err := s.db.Begin(ctx)
if err != nil {
return "", err
}
for rows.Next() {
var active bool
if err = rows.Scan(&active); err != nil || !active {
rows.Close()
if err != nil {
return "", err
}
return "", ErrVersion
}
}
if err = rows.Err(); err != nil {
defer tx.Rollback(ctx)
if err = lockSendingEpoch(ctx, tx, group, user, device, message.Epoch); err != nil {
return "", err
}
if message.AttachmentID != nil {
var found string
err = tx.QueryRow(ctx, `
SELECT id FROM group_attachments WHERE group_id=$1 AND epoch_id=$2 AND sender_device=$3 AND id=$4 FOR KEY SHARE
`, group, message.Epoch, device, message.AttachmentID).Scan(&found)
if errors.Is(err, pgx.ErrNoRows) {
return "", ErrAttachment
}
if err != nil {
return "", err
}
}
var sequence string
err = tx.QueryRow(ctx, `
INSERT INTO group_messages(group_id,epoch_id,sender_device,message_id,ciphertext)
VALUES($1,$2,$3,$4,$5) ON CONFLICT DO NOTHING RETURNING sequence::text
`, group, message.Epoch, device, message.ID, message.Wire).Scan(&sequence)
INSERT INTO group_messages(group_id,epoch_id,sender_device,message_id,ciphertext,attachment_id)
VALUES($1,$2,$3,$4,$5,$6) ON CONFLICT DO NOTHING RETURNING sequence::text
`, group, message.Epoch, device, message.ID, message.Wire, message.AttachmentID).Scan(&sequence)
if errors.Is(err, pgx.ErrNoRows) {
var previous []byte
var same bool
err = tx.QueryRow(ctx, `
SELECT sequence::text,ciphertext FROM group_messages
SELECT sequence::text,ciphertext=$5 AND attachment_id IS NOT DISTINCT FROM $6::uuid FROM group_messages
WHERE group_id=$1 AND epoch_id=$2 AND sender_device=$3 AND message_id=$4
`, group, message.Epoch, device, message.ID).Scan(&sequence, &previous)
if err == nil && !bytes.Equal(previous, message.Wire) {
`, group, message.Epoch, device, message.ID, message.Wire, message.AttachmentID).Scan(&sequence, &same)
if errors.Is(err, pgx.ErrNoRows) || err == nil && !same {
return "", ErrVersion
}
}
@@ -226,7 +243,7 @@ func (s *Store) Inbox(ctx context.Context, group, user, device string, after int
return nil, err
}
rows, err := tx.Query(ctx, `
SELECT x.sequence::text,x.epoch_id,x.message_id,x.sender_device,x.ciphertext,x.created_at
SELECT x.sequence::text,x.epoch_id,x.message_id,x.sender_device,x.ciphertext,x.created_at,x.attachment_id
FROM group_messages x JOIN group_epochs g ON g.group_id=x.group_id AND g.id=x.epoch_id
JOIN group_epoch_devices e ON e.group_id=g.group_id AND e.epoch_id=g.id AND e.device_id=$3
JOIN group_members m ON m.group_id=x.group_id AND m.user_id=$2 AND m.state='active'
@@ -241,7 +258,7 @@ func (s *Store) Inbox(ctx context.Context, group, user, device string, after int
result := make([]Message, 0)
for rows.Next() {
var m Message
if err := rows.Scan(&m.Sequence, &m.Epoch, &m.ID, &m.Sender, &m.Wire, &m.CreatedAt); err != nil {
if err := rows.Scan(&m.Sequence, &m.Epoch, &m.ID, &m.Sender, &m.Wire, &m.CreatedAt, &m.AttachmentID); err != nil {
return nil, err
}
result = append(result, m)
@@ -317,6 +334,13 @@ func (s *Store) Cleanup(ctx context.Context, retention time.Duration) error {
if _, err = tx.Exec(ctx, "DELETE FROM group_messages WHERE created_at<$1", cutoff); err != nil {
return err
}
if _, err = tx.Exec(ctx, `
DELETE FROM group_attachments a WHERE created_at<$1 AND NOT EXISTS (
SELECT 1 FROM group_messages m WHERE m.group_id=a.group_id AND m.attachment_id=a.id
)
`, cutoff); err != nil {
return err
}
// Keep the current epoch and any old epoch with retained application messages.
_, err = tx.Exec(ctx, `
DELETE FROM group_epochs e USING chat_groups g
+2
View File
@@ -68,6 +68,8 @@ func New(accounts *account.Store, contactStore *contacts.Store, envelopeStore *d
mux.HandleFunc("POST /api/v1/groups/{id}/epochs", a.authorized(a.publishGroupEpoch))
mux.HandleFunc("GET /api/v1/groups/{id}/messages", a.authorized(a.groupInbox))
mux.HandleFunc("POST /api/v1/groups/{id}/messages", a.authorized(a.sendGroupMessage))
mux.HandleFunc("POST /api/v1/groups/{id}/attachments/{attachment}", a.authorized(a.uploadGroupAttachment))
mux.HandleFunc("GET /api/v1/groups/{id}/attachments/{attachment}", a.authorized(a.downloadGroupAttachment))
mux.HandleFunc("GET /api/v1/groups/{id}/devices", a.authorized(a.groupDevices))
mux.HandleFunc("POST /api/v1/groups/{id}/devices/{device}/claim", a.authorized(a.claimGroupPackage))
return a.boundary(mux)
+25 -12
View File
@@ -19,17 +19,11 @@ func (a *API) uploadAttachment(w http.ResponseWriter, r *http.Request, _ string,
if !a.limit(w, r, "attachment-upload:"+identity.Device.ID, 12) {
return
}
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, delivery.AttachmentLimit))
var tooLarge *http.MaxBytesError
switch {
case errors.As(err, &tooLarge):
problem(w, 413, "attachment_too_large", "附件超过 8 MiB")
return
case err != nil || len(body) < 16:
problem(w, 400, "invalid_attachment", "附件请求格式无效")
body, ok := readAttachment(w, r)
if !ok {
return
}
err = a.delivery.UploadAttachment(r.Context(), identity.Device.ID, recipient, id, body)
err := a.delivery.UploadAttachment(r.Context(), identity.Device.ID, recipient, id, body)
switch {
case errors.Is(err, delivery.ErrUnavailable):
problem(w, 404, "attachment_unavailable", "无法向指定设备发送附件")
@@ -42,6 +36,20 @@ func (a *API) uploadAttachment(w http.ResponseWriter, r *http.Request, _ string,
}
}
func readAttachment(w http.ResponseWriter, r *http.Request) ([]byte, bool) {
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, delivery.AttachmentLimit))
var tooLarge *http.MaxBytesError
switch {
case errors.As(err, &tooLarge):
problem(w, 413, "attachment_too_large", "附件超过 8 MiB")
return nil, false
case err != nil || len(body) < 16:
problem(w, 400, "invalid_attachment", "附件请求格式无效")
return nil, false
}
return body, true
}
func (a *API) downloadAttachment(w http.ResponseWriter, r *http.Request, _ string, identity account.Identity) {
id := r.PathValue("id")
if !uuidPattern.MatchString(id) {
@@ -58,8 +66,13 @@ func (a *API) downloadAttachment(w http.ResponseWriter, r *http.Request, _ strin
case err != nil:
unavailable(w, err)
default:
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Disposition", "attachment")
_, _ = w.Write(body)
writeAttachment(w, body)
}
}
func writeAttachment(w http.ResponseWriter, body []byte) {
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Disposition", "attachment")
w.Header().Set("Cache-Control", "no-store")
_, _ = w.Write(body)
}
@@ -0,0 +1,50 @@
package httpapi
import (
"errors"
"net/http"
"xhchat/server/internal/account"
"xhchat/server/internal/groups"
)
func (a *API) uploadGroupAttachment(w http.ResponseWriter, r *http.Request, _ string, actor account.Identity) {
group, id, epoch := r.PathValue("id"), r.PathValue("attachment"), r.URL.Query().Get("epoch")
if !uuidPattern.MatchString(group) || !uuidPattern.MatchString(id) || !uuidPattern.MatchString(epoch) ||
r.Header.Get("Content-Type") != "application/octet-stream" {
problem(w, 400, "invalid_attachment", "群附件请求格式无效")
return
}
if !a.limit(w, r, "attachment-upload:"+actor.Device.ID, 12) {
return
}
body, ok := readAttachment(w, r)
if !ok {
return
}
if err := a.groups.UploadAttachment(r.Context(), group, actor.User.ID, actor.Device.ID, epoch, id, body); err != nil {
groupError(w, err)
return
}
w.WriteHeader(204)
}
func (a *API) downloadGroupAttachment(w http.ResponseWriter, r *http.Request, _ string, actor account.Identity) {
group, id := r.PathValue("id"), r.PathValue("attachment")
if !uuidPattern.MatchString(group) || !uuidPattern.MatchString(id) {
problem(w, 404, "attachment_unavailable", "群附件不可用")
return
}
if !a.limit(w, r, "attachment-download:"+actor.Device.ID, 30) {
return
}
body, err := a.groups.DownloadAttachment(r.Context(), group, actor.User.ID, actor.Device.ID, id)
switch {
case errors.Is(err, groups.ErrUnavailable):
problem(w, 404, "attachment_unavailable", "群附件不可用或已过期")
case err != nil:
unavailable(w, err)
default:
writeAttachment(w, body)
}
}
+8 -5
View File
@@ -108,18 +108,21 @@ func (a *API) sendGroupMessage(w http.ResponseWriter, r *http.Request, _ string,
return
}
var input struct {
ID string `json:"id"`
Epoch string `json:"epoch"`
Wire []byte `json:"wire"`
ID string `json:"id"`
Epoch string `json:"epoch"`
Wire []byte `json:"wire"`
AttachmentID *string `json:"attachmentId,omitempty"`
}
if !parseLimit(w, r, &input, 96<<10) {
return
}
if !uuidPattern.MatchString(input.ID) || !uuidPattern.MatchString(input.Epoch) || !mlsFrame(input.Wire) || input.Wire[3] != 2 {
if !uuidPattern.MatchString(input.ID) || !uuidPattern.MatchString(input.Epoch) || !mlsFrame(input.Wire) || input.Wire[3] != 2 ||
input.AttachmentID != nil && !uuidPattern.MatchString(*input.AttachmentID) {
problem(w, 400, "invalid_group_message", "群消息密文无效")
return
}
sequence, err := a.groups.Send(r.Context(), id, actor.User.ID, actor.Device.ID, groups.Message{ID: input.ID, Epoch: input.Epoch, Wire: input.Wire})
message := groups.Message{ID: input.ID, Epoch: input.Epoch, Wire: input.Wire, AttachmentID: input.AttachmentID}
sequence, err := a.groups.Send(r.Context(), id, actor.User.ID, actor.Device.ID, message)
if err != nil {
groupError(w, err)
return
+2
View File
@@ -25,6 +25,8 @@ func groupError(w http.ResponseWriter, err error) {
problem(w, 409, "group_member_limit", "群成员与待处理邀请已达上限")
case errors.Is(err, groups.ErrVersion):
problem(w, 409, "group_epoch_changed", "群成员或设备版本已变更")
case errors.Is(err, groups.ErrAttachment):
problem(w, 409, "group_attachment_conflict", "群附件不可用或已绑定到其他消息")
default:
unavailable(w, err)
}
+34
View File
@@ -157,11 +157,43 @@ test('群密文只投递到当前名单中的设备,离开或撤销后阻断
assert.deepEqual(await json(path + '/messages', 'POST', member.token, message), sequence)
const received = await json(path + '/messages', 'GET', member.token)
assert.deepEqual(received.map((item) => item.wire), [message.wire])
const attachmentId = randomUUID()
const attachment = path + '/attachments/' + attachmentId
const bytes = randomBytes(16_384)
const upload = (token, epoch, body) => fetch(base + '/api/v1' + attachment + '?epoch=' + epoch, {
method: 'POST', headers: { Authorization: 'Bearer ' + token, 'Content-Type': 'application/octet-stream' },
body, signal: AbortSignal.timeout(10_000),
})
const linked = { id: randomUUID(), epoch: second.id, wire: frame(), attachmentId }
await json(path + '/messages', 'POST', owner.token, linked, 409)
assert.equal((await upload(outsider.token, second.id, bytes)).status, 404)
assert.equal((await upload(owner.token, first.id, bytes)).status, 409)
assert.equal((await upload(owner.token, second.id, Buffer.alloc(8 * 1024 * 1024 + 17))).status, 413)
assert.equal((await upload(owner.token, second.id, bytes)).status, 204)
assert.equal((await upload(owner.token, second.id, bytes)).status, 204)
assert.equal((await upload(owner.token, second.id, randomBytes(64))).status, 409)
await json(attachment, 'GET', owner.token, undefined, 404)
await json(attachment, 'GET', member.token, undefined, 404)
await json(path + '/messages', 'POST', member.token, linked, 409)
const posted = await json(path + '/messages', 'POST', owner.token, linked)
assert.deepEqual(await json(path + '/messages', 'POST', owner.token, linked), posted)
await json(path + '/messages', 'POST', owner.token, { ...linked, id: randomUUID() }, 409)
await json(path + '/messages', 'POST', owner.token, { ...linked, attachmentId: undefined }, 409)
assert.equal((await json(path + '/messages?after=' + sequence.sequence, 'GET', member.token))[0].attachmentId, attachmentId)
const download = await call(attachment, 'GET', member.token)
assert.equal(download.status, 200)
assert.equal(download.headers.get('cache-control'), 'no-store')
assert.deepEqual(Buffer.from(await download.arrayBuffer()), bytes)
await json(attachment, 'GET', outsider.token, undefined, 404)
await change(owner, 'remove', member.user.id)
await json(path + '/messages', 'GET', member.token, undefined, 404)
await json(attachment, 'GET', member.token, undefined, 404)
await json(path + '/messages', 'POST', owner.token, { ...message, id: randomUUID() }, 409)
await change(owner, 'invite', member.user.id)
await change(member, 'accept')
await json(attachment, 'GET', member.token, undefined, 404)
assert.equal((await call(attachment, 'GET', owner.token)).status, 200)
assert.deepEqual(await json(path + '/messages', 'GET', member.token), [])
assert.deepEqual(await json(path + '/epochs', 'GET', member.token), [])
const next = { ...second, id: randomUUID(), membershipVersion: '4' }
@@ -169,6 +201,8 @@ test('群密文只投递到当前名单中的设备,离开或撤销后阻断
assert.equal((await call('/devices/' + member.device.id, 'DELETE', member.token)).status, 204)
await json(path + '/messages', 'POST', owner.token, { id: randomUUID(), epoch: next.id, wire: frame() }, 409)
assert.equal((await json(path + '/epochs', 'GET', owner.token)).every((epoch) => !epoch.current), true)
await change(owner, 'disband')
await json(attachment, 'GET', owner.token, undefined, 404)
})
test('设备密文投递、幂等、离线补拉、实时提示和撤销', { timeout: 25000 }, async () => {
+49 -2
View File
@@ -1,6 +1,6 @@
import assert from 'node:assert/strict'
import { randomBytes } from 'node:crypto'
import { mkdtemp, rm } from 'node:fs/promises'
import { mkdtemp, readFile, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import test from 'node:test'
@@ -67,7 +67,21 @@ async function send(sender, text) {
return packet.epoch
}
test('群聊跨 Windows 与 Web 加密投递,移除成员后切换新会话', { timeout: 120_000 }, async () => {
async function sendFile(sender, name, mimeType, contents) {
const upload = sender.waitForResponse((item) => /\/groups\/[^/]+\/attachments\/[^/]+\?epoch=/.test(item.url()) && item.request().method() === 'POST')
const message = sender.waitForResponse((item) => /\/groups\/[^/]+\/messages$/.test(item.url()) && item.request().method() === 'POST')
await chat(sender).getByLabel('选择群附件').setInputFiles({ name, mimeType, buffer: contents })
await chat(sender).getByRole('button', { name: '发送群附件' }).click()
const uploaded = await upload
assert.equal(uploaded.status(), 204)
const sent = await message
assert.equal(sent.status(), 200, await sent.text())
const packet = sent.request().postDataJSON()
assert.match(new URL(uploaded.url()).pathname, new RegExp('/' + packet.attachmentId + '$'))
assert.ok(!Buffer.from(packet.wire, 'base64').includes(Buffer.from(name)))
}
test('群聊跨 Windows 与 Web 加密投递文字和附件,移除成员后切换会话', { timeout: 120_000 }, async () => {
const browser = await chromium.launch({ channel: process.env.XHCHAT_TEST_BROWSER ?? 'msedge' })
const profile = await mkdtemp(join(tmpdir(), 'xhchat-group-test-'))
let desktop
@@ -119,6 +133,39 @@ test('群聊跨 Windows 与 Web 加密投递,移除成员后切换新会话',
await chat(removed).getByRole('button', { name: '启用本机群聊' }).click()
await chat(removed).getByText(first, { exact: true }).waitFor()
const contents = Buffer.from('Windows 群文件内容 ' + id() + '\n'.repeat(9000))
const filename = '群附件-' + id() + '.txt'
await sendFile(windows, filename, 'text/plain', contents)
const fileDownload = owner.waitForEvent('download')
const fileResponse = owner.waitForResponse((item) => item.url().includes('/attachments/') && item.request().method() === 'GET')
await chat(owner).getByRole('button', { name: '下载 ' + filename }).click()
const response = await fileResponse
assert.equal(response.status(), 200)
const ciphertext = await response.body()
assert.equal(ciphertext.length, contents.length + 16)
assert.ok(!ciphertext.includes(contents.subarray(0, 24)))
const downloaded = await fileDownload
assert.equal(downloaded.suggestedFilename(), filename)
assert.deepEqual(await readFile(await downloaded.path()), contents)
await owner.route(response.url(), async (route) => {
const tampered = Buffer.from(ciphertext)
tampered[0] ^= 1
await route.fulfill({ status: 200, contentType: 'application/octet-stream', body: tampered })
}, { times: 1 })
await chat(owner).getByRole('button', { name: '下载 ' + filename }).click()
await chat(owner).getByRole('alert').getByText('附件完整性校验失败').waitFor()
const photo = '群图片-' + id() + '.png'
const pixel = Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+aX2sAAAAASUVORK5CYII=', 'base64')
await sendFile(owner, photo, 'image/png', pixel)
await chat(windows).getByRole('button', { name: '预览 ' + photo }).click()
const image = chat(windows).getByRole('img', { name: photo })
await image.evaluate((element) => element.decode())
assert.equal(await image.evaluate((element) => element.naturalWidth), 1)
await chat(windows).getByRole('button', { name: '关闭预览' }).click()
await image.waitFor({ state: 'detached' })
const member = owner.getByRole('list', { name: '群成员' }).getByRole('listitem').filter({ hasText: names[2] })
await member.getByRole('button', { name: '移除成员' }).click()
await owner.getByRole('button', { name: '确认操作' }).click()
+40 -2
View File
@@ -1,8 +1,11 @@
<script setup lang="ts">
import { computed, onBeforeUnmount, onMounted, ref, watch } from 'vue'
import { useSession } from '../session'
import { downloadAttachment } from './attachments'
import type { Attachment } from './attachments'
import MediaPreview from './MediaPreview.vue'
import { enableChat, trust } from './chat'
import { discardPendingGroupMessage, publishGroup, sendGroupMessage, synchronizeGroup } from './group-chat'
import { discardPendingGroupMessage, groupAttachmentPath, publishGroup, sendGroupMessage, synchronizeGroup } from './group-chat'
import type { GroupDetail, GroupDevice } from './group-chat'
const props = defineProps<{ group: GroupDetail }>()
@@ -12,6 +15,7 @@ const code = ref('')
const codes = ref<Record<string, string>>({})
const selected = ref<Record<string, string>>({})
const draft = ref('')
const file = ref<File>()
const busy = ref(false)
const error = ref('')
const ready = computed(() => view.value?.ready && view.value.current?.membershipVersion === props.group.membershipVersion)
@@ -84,6 +88,26 @@ function send() {
})
}
function chooseFile(event: Event) {
const input = event.target as HTMLInputElement
file.value = input.files?.[0]
input.value = ''
}
function sendFile() {
const identity = session.identity
const selectedFile = file.value
if (identity && selectedFile) void run(async () => {
await sendGroupMessage(identity, props.group, selectedFile, session.authorized)
file.value = undefined
await refresh()
})
}
function download(attachment: Attachment) {
void run(() => downloadAttachment(attachment, session.authorized, groupAttachmentPath(props.group, attachment.id)))
}
function discard() {
const identity = session.identity
if (identity) void run(async () => {
@@ -135,7 +159,12 @@ onBeforeUnmount(() => clearInterval(timer))
<ul v-if="view?.lines.length" class="group-chat-lines" aria-label="群消息记录">
<li v-for="line in view.lines" :key="line.id">
<small>{{ line.name }} · {{ new Date(line.time).toLocaleString() }}</small>
<p>{{ line.text }}</p>
<template v-if="line.attachment">
<p>{{ line.attachment.name }} · {{ new Intl.NumberFormat().format(line.attachment.size) }} 字节</p>
<MediaPreview v-if="line.attachment.media" :attachment="line.attachment" :path="groupAttachmentPath(group, line.attachment.id)" />
<button type="button" class="plain" :disabled="busy" @click="download(line.attachment)">下载 {{ line.attachment.name }}</button>
</template>
<p v-else>{{ line.text }}</p>
</li>
</ul>
<p v-if="view?.hasPending" role="status">
@@ -146,6 +175,14 @@ onBeforeUnmount(() => clearInterval(timer))
<label>群消息<textarea v-model="draft" rows="3" maxlength="8000" required /></label>
<button class="primary" :disabled="busy || view?.hasPending || !draft.trim()">发送群消息</button>
</form>
<div v-if="ready" class="group-chat-composer">
<label>选择群附件 <input type="file" :disabled="busy || view?.hasPending" @change="chooseFile"></label>
<div v-if="file" class="message-actions">
<span>{{ file.name }}</span>
<button type="button" class="plain" :disabled="busy || view?.hasPending" @click="sendFile">发送群附件</button>
<button type="button" class="plain" :disabled="busy" @click="file = undefined">取消</button>
</div>
</div>
</template>
</section>
</template>
@@ -160,4 +197,5 @@ onBeforeUnmount(() => clearInterval(timer))
.group-chat-composer { display: grid; gap: .55rem; }
.group-chat-composer textarea { display: block; width: 100%; resize: vertical; font: inherit; }
.group-chat button { justify-self: start; }
.media-preview :deep(img), .media-preview :deep(video) { display: block; max-width: 100%; max-height: 18rem; }
</style>
+2 -2
View File
@@ -4,7 +4,7 @@ import { readAttachment } from './attachments'
import type { Attachment } from './attachments'
import { useSession } from '../session'
const props = defineProps<{ attachment: Attachment }>()
const props = defineProps<{ attachment: Attachment; path?: string }>()
const session = useSession()
const url = ref('')
const busy = ref(false)
@@ -21,7 +21,7 @@ async function open() {
busy.value = true
error.value = ''
try {
const bytes = await readAttachment(props.attachment, session.authorized)
const bytes = await readAttachment(props.attachment, session.authorized, props.path)
if (active) url.value = URL.createObjectURL(new Blob([bytes], { type: props.attachment.media }))
} catch (cause) {
if (active) error.value = cause instanceof Error ? cause.message : '无法加载媒体预览'
+4 -4
View File
@@ -137,8 +137,8 @@ function encodeContent(kind: number, value: object) {
return message
}
export async function readAttachment(attachment: Attachment, call: Call) {
const ciphertext = await call<ArrayBuffer>(attachmentPath(attachment.id), 'GET', undefined, true)
export async function readAttachment(attachment: Attachment, call: Call, path = attachmentPath(attachment.id)) {
const ciphertext = await call<ArrayBuffer>(path, 'GET', undefined, true)
if (ciphertext.byteLength !== attachment.size + 16) throw new Error('附件完整性校验失败')
const raw = decode(attachment.key)
let bytes: ArrayBuffer
@@ -153,8 +153,8 @@ export async function readAttachment(attachment: Attachment, call: Call) {
return bytes
}
export async function downloadAttachment(attachment: Attachment, call: Call) {
const bytes = await readAttachment(attachment, call)
export async function downloadAttachment(attachment: Attachment, call: Call, path?: string) {
const bytes = await readAttachment(attachment, call, path)
const url = URL.createObjectURL(new Blob([bytes], { type: 'application/octet-stream' }))
const link = document.createElement('a')
link.download = attachment.name
+27 -13
View File
@@ -2,6 +2,8 @@ import { APIError } from '../api'
import { base64, decode } from './engine'
import { replenishPackages, update, verifiedPeer, verify } from './chat'
import type { Call, ChatIdentity, Package, Peer } from './chat'
import { decodeMessage, encodeAttachment, sealAttachment } from './attachments'
import type { Attachment } from './attachments'
export type GroupMember = { id: string; username: string; state: 'active' | 'invited'; role: 'owner' | 'admin' | 'member' }
export type GroupDetail = { id: string; membershipVersion: string; members: GroupMember[]; role: GroupMember['role'] }
@@ -9,8 +11,8 @@ export type GroupDevice = Peer & { user: string }
type Epoch = { id: string; creatorDevice: string; membershipVersion: string; welcome?: string; current: boolean; roster: string[] }
type Publishing = { id: string; membershipVersion: string; devices: { device: string; wire?: string }[] }
type Outgoing = { id: string; epoch: string; wire: string }
type Packet = Outgoing & { sequence: string; senderDevice: string; createdAt: string }
type Line = { id: string; user: string; sender: string; name: string; text: string; time: string }
type Packet = Outgoing & { sequence: string; senderDevice: string; createdAt: string; attachmentId?: string }
type Line = { id: string; user: string; sender: string; name: string; text: string; time: string; attachment?: Attachment }
type Verified = { user: string; key: string; name: string }
type VerifiedEpoch = { version: string; roster: Record<string, Verified> }
export type GroupChatState = {
@@ -22,8 +24,8 @@ export type GroupChatState = {
}
const encoder = new TextEncoder()
const decoder = new TextDecoder('utf-8', { fatal: true })
const path = (group: GroupDetail) => '/groups/' + encodeURIComponent(group.id)
export const groupAttachmentPath = (group: GroupDetail, id: string) => path(group) + '/attachments/' + encodeURIComponent(id)
const aad = (group: string, epoch: string, id: string, sender: string) =>
encoder.encode(['XHChat group message v1', group, epoch, id, sender].join('\n'))
@@ -132,7 +134,9 @@ export function publishGroup(identity: ChatIdentity, group: GroupDetail, selecti
async function sendPending(group: GroupDetail, chat: GroupChatState, call: Call) {
const pending = chat.pendingMessage
if (!pending) return
await call(path(group) + '/messages', 'POST', { id: pending.id, epoch: pending.epoch, wire: pending.wire })
await call(path(group) + '/messages', 'POST', {
id: pending.id, epoch: pending.epoch, wire: pending.wire, attachmentId: pending.line.attachment?.id,
})
chat.lines.push(pending.line)
delete chat.pendingMessage
}
@@ -186,7 +190,8 @@ export function synchronizeGroup(identity: ChatIdentity, group: GroupDetail, cal
for (const item of packets) {
if (item.senderDevice === identity.device.id) {
const pending = chat.pendingMessage
if (pending && pending.id === item.id && pending.epoch === item.epoch && pending.wire === item.wire) {
if (pending && pending.id === item.id && pending.epoch === item.epoch && pending.wire === item.wire &&
pending.line.attachment?.id === item.attachmentId) {
chat.lines.push(pending.line)
delete chat.pendingMessage
}
@@ -198,11 +203,12 @@ export function synchronizeGroup(identity: ChatIdentity, group: GroupDetail, cal
try {
const bytes = engine.decrypt_group(await groupID(group.id, item.epoch), item.senderDevice, decode(sender.key),
aad(group.id, item.epoch, item.id, item.senderDevice), decode(item.wire))
const text = decoder.decode(bytes)
if (text.length > 0 && bytes.length <= 8000) {
chat.lines.push({ id: item.id, user: sender.user, name: sender.name, sender: item.senderDevice, text, time: item.createdAt })
}
state.engine = base64(engine.snapshot())
const { text, attachment, quote } = decodeMessage(bytes)
if (attachment?.id !== item.attachmentId) throw new Error('群附件标识不匹配')
if (attachment || (!quote && text.length > 0 && bytes.length <= 8000)) {
chat.lines.push({ id: item.id, user: sender.user, name: sender.name, sender: item.senderDevice, text, attachment, time: item.createdAt })
}
} catch {
warning ||= '已跳过无法通过验证的群消息'
} finally {
@@ -233,10 +239,11 @@ export function discardPendingGroupMessage(identity: ChatIdentity, group: GroupD
})
}
export function sendGroupMessage(identity: ChatIdentity, group: GroupDetail, text: string, call: Call) {
export function sendGroupMessage(identity: ChatIdentity, group: GroupDetail, message: string | File, call: Call) {
return update(identity, async (state, save, factory) => {
const content = encoder.encode(text.trim())
if (content.length === 0 || content.length > 8000) throw new Error('消息须为 1 至 8000 字节')
const text = typeof message === 'string' ? message.trim() : ''
let content: Uint8Array = encoder.encode(text)
if (typeof message === 'string' && (content.length === 0 || content.length > 8000)) throw new Error('消息须为 1 至 8000 字节')
const chat = local(state.groupChats ??= {}, group.id)
const epochs = await call<Epoch[]>(path(group) + '/epochs')
const epoch = epochs.find((item) => item.current && item.membershipVersion === group.membershipVersion)
@@ -245,11 +252,18 @@ export function sendGroupMessage(identity: ChatIdentity, group: GroupDetail, tex
const id = crypto.randomUUID()
const own = chat.epochs[epoch.id]!.roster[identity.device.id]
if (!own) throw new Error('当前设备未加入群加密世代')
let attachment: Attachment | undefined
if (typeof message !== 'string') {
const sealed = await sealAttachment(message)
attachment = sealed.attachment
await call(groupAttachmentPath(group, attachment.id) + '?' + new URLSearchParams({ epoch: epoch.id }), 'POST', sealed.ciphertext)
content = encodeAttachment(attachment)
}
const engine = factory.DeviceMls.restore(decode(state.engine))
try {
const wire = engine.encrypt(await groupID(group.id, epoch.id), aad(group.id, epoch.id, id, identity.device.id), content)
chat.pendingMessage = { id, epoch: epoch.id, wire: base64(wire),
line: { id, user: identity.user.id, name: own.name, sender: identity.device.id, text: text.trim(), time: new Date().toISOString() } }
line: { id, user: identity.user.id, name: own.name, sender: identity.device.id, text, attachment, time: new Date().toISOString() } }
state.engine = base64(engine.snapshot())
await save(state)
} finally {