yiekheng cfd3308477 feat(media): unsupported image/video/audio formats fall back to document delivery
Old behaviour: HEIC/AVIF photos, .mov / .webm / .mkv videos, and niche
audio (FLAC, etc.) got rejected outright at upload with "Images are
not supported" / "Videos are not supported" errors. Strict but
unfriendly — recipients could still receive these as a downloadable
file via WhatsApp's document path; we just weren't using it.

New behaviour: anything not playable inline gets routed through the
document path automatically. The recipient downloads the file and
opens it in their default app. The 100 MB document cap applies
instead of the inline 5 / 16 / 16 MB caps. Only oversized uploads
get rejected.

Where the policy lives
----------------------
The classifier moved into a new `@cmbot/shared/whatsapp-media`
module so the web upload validator AND the bot's fire-reminder send
path share one source of truth:

  - resolveDeliveryKind(mime, bytes?) → "image" | "video" | "audio"
    | "document". Native types stay as-is; HEIF / AVIF / QuickTime /
    WebM / Matroska / non-MP3-or-M4A audio all collapse to "document".
  - Bytes argument is optional but recommended — sniffing the first
    12 bytes of the file catches iOS Safari's habit of labelling
    a HEIC as image/jpeg or a .mov as video/mp4. Bytes win when they
    disagree with the mime.

Web side
--------
- `lib/whatsapp-media.ts` re-exports the shared helpers and keeps
  only the validator + byte-formatter. `validateForWhatsApp` calls
  resolveDeliveryKind internally; the size cap it returns is for the
  RESOLVED kind (so a HEIC routes to document and gets the 100 MB
  cap). The "Images are not supported" / "Videos are not supported"
  rejection messages are gone — there's no format rejection anymore.
- `actions/media.ts` collapses the previous explicit-mime + byte-sniff
  pair into a single `validateForWhatsApp(mime, size, bytes)` call.
- Compose-step upload-zone hint updated to spell out the per-kind
  caps: "JPEG/PNG up to 5 MB · MP4/3GP up to 16 MB · MP3/M4A/OGG
  up to 16 MB · documents up to 100 MB".

Bot side
--------
- `fire-reminder.ts` reads the first 12 bytes of the file before
  dispatching and calls `resolveDeliveryKind(mimeType, head)` to
  pick the senderKind. So a HEIC on disk (whose mime claims
  image/jpeg) gets sent via Baileys' document path — no failed
  thumbnail extraction, message arrives as a downloadable .heic.
- New `readHeadBytes(filePath, n)` helper opens, reads N bytes,
  closes — no full-file slurp.

Tests
-----
249 web + 31 shared + 26 bot = 306 passing total.

Web (`lib/whatsapp-media.test.ts`):
- "HEIC at 30 MB allowed: routes to document (100 MB cap)"
- "HEIC at 110 MB rejects: exceeds the document cap"
- "MOV at 50 MB allowed (would be 16 MB cap as video, 100 MB as
  document)"
- "MOV pretending to be mp4 demotes to document (50 MB allowed)"
- "FLAC audio routes to document path"
- "genuine MP4 byte-sniff path keeps it as video"

Shared (`packages/shared/src/whatsapp-media.test.ts`, new):
- The cross-package contract: 11 tests covering size limits,
  classifyMediaKind, resolveDeliveryKind for native + demoted +
  byte-sniff cases, plus the underlying helpers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 13:07:54 +08:00

405 lines
13 KiB
TypeScript

"use client";
import { useRef, useState } from "react";
import {
ArrowDownIcon,
ArrowUpIcon,
FileIcon,
ImageIcon,
MessageSquareTextIcon,
PaperclipIcon,
PlusIcon,
Trash2Icon,
UploadIcon,
} from "lucide-react";
import { Button } from "@/components/ui/button";
import { Textarea } from "@/components/ui/textarea";
import { Input } from "@/components/ui/input";
import { uploadMediaAction } from "@/actions/media";
import { cn } from "@/lib/utils";
import type { MessagePart } from "@/lib/reminder-messages";
const MAX_PARTS = 10;
/**
* Per-part draft. Mirrors `MessagePart` but adds UI-only metadata
* (filename / mime / preview URL) that's discarded before the parts
* cross the wire to the server action.
*/
interface PartDraft extends MessagePart {
/** Local id so React's key stays stable across reorders. */
uid: string;
filename?: string;
mimeType?: string;
/** ObjectURL preview for image media — not persisted. */
previewUrl?: string;
}
function newUid(): string {
return Math.random().toString(36).slice(2);
}
function emptyText(): PartDraft {
return { uid: newUid(), kind: "text", textContent: "", mediaId: null };
}
function emptyMedia(): PartDraft {
return { uid: newUid(), kind: "media", textContent: "", mediaId: null };
}
interface MessageStackProps {
initial: MessagePart[];
/** Called whenever the stack changes — parent owns the canonical state. */
onChange: (parts: MessagePart[]) => void;
/** Optional resolver for already-uploaded mediaId → display info. */
initialMediaInfo?: Record<string, { filename: string; mimeType: string }>;
}
/**
* Stack-of-blocks editor for a reminder's message parts.
*
* Each row is either a `text` block (Textarea) or a `media` block
* (file picker + caption Input + per-block delete/move buttons). The
* stack supports reorder via up/down buttons and ships at most
* MAX_PARTS rows. The bot fires the parts in this order with ~1.5 s
* spacing between sends.
*/
export function MessageStack({ initial, onChange, initialMediaInfo }: MessageStackProps) {
const [parts, setParts] = useState<PartDraft[]>(() => {
if (initial.length === 0) return [emptyText()];
return initial.map((p) => {
const info = p.mediaId ? initialMediaInfo?.[p.mediaId] : undefined;
return {
uid: newUid(),
kind: p.kind,
textContent: p.textContent ?? "",
mediaId: p.mediaId ?? null,
filename: info?.filename,
mimeType: info?.mimeType,
};
});
});
function emit(next: PartDraft[]) {
setParts(next);
onChange(
next
.map<MessagePart>((p) => ({
kind: p.kind,
textContent: p.textContent && p.textContent.trim() ? p.textContent : null,
mediaId: p.mediaId,
}))
// Drop empty text blocks and orphan media blocks at compile-out
// — onChange should reflect only "real" message parts.
.filter((p) =>
p.kind === "text" ? Boolean(p.textContent) : Boolean(p.mediaId),
),
);
}
function update(uid: string, patch: Partial<PartDraft>) {
emit(parts.map((p) => (p.uid === uid ? { ...p, ...patch } : p)));
}
function remove(uid: string) {
if (parts.length === 1) {
// Don't go below one block; reset it instead.
emit([emptyText()]);
return;
}
emit(parts.filter((p) => p.uid !== uid));
}
function move(uid: string, direction: -1 | 1) {
const idx = parts.findIndex((p) => p.uid === uid);
const target = idx + direction;
if (idx < 0 || target < 0 || target >= parts.length) return;
const next = parts.slice();
[next[idx], next[target]] = [next[target]!, next[idx]!];
emit(next);
}
function add(kind: "text" | "media") {
if (parts.length >= MAX_PARTS) return;
emit([...parts, kind === "text" ? emptyText() : emptyMedia()]);
}
return (
<div className="space-y-3">
{parts.map((part, idx) => (
<Block
key={part.uid}
part={part}
index={idx}
total={parts.length}
onChange={(patch) => update(part.uid, patch)}
onRemove={() => remove(part.uid)}
onMoveUp={() => move(part.uid, -1)}
onMoveDown={() => move(part.uid, 1)}
/>
))}
<div className="flex flex-wrap gap-2">
<Button
type="button"
size="sm"
variant="outline"
onClick={() => add("text")}
disabled={parts.length >= MAX_PARTS}
className="gap-1.5"
>
<PlusIcon className="size-3.5" />
<MessageSquareTextIcon className="size-3.5" />
Add text
</Button>
<Button
type="button"
size="sm"
variant="outline"
onClick={() => add("media")}
disabled={parts.length >= MAX_PARTS}
className="gap-1.5"
>
<PlusIcon className="size-3.5" />
<PaperclipIcon className="size-3.5" />
Add file
</Button>
{parts.length >= MAX_PARTS && (
<span className="self-center text-xs text-muted-foreground">
Up to {MAX_PARTS} parts per reminder
</span>
)}
</div>
</div>
);
}
interface BlockProps {
part: PartDraft;
index: number;
total: number;
onChange: (patch: Partial<PartDraft>) => void;
onRemove: () => void;
onMoveUp: () => void;
onMoveDown: () => void;
}
function Block({ part, index, total, onChange, onRemove, onMoveUp, onMoveDown }: BlockProps) {
return (
<div className="rounded-xl border border-border bg-card p-3 space-y-2.5">
<div className="flex items-center justify-between gap-2">
<span className="text-xs font-medium uppercase tracking-wide text-muted-foreground">
{part.kind === "text" ? "Text" : "File"} · #{index + 1}
</span>
<div className="flex items-center gap-0.5">
<Button
type="button"
size="icon-sm"
variant="ghost"
onClick={onMoveUp}
disabled={index === 0}
aria-label="Move up"
className="size-7"
>
<ArrowUpIcon className="size-3.5" />
</Button>
<Button
type="button"
size="icon-sm"
variant="ghost"
onClick={onMoveDown}
disabled={index === total - 1}
aria-label="Move down"
className="size-7"
>
<ArrowDownIcon className="size-3.5" />
</Button>
<Button
type="button"
size="icon-sm"
variant="ghost"
onClick={onRemove}
aria-label="Remove this part"
className="size-7 text-muted-foreground hover:text-destructive"
>
<Trash2Icon className="size-3.5" />
</Button>
</div>
</div>
{part.kind === "text" ? (
<Textarea
value={part.textContent ?? ""}
onChange={(e) => onChange({ textContent: e.target.value })}
placeholder="Type your message…"
rows={3}
className="resize-none"
/>
) : (
<MediaBlock part={part} onChange={onChange} />
)}
</div>
);
}
interface MediaBlockProps {
part: PartDraft;
onChange: (patch: Partial<PartDraft>) => void;
}
function MediaBlock({ part, onChange }: MediaBlockProps) {
const fileInputRef = useRef<HTMLInputElement>(null);
const [uploading, setUploading] = useState(false);
const [error, setError] = useState<string | null>(null);
const [dragOver, setDragOver] = useState(false);
async function handleFile(file: File) {
setError(null);
setUploading(true);
const fd = new FormData();
fd.append("file", file);
const result = await uploadMediaAction(null, fd);
setUploading(false);
if (result.ok) {
// Revoke prior preview URL to avoid leaks across replacements.
if (part.previewUrl) URL.revokeObjectURL(part.previewUrl);
const previewUrl = result.mimeType.startsWith("image/")
? URL.createObjectURL(file)
: undefined;
onChange({
mediaId: result.mediaId,
filename: result.filename,
mimeType: result.mimeType,
previewUrl,
});
} else {
setError(result.error);
}
}
function clearMedia() {
if (part.previewUrl) URL.revokeObjectURL(part.previewUrl);
onChange({ mediaId: null, filename: undefined, mimeType: undefined, previewUrl: undefined });
if (fileInputRef.current) fileInputRef.current.value = "";
}
const isImage = part.mimeType?.startsWith("image/");
const hasMedia = Boolean(part.mediaId);
return (
<div className="space-y-2">
{hasMedia ? (
<div className="rounded-lg border border-border bg-muted/30 p-2.5">
<div className="flex items-start gap-2.5">
{isImage && part.previewUrl ? (
// eslint-disable-next-line @next/next/no-img-element
<img
src={part.previewUrl}
alt={part.filename ?? "Uploaded image"}
className="size-14 rounded-md object-cover shrink-0 border border-border"
/>
) : (
<div className="flex size-14 shrink-0 items-center justify-center rounded-md bg-muted border border-border">
<FileIcon className="size-5 text-muted-foreground" />
</div>
)}
<div className="min-w-0 flex-1 space-y-0.5">
<p className="text-sm font-medium truncate">
{part.filename ?? "Uploaded file"}
</p>
{part.mimeType && (
<p className="text-xs text-muted-foreground truncate">{part.mimeType}</p>
)}
</div>
<Button
type="button"
size="sm"
variant="ghost"
onClick={() => fileInputRef.current?.click()}
className="text-xs"
>
Replace
</Button>
<Button
type="button"
size="icon-sm"
variant="ghost"
onClick={clearMedia}
aria-label="Remove file"
className="size-7 text-muted-foreground hover:text-destructive"
>
<Trash2Icon className="size-3.5" />
</Button>
</div>
<Input
value={part.textContent ?? ""}
onChange={(e) => onChange({ textContent: e.target.value })}
placeholder="Caption (optional)"
className="mt-2 h-8 text-sm"
/>
</div>
) : (
<div
role="button"
tabIndex={0}
aria-label="Click or drag a file to upload"
onDragOver={(e) => {
e.preventDefault();
setDragOver(true);
}}
onDragLeave={() => setDragOver(false)}
onDrop={(e) => {
e.preventDefault();
setDragOver(false);
const file = e.dataTransfer.files?.[0];
if (file) void handleFile(file);
}}
onClick={() => fileInputRef.current?.click()}
onKeyDown={(e) => {
if (e.key === "Enter" || e.key === " ") fileInputRef.current?.click();
}}
className={cn(
"flex flex-col items-center gap-1.5 rounded-lg border-2 border-dashed px-4 py-5 text-center cursor-pointer transition-colors",
dragOver
? "border-primary bg-primary/5"
: "border-border hover:border-primary/50 hover:bg-muted/30",
uploading && "pointer-events-none opacity-60",
)}
>
{uploading ? (
<>
<UploadIcon className="size-5 text-muted-foreground animate-pulse" />
<p className="text-xs text-muted-foreground">Uploading</p>
</>
) : (
<>
<div className="flex items-center gap-1.5 text-muted-foreground">
<ImageIcon className="size-4" />
<PaperclipIcon className="size-3.5" />
</div>
<div>
<p className="text-xs font-medium">Click to upload or drag & drop</p>
<p className="text-[11px] text-muted-foreground mt-0.5">
JPEG/PNG up to 5 MB · MP4/3GP up to 16 MB · MP3/M4A/OGG up to 16 MB · documents up to 100 MB
</p>
</div>
</>
)}
</div>
)}
<input
ref={fileInputRef}
type="file"
aria-hidden="true"
className="sr-only"
onChange={(e) => {
const file = e.target.files?.[0];
if (file) void handleFile(file);
}}
/>
{error && (
<p className="text-xs text-destructive">{error}</p>
)}
</div>
);
}