diff --git a/docs/Features/3D-Layers.md b/docs/Features/3D-Layers.md index 2eb2a182..a14690fc 100644 --- a/docs/Features/3D-Layers.md +++ b/docs/Features/3D-Layers.md @@ -44,6 +44,7 @@ Prepared splat runtime metadata, native splat rasterization, preview, nested com - Any video or image clip can be toggled to 3D from the Transform panel. - 3D layers become textured planes in the common 3D scene. - Video and image clips that still use the default `position.z = 0` start slightly behind the scene target when first toggled to 3D, so a full-frame plane does not depth-mask splats or meshes. +- During export, 3D video planes sample the per-frame `VideoFrame` produced by the export decoder instead of relying on the preview `HTMLVideoElement`, so animated video planes advance correctly in fast and precise exports. - While scrubbing, 3D video planes keep their last uploaded texture if the browser video element is briefly between decoded frames, avoiding full shared-scene flicker. - Turning 3D off resets the 3D-specific transform state back to 2D defaults. @@ -182,6 +183,7 @@ The Transform tab is context-sensitive: 3D layers are included in export. - Scene camera resolution, scene-layer collection, splat runtime preparation, preload, and readiness now share the same scene contract across preview, nested, and export. +- Exported 3D video planes use decoder-provided `VideoFrame` textures when available, matching the 2D export compositor frame timing. - Gaussian splats can export through prepared or direct native scene modes while keeping identical scene-camera semantics. - Export waits for shared 3D and splat readiness before capture so preview and export stay aligned. diff --git a/docs/Features/Keyframes.md b/docs/Features/Keyframes.md index 3abe9f99..da9cf8d3 100644 --- a/docs/Features/Keyframes.md +++ b/docs/Features/Keyframes.md @@ -68,8 +68,8 @@ mask.{maskId}.featherQuality ``` `mask.*.path` stores the whole path as one keyframe value: all vertices, bezier handles, handle modes, and the closed/open state. -It interpolates between compatible neighboring paths and holds when the topology does not match. -The numeric mask properties use the same curve and easing behavior as transform and effect values. +It interpolates between neighboring paths and handles added or removed vertices by tweening them from or into a collapsed neighbor point. +The numeric mask properties use the same curve and easing behavior as transform and effect values, but the Mask tab presents `mask.*.path` as the primary stopwatch for shape animation. ### Vector Animation Properties @@ -132,7 +132,7 @@ The diamond button writes a keyframe at the playhead. If a keyframe already exis - Transform panel stopwatch buttons are per value, including Position X/Y/Z, Scale All/X/Y/Z, and Rotation X/Y/Z. Group stopwatches are not used for these rows. - `scale.all` does not overwrite `scale.x`, `scale.y`, or `scale.z`; render, export, and scene-gizmo paths multiply it into the final visible scale only at evaluation time. - Camera stopwatch buttons are per camera value. FOV and mm both write `camera.fov`; Near, Far, Resolution X, and Resolution Y write their own camera properties. -- Mask panel stopwatch buttons are available for Feather, Feather Quality, Position X/Y, and the whole Mask Path. +- Mask panel stopwatch buttons are available for the whole Mask Path, Feather, and Feather Quality. Position X/Y remain animatable for compatibility and automation, but the visible mask-shape workflow uses the Mask Path stopwatch. - Motion shape stopwatch buttons are available for size, corner radius, fill opacity, and stroke width in the Motion tab. ### Recording Mode diff --git a/docs/Features/Masks.md b/docs/Features/Masks.md index 4d64bda8..16971ded 100644 --- a/docs/Features/Masks.md +++ b/docs/Features/Masks.md @@ -9,11 +9,11 @@ MasterSelects supports per-clip vector masks with preview-overlay editing, selec - Masks are stored on timeline clips as `ClipMask[]`. - The properties panel exposes rectangle, ellipse, and pen creation. - The preview overlay supports vertex selection, handle mode toggles, edge insertion, edge dragging, and whole-mask dragging. -- Whole-mask dragging updates the mask `position.x` / `position.y` offset, so the Mask tab values stay in sync while dragging. +- Whole-mask dragging uses an internal mask offset; visible shape animation is driven by the `Mask Path` stopwatch. - Mask outlines are projected through the active layer transform, so 2D and 3D movement, scale, and rotation keep the editable overlay aligned with the rendered mask. - When the mask tab is active, the normal preview Edit Mode toggle becomes navigation-only: wheel zoom and Alt/MMB pan stay available, but layer transform handles are disabled. - Mask outlines are only shown while the mask tab is open. Opening the tab activates the current mask for editing; leaving the tab hides the overlay again. -- Mask paths can be keyframed as a single whole-path property. +- Mask path animation is exposed as one `Mask Path` stopwatch, not separate vertex X/Y stopwatches. - Mask changes are serialized with the project. ## Data Model @@ -32,10 +32,12 @@ MasterSelects supports per-clip vector masks with preview-overlay editing, selec - `position` - `enabled` - `visible` +- `outlineColor` `MaskMode` is currently `add`, `subtract`, or `intersect`. `enabled` controls whether a mask contributes to the rendered mask texture. `visible` controls only the preview overlay outline and edit handles. +`outlineColor` controls the SVG stroke color used for that mask in the preview overlay. Each `MaskVertex` can store `handleMode` as `none`, `mirrored`, or `split`. Per-mask inversion is baked into the generated mask texture before GPU compositing, so mixed normal/inverted masks on the same clip do not rely on a clip-wide inversion flag. @@ -66,12 +68,12 @@ Clicking the first point closes the path once at least three vertices exist. The preview overlay is implemented in `src/components/preview/MaskOverlay.tsx`. -- Active masks render as SVG paths over the preview. +- Every visible mask renders its SVG outline over the preview. - The SVG overlay is sized to the displayed canvas, not the full preview wrapper, so pointer coordinates stay aligned when the preview is letterboxed. - Visible masks show vertex squares, selected-vertex highlights, bezier handle circles, and edge hit areas. - Selected bezier vertices always show their handles, including when the outline is hidden or a handle is currently zero-length. - Mask geometry is edited in layer-local UV space and projected to the preview with the current layer transform. -- Whole-mask dragging moves `position.x` and `position.y`, leaving the stored vertex topology unchanged. +- Whole-mask dragging moves the internal `position.x` and `position.y` offset, leaving the stored vertex topology unchanged. - Dragging an edge moves the two adjacent vertices together. - Clicking an edge with the pen tool inserts a new vertex. - Dragging a vertex moves that vertex. @@ -95,31 +97,37 @@ The overlay uses normalized layer-local coordinates internally. Pointer input is The properties panel exposes the following controls per mask: - Name +- Outline color +- Mask Path stopwatch - Visibility toggle - Mode dropdown - Render enabled toggle - Feather - Feather quality -- Position X / Y - Inverted - Selected vertex handle mode `featherQuality` is stored as a 1-100 value in the UI and defaults to `50` for new masks. Lower values use a lower-resolution CPU blur path for faster previews; higher values preserve more edge detail. Feather is applied per mask before mask-mode compositing, so a later subtract mask can still cut into an earlier feathered add mask. +Right-clicking a mask row, or clicking its color swatch, opens the outline color palette for that mask. Mask opacity is intentionally not exposed in the mask panel. Layer opacity is handled by the normal transform controls. ## Mask Keyframes -Mask feather, feather quality, and Position X / Y use the normal numeric keyframe workflow. -The active mask name also exposes a stopwatch for `mask.{maskId}.path`. +The active mask exposes a dedicated `Mask Path` stopwatch for `mask.{maskId}.path`. That path keyframe stores the whole mask shape at once: all vertices, bezier handles, handle modes, and the closed/open state. +This is the primary animation workflow for changing individual mask vertices over time. + +Mask feather and feather quality use the normal numeric keyframe workflow. +The underlying `mask.{maskId}.position.x` and `mask.{maskId}.position.y` properties stay supported for older projects and automation paths, but they are not exposed in the Mask tab. Editing a vertex, handle, edge, or keyboard-nudging selected vertices records a new path keyframe when the path stopwatch is active or path keyframes already exist. This matches the After Effects-style workflow where the mask path is one animatable property instead of one property per vertex. -Path interpolation expects matching topology between neighboring path keyframes. -When the vertex count or open/closed state differs, playback holds the previous path until the next compatible path keyframe. +Path interpolation can morph between different vertex counts. +When a vertex exists on only one side of a keyframe segment, playback creates a temporary collapsed vertex on the nearest surviving neighbor and tweens it into, or out of, that point. +The open/closed state remains discrete and switches at the destination keyframe. ## Shortcuts @@ -167,6 +175,6 @@ Relevant files: ## Limitations - Mask tracking is not implemented. -- Mask path interpolation requires matching topology between neighboring keyframes. +- Mask path interpolation morphs added and removed vertices through collapsed neighbor points. - Mask mode is applied while generating the combined CPU mask texture. diff --git a/src/App.css b/src/App.css index 9d9a5fe5..e293946a 100644 --- a/src/App.css +++ b/src/App.css @@ -6252,6 +6252,18 @@ input[type="checkbox"] { font-weight: 500; color: var(--text-primary); cursor: text; + min-width: 0; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; + max-width: 62px; +} + +.track-layer-id { + flex-shrink: 0; + font-size: 10px; + color: var(--text-secondary); + line-height: 1; } .track-name-input { @@ -9120,6 +9132,60 @@ input[type="checkbox"] { font-size: 11px; color: var(--text-secondary); margin-right: auto; + white-space: nowrap; +} + +.media-panel-search { + position: relative; + display: flex; + align-items: center; + width: min(260px, 28vw); + min-width: 140px; + height: 24px; + color: var(--text-muted); + flex: 0 1 260px; +} + +.media-panel-search svg { + position: absolute; + left: 8px; + pointer-events: none; +} + +.media-panel-search input { + width: 100%; + height: 24px; + padding: 0 24px 0 26px; + border: 1px solid var(--border-color); + border-radius: 4px; + background: var(--bg-primary); + color: var(--text-primary); + font-size: 11px; + outline: none; +} + +.media-panel-search input:focus { + border-color: var(--accent); + box-shadow: 0 0 0 1px rgba(45, 140, 235, 0.28); +} + +.media-panel-search-clear { + position: absolute; + right: 4px; + width: 17px; + height: 17px; + border: 0; + border-radius: 3px; + background: transparent; + color: var(--text-secondary); + font-size: 12px; + line-height: 17px; + cursor: pointer; +} + +.media-panel-search-clear:hover { + background: var(--bg-hover); + color: var(--text-primary); } .media-panel-actions { @@ -9635,6 +9701,10 @@ input[type="checkbox"] { 0 0; } +.media-board-wrapper.board-interacting { + background: var(--bg-primary); +} + .media-board-toolbar { display: flex; align-items: center; @@ -9688,17 +9758,19 @@ input[type="checkbox"] { cursor: grabbing; } -.media-board-wrapper.board-interacting .media-board-node { - will-change: transform; -} - -.media-board-wrapper.board-interacting .media-board-group { +.media-board-wrapper.board-interacting .media-board-canvas-inner { will-change: transform; } .media-board-wrapper.board-interacting .media-board-node, .media-board-wrapper.board-interacting .media-board-group, .media-board-wrapper.board-interacting .media-board-insert-gap { + box-shadow: none; + transition: none; +} + +.media-board-wrapper.board-interacting .media-board-node-body, +.media-board-wrapper.board-interacting .media-board-node-timeline-drag { transition: none; } @@ -9715,6 +9787,13 @@ input[type="checkbox"] { transform-origin: 0 0; } +.media-board-overview-canvas { + position: absolute; + display: block; + pointer-events: none; + z-index: 4; +} + .media-board-group { position: absolute; contain: layout paint style; @@ -9723,7 +9802,7 @@ input[type="checkbox"] { background: rgba(255, 255, 255, 0.035); box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.04); pointer-events: auto; - transition: left 0.16s ease, top 0.16s ease, width 0.16s ease, height 0.16s ease, border-color 0.1s, box-shadow 0.1s; + transition: left 0.16s ease, top 0.16s ease, width 0.16s ease, height 0.16s ease, border-color 0.1s, box-shadow 0.1s, opacity 0.2s ease, filter 0.2s ease; } .media-board-group.folder-group { @@ -9756,6 +9835,18 @@ input[type="checkbox"] { z-index: 15; } +.media-board-group.folder-group.selected { + border-color: var(--accent); + box-shadow: + inset 0 1px 0 rgba(255, 255, 255, 0.06), + 0 0 0 2px rgba(45, 140, 235, 0.24), + 0 10px 22px rgba(0, 0, 0, 0.24); +} + +.media-board-group.folder-group.drag-source-preview { + z-index: 14; +} + .media-board-group-header { height: 42px; display: flex; @@ -9831,7 +9922,7 @@ input[type="checkbox"] { background: var(--bg-secondary); box-shadow: 0 8px 18px rgba(0, 0, 0, 0.22); cursor: grab; - transition: left 0.16s ease, top 0.16s ease, width 0.16s ease, height 0.16s ease, border-color 0.1s, box-shadow 0.1s, background 0.1s; + transition: left 0.16s ease, top 0.16s ease, width 0.16s ease, height 0.16s ease, border-color 0.1s, box-shadow 0.1s, background 0.1s, opacity 0.2s ease, filter 0.2s ease; pointer-events: auto; z-index: 5; } @@ -9859,6 +9950,12 @@ input[type="checkbox"] { opacity: 0.55; } +.media-board-node.search-dimmed, +.media-board-group.search-dimmed { + opacity: 0.26; + filter: grayscale(1) saturate(0.2); +} + .media-board-node.lod-compact { box-shadow: 0 3px 8px rgba(0, 0, 0, 0.18); } @@ -11026,6 +11123,7 @@ input[type="checkbox"] { display: flex; align-items: center; gap: 4px; + min-width: 0; } .track-expand-arrow { @@ -12668,10 +12766,11 @@ input[type="checkbox"] { } .mask-item { + position: relative; background: var(--bg-secondary); border: 1px solid var(--border-color); border-radius: 6px; - overflow: hidden; + overflow: visible; } .mask-item.active { @@ -12679,6 +12778,10 @@ input[type="checkbox"] { box-shadow: inset 3px 0 0 rgba(45, 140, 235, 0.9); } +.mask-item.color-menu-open { + z-index: 25; +} + .mask-item.disabled { opacity: 0.68; } @@ -12696,6 +12799,64 @@ input[type="checkbox"] { background: var(--bg-hover); } +.mask-outline-swatch { + width: 14px; + height: 14px; + flex: 0 0 14px; + padding: 0; + border: 1px solid rgba(255, 255, 255, 0.48); + border-radius: 4px; + box-shadow: 0 0 0 1px rgba(0, 0, 0, 0.38); + cursor: pointer; +} + +.mask-outline-swatch:hover, +.mask-outline-swatch[aria-expanded="true"] { + border-color: rgba(255, 255, 255, 0.82); +} + +.mask-outline-color-menu { + position: absolute; + top: 34px; + left: 40px; + z-index: 30; + display: grid; + grid-template-columns: repeat(6, 18px) 26px; + align-items: center; + gap: 6px; + padding: 7px; + background: var(--bg-tertiary); + border: 1px solid var(--border-color); + border-radius: 6px; + box-shadow: var(--shadow-lg); +} + +.mask-outline-color-option { + position: relative; + width: 18px; + height: 18px; + padding: 0; + border: 1px solid rgba(255, 255, 255, 0.42); + border-radius: 4px; + cursor: pointer; +} + +.mask-outline-color-option:hover, +.mask-outline-color-option.active { + border-color: #fff; + box-shadow: 0 0 0 2px rgba(45, 140, 235, 0.45); +} + +.mask-outline-color-input { + width: 26px; + height: 22px; + padding: 0; + border: 1px solid var(--border-color); + border-radius: 4px; + background: transparent; + cursor: pointer; +} + .mask-name { flex: 1; min-width: 0; @@ -12760,13 +12921,27 @@ input[type="checkbox"] { margin-bottom: 2px; } -.mask-active-header strong .keyframe-toggle { - flex: 0 0 auto; +.mask-active-header span { + color: var(--text-secondary); + font-size: 10px; } -.mask-active-header span { +.mask-path-row { + display: flex; + align-items: center; + gap: var(--sp-2); +} + +.mask-path-row label { + min-width: 70px; +} + +.mask-path-row span { + margin-left: auto; + min-width: auto; color: var(--text-secondary); font-size: 10px; + white-space: nowrap; } .mask-vertex-tools { diff --git a/src/changelog-data.json b/src/changelog-data.json index 667fd241..2e560e33 100644 --- a/src/changelog-data.json +++ b/src/changelog-data.json @@ -1,4 +1,11 @@ [ + { + "date": "2026-05-09", + "type": "improve", + "title": "Large Media Board Search and Loading", + "description": "Project search now supports text and glob-style filters like *.mp4, board search dims non-matching items instead of hiding layout context, and large board layouts restore from cached placement snapshots for faster startup.", + "section": "Project Board" + }, { "date": "2026-05-09", "type": "improve", diff --git a/src/components/common/EditableDraggableNumber.tsx b/src/components/common/EditableDraggableNumber.tsx index 5073d5cd..79dd3667 100644 --- a/src/components/common/EditableDraggableNumber.tsx +++ b/src/components/common/EditableDraggableNumber.tsx @@ -103,6 +103,8 @@ export function EditableDraggableNumber({ const startValue = useRef(0); const lastClientX = useRef(0); const dragStarted = useRef(false); + const pointerLockRequested = useRef(false); + const pointerLockActive = useRef(false); const [isEditing, setIsEditing] = useState(false); const [draftValue, setDraftValue] = useState(() => formatEditableValue(value, decimals)); const [draftDefaultValue, setDraftDefaultValue] = useState(''); @@ -248,9 +250,28 @@ export function EditableDraggableNumber({ }, [showBoundsPopover, updatePopoverPlacement]); const readDragDeltaX = useCallback((event: MouseEvent) => { - const dx = event.clientX - lastClientX.current; + const element = spanRef.current; + const isPointerLocked = + pointerLockActive.current || + (element !== null && document.pointerLockElement === element); + const movementX = Number.isFinite(event.movementX) ? event.movementX : 0; + + if (isPointerLocked) { + return movementX; + } + + const clientDx = event.clientX - lastClientX.current; lastClientX.current = event.clientX; - return dx; + + if ( + pointerLockRequested.current && + movementX !== 0 && + Math.abs(clientDx) > Math.abs(movementX) * 4 + 8 + ) { + return movementX; + } + + return clientDx; }, []); const handleMouseDown = useCallback((e: React.MouseEvent) => { @@ -269,6 +290,37 @@ export function EditableDraggableNumber({ startValue.current = value; lastClientX.current = e.clientX; dragStarted.current = false; + pointerLockRequested.current = false; + pointerLockActive.current = false; + + const element = spanRef.current; + + const handlePointerLockChange = () => { + pointerLockActive.current = element !== null && document.pointerLockElement === element; + }; + + document.addEventListener('pointerlockchange', handlePointerLockChange); + + if (element?.requestPointerLock) { + pointerLockRequested.current = true; + try { + const result = element.requestPointerLock(); + if (result && typeof result.then === 'function') { + void result.then( + () => { + pointerLockActive.current = document.pointerLockElement === element; + }, + () => { + pointerLockRequested.current = false; + pointerLockActive.current = false; + }, + ); + } + } catch { + pointerLockRequested.current = false; + pointerLockActive.current = false; + } + } const handleMouseMove = (event: MouseEvent) => { if ((event.buttons & 1) !== 1) { @@ -305,12 +357,19 @@ export function EditableDraggableNumber({ }; const handleMouseUp = () => { + const lockedElement = document.pointerLockElement; + if (element !== null && lockedElement === element) { + document.exitPointerLock?.(); + } + document.removeEventListener('pointerlockchange', handlePointerLockChange); window.removeEventListener('mousemove', handleMouseMove); window.removeEventListener('mouseup', handleMouseUp); if (dragStarted.current) { onDragEnd?.(); } dragStarted.current = false; + pointerLockRequested.current = false; + pointerLockActive.current = false; }; window.addEventListener('mousemove', handleMouseMove); diff --git a/src/components/panels/MediaPanel.tsx b/src/components/panels/MediaPanel.tsx index 98271fb7..94321dfb 100644 --- a/src/components/panels/MediaPanel.tsx +++ b/src/components/panels/MediaPanel.tsx @@ -30,7 +30,7 @@ import { // Column definitions type ColumnId = 'label' | 'name' | 'duration' | 'resolution' | 'fps' | 'container' | 'codec' | 'audio' | 'bitrate' | 'size'; type MediaPanelViewMode = 'classic' | 'icons' | 'board'; -type MediaBoardItem = Exclude; +type MediaBoardItem = ProjectItem; const CLASSIC_ROW_HEIGHT = 20; const CLASSIC_OVERSCAN_ROWS = 12; @@ -80,6 +80,7 @@ interface MediaBoardGroupLayout { height: number; itemCount: number; depth: number; + isDraggingPreview?: boolean; } interface MediaBoardNodePlacement { @@ -98,6 +99,40 @@ interface MediaBoardInsertGapPlacement { slotIndex: number; } +interface MediaBoardSlotPlacement { + id: string; + layout: MediaBoardNodeLayout; + groupId: string | null; + slotIndex: number; + itemId?: string; + isEmptySlot?: boolean; +} + +interface MediaBoardLayoutResult { + groups: MediaBoardGroupLayout[]; + placements: MediaBoardNodePlacement[]; + insertGaps: MediaBoardInsertGapPlacement[]; + slots: MediaBoardSlotPlacement[]; +} + +interface MediaBoardNodePlacementSnapshot { + itemId: string; + layout: MediaBoardNodeLayout; + defaultLayout: MediaBoardNodeLayout; + groupId: string | null; + slotIndex: number; + isDraggingPreview?: boolean; +} + +interface MediaBoardLayoutSnapshot { + version: number; + signature: string; + groups: MediaBoardGroupLayout[]; + placements: MediaBoardNodePlacementSnapshot[]; + insertGaps: MediaBoardInsertGapPlacement[]; + slots: MediaBoardSlotPlacement[]; +} + interface MediaBoardMarquee { startX: number; startY: number; @@ -108,7 +143,7 @@ interface MediaBoardMarquee { interface MediaBoardInsertionPreview { movingIds: string[]; targetGroupId: string | null; - targetIndex: number; + targetPosition: MediaBoardGroupOffset; sourceLayouts: Record; } @@ -138,8 +173,13 @@ const VIEW_MODE_STORAGE_KEY = 'media-panel-view-mode'; const BOARD_VIEWPORT_STORAGE_KEY = 'media-panel-board-viewport'; const BOARD_ORDER_STORAGE_KEY = 'media-panel-board-order'; const BOARD_GROUP_OFFSETS_STORAGE_KEY = 'media-panel-board-group-offsets'; +const BOARD_LAYOUTS_STORAGE_KEY = 'media-panel-board-layouts'; +const BOARD_LAYOUT_SNAPSHOT_STORAGE_KEY = 'media-panel-board-layout-snapshot'; +const BOARD_LAYOUT_SNAPSHOT_VERSION = 1; const MEDIA_PANEL_PROJECT_UI_LOADED_EVENT = 'media-panel-project-ui-loaded'; const MEDIA_BOARD_ROOT_ORDER_KEY = '__root__'; +const MEDIA_BOARD_EMPTY_SLOT_ID = '__media_board_empty_slot__'; +const MEDIA_BOARD_EMPTY_SLOT_SIZE_SEPARATOR = ':'; const DEFAULT_BOARD_VIEWPORT: MediaBoardViewport = { zoom: 0.82, panX: 32, panY: 28 }; const MEDIA_BOARD_NODE_TARGET_AREA = 20500; @@ -152,10 +192,15 @@ const MEDIA_BOARD_NODE_ASPECT_MAX = 2.75; const MEDIA_BOARD_NODE_GAP = 14; const MEDIA_BOARD_GROUP_HEADER_HEIGHT = 42; const MEDIA_BOARD_GROUP_PADDING = 18; -const MEDIA_BOARD_FOLDER_GAP = 28; const MEDIA_BOARD_GROUP_MIN_WIDTH = 260; const MEDIA_BOARD_GROUP_MAX_BODY_WIDTH = 700; const MEDIA_BOARD_FOLDER_ROW_MAX_WIDTH = 1480; +const MEDIA_BOARD_EMPTY_FOLDER_BODY_MIN_HEIGHT = 128; +const MEDIA_BOARD_EMPTY_SLOT_WIDTH = 192; +const MEDIA_BOARD_EMPTY_SLOT_HEIGHT = 108; +const MEDIA_BOARD_SLOT_CELL_WIDTH = 32; +const MEDIA_BOARD_SLOT_CELL_HEIGHT = 32; +const MEDIA_BOARD_ROOT_PADDING = 0; const MEDIA_BOARD_PAN_ZOOM_MIN = 0.18; const MEDIA_BOARD_PAN_ZOOM_MAX = 2.4; const MEDIA_BOARD_DRAG_START_DISTANCE = 4; @@ -163,9 +208,15 @@ const MEDIA_BOARD_GRID_PARALLAX = 0.18; const MEDIA_BOARD_AUTOPAN_EDGE_PX = 72; const MEDIA_BOARD_AUTOPAN_MAX_SPEED = 620; const MEDIA_BOARD_TIMELINE_HANDOFF_DISTANCE_PX = 96; -const MEDIA_BOARD_RENDER_BUFFER_PX = 760; -const MEDIA_BOARD_COMPACT_LOD_ZOOM = 0.34; -const MEDIA_BOARD_THUMBNAIL_LOD_MIN_ZOOM = 0.42; +const MEDIA_BOARD_RENDER_BUFFER_PX = 420; +const MEDIA_BOARD_COMPACT_RENDER_BUFFER_PX = 220; +const MEDIA_BOARD_COMPACT_LOD_ZOOM = 0.22; +const MEDIA_BOARD_OVERVIEW_CANVAS_ZOOM = 0.34; +const MEDIA_BOARD_THUMBNAIL_LOD_MIN_ZOOM = MEDIA_BOARD_OVERVIEW_CANVAS_ZOOM; +const MEDIA_BOARD_THUMBNAIL_REQUEST_MIN_ZOOM = MEDIA_BOARD_PAN_ZOOM_MIN; +const MEDIA_BOARD_THUMBNAIL_REQUEST_LIMIT = 180; +const MEDIA_BOARD_OVERVIEW_THUMBNAIL_REQUEST_LIMIT = 64; +const MEDIA_BOARD_THUMBNAIL_WORKER_COUNT = 2; const MEDIA_PANEL_VIEW_TRANSITION_MS = 500; interface MediaPanelTransitionBox { @@ -292,6 +343,27 @@ function loadMediaBoardGroupOffsets(): Record { } } +function loadMediaBoardLayouts(): Record { + try { + const stored = localStorage.getItem(BOARD_LAYOUTS_STORAGE_KEY); + if (!stored) return {}; + const parsed = JSON.parse(stored) as Record>; + if (!parsed || typeof parsed !== 'object') return {}; + + const valid: Record = {}; + Object.entries(parsed).forEach(([itemId, layout]) => { + if (!itemId || !layout || typeof layout !== 'object') return; + const x = Number(layout.x); + const y = Number(layout.y); + if (!Number.isFinite(x) || !Number.isFinite(y)) return; + valid[itemId] = { x, y }; + }); + return valid; + } catch { + return {}; + } +} + function getProjectItemIconType(item: ProjectItem | undefined): string | undefined { if (!item || !('type' in item)) return undefined; if (item.type === 'model') { @@ -310,6 +382,77 @@ function formatCompactCount(value: number | undefined): string | null { return `${(value / 1_000_000_000).toFixed(1)}B`; } +interface MediaSearchToken { + value: string; + glob?: RegExp; +} + +function escapeRegExp(value: string): string { + return value.replace(/[\\^$.*+?()[\]{}|]/g, '\\$&'); +} + +function globToRegExp(pattern: string): RegExp { + let source = '^'; + for (const char of pattern) { + if (char === '*') { + source += '.*'; + } else if (char === '?') { + source += '.'; + } else { + source += escapeRegExp(char); + } + } + source += '$'; + return new RegExp(source, 'i'); +} + +function createMediaSearchTokens(query: string): MediaSearchToken[] { + return query + .trim() + .split(/\s+/) + .filter(Boolean) + .map((token) => ({ + value: token.toLowerCase(), + glob: /[*?]/.test(token) ? globToRegExp(token) : undefined, + })); +} + +function getProjectItemSearchValues(item: ProjectItem): string[] { + const values = [item.name, 'isExpanded' in item ? 'folder' : '']; + if ('type' in item) { + values.push(item.type); + } + + if (isImportedMediaFileItem(item)) { + values.push( + item.file?.name ?? '', + item.file?.type ?? '', + item.codec ?? '', + item.audioCodec ?? '', + item.container ?? '', + item.filePath ?? '', + item.absolutePath ?? '', + item.projectPath ?? '', + ); + } + + return values.filter(Boolean); +} + +function projectItemMatchesMediaSearch(item: ProjectItem, tokens: MediaSearchToken[]): boolean { + if (tokens.length === 0) return true; + + const values = getProjectItemSearchValues(item); + const searchableText = values.join(' ').toLowerCase(); + + return tokens.every((token) => { + if (token.glob) { + return values.some((value) => token.glob!.test(value)); + } + return searchableText.includes(token.value); + }); +} + function getGaussianSplatFrameCount(mediaFile: MediaFile): number | undefined { return mediaFile.splatFrameCount ?? mediaFile.gaussianSplatSequence?.frameCount; } @@ -400,7 +543,38 @@ function getMediaBoardGroupName(folderId: string | null, folders: Array<{ id: st return path.length ? path.join(' / ') : 'Folder'; } +function isMediaBoardFolder(item: ProjectItem): item is MediaFolder { + return 'isExpanded' in item; +} + +function isMediaBoardEmptySlotId(id: string): boolean { + return id === MEDIA_BOARD_EMPTY_SLOT_ID || id.startsWith(`${MEDIA_BOARD_EMPTY_SLOT_ID}${MEDIA_BOARD_EMPTY_SLOT_SIZE_SEPARATOR}`); +} + +function normalizeMediaBoardOrderIds(ids: string[], validItemIds: Set): string[] { + const seenItemIds = new Set(); + const normalized: string[] = []; + + ids.forEach((id) => { + if (isMediaBoardEmptySlotId(id)) { + normalized.push(MEDIA_BOARD_EMPTY_SLOT_ID); + return; + } + + if (!validItemIds.has(id) || seenItemIds.has(id)) return; + seenItemIds.add(id); + normalized.push(id); + }); + + while (normalized.length > 0 && isMediaBoardEmptySlotId(normalized[normalized.length - 1])) { + normalized.pop(); + } + + return normalized.some((id) => !isMediaBoardEmptySlotId(id)) ? normalized : []; +} + function getMediaBoardTypeLabel(item: MediaBoardItem): string { + if (isMediaBoardFolder(item)) return 'Folder'; if (item.type === 'composition') return 'Composition'; if (item.type === 'gaussian-splat') { return isImportedMediaFileItem(item) && (getGaussianSplatFrameCount(item) ?? 1) > 1 @@ -422,6 +596,8 @@ function clampMediaBoardNumber(value: number, min: number, max: number): number } function getMediaBoardItemAspectRatio(item: MediaBoardItem): number { + if (isMediaBoardFolder(item)) return 16 / 9; + const width = 'width' in item ? Number(item.width) : undefined; const height = 'height' in item ? Number(item.height) : undefined; if (width && height && Number.isFinite(width) && Number.isFinite(height) && width > 0 && height > 0) { @@ -470,12 +646,20 @@ function getMediaBoardNodeSize(item: MediaBoardItem): { width: number; height: n }; } +function getMediaBoardGroupChrome(groupId: string | null): { headerHeight: number; padding: number } { + return groupId === null + ? { headerHeight: 0, padding: MEDIA_BOARD_ROOT_PADDING } + : { headerHeight: MEDIA_BOARD_GROUP_HEADER_HEIGHT, padding: MEDIA_BOARD_GROUP_PADDING }; +} + function getMediaBoardVisibleRect( viewport: MediaBoardViewport, viewportSize: MediaBoardViewportSize, ): MediaBoardVisibleRect { const zoom = Math.max(viewport.zoom, MEDIA_BOARD_PAN_ZOOM_MIN); - const buffer = MEDIA_BOARD_RENDER_BUFFER_PX; + const buffer = zoom <= MEDIA_BOARD_COMPACT_LOD_ZOOM + ? MEDIA_BOARD_COMPACT_RENDER_BUFFER_PX + : MEDIA_BOARD_RENDER_BUFFER_PX; return { left: (-viewport.panX - buffer) / zoom, @@ -485,6 +669,145 @@ function getMediaBoardVisibleRect( }; } +function hashMediaBoardSignature(value: string): string { + let hash = 2166136261; + for (let index = 0; index < value.length; index += 1) { + hash ^= value.charCodeAt(index); + hash = Math.imul(hash, 16777619); + } + return (hash >>> 0).toString(36); +} + +function createMediaBoardLayoutSignature( + items: MediaBoardItem[], + layouts: Record, +): string { + const itemPayload = items.map((item) => [ + item.id, + item.parentId ?? '', + isMediaBoardFolder(item) ? 'folder' : item.type, + 'width' in item ? item.width ?? 0 : 0, + 'height' in item ? item.height ?? 0 : 0, + ]); + const layoutPayload = Object.keys(layouts) + .sort() + .map((id) => [ + id, + Math.round((layouts[id]?.x ?? 0) * 100) / 100, + Math.round((layouts[id]?.y ?? 0) * 100) / 100, + ]); + + return `${BOARD_LAYOUT_SNAPSHOT_VERSION}:${hashMediaBoardSignature(JSON.stringify([itemPayload, layoutPayload]))}`; +} + +function restoreMediaBoardLayoutItems( + layout: MediaBoardLayoutResult, + itemsById: Map, + folders: MediaFolder[], +): MediaBoardLayoutResult { + const placements = layout.placements + .map((placement) => { + const item = itemsById.get(placement.item.id); + return item ? { ...placement, item } : null; + }) + .filter((placement): placement is MediaBoardNodePlacement => placement !== null); + + return { + groups: layout.groups.map((group) => ({ + ...group, + name: getMediaBoardGroupName(group.id, folders), + })), + placements, + insertGaps: layout.insertGaps, + slots: layout.slots, + }; +} + +function loadMediaBoardLayoutSnapshot( + signature: string, + itemsById: Map, + folders: MediaFolder[], +): MediaBoardLayoutResult | null { + try { + const stored = localStorage.getItem(BOARD_LAYOUT_SNAPSHOT_STORAGE_KEY); + if (!stored) return null; + + const snapshot = JSON.parse(stored) as Partial; + if ( + snapshot.version !== BOARD_LAYOUT_SNAPSHOT_VERSION + || snapshot.signature !== signature + || !Array.isArray(snapshot.groups) + || !Array.isArray(snapshot.placements) + || !Array.isArray(snapshot.insertGaps) + || !Array.isArray(snapshot.slots) + ) { + return null; + } + + const placements = snapshot.placements + .map((placement): MediaBoardNodePlacement | null => { + if (!placement || typeof placement.itemId !== 'string') return null; + const item = itemsById.get(placement.itemId); + if (!item || !placement.layout || !placement.defaultLayout) return null; + return { + item, + layout: placement.layout, + defaultLayout: placement.defaultLayout, + groupId: placement.groupId ?? null, + slotIndex: Number(placement.slotIndex) || 0, + isDraggingPreview: placement.isDraggingPreview, + }; + }) + .filter((placement): placement is MediaBoardNodePlacement => placement !== null); + + if (placements.length !== snapshot.placements.length) return null; + + return restoreMediaBoardLayoutItems({ + groups: snapshot.groups, + placements, + insertGaps: snapshot.insertGaps, + slots: snapshot.slots, + }, itemsById, folders); + } catch { + return null; + } +} + +function saveMediaBoardLayoutSnapshot(signature: string, layout: MediaBoardLayoutResult) { + try { + const snapshot: MediaBoardLayoutSnapshot = { + version: BOARD_LAYOUT_SNAPSHOT_VERSION, + signature, + groups: layout.groups.map((group) => ({ ...group, isDraggingPreview: undefined })), + placements: layout.placements.map((placement) => ({ + itemId: placement.item.id, + layout: placement.layout, + defaultLayout: placement.defaultLayout, + groupId: placement.groupId, + slotIndex: placement.slotIndex, + isDraggingPreview: undefined, + })), + insertGaps: layout.insertGaps, + slots: layout.slots, + }; + localStorage.setItem(BOARD_LAYOUT_SNAPSHOT_STORAGE_KEY, JSON.stringify(snapshot)); + } catch { + // Snapshot cache is an optimization only. + } +} + +function waitForMediaBoardThumbnailTurn(): Promise { + return new Promise((resolve) => { + const requestIdle = typeof window === 'undefined' ? undefined : window.requestIdleCallback; + if (typeof requestIdle === 'function') { + requestIdle(() => resolve(), { timeout: 120 }); + return; + } + + globalThis.setTimeout(resolve, 8); + }); +} + function mediaBoardNodeIntersectsVisibleRect( layout: MediaBoardNodeLayout, visibleRect: MediaBoardVisibleRect, @@ -509,6 +832,123 @@ function mediaBoardGroupIntersectsVisibleRect( ); } +function drawMediaBoardOverviewRoundedRect( + ctx: CanvasRenderingContext2D, + x: number, + y: number, + width: number, + height: number, + radius: number, +) { + const r = Math.max(0, Math.min(radius, width / 2, height / 2)); + ctx.beginPath(); + ctx.moveTo(x + r, y); + ctx.lineTo(x + width - r, y); + ctx.quadraticCurveTo(x + width, y, x + width, y + r); + ctx.lineTo(x + width, y + height - r); + ctx.quadraticCurveTo(x + width, y + height, x + width - r, y + height); + ctx.lineTo(x + r, y + height); + ctx.quadraticCurveTo(x, y + height, x, y + height - r); + ctx.lineTo(x, y + r); + ctx.quadraticCurveTo(x, y, x + r, y); + ctx.closePath(); +} + +function drawMediaBoardOverviewImageCover( + ctx: CanvasRenderingContext2D, + image: HTMLImageElement, + x: number, + y: number, + width: number, + height: number, +) { + const sourceWidth = image.naturalWidth || image.width; + const sourceHeight = image.naturalHeight || image.height; + if (sourceWidth <= 0 || sourceHeight <= 0) return; + + const scale = Math.max(width / sourceWidth, height / sourceHeight); + const drawWidth = sourceWidth * scale; + const drawHeight = sourceHeight * scale; + ctx.drawImage( + image, + x + (width - drawWidth) / 2, + y + (height - drawHeight) / 2, + drawWidth, + drawHeight, + ); +} + +function getMediaBoardOverviewFill(item: MediaBoardItem): string { + if (!('type' in item)) return 'rgba(32, 34, 38, 0.9)'; + if (item.type === 'solid' && 'color' in item) return item.color; + if (item.type === 'composition') return 'rgba(100, 58, 138, 0.82)'; + if (item.type === 'text') return 'rgba(46, 59, 78, 0.92)'; + if (item.type === 'camera') return 'rgba(139, 108, 45, 0.86)'; + if (item.type === 'image') return 'rgba(38, 64, 84, 0.88)'; + if (item.type === 'video') return 'rgba(45, 43, 64, 0.9)'; + if (item.type === 'audio') return 'rgba(52, 71, 55, 0.88)'; + return 'rgba(23, 24, 28, 0.94)'; +} + +function drawMediaBoardOverviewItem( + ctx: CanvasRenderingContext2D, + placement: MediaBoardNodePlacement, + image: HTMLImageElement | null, + zoom: number, + isDimmed = false, +) { + const { item, layout } = placement; + const screenWidth = layout.width * zoom; + const screenHeight = layout.height * zoom; + if (screenWidth < 1.2 || screenHeight < 1.2) return; + + ctx.save(); + if (isDimmed) { + ctx.globalAlpha = 0.24; + } + + const radius = Math.min(6, Math.max(1.5 / zoom, Math.min(layout.width, layout.height) * 0.06)); + drawMediaBoardOverviewRoundedRect(ctx, layout.x, layout.y, layout.width, layout.height, radius); + ctx.fillStyle = getMediaBoardOverviewFill(item); + ctx.fill(); + + if (image) { + ctx.save(); + drawMediaBoardOverviewRoundedRect(ctx, layout.x, layout.y, layout.width, layout.height, radius); + ctx.clip(); + drawMediaBoardOverviewImageCover(ctx, image, layout.x, layout.y, layout.width, layout.height); + ctx.restore(); + } else if (screenWidth >= 8 && screenHeight >= 8) { + const markWidth = Math.max(8 / zoom, layout.width * 0.28); + const markHeight = Math.max(5 / zoom, layout.height * 0.18); + ctx.fillStyle = 'rgba(255, 255, 255, 0.16)'; + drawMediaBoardOverviewRoundedRect( + ctx, + layout.x + (layout.width - markWidth) / 2, + layout.y + (layout.height - markHeight) / 2, + markWidth, + markHeight, + Math.min(3 / zoom, markHeight / 2), + ); + ctx.fill(); + } + + const labelHex = 'labelColor' in item ? getLabelHex(item.labelColor) : 'transparent'; + const borderWidth = Math.max(0.75 / zoom, 1); + ctx.lineWidth = borderWidth; + ctx.strokeStyle = labelHex === 'transparent' ? 'rgba(255, 255, 255, 0.1)' : labelHex; + drawMediaBoardOverviewRoundedRect( + ctx, + layout.x + borderWidth / 2, + layout.y + borderWidth / 2, + Math.max(0, layout.width - borderWidth), + Math.max(0, layout.height - borderWidth), + radius, + ); + ctx.stroke(); + ctx.restore(); +} + function rectToTransitionBox(rect: DOMRect): MediaPanelTransitionBox { return { left: rect.left, @@ -596,8 +1036,11 @@ export function MediaPanel() { const boardWrapperRef = useRef(null); const boardCanvasRef = useRef(null); const boardCanvasInnerRef = useRef(null); + const boardOverviewCanvasRef = useRef(null); const boardInteractionFrameRef = useRef(null); const boardAutoPanFrameRef = useRef(null); + const boardOverviewRedrawFrameRef = useRef(null); + const boardOverviewImageCacheRef = useRef(new Map()); const pendingViewTransitionRef = useRef(null); const activeViewTransitionRef = useRef(null); const [renamingId, setRenamingId] = useState(null); @@ -619,15 +1062,20 @@ export function MediaPanel() { const [labelPickerItemId, setLabelPickerItemId] = useState(null); const [labelPickerPos, setLabelPickerPos] = useState<{ x: number; y: number } | null>(null); const [viewMode, setViewMode] = useState(loadMediaPanelViewMode); + const [mediaSearchQuery, setMediaSearchQuery] = useState(''); // Grid view: current open folder (null = root) const [gridFolderId, setGridFolderId] = useState(null); const [mediaBoardViewport, setMediaBoardViewport] = useState(loadMediaBoardViewport); + const mediaBoardViewportRef = useRef(mediaBoardViewport); + const boardWheelCommitTimerRef = useRef(null); const [mediaBoardOrder, setMediaBoardOrder] = useState>(loadMediaBoardOrder); const [mediaBoardGroupOffsets, setMediaBoardGroupOffsets] = useState>(loadMediaBoardGroupOffsets); + const [mediaBoardLayouts, setMediaBoardLayouts] = useState>(loadMediaBoardLayouts); const [mediaBoardCanvasSize, setMediaBoardCanvasSize] = useState(() => ({ width: typeof window === 'undefined' ? 1280 : Math.max(1, window.innerWidth), height: typeof window === 'undefined' ? 720 : Math.max(1, window.innerHeight), })); + const [mediaBoardOverviewImageVersion, setMediaBoardOverviewImageVersion] = useState(0); const [mediaBoardMarquee, setMediaBoardMarquee] = useState(null); const [mediaBoardInsertionPreview, setMediaBoardInsertionPreview] = useState(null); const suppressMediaBoardContextMenuRef = useRef(false); @@ -652,9 +1100,17 @@ export function MediaPanel() { }, [viewMode]); useEffect(() => { + mediaBoardViewportRef.current = mediaBoardViewport; localStorage.setItem(BOARD_VIEWPORT_STORAGE_KEY, JSON.stringify(mediaBoardViewport)); }, [mediaBoardViewport]); + useEffect(() => () => { + if (boardWheelCommitTimerRef.current !== null) { + window.clearTimeout(boardWheelCommitTimerRef.current); + boardWheelCommitTimerRef.current = null; + } + }, []); + useEffect(() => { localStorage.setItem(BOARD_ORDER_STORAGE_KEY, JSON.stringify(mediaBoardOrder)); }, [mediaBoardOrder]); @@ -663,6 +1119,10 @@ export function MediaPanel() { localStorage.setItem(BOARD_GROUP_OFFSETS_STORAGE_KEY, JSON.stringify(mediaBoardGroupOffsets)); }, [mediaBoardGroupOffsets]); + useEffect(() => { + localStorage.setItem(BOARD_LAYOUTS_STORAGE_KEY, JSON.stringify(mediaBoardLayouts)); + }, [mediaBoardLayouts]); + useLayoutEffect(() => { if (viewMode !== 'board') return; @@ -727,6 +1187,9 @@ export function MediaPanel() { if (boardAutoPanFrameRef.current !== null) { window.cancelAnimationFrame(boardAutoPanFrameRef.current); } + if (boardOverviewRedrawFrameRef.current !== null) { + window.cancelAnimationFrame(boardOverviewRedrawFrameRef.current); + } if (suppressMediaBoardContextMenuTimerRef.current !== null) { window.clearTimeout(suppressMediaBoardContextMenuTimerRef.current); } @@ -1866,6 +2329,7 @@ export function MediaPanel() { setMediaBoardViewport(loadMediaBoardViewport()); setMediaBoardOrder(loadMediaBoardOrder()); setMediaBoardGroupOffsets(loadMediaBoardGroupOffsets()); + setMediaBoardLayouts(loadMediaBoardLayouts()); const storedNameWidth = localStorage.getItem('media-panel-name-width'); setNameColumnWidth(storedNameWidth ? parseInt(storedNameWidth, 10) : 250); setGridFolderId(null); @@ -2246,20 +2710,58 @@ export function MediaPanel() { ); }; - const totalItems = ( - files.length + - compositions.length + - folders.length + - textItems.length + - solidItems.length + - meshItems.length + - cameraItems.length + - splatEffectorItems.length - ); + const allProjectItems = useMemo(() => ([ + ...files, + ...compositions, + ...folders, + ...textItems, + ...solidItems, + ...meshItems, + ...cameraItems, + ...splatEffectorItems, + ]), [files, compositions, folders, textItems, solidItems, meshItems, cameraItems, splatEffectorItems]); + + const projectListItems = useMemo(() => ([ + ...folders, + ...compositions, + ...textItems, + ...solidItems, + ...meshItems, + ...cameraItems, + ...splatEffectorItems, + ...files, + ]), [folders, compositions, textItems, solidItems, meshItems, cameraItems, splatEffectorItems, files]); + + const allProjectItemsById = useMemo(() => new Map(allProjectItems.map((item) => [item.id, item])), [allProjectItems]); + const totalItems = allProjectItems.length; + const mediaSearchTokens = useMemo(() => createMediaSearchTokens(mediaSearchQuery), [mediaSearchQuery]); + const isMediaSearchActive = mediaSearchTokens.length > 0; + const mediaSearchDirectMatches = useMemo(() => ( + isMediaSearchActive + ? projectListItems.filter((item) => projectItemMatchesMediaSearch(item, mediaSearchTokens)) + : projectListItems + ), [isMediaSearchActive, mediaSearchTokens, projectListItems]); + const mediaSearchDirectMatchIds = useMemo(() => new Set(mediaSearchDirectMatches.map((item) => item.id)), [mediaSearchDirectMatches]); + const mediaSearchVisibleItemIds = useMemo(() => { + if (!isMediaSearchActive) return null; + + const visibleIds = new Set(mediaSearchDirectMatchIds); + mediaSearchDirectMatches.forEach((item) => { + let parentId = item.parentId ?? null; + while (parentId) { + visibleIds.add(parentId); + parentId = allProjectItemsById.get(parentId)?.parentId ?? null; + } + }); + return visibleIds; + }, [allProjectItemsById, isMediaSearchActive, mediaSearchDirectMatchIds, mediaSearchDirectMatches]); + const mediaSearchResultCount = isMediaSearchActive ? mediaSearchDirectMatches.length : totalItems; const projectItemsByParentId = useMemo(() => { const itemsByParentId = new Map(); const append = (item: ProjectItem) => { + if (mediaSearchVisibleItemIds && !mediaSearchVisibleItemIds.has(item.id)) return; + const parentId = item.parentId ?? null; const items = itemsByParentId.get(parentId); if (items) { @@ -2269,17 +2771,10 @@ export function MediaPanel() { } }; - folders.forEach(append); - compositions.forEach(append); - textItems.forEach(append); - solidItems.forEach(append); - meshItems.forEach(append); - cameraItems.forEach(append); - splatEffectorItems.forEach(append); - files.forEach(append); + projectListItems.forEach(append); return itemsByParentId; - }, [files, compositions, folders, textItems, solidItems, meshItems, cameraItems, splatEffectorItems]); + }, [mediaSearchVisibleItemIds, projectListItems]); const getItemsForParent = useCallback( (parentId: string | null) => projectItemsByParentId.get(parentId) ?? [], @@ -2292,7 +2787,7 @@ export function MediaPanel() { const appendRows = (items: ProjectItem[], depth: number) => { for (const item of sortItems(items)) { rows.push({ item, depth }); - if ('isExpanded' in item && classicExpandedFolderIdSet.has(item.id)) { + if ('isExpanded' in item && (classicExpandedFolderIdSet.has(item.id) || isMediaSearchActive)) { appendRows(getItemsForParent(item.id), depth + 1); } } @@ -2304,6 +2799,7 @@ export function MediaPanel() { sortItems, getItemsForParent, classicExpandedFolderIdSet, + isMediaSearchActive, ]); const classicVisibleRange = useMemo(() => { @@ -2322,18 +2818,58 @@ export function MediaPanel() { const classicTopSpacerHeight = classicVisibleRange.start * CLASSIC_ROW_HEIGHT; const classicBottomSpacerHeight = Math.max(0, (classicRows.length - classicVisibleRange.end) * CLASSIC_ROW_HEIGHT); - const mediaBoardItems = useMemo(() => ([ - ...files, - ...compositions, - ...textItems, - ...solidItems, - ...meshItems, - ...cameraItems, - ...splatEffectorItems, - ]), [files, compositions, textItems, solidItems, meshItems, cameraItems, splatEffectorItems]); + const mediaBoardItems = allProjectItems; const mediaBoardItemIds = useMemo(() => new Set(mediaBoardItems.map((item) => item.id)), [mediaBoardItems]); + const mediaBoardItemsById = useMemo(() => new Map(mediaBoardItems.map((item) => [item.id, item])), [mediaBoardItems]); + const mediaBoardFoldersById = useMemo(() => new Map(folders.map((folder) => [folder.id, folder])), [folders]); const selectedIdSet = useMemo(() => new Set(selectedIds), [selectedIds]); + const mediaBoardLayoutSignature = useMemo( + () => createMediaBoardLayoutSignature(mediaBoardItems, mediaBoardLayouts), + [mediaBoardItems, mediaBoardLayouts], + ); + const mediaBoardInsertionPreviewKey = useMemo(() => { + if (!mediaBoardInsertionPreview) return ''; + return JSON.stringify([ + mediaBoardInsertionPreview.movingIds, + mediaBoardInsertionPreview.targetGroupId, + mediaBoardInsertionPreview.targetPosition.x, + mediaBoardInsertionPreview.targetPosition.y, + ]); + }, [mediaBoardInsertionPreview]); + const mediaBoardGeometryInputRef = useRef({ + mediaBoardItems, + folders, + mediaBoardLayouts, + mediaBoardInsertionPreview, + }); + mediaBoardGeometryInputRef.current = { + mediaBoardItems, + folders, + mediaBoardLayouts, + mediaBoardInsertionPreview, + }; + + const getMediaBoardTopLevelMoveIds = useCallback((itemIds: string[]) => { + const requestedIds = new Set(itemIds.filter((id) => mediaBoardItemIds.has(id))); + const seenIds = new Set(); + + const hasSelectedAncestor = (itemId: string) => { + const item = mediaBoardItemsById.get(itemId); + let parentId = item?.parentId ?? null; + while (parentId) { + if (requestedIds.has(parentId)) return true; + parentId = mediaBoardFoldersById.get(parentId)?.parentId ?? null; + } + return false; + }; + + return itemIds.filter((id) => { + if (!requestedIds.has(id) || seenIds.has(id) || hasSelectedAncestor(id)) return false; + seenIds.add(id); + return true; + }); + }, [mediaBoardFoldersById, mediaBoardItemIds, mediaBoardItemsById]); useEffect(() => { setMediaBoardOrder((current) => { @@ -2350,7 +2886,7 @@ export function MediaPanel() { return; } - const filteredIds = ids.filter((id, index) => mediaBoardItemIds.has(id) && ids.indexOf(id) === index); + const filteredIds = normalizeMediaBoardOrderIds(ids, mediaBoardItemIds); if (filteredIds.length !== ids.length) { changed = true; } @@ -2381,7 +2917,192 @@ export function MediaPanel() { }); }, [folders]); - const mediaBoardLayout = useMemo(() => { + useEffect(() => { + setMediaBoardLayouts((current) => { + const { mediaBoardItems } = mediaBoardGeometryInputRef.current; + const currentMediaBoardItemIds = new Set(mediaBoardItems.map((item) => item.id)); + const columnPitch = MEDIA_BOARD_SLOT_CELL_WIDTH; + const rowPitch = MEDIA_BOARD_SLOT_CELL_HEIGHT; + let changed = false; + const next: Record = {}; + const usedSlotsByGroup = new Map>(); + const itemsByParent = new Map(); + + mediaBoardItems.forEach((item) => { + const parentId = item.parentId ?? null; + const siblings = itemsByParent.get(parentId) ?? []; + siblings.push(item); + itemsByParent.set(parentId, siblings); + }); + + const canPlace = ( + usedSlots: Set, + column: number, + row: number, + span: { columns: number; rows: number }, + columnCount: number, + ) => { + if (column + span.columns > columnCount) return false; + for (let y = row; y < row + span.rows; y += 1) { + for (let x = column; x < column + span.columns; x += 1) { + if (usedSlots.has(`${x}:${y}`)) return false; + } + } + return true; + }; + + const markSpan = ( + usedSlots: Set, + column: number, + row: number, + span: { columns: number; rows: number }, + ) => { + for (let y = row; y < row + span.rows; y += 1) { + for (let x = column; x < column + span.columns; x += 1) { + usedSlots.add(`${x}:${y}`); + } + } + }; + + const getSpanForSize = (size: { width: number; height: number }) => ({ + columns: Math.max(1, Math.ceil((size.width + MEDIA_BOARD_NODE_GAP) / columnPitch)), + rows: Math.max(1, Math.ceil((size.height + MEDIA_BOARD_NODE_GAP) / rowPitch)), + }); + + const getPackColumnsForSpans = (groupId: string | null, spans: Array<{ columns: number; rows: number }>) => { + if (spans.length === 0) return 1; + const widestItem = Math.max(1, ...spans.map((span) => span.columns)); + const totalCells = spans.reduce((sum, span) => sum + (span.columns * span.rows), 0); + const targetColumns = Math.ceil(Math.sqrt(totalCells) * (groupId === null ? 1.35 : 1.22)); + const hardMaxColumns = groupId === null ? 128 : 84; + return Math.max(widestItem, Math.min(hardMaxColumns, targetColumns)); + }; + + const packSpans = ( + spans: Array<{ columns: number; rows: number }>, + columnCount: number, + ) => { + const usedSlots = new Set(); + let maxColumn = 0; + let maxRow = 0; + + spans.forEach((span) => { + let slotIndex = 0; + while (!canPlace(usedSlots, slotIndex % columnCount, Math.floor(slotIndex / columnCount), span, columnCount)) { + slotIndex += 1; + } + const column = slotIndex % columnCount; + const row = Math.floor(slotIndex / columnCount); + markSpan(usedSlots, column, row, span); + maxColumn = Math.max(maxColumn, column + span.columns); + maxRow = Math.max(maxRow, row + span.rows); + }); + + return { + width: maxColumn * columnPitch, + height: maxRow * rowPitch, + }; + }; + + const estimatedSizeCache = new Map(); + const estimateBoardItemSize = (item: MediaBoardItem, stack: Set = new Set()): { width: number; height: number } => { + if (!isMediaBoardFolder(item)) { + return getMediaBoardNodeSize(item); + } + + const cached = estimatedSizeCache.get(item.id); + if (cached) return cached; + + if (stack.has(item.id)) { + return { + width: MEDIA_BOARD_GROUP_MIN_WIDTH, + height: MEDIA_BOARD_GROUP_HEADER_HEIGHT + (MEDIA_BOARD_GROUP_PADDING * 2) + MEDIA_BOARD_EMPTY_FOLDER_BODY_MIN_HEIGHT, + }; + } + + const nextStack = new Set(stack); + nextStack.add(item.id); + const children = sortItems([...(itemsByParent.get(item.id) ?? [])]) as MediaBoardItem[]; + const childSpans = children.map((child) => getSpanForSize(estimateBoardItemSize(child, nextStack))); + const body = childSpans.length > 0 + ? packSpans(childSpans, getPackColumnsForSpans(item.id, childSpans)) + : { width: 0, height: MEDIA_BOARD_EMPTY_FOLDER_BODY_MIN_HEIGHT }; + const estimated = { + width: Math.max(MEDIA_BOARD_GROUP_MIN_WIDTH, Math.ceil(body.width + (MEDIA_BOARD_GROUP_PADDING * 2))), + height: MEDIA_BOARD_GROUP_HEADER_HEIGHT + (MEDIA_BOARD_GROUP_PADDING * 2) + Math.max(body.height, MEDIA_BOARD_EMPTY_FOLDER_BODY_MIN_HEIGHT), + }; + estimatedSizeCache.set(item.id, estimated); + return estimated; + }; + + const getSpan = (item: MediaBoardItem) => getSpanForSize(estimateBoardItemSize(item)); + + const markUsed = (groupId: string | null, position: MediaBoardGroupOffset, span: { columns: number; rows: number }) => { + const usedSlots = usedSlotsByGroup.get(groupId) ?? new Set(); + const column = Math.max(0, Math.round(position.x / columnPitch)); + const row = Math.max(0, Math.round(position.y / rowPitch)); + markSpan(usedSlots, column, row, span); + usedSlotsByGroup.set(groupId, usedSlots); + }; + + mediaBoardItems.forEach((item) => { + const parentId = item.parentId ?? null; + const layout = current[item.id]; + if (!layout) return; + next[item.id] = layout; + markUsed(parentId, layout, getSpan(item)); + }); + + Object.keys(current).forEach((itemId) => { + if (!currentMediaBoardItemIds.has(itemId)) { + changed = true; + } + }); + + itemsByParent.forEach((items, parentId) => { + const sortedItems = sortItems([...items]) as MediaBoardItem[]; + const columnCount = getPackColumnsForSpans(parentId, sortedItems.map(getSpan)); + sortedItems.forEach((item) => { + if (next[item.id]) return; + + const usedSlots = usedSlotsByGroup.get(parentId) ?? new Set(); + const span = getSpan(item); + let slotIndex = 0; + while (!canPlace(usedSlots, slotIndex % columnCount, Math.floor(slotIndex / columnCount), span, columnCount)) { + slotIndex += 1; + } + + const position = { + x: (slotIndex % columnCount) * columnPitch, + y: Math.floor(slotIndex / columnCount) * rowPitch, + }; + next[item.id] = position; + markUsed(parentId, position, span); + changed = true; + }); + }); + + if (Object.keys(next).length !== Object.keys(current).length) { + changed = true; + } + + return changed ? next : current; + }); + }, [mediaBoardLayoutSignature, sortItems]); + + const mediaBoardLayoutGeometry = useMemo(() => { + const { + mediaBoardItems, + folders, + mediaBoardLayouts, + mediaBoardInsertionPreview, + } = mediaBoardGeometryInputRef.current; + const itemsById = new Map(mediaBoardItems.map((item) => [item.id, item])); + if (!mediaBoardInsertionPreviewKey) { + const snapshot = loadMediaBoardLayoutSnapshot(mediaBoardLayoutSignature, itemsById, folders); + if (snapshot) return snapshot; + } + const groupsByParent = new Map(); groupsByParent.set(null, []); folders.forEach((folder) => groupsByParent.set(folder.id, [])); @@ -2394,10 +3115,10 @@ export function MediaPanel() { } foldersByParent.get(parentId)!.push(folder); }); - const itemsById = new Map(mediaBoardItems.map((item) => [item.id, item])); const movingIdSet = new Set(mediaBoardInsertionPreview?.movingIds ?? []); mediaBoardItems.forEach((item) => { + if (isMediaBoardFolder(item)) return; const parentId = item.parentId ?? null; if (!groupsByParent.has(parentId)) { groupsByParent.set(parentId, []); @@ -2408,29 +3129,20 @@ export function MediaPanel() { const groups: MediaBoardGroupLayout[] = []; const placements: MediaBoardNodePlacement[] = []; const insertGaps: MediaBoardInsertGapPlacement[] = []; - - const orderItemsForGroup = (groupId: string | null, items: MediaBoardItem[]): MediaBoardItem[] => { - const sortedItems = sortItems([...items]) as MediaBoardItem[]; - const savedOrder = mediaBoardOrder[getMediaBoardOrderKey(groupId)] ?? []; - if (savedOrder.length === 0) return sortedItems; - - const byId = new Map(sortedItems.map((item) => [item.id, item])); - const orderedItems = savedOrder - .map((id) => byId.get(id)) - .filter((item): item is MediaBoardItem => Boolean(item)); - const orderedIds = new Set(orderedItems.map((item) => item.id)); - return [ - ...orderedItems, - ...sortedItems.filter((item) => !orderedIds.has(item.id)), - ]; - }; + const slots: MediaBoardSlotPlacement[] = []; type MediaBoardLayoutEntry = { id: string; item?: MediaBoardItem; width: number; height: number; + desiredX: number; + desiredY: number; isInsertGap: boolean; + isEmptySlot?: boolean; + offsetX?: number; + offsetY?: number; + resolvedSlotIndex?: number; }; type MediaBoardLayoutRow = { @@ -2439,111 +3151,150 @@ export function MediaPanel() { height: number; }; - type MediaBoardFolderBlock = { - folder: MediaFolder; - width: number; - height: number; - desiredX: number; - desiredY: number; - }; - type MediaBoardGroupMeasure = { width: number; height: number; itemRows: MediaBoardLayoutRow[]; - folderRows: Array<{ entries: MediaBoardFolderBlock[]; width: number; height: number }>; itemCount: number; bodyHeight: number; }; - const getEntriesForGroup = (groupId: string | null): MediaBoardLayoutEntry[] => { - const entries = orderItemsForGroup(groupId, groupsByParent.get(groupId) ?? []) - .filter((item) => !movingIdSet.has(item.id)) - .map((item) => ({ - id: item.id, - item, - ...getMediaBoardNodeSize(item), - isInsertGap: false, - })); + const getDirectBoardItems = (groupId: string | null): MediaBoardItem[] => [ + ...(groupsByParent.get(groupId) ?? []), + ...(foldersByParent.get(groupId) ?? []), + ]; - if (!mediaBoardInsertionPreview || mediaBoardInsertionPreview.targetGroupId !== groupId) { - return entries; + function getLayoutSizeForItem(item: MediaBoardItem, stack: Set): { width: number; height: number } { + if (!isMediaBoardFolder(item)) { + return getMediaBoardNodeSize(item); } - const gapEntries = mediaBoardInsertionPreview.movingIds - .map((id, index) => { - const item = itemsById.get(id); - if (!item) return null; - return { - id: `insert-gap-${id}-${index}`, - ...getMediaBoardNodeSize(item), - isInsertGap: true, - }; - }) - .filter((entry): entry is MediaBoardLayoutEntry => Boolean(entry)); - - if (gapEntries.length === 0) { - return entries; + if (stack.has(item.id)) { + return { + width: MEDIA_BOARD_GROUP_MIN_WIDTH, + height: MEDIA_BOARD_GROUP_HEADER_HEIGHT + (MEDIA_BOARD_GROUP_PADDING * 2) + MEDIA_BOARD_NODE_MIN_HEIGHT, + }; } - const insertIndex = Math.max(0, Math.min(mediaBoardInsertionPreview.targetIndex, entries.length)); - return [ - ...entries.slice(0, insertIndex), - ...gapEntries, - ...entries.slice(insertIndex), - ]; + const measure = measureGroup(item.id, stack); + return { width: measure.width, height: measure.height }; + } + + const getEntriesForGroup = (groupId: string | null, stack: Set): MediaBoardLayoutEntry[] => { + const columnPitch = MEDIA_BOARD_SLOT_CELL_WIDTH; + const entries: MediaBoardLayoutEntry[] = []; + + getDirectBoardItems(groupId).forEach((item) => { + if (movingIdSet.has(item.id)) return; + const position = mediaBoardLayouts[item.id]; + if (!position) return; + entries.push({ + id: item.id, + item, + ...getLayoutSizeForItem(item, stack), + desiredX: position.x, + desiredY: position.y, + isInsertGap: false, + }); + }); + + if (mediaBoardInsertionPreview?.targetGroupId === groupId) { + mediaBoardInsertionPreview.movingIds.forEach((id, index) => { + const item = itemsById.get(id); + if (!item) return; + entries.push({ + id: `insert-gap-${id}-${index}`, + ...getLayoutSizeForItem(item, stack), + desiredX: mediaBoardInsertionPreview.targetPosition.x + (index * columnPitch), + desiredY: mediaBoardInsertionPreview.targetPosition.y, + isInsertGap: true, + }); + }); + } + + return entries; }; - function wrapEntriesIntoRows( + function placeEntriesOnGrid( entries: T[], - maxRowWidth: number, + maxBodyWidth: number, + allowNegativePositions: boolean, ): Array<{ entries: T[]; width: number; height: number }> { - const rows: Array<{ entries: T[]; width: number; height: number }> = []; - let currentEntries: T[] = []; - let currentWidth = 0; - let currentHeight = 0; - - const flushRow = () => { - if (currentEntries.length === 0) return; - rows.push({ - entries: currentEntries, - width: currentWidth, - height: currentHeight, - }); - currentEntries = []; - currentWidth = 0; - currentHeight = 0; + const columnPitch = MEDIA_BOARD_SLOT_CELL_WIDTH; + const rowPitch = MEDIA_BOARD_SLOT_CELL_HEIGHT; + const occupied = new Set(); + const rowsByIndex = new Map>>>(); + + const getSpan = (entry: T) => ({ + columns: Math.max(1, Math.ceil((entry.width + MEDIA_BOARD_NODE_GAP) / columnPitch)), + rows: Math.max(1, Math.ceil((entry.height + MEDIA_BOARD_NODE_GAP) / rowPitch)), + }); + const columnCount = Math.max( + 1, + Math.floor(maxBodyWidth / columnPitch), + ...entries.map((entry) => Math.max(0, Math.round(entry.desiredX / columnPitch)) + getSpan(entry).columns), + ); + + const canPlace = (column: number, row: number, span: { columns: number; rows: number }) => { + if (!allowNegativePositions && (column < 0 || row < 0)) return false; + if (column + span.columns > columnCount) return false; + for (let y = row; y < row + span.rows; y += 1) { + for (let x = column; x < column + span.columns; x += 1) { + if (occupied.has(`${x}:${y}`)) return false; + } + } + return true; + }; + + const markOccupied = (column: number, row: number, span: { columns: number; rows: number }) => { + for (let y = row; y < row + span.rows; y += 1) { + for (let x = column; x < column + span.columns; x += 1) { + occupied.add(`${x}:${y}`); + } + } }; entries.forEach((entry) => { - const nextWidth = currentEntries.length === 0 - ? entry.width - : currentWidth + MEDIA_BOARD_NODE_GAP + entry.width; - if (currentEntries.length > 0 && nextWidth > maxRowWidth) { - flushRow(); + const span = getSpan(entry); + const initialColumn = allowNegativePositions + ? Math.round(entry.desiredX / columnPitch) + : Math.max(0, Math.round(entry.desiredX / columnPitch)); + const initialRow = allowNegativePositions + ? Math.round(entry.desiredY / rowPitch) + : Math.max(0, Math.round(entry.desiredY / rowPitch)); + let column = initialColumn; + let row = initialRow; + while (!canPlace(column, row, span)) { + column += 1; + if (column + span.columns > columnCount) { + row += 1; + column = allowNegativePositions ? initialColumn : 0; + } } + markOccupied(column, row, span); - currentWidth = currentEntries.length === 0 - ? entry.width - : currentWidth + MEDIA_BOARD_NODE_GAP + entry.width; - currentHeight = Math.max(currentHeight, entry.height); - currentEntries.push(entry); + const placedEntry = { + ...entry, + offsetX: column * columnPitch, + offsetY: row * rowPitch, + resolvedSlotIndex: (row * 100000) + column, + }; + const rowEntries = rowsByIndex.get(row) ?? []; + rowEntries.push(placedEntry); + rowsByIndex.set(row, rowEntries); }); - flushRow(); - return rows; + return [...rowsByIndex.entries()] + .sort(([a], [b]) => a - b) + .map(([, rowEntries]) => ({ + entries: rowEntries.sort((a, b) => (a.offsetX - b.offsetX) || (a.resolvedSlotIndex - b.resolvedSlotIndex)), + width: Math.max(0, ...rowEntries.map((entry) => entry.offsetX + entry.width)), + height: Math.max(0, ...rowEntries.map((entry) => entry.offsetY + entry.height)), + })); } - const getRowsHeight = (rows: Array<{ height: number }>, gap: number): number => { - return rows.reduce((height, row, index) => height + row.height + (index > 0 ? gap : 0), 0); - }; - - const getChildFolders = (parentId: string | null): MediaFolder[] => { - return [...(foldersByParent.get(parentId) ?? [])].sort((a, b) => a.name.localeCompare(b.name)); - }; - const measureCache = new Map(); - const measureGroup = (groupId: string | null, stack: Set = new Set()): MediaBoardGroupMeasure => { + function measureGroup(groupId: string | null, stack: Set = new Set()): MediaBoardGroupMeasure { const cacheKey = getMediaBoardOrderKey(groupId); const cached = measureCache.get(cacheKey); if (cached) return cached; @@ -2551,11 +3302,10 @@ export function MediaPanel() { if (groupId && stack.has(groupId)) { return { width: MEDIA_BOARD_GROUP_MIN_WIDTH, - height: MEDIA_BOARD_GROUP_HEADER_HEIGHT + (MEDIA_BOARD_GROUP_PADDING * 2) + MEDIA_BOARD_NODE_MIN_HEIGHT, + height: MEDIA_BOARD_GROUP_HEADER_HEIGHT + (MEDIA_BOARD_GROUP_PADDING * 2) + MEDIA_BOARD_EMPTY_FOLDER_BODY_MIN_HEIGHT, itemRows: [], - folderRows: [], itemCount: 0, - bodyHeight: MEDIA_BOARD_NODE_MIN_HEIGHT, + bodyHeight: MEDIA_BOARD_EMPTY_FOLDER_BODY_MIN_HEIGHT, }; } @@ -2564,46 +3314,32 @@ export function MediaPanel() { nextStack.add(groupId); } - const itemRows = wrapEntriesIntoRows(getEntriesForGroup(groupId), MEDIA_BOARD_GROUP_MAX_BODY_WIDTH) as MediaBoardLayoutRow[]; - const childFolderBlocks = getChildFolders(groupId) - .map((folder, index) => { - const childMeasure = measureGroup(folder.id, nextStack); - const offset = mediaBoardGroupOffsets[folder.id] ?? { x: 0, y: 0 }; - return { - folder, - width: childMeasure.width, - height: childMeasure.height, - desiredX: Math.max(0, (index * (MEDIA_BOARD_GROUP_MIN_WIDTH + MEDIA_BOARD_FOLDER_GAP)) + offset.x), - desiredY: Math.max(0, offset.y), - }; - }) - .sort((a, b) => (a.desiredY - b.desiredY) || (a.desiredX - b.desiredX) || a.folder.name.localeCompare(b.folder.name)); - const folderRows = wrapEntriesIntoRows(childFolderBlocks, MEDIA_BOARD_FOLDER_ROW_MAX_WIDTH); - const itemRowsHeight = getRowsHeight(itemRows, MEDIA_BOARD_NODE_GAP); - const folderRowsHeight = getRowsHeight(folderRows, MEDIA_BOARD_FOLDER_GAP); + const maxBodyWidth = groupId === null ? MEDIA_BOARD_FOLDER_ROW_MAX_WIDTH : MEDIA_BOARD_GROUP_MAX_BODY_WIDTH; + const itemRows = placeEntriesOnGrid(getEntriesForGroup(groupId, nextStack), maxBodyWidth, groupId === null) as MediaBoardLayoutRow[]; const hasItems = itemRows.length > 0; - const hasFolders = folderRows.length > 0; - const bodyWidth = Math.max( - 0, - ...itemRows.map((row) => row.width), - ...folderRows.map((row) => row.width), - ); - const bodyHeight = hasItems || hasFolders - ? itemRowsHeight + folderRowsHeight + (hasItems && hasFolders ? MEDIA_BOARD_FOLDER_GAP : 0) - : MEDIA_BOARD_NODE_MIN_HEIGHT; + const bodyWidth = Math.max(0, ...itemRows.map((row) => row.width)); + const bodyHeight = hasItems ? Math.max(0, ...itemRows.map((row) => row.height)) : MEDIA_BOARD_EMPTY_FOLDER_BODY_MIN_HEIGHT; + const chrome = getMediaBoardGroupChrome(groupId); + const minWidth = groupId === null ? Math.max(MEDIA_BOARD_GROUP_MAX_BODY_WIDTH, bodyWidth) : MEDIA_BOARD_GROUP_MIN_WIDTH; const measure: MediaBoardGroupMeasure = { - width: Math.max(MEDIA_BOARD_GROUP_MIN_WIDTH, Math.ceil(bodyWidth + (MEDIA_BOARD_GROUP_PADDING * 2))), - height: MEDIA_BOARD_GROUP_HEADER_HEIGHT + (MEDIA_BOARD_GROUP_PADDING * 2) + bodyHeight, + width: Math.max(minWidth, Math.ceil(bodyWidth + (chrome.padding * 2))), + height: chrome.headerHeight + (chrome.padding * 2) + bodyHeight, itemRows, - folderRows, - itemCount: (groupsByParent.get(groupId)?.length ?? 0) + (foldersByParent.get(groupId)?.length ?? 0), + itemCount: getDirectBoardItems(groupId).length, bodyHeight, }; measureCache.set(cacheKey, measure); return measure; - }; + } - const placeGroup = (groupId: string | null, x: number, y: number, depth: number, parentId: string | null) => { + const placeGroup = ( + groupId: string | null, + x: number, + y: number, + depth: number, + parentId: string | null, + options?: { draggingPreview?: boolean }, + ) => { const measure = measureGroup(groupId); const group: MediaBoardGroupLayout = { id: groupId, @@ -2615,28 +3351,36 @@ export function MediaPanel() { height: measure.height, itemCount: measure.itemCount, depth, + isDraggingPreview: options?.draggingPreview, }; groups.push(group); - let entryTop = y + MEDIA_BOARD_GROUP_HEADER_HEIGHT + MEDIA_BOARD_GROUP_PADDING; - let entrySlotIndex = 0; - measure.itemRows.forEach((layoutRow, rowIndex) => { - let entryLeft = x + MEDIA_BOARD_GROUP_PADDING; - if (rowIndex > 0) { - entryTop += MEDIA_BOARD_NODE_GAP; - } - - layoutRow.entries.forEach((entry, entryIndex) => { - if (entryIndex > 0) { - entryLeft += MEDIA_BOARD_NODE_GAP; - } + const chrome = getMediaBoardGroupChrome(groupId); + const entryOriginX = x + chrome.padding; + const entryOriginY = y + chrome.headerHeight + chrome.padding; + measure.itemRows.forEach((layoutRow) => { + layoutRow.entries.forEach((entry) => { + const entryOffsetX = entry.offsetX ?? 0; + const entryOffsetY = entry.offsetY ?? 0; + const entrySlotIndex = entry.resolvedSlotIndex ?? 0; const layout: MediaBoardNodeLayout = { - x: entryLeft, - y: entryTop + ((layoutRow.height - entry.height) / 2), + x: entryOriginX + entryOffsetX, + y: entryOriginY + entryOffsetY, width: entry.width, height: entry.height, }; + if (!entry.isInsertGap) { + slots.push({ + id: entry.isEmptySlot ? entry.id : entry.item?.id ?? entry.id, + itemId: entry.item?.id, + layout, + groupId, + slotIndex: entrySlotIndex, + isEmptySlot: entry.isEmptySlot, + }); + } + if (entry.isInsertGap) { insertGaps.push({ id: entry.id, @@ -2649,49 +3393,24 @@ export function MediaPanel() { item: entry.item, defaultLayout: layout, groupId, + isDraggingPreview: options?.draggingPreview, layout, slotIndex: entrySlotIndex, }); - } - - entryLeft += entry.width; - entrySlotIndex += 1; - }); - entryTop += layoutRow.height; - }); - - if (measure.itemRows.length > 0 && measure.folderRows.length > 0) { - entryTop += MEDIA_BOARD_FOLDER_GAP; - } - measure.folderRows.forEach((folderRow, rowIndex) => { - let rowCursorX = x + MEDIA_BOARD_GROUP_PADDING; - let rowMaxBottom = entryTop; - if (rowIndex > 0) { - entryTop += MEDIA_BOARD_FOLDER_GAP; - rowCursorX = x + MEDIA_BOARD_GROUP_PADDING; - rowMaxBottom = entryTop; - } + if (isMediaBoardFolder(entry.item)) { + placeGroup( + entry.item.id, + layout.x, + layout.y, + depth + 1, + groupId, + { draggingPreview: options?.draggingPreview }, + ); + } + } - folderRow.entries.forEach((block) => { - const desiredX = x + MEDIA_BOARD_GROUP_PADDING + block.desiredX; - const desiredY = entryTop + block.desiredY; - const childX = Math.max(rowCursorX, desiredX); - const childY = Math.max( - y + MEDIA_BOARD_GROUP_HEADER_HEIGHT + MEDIA_BOARD_GROUP_PADDING, - desiredY, - ); - placeGroup( - block.folder.id, - childX, - childY, - depth + 1, - groupId, - ); - rowCursorX = childX + block.width + MEDIA_BOARD_FOLDER_GAP; - rowMaxBottom = Math.max(rowMaxBottom, childY + block.height); }); - entryTop = Math.max(entryTop + folderRow.height, rowMaxBottom); }); }; @@ -2728,11 +3447,36 @@ export function MediaPanel() { layout: sourceLayout, slotIndex: index, }); + if (isMediaBoardFolder(item)) { + placeGroup(item.id, sourceLayout.x, sourceLayout.y, 1, item.parentId ?? null, { + draggingPreview: true, + }); + } }); } - return { groups, placements, insertGaps }; - }, [folders, mediaBoardGroupOffsets, mediaBoardInsertionPreview, mediaBoardItems, mediaBoardOrder, sortItems]); + return { groups, placements, insertGaps, slots }; + }, [mediaBoardInsertionPreviewKey, mediaBoardLayoutSignature]); + + const mediaBoardLayout = useMemo(() => ( + restoreMediaBoardLayoutItems(mediaBoardLayoutGeometry, mediaBoardItemsById, folders) + ), [folders, mediaBoardItemsById, mediaBoardLayoutGeometry]); + + useEffect(() => { + if (viewMode !== 'board' || mediaBoardInsertionPreviewKey) return; + + const saveSnapshot = () => { + saveMediaBoardLayoutSnapshot(mediaBoardLayoutSignature, mediaBoardLayoutGeometry); + }; + const requestIdle = window.requestIdleCallback; + if (typeof requestIdle === 'function') { + const idleId = requestIdle(saveSnapshot, { timeout: 1200 }); + return () => window.cancelIdleCallback?.(idleId); + } + + const timeoutId = window.setTimeout(saveSnapshot, 250); + return () => window.clearTimeout(timeoutId); + }, [mediaBoardInsertionPreviewKey, mediaBoardLayoutGeometry, mediaBoardLayoutSignature, viewMode]); const mediaBoardPlacementsById = useMemo(() => { return new Map(mediaBoardLayout.placements.map((placement) => [placement.item.id, placement])); @@ -2744,8 +3488,10 @@ export function MediaPanel() { ), [mediaBoardCanvasSize, mediaBoardViewport]); const mediaBoardRenderLod = useMemo(() => ({ + overviewCanvas: mediaBoardViewport.zoom <= MEDIA_BOARD_OVERVIEW_CANVAS_ZOOM, compact: mediaBoardViewport.zoom <= MEDIA_BOARD_COMPACT_LOD_ZOOM, - showImages: mediaBoardViewport.zoom >= MEDIA_BOARD_THUMBNAIL_LOD_MIN_ZOOM, + showImages: mediaBoardViewport.zoom > MEDIA_BOARD_THUMBNAIL_LOD_MIN_ZOOM, + requestThumbnails: mediaBoardViewport.zoom >= MEDIA_BOARD_THUMBNAIL_REQUEST_MIN_ZOOM, }), [mediaBoardViewport.zoom]); const visibleMediaBoardGroups = useMemo(() => ( @@ -2764,20 +3510,57 @@ export function MediaPanel() { )) ), [mediaBoardLayout.placements, mediaBoardVisibleRect, selectedIdSet]); + const getMediaBoardPlacementAtPoint = useCallback((point: { x: number; y: number }) => { + for (let index = mediaBoardLayout.placements.length - 1; index >= 0; index -= 1) { + const placement = mediaBoardLayout.placements[index]; + const { layout } = placement; + if ( + point.x >= layout.x + && point.x <= layout.x + layout.width + && point.y >= layout.y + && point.y <= layout.y + layout.height + ) { + return placement; + } + } + return null; + }, [mediaBoardLayout.placements]); + const visibleMediaBoardThumbnailKey = useMemo(() => { - if (!mediaBoardRenderLod.showImages) return ''; + if (!mediaBoardRenderLod.requestThumbnails) return ''; + + const centerX = (mediaBoardVisibleRect.left + mediaBoardVisibleRect.right) / 2; + const centerY = (mediaBoardVisibleRect.top + mediaBoardVisibleRect.bottom) / 2; + const requestLimit = mediaBoardRenderLod.overviewCanvas + ? MEDIA_BOARD_OVERVIEW_THUMBNAIL_REQUEST_LIMIT + : MEDIA_BOARD_THUMBNAIL_REQUEST_LIMIT; + return visibleMediaBoardPlacements - .map((placement) => placement.item) - .filter((item): item is MediaFile => ( - isImportedMediaFileItem(item) - && !item.thumbnailUrl - && !item.isImporting - && (item.type === 'image' || item.type === 'video') - )) - .slice(0, 48) - .map((item) => item.id) + .map((placement) => { + const { item, layout } = placement; + if ( + !isImportedMediaFileItem(item) + || item.thumbnailUrl + || item.isImporting + || (item.type !== 'image' && item.type !== 'video') + ) { + return null; + } + + const itemCenterX = layout.x + layout.width / 2; + const itemCenterY = layout.y + layout.height / 2; + return { + id: item.id, + area: layout.width * layout.height, + distance: Math.hypot(itemCenterX - centerX, itemCenterY - centerY), + }; + }) + .filter((entry): entry is { id: string; area: number; distance: number } => entry !== null) + .toSorted((a, b) => (b.area - a.area) || (a.distance - b.distance)) + .slice(0, requestLimit) + .map((entry) => entry.id) .join('\n'); - }, [mediaBoardRenderLod.showImages, visibleMediaBoardPlacements]); + }, [mediaBoardRenderLod.overviewCanvas, mediaBoardRenderLod.requestThumbnails, mediaBoardVisibleRect, visibleMediaBoardPlacements]); useEffect(() => { if (viewMode !== 'board' || !visibleMediaBoardThumbnailKey) return; @@ -2785,13 +3568,15 @@ export function MediaPanel() { const thumbnailIds = visibleMediaBoardThumbnailKey.split('\n').filter(Boolean); let cancelled = false; let nextIndex = 0; - const workerCount = Math.min(3, thumbnailIds.length); + const workerCount = Math.min(MEDIA_BOARD_THUMBNAIL_WORKER_COUNT, thumbnailIds.length); const runWorker = async () => { while (!cancelled) { const id = thumbnailIds[nextIndex]; nextIndex += 1; if (!id) return; + await waitForMediaBoardThumbnailTurn(); + if (cancelled) return; await ensureFileThumbnail(id); } }; @@ -2805,14 +3590,100 @@ export function MediaPanel() { }; }, [ensureFileThumbnail, viewMode, visibleMediaBoardThumbnailKey]); + const scheduleMediaBoardOverviewRedraw = useCallback(() => { + if (boardOverviewRedrawFrameRef.current !== null) return; + boardOverviewRedrawFrameRef.current = window.requestAnimationFrame(() => { + boardOverviewRedrawFrameRef.current = null; + setMediaBoardOverviewImageVersion((version) => (version + 1) % 100000); + }); + }, []); + + useLayoutEffect(() => { + if (viewMode !== 'board' || !mediaBoardRenderLod.overviewCanvas) return; + const canvas = boardOverviewCanvasRef.current; + if (!canvas) return; + + const zoom = Math.max(mediaBoardViewport.zoom, MEDIA_BOARD_PAN_ZOOM_MIN); + const dpr = Math.min(2, Math.max(1, window.devicePixelRatio || 1)); + const rect = mediaBoardVisibleRect; + const boardWidth = Math.max(1, rect.right - rect.left); + const boardHeight = Math.max(1, rect.bottom - rect.top); + const pixelWidth = Math.max(1, Math.ceil(boardWidth * zoom * dpr)); + const pixelHeight = Math.max(1, Math.ceil(boardHeight * zoom * dpr)); + + if (canvas.width !== pixelWidth) canvas.width = pixelWidth; + if (canvas.height !== pixelHeight) canvas.height = pixelHeight; + + const ctx = canvas.getContext('2d', { alpha: true }); + if (!ctx) return; + + ctx.setTransform(1, 0, 0, 1, 0, 0); + ctx.clearRect(0, 0, pixelWidth, pixelHeight); + ctx.setTransform(zoom * dpr, 0, 0, zoom * dpr, -rect.left * zoom * dpr, -rect.top * zoom * dpr); + ctx.imageSmoothingEnabled = true; + ctx.imageSmoothingQuality = 'low'; + + const cache = boardOverviewImageCacheRef.current; + const visibleItemIds = new Set(); + const getLoadedOverviewImage = (item: MediaBoardItem): HTMLImageElement | null => { + if (!isImportedMediaFileItem(item) || !item.thumbnailUrl) return null; + + visibleItemIds.add(item.id); + const cached = cache.get(item.id); + if (cached?.src === item.thumbnailUrl) { + return cached.status === 'loaded' ? cached.image : null; + } + + const image = new Image(); + const record = { src: item.thumbnailUrl, image, status: 'loading' as const }; + cache.set(item.id, record); + image.onload = () => { + cache.set(item.id, { ...record, status: 'loaded' }); + scheduleMediaBoardOverviewRedraw(); + }; + image.onerror = () => { + cache.set(item.id, { ...record, status: 'error' }); + }; + image.decoding = 'async'; + image.src = item.thumbnailUrl; + return null; + }; + + visibleMediaBoardPlacements.forEach((placement) => { + if (placement.isDraggingPreview || selectedIdSet.has(placement.item.id)) return; + drawMediaBoardOverviewItem( + ctx, + placement, + getLoadedOverviewImage(placement.item), + zoom, + Boolean(mediaSearchVisibleItemIds && !mediaSearchVisibleItemIds.has(placement.item.id)), + ); + }); + + cache.forEach((_, itemId) => { + if (!visibleItemIds.has(itemId)) cache.delete(itemId); + }); + }, [ + mediaBoardOverviewImageVersion, + mediaBoardRenderLod.overviewCanvas, + mediaBoardViewport.zoom, + mediaBoardVisibleRect, + mediaSearchVisibleItemIds, + scheduleMediaBoardOverviewRedraw, + selectedIdSet, + viewMode, + visibleMediaBoardPlacements, + ]); + const screenToMediaBoard = useCallback((clientX: number, clientY: number) => { const rect = boardCanvasRef.current?.getBoundingClientRect(); if (!rect) return { x: 0, y: 0 }; + const viewport = mediaBoardViewportRef.current; return { - x: (clientX - rect.left - mediaBoardViewport.panX) / mediaBoardViewport.zoom, - y: (clientY - rect.top - mediaBoardViewport.panY) / mediaBoardViewport.zoom, + x: (clientX - rect.left - viewport.panX) / viewport.zoom, + y: (clientY - rect.top - viewport.panY) / viewport.zoom, }; - }, [mediaBoardViewport.panX, mediaBoardViewport.panY, mediaBoardViewport.zoom]); + }, []); const openBoardAI = useCallback(() => { useDockStore.getState().activatePanelType('ai-video'); @@ -2823,6 +3694,12 @@ export function MediaPanel() { boardWrapperRef.current?.classList.toggle('board-interacting', enabled); }, []); + const applyMediaBoardViewportPreview = useCallback((viewport: MediaBoardViewport) => { + const inner = boardCanvasInnerRef.current; + if (!inner) return; + inner.style.transform = `translate(${viewport.panX}px, ${viewport.panY}px) scale(${viewport.zoom})`; + }, []); + const startMediaBoardPanGesture = useCallback((e: React.MouseEvent, options?: { clearSelectionOnTap?: boolean }) => { if (e.button === 1) { e.preventDefault(); @@ -2831,7 +3708,7 @@ export function MediaPanel() { const startX = e.clientX; const startY = e.clientY; - const startViewport = { ...mediaBoardViewport }; + const startViewport = { ...mediaBoardViewportRef.current }; let pendingViewport = startViewport; let didPan = false; @@ -2839,11 +3716,7 @@ export function MediaPanel() { if (boardInteractionFrameRef.current !== null) return; boardInteractionFrameRef.current = window.requestAnimationFrame(() => { boardInteractionFrameRef.current = null; - const inner = boardCanvasInnerRef.current; - if (!inner) return; - inner.style.transform = `translate(${pendingViewport.panX}px, ${pendingViewport.panY}px) scale(${pendingViewport.zoom})`; - boardWrapperRef.current?.style.setProperty('--media-board-grid-x', `${pendingViewport.panX * MEDIA_BOARD_GRID_PARALLAX}px`); - boardWrapperRef.current?.style.setProperty('--media-board-grid-y', `${pendingViewport.panY * MEDIA_BOARD_GRID_PARALLAX}px`); + applyMediaBoardViewportPreview(pendingViewport); }); }; @@ -2876,6 +3749,7 @@ export function MediaPanel() { setMediaBoardPerformanceMode(false); if (didPan) { + mediaBoardViewportRef.current = pendingViewport; setMediaBoardViewport(pendingViewport); } else if (options?.clearSelectionOnTap) { setSelection([]); @@ -2889,7 +3763,7 @@ export function MediaPanel() { window.addEventListener('mousemove', handleMouseMove); window.addEventListener('mouseup', handleMouseUp); window.addEventListener('blur', handleMouseUp); - }, [closeContextMenu, mediaBoardViewport, setMediaBoardPerformanceMode, setSelection]); + }, [applyMediaBoardViewportPreview, closeContextMenu, setMediaBoardPerformanceMode, setSelection]); const startMediaBoardMarqueeGesture = useCallback((e: React.MouseEvent) => { const startPoint = screenToMediaBoard(e.clientX, e.clientY); @@ -2946,122 +3820,7 @@ export function MediaPanel() { window.addEventListener('blur', handleMouseUp); }, [closeContextMenu, mediaBoardLayout.placements, screenToMediaBoard, selectedIds, setSelection, suppressNextMediaBoardContextMenu]); - const startMediaBoardGroupMoveGesture = useCallback((e: React.MouseEvent, group: MediaBoardGroupLayout) => { - if (!group.id) return; - - e.stopPropagation(); - - const movingGroupIds = new Set([group.id]); - let changed = true; - while (changed) { - changed = false; - folders.forEach((folder) => { - if (folder.parentId && movingGroupIds.has(folder.parentId) && !movingGroupIds.has(folder.id)) { - movingGroupIds.add(folder.id); - changed = true; - } - }); - } - - const groupElements = [...movingGroupIds] - .map((folderId) => boardCanvasRef.current?.querySelector(`.media-board-group[data-board-group-key="${CSS.escape(folderId)}"]`) ?? null) - .filter((element): element is HTMLElement => Boolean(element)); - const nodeElements = mediaBoardLayout.placements - .filter((placement) => placement.groupId !== null && movingGroupIds.has(placement.groupId)) - .map((placement) => boardCanvasRef.current?.querySelector(`.media-board-node[data-item-id="${CSS.escape(placement.item.id)}"]`) ?? null) - .filter((element): element is HTMLElement => Boolean(element)); - const movingElements = [...groupElements, ...nodeElements]; - if (movingElements.length === 0) return; - - const movingFolderIds = [group.id]; - const startX = e.clientX; - const startY = e.clientY; - let didDrag = false; - let previewDx = 0; - let previewDy = 0; - - const clearPreview = () => { - movingElements.forEach((element) => { - element.style.transform = ''; - element.classList.remove('drag-preview'); - }); - document.body.style.cursor = ''; - document.body.style.userSelect = ''; - }; - - const schedulePreview = () => { - if (boardInteractionFrameRef.current !== null) return; - boardInteractionFrameRef.current = window.requestAnimationFrame(() => { - boardInteractionFrameRef.current = null; - movingElements.forEach((element) => { - element.style.transform = `translate3d(${previewDx}px, ${previewDy}px, 0)`; - element.classList.add('drag-preview'); - }); - }); - }; - - const handleMouseMove = (moveEvent: MouseEvent) => { - const distance = Math.hypot(moveEvent.clientX - startX, moveEvent.clientY - startY); - if (!didDrag && distance < MEDIA_BOARD_DRAG_START_DISTANCE) return; - - if (!didDrag) { - didDrag = true; - closeContextMenu(); - setMediaBoardPerformanceMode(true); - document.body.style.cursor = 'grabbing'; - document.body.style.userSelect = 'none'; - } - - moveEvent.preventDefault(); - previewDx = (moveEvent.clientX - startX) / mediaBoardViewport.zoom; - previewDy = (moveEvent.clientY - startY) / mediaBoardViewport.zoom; - schedulePreview(); - }; - - const handleMouseUp = () => { - if (boardInteractionFrameRef.current !== null) { - window.cancelAnimationFrame(boardInteractionFrameRef.current); - boardInteractionFrameRef.current = null; - } - clearPreview(); - setMediaBoardPerformanceMode(false); - - if (didDrag) { - suppressNextMediaBoardContextMenu(); - setMediaBoardGroupOffsets((current) => { - const next: Record = { ...current }; - movingFolderIds.forEach((folderId) => { - const currentOffset = next[folderId] ?? { x: 0, y: 0 }; - const x = currentOffset.x + previewDx; - const y = currentOffset.y + previewDy; - if (Math.abs(x) < 0.5 && Math.abs(y) < 0.5) { - delete next[folderId]; - } else { - next[folderId] = { x, y }; - } - }); - return next; - }); - } - - window.removeEventListener('mousemove', handleMouseMove); - window.removeEventListener('mouseup', handleMouseUp); - window.removeEventListener('blur', handleMouseUp); - }; - - window.addEventListener('mousemove', handleMouseMove); - window.addEventListener('mouseup', handleMouseUp); - window.addEventListener('blur', handleMouseUp); - }, [ - closeContextMenu, - folders, - mediaBoardLayout.placements, - mediaBoardViewport.zoom, - setMediaBoardPerformanceMode, - suppressNextMediaBoardContextMenu, - ]); - - const getMediaBoardGroupAtPoint = useCallback((point: { x: number; y: number }) => { + const getMediaBoardGroupsAtPoint = useCallback((point: { x: number; y: number }) => { return mediaBoardLayout.groups .filter((group) => ( point.x >= group.x @@ -3069,9 +3828,14 @@ export function MediaPanel() { && point.y >= group.y && point.y <= group.y + group.height )) - .sort((a, b) => b.depth - a.depth)[0] ?? null; + .sort((a, b) => b.depth - a.depth); }, [mediaBoardLayout.groups]); + const getMediaBoardGroupAtPoint = useCallback((point: { x: number; y: number }) => { + const groupsAtPoint = getMediaBoardGroupsAtPoint(point); + return groupsAtPoint[0] ?? mediaBoardLayout.groups.find((group) => group.id === null) ?? null; + }, [getMediaBoardGroupsAtPoint, mediaBoardLayout.groups]); + const canMoveItemsToMediaBoardGroup = useCallback((itemIds: string[], targetGroupId: string | null) => { if (!targetGroupId) return true; @@ -3098,35 +3862,75 @@ export function MediaPanel() { handleContextMenu(e, undefined, targetGroup?.id ?? null); }, [consumeSuppressedMediaBoardContextMenu, getMediaBoardGroupAtPoint, handleContextMenu, screenToMediaBoard]); - const getMediaBoardInsertTarget = useCallback((point: { x: number; y: number }, movingIds: string[]) => { - const targetGroup = getMediaBoardGroupAtPoint(point); + const getMediaBoardInsertTarget = useCallback(( + point: { x: number; y: number }, + movingIds: string[], + groupPoint = point, + ) => { + const groupsAtPoint = getMediaBoardGroupsAtPoint(groupPoint); + const rootGroup = mediaBoardLayout.groups.find((group) => group.id === null) ?? null; + const isPointInsideGroupBody = (group: MediaBoardGroupLayout) => { + if (group.id === null) return true; + const chrome = getMediaBoardGroupChrome(group.id); + if (group.itemCount === 0) { + return ( + groupPoint.x >= group.x + && groupPoint.x <= group.x + group.width + && groupPoint.y >= group.y + && groupPoint.y <= group.y + group.height + ); + } + return ( + groupPoint.x >= group.x + chrome.padding + && groupPoint.x <= group.x + group.width - chrome.padding + && groupPoint.y >= group.y + chrome.headerHeight + chrome.padding + && groupPoint.y <= group.y + group.height - chrome.padding + ); + }; + const targetGroup = [ + ...groupsAtPoint.filter(isPointInsideGroupBody), + ...(rootGroup && !groupsAtPoint.some((group) => group.id === rootGroup.id) ? [rootGroup] : []), + ].find((group) => canMoveItemsToMediaBoardGroup(movingIds, group.id)) ?? null; if (!targetGroup) return null; const movingIdSet = new Set(movingIds); - const targetPlacements = mediaBoardLayout.placements - .filter((placement) => placement.groupId === targetGroup.id && !movingIdSet.has(placement.item.id)) + const targetSlots = mediaBoardLayout.slots + .filter((slot) => slot.groupId === targetGroup.id && (!slot.itemId || !movingIdSet.has(slot.itemId))) .sort((a, b) => a.slotIndex - b.slotIndex); - let targetIndex = targetPlacements.length; - for (let index = 0; index < targetPlacements.length; index += 1) { - const { layout } = targetPlacements[index]; - const centerX = layout.x + layout.width / 2; - const centerY = layout.y + layout.height / 2; - if (point.y < centerY || (point.y < layout.y + layout.height && point.x < centerX)) { - targetIndex = index; - break; - } - } + const chrome = getMediaBoardGroupChrome(targetGroup.id); + const bodyLeft = targetGroup.x + chrome.padding; + const bodyTop = targetGroup.y + chrome.headerHeight + chrome.padding; + const columnPitch = MEDIA_BOARD_SLOT_CELL_WIDTH; + const rowPitch = MEDIA_BOARD_SLOT_CELL_HEIGHT; + const hoveredSlot = targetSlots.find(({ layout }) => ( + groupPoint.x >= layout.x + && groupPoint.x <= layout.x + layout.width + && groupPoint.y >= layout.y + && groupPoint.y <= layout.y + layout.height + )); + const clampToFolderBody = targetGroup.id !== null; + const clampBoardPosition = (value: number) => clampToFolderBody ? Math.max(0, value) : value; + const targetPosition = hoveredSlot + ? { + x: clampBoardPosition(hoveredSlot.layout.x - bodyLeft), + y: clampBoardPosition(hoveredSlot.layout.y - bodyTop), + } + : { + x: clampBoardPosition(Math.round((point.x - bodyLeft) / columnPitch) * columnPitch), + y: clampBoardPosition(Math.round((point.y - bodyTop) / rowPitch) * rowPitch), + }; - return { groupId: targetGroup.id, index: targetIndex }; - }, [getMediaBoardGroupAtPoint, mediaBoardLayout.placements]); + return { groupId: targetGroup.id, position: targetPosition }; + }, [canMoveItemsToMediaBoardGroup, getMediaBoardGroupsAtPoint, mediaBoardLayout.groups, mediaBoardLayout.slots]); const updateMediaBoardInsertionPreview = useCallback(( point: { x: number; y: number }, movingIds: string[], sourceLayouts: Record, + groupPoint = point, ) => { - const target = getMediaBoardInsertTarget(point, movingIds); + const target = getMediaBoardInsertTarget(point, movingIds, groupPoint); if (!target) { setMediaBoardInsertionPreview(null); return null; @@ -3137,7 +3941,8 @@ export function MediaPanel() { if ( current && current.targetGroupId === target.groupId - && current.targetIndex === target.index + && current.targetPosition.x === target.position.x + && current.targetPosition.y === target.position.y && current.movingIds.join('\u0000') === movingKey ) { return current; @@ -3146,63 +3951,161 @@ export function MediaPanel() { movingIds, sourceLayouts, targetGroupId: target.groupId, - targetIndex: target.index, + targetPosition: target.position, }; }); return target; }, [getMediaBoardInsertTarget]); - const getMediaBoardAppendIndex = useCallback((groupId: string | null, movingIds: string[]) => { - const movingIdSet = new Set(movingIds); - return mediaBoardLayout.placements.filter((placement) => ( - placement.groupId === groupId && !movingIdSet.has(placement.item.id) - )).length; - }, [mediaBoardLayout.placements]); - - const commitMediaBoardOrderChange = useCallback((movingIds: string[], targetGroupId: string | null, targetIndex: number) => { + const commitMediaBoardOrderChange = useCallback(( + movingIds: string[], + targetGroupId: string | null, + targetPosition: MediaBoardGroupOffset, + options?: { sourceLayouts?: Record; anchorId?: string }, + ) => { if (movingIds.length === 0) return; - const movingIdSet = new Set(movingIds); - const targetGroupKey = getMediaBoardOrderKey(targetGroupId); - const groupIds = [ - MEDIA_BOARD_ROOT_ORDER_KEY, - ...folders.map((folder) => folder.id), - ]; + const normalizedMovingIds = movingIds.filter((id) => mediaBoardItemsById.has(id)); + if (normalizedMovingIds.length === 0) return; + const movingIdSet = new Set(normalizedMovingIds); + + const columnPitch = MEDIA_BOARD_SLOT_CELL_WIDTH; + const rowPitch = MEDIA_BOARD_SLOT_CELL_HEIGHT; + const targetGroup = mediaBoardLayout.groups.find((group) => group.id === targetGroupId) ?? null; + const targetChrome = getMediaBoardGroupChrome(targetGroupId); + const targetBodyLeft = targetGroup ? targetGroup.x + targetChrome.padding : 0; + const targetBodyTop = targetGroup ? targetGroup.y + targetChrome.headerHeight + targetChrome.padding : 0; + const allowNegativePositions = targetGroupId === null; + const clampLocalPosition = (value: number) => allowNegativePositions ? value : Math.max(0, value); + + const getItemSize = (id: string) => { + const placement = mediaBoardPlacementsById.get(id); + if (placement) { + return { width: placement.layout.width, height: placement.layout.height }; + } + const item = mediaBoardItemsById.get(id); + return item ? getMediaBoardNodeSize(item) : { width: MEDIA_BOARD_EMPTY_SLOT_WIDTH, height: MEDIA_BOARD_EMPTY_SLOT_HEIGHT }; + }; - setMediaBoardOrder((current) => { - const next: Record = { ...current }; - - groupIds.forEach((groupKey) => { - const existingOrder = next[groupKey] - ?? mediaBoardLayout.placements - .filter((placement) => getMediaBoardOrderKey(placement.groupId) === groupKey) - .sort((a, b) => a.slotIndex - b.slotIndex) - .map((placement) => placement.item.id); - const filteredOrder = existingOrder.filter((id) => !movingIdSet.has(id)); - if (filteredOrder.length > 0) { - next[groupKey] = filteredOrder; - } else { - delete next[groupKey]; + const getFallbackLocalPosition = (id: string, fallbackIndex: number): MediaBoardGroupOffset => { + const placement = mediaBoardPlacementsById.get(id); + if (placement && placement.groupId === targetGroupId) { + return { + x: clampLocalPosition(placement.layout.x - targetBodyLeft), + y: clampLocalPosition(placement.layout.y - targetBodyTop), + }; + } + return { + x: fallbackIndex * columnPitch, + y: 0, + }; + }; + + const sourceLayouts = options?.sourceLayouts ?? {}; + const anchorSourceLayout = (options?.anchorId ? sourceLayouts[options.anchorId] : undefined) + ?? normalizedMovingIds.map((id) => sourceLayouts[id]).find((layout): layout is MediaBoardNodeLayout => Boolean(layout)) + ?? null; + + const getMovingDesiredPosition = (id: string, index: number): MediaBoardGroupOffset => { + const sourceLayout = sourceLayouts[id]; + if (sourceLayout && anchorSourceLayout) { + return { + x: targetPosition.x + (sourceLayout.x - anchorSourceLayout.x), + y: targetPosition.y + (sourceLayout.y - anchorSourceLayout.y), + }; + } + return { + x: targetPosition.x + (index * columnPitch), + y: targetPosition.y, + }; + }; + + setMediaBoardLayouts((current) => { + const next = { ...current }; + const occupied = new Set(); + let changed = false; + + const getSpan = (size: { width: number; height: number }) => ({ + columns: Math.max(1, Math.ceil((size.width + MEDIA_BOARD_NODE_GAP) / columnPitch)), + rows: Math.max(1, Math.ceil((size.height + MEDIA_BOARD_NODE_GAP) / rowPitch)), + }); + + const canPlace = (column: number, row: number, span: { columns: number; rows: number }) => { + if (!allowNegativePositions && (column < 0 || row < 0)) return false; + for (let y = row; y < row + span.rows; y += 1) { + for (let x = column; x < column + span.columns; x += 1) { + if (occupied.has(`${x}:${y}`)) return false; + } + } + return true; + }; + + const markOccupied = (column: number, row: number, span: { columns: number; rows: number }) => { + for (let y = row; y < row + span.rows; y += 1) { + for (let x = column; x < column + span.columns; x += 1) { + occupied.add(`${x}:${y}`); + } + } + }; + + mediaBoardItems + .filter((item) => !movingIdSet.has(item.id) && (item.parentId ?? null) === targetGroupId) + .forEach((item, index) => { + const size = getItemSize(item.id); + const desired = current[item.id] ?? getFallbackLocalPosition(item.id, index); + const span = getSpan(size); + const column = allowNegativePositions + ? Math.round(desired.x / columnPitch) + : Math.max(0, Math.round(desired.x / columnPitch)); + const row = allowNegativePositions + ? Math.round(desired.y / rowPitch) + : Math.max(0, Math.round(desired.y / rowPitch)); + markOccupied(column, row, span); + }); + + normalizedMovingIds.forEach((id, index) => { + const desired = getMovingDesiredPosition(id, index); + const size = getItemSize(id); + const entry = { id, desired, size }; + const span = getSpan(entry.size); + const initialColumn = allowNegativePositions + ? Math.round(entry.desired.x / columnPitch) + : Math.max(0, Math.round(entry.desired.x / columnPitch)); + const initialRow = allowNegativePositions + ? Math.round(entry.desired.y / rowPitch) + : Math.max(0, Math.round(entry.desired.y / rowPitch)); + let column = initialColumn; + let row = initialRow; + let attempts = 0; + while (!canPlace(column, row, span)) { + column += 1; + attempts += 1; + if (attempts > 10000) { + row += 1; + column = initialColumn; + attempts = 0; + } + } + markOccupied(column, row, span); + + const resolvedPosition = { + x: column * columnPitch, + y: row * rowPitch, + }; + if (next[entry.id]?.x !== resolvedPosition.x || next[entry.id]?.y !== resolvedPosition.y) { + next[entry.id] = resolvedPosition; + changed = true; } }); - const targetOrder = [ - ...(next[targetGroupKey] - ?? mediaBoardLayout.placements - .filter((placement) => placement.groupId === targetGroupId && !movingIdSet.has(placement.item.id)) - .sort((a, b) => a.slotIndex - b.slotIndex) - .map((placement) => placement.item.id)), - ]; - const insertIndex = Math.max(0, Math.min(targetIndex, targetOrder.length)); - targetOrder.splice(insertIndex, 0, ...movingIds); - next[targetGroupKey] = targetOrder; - - return next; + return changed ? next : current; }); - moveToFolder(movingIds, targetGroupId); - }, [folders, mediaBoardLayout.placements, moveToFolder]); + moveToFolder(normalizedMovingIds, targetGroupId); + }, [mediaBoardItems, mediaBoardItemsById, mediaBoardLayout.groups, mediaBoardPlacementsById, moveToFolder, setMediaBoardLayouts]); const getMediaBoardExternalDragPayload = useCallback((item: MediaBoardItem): ExternalDragPayload | null => { + if (isMediaBoardFolder(item)) return null; + if (item.type === 'composition') { const comp = item as Composition; const inSlotView = useTimelineStore.getState().slotGridProgress > 0.5; @@ -3292,16 +4195,13 @@ export function MediaPanel() { }, [activeCompositionId]); const startMediaBoardNodeMoveGesture = useCallback((e: React.MouseEvent, item: MediaBoardItem) => { - if (e.button === 2) { - e.preventDefault(); - } - - const selectedMoveIds = selectedIds.includes(item.id) + const requestedMoveIds = selectedIds.includes(item.id) ? selectedIds.filter((id) => mediaBoardItemIds.has(id)) : [item.id]; + const selectedMoveIds = getMediaBoardTopLevelMoveIds(requestedMoveIds); const boardOrderedMoveIds = mediaBoardLayout.placements .filter((placement) => selectedMoveIds.includes(placement.item.id)) - .sort((a, b) => a.slotIndex - b.slotIndex) + .sort((a, b) => (a.layout.y - b.layout.y) || (a.layout.x - b.layout.x) || (a.slotIndex - b.slotIndex)) .map((placement) => placement.item.id); const moveIds = boardOrderedMoveIds.length > 0 ? boardOrderedMoveIds : selectedMoveIds; const startLayouts = moveIds.map((id) => { @@ -3319,10 +4219,27 @@ export function MediaPanel() { layouts[entry.id] = entry.layout; return layouts; }, {}); + const anchorLayout = sourceLayouts[item.id] ?? startLayouts[0]?.layout ?? null; + const getMediaBoardElementById = (id: string) => ( + boardCanvasRef.current?.querySelector( + `.media-board-node[data-item-id="${CSS.escape(id)}"], .media-board-group[data-item-id="${CSS.escape(id)}"]`, + ) ?? null + ); + const getMediaBoardPreviewElements = () => { + const elements = new Set(); + startLayouts.forEach(({ id }) => { + const node = getMediaBoardElementById(id); + if (node) elements.add(node); + }); + boardCanvasRef.current + ?.querySelectorAll('.media-board-node.drag-source-preview, .media-board-group.drag-source-preview') + .forEach((node) => elements.add(node)); + return [...elements]; + }; const startX = e.clientX; const startY = e.clientY; - const startViewport = { ...mediaBoardViewport }; - let liveViewport = { ...mediaBoardViewport }; + const startViewport = { ...mediaBoardViewportRef.current }; + let liveViewport = { ...startViewport }; let didDrag = false; let previewDx = 0; let previewDy = 0; @@ -3330,7 +4247,7 @@ export function MediaPanel() { let latestClientY = startY; let latestTimelineHandoffActive = false; let timelineBridgeActive = false; - let latestInsertTarget: { groupId: string | null; index: number } | null = null; + let latestInsertTarget: { groupId: string | null; position: MediaBoardGroupOffset } | null = null; let autoPanVelocity = { x: 0, y: 0 }; let lastAutoPanTime: number | null = null; @@ -3344,11 +4261,7 @@ export function MediaPanel() { }; const applyLiveViewportPreview = () => { - const inner = boardCanvasInnerRef.current; - if (!inner) return; - inner.style.transform = `translate(${liveViewport.panX}px, ${liveViewport.panY}px) scale(${liveViewport.zoom})`; - boardWrapperRef.current?.style.setProperty('--media-board-grid-x', `${liveViewport.panX * MEDIA_BOARD_GRID_PARALLAX}px`); - boardWrapperRef.current?.style.setProperty('--media-board-grid-y', `${liveViewport.panY * MEDIA_BOARD_GRID_PARALLAX}px`); + applyMediaBoardViewportPreview(liveViewport); }; const isTimelineHandoffTarget = () => { @@ -3411,10 +4324,15 @@ export function MediaPanel() { setMediaBoardInsertionPreview(null); return; } + const insertionPoint = anchorLayout + ? { x: anchorLayout.x + previewDx, y: anchorLayout.y + previewDy } + : pointToBoard(latestClientX, latestClientY); + const groupPoint = pointToBoard(latestClientX, latestClientY); latestInsertTarget = updateMediaBoardInsertionPreview( - pointToBoard(latestClientX, latestClientY), + insertionPoint, moveIds, sourceLayouts, + groupPoint, ); }; @@ -3424,9 +4342,7 @@ export function MediaPanel() { }; const clearPreview = () => { - startLayouts.forEach(({ id }) => { - const node = boardCanvasRef.current?.querySelector(`.media-board-node[data-item-id="${CSS.escape(id)}"]`); - if (!node) return; + getMediaBoardPreviewElements().forEach((node) => { node.style.transform = ''; node.classList.remove('drag-preview'); }); @@ -3437,9 +4353,7 @@ export function MediaPanel() { boardInteractionFrameRef.current = window.requestAnimationFrame(() => { boardInteractionFrameRef.current = null; applyLiveViewportPreview(); - startLayouts.forEach(({ id }) => { - const node = boardCanvasRef.current?.querySelector(`.media-board-node[data-item-id="${CSS.escape(id)}"]`); - if (!node) return; + getMediaBoardPreviewElements().forEach((node) => { node.style.transform = `translate3d(${previewDx}px, ${previewDy}px, 0)`; node.classList.add('drag-preview'); }); @@ -3516,6 +4430,7 @@ export function MediaPanel() { if (!didDrag) { didDrag = true; + moveEvent.preventDefault(); suppressNextMediaBoardContextMenu(); closeContextMenu(); setMediaBoardPerformanceMode(true); @@ -3523,6 +4438,7 @@ export function MediaPanel() { document.body.style.cursor = 'grabbing'; } + moveEvent.preventDefault(); syncTimelineBridge('move'); updatePreviewDelta(); updateInsertionPreview(); @@ -3537,13 +4453,13 @@ export function MediaPanel() { } stopAutoPan(); clearPreview(); - setMediaBoardPerformanceMode(false); setMediaBoardInsertionPreview(null); document.body.style.cursor = ''; document.body.style.userSelect = ''; if (didDrag) { suppressNextMediaBoardContextMenu(); + mediaBoardViewportRef.current = liveViewport; setMediaBoardViewport(liveViewport); if (latestTimelineHandoffActive && timelineDragPayload) { @@ -3552,13 +4468,24 @@ export function MediaPanel() { clearExternalDragPayload(); } else { syncTimelineBridge('cancel'); - const target = latestInsertTarget ?? getMediaBoardInsertTarget(pointToBoard(latestClientX, latestClientY), moveIds); + const insertionPoint = anchorLayout + ? { x: anchorLayout.x + previewDx, y: anchorLayout.y + previewDy } + : pointToBoard(latestClientX, latestClientY); + const groupPoint = pointToBoard(latestClientX, latestClientY); + const target = latestInsertTarget ?? getMediaBoardInsertTarget(insertionPoint, moveIds, groupPoint); if (target) { - commitMediaBoardOrderChange(moveIds, target.groupId, target.index); + commitMediaBoardOrderChange(moveIds, target.groupId, target.position, { + sourceLayouts, + anchorId: item.id, + }); } } + window.requestAnimationFrame(() => { + window.requestAnimationFrame(() => setMediaBoardPerformanceMode(false)); + }); } else { syncTimelineBridge('cancel'); + setMediaBoardPerformanceMode(false); } window.removeEventListener('mousemove', handleMouseMove); @@ -3588,10 +4515,11 @@ export function MediaPanel() { commitMediaBoardOrderChange, getMediaBoardExternalDragPayload, getMediaBoardInsertTarget, + getMediaBoardTopLevelMoveIds, + applyMediaBoardViewportPreview, mediaBoardItemIds, mediaBoardLayout.placements, mediaBoardPlacementsById, - mediaBoardViewport, selectedIds, setMediaBoardPerformanceMode, suppressNextMediaBoardContextMenu, @@ -3604,25 +4532,72 @@ export function MediaPanel() { if (!rect) return; const zoomDelta = e.deltaY > 0 ? 0.9 : 1.1; - setMediaBoardViewport((current) => { - const nextZoom = Math.min( - MEDIA_BOARD_PAN_ZOOM_MAX, - Math.max(MEDIA_BOARD_PAN_ZOOM_MIN, current.zoom * zoomDelta), - ); - const cursorX = e.clientX - rect.left; - const cursorY = e.clientY - rect.top; - return { - zoom: nextZoom, - panX: cursorX - ((cursorX - current.panX) * (nextZoom / current.zoom)), - panY: cursorY - ((cursorY - current.panY) * (nextZoom / current.zoom)), - }; - }); - }, []); + const current = mediaBoardViewportRef.current; + const nextZoom = Math.min( + MEDIA_BOARD_PAN_ZOOM_MAX, + Math.max(MEDIA_BOARD_PAN_ZOOM_MIN, current.zoom * zoomDelta), + ); + const cursorX = e.clientX - rect.left; + const cursorY = e.clientY - rect.top; + const nextViewport = { + zoom: nextZoom, + panX: cursorX - ((cursorX - current.panX) * (nextZoom / current.zoom)), + panY: cursorY - ((cursorY - current.panY) * (nextZoom / current.zoom)), + }; + + mediaBoardViewportRef.current = nextViewport; + setMediaBoardPerformanceMode(true); + + if (boardInteractionFrameRef.current === null) { + boardInteractionFrameRef.current = window.requestAnimationFrame(() => { + boardInteractionFrameRef.current = null; + applyMediaBoardViewportPreview(mediaBoardViewportRef.current); + }); + } + + if (boardWheelCommitTimerRef.current !== null) { + window.clearTimeout(boardWheelCommitTimerRef.current); + } + boardWheelCommitTimerRef.current = window.setTimeout(() => { + boardWheelCommitTimerRef.current = null; + const committedViewport = mediaBoardViewportRef.current; + setMediaBoardViewport(committedViewport); + window.requestAnimationFrame(() => { + window.requestAnimationFrame(() => setMediaBoardPerformanceMode(false)); + }); + }, 90); + }, [applyMediaBoardViewportPreview, setMediaBoardPerformanceMode]); const handleMediaBoardMouseDown = useCallback((e: React.MouseEvent) => { const target = e.target as HTMLElement; if (target.closest('.media-board-node, .media-board-group.folder-group, button, input, .context-menu')) return; + if (mediaBoardRenderLod.overviewCanvas) { + const hitPlacement = getMediaBoardPlacementAtPoint(screenToMediaBoard(e.clientX, e.clientY)); + if (hitPlacement && !isMediaBoardFolder(hitPlacement.item)) { + if (e.button === 2) { + e.stopPropagation(); + if (e.ctrlKey || e.metaKey) { + startMediaBoardMarqueeGesture(e); + return; + } + if (!selectedIds.includes(hitPlacement.item.id)) { + setSelection([hitPlacement.item.id]); + } + startMediaBoardNodeMoveGesture(e, hitPlacement.item); + return; + } + + if (e.button === 0) { + e.stopPropagation(); + if (e.detail >= 2) return; + handleItemClick(hitPlacement.item.id, e); + startMediaBoardPanGesture(e); + return; + } + } + } + if (e.button === 2) { startMediaBoardMarqueeGesture(e); return; @@ -3630,14 +4605,65 @@ export function MediaPanel() { if (e.button !== 0 && e.button !== 1) return; startMediaBoardPanGesture(e, { clearSelectionOnTap: e.button === 0 && !e.ctrlKey && !e.metaKey }); - }, [startMediaBoardMarqueeGesture, startMediaBoardPanGesture]); + }, [ + getMediaBoardPlacementAtPoint, + handleItemClick, + mediaBoardRenderLod.overviewCanvas, + screenToMediaBoard, + selectedIds, + setSelection, + startMediaBoardMarqueeGesture, + startMediaBoardNodeMoveGesture, + startMediaBoardPanGesture, + ]); + + const handleMediaBoardDoubleClick = useCallback((e: React.MouseEvent) => { + if (!mediaBoardRenderLod.overviewCanvas) return; + const target = e.target as HTMLElement; + if (target.closest('.media-board-node, .media-board-group.folder-group, button, input, .context-menu')) return; + const hitPlacement = getMediaBoardPlacementAtPoint(screenToMediaBoard(e.clientX, e.clientY)); + if (!hitPlacement || isMediaBoardFolder(hitPlacement.item)) return; + e.preventDefault(); + e.stopPropagation(); + void handleItemDoubleClick(hitPlacement.item); + }, [ + getMediaBoardPlacementAtPoint, + handleItemDoubleClick, + mediaBoardRenderLod.overviewCanvas, + screenToMediaBoard, + ]); + + const handleMediaBoardContextMenu = useCallback((e: React.MouseEvent) => { + const target = e.target as HTMLElement; + if (target.closest('.media-board-node, .media-board-group.folder-group, button, input, .context-menu')) return; + + if (mediaBoardRenderLod.overviewCanvas) { + const hitPlacement = getMediaBoardPlacementAtPoint(screenToMediaBoard(e.clientX, e.clientY)); + if (hitPlacement && !isMediaBoardFolder(hitPlacement.item)) { + if (consumeSuppressedMediaBoardContextMenu()) { + e.preventDefault(); + return; + } + handleContextMenu(e, hitPlacement.item.id); + return; + } + } + + handleMediaBoardWorkspaceContextMenu(e); + }, [ + consumeSuppressedMediaBoardContextMenu, + getMediaBoardPlacementAtPoint, + handleContextMenu, + handleMediaBoardWorkspaceContextMenu, + mediaBoardRenderLod.overviewCanvas, + screenToMediaBoard, + ]); const handleMediaBoardNodeMouseDown = useCallback((e: React.MouseEvent, item: MediaBoardItem) => { const target = e.target as HTMLElement; if (target.closest('.media-board-node-timeline-drag, button, input')) return; if (e.button === 2) { - e.preventDefault(); e.stopPropagation(); if (e.ctrlKey || e.metaKey) { startMediaBoardMarqueeGesture(e); @@ -3657,9 +4683,7 @@ export function MediaPanel() { handleItemClick(item.id, e); - if (!e.ctrlKey && !e.metaKey && !e.shiftKey) { - startMediaBoardPanGesture(e); - } + startMediaBoardPanGesture(e); }, [ handleItemClick, setSelection, @@ -3682,12 +4706,7 @@ export function MediaPanel() { } const itemIds = selectedIds.includes(itemId) ? selectedIds : [itemId]; - if (itemIds.some((id) => folders.some((folder) => folder.id === id))) { - setMediaBoardInsertionPreview(null); - return false; - } - - const movingIds = itemIds.filter((id) => mediaBoardItemIds.has(id)); + const movingIds = getMediaBoardTopLevelMoveIds(itemIds); if (movingIds.length === 0) { setMediaBoardInsertionPreview(null); return false; @@ -3701,12 +4720,12 @@ export function MediaPanel() { return layouts; }, {}); - updateMediaBoardInsertionPreview(screenToMediaBoard(e.clientX, e.clientY), movingIds, sourceLayouts); + const point = screenToMediaBoard(e.clientX, e.clientY); + updateMediaBoardInsertionPreview(point, movingIds, sourceLayouts, point); return true; }, [ - folders, + getMediaBoardTopLevelMoveIds, internalDragId, - mediaBoardItemIds, mediaBoardPlacementsById, screenToMediaBoard, selectedIds, @@ -3722,20 +4741,11 @@ export function MediaPanel() { if (e.dataTransfer.types.includes('application/x-media-panel-item')) { const itemId = e.dataTransfer.getData('application/x-media-panel-item'); if (itemId) { - const itemsToMove = selectedIds.includes(itemId) ? selectedIds : [itemId]; - const isFolderMove = itemsToMove.some((id) => folders.some((folder) => folder.id === id)); - if (isFolderMove) { - const point = screenToMediaBoard(e.clientX, e.clientY); - const targetGroup = getMediaBoardGroupAtPoint(point); - if (targetGroup && canMoveItemsToMediaBoardGroup(itemsToMove, targetGroup.id)) { - moveToFolder(itemsToMove, targetGroup.id); - } - } else { - const point = screenToMediaBoard(e.clientX, e.clientY); - const target = getMediaBoardInsertTarget(point, itemsToMove); - const groupId = target?.groupId ?? null; - const index = target?.index ?? getMediaBoardAppendIndex(groupId, itemsToMove); - commitMediaBoardOrderChange(itemsToMove, groupId, index); + const itemsToMove = getMediaBoardTopLevelMoveIds(selectedIds.includes(itemId) ? selectedIds : [itemId]); + const point = screenToMediaBoard(e.clientX, e.clientY); + const target = getMediaBoardInsertTarget(point, itemsToMove); + if (target && canMoveItemsToMediaBoardGroup(itemsToMove, target.groupId)) { + commitMediaBoardOrderChange(itemsToMove, target.groupId, target.position); } } setDragOverFolderId(null); @@ -3746,7 +4756,7 @@ export function MediaPanel() { const point = screenToMediaBoard(e.clientX, e.clientY); const targetGroup = getMediaBoardGroupAtPoint(point); await handleExternalDropImport(e.dataTransfer, targetGroup?.id ?? null); - }, [canMoveItemsToMediaBoardGroup, commitMediaBoardOrderChange, folders, getMediaBoardAppendIndex, getMediaBoardGroupAtPoint, getMediaBoardInsertTarget, handleExternalDropImport, moveToFolder, screenToMediaBoard, selectedIds]); + }, [canMoveItemsToMediaBoardGroup, commitMediaBoardOrderChange, getMediaBoardGroupAtPoint, getMediaBoardInsertTarget, getMediaBoardTopLevelMoveIds, handleExternalDropImport, screenToMediaBoard, selectedIds]); const handleMediaBoardGroupDrop = useCallback(async (e: React.DragEvent, groupId: string | null) => { e.preventDefault(); @@ -3756,21 +4766,16 @@ export function MediaPanel() { if (e.dataTransfer.types.includes('application/x-media-panel-item')) { const itemId = e.dataTransfer.getData('application/x-media-panel-item'); if (itemId) { - const itemsToMove = selectedIds.includes(itemId) ? selectedIds : [itemId]; + const itemsToMove = getMediaBoardTopLevelMoveIds(selectedIds.includes(itemId) ? selectedIds : [itemId]); if (!canMoveItemsToMediaBoardGroup(itemsToMove, groupId)) { setDragOverFolderId(null); setInternalDragId(null); return; } - const isFolderMove = itemsToMove.some((id) => folders.some((folder) => folder.id === id)); - if (isFolderMove) { - moveToFolder(itemsToMove, groupId); - } else { - const point = screenToMediaBoard(e.clientX, e.clientY); - const target = getMediaBoardInsertTarget(point, itemsToMove); - const targetGroupId = target?.groupId ?? groupId; - const targetIndex = target?.index ?? getMediaBoardAppendIndex(targetGroupId, itemsToMove); - commitMediaBoardOrderChange(itemsToMove, targetGroupId, targetIndex); + const point = screenToMediaBoard(e.clientX, e.clientY); + const target = getMediaBoardInsertTarget(point, itemsToMove); + if (target) { + commitMediaBoardOrderChange(itemsToMove, target.groupId, target.position); } } setDragOverFolderId(null); @@ -3780,7 +4785,7 @@ export function MediaPanel() { await handleExternalDropImport(e.dataTransfer, groupId); setIsExternalDragOver(false); - }, [canMoveItemsToMediaBoardGroup, commitMediaBoardOrderChange, folders, getMediaBoardAppendIndex, getMediaBoardInsertTarget, handleExternalDropImport, moveToFolder, screenToMediaBoard, selectedIds]); + }, [canMoveItemsToMediaBoardGroup, commitMediaBoardOrderChange, getMediaBoardInsertTarget, getMediaBoardTopLevelMoveIds, handleExternalDropImport, screenToMediaBoard, selectedIds]); const handleMediaBoardGroupDragOver = useCallback((e: React.DragEvent) => { if (!e.dataTransfer.types.includes('application/x-media-panel-item') && !e.dataTransfer.types.includes('Files')) return; @@ -3793,11 +4798,21 @@ export function MediaPanel() { const resetMediaBoardLayout = useCallback(() => { setMediaBoardOrder({}); setMediaBoardGroupOffsets({}); + setMediaBoardLayouts({}); setMediaBoardViewport(DEFAULT_BOARD_VIEWPORT); }, []); + const mediaBoardOverviewCanvasStyle = useMemo(() => ({ + left: mediaBoardVisibleRect.left, + top: mediaBoardVisibleRect.top, + width: Math.max(1, mediaBoardVisibleRect.right - mediaBoardVisibleRect.left), + height: Math.max(1, mediaBoardVisibleRect.bottom - mediaBoardVisibleRect.top), + }), [mediaBoardVisibleRect]); + const renderMediaBoardNode = (placement: MediaBoardNodePlacement) => { const { item, layout } = placement; + if (isMediaBoardFolder(item)) return null; + const isSelected = selectedIdSet.has(item.id); const isMediaFile = isImportedMediaFileItem(item); const mediaFile = isMediaFile ? item : null; @@ -3826,6 +4841,7 @@ export function MediaPanel() { : getMediaFileCodecLabel(mediaFile); const isCompactNode = mediaBoardRenderLod.compact; const shouldRenderThumb = Boolean(thumbUrl && mediaBoardRenderLod.showImages); + if (mediaBoardRenderLod.overviewCanvas && !isSelected && !placement.isDraggingPreview) return null; return (
{ void refreshFileUrls(mediaFile.id); } : undefined} /> @@ -3923,7 +4940,11 @@ export function MediaPanel() {
Board - {mediaBoardItems.length} assets in {mediaBoardLayout.groups.length} groups + + {isMediaSearchActive + ? `${mediaSearchResultCount} of ${totalItems} items` + : `${mediaBoardItems.length} items in ${mediaBoardLayout.groups.filter((group) => group.id !== null).length} folders`} +
+ ) : null} +
+ ) : isMediaSearchActive && mediaSearchResultCount === 0 ? ( +
handleContextMenu(e)}> +

No matching items

+

{mediaSearchQuery}

+
) : viewMode === 'classic' ? (
{/* Column headers */} @@ -4309,7 +5383,7 @@ export function MediaPanel() { style={{ position: 'relative' }} > {/* Breadcrumb for folder navigation */} - {gridFolderId && ( + {!isMediaSearchActive && gridFolderId && (
{gridBreadcrumb.map((crumb, i) => ( diff --git a/src/components/panels/TextTab.tsx b/src/components/panels/TextTab.tsx index c035e78e..8b195ef9 100644 --- a/src/components/panels/TextTab.tsx +++ b/src/components/panels/TextTab.tsx @@ -20,31 +20,116 @@ interface CompactNumberProps { title: string; } +interface CompactNumberDragState { + startValue: number; + lastClientX: number; + accumulatedDelta: number; + pointerLockRequested: boolean; + pointerLockActive: boolean; + element: HTMLElement; +} + function CompactNumber({ value, onChange, min = 0, max = 999, step = 1, unit = 'px', icon, title }: CompactNumberProps) { - const dragStartRef = useRef<{ x: number; value: number } | null>(null); + const dragStateRef = useRef(null); + + const readDragDeltaX = useCallback((event: MouseEvent) => { + const state = dragStateRef.current; + if (!state) return 0; + + const isPointerLocked = state.pointerLockActive || document.pointerLockElement === state.element; + const movementX = Number.isFinite(event.movementX) ? event.movementX : 0; + if (isPointerLocked) return movementX; + + const clientDx = event.clientX - state.lastClientX; + state.lastClientX = event.clientX; + + if ( + state.pointerLockRequested && + movementX !== 0 && + Math.abs(clientDx) > Math.abs(movementX) * 4 + 8 + ) { + return movementX; + } + + return clientDx; + }, []); const handleMouseDown = useCallback((e: React.MouseEvent) => { + if (e.button !== 0) return; if ((e.target as HTMLElement).tagName === 'INPUT') return; e.preventDefault(); - dragStartRef.current = { x: e.clientX, value }; + const element = e.currentTarget as HTMLElement; + dragStateRef.current = { + startValue: value, + lastClientX: e.clientX, + accumulatedDelta: 0, + pointerLockRequested: false, + pointerLockActive: false, + element, + }; + + const handlePointerLockChange = () => { + const state = dragStateRef.current; + if (state) { + state.pointerLockActive = document.pointerLockElement === state.element; + } + }; + + document.addEventListener('pointerlockchange', handlePointerLockChange); + + if (element.requestPointerLock) { + dragStateRef.current.pointerLockRequested = true; + try { + const result = element.requestPointerLock(); + if (result && typeof result.then === 'function') { + void result.then( + () => { + const state = dragStateRef.current; + if (state) state.pointerLockActive = document.pointerLockElement === state.element; + }, + () => { + const state = dragStateRef.current; + if (state) { + state.pointerLockRequested = false; + state.pointerLockActive = false; + } + }, + ); + } + } catch { + dragStateRef.current.pointerLockRequested = false; + dragStateRef.current.pointerLockActive = false; + } + } const handleMouseMove = (moveEvent: MouseEvent) => { - if (!dragStartRef.current) return; - const delta = moveEvent.clientX - dragStartRef.current.x; + const state = dragStateRef.current; + if (!state) return; + if ((moveEvent.buttons & 1) !== 1) { + handleMouseUp(); + return; + } + + state.accumulatedDelta += readDragDeltaX(moveEvent); const sensitivity = step < 1 ? 0.5 : (step >= 10 ? 5 : 1); - const newValue = dragStartRef.current.value + Math.round(delta / sensitivity) * step; + const newValue = state.startValue + Math.round(state.accumulatedDelta / sensitivity) * step; onChange(Math.max(min, Math.min(max, newValue))); }; const handleMouseUp = () => { - dragStartRef.current = null; + const state = dragStateRef.current; + if (state && document.pointerLockElement === state.element) { + document.exitPointerLock?.(); + } + document.removeEventListener('pointerlockchange', handlePointerLockChange); + dragStateRef.current = null; document.removeEventListener('mousemove', handleMouseMove); document.removeEventListener('mouseup', handleMouseUp); }; document.addEventListener('mousemove', handleMouseMove); document.addEventListener('mouseup', handleMouseUp); - }, [value, onChange, min, max, step]); + }, [value, readDragDeltaX, onChange, min, max, step]); return (
diff --git a/src/components/panels/properties/MasksTab.tsx b/src/components/panels/properties/MasksTab.tsx index a53589ea..c6937306 100644 --- a/src/components/panels/properties/MasksTab.tsx +++ b/src/components/panels/properties/MasksTab.tsx @@ -1,5 +1,5 @@ // Masks Tab - focused clip mask creation and editing controls -import { useCallback, useEffect, useMemo, useState } from 'react'; +import { useCallback, useEffect, useMemo, useRef, useState } from 'react'; import { useTimelineStore } from '../../../stores/timeline'; import { startBatch, endBatch } from '../../../stores/historyStore'; import { getShortcutRegistry } from '../../../services/shortcutRegistry'; @@ -16,7 +16,7 @@ import { type MaskPathKeyframeValue, type MaskVertexHandleMode, } from '../../../types'; -import { DraggableNumber, KeyframeToggle } from './shared'; +import { DraggableNumber, KeyframeToggle, PrecisionSlider } from './shared'; import { MIDIParameterLabel } from './MIDIParameterLabel'; const MASK_MODES: { value: MaskMode; label: string }[] = [ @@ -24,8 +24,14 @@ const MASK_MODES: { value: MaskMode; label: string }[] = [ { value: 'subtract', label: 'Subtract' }, { value: 'intersect', label: 'Intersect' }, ]; +const DEFAULT_MASK_OUTLINE_COLOR = '#2997E5'; +const MASK_OUTLINE_COLORS = ['#2997E5', '#ff9900', '#7ddc7a', '#d16bff', '#ff5f6d', '#f8d34f']; const EMPTY_KEYFRAMES: Keyframe[] = []; +function getColorInputValue(color: string | undefined): string { + return /^#[0-9a-f]{6}$/i.test(color || '') ? color! : DEFAULT_MASK_OUTLINE_COLOR; +} + function isTypingTarget(target: EventTarget | null): boolean { return ( target instanceof HTMLInputElement || @@ -194,7 +200,7 @@ function MaskPathKeyframeToggle({ clipId, mask }: { clipId: string; mask: ClipMa
+ {colorMenuOpen && ( +
e.stopPropagation()} + onPointerDown={(e) => e.stopPropagation()} + onContextMenu={(e) => { + e.preventDefault(); + e.stopPropagation(); + }} + > + {MASK_OUTLINE_COLORS.map(color => ( +
+ )}
); } @@ -520,8 +625,6 @@ export function MasksTab({ clipId, masks }: MasksTabProps) { const activeMaskFeatherProperty = activeMask ? createMaskNumericProperty(activeMask.id, 'feather') : null; const activeMaskFeatherQualityProperty = activeMask ? createMaskNumericProperty(activeMask.id, 'featherQuality') : null; - const activeMaskPositionXProperty = activeMask ? createMaskNumericProperty(activeMask.id, 'position.x') : null; - const activeMaskPositionYProperty = activeMask ? createMaskNumericProperty(activeMask.id, 'position.y') : null; return (
@@ -572,7 +675,6 @@ export function MasksTab({ clipId, masks }: MasksTabProps) {
{activeMask.name} - {activeMask.closed ? 'Closed path' : 'Open path'} / {activeMask.vertices.length} vertices / {selectedVertexIds.size} selected
@@ -657,6 +759,11 @@ export function MasksTab({ clipId, masks }: MasksTabProps) {
Edge
+
+ + + {activeMask.vertices.length} vertices +
)} + activeMaskFeatherProperty + ? setPropertyValue(clipId, activeMaskFeatherProperty, Math.max(0, v)) + : updateMask(clipId, activeMask.id, { feather: Math.max(0, v) })} + defaultValue={0} + min={0} + max={500} + step={1} + onDragStart={handleBatchStart} + onDragEnd={handleBatchEnd} + /> activeMaskFeatherProperty @@ -721,68 +840,6 @@ export function MasksTab({ clipId, masks }: MasksTabProps) { />
- -
-
Transform
-
- - X - - {activeMaskPositionXProperty && ( - - )} - activeMaskPositionXProperty - ? setPropertyValue(clipId, activeMaskPositionXProperty, v) - : updateMask(clipId, activeMask.id, { position: { ...activeMask.position, x: v } })} - defaultValue={0} - sensitivity={100} - decimals={3} - onDragStart={handleBatchStart} - onDragEnd={handleBatchEnd} - /> -
-
- - Y - - {activeMaskPositionYProperty && ( - - )} - activeMaskPositionYProperty - ? setPropertyValue(clipId, activeMaskPositionYProperty, v) - : updateMask(clipId, activeMask.id, { position: { ...activeMask.position, y: v } })} - defaultValue={0} - sensitivity={100} - decimals={3} - onDragStart={handleBatchStart} - onDragEnd={handleBatchEnd} - /> -
-
)} diff --git a/src/components/preview/MaskOverlay.tsx b/src/components/preview/MaskOverlay.tsx index d855999a..26504ee9 100644 --- a/src/components/preview/MaskOverlay.tsx +++ b/src/components/preview/MaskOverlay.tsx @@ -15,6 +15,8 @@ import { useMaskDrag } from './useMaskDrag'; import { useMaskEdgeDrag } from './useMaskEdgeDrag'; import { useMaskShapeDraw } from './useMaskShapeDraw'; +const DEFAULT_MASK_OUTLINE_COLOR = '#2997E5'; + interface MaskOverlayProps { canvasWidth: number; canvasHeight: number; @@ -348,6 +350,10 @@ function buildProjectedMaskPath( return d; } +function getMaskOutlineColor(mask: ClipMask): string { + return mask.outlineColor || DEFAULT_MASK_OUTLINE_COLOR; +} + export function MaskOverlay({ canvasWidth, canvasHeight, displayWidth, displayHeight }: MaskOverlayProps) { const svgRef = useRef(null); const suppressNextSvgClickRef = useRef(false); @@ -614,6 +620,18 @@ export function MaskOverlay({ canvasWidth, canvasHeight, displayWidth, displayHe if (!activeMask) return ''; return buildProjectedMaskPath(activeMask, projectMaskPoint); }, [activeMask, projectMaskPoint]); + const visibleMaskPaths = useMemo( + () => (selectedClipMasks || []) + .filter(mask => mask.visible && mask.vertices.length >= 2) + .map(mask => ({ + id: mask.id, + d: buildProjectedMaskPath(mask, projectMaskPoint), + closed: mask.closed, + color: getMaskOutlineColor(mask), + })) + .filter(path => path.d.length > 0), + [projectMaskPoint, selectedClipMasks], + ); // Generate individual edge path segments for hit testing const edgeSegments = useMemo(() => { @@ -1048,17 +1066,18 @@ export function MaskOverlay({ canvasWidth, canvasHeight, displayWidth, displayHe /> )} - {/* Mask path stroke - only when visible */} - {activeMask && activeMask.visible && pathData && ( + {/* Mask path strokes - show every visible mask outline */} + {visibleMaskPaths.map(maskPath => ( - )} + ))} {maskEditMode === 'drawingPen' && penInsertPreview && ( diff --git a/src/components/timeline/Timeline.tsx b/src/components/timeline/Timeline.tsx index 7cb4c982..ad318c13 100644 --- a/src/components/timeline/Timeline.tsx +++ b/src/components/timeline/Timeline.tsx @@ -179,6 +179,7 @@ export function Timeline() { // Keyframe state const { selectedKeyframeIds, clipKeyframes, expandedCurveProperties } = useTimelineStore(useShallow(selectKeyframeState)); + const expandedTracks = useTimelineStore(state => state.expandedTracks); // =========================================== // STABLE ACTION REFERENCES @@ -193,7 +194,7 @@ export function Timeline() { const { play, pause, stop, playForward, playReverse, setPlayheadPosition, setDraggingPlayhead } = store; // Track actions - const { addTrack, isTrackExpanded, toggleTrackExpanded, getExpandedTrackHeight, trackHasKeyframes, setTrackParent } = store; + const { addTrack, toggleTrackExpanded, getExpandedTrackHeight, trackHasKeyframes, setTrackParent } = store; // Clip actions const { @@ -277,6 +278,14 @@ export function Timeline() { const compositionSwitchTargetTracks = useTimelineStore(s => s.compositionSwitchTargetTracks); const compositionSwitchSourceTracksRef = useRef(null); const isCompositionTrackMorphing = clipAnimationPhase !== 'idle' && compositionSwitchTargetTracks !== null; + const isTrackExpandedFromState = useCallback( + (trackId: string) => expandedTracks.has(trackId), + [expandedTracks], + ); + const isTrackExpandedForRender = useCallback( + (trackId: string) => !isCompositionTrackMorphing && expandedTracks.has(trackId), + [expandedTracks, isCompositionTrackMorphing], + ); const timelineViewTracks = useMemo( () => isCompositionTrackMorphing ? buildCompositionSwitchTracks( @@ -568,7 +577,7 @@ export function Timeline() { selectKeyframe, deselectAllKeyframes, pixelToTime, - isTrackExpanded, + isTrackExpanded: isTrackExpandedForRender, getExpandedTrackHeight, }); @@ -621,17 +630,26 @@ export function Timeline() { // Calculate total content height and track snap positions for vertical scrollbar // Dependencies: tracks, expansion state, curve editor state, selected clips (affects property rows) - const { contentHeight, trackSnapPositions } = useMemo(() => { + const { contentHeight, trackSnapPositions, renderedTrackHeights } = useMemo(() => { let totalHeight = 0; const snapPositions: number[] = [0]; + const trackHeights = new Map(); for (const track of timelineViewTracks) { - const isExpanded = !isCompositionTrackMorphing && isTrackExpanded(track.id); - totalHeight += isExpanded ? getExpandedTrackHeight(track.id, track.height) : track.height; + const isExpanded = isTrackExpandedForRender(track.id); + const trackHeight = isExpanded ? getExpandedTrackHeight(track.id, track.height) : track.height; + trackHeights.set(track.id, trackHeight); + totalHeight += trackHeight; snapPositions.push(totalHeight); } - return { contentHeight: totalHeight, trackSnapPositions: snapPositions }; + return { contentHeight: totalHeight, trackSnapPositions: snapPositions, renderedTrackHeights: trackHeights }; // eslint-disable-next-line react-hooks/exhaustive-deps - }, [timelineViewTracks, isCompositionTrackMorphing, isTrackExpanded, getExpandedTrackHeight, expandedCurveProperties, curveEditorHeight, selectedClipIds, clipKeyframes]); + }, [timelineViewTracks, isTrackExpandedForRender, getExpandedTrackHeight, expandedCurveProperties, curveEditorHeight, selectedClipIds, clipKeyframes]); + const getRenderedTrackHeight = useCallback( + (trackId: string, baseHeight: number) => + renderedTrackHeights.get(trackId) ?? + (isTrackExpandedForRender(trackId) ? getExpandedTrackHeight(trackId, baseHeight) : baseHeight), + [renderedTrackHeights, isTrackExpandedForRender, getExpandedTrackHeight], + ); // Track viewport height for scrollbar const [viewportHeight, setViewportHeight] = useState(300); @@ -1172,8 +1190,8 @@ export function Timeline() { const isDimmed = (track.type === 'video' && anyViewVideoSolo && !track.solo) || (track.type === 'audio' && anyViewAudioSolo && !track.solo); - const isExpanded = !isCompositionTrackMorphing && isTrackExpanded(track.id); - const dynamicHeight = isExpanded ? getExpandedTrackHeight(track.id, track.height) : track.height; + const isExpanded = isTrackExpandedForRender(track.id); + const dynamicHeight = getRenderedTrackHeight(track.id, track.height); return ( {tracks.map((track) => { - const isExpanded = isTrackExpanded(track.id); + const isExpanded = isTrackExpandedFromState(track.id); const dynamicHeight = isExpanded ? getExpandedTrackHeight(track.id, track.height) : track.height; const trackClips = clips.filter((clip) => clip.trackId === track.id); @@ -1369,8 +1387,8 @@ export function Timeline() { clips={clips} tracks={tracks} timeToPixel={timeToPixel} - isTrackExpanded={isTrackExpanded} - getExpandedTrackHeight={getExpandedTrackHeight} + isTrackExpanded={isTrackExpandedForRender} + getExpandedTrackHeight={getRenderedTrackHeight} /> )} @@ -1378,8 +1396,8 @@ export function Timeline() { )} @@ -1500,7 +1518,7 @@ export function Timeline() { timelineRef={timelineRef} scrollX={scrollX} zoom={zoom} - getExpandedTrackHeight={getExpandedTrackHeight} + getExpandedTrackHeight={getRenderedTrackHeight} /> )} diff --git a/src/components/timeline/TimelineHeader.tsx b/src/components/timeline/TimelineHeader.tsx index 7965ac17..a65286b9 100644 --- a/src/components/timeline/TimelineHeader.tsx +++ b/src/components/timeline/TimelineHeader.tsx @@ -907,6 +907,7 @@ function TrackPropertyLabels({ function TimelineHeaderComponent({ track, + tracks, isDimmed, isExpanded, dynamicHeight, @@ -935,6 +936,8 @@ function TimelineHeaderComponent({ // Get the first selected clip in this track const trackClips = clips.filter((c) => c.trackId === track.id); const selectedTrackClip = trackClips.find((c) => selectedClipIds.has(c.id)); + const videoLayerIndex = tracks.filter((timelineTrack) => timelineTrack.type === 'video').findIndex((timelineTrack) => timelineTrack.id === track.id); + const layerDisplayId = videoLayerIndex >= 0 ? videoLayerIndex + 1 : null; // Editing state for track name const [isEditing, setIsEditing] = useState(false); @@ -949,13 +952,22 @@ function TimelineHeaderComponent({ } }, [isEditing]); - // Handle double-click on name to edit - const handleDoubleClick = (e: React.MouseEvent) => { - e.stopPropagation(); + const startNameEdit = () => { setEditValue(track.name); setIsEditing(true); }; + // Handle click on name to edit without toggling track expansion + const handleNameClick = (e: React.MouseEvent) => { + e.stopPropagation(); + startNameEdit(); + }; + + const handleNameDoubleClick = (e: React.MouseEvent) => { + e.stopPropagation(); + startNameEdit(); + }; + // Handle finishing edit const handleFinishEdit = () => { const trimmed = editValue.trim(); @@ -1024,13 +1036,26 @@ function TimelineHeaderComponent({ onClick={(e) => e.stopPropagation()} /> ) : ( - - {track.name} - + <> + + {track.name} + + {layerDisplayId !== null && ( + + {`(id:${layerDisplayId})`} + + )} + )}
diff --git a/src/engine/native3d/NativeSceneRenderer.ts b/src/engine/native3d/NativeSceneRenderer.ts index 7b852650..7cfb446f 100644 --- a/src/engine/native3d/NativeSceneRenderer.ts +++ b/src/engine/native3d/NativeSceneRenderer.ts @@ -35,7 +35,7 @@ const MODEL_SEQUENCE_GPU_RETAIN_AHEAD = 8; const MODEL_SEQUENCE_GPU_RETAIN_BEHIND = 3; interface CachedPlaneTexture { - source: HTMLVideoElement | HTMLImageElement | HTMLCanvasElement; + source: HTMLVideoElement | HTMLImageElement | HTMLCanvasElement | VideoFrame; texture: GPUTexture; view: GPUTextureView; width: number; @@ -726,18 +726,19 @@ export class NativeSceneRenderer { const current = this.planeTextures.get(layer.layerId); const sourceState = this.resolvePlaneTextureSource(layer, current); if (!sourceState) { - return layer.videoElement ? current?.view ?? null : null; + return layer.videoElement || layer.videoFrame ? current?.view ?? null : null; } + const sameSource = sourceState.transient === true || current?.source === sourceState.source; const canReuseCurrent = !!current && - current.source === sourceState.source && + sameSource && current.width === sourceState.width && current.height === sourceState.height; let cached = current; if ( !cached || - cached.source !== sourceState.source || + (sourceState.transient !== true && cached.source !== sourceState.source) || cached.width !== sourceState.width || cached.height !== sourceState.height ) { @@ -759,6 +760,8 @@ export class NativeSceneRenderer { } else if (sourceState.videoCanvas) { cached.videoCanvas = sourceState.videoCanvas; cached.source = sourceState.source; + } else if (sourceState.transient) { + cached.source = sourceState.source; } try { @@ -973,11 +976,29 @@ export class NativeSceneRenderer { layer: ScenePlaneLayer, cached?: CachedPlaneTexture, ): { - source: HTMLVideoElement | HTMLImageElement | HTMLCanvasElement; + source: HTMLVideoElement | HTMLImageElement | HTMLCanvasElement | VideoFrame; width: number; height: number; + transient?: boolean; videoCanvas?: HTMLCanvasElement; } | null { + if (layer.videoFrame) { + const width = Math.max( + 1, + Math.floor(layer.videoFrame.displayWidth || layer.videoFrame.codedWidth || layer.sourceWidth || 1), + ); + const height = Math.max( + 1, + Math.floor(layer.videoFrame.displayHeight || layer.videoFrame.codedHeight || layer.sourceHeight || 1), + ); + return { + source: layer.videoFrame, + width, + height, + transient: true, + }; + } + if (layer.videoElement) { const width = Math.max( 1, @@ -1126,7 +1147,7 @@ export class NativeSceneRenderer { if (layer.alphaMode === 'opaque') { return true; } - return !!layer.videoElement; + return !!(layer.videoElement || layer.videoFrame); } private prunePlaneTextureCache(activeLayerIds: Set): void { diff --git a/src/engine/scene/SceneLayerCollector.ts b/src/engine/scene/SceneLayerCollector.ts index 25dca783..61682e61 100644 --- a/src/engine/scene/SceneLayerCollector.ts +++ b/src/engine/scene/SceneLayerCollector.ts @@ -166,6 +166,7 @@ export function collectScene3DLayers( ...base, kind: 'plane', alphaMode: source?.videoElement + || source?.videoFrame ? 'opaque' : source?.imageElement ? 'straight' @@ -173,9 +174,10 @@ export function collectScene3DLayers( ? 'premultiplied' : undefined, doubleSided: true, - castsDepth: !!source?.videoElement, + castsDepth: !!(source?.videoElement || source?.videoFrame), receivesDepth: true, videoElement: source?.videoElement ?? undefined, + videoFrame: source?.videoFrame ?? undefined, preciseVideoSampling: options.preciseVideoSampling, imageElement: source?.imageElement ?? undefined, canvas: source?.textCanvas ?? undefined, diff --git a/src/engine/scene/types.ts b/src/engine/scene/types.ts index b37e4c87..47305a6f 100644 --- a/src/engine/scene/types.ts +++ b/src/engine/scene/types.ts @@ -43,6 +43,7 @@ export interface SceneLayerBase { export interface ScenePlaneLayer extends SceneLayerBase { kind: 'plane'; videoElement?: HTMLVideoElement; + videoFrame?: VideoFrame; preciseVideoSampling?: boolean; imageElement?: HTMLImageElement; canvas?: HTMLCanvasElement; diff --git a/src/services/project/projectLoad.ts b/src/services/project/projectLoad.ts index 59289b2e..4f7f9f08 100644 --- a/src/services/project/projectLoad.ts +++ b/src/services/project/projectLoad.ts @@ -65,6 +65,7 @@ import type { const log = Logger.create('ProjectSync'); const MEDIA_PANEL_PROJECT_UI_LOADED_EVENT = 'media-panel-project-ui-loaded'; +const CACHED_THUMBNAIL_RESTORE_BATCH_SIZE = 48; type ProjectLoadProgressUpdate = Partial> & { message: string; @@ -666,6 +667,7 @@ function convertProjectCompositionToStore( featherQuality: mask.featherQuality ?? 50, enabled: mask.enabled !== false, visible: mask.visible !== false, + outlineColor: mask.outlineColor, closed: mask.closed, expanded: false, position: mask.position, @@ -1100,6 +1102,11 @@ export async function loadProjectToStores(): Promise { } else { removeLocalStorageKey('media-panel-board-group-offsets'); } + if (projectData.uiState?.mediaPanelBoardLayouts) { + localStorage.setItem('media-panel-board-layouts', JSON.stringify(projectData.uiState.mediaPanelBoardLayouts)); + } else { + removeLocalStorageKey('media-panel-board-layouts'); + } removeLocalStorageKey('media-panel-board-layout'); window.dispatchEvent(new CustomEvent(MEDIA_PANEL_PROJECT_UI_LOADED_EVENT)); if (projectData.uiState?.transcriptLanguage) { @@ -1182,14 +1189,92 @@ async function applyProjectRestoreMediaUpdate( }); } -async function runPostLoadRestoration(projectData: ProjectFile, hydrateFiles: boolean): Promise { - try { - if (!hydrateFiles) { - log.info('Skipping eager post-load restoration for native backend; media details are restored lazily'); - completeProjectLoadProgress('Project ready'); - return; +function isProjectMediaThumbnailCandidate(media: ProjectMediaFile): boolean { + return Boolean(media.fileHash) && (media.type === 'image' || media.type === 'video'); +} + +async function applyCachedThumbnailBatch(thumbnailsById: Map): Promise { + if (thumbnailsById.size === 0) return 0; + + const thumbnailEntries = [...thumbnailsById.entries()]; + const appliedUrls = new Set(); + await applyProjectRestoreMediaUpdate((state) => ({ + files: state.files.map((file) => { + const thumbnailUrl = thumbnailsById.get(file.id); + if (!thumbnailUrl || file.thumbnailUrl) return file; + appliedUrls.add(thumbnailUrl); + return { ...file, thumbnailUrl }; + }), + })); + + thumbnailEntries.forEach(([, thumbnailUrl]) => { + if (!appliedUrls.has(thumbnailUrl) && thumbnailUrl.startsWith('blob:')) { + URL.revokeObjectURL(thumbnailUrl); + } + }); + thumbnailsById.clear(); + return appliedUrls.size; +} + +async function restoreCachedMediaThumbnails( + projectMedia: ProjectMediaFile[], + onProgress?: (done: number, total: number, name: string) => void, +): Promise { + const candidates = projectMedia.filter(isProjectMediaThumbnailCandidate); + const thumbnailsById = new Map(); + let restoredCount = 0; + + for (let index = 0; index < candidates.length; index += 1) { + const media = candidates[index]; + onProgress?.(index, candidates.length, media.name); + + const currentFile = useMediaStore.getState().files.find((file) => file.id === media.id); + if (currentFile?.thumbnailUrl || !media.fileHash) { + continue; + } + + try { + const storedThumbnail = await projectDB.getThumbnail(media.fileHash); + let thumbnailBlob = storedThumbnail?.blob ?? null; + + if ((!thumbnailBlob || thumbnailBlob.size <= 0) && projectFileService.isProjectOpen()) { + thumbnailBlob = await projectFileService.getThumbnail(media.fileHash); + if (thumbnailBlob && thumbnailBlob.size > 0) { + void projectDB.saveThumbnail({ + fileHash: media.fileHash, + blob: thumbnailBlob, + createdAt: Date.now(), + }); + } + } + + if (thumbnailBlob && thumbnailBlob.size > 0) { + thumbnailsById.set(media.id, URL.createObjectURL(thumbnailBlob)); + } + + if (thumbnailsById.size >= CACHED_THUMBNAIL_RESTORE_BATCH_SIZE) { + restoredCount += await applyCachedThumbnailBatch(thumbnailsById); + } + } catch (error) { + log.debug('Cached thumbnail restore skipped', { + id: media.id, + name: media.name, + error, + }); + } + + if (index % 12 === 0) { + await yieldToBrowser(); } + } + + restoredCount += await applyCachedThumbnailBatch(thumbnailsById); + onProgress?.(candidates.length, candidates.length, ''); + return restoredCount; +} +async function runPostLoadRestoration(projectData: ProjectFile, hydrateFiles: boolean): Promise { + try { if (hydrateFiles) { setProjectLoadProgress({ phase: 'relink', @@ -1198,11 +1283,44 @@ async function runPostLoadRestoration(projectData: ProjectFile, hydrateFiles: bo blocking: false, }); await autoRelinkFromRawFolder(); + } else { + log.info('Skipping eager file restoration for native backend; media details are restored lazily'); } await yieldToBrowser(); - log.info('Skipping eager thumbnail restoration; media panel restores visible thumbnails lazily'); + const cachedThumbnailCandidates = projectData.media.filter(isProjectMediaThumbnailCandidate).length; + if (cachedThumbnailCandidates > 0) { + setProjectLoadProgress({ + phase: 'thumbnails', + percent: 78, + message: 'Restoring cached thumbnails', + itemsDone: 0, + itemsTotal: cachedThumbnailCandidates, + blocking: false, + }); + const restoredCount = await restoreCachedMediaThumbnails(projectData.media, (done, total, name) => { + const ratio = total > 0 ? done / total : 1; + setProjectLoadProgress({ + phase: 'thumbnails', + percent: 78 + ratio * 8, + message: 'Restoring cached thumbnails', + detail: name, + itemsDone: done, + itemsTotal: total, + blocking: false, + }); + }); + log.info('Restored cached media thumbnails', { + restoredCount, + candidateCount: cachedThumbnailCandidates, + }); + } + + if (!hydrateFiles) { + completeProjectLoadProgress('Project ready'); + return; + } const eagerMetadataLimit = 120; if (projectData.media.length <= eagerMetadataLimit) { diff --git a/src/services/project/projectSave.ts b/src/services/project/projectSave.ts index 0fd6d411..7fe6b378 100644 --- a/src/services/project/projectSave.ts +++ b/src/services/project/projectSave.ts @@ -35,6 +35,7 @@ import type { } from '../../types'; import type { ProjectMediaBoardGroupOffsets, + ProjectMediaBoardNodeLayout, ProjectMediaBoardOrder, ProjectMediaBoardViewport, } from './types/project.types'; @@ -216,6 +217,7 @@ function convertCompositions(compositions: Composition[]): ProjectComposition[] featherQuality: m.featherQuality ?? 50, enabled: m.enabled !== false, visible: m.visible !== false, + outlineColor: m.outlineColor, closed: m.closed !== false, vertices: (m.vertices || []).map((vertex) => ({ x: vertex.x, @@ -492,6 +494,7 @@ export async function syncStoresToProject(): Promise { const mediaPanelBoardViewport = parseLocalStorageJson('media-panel-board-viewport'); const mediaPanelBoardOrder = parseLocalStorageJson('media-panel-board-order'); const mediaPanelBoardGroupOffsets = parseLocalStorageJson('media-panel-board-group-offsets'); + const mediaPanelBoardLayouts = parseLocalStorageJson>('media-panel-board-layouts'); const transcriptLanguage = localStorage.getItem('transcriptLanguage'); const settingsState = useSettingsStore.getState(); const midiState = useMIDIStore.getState(); @@ -505,6 +508,7 @@ export async function syncStoresToProject(): Promise { mediaPanelBoardViewport, mediaPanelBoardOrder, mediaPanelBoardGroupOffsets, + mediaPanelBoardLayouts, transcriptLanguage: transcriptLanguage || undefined, thumbnailsEnabled: timelineState.thumbnailsEnabled, waveformsEnabled: timelineState.waveformsEnabled, diff --git a/src/services/project/types/project.types.ts b/src/services/project/types/project.types.ts index 3ddc3826..7673c32f 100644 --- a/src/services/project/types/project.types.ts +++ b/src/services/project/types/project.types.ts @@ -56,8 +56,8 @@ export interface ProjectMediaBoardViewport { export interface ProjectMediaBoardNodeLayout { x: number; y: number; - width: number; - height: number; + width?: number; + height?: number; } export type ProjectMediaBoardOrder = Record; @@ -82,7 +82,6 @@ export interface ProjectUIState { mediaPanelBoardViewport?: ProjectMediaBoardViewport; mediaPanelBoardOrder?: ProjectMediaBoardOrder; mediaPanelBoardGroupOffsets?: ProjectMediaBoardGroupOffsets; - /** @deprecated Board nodes now snap to folder slot grids; retained only to ignore older project files safely. */ mediaPanelBoardLayouts?: Record; // Transcript settings transcriptLanguage?: string; diff --git a/src/services/project/types/timeline.types.ts b/src/services/project/types/timeline.types.ts index 3b9998e1..29aca71f 100644 --- a/src/services/project/types/timeline.types.ts +++ b/src/services/project/types/timeline.types.ts @@ -43,6 +43,7 @@ export interface ProjectMask { featherQuality: number; enabled: boolean; visible: boolean; + outlineColor?: string; closed: boolean; vertices: ProjectMaskVertex[]; position: { x: number; y: number }; diff --git a/src/stores/mediaStore/helpers/thumbnailHelpers.ts b/src/stores/mediaStore/helpers/thumbnailHelpers.ts index a6539349..9c349bca 100644 --- a/src/stores/mediaStore/helpers/thumbnailHelpers.ts +++ b/src/stores/mediaStore/helpers/thumbnailHelpers.ts @@ -2,13 +2,15 @@ import { THUMBNAIL_TIMEOUT } from '../constants'; import { projectFileService } from '../../../services/projectFileService'; +import { projectDB } from '../../../services/projectDB'; import { Logger } from '../../../services/logger'; const log = Logger.create('Thumbnail'); -const THUMBNAIL_MAX_WIDTH = 320; -const THUMBNAIL_MAX_HEIGHT = 240; -const THUMBNAIL_QUALITY = 0.72; +const THUMBNAIL_MAX_WIDTH = 256; +const THUMBNAIL_MAX_HEIGHT = 192; +const THUMBNAIL_QUALITY = 0.68; +const isBlobUrl = (value?: string): value is string => typeof value === 'string' && value.startsWith('blob:'); /** * Create thumbnail for video or image. @@ -55,18 +57,9 @@ export async function createThumbnail( }; video.onseeked = () => { - const canvas = document.createElement('canvas'); - const size = getThumbnailCanvasSize(video.videoWidth || 16, video.videoHeight || 9); - canvas.width = size.width; - canvas.height = size.height; - const ctx = canvas.getContext('2d'); - if (ctx) { - ctx.drawImage(video, 0, 0, canvas.width, canvas.height); - resolve(canvasToThumbnailDataUrl(canvas)); - } else { - resolve(undefined); - } - cleanup(); + void drawThumbnailFromSource(video, video.videoWidth || 16, video.videoHeight || 9) + .then(resolve) + .finally(cleanup); }; video.onerror = () => { @@ -82,6 +75,19 @@ export async function createThumbnail( } async function createImageThumbnail(file: File): Promise { + if (typeof createImageBitmap === 'function') { + try { + const bitmap = await createImageBitmap(file); + try { + return await drawThumbnailFromSource(bitmap, bitmap.width, bitmap.height); + } finally { + bitmap.close(); + } + } catch { + // Fall back to HTMLImageElement decoding for formats createImageBitmap cannot decode. + } + } + return new Promise((resolve) => { const image = new Image(); const url = URL.createObjectURL(file); @@ -108,22 +114,9 @@ async function createImageThumbnail(file: File): Promise { return; } - const size = getThumbnailCanvasSize(width, height); - const canvas = document.createElement('canvas'); - canvas.width = size.width; - canvas.height = size.height; - const ctx = canvas.getContext('2d'); - - if (!ctx) { - cleanup(); - resolve(undefined); - return; - } - - ctx.drawImage(image, 0, 0, canvas.width, canvas.height); - const thumbnail = canvasToThumbnailDataUrl(canvas); - cleanup(); - resolve(thumbnail); + void drawThumbnailFromSource(image, width, height) + .then(resolve) + .finally(cleanup); }; image.onerror = () => { @@ -136,6 +129,25 @@ async function createImageThumbnail(file: File): Promise { }); } +async function drawThumbnailFromSource( + source: CanvasImageSource, + sourceWidth: number, + sourceHeight: number, +): Promise { + const size = getThumbnailCanvasSize(sourceWidth, sourceHeight); + const canvas = document.createElement('canvas'); + canvas.width = size.width; + canvas.height = size.height; + const ctx = canvas.getContext('2d'); + + if (!ctx) return undefined; + + ctx.imageSmoothingEnabled = true; + ctx.imageSmoothingQuality = 'medium'; + ctx.drawImage(source, 0, 0, canvas.width, canvas.height); + return canvasToThumbnailUrl(canvas); +} + function getThumbnailCanvasSize(sourceWidth: number, sourceHeight: number): { width: number; height: number } { if (!Number.isFinite(sourceWidth) || !Number.isFinite(sourceHeight) || sourceWidth <= 0 || sourceHeight <= 0) { return { width: THUMBNAIL_MAX_WIDTH, height: Math.round(THUMBNAIL_MAX_WIDTH * 9 / 16) }; @@ -153,14 +165,26 @@ function getThumbnailCanvasSize(sourceWidth: number, sourceHeight: number): { wi }; } -function canvasToThumbnailDataUrl(canvas: HTMLCanvasElement): string { - const webp = canvas.toDataURL('image/webp', THUMBNAIL_QUALITY); - if (webp.startsWith('data:image/webp')) { - return webp; +async function canvasToThumbnailUrl(canvas: HTMLCanvasElement): Promise { + const webp = await canvasToBlob(canvas, 'image/webp', THUMBNAIL_QUALITY); + if (webp?.type === 'image/webp' && webp.size > 0) { + return URL.createObjectURL(webp); + } + + const jpeg = await canvasToBlob(canvas, 'image/jpeg', THUMBNAIL_QUALITY); + if (jpeg && jpeg.size > 0) { + return URL.createObjectURL(jpeg); } + return canvas.toDataURL('image/jpeg', THUMBNAIL_QUALITY); } +function canvasToBlob(canvas: HTMLCanvasElement, type: string, quality: number): Promise { + return new Promise((resolve) => { + canvas.toBlob(resolve, type, quality); + }); +} + /** * Handle thumbnail deduplication - check for existing, save new. * UNIFIED: Replaces 3 duplicate blocks in original code. @@ -169,15 +193,30 @@ export async function handleThumbnailDedup( fileHash: string | undefined, thumbnailUrl: string | undefined ): Promise { - if (!fileHash || !projectFileService.isProjectOpen()) { + if (!fileHash) { return thumbnailUrl; } try { - // Check for existing thumbnail - const existingBlob = await projectFileService.getThumbnail(fileHash); + let existingBlob: Blob | null = null; + + if (projectFileService.isProjectOpen()) { + existingBlob = await projectFileService.getThumbnail(fileHash); + } + + if (!existingBlob || existingBlob.size <= 0) { + const storedThumbnail = await projectDB.getThumbnail(fileHash); + existingBlob = storedThumbnail?.blob ?? null; + } + if (existingBlob && existingBlob.size > 0) { log.debug('Reusing existing for hash:', fileHash.slice(0, 8)); + if (isBlobUrl(thumbnailUrl)) { + URL.revokeObjectURL(thumbnailUrl); + } + if (projectFileService.isProjectOpen()) { + void projectFileService.saveThumbnail(fileHash, existingBlob); + } return URL.createObjectURL(existingBlob); } @@ -185,8 +224,15 @@ export async function handleThumbnailDedup( if (thumbnailUrl) { const blob = await fetchThumbnailBlob(thumbnailUrl); if (blob && blob.size > 0) { - await projectFileService.saveThumbnail(fileHash, blob); - log.debug('Saved to project folder:', fileHash.slice(0, 8)); + await projectDB.saveThumbnail({ + fileHash, + blob, + createdAt: Date.now(), + }); + if (projectFileService.isProjectOpen()) { + await projectFileService.saveThumbnail(fileHash, blob); + } + log.debug('Saved thumbnail cache:', fileHash.slice(0, 8)); } } } catch (e) { diff --git a/src/stores/mediaStore/slices/fileManageSlice.ts b/src/stores/mediaStore/slices/fileManageSlice.ts index 507cc973..c7080c18 100644 --- a/src/stores/mediaStore/slices/fileManageSlice.ts +++ b/src/stores/mediaStore/slices/fileManageSlice.ts @@ -66,6 +66,21 @@ export const createFileManageSlice: MediaSliceCreator = (set, const existingBlob = await projectFileService.getThumbnail(mediaFile.fileHash); if (existingBlob && existingBlob.size > 0) { thumbnailUrl = URL.createObjectURL(existingBlob); + void projectDB.saveThumbnail({ + fileHash: mediaFile.fileHash, + blob: existingBlob, + createdAt: Date.now(), + }); + } + } + + if (!thumbnailUrl && mediaFile.fileHash) { + const storedThumbnail = await projectDB.getThumbnail(mediaFile.fileHash); + if (storedThumbnail?.blob && storedThumbnail.blob.size > 0) { + thumbnailUrl = URL.createObjectURL(storedThumbnail.blob); + if (projectFileService.isProjectOpen()) { + void projectFileService.saveThumbnail(mediaFile.fileHash, storedThumbnail.blob); + } } } diff --git a/src/stores/mediaStore/slices/projectSlice.ts b/src/stores/mediaStore/slices/projectSlice.ts index e3edaa61..b445d14a 100644 --- a/src/stores/mediaStore/slices/projectSlice.ts +++ b/src/stores/mediaStore/slices/projectSlice.ts @@ -103,10 +103,17 @@ export const createProjectSlice: MediaSliceCreator = (set, get) } } - // Restore thumbnail from project folder - if (stored.fileHash && projectFileService.isProjectOpen()) { - const thumbBlob = await projectFileService.getThumbnail(stored.fileHash); - if (thumbBlob) { + // Restore thumbnail from project folder or IndexedDB cache. + if (stored.fileHash) { + let thumbBlob: Blob | null = null; + if (projectFileService.isProjectOpen()) { + thumbBlob = await projectFileService.getThumbnail(stored.fileHash); + } + if (!thumbBlob || thumbBlob.size <= 0) { + const storedThumbnail = await projectDB.getThumbnail(stored.fileHash); + thumbBlob = storedThumbnail?.blob ?? null; + } + if (thumbBlob && thumbBlob.size > 0) { thumbnailUrl = URL.createObjectURL(thumbBlob); } } diff --git a/src/stores/timeline/keyframeSlice.ts b/src/stores/timeline/keyframeSlice.ts index e420bca9..f61f7bdb 100644 --- a/src/stores/timeline/keyframeSlice.ts +++ b/src/stores/timeline/keyframeSlice.ts @@ -40,6 +40,8 @@ import { composeTransforms } from '../../utils/transformComposition'; import { calculateSourceTime, getSpeedAtTime, calculateTimelineDuration } from '../../utils/speedIntegration'; import { dispatchKeyframeRecordingFeedback } from '../../utils/keyframeRecordingFeedback'; +type MaskPathVertex = MaskPathKeyframeValue['vertices'][number]; + function findClipById(clips: TimelineClip[], clipId: string): TimelineClip | undefined { for (const clip of clips) { if (clip.id === clipId) { @@ -217,6 +219,14 @@ function applyMaskPathValue(mask: ClipMask, value: MaskPathKeyframeValue): ClipM }; } +function cloneMaskVertex(vertex: MaskPathVertex): MaskPathVertex { + return { + ...vertex, + handleIn: { ...vertex.handleIn }, + handleOut: { ...vertex.handleOut }, + }; +} + function maskPathsHaveMatchingTopology( from: MaskPathKeyframeValue, to: MaskPathKeyframeValue, @@ -225,6 +235,232 @@ function maskPathsHaveMatchingTopology( return from.vertices.every((vertex, index) => vertex.id === to.vertices[index]?.id); } +function collapseMaskVertexToAnchor(vertex: MaskPathVertex, anchor: MaskPathVertex): MaskPathVertex { + return { + ...vertex, + x: anchor.x, + y: anchor.y, + handleIn: { x: 0, y: 0 }, + handleOut: { x: 0, y: 0 }, + handleMode: 'none', + }; +} + +function getWrappedTopologyIndex(index: number, count: number): number { + return ((index % count) + count) % count; +} + +function getTopologyRunIndices(startIndex: number, endIndex: number, count: number): number[] { + const indices: number[] = []; + let index = getWrappedTopologyIndex(startIndex + 1, count); + while (index !== endIndex) { + indices.push(index); + index = getWrappedTopologyIndex(index + 1, count); + } + return indices; +} + +function getPointDistance(from: { x: number; y: number }, to: { x: number; y: number }): number { + return Math.hypot(to.x - from.x, to.y - from.y); +} + +function getTopologyRatios(topologyVertices: MaskPathVertex[], segmentIndices: number[]): number[] { + if (segmentIndices.length < 3) return []; + + const distances: number[] = []; + let total = 0; + for (let index = 1; index < segmentIndices.length; index += 1) { + const prev = topologyVertices[segmentIndices[index - 1]]; + const next = topologyVertices[segmentIndices[index]]; + const distance = getPointDistance(prev, next); + distances.push(distance); + total += distance; + } + + if (total <= 1e-9) { + return segmentIndices.slice(1, -1).map((_, index) => (index + 1) / (segmentIndices.length - 1)); + } + + let cumulative = 0; + return distances.slice(0, -1).map(distance => { + cumulative += distance; + return cumulative / total; + }); +} + +function cubicPoint( + p0: { x: number; y: number }, + p1: { x: number; y: number }, + p2: { x: number; y: number }, + p3: { x: number; y: number }, + t: number, +): { x: number; y: number } { + const mt = 1 - t; + const mt2 = mt * mt; + const t2 = t * t; + return { + x: mt2 * mt * p0.x + 3 * mt2 * t * p1.x + 3 * mt * t2 * p2.x + t2 * t * p3.x, + y: mt2 * mt * p0.y + 3 * mt2 * t * p1.y + 3 * mt * t2 * p2.y + t2 * t * p3.y, + }; +} + +function cubicDerivative( + p0: { x: number; y: number }, + p1: { x: number; y: number }, + p2: { x: number; y: number }, + p3: { x: number; y: number }, + t: number, +): { x: number; y: number } { + const mt = 1 - t; + return { + x: 3 * mt * mt * (p1.x - p0.x) + 6 * mt * t * (p2.x - p1.x) + 3 * t * t * (p3.x - p2.x), + y: 3 * mt * mt * (p1.y - p0.y) + 6 * mt * t * (p2.y - p1.y) + 3 * t * t * (p3.y - p2.y), + }; +} + +function getSourceSegmentControls(fromVertex: MaskPathVertex, toVertex: MaskPathVertex) { + return { + p0: { x: fromVertex.x, y: fromVertex.y }, + p1: { x: fromVertex.x + fromVertex.handleOut.x, y: fromVertex.y + fromVertex.handleOut.y }, + p2: { x: toVertex.x + toVertex.handleIn.x, y: toVertex.y + toVertex.handleIn.y }, + p3: { x: toVertex.x, y: toVertex.y }, + }; +} + +function applySplitSourceSegment( + outputVertices: MaskPathVertex[], + topologyVertices: MaskPathVertex[], + sourceVerticesById: Map, + segmentIndices: number[], +): void { + if (segmentIndices.length < 3) return; + + const startVertex = sourceVerticesById.get(topologyVertices[segmentIndices[0]].id); + const endVertex = sourceVerticesById.get(topologyVertices[segmentIndices[segmentIndices.length - 1]].id); + if (!startVertex || !endVertex) return; + + const controls = getSourceSegmentControls(startVertex, endVertex); + const ratios = getTopologyRatios(topologyVertices, segmentIndices); + const breakpoints = [0, ...ratios, 1]; + + for (let index = 1; index < segmentIndices.length - 1; index += 1) { + const point = cubicPoint(controls.p0, controls.p1, controls.p2, controls.p3, breakpoints[index]); + const vertexIndex = segmentIndices[index]; + outputVertices[vertexIndex] = { + ...outputVertices[vertexIndex], + x: point.x, + y: point.y, + handleIn: { x: 0, y: 0 }, + handleOut: { x: 0, y: 0 }, + handleMode: 'split', + }; + } + + for (let index = 0; index < segmentIndices.length - 1; index += 1) { + const fromIndex = segmentIndices[index]; + const toIndex = segmentIndices[index + 1]; + const t0 = breakpoints[index]; + const t1 = breakpoints[index + 1]; + const dt = t1 - t0; + const fromPoint = cubicPoint(controls.p0, controls.p1, controls.p2, controls.p3, t0); + const toPoint = cubicPoint(controls.p0, controls.p1, controls.p2, controls.p3, t1); + const fromDerivative = cubicDerivative(controls.p0, controls.p1, controls.p2, controls.p3, t0); + const toDerivative = cubicDerivative(controls.p0, controls.p1, controls.p2, controls.p3, t1); + + outputVertices[fromIndex] = { + ...outputVertices[fromIndex], + handleOut: { + x: (fromDerivative.x * dt) / 3, + y: (fromDerivative.y * dt) / 3, + }, + }; + outputVertices[toIndex] = { + ...outputVertices[toIndex], + handleIn: { + x: -(toDerivative.x * dt) / 3, + y: -(toDerivative.y * dt) / 3, + }, + x: toPoint.x, + y: toPoint.y, + }; + + if (index === 0) { + outputVertices[fromIndex] = { + ...outputVertices[fromIndex], + x: fromPoint.x, + y: fromPoint.y, + }; + } + } +} + +function applyCollapsedTopologyRuns( + outputVertices: MaskPathVertex[], + topologyVertices: MaskPathVertex[], + sourceVerticesById: Map, + closed: boolean, +): void { + const existingIndices = topologyVertices + .map((vertex, index) => sourceVerticesById.has(vertex.id) ? index : -1) + .filter(index => index >= 0); + + if (existingIndices.length === 0) return; + if (existingIndices.length === 1) { + const anchor = sourceVerticesById.get(topologyVertices[existingIndices[0]].id); + if (!anchor) return; + outputVertices.forEach((vertex, index) => { + if (!sourceVerticesById.has(vertex.id)) { + outputVertices[index] = collapseMaskVertexToAnchor(vertex, anchor); + } + }); + return; + } + + for (let index = 0; index < existingIndices.length - 1; index += 1) { + const segmentIndices = [existingIndices[index], ...getTopologyRunIndices(existingIndices[index], existingIndices[index + 1], topologyVertices.length), existingIndices[index + 1]]; + applySplitSourceSegment(outputVertices, topologyVertices, sourceVerticesById, segmentIndices); + } + + if (closed) { + const firstIndex = existingIndices[0]; + const lastIndex = existingIndices[existingIndices.length - 1]; + const segmentIndices = [lastIndex, ...getTopologyRunIndices(lastIndex, firstIndex, topologyVertices.length), firstIndex]; + applySplitSourceSegment(outputVertices, topologyVertices, sourceVerticesById, segmentIndices); + } +} + +function buildMaskPathForTopology( + source: MaskPathKeyframeValue, + topology: MaskPathKeyframeValue, +): MaskPathKeyframeValue { + const sourceVerticesById = new Map(source.vertices.map(vertex => [vertex.id, vertex])); + const fallbackAnchor = source.vertices[0] ?? topology.vertices[0]; + const vertices = topology.vertices.map((topologyVertex) => { + const sourceVertex = sourceVerticesById.get(topologyVertex.id); + return sourceVertex + ? cloneMaskVertex(sourceVertex) + : collapseMaskVertexToAnchor(topologyVertex, fallbackAnchor ?? topologyVertex); + }); + + applyCollapsedTopologyRuns(vertices, topology.vertices, sourceVerticesById, source.closed); + + return { + closed: source.closed, + vertices, + }; +} + +function buildMorphableMaskPaths( + from: MaskPathKeyframeValue, + to: MaskPathKeyframeValue, +): { from: MaskPathKeyframeValue; to: MaskPathKeyframeValue } { + const topology = to.vertices.length >= from.vertices.length ? to : from; + return { + from: buildMaskPathForTopology(from, topology), + to: buildMaskPathForTopology(to, topology), + }; +} + function lerpValue(from: number, to: number, t: number): number { return from + (to - from) * t; } @@ -287,15 +523,14 @@ function getInterpolatedMaskPathValue( const nextPath = nextKey.pathValue; if (!prevPath || !nextPath) return defaultValue; - if (!maskPathsHaveMatchingTopology(prevPath, nextPath)) { - return cloneMaskPathValue(prevPath); - } - const range = nextKey.time - prevKey.time; const localTime = time - prevKey.time; const t = range > 0 ? localTime / range : 0; const easedT = Math.max(0, Math.min(1, interpolateKeyframeProgress(prevKey, nextKey, t))); - return interpolateMaskPathValue(prevPath, nextPath, easedT); + const morphPaths = maskPathsHaveMatchingTopology(prevPath, nextPath) + ? { from: prevPath, to: nextPath } + : buildMorphableMaskPaths(prevPath, nextPath); + return interpolateMaskPathValue(morphPaths.from, morphPaths.to, easedT); } export const createKeyframeSlice: SliceCreator = (set, get) => ({ diff --git a/src/stores/timeline/maskSlice.ts b/src/stores/timeline/maskSlice.ts index 283af464..b869da3a 100644 --- a/src/stores/timeline/maskSlice.ts +++ b/src/stores/timeline/maskSlice.ts @@ -3,6 +3,8 @@ import type { MaskActions, SliceCreator, ClipMask, MaskVertex, MaskEditMode } from './types'; import { getMaskVerticesHandleModeUpdates, inferMaskVertexHandleMode } from '../../utils/maskVertexHandles'; +const DEFAULT_MASK_OUTLINE_COLORS = ['#2997E5', '#ff9900', '#7ddc7a', '#d16bff', '#ff5f6d', '#f8d34f']; + export const createMaskSlice: SliceCreator = (set, get) => ({ setMaskEditMode: (mode: MaskEditMode) => { set({ maskEditMode: mode, maskDrawStart: null }); @@ -75,6 +77,7 @@ export const createMaskSlice: SliceCreator = (set, get) => ({ position: maskData?.position ?? { x: 0, y: 0 }, enabled: maskData?.enabled ?? true, visible: maskData?.visible ?? true, + outlineColor: maskData?.outlineColor ?? DEFAULT_MASK_OUTLINE_COLORS[(maskCount - 1) % DEFAULT_MASK_OUTLINE_COLORS.length], }; set({ diff --git a/src/types/index.ts b/src/types/index.ts index 39c7a280..3a838dbc 100644 --- a/src/types/index.ts +++ b/src/types/index.ts @@ -968,6 +968,7 @@ export interface ClipMask { position: { x: number; y: number }; // Offset in normalized coords (0-1) enabled: boolean; // Whether the mask affects rendering visible: boolean; // Toggle outline visibility + outlineColor?: string; // Preview overlay stroke color } export interface MaskPathKeyframeValue { diff --git a/src/version.ts b/src/version.ts index 1ede0933..335273ce 100644 --- a/src/version.ts +++ b/src/version.ts @@ -1,6 +1,6 @@ // App version // Format: MAJOR.MINOR.PATCH -export const APP_VERSION = '1.7.7'; +export const APP_VERSION = '1.7.8'; export interface ChangelogNotice { type: 'info' | 'warning' | 'success' | 'danger'; @@ -36,15 +36,15 @@ export const FEATURED_VIDEO: { // Build/Platform notice shown at top of changelog (set to null to hide) export const BUILD_NOTICE: ChangelogNotice | null = { type: 'success', - title: 'Timeline cache and slider polish', - message: 'Proxy and scrub cache ranges now live in the timeline ruler, and custom slider ranges stay in sync across color, effect, and blendshape controls.', + title: 'Board search and fast large projects', + message: 'Project search now supports filters like *.mp4, while the board keeps layouts stable and restores cached placement data for faster large-project loading.', animated: true, }; export const WIP_NOTICE: ChangelogNotice | null = { type: 'info', - title: 'Timeline and mask editing polish', - message: 'Keyframe rows, camera tracks, mask editing, and native-helper workflows received another round of production hardening.', + title: 'Large media board polish', + message: 'Board folders, loose items, search dimming, and overview rendering received another round of interaction and performance hardening.', animated: true, }; diff --git a/tests/stores/timeline/keyframeSlice.test.ts b/tests/stores/timeline/keyframeSlice.test.ts index becc84e6..e815d58c 100644 --- a/tests/stores/timeline/keyframeSlice.test.ts +++ b/tests/stores/timeline/keyframeSlice.test.ts @@ -2,7 +2,7 @@ import { describe, it, expect, beforeEach, vi } from 'vitest'; import { createTestTimelineStore } from '../../helpers/storeFactory'; import { createMockClip } from '../../helpers/mockData'; import { KEYFRAME_RECORDING_FEEDBACK_EVENT } from '../../../src/utils/keyframeRecordingFeedback'; -import { createMaskPathProperty, createMaskNumericProperty, type ClipMask } from '../../../src/types'; +import { createMaskPathProperty, createMaskNumericProperty, type ClipMask, type MaskPathKeyframeValue } from '../../../src/types'; describe('keyframeSlice', () => { let store: ReturnType; @@ -162,6 +162,103 @@ describe('keyframeSlice', () => { expect(interpolatedMask?.position.x).toBeCloseTo(0.5); }); + it('getInterpolatedMasks: tweens an added mask vertex from a collapsed neighbor point', () => { + const fromPath: MaskPathKeyframeValue = { + closed: true, + vertices: [ + { id: 'v1', x: 0, y: 0, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + { id: 'v2', x: 1, y: 0, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + { id: 'v3', x: 1, y: 1, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + ], + }; + const toPath: MaskPathKeyframeValue = { + closed: true, + vertices: [ + { id: 'v1', x: 0, y: 0, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + { id: 'v-new', x: 0.5, y: 0.5, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + { id: 'v2', x: 1, y: 0, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + { id: 'v3', x: 1, y: 1, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + ], + }; + const mask: ClipMask = { + id: 'mask-1', + name: 'Mask 1', + vertices: fromPath.vertices, + closed: true, + opacity: 1, + feather: 0, + featherQuality: 50, + inverted: false, + mode: 'add', + expanded: true, + position: { x: 0, y: 0 }, + enabled: true, + visible: true, + }; + store = createTestTimelineStore({ + clips: [createMockClip({ id: 'clip-1', trackId: 'video-1', startTime: 0, duration: 10, masks: [mask] })], + }); + + store.getState().addMaskPathKeyframe('clip-1', 'mask-1', fromPath, 0, 'linear'); + store.getState().addMaskPathKeyframe('clip-1', 'mask-1', toPath, 10, 'linear'); + + const interpolatedMask = store.getState().getInterpolatedMasks('clip-1', 5)?.[0]; + expect(interpolatedMask?.vertices.map(vertex => vertex.id)).toEqual(['v1', 'v-new', 'v2', 'v3']); + const newVertex = interpolatedMask?.vertices.find(vertex => vertex.id === 'v-new'); + expect(newVertex?.x).toBeCloseTo(0.5); + expect(newVertex?.y).toBeCloseTo(0.25); + }); + + it('getInterpolatedMasks: tweens a removed mask vertex into a collapsed neighbor point', () => { + const fromPath: MaskPathKeyframeValue = { + closed: true, + vertices: [ + { id: 'v1', x: 0, y: 0, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + { id: 'v-remove', x: 0.5, y: 0.5, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + { id: 'v2', x: 1, y: 0, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + { id: 'v3', x: 1, y: 1, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + ], + }; + const toPath: MaskPathKeyframeValue = { + closed: true, + vertices: [ + { id: 'v1', x: 0, y: 0, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + { id: 'v2', x: 1, y: 0, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + { id: 'v3', x: 1, y: 1, handleIn: { x: 0, y: 0 }, handleOut: { x: 0, y: 0 }, handleMode: 'none' }, + ], + }; + const mask: ClipMask = { + id: 'mask-1', + name: 'Mask 1', + vertices: fromPath.vertices, + closed: true, + opacity: 1, + feather: 0, + featherQuality: 50, + inverted: false, + mode: 'add', + expanded: true, + position: { x: 0, y: 0 }, + enabled: true, + visible: true, + }; + store = createTestTimelineStore({ + clips: [createMockClip({ id: 'clip-1', trackId: 'video-1', startTime: 0, duration: 10, masks: [mask] })], + }); + + store.getState().addMaskPathKeyframe('clip-1', 'mask-1', fromPath, 0, 'linear'); + store.getState().addMaskPathKeyframe('clip-1', 'mask-1', toPath, 10, 'linear'); + + const midwayMask = store.getState().getInterpolatedMasks('clip-1', 5)?.[0]; + expect(midwayMask?.vertices.map(vertex => vertex.id)).toEqual(['v1', 'v-remove', 'v2', 'v3']); + const removedVertex = midwayMask?.vertices.find(vertex => vertex.id === 'v-remove'); + expect(removedVertex?.x).toBeCloseTo(0.5); + expect(removedVertex?.y).toBeCloseTo(0.25); + + const finalMask = store.getState().getInterpolatedMasks('clip-1', 10)?.[0]; + expect(finalMask?.vertices.map(vertex => vertex.id)).toEqual(['v1', 'v2', 'v3']); + }); + it('addKeyframe: defaults easing to linear', () => { store.getState().addKeyframe('clip-1', 'opacity', 0.5, 1); const kfs = store.getState().clipKeyframes.get('clip-1')!; diff --git a/tests/stores/timeline/maskSlice.test.ts b/tests/stores/timeline/maskSlice.test.ts index 292f8ecd..99a791c3 100644 --- a/tests/stores/timeline/maskSlice.test.ts +++ b/tests/stores/timeline/maskSlice.test.ts @@ -29,6 +29,7 @@ describe('maskSlice', () => { expect(mask.mode).toBe('add'); expect(mask.enabled).toBe(true); expect(mask.visible).toBe(true); + expect(mask.outlineColor).toBe('#2997E5'); }); it('addMask: uses provided partial mask data', () => { @@ -38,6 +39,7 @@ describe('maskSlice', () => { feather: 10, inverted: true, mode: 'subtract', + outlineColor: '#abcdef', }); const mask = store.getState().clips.find(c => c.id === 'clip-1')!.masks![0]; expect(mask.name).toBe('Custom Mask'); @@ -45,6 +47,7 @@ describe('maskSlice', () => { expect(mask.feather).toBe(10); expect(mask.inverted).toBe(true); expect(mask.mode).toBe('subtract'); + expect(mask.outlineColor).toBe('#abcdef'); }); it('addMask: increments mask name based on existing count', () => { @@ -53,6 +56,7 @@ describe('maskSlice', () => { const masks = store.getState().clips.find(c => c.id === 'clip-1')!.masks!; expect(masks[0].name).toBe('Mask 1'); expect(masks[1].name).toBe('Mask 2'); + expect(masks.map(m => m.outlineColor)).toEqual(['#2997E5', '#ff9900']); }); it('addMask: sets all default properties including expanded, position, featherQuality', () => { @@ -214,6 +218,13 @@ describe('maskSlice', () => { expect(mask.featherQuality).toBe(90); }); + it('updateMask: updates outline color', () => { + const maskId = store.getState().addMask('clip-1'); + store.getState().updateMask('clip-1', maskId, { outlineColor: '#d16bff' }); + const mask = store.getState().clips.find(c => c.id === 'clip-1')!.masks![0]; + expect(mask.outlineColor).toBe('#d16bff'); + }); + it('updateMask: only updates the targeted mask, not others', () => { const maskId1 = store.getState().addMask('clip-1', { opacity: 1 }); const maskId2 = store.getState().addMask('clip-1', { opacity: 1 }); diff --git a/tests/unit/EditableDraggableNumber.test.tsx b/tests/unit/EditableDraggableNumber.test.tsx index 5ff5fdbb..da1dfa94 100644 --- a/tests/unit/EditableDraggableNumber.test.tsx +++ b/tests/unit/EditableDraggableNumber.test.tsx @@ -2,12 +2,19 @@ import { cleanup, fireEvent, render } from '@testing-library/react'; import { afterEach, describe, expect, it, vi } from 'vitest'; import { EditableDraggableNumber } from '../../src/components/common/EditableDraggableNumber'; -function dispatchMouseMove(target: EventTarget, init: MouseEventInit) { +function dispatchMouseMove(target: EventTarget, init: MouseEventInit & { movementX?: number }) { + const { movementX, ...mouseInit } = init; const event = new MouseEvent('mousemove', { bubbles: true, cancelable: true, - ...init, + ...mouseInit, }); + if (movementX !== undefined) { + Object.defineProperty(event, 'movementX', { + configurable: true, + value: movementX, + }); + } fireEvent(target, event); } @@ -22,45 +29,83 @@ describe('EditableDraggableNumber drag behavior', () => { cleanup(); }); - it('drags from the current value without requesting pointer lock', () => { + it('drags from the current value with pointer lock and no initial jump', () => { const onChange = vi.fn(); const requestPointerLock = vi.fn(); + const exitPointerLock = vi.fn(); + let lockedElement: Element | null = null; + let pointerLockTarget: Element | null = null; const originalRequestPointerLockDescriptor = Object.getOwnPropertyDescriptor(HTMLElement.prototype, 'requestPointerLock'); + const originalExitPointerLockDescriptor = Object.getOwnPropertyDescriptor(document, 'exitPointerLock'); + const originalPointerLockElementDescriptor = Object.getOwnPropertyDescriptor(document, 'pointerLockElement'); Object.defineProperty(HTMLElement.prototype, 'requestPointerLock', { configurable: true, - value: requestPointerLock, + value: () => { + requestPointerLock(); + lockedElement = pointerLockTarget; + document.dispatchEvent(new Event('pointerlockchange')); + return Promise.resolve(); + }, + }); + Object.defineProperty(document, 'exitPointerLock', { + configurable: true, + value: () => { + exitPointerLock(); + lockedElement = null; + document.dispatchEvent(new Event('pointerlockchange')); + }, + }); + Object.defineProperty(document, 'pointerLockElement', { + configurable: true, + get: () => lockedElement, }); - const { container } = render( - , - ); - const valueElement = container.querySelector('.draggable-number') as HTMLElement; + try { + const { container } = render( + , + ); + const valueElement = container.querySelector('.draggable-number') as HTMLElement; + pointerLockTarget = valueElement; - fireEvent.mouseDown(valueElement, { button: 0, clientX: 100, buttons: 1 }); - dispatchMouseMove(window, { clientX: 103, buttons: 1 }); + fireEvent.mouseDown(valueElement, { button: 0, clientX: 100, buttons: 1 }); + expect(requestPointerLock).toHaveBeenCalledTimes(1); + expect(onChange).not.toHaveBeenCalled(); - expect(lastChangedValue(onChange)).toBeGreaterThan(102); - expect(lastChangedValue(onChange)).toBeLessThan(103); + dispatchMouseMove(window, { clientX: 0, movementX: 3, buttons: 1 }); - dispatchMouseMove(window, { clientX: 104, buttons: 1 }); + expect(lastChangedValue(onChange)).toBeGreaterThan(102); + expect(lastChangedValue(onChange)).toBeLessThan(103); - expect(lastChangedValue(onChange)).toBeGreaterThan(103); - expect(lastChangedValue(onChange)).toBeLessThan(104); - expect(requestPointerLock).not.toHaveBeenCalled(); + dispatchMouseMove(window, { clientX: 0, movementX: 1, buttons: 1 }); - fireEvent.mouseUp(window, { button: 0, buttons: 0 }); + expect(lastChangedValue(onChange)).toBeGreaterThan(103); + expect(lastChangedValue(onChange)).toBeLessThan(104); - if (originalRequestPointerLockDescriptor) { - Object.defineProperty(HTMLElement.prototype, 'requestPointerLock', originalRequestPointerLockDescriptor); - } else { - delete (HTMLElement.prototype as HTMLElement & { requestPointerLock?: () => void }).requestPointerLock; + fireEvent.mouseUp(window, { button: 0, buttons: 0 }); + expect(exitPointerLock).toHaveBeenCalledTimes(1); + } finally { + if (originalRequestPointerLockDescriptor) { + Object.defineProperty(HTMLElement.prototype, 'requestPointerLock', originalRequestPointerLockDescriptor); + } else { + delete (HTMLElement.prototype as HTMLElement & { requestPointerLock?: () => void }).requestPointerLock; + } + if (originalExitPointerLockDescriptor) { + Object.defineProperty(document, 'exitPointerLock', originalExitPointerLockDescriptor); + } else { + delete (document as Document & { exitPointerLock?: () => void }).exitPointerLock; + } + if (originalPointerLockElementDescriptor) { + Object.defineProperty(document, 'pointerLockElement', originalPointerLockElementDescriptor); + } else { + delete (document as Document & { pointerLockElement?: Element | null }).pointerLockElement; + } } }); diff --git a/tests/unit/TimelineHeaderCameraLook.test.tsx b/tests/unit/TimelineHeaderCameraLook.test.tsx index fd3e1623..c38bcbd2 100644 --- a/tests/unit/TimelineHeaderCameraLook.test.tsx +++ b/tests/unit/TimelineHeaderCameraLook.test.tsx @@ -4,6 +4,65 @@ import { TimelineHeader } from '../../src/components/timeline/TimelineHeader'; import type { ClipTransform, TimelineClip, TimelineTrack } from '../../src/types'; describe('TimelineHeader camera look controls', () => { + it('edits the track name instead of expanding when the name is clicked', () => { + const track = { + id: 'video-1', + name: 'Video 1', + type: 'video', + height: 48, + visible: true, + muted: false, + solo: false, + } as TimelineTrack; + const onToggleExpand = vi.fn(); + + const { container } = render( + []} + getInterpolatedTransform={() => ({ + opacity: 1, + blendMode: 'normal', + position: { x: 0, y: 0, z: 0 }, + scale: { x: 1, y: 1 }, + rotation: { x: 0, y: 0, z: 0 }, + })} + getInterpolatedEffects={() => []} + addKeyframe={vi.fn()} + setPlayheadPosition={vi.fn()} + setPropertyValue={vi.fn()} + expandedCurveProperties={new Map()} + onToggleCurveExpanded={vi.fn()} + onSetTrackParent={vi.fn()} + onTrackPickWhipDragStart={vi.fn()} + onTrackPickWhipDragEnd={vi.fn()} + />, + ); + + expect(container.textContent).toContain('(id:1)'); + + fireEvent.click(container.querySelector('.track-name') as HTMLElement); + + expect(onToggleExpand).not.toHaveBeenCalled(); + expect(container.querySelector('.track-name-input')).not.toBeNull(); + }); + it('scrubs camera yaw as a look keyframe without moving the camera position', () => { const transform: ClipTransform = { opacity: 1, diff --git a/tests/unit/nativeSceneRenderer.test.ts b/tests/unit/nativeSceneRenderer.test.ts index a7a38eb9..a32a2bbc 100644 --- a/tests/unit/nativeSceneRenderer.test.ts +++ b/tests/unit/nativeSceneRenderer.test.ts @@ -430,6 +430,38 @@ describe('NativeSceneRenderer shared depth contract', () => { expect(depthMaskOptions.depthAlphaCutoff).toBeGreaterThan(0.05); }); + it('uploads VideoFrame sources for exported 3D video planes', async () => { + const renderer = await createInitializedRenderer(); + + const { device } = createFakeDevice(); + const videoFrame = { + displayWidth: 640, + displayHeight: 360, + codedWidth: 640, + codedHeight: 360, + } as VideoFrame; + const result = renderer.renderScene( + device, + [{ + ...makePlaneLayer('video-frame-plane', 1), + sourceWidth: 640, + sourceHeight: 360, + videoElement: undefined, + videoFrame, + }], + makeCamera(), + [], + false, + ); + + expect(result).toEqual((renderer as NativeSceneRendererTestAccess).sceneView); + expect(device.queue.copyExternalImageToTexture).toHaveBeenCalledWith( + { source: videoFrame }, + expect.anything(), + { width: 640, height: 360 }, + ); + }); + it('keeps source alpha for non-opaque video planes', async () => { const renderer = await createInitializedRenderer(); diff --git a/tests/unit/sceneLayerCollector.test.ts b/tests/unit/sceneLayerCollector.test.ts index 200b0f17..43d3a026 100644 --- a/tests/unit/sceneLayerCollector.test.ts +++ b/tests/unit/sceneLayerCollector.test.ts @@ -203,4 +203,53 @@ describe('SceneLayerCollector', () => { expect(nativeSplat.worldTransform?.rotationDegrees).toEqual({ x: 0, y: 0, z: 0 }); expect(nativeSplat.worldTransform?.scale).toEqual({ x: 1, y: 1, z: 1 }); }); + + it('forwards VideoFrame sources to native 3D video planes for export', () => { + const videoElement = document.createElement('video'); + const videoFrame = { + displayWidth: 1920, + displayHeight: 1080, + } as VideoFrame; + const layerData: LayerRenderData[] = [ + { + layer: { + id: 'video-plane', + name: '3D Video', + sourceClipId: 'clip-video', + visible: true, + opacity: 1, + blendMode: 'normal', + source: { + type: 'video', + videoElement, + videoFrame, + }, + effects: [], + position: { x: 0, y: 0, z: 0 }, + scale: { x: 1, y: 1 }, + rotation: { x: 0, y: 0, z: 0 }, + is3D: true, + }, + isVideo: true, + externalTexture: null, + textureView: null, + sourceWidth: 1920, + sourceHeight: 1080, + }, + ]; + + const collected = collectScene3DLayers(layerData, { + width: 1920, + height: 1080, + }); + + expect(collected).toHaveLength(1); + expect(collected[0]).toMatchObject({ + kind: 'plane', + alphaMode: 'opaque', + castsDepth: true, + videoElement, + videoFrame, + }); + }); });