absorb.md

Karen Hao

Chronological feed of everything captured from Karen Hao.

paper / _karenhao / Jun 11

ISAP-3D: Enhancing 3D Generation with Identity-Slot Alignment for Improved Part Awareness

ISAP-3D is a novel framework for part-aware 3D generation that tackles the identity-slot permutation freedom issue, a key source of structural ambiguity in existing methods. By enforcing identity-aligned one-to-one slot modeling through identity-conditioned layout prediction and layout-conditioned geometry synthesis, ISAP-3D enhances structural stability, controllability, and robustness. The framework leverages structured local-global conditioning and is supported by a new part-level dataset designed for consistent identity-slot alignment.

3d-generationcomputer-visiongenerative-modelspart-aware-generationidentity-slot-alignment
“ISAP-3D addresses the issue of identity-slot permutation freedom in part-aware 3D generation by enforcing identity-aligned one-to-one slot modeling”
paper / _karenhao / Jun 9

MAVIS: Multi-Agent Framework Revolutionizes Video Retrieval with Cooperative Reasoning and Structured Understanding

MAVIS introduces a multi-agent framework for video retrieval that shifts from brute-force search to cooperative reasoning. It addresses semantic and granularity mismatches by creating a Structured Semantic Library and utilizes a planner-agent system to decompose queries. A 'Logic-aware Debate' mechanism refines candidate selection, leading to efficient and competitive retrieval performance without task-specific tuning.

multi-agent-systemsvideo-retrievalstructured-video-understandingretrieval-augmented-modelsdebate-mechanismcomputer-visionsemantic-indexing
“MAVIS improves video retrieval by using a multi-agent system for cooperative reasoning instead of brute-force search.”
paper / _karenhao / Jun 9

Layer-Ratio Monte Carlo Estimates of Dedekind Numbers Reveals Non-Unimodal Whitney Number Structure

Chen et al. introduce a layer-ratio Monte Carlo method to estimate Dedekind numbers M(n)—the count of monotone Boolean functions—by reformulating the problem as reconstructing Whitney numbers of the ideal lattice. Their approach uses reversible fixed-layer Markov chains to estimate local averages of addable and removable elements, then backtests at M(8) and M(9) to calibrate variability. The estimate for M(10) is (8.9360 ± 0.0010) × 10^78. Notably, the high-precision n=9 estimate reveals a shallow two-shoulder feature around the central rank, contradicting the previously assumed unimodality of the Whitney-number sequence, and this pattern becomes more pronounced at n=11 and n=13.

combinatoricsmonotone-boolean-functionsdedekind-numbersmonte-carlo-methodsmathematical-analysis
“M(10) is estimated as (8.9360 ± 0.0010) × 10^78”
paper / _karenhao / Jun 9

MemoryVLA++: Enhancing Robotic Manipulation with Temporal Modeling via Memory and Imagination

MemoryVLA++ introduces a novel temporal modeling framework for Vision-Language-Action (VLA) models in robotic manipulation. It enhances performance on long-horizon tasks by integrating working memory, episodic memory retrieval from a Perceptual-Cognitive Memory Bank, and future state imagination via a world model. This approach conditions a diffusion action expert on temporally aware tokens, leading to significant improvements in general, memory-dependent, and imagination-dependent tasks on both simulated and real-world robotic platforms.

temporal-modelingvision-language-action-modelsrobotic-manipulationworld-modelsmemory-mechanismsdiffusion-policies
“MemoryVLA++ improves robotic manipulation by incorporating temporal modeling through memory and imagination, addressing limitations of models that rely solely on current observations for long-horizon tasks.”
paper / _karenhao / May 31

Fine-Tuning-Free Talking Face Generation via Pretrained Stable Diffusion and IP-Adapter

This paper proposes a paradigm for talking face generation that eliminates task-specific fine-tuning by leveraging pretrained Stable Diffusion and IP-Adapter weights directly, bypassing the need for large-scale audiovisual training datasets. Three trainable-parameter-free modules — the Structurist, Structure Controller, and Noise Sensor — address identity drift, lip synchronization errors, and temporal instability respectively. The approach outperforms current SOTA methods on both lip-sync accuracy (PCLD gain ≥0.16) and visual fidelity (FID improvement ≥0.7), demonstrating that frozen pretrained diffusion backbones can be effectively repurposed for complex video generation tasks without additional fine-tuning overhead.

talking-face-generationdiffusion-modelsip-adapterstable-diffusionlip-synccomputer-visionfine-tuning-free
“Talking face generation can be performed without any task-specific fine-tuning by directly using pretrained Stable Diffusion and IP-Adapter weights.”
paper / _karenhao / May 31

Sparse Attention Quality Depends on Mask Geometry, Not Sparsity Ratio — Veda Exploits This for 10x Video Diffusion Speedup

Veda is a distilled sparse attention framework for video Diffusion Transformers that reframes tile selection as an explicit reconstruction problem relative to full attention. The key empirical finding is that generation quality is governed by how well the sparse mask aligns with the tile-wise geometry of full attention, not by the sparsity ratio itself — a finding that invalidates the design assumptions of prior sparse attention methods. By combining statistics-aware tile scoring, head-aware tiling, and a hardware-efficient tile-skipping kernel, Veda converts high theoretical sparsity into real wall-clock gains that scale favorably with sequence length. On a 12B-parameter model generating 720P 10-second video, it achieves 5.1× end-to-end and 10.5× self-attention speedup with no noticeable quality degradation.

video-generationdiffusion-modelssparse-attentiontransformer-efficiencycomputer-visionmodel-distillationinference-optimization
“Generation quality in sparse attention is determined by sparse mask alignment with tile-wise full attention geometry, not by the sparsity ratio itself.”