多模式生成模型可产生流畅的输出，但当生成必须尊重结构化, 特定领域, 或安全关键知识时，它仍然不可靠。现有方法通过诸如提示增强,指导,潜在编辑,或微调,等机制整合知识，但它们通常按技术而不是按它们修改的生成过程的组成部分进行分类。我们认为迭代生成模型中的知识注入从根本上来说是一个干预层问题。由于生成过程作为内部状态,的轨迹展开，知识可以作用于该过程的四个结构上不同的组成部分:、输入/输出边界,、转换函数,、中间状态,和模型参数。这映射到四个干预层:表面,轨迹,潜在,和参数输注。我们实例化扩散模型, 中的框架，将代表性方法映射到所有四个层, 并推导出多层组合的设计原则。在使用具有两个扩散主干,的多模态知识图的受控安全对齐实验中，我们累计实现了四层中的三层,表面(输入侧和输出侧)以及轨迹-潜在(中期)。我们凭经验证明，每个附加层都解决了先前层无法达到的故障类别,，与普通生成相比，知识违规输出减少了 70.97%，并凭经验证实了框架的 互补性预测。
Multimodal generative models produce fluent outputs but remain unreliable when generation must respect structured, domain-specific, or safety-critical knowledge. Existing methods incorporate knowledge through mechanisms such as prompt augmentation, guidance, latent editing, or fine-tuning, yet they are typically categorized by technique rather than by the component of the generative process they modify. We argue that knowledge infusion in iterative generative models is fundamentally anintervention-layer problem. Since thegenerative process unfolds as a trajectory of internal states, knowledge can act on four structurally distinct components of this process: the input/output boundary, the transition function, the intermediate state, and the model parameters. This maps to four intervention layers: surface, trajectory, latent, and parametric infusion. We instantiate the framework in diffusion models, map representative methods to all four layers, and derive design principles for multi-layer composition. In a controlled safety-alignment experiment using a multimodal knowledge graph with two diffusion backbones, we implement three of the four layers cumulatively, surface (input-side and output-side) and trajectory--latent (mid-generation). We show empirically that each additional layer addresses failure classes that prior layers cannot reach, reducing knowledge-violating outputs by 70.97% compared to vanilla generation and empirically confirming the framework的 complementarity prediction.
科目:人工智能(cs.AI)
Subjects: Artificial Intelligence (cs.AI)