Northwestern Polytechnical University Unveils ROSE for Lifelong Multimodal Knowledge Editing
A team at Northwestern Polytechnical University proposes ROSE, a lifelong multimodal knowledge editing framework that constrains parameter update directions and activates relevant edits on demand at inference, mitigating catastrophic forgetting and locality erosion during continuous knowledge updates. The paper was accepted as a NeurIPS 2026 Spotlight, offering a new paradigm for keeping multimodal LLMs current without degrading general capabilities.
Related Entities
Full Report

Achieving a better balance between learning new knowledge and preserving existing capabilities

Factual knowledge in the real world is not static: corporate CEOs change, product information is updated with releases and iterations, and competition results are constantly produced. Multimodal large models need to absorb this new information in a timely manner to keep their knowledge current. However, in the process of continuously introducing new facts, how to avoid mutual interference between new and old knowledge and preserve as much as possible the general capabilities formed during the model's pre-training stage remains a key problem to be solved.
To address this challenge, under the joint guidance of Professor Song Lingyun and Professor Shang Xuequn at Northwestern Polytechnical University, the research team proposed a multimodal lifelong knowledge editing framework, ROSE. This framework reduces interference between new and old knowledge by constraining the direction of parameter updates during the knowledge editing stage. At the same time, during inference, it activates the knowledge of relevant edits on demand, thereby reducing knowledge forgetting and degradation of original capabilities caused by continuous knowledge updates. The paper was accepted as a NeurIPS 2026 Spotlight.

- Paper title: ROSE: Risk-Aware Orthogonal Subspace Navigation for Lifelong Knowledge Editing in Multimodal Large Language Models
- Authors: Song Lingyun, Chen Ziyao, Pan Kang, Han Xiaolin, Zhang Yifei, Wang Xiaoqi, Pan Yudai, Shang Xuequn
- Paper link: https://arxiv.org/abs/2611.09770
- Code link: https://github.com/sunlitsong/ROSE
What challenges does continuous knowledge updating face?
Knowledge editing aims to correct errors or update specific facts without retraining the entire model. In real application scenarios, new facts continuously emerge with real-world events, and existing facts may also change. Therefore, multimodal lifelong knowledge editing requires the model to learn new knowledge while preserving previous editing results and maintaining its original performance on unrelated inputs. This paper summarizes this challenge into two aspects:
-
Catastrophic Forgetting: Subsequent knowledge updates interfere with previous editing results, making it difficult for the model to retain factual knowledge that has already been edited (for example, after previously correcting the CEO of Apple, the model answers incorrectly again after subsequent updates).
-
Locality Erosion: Knowledge updates cause unexpected changes in the model's outputs on unrelated knowledge, and may even damage the model's original pre-training capabilities (for example, after updating information about Apple's CEO, the model also produces unnecessary changes in its answers about the CEOs of other companies).
The association between visual and textual representations further increases the difficulty of controlling the scope of editing. A single fact update may affect both image understanding and language responses, so it is necessary to take into account both the preservation of historical edits and the stability of unrelated knowledge and original capabilities.
Why continuous edits easily interfere with each other
Knowledge editing aims to correct specific facts in a model at a low cost, avoiding retraining the entire model for each update. For example, the model may receive multiple knowledge updates in succession: first updating Apple's CEO from Tim Cook to John Ternus, then updating Ternus's work experience and the business he previously led, and then supplementing the changes in Cook's position after stepping down as CEO. After completing these edits, the model must not only accurately grasp each piece of new information, but also ensure that previously written knowledge is not incorrectly modified due to subsequent updates of other facts.
The difficulty, however, is that the knowledge spaces on which different facts in large models depend are not independent of one another. A new edit may change the feature directions on which previous edits depend, and may also affect outputs on unrelated inputs. Therefore, even if the model successfully writes a new fact, subsequent edits may still cause historical knowledge to be forgotten or change its answers to unrelated questions.
The figure below uses optimization paths to intuitively explain this problem: if the update is too conservative, it may fail to successfully edit the new fact. If the update is too aggressive, it is likely to interfere with other knowledge. Therefore, ROSE aims to avoid high-risk parameter subspaces that may damage pre-training capabilities during the knowledge editing process, while reducing conflicts with historical editing directions, thereby achieving a balance between writing new knowledge and preserving existing knowledge.

Figure 1 Schematic of the optimization paths of traditional knowledge editing methods and ROSE.
How ROSE writes and uses new knowledge
During the editing stage, ROSE freezes the backbone of the original model and introduces an auxiliary branch into the down-projection matrix of the FFN in selected Transformer layers. Continuously arriving new knowledge is written into this branch. During the inference stage, the model first determines whether the input is knowledge that has been edited, and then decides how to combine the representations of the two branches.

Figure 2 ROSE framework diagram.
Risk-aware orthogonal editing mechanism
To reduce the interference of subsequent edits with existing modifications, ROSE separately extracts the visual and textual features from historical edits and constructs corresponding knowledge subspaces to record the feature directions involved in historical edits. In a new round of editing, ROSE projects the gradient onto the orthogonal complement space of the historical knowledge subspace, removing gradient components that overlap with historical editing directions, so that new parameter updates avoid as much as possible the feature directions involved in existing edits, thereby reducing mutual interference between new and old knowledge.
However, protecting only the feature subspaces involved in historical edits is not enough to guarantee the stability of the model's original knowledge and general capabilities. To this end, ROSE introduces a dynamic risk mask mechanism, using the fixed original weights as anchors to select parameter spaces with large changes for subsequent updates and suppress updates at other parameter positions. This mask is dynamically recomputed during gradient updates to further protect the general knowledge and capabilities formed during the large model's pre-training stage.
Multi-granularity knowledge integration strategy
The edited knowledge in the auxiliary branch is not applicable to all inputs. For example, updating information about Apple's CEO should not affect the model's answers to other unrelated enterprises. To this end, ROSE extracts visual and textual semantic prototypes for each edit separately, recording the knowledge features involved in that modification. When the model receives a new question, it determines whether the current input is relevant to existing edits by computing the similarity between the input features and historical semantic prototypes. Inputs that fail the semantic gate directly reuse the output of the original backbone.
For inputs that pass the gate, ROSE further compares the difference in feature representations between the original backbone and the auxiliary branch, and accordingly constructs a dynamic fusion mask, introducing the edited feature representation only at positions with more significant differences while retaining the original representation at the remaining positions. Through this coarse-to-fine knowledge integration approach, selective invocation of edited knowledge is achieved, reducing interference with irrelevant knowledge and original capabilities.
Performance of ROSE under Long Edit Sequences
The paper evaluates ROSE on two benchmarks, VLKEB and MMEdit, with tasks including visual question answering and image captioning. Reliability focuses on whether the model correctly completes the update of target knowledge; generalization examines whether the model can still answer accurately when the question phrasing or the image changes; and locality focuses on whether knowledge editing affects other unrelated inputs, ensuring that the model's outputs on these inputs remain as consistent as possible with those before editing.
The table below shows the results of LLaVA-v1.5-7B on VLKEB after 1000 consecutive edits.

From the results, it is easy to see:
- ROSE effectively mitigates catastrophic forgetting.
As shown in the table, as the number of edits increases, the reliability of most baseline methods drops significantly. After 1000 edits on VLKEB, the reliability of FT-L falls to 68.14%, while ROSE still reaches 95.24%. From 1 edit to 1000 edits, ROSE drops by only 4.76 percentage points, whereas FT-L drops by 26.15 percentage points. This benefits from the risk-guided orthogonal editing mechanism: the orthogonal constraint reduces directional conflicts between new and old edits, while the dynamic risk mask limits the scope of parameter updates, reducing interference with pre-trained knowledge and capabilities.
- Better knowledge generalization ability.
On VLKEB, ROSE performs well in both textual and multimodal generalization, and outperforms the strongest baseline LiveEdit after 1000 edits. Its textual generalization reaches 92.74% (LiveEdit: 83.97%), and multimodal generalization reaches 84.53% (LiveEdit: 82.75%). This benefits from the perturbation-guided knowledge subspace fusion mechanism, which compares the feature differences between the two branches and selectively fuses the parts with significant changes while retaining the remaining original representations.
- Excellent locality.
Preserving the integrity of irrelevant knowledge is a difficulty faced by many editing methods. The multimodal locality of SERAC on VLKEB drops to 15.04%. In contrast, ROSE achieves 100% textual locality and 88.39% multimodal locality. This benefits from the coarse-to-fine knowledge integration strategy. The semantic alignment gate guides irrelevant queries to bypass the auxiliary branch, while perturbation-guided fusion performs fine-grained modifications for relevant inputs, thereby reducing unintended interference with other factual knowledge.
In terms of running cost, under the test configuration of a single RTX 4090 and LLaVA-v1.5-7B, the average time per edit in the 1000-edit setting is about 1.31 seconds, and inference per query is about 1.13 seconds, demonstrating certain practical value.

Conclusion
ROSE reduces the interference of continuous knowledge updates with existing knowledge through a risk-aware orthogonal editing mechanism, and leverages a multi-granularity knowledge integration strategy to invoke edited knowledge on demand, lowering the risk of locality degradation, thereby achieving a better balance between learning new knowledge and preserving original capabilities. Experimental results further validate the effectiveness of this framework in multimodal lifelong knowledge editing tasks.
About the First Author
Song Lingyun is an associate professor and doctoral supervisor at the School of Computer Science, Northwestern Polytechnical University. His research interests include reinforcement fine-tuning of multimodal large models, agent technology, cross-media cognition and reasoning, and computer vision. He has published more than 60 papers in SCI journals and high-level conferences in the computer field, and has applied for more than 20 national invention patents and multiple software copyrights.

© THE END
For reprinting, please contact this official account for authorization.
Submission or reporting: [email protected]