Model Merging

Diagram

Is this really working? Neural Network Can Add?

• Paper: Editing Models with Task Arithmetic
• paper: Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages

learning via addition

• 通常調整相加比例會有更好的結果
• paper: Model Stock: All we need is just a few fine-tuned models

Example

• paper: DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging

• paper: Transferring Textual Preferences to Vision-Language Understanding through Model Merging

Forgetting via negation (Machine Unlearning)

• example: 台灣化LLM的實踐經驗分享 3 |Chat Vector—神奇的 LLM 積木

Task analogies

• Task A : Task B = Task C : Task D
• 在沒Task D 的情況下，可以用 Task A, B, C 來推論 Task D ，model 學會task D
• Paper