Vibe Coding DiscoverContinualLM
View on GitHubAn Extensible Continual Learning Framework Focused on Language Models (LMs)
PyTorch framework for continual learning of language models: implements DAS, CPT, DGA, EWC, HAT, DER++ and baselines for sequential domain-adaptive pretraining with end-task fine-tuning, forgetting-rate tools, and Hugging Face checkpoints.
Use Cases
continual pretraining of language models over domain corporadomain-adaptive pretraining (post-training)mitigating catastrophic forgetting in LMsknowledge transfer across sequential domainsend-task fine-tuning after continual learningaspect sentiment classificationcitation intent classificationrelation classificationchemical-protein interaction predictionmeasuring forgetting ratereproducing continual learning baselines (DAS, CPT, DGA, EWC, HAT, DER++)self-contained soft-masking notebook example without GPUs
Built With
- Language
- Python
- Frameworks
- PyTorch · Hugging Face Transformers · adapter-transformers · Hugging Face Datasets · evaluate · accelerate · conda
Tags
continual-learning · catastrophic-forgetting · domain-adaptive-pretraining · language-models · knowledge-transfer · transfer-learning · transformer · soft-masking · adapters · prompts · knowledge-distillation · fine-tuning · pytorch · research · nlp