【ICLR 2026】Official implementation of TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching
This repository contains the official implementation of TokenSeek.
I am Run, the project lead.
The complete implementation, including core code and training scripts, will be released prior to the ICLR conference (April 23). Due to ongoing PhD commitments, reorganizing the repository may take some time.
Below are the core codes of TokenSeek:
.
├── tokenseek/ # Core source code
│ ├── backgradient.py # [TokenSeek] Backward gradient extraction
│ ├── forward_attention.py # [TokenSeek] Forward attention extraction
│ ├── attention_score_gradient.py # [TokenSeek] Process gradient scores
│ ├── attention_score_pt.py # [TokenSeek] Process attention scores
│ ├── combine_score.py # [TokenSeek] Combine gradient + attention scores
│ ├── llama_tokentune.py # TokenTune model implementation for Llama
│ ├── prompter.py # Prompt template handling
│ └── alpaca_template.json # Alpaca-style prompt template
└── README.md
Thank you for your understanding, and we sincerely appreciate the excellent work by the TokenTune team.