Skip to content
Tang-Lab-RutgersPublic
forked from runtsang/TokenSeek

About

Code of ICLR 2026 "TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching"

Resources

Stars

1 star

Watchers

0 watching

Forks

 
 

Latest commit

 

History

4 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 

Repository files navigation

【ICLR 2026】Official implementation of TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching

This repository contains the official implementation of TokenSeek.
I am Run, the project lead.

The complete implementation, including core code and training scripts, will be released prior to the ICLR conference (April 23). Due to ongoing PhD commitments, reorganizing the repository may take some time.

Below are the core codes of TokenSeek:

Directory Structure

.
├── tokenseek/                       # Core source code
│   ├── backgradient.py              # [TokenSeek] Backward gradient extraction
│   ├── forward_attention.py         # [TokenSeek] Forward attention extraction
│   ├── attention_score_gradient.py  # [TokenSeek] Process gradient scores
│   ├── attention_score_pt.py        # [TokenSeek] Process attention scores
│   ├── combine_score.py             # [TokenSeek] Combine gradient + attention scores
│   ├── llama_tokentune.py           # TokenTune model implementation for Llama
│   ├── prompter.py                  # Prompt template handling
│   └── alpaca_template.json         # Alpaca-style prompt template
└── README.md

Thank you for your understanding, and we sincerely appreciate the excellent work by the TokenTune team.

About

Code of ICLR 2026 "TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching"

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages