Today News Research Fields Stacks Market Analytics

⌘K

Tool

ByteTrail · AI and developer-tools intelligence

News Research Stacks API

Today News Research Fields Stacks Market Analytics

Fields · this week

Multimodal497 Agents486 Frontier LLMs423 Inference & serving245 Learning verifiable rewards164 Image segmentation161 Graph learning157 3D scene140 Evals108 Voice AI88 Flow matching75 Forecasting time75 Classifier & causal discovery74 Training & fine-tuning64 Teacher policy distillation55 All 32 fields

Tokenizers · ByteTrail

Model training & ML

Tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

This week

Coming with more sources: stars history

Downloads / week46.6M+14.2%

Stars11.2kgrowth from late November

Releases · 90 days4last Oct 19

Rising tools#19

Weekly downloads · last year

Coming with more sources: stars history

PyPItokenizers46.6M/wk

GitHub stars

Coming with more sources: stars history

Links

Coming with more sources: stars history

GitHub: huggingface/tokenizers
Language: Rust
PyPI: tokenizers