DistilBERT

Hugging Face's distilled, 40%-smaller version of BERT that retains most of its performance

Free Language
Visit Product Page →
512 context tokens
66M parameters
Apache 2.0 license
Oct 2019 released

// about DistilBERT

Hugging Face's distilled, 40%-smaller version of BERT that retains most of its performance