DistilBERT
Hugging Face's distilled, 40%-smaller version of BERT that retains most of its performance
Free
Language
512
context tokens
66M
parameters
Apache 2.0
license
Oct 2019
released
// about DistilBERT
Hugging Face's distilled, 40%-smaller version of BERT that retains most of its performance