ELECTRA

Google's more sample-efficient pretraining approach using a replaced-token-detection objective

Free Language
Visit Product Page →
512 context tokens
335M parameters
Apache 2.0 license
Mar 2020 released

// about ELECTRA

Google's more sample-efficient pretraining approach using a replaced-token-detection objective