lr scheduler #150
Description
Hi, I am trying to pretrain a model on DNABERT-2, but the model's loss is not converging. I wanted to as,k did you use this scheduler for learning rate?
composer.optim.LinearWithWarmupScheduler
{{ message }}
Hi, I am trying to pretrain a model on DNABERT-2, but the model's loss is not converging. I wanted to as,k did you use this scheduler for learning rate?
composer.optim.LinearWithWarmupScheduler