I would say it depends on your task, if pre-trained ones performs better, you use pre-trained ones, otherwise you let the model learns the embedding itself.