On the Knowledge Transfer via Pretraining, Distillation and Federated Learning