When Do Curricula Work in Federated Learning?

Saeed Vahidian; Sreevatsank Kadaveru; Woonjoon Baek; Weijia Wang; Vyacheslav Kungurtsev; Chen Chen; Mubarak Shah; Bill Lin

doi:10.1109/ICCV51070.2023.00469

When Do Curricula Work in Federated Learning?

2022 ICCV 2023 with Saeed Vahidian, Mubarak Shah, Bill Lin, and others

Illustration of training data split across two clients and an easy-first curriculum

A study of curriculum learning in a federated context, showing that curricula can significantly improve the convergence time of training. Ordered learning substantially offsets the effect of non-IID data, and clients with more divergent distributions benefit more.

Curriculum learning orders training data from easy to hard, while in federated learning each client trains on its own skewed slice of the data. A convergence analysis models curriculum training as local SGD with locally biased gradients, and a curriculum over clients, introduced from easy to hard, further improves performance.

Paper arXiv

Publication

When Do Curricula Work in Federated Learning?

Saeed Vahidian, Sreevatsank Kadaveru, Woonjoon Baek, Weijia Wang, Vyacheslav Kungurtsev, Chen Chen, Mubarak Shah, Bill Lin

ICCV 2023 Paris, France

Paper PDF arXiv

@inproceedings{vahidian2023curricula,
  author    = {Vahidian, Saeed and Kadaveru, Sreevatsank and Baek, Woonjoon and Wang, Weijia and Kungurtsev, Vyacheslav and Chen, Chen and Shah, Mubarak and Lin, Bill},
  title     = {When Do Curricula Work in Federated Learning?},
  booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  pages     = {5084--5094},
  year      = {2023}
}

All research