Arto Maranjyan

I’m a PhD student at KAUST, working under the guidance of Prof. Peter Richtárik. My research focuses on optimization for machine learning and federated learning, where I contribute to the development of distributed and randomized optimization algorithms.

Before starting my PhD, I earned my MSc and BSc from Yerevan State University. During my bachelor’s studies, I coauthored several papers in Harmonic Analysis under the supervision of Prof. Martin Grigoryan.

Outside of academics, I enjoy dancing bachata, playing board games, ultimate frisbee and foosball.

News

Oct 17, 2024	I will be giving a talk at the International Conference on Algebra, Logic, and their Applications on October 18, on our paper, MindFlayer: Efficient Asynchronous Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times.
Oct 17, 2024	I am reviewing for SIAM Journal on Mathematics of Data Science (SIMODS).
Oct 10, 2024	Three papers are accepted to Optimization for Machine Learning Workshop (NeurIPS 2024): MindFlayer: Efficient Asynchronous Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times Differentially Private Random Block Coordinate Descent LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression
Oct 08, 2024	New paper out; we have MindFlayer, Vecna, and other Stranger Things: MindFlayer: Efficient Asynchronous Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times
Sep 11, 2024	I am reviewing for Transactions on Machine Learning Research (TMLR).
Sep 08, 2024	I am reviewing for The Journal of Machine Learning Research (JMLR).
Oct 06, 2023	I am invited to give a talk at the Algorithms & Computationally Intensive Inference seminars at the University of Warwick, Coventry, England.

Selected Publications

MindFlayer: Efficient Asynchronous Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times

Artavazd Maranjyan, Omar Shaikh Omar, and Peter Richtárik

arXiv preprint arXiv:2410.04285, 2024

Abs Bib HTML

We study the problem of minimizing the expectation of smooth nonconvex functions with the help of several parallel workers whose role is to compute stochastic gradients. In particular, we focus on the challenging situation where the workers’ compute times are arbitrarily heterogeneous and random. In the simpler regime characterized by arbitrarily heterogeneous but deterministic compute times, Tyurin and Richtárik (NeurIPS 2023) recently designed the first theoretically optimal asynchronous SGD method, called Rennala SGD, in terms of a novel complexity notion called time complexity. The starting point of our work is the observation that Rennala SGD can have arbitrarily bad performance in the presence of random compute times – a setting it was not designed to handle. To advance our understanding of stochastic optimization in this challenging regime, we propose a new asynchronous SGD method, for which we coin the name MindFlayer SGD. Our theory and empirical results demonstrate the superiority of MindFlayer SGD over existing baselines, including Rennala SGD, in cases when the noise is heavy tailed.
@article{maranjyan2024mindflayer, title = {MindFlayer: Efficient Asynchronous Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times}, author = {Maranjyan, Artavazd and Omar, Omar Shaikh and Richt{\'a}rik, Peter}, journal = {arXiv preprint arXiv:2410.04285}, url = {https://arxiv.org/abs/2410.04285}, year = {2024} }
Gradskip: Communication-accelerated local gradient methods with better computational complexity

Artavazd Maranjyan, Mher Safaryan, and Peter Richtárik

arXiv preprint arXiv:2210.16402, 2022

Abs Bib HTML Video Poster Slides

We study a class of distributed optimization algorithms that aim to alleviate high communication costs by allowing clients to perform multiple local gradient-type training steps prior to communication. In a recent breakthrough, Mishchenko et al. (2022) proved that local training, when properly executed, leads to provable communication acceleration, and this holds in the strongly convex regime without relying on any data similarity assumptions. However, their ProxSkip method requires all clients to take the same number of local training steps in each communication round. We propose a redesign of the original ProxSkip method, allowing clients with “less important” data to get away with fewer local training steps without impacting the overall communication complexity of the method. In particular, we prove that our modified method, GradSkip, converges linearly under the same assumptions and has the same accelerated communication complexity, while the number of local gradient steps can be reduced relative to a local condition number. We further generalize our method by extending the randomness of probabilistic alternations to arbitrary unbiased compression operators and by considering a generic proximable regularizer. This generalization, which we call GradSkip+, recovers several related methods in the literature as special cases. Finally, we present an empirical study on carefully designed toy problems that confirm our theoretical claims.
@article{maranjyan2022gradskip, title = {Gradskip: Communication-accelerated local gradient methods with better computational complexity}, author = {Maranjyan, Artavazd and Safaryan, Mher and Richt{\'a}rik, Peter}, journal = {arXiv preprint arXiv:2210.16402}, url = {https://arxiv.org/abs/2210.16402}, year = {2022}, }