# On Theory of Model-Agnostic Meta-Learning Algorithms

December 12, 2019

Alireza Fallah, Ph.D. Candidate, EECS. Based on a joint work with Aryan Mokhtari, UT Austin, and Asu Ozdaglar, MIT.

Introduction

Imagine sitting in your autonomous car, going for a vacation. Your vehicle should follow the directions provided by the navigation app, and also use multiple sensors to monitor other vehicles, road signs, street light, etc. As a result, during the course of your journey, your car might need to take actions within a few seconds, such as turning or stopping. The question is how should your vehicle be programmed to be able to adapt to the new tasks within a short amount of time and limited data.

In a large fraction of artificial intelligence problems, ranging from robotics to image classification and pattern recognition, the goal is to design systems that use prior experience and knowledge to learn new skills more efficiently. Meta-learning or learning to learn formalizes this goal by using data from previous tasks to learn update rules or model parameters that can be fine-tuned to perform well on new tasks with small amount of data.

A particularly simple and effective approach for this problem, proposed by Finn et al., is model-agnostic meta learning (MAML). This approach finds a meta initialization which can be updated after the arrival of the new task and by using a gradient-based method. Several recent papers studied the empirical performance of various methods in this setting. In our work Fallah et al., we establish the first convergence guarantees of MAML type methods for general non-convex setting.

Problem Formulation

Let $\inline&space;T&space;=&space;\{T_i\}_{i&space;\in&space;I}$ denote the set of all tasks and let p be the probability distribution over tasks T
Each task T has a corresponding loss function $\inline&space;f_i(w):&space;R^d&space;\to&space;R$ which measures how well the model w performs on that task. We also define $\inline&space;f(w)$ as the expectation of fi over all tasks, i.e.,
$\inline&space;f(w)&space;:=&space;E_{i\sim&space;p}[f_i(w)].$

Figure 1a: Traditional statistical learning problem, Figure 1b: MAML problem

In traditional statistical learning, we aim to minimize the loss function f, as we expect its solution to be a proper approximation for the optimal solution of a new unseen task Ti (see Figure 1).
However, in the model-agnostic meta-learning problem (Figure 1), we aim to find the best point w* that performs well as an initial point for learning a new task Ti when we have budget for running a few steps of gradient descent. For simplicity, we focus on finding an initialization w* such that, after observing a new task Ti, one gradient step would lead to a good approximation for the minimizer of $\inline&space;f_i(w)$. We can formulate this goal as the following optimization problem

$\inline&space;\min_{w&space;\in&space;R^d}&space;F(w)&space;:=&space;E_{i&space;\sim&space;p}\left[&space;F_i(w)\right]&space;:=&space;E_{i&space;\sim&space;p}\left[f_i(w&space;-&space;\al&space;\nabla{f_i(w)})\right]$

The MAML algorithm proposed in Finn et al., at each iteration k, first selects a batch of tasks Bk, and then proceeds in two stages: the inner loop and the outer loop. In the inner loop, for each chosen task Ti in Bk, MAML computes a mid-point using a step of stochastic gradient on fi. Then, in the outer loop, MAML runs one step of stochastic gradient descent with an estimate of $\inline&space;\nabla&space;F$ computed over Bk using the the mid-points for each task Ti evaluated in the inner loop. The final update rule of MAML can be written as:

$\inline&space;w_{k+1}&space;=&space;w_k&space;-&space;\beta_k&space;\frac{1}{|B_k|}&space;\sum_{i&space;\in&space;B_k}&space;\left(I-\alpha&space;\tilde{\nabla}^2&space;f_i(w_{k})\right)\&space;\!&space;\tilde{\nabla}&space;f_i\!\left(w_k-\alpha&space;\tilde{\nabla}&space;f_i(w_k)\right)$

Table 1 : Our theoretical results for convergence of MAML, first-order approximation of MAML (FO-MAML), and our proposed Hessian-free MAML (HF-MAML) to a first-order stationary point (FOSP) in nonconvex settings. Here, d is the problem dimension, $\inline&space;\sigma$ is a bound on the standard deviation of $\inline&space;\nabla&space;f_i(w)$ from its mean $\inline&space;\nabla&space;f(w)$, and $\inline&space;\tilde&space;\sigma$ is a bound on the standard deviation of estimating $\inline&space;\nabla&space;f_i(w)$ using a single data point.

Results and Conclusion

As shown above, the exact implementation of MAML requires access to second-order information of the loss functions $\inline&space;\nabla^2&space;f_i$, and, therefore, the computational complexity of MAML per iteration scales poorly with the  problem dimension d. To resolve this issue, the authors in Finn et al. propose the first-order approximation of MAML (FO-MAML) which ignores the second-order term in the update rule of MAML. In our work, we show that if the learning rate $\inline&space;\alpha$ used for updating each individual task is small or the tasks are statistically close to each other, then the error induced by this first-order approximation is negligible. Nevertheless, in general, in contrast to MAML which can find an epsilon-first order stationary point for any arbitrary epsilon, the performance of FO-MAML is limited and it cannot achieve any small desired level of accuracy.

To address this issue, we introduce a new method, Hessian-free MAML (HF-MAML), which recovers the complexity bounds for MAML while it does not require computation of the Hessian or a Hessian-vector product. In fact, we show that HF-MAML enjoys a better convergence rate in comparison to FO-MAML, and, for any positive epsilon, it can find an epsilon-first-order stationary point after while keeping the computational cost at each iteration of $\inline&space;O(d)$. Hence, HF-MAML has the best of both worlds meaning that it has a low computational complexity as in FO-MAML and it can achieve any arbitrary accuracy for first-order stationarity as in MAML. These results are briefly provided in Table 1 and the details can be found in Fallah et al..

Figure 2: Comparison of MAML, FO-MAML, and HF-MAML. Figure 2a: Having exact gradients available, Figure 2b: Stochastic gradients with similar tasks, Figure 2c: Stochastic gradients with less similar tasks

Finally, we highlight our results through a simple experiment: Consider a 1-rank matrix factorization problem where the loss for task i is $\inline&space;f_i(x)&space;=&space;1/4&space;\|&space;x&space;x^T&space;-&space;M_i&space;\|_F^2$. Here, Mi is rank one, generated as $\inline&space;M_i=g_i*g_i^T$ where gi is a random Gaussian vector. The variance of gi controls task similarity $\inline&space;\sigma^2$. To capture noisy gradients and Hessians we add random Gaussian noise with variance $\inline&space;\tilde{\sigma^2}$ to the tasks gradients and Hessians.

Recall from our results that the best accuracy for MAML and HF-MAML is $\inline&space;O(\tilde{\sigma}/\sqrt{K})$ while the one for FO-MAML is $\inline&space;O(\alpha&space;\sigma&space;+&space;\tilde{\sigma}/\sqrt{K})$, which has the additional term $\inline&space;O(\alpha&space;\sigma)$ (it does not go to zero even with exact gradients and Hessians).

•  In Figure 2a, we assume gradients and Hessians are exact $\inline&space;(\tilde{\sigma}=0)$ and focus on task variation. We assume that the batch-size for tasks is equal to number of tasks equal to 20. In this case, even though the gradients are exact, FO-MAML converges to an error level with a gap compared to two others. This is consistent with our results.
•  We next consider noisy gradients and Hessians. We also set number of tasks equal to 50 and take a batch of tasks with size 10 at each iteration. In Figure 2b, we choose the variance of gi small to ensure the tasks are relatively similar. Here the additional persistent error of FO-MAML is negligible compared to the other term and all three methods behave similarly.
•  In Figure 2c, we increase the variance of gi (tasks are less similar). In this case, the $\inline&space;O(\alpha&space;\sigma)$ term dominates the other term in the error of FO-MAML, and it has worse performance compared to two others.

Alireza Fallah is a third-year Ph.D. student in the Electrical Engineering and Computer Science department, advised by Professor Asuman Ozdaglar. Before that, he earned a dual B.Sc. degree in Electrical Engineering and Pure Mathematics from Sharif University of Technology, Tehran, Iran. His research interest is mainly in optimization, the theory of machine learning, statistical inference, and Game Theory. His current research focuses on the theoretical understanding of optimization methods as well as developing fast and robust algorithms for machine learning applications. He has received several awards and fellowships, including Mathworks Engineering Fellowship, Siebel Scholarship, First prize in International Mathematics Competition (IMC) in 2016, and the silver medal in International Mathematical Olympiad (IMO) in 2011 and 2012. He can be contacted here