LAUSR.org creates dashboard-style pages of related content for over 1.5 million academic articles. Sign Up to like articles & get recommendations!

Parallel Multi-Block ADMM with o(1 / k) Convergence

Photo by lgnwvr from unsplash

This paper introduces a parallel and distributed algorithm for solving the following minimization problem with linear constraints: $$\begin{aligned} \text {minimize} ~~&f_1(\mathbf{x}_1) + \cdots + f_N(\mathbf{x}_N)\\ \text {subject to}~~&A_1 \mathbf{x}_1 ~+… Click to show full abstract

This paper introduces a parallel and distributed algorithm for solving the following minimization problem with linear constraints: $$\begin{aligned} \text {minimize} ~~&f_1(\mathbf{x}_1) + \cdots + f_N(\mathbf{x}_N)\\ \text {subject to}~~&A_1 \mathbf{x}_1 ~+ \cdots + A_N\mathbf{x}_N =c,\\&\mathbf{x}_1\in {\mathcal {X}}_1,~\ldots , ~\mathbf{x}_N\in {\mathcal {X}}_N, \end{aligned}$$minimizef1(x1)+⋯+fN(xN)subject toA1x1+⋯+ANxN=c,x1∈X1,…,xN∈XN,where $$N \ge 2$$N≥2, $$f_i$$fi are convex functions, $$A_i$$Ai are matrices, and $${\mathcal {X}}_i$$Xi are feasible sets for variable $$\mathbf{x}_i$$xi. Our algorithm extends the alternating direction method of multipliers (ADMM) and decomposes the original problem into N smaller subproblems and solves them in parallel at each iteration. This paper shows that the classic ADMM can be extended to the N-block Jacobi fashion and preserve convergence in the following two cases: (i) matrices $$A_i$$Ai are mutually near-orthogonal and have full column-rank, or (ii) proximal terms are added to the N subproblems (but without any assumption on matrices $$A_i$$Ai). In the latter case, certain proximal terms can let the subproblem be solved in more flexible and efficient ways. We show that $$\Vert {\mathbf {x}}^{k+1} - {\mathbf {x}}^k\Vert _M^2$$‖xk+1-xk‖M2 converges at a rate of o(1 / k) where M is a symmetric positive semi-definte matrix. Since the parameters used in the convergence analysis are conservative, we introduce a strategy for automatically tuning the parameters to substantially accelerate our algorithm in practice. We implemented our algorithm (for the case ii above) on Amazon EC2 and tested it on basis pursuit problems with >300 GB of distributed data. This is the first time that successfully solving a compressive sensing problem of such a large scale is reported.

Keywords: admm; parallel multi; multi block; mathbf; convergence

Journal Title: Journal of Scientific Computing
Year Published: 2017

Link to full text (if available)


Share on Social Media:                               Sign Up to like & get
recommendations!

Related content

More Information              News              Social Media              Video              Recommended



                Click one of the above tabs to view related content.