Loading...

Stochastic successive convex approximation for non-convex constrained stochastic optimization

Liu, A ; Sharif University of Technology | 2019

629 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/TSP.2019.2925601
  3. Publisher: Institute of Electrical and Electronics Engineers Inc , 2019
  4. Abstract:
  5. This paper proposes a constrained stochastic successive convex approximation (CSSCA) algorithm to find a stationary point for a general non-convex stochastic optimization problem, whose objective and constraint functions are non-convex and involve expectations over random states. Most existing methods for non-convex stochastic optimization, such as the stochastic (average) gradient and stochastic majorization-minimization, only consider minimizing a stochastic non-convex objective over a deterministic convex set. The proposed CSSCA algorithm can also handle stochastic non-convex constraints in optimization problems, and it opens the way to solving more challenging optimization problems that occur in many applications. The algorithm is based on solving a sequence of convex objective/feasibility optimization problems obtained by replacing the objective/constraint functions in the original problems with some convex surrogate functions. The CSSCA algorithm allows a wide class of surrogate functions and thus provides many freedoms to design good surrogate functions for specific applications. Moreover, it also facilitates parallel implementation for solving large-scale stochastic optimization problems, which arise naturally in today's signal processing such as machine learning and big data analysis. We establish the convergence of the CSSCA algorithm with a feasible initial point, and customize the algorithmic framework to solve several important application problems. Simulations show that the CSSCA algorithm can achieve superior performance over existing solutions. © 1991-2012 IEEE
  6. Keywords:
  7. Non-convex stochastic optimization ; Successive convex approximation ; Approximation algorithms ; Constrained optimization ; Set theory ; Signal processing ; Algorithmic framework ; Non-convex constraints ; Optimization problems ; Parallel implementations ; Parallel optimization ; Stochastic optimization problems ; Stochastic optimizations ; Successive convex approximations ; Stochastic systems
  8. Source: IEEE Transactions on Signal Processing ; Volume 67, Issue 16 , 2019 , Pages 4189-4203 ; 1053587X (ISSN)
  9. URL: https://ieeexplore.ieee.org/document/8752072