Loading...

Submodularity in action: from machine learning to signal processing applications

Tohidi, E ; Sharif University of Technology | 2020

337 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/MSP.2020.3003836
  3. Publisher: Institute of Electrical and Electronics Engineers Inc , 2020
  4. Abstract:
  5. Submodularity is a discrete domain functional property that can be interpreted as mimicking the role of well-known convexity/concavity properties in the continuous domain. Submodular functions exhibit strong structure that lead to efficient optimization algorithms with provable near-optimality guarantees. These characteristics, namely, efficiency and provable performance bounds, are of particular interest for signal processing (SP) and machine learning (ML) practitioners, as a variety of discrete optimization problems are encountered in a wide range of applications. Conventionally, two general approaches exist to solve discrete problems: 1) relaxation into the continuous domain to obtain an approximate solution or 2) the development of a tailored algorithm that applies directly in the discrete domain. In both approaches, worst-case performance guarantees are often hard to establish. Furthermore, they are often complex and thus not practical for large-scale problems. In this article, we show how certain scenarios lend themselves to exploiting submodularity for constructing scalable solutions with provable worst-case performance guarantees. We introduce a variety of submodular-friendly applications and elucidate the relation of submodularity to convexity and concavity, which enables efficient optimization. With a mixture of theory and practice, we present different flavors of submodularity accompanying illustrative real-world case studies from modern SP and ML. In all of the cases, optimization algorithms are presented along with hints on how optimality guarantees can be established. © 1991-2012 IEEE
  6. Keywords:
  7. Machine learning ; Optimization ; Approximate solution ; Discrete optimization problems ; Functional properties ; Optimization algorithms ; Signal processing applications ; Submodular functions ; Theory and practice ; Worst-case performance ; Signal processing
  8. Source: IEEE Signal Processing Magazine ; Volume 37, Issue 5 , 2020 , Pages 120-133
  9. URL: https://ieeexplore.ieee.org/document/9186137