Maths Seminar Series: Andreas Stathopoulos, College of William and Mary – State-of-the-art SVD for Big Data

The singular value decomposition (SVD) is one of the core computations of today’s scien- ti􏰏ic applications and data analysis tools. The main goal is to compute a compact representation of a high dimensional operator, a matrix, or a set of data that best resembles the original in its most important features. Thus, the SVD is widely used in scienti􏰏ic computing and machine learning, including low rank factorizations, graph learning, unsupervised learning, matrix completion, compression and analysis of images and text. The popularity of the SVD has resulted in an increased diversity of methods and im- plementations that exploit speci􏰏ic features of the input data (e.g., dense/sparse matrix, data distributed among the computing devices, data from queries or batch access, spectral decay) and certain constraints on the computed solutions (e.g., few/many number of singular values and singular vectors computed, targeted part of the spectrum, accuracy). The use of the proper method and the customization of the settings can signi􏰏icantly reduce the cost. In this talk, we overview the most relevant methods in terms of computing cost and accuracy (direct methods, iterative methods, online methods), including the most recent advances in randomized and online SVD solvers. For iterative methods, the stopping criterion has arguably the biggest impact in terms of cost and quality of the solution. We present two case studies where custom stopping criteria reduce computational cost drastically over the traditional criteria used in iterative software. Finally, we discuss the current state of the software on widely used platforms (MAT- LAB, Python’s numpy/scipy and R) as well as high-performance solvers with support for multicore, GPU, and distributed memory.

Bio