Inexact higher-order proximal algorithms for tensor factorization

In the last decades, Matrix Factorization (MF) models and their multilinear extension-Tensor Factorization (TF) models have been shown to be powerful tools for high dimensional data analysis and features extraction. Computing MF's or TF's are commonly achieved by solving a constrained optimization subproblem on each block of variables, where the subproblems usually have a huge problem size that one has to rely on First-order Methods (FoM), i.e., gradient-based optimization methods. In this work, we consider Higher-order Methods (HoM), which are based on higher-order derivatives of the objective function. Compared to FoM, HoM are faster both in theory and practice. However, HoM has a higher per-iteration cost than FoM. Based on the recent development of efficient and implementable HoM, we consider higher-order proximal point methods within the BLUM framework which is potentially tractable for large-scale problems. For the newly proposed HoM, we introduce the appropriate objective functions, derive the algorithm, and show experimentally that the drop in the number of iterations with respect to their per-iteration cost make these HoM-based algorithms attractive for computing MF's and TF's.

matrix factorization, tensor decomposition, higher-order proximal point methods, constrained optimization

IEEE

Leplat, Valentin

019d30cb-499a-4996-967f-0d5566fcef56

Phan, Anh-Huy

6f26873e-826a-4a02-8384-87ad0d2a1026

Ang, Andersen

ed509ecd-39a3-4887-a709-339fdaded867

Leplat, Valentin

019d30cb-499a-4996-967f-0d5566fcef56

Phan, Anh-Huy

6f26873e-826a-4a02-8384-87ad0d2a1026

Ang, Andersen

ed509ecd-39a3-4887-a709-339fdaded867

Leplat, Valentin, Phan, Anh-Huy and Ang, Andersen (2023) Inexact higher-order proximal algorithms for tensor factorization. In Proceedings of the 2023 IEEE Statistical Signal Processing Workshop (SSP). IEEE. 5 pp . (In Press)