中国科学院重庆绿色智能技术研究院机构知识库

QQ客服 官方微博 反馈留言

	Parallel Adaptive Stochastic Gradient Descent Algorithms for Latent Factor Analysis of High-Dimensional and Incomplete Industrial Data
	Qin, Wen1,2; Luo, Xin3; Li, Shuai4,5; Zhou, MengChu6,7
	2023-06-01
摘要	Latent factor analysis (LFA) is efficient in knowledge discovery from a high-dimensional and incomplete (HDI) matrix frequently encountered in industrial big data-related applications. A stochastic gradient descent (SGD) algorithm is commonly adopted as a learning algorithm for LFA owing to its high efficiency. However, its sequential nature makes it less scalable when processing large-scale data. Although alternating SGD decouples an LFA process to achieve parallelization, its performance relies on its hyper-parameters that are highly expensive to tune. To address this issue, this paper presents three extended alternating SGD algorithms whose hyper-parameters are made adaptive through particle swarm optimization. Correspondingly, three Parallel Adaptive LFA (PAL) models are proposed and achieve highly efficient latent factor acquisition from an HDI matrix. Experiments have been conducted on four HDI matrices collected from industrial applications, and the benchmark models are LFA models based on state-of-the-art parallel SGD algorithms including the alternative SGD, Hogwild!, distributed gradient descent, and sparse matrix factorization parallelization. The results demonstrate that compared with the benchmarks, with 32 threads, the proposed PAL models achieve much speedup gain. They achieve the highest prediction accuracy for missing data on most cases. Note to Practitioners-HDI data are commonly encountered in many industrial big data-related applications, where rich knowledge and patterns can be extracted efficiently. An SGD based-LFA model is popular in addressing HDI data due to its efficiency. Yet when dealing with large-scale HDI data, its serial nature greatly reduces its scalability. Although alternating SGD can decouple an LFA process to implement parallelization, its performance depends on its hyper-parameter whose tuning is tedious. To address this vital issue, this study proposes three extended alternating SGD algorithms whose hyper-parameters are made via through a particle swarm optimizer. Based on them, three models are realized, which are able to efficiently obtain latent factors from HDI matrices. Compared with the existing and state-of-the-art models, they enjoy their hyper-parameter-adaptive learning process, as well as highly competitive computational efficiency and representation learning ability. Hence, they provide practitioners with more scalable solutions when addressing large HDI data from industrial applications.
关键词	Adaptation models Training Data models Convergence Stochastic processes Sparse matrices Tuning Big data latent factor analysis Index Terms adaptive model parallelization machine learning stochastic gradient descent high-dimensional and incomplete matrix
DOI	10.1109/TASE.2023.3267609
发表期刊	IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING
ISSN	1545-5955
页码	14
通讯作者	Luo, Xin(luoxin@swu.edu.cn) ; Zhou, MengChu(zhou@njit.edu)
收录类别	SCI
WOS记录号	WOS:001008043400001
语种	英语