Job Description

Research area: advanced mathematical methods applied to big data processing.
Keywords: compressed sensing, tensor models, sparse tensor recovery, massive antennas, large-scale systems, parametric estimation theory, random matrix theory, Bayesian inference, machine learning

Profil of the applicant : An outstanding, self and highly motivated candidate is solicited with a strong interest in the field of advanced mathematical methods applied to signal processing. Candidates with
- a solid background in probability, statistics and algebra,
- excellent analytical, technical and problem solving skills,
- good written, oral communication and organization skills,
are encouraged to apply. Above all, the applicants must be motivated to learn quickly and work effectively on challenging research problems.

Context of the proposal : The thesis will be conducted in collaboration between several national and international laboratories :
1. Rémy Boyer (Associated Professor, HDR), L2S lab., Signals and Statistics Dep.
2. Pascal Larzabal (Professor and lab. Director), ENS/Cachan, SATIE lab.
3. Gérard Favier (Emeritus Research Director at CNRS), I3S lab.UNS/CNRS
4. André L. F. de Almeida (Full Professor), UFC , Teleinformatics Engineering Department

The "massive/big data" processing challenge which is at the heart of the proposed thesis is a major research theme for the Paris-Saclay University and for the Labex (Laboratoire d'Excellence) DIGICOSME and UNC@SOPHIA where the L2S, SATIE and I3S are involved. Depending on the interest of the PhD student, a wide range of applications can be considered as for instance source localizaion based on very large arrays, cooperative wireless communications or bio-medical applications as the Magnetic Resonance Imaging where the involved laboratories have a strong and world-wide recognized expertise. The research work will be done in CentraleSupélec/L2S premises but travel opportunities for meeting with collaborators are encouraged.



Subject: From Internet to large research infrastructures, the volume of data generated by our societies is continuously increasing. Such massive datasets which are characterized not only by their huge volume, but also their heterogeneity (multimodal data), precision (noisy data) and incompleteness (missing data), define what is now commonly named under the term of big data. In many disciplines and areas of application, data inherently has more than two axes of variation, also called modes, and can be arranged as tensors (i.e. multi-way arrays) [1-3]. Thus, tensor decompositions/models of multi-way datasets are particularly useful to represent and analyze big data. Large-scale tensor decompositions have received a particular attention very recently, due to their capability to handle a variety of mining tasks that are increasingly being applied to massive datasets [4-7]. Application fields are diverse, and include telecommunications, audio/video, medical imaging, chemometrics, and intelligent transport systems, to mention a few. Main advantages of tensor decompositions are:
- Uniqueness properties under mild conditions.
- Reduced parametric complexity in comparison with the dimensions of data tensors.
- Existence of simple and efficient parameter estimation methods, like the alternating least-squares (ALS) algorithm.

In this thesis, first from a fundamental perspective, different tensor models and algorithms will be studied for big data modelling and processing. Compact representations can be achieved by resorting to higher order singular value decomposition (HOSVD)-based models [8,24], tensor models with constraints [9], tensor train models [10,11], hierarchical Tucker models [12-14], and tensor networks [4]. In practice, low-rank approximations are crucial for reducing the data storage requirements and the processing complexity. This low-rank assumption is directly linked with the notion of rank of a tensor that, unlike matrices, is not unique since it depends on the considered decomposition. After an in-depth study of the different tensor decompositions and their uniqueness properties, the research work will be dedicated in the development of algorithms and analytical performance for recovering low-rank tensors from undercomplete big data, i.e. the completion of data tensors having missing entries under the low-rank assumption. This low-rank tensor recovery problem which can be viewed as an extension of the matrix completion and compressive sensing (CS) problems, arises in numerous real-world applications such as medical imaging, hyperspectral imaging, seismic data, and road traffic data, among many others. It consists in reconstructing randomly under-sampled low-rank tensors. In CS, under-sampled signals are recovered using their representation in a prespecified basis by means of a small number of nonzero coefficients, which defines the sparsity constraint. In tensor completion, the sparsity is played by the low-rank property. Various approaches exist for solving this problem depending on the choice of the considered tensor decomposition, of the cost function to be minimized under low-rank constraints, and of the used optimization algorithm.

The tensor completion problem is linked with the tensor learning and tensor prediction problems which consist in using training data for estimating/predicting missing entries, with the aim to use as few observations as possible. This is to be compared with a semi-supervised learning. Important examples of such problems are the recommender systems, like for instance the famous Netflix system for renting videos. Other applications concern the design of acquisition systems with the goal to reduce the number of measurements and consequently the storage requirement, and therefore to accelerate the acquisition, while ensuring good recovery performance. These problems are generally solved by minimizing the normalized mean square error (NMSE) under low-rank constraints. Two main approaches exist. One consists in combining an iterative gradient algorithm with repeated low-rank truncation, which leads to the class of iterative hard thresholding (IHT)-based methods [15,16]. A second approach is based on a reformulation of the problem as an optimization problem under low-rank constraint, and the application of an optimization technique, as for instance the alternating direction method of Lagrange multipliers (ADMM) [17,18].

Some link with machine learning (ML) can also be underlined via a formulation of the problem as a convex optimization under the constraint of low multilinear rank.
Fundamental questions concerning recovery guarantees and the minimum number of data needed for tensor completion will be addressed. Moreover, closed-form expressions of Cramér-Rao bounds of the mean square error of the parameter estimates will be established in presence of additive white Gaussian noise, as recently done for structured CP tensor decompositions [19]. Finally, distributed algorithms to compute tensor decompositions will be also developed to achieve reasonable performance in scenarios where the processing power can be distributed across a network of processing units, or sensors [20-23].
From an applications perspective, an important example is that of massive multiple-input multiple-output (MIMO) antenna systems, which is one of the key technologies currently under consideration for future fifth-generation (5G) wireless communication standards. Massive MIMO systems shall employ VLA of hundreds of antennas at the base station to serve many users at the same time. In this context, sparsity-aware tensor-based methods can be useful for channel estimation, due to the multidimensionality (i.e. space, time, frequency, and polarization domains), as well as the low-rank nature of the wireless channel.

References
[1] T. Kolda and B. Bader, “Tensor decompositions and applications,” SIAM Review, vol. 51, no. 3, pp. 455–500, September 2009.

[2] A. Smilde, R. Bro, and P. Geladi, Multi-way Analysis: Applications in the Chemical Sciences, New York: John Wiley & Sons Ltd, 2004.

[3] P. Comon, X. Luciani, and A. L. F. de Almeida, “Tensor decompositions, alternating least squares and other tales,” Jour. Chemometrics, vol. 23, pp. 393–405, 2009.

[4] M. Mørup, “Applications of tensor (multiway array) factorizations and decompositions in data mining,” Wiley Interdisc. Rew.: Data Mining and Knowledge Discovery, vol. 1, no. 1, pp. 24–40, 2011.

[5] A Cichocki Era of Big Data Processing: A New Approach via Tensor Networks and Tensor Decompositions. In: International Workshop on Smart Info-Media Systems in Asia (SISA-2013), (2013).

[6] A Cichocki, C Mandic, AH Phan, C Caiafa, G Zhou, Q Zhao, L De Lathauwer Tensor Decompositions for Signal Processing Applications. From two-way to multiway component analysis. IEEE Signal Processing Magazine, 32(2), 145-163, (2015).

[7] Mardani, M.; Mateos, G.; Giannakis, G.B., "Subspace Learning and Imputation for Streaming Big Data Matrices and Tensors," in Trans. Signal Processing, vol.63, no.10, pp.2663-2677, 2015.

[8] L. De Lathauwer, B. De Moor, and J. Vandewalle, “A multilinear singular value decomposition,” SIAM Journal of Matrix Analysis and Applications, vol. 24, pp. 1253–1278, 2000.

[9] G. Favier and A. L. F. de Almeida, “Overview of constrained PARAFAC models,” EURASIP J. Advances in Signal Processing, vol. 2014, no. 142, pp. 1-25, 2014.

[10] I. V. Osedelets, Tensor-Train decomposition, SIAM Journal on Scientific Computing, vol. 33, n. 5, pp. 2295-2317, 2011.

[11] I. V. Oseledets and E. Tyrtyshnikov, “Breaking the curse of dimensionality, or how to use SVD in many dimensions,” SIAM J. Scientific Computing, vol. 31, no. 5, pp. 3744–3759, 2009.

[12] A. Uschmajew and B. Vandereycken, “The geometry of algorithms using hierarchical tensors,” Linear Algebra and its Applications, vol. 439, 2013.

[13] L. Grasedyck, “Hierarchical singular value decomposition of tensors,” SIAM J. Matrix Analysis Applications, vol. 31, no. 4, pp. 2029–2054, 2010.
[14] L. Grasedyck and W. Hackbusch, “An introduction to hierarchical (H-)rank and TT-rank of tensors with examples,”Comput. Meth. In Appl. Math., vol. 11, no. 3, pp. 291–304, 2011.
[15] H. Rauhut, R. Schneider, and Z. Stojanac, “Low rank tensor recovery via iterative hard thresholding”, Proc; 10th Int. Conf. Sampling Theory Applicat., 2013.
[16] J.H. Goulart and G. Favier, “An iterative hard thresholding algorithm with improved convergence for low-rank tensor recovery”, 23rd European Signal Processing Conference (EUSIPCO'2015), Nice, Sept. 2015.
[17] S. Gandy, B. Recht, and I. Yamada, “Tensor completion and low-n-rank tensor recovery via convex optimization”, Inverse Problems, 27(2) 025010, 2011.

[18] D. Goldfarb, and Z. Qin, “Robust low-rank tensor recovery: Models and algorithms”, arXiv: 1311.6182, 2013.
[19] J.H. Goulart, M. Boizard, R. Boyer, G. Favier, and P. Comon, “Tensor CP Decomposition with Structured Factor Matrices: Algorithms and Performance” to appear in IEEE Journal of Selected Topics in Signal Processing, June 2016.

[20] A.-H. Phan and A. Cichocki, “PARAFAC algorithms for large-scale problems,” Neurocomputing, vol. 74, no. 11, pp. 1970–1984, 2011.

[21] E. Papalexakis, C. Faloutsos, and N. Sidiropoulos, “ParCube: Sparse parallelizable tensor decompositions,” Proc. European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 2012.

[22] A. L.F. de Almeida, A. Y. Kibangou, "Distributed computation of tensor decompositions in collaborative networks," in Proc. IEEE CAMSAP 2013, vol. 1, no. 1, pp.232-235, Dec. 2013.

[23] A.L.F. de Almeida, A.Y. Kibangou, "Distributed large-scale tensor decomposition," in Proc. IEEE ICASSP 2014, vol. 1, no.1, pp.26-30, May 2014.

[24] R. Badeau, and R. Boyer, "Fast multilinear singular value decomposition for structured tensors", SIAM J. on Matrix Analysis and Applications, Special Issue on Tensor Decompositions and Applications, Volume 30, No. 3, 2008, pp. 1008-1021.

Follow @eurasip