LAUSR

Abstract The emergence of single-cell RNA-sequencing is ideally placed to unravel cellular heterogeneity in biological systems, an extremely challenging problem in single cell RNA-sequencing studies. However, most current computational approaches lack the sensitivity to reliably detect nonlinear gene-gene relationships masked by dropout events. We proposed a kernel non-negative matrix factorization framework for detecting nonlinear relationships among genes, where the new kernel is developed using kernel tricks on cellular differentiability correlation. The newly constructed kernel not only provides a description on the gene-gene relationship, but also helps to build a new low-dimensional representation on the original data. Besides, we developed an efficient method for determining the optimal cluster number within each data set with the usage of Diffusion Maps. The proposed algorithm is further compared with representative algorithms: SC3 and several other state-of-the-art clustering methods, on several benchmark or real scRNA-Seq datasets using internal criteria (clustering number accuracy) and external criteria (Adjusted rand index and Normalized mutual information) to show effectiveness of our method.