LAUSR

In a recent experiment [Vochezer {\it et al.,} Phys. Rev. Lett. \textbf{120}, 073602 (2018)], a novel kind of hybrid atom-opto-mechanical system has been realized by coupling atoms in a lattice to a membrane. While such system promises a viable contender in the competitive field of simulating non-equilibrium many-body physics, its complete steady-state phase diagram is still lacking. Here we study the phase diagram of this hybrid system based on an atomic Bose-Hubbard model coupled to a quantum harmonic oscillator. We take both the expectation value of the bosonic operator and the mechanical motion of the membrane as order parameters, and thereby identify four different quantum phases. Importantly, we find the atomic gas in the steady state of such non-equilibrium setting undergoes a superfluid-Mott-insulator transition when the atom-membrane coupling is tuned to increase. Such steady-state phase transition can be seen as the non-equilibrium counterpart of the conventional superfluid-Mott-insulator transition in the ground state of Bose-Hubbard model. Further, no matter which phase the quantum gas is in, the mechanic motion of the membrane exhibits a transition from an incoherent vibration to a coherent one when the atom-membrane coupling increases, agreeing with the experimental observations. Our present study provides a simple way to study non-equilibrium many-body physics that is complementary to ongoing experiments on the hybrid atomic and opto-mechanical systems.