Nonequilibrium Quantum Phase Transition in a Hybrid Atom-Optomechanical System.

We consider a hybrid quantum many-body system formed by both a vibrational mode of a nanomembrane, which interacts optomechanically with light in a cavity, and an ultracold atom gas in the optical lattice of the out-coupled light. After integrating over the light field, an effective Hamiltonian reveals a competition between the localizing potential force and the membrane displacement force. For increasing atom-membrane interaction we find a nonequilibrium quantum phase transition from a localized non-motional phase of the atom cloud to a phase of collective motion. Near the quantum critical point, the energy of the lowest collective excitation vanishes, while the order parameter of the condensate becomes non-zero in the symmetry-broken state. The effect occurs when the atoms and the membrane are non-resonantly coupled.