Non-conforming approximation methods are becoming increasingly popular because of the potential to apply to multi-material and multi-model analysis for both bounded and unbounded domains. In this paper, we present a least-square approximation based method to solve the one or two dimensional elliptic problems on an unbounded domain. The method gives exponential accuracy and shows superior performance when compared to other numerical methods. Differentiability estimates and the main stability estimate theorem, using a non-conforming spectral element method, are also discussed. The exponential convergence rate of the proposed method is also shown through rigorous error estimate and specific numerical examples. © 2016 Elsevier B.V.