We have investigated the non-equilibrium solubility of Mn and Co in Zn 1-xTMxO (x = 5-30 at.% TM = Co, Mn) single crystal thin film and the role of Co/Mn concentration on the structural and magnetic properties. The films have been grown by pulsed laser deposition and the structural and magnetic characterization has been performed using high resolution transmission electron microscopy and superconducting quantum interference device respectively. Cobalt shows high solubility in single crystalline ZnO films with no tendency to form incoherent secondary precipitates up to 30 at.%. Whereas for Mn:ZnO, secondary precipitate started forming from Mn concentration of 25 at.% and above. HREELS of both Co and Mn L2,3 absorption edges shows that the Co and Mn atoms have substituted the Zn positions in the lattice. ZnO doped with 30 at.% Mn forms single crystalline spinel Mn3O4 phase on the sapphire substrate in the form of an interlayer which is responsible for aligning subsequent Mn:ZnO film with respect to sapphire as [1 1 -2 0]Al2O3 || [1 1 -2 0] ZnO compared to the usual [1 1 -2 0]Al2O3 || [0 1 -1 0] ZnO orientation. Saturation magnetization remained almost the same (∼10 20-1021 μB/cm3) for Co:ZnO but a composition dependent coercivity has been observed. For Zn0.75Co 0.25O film, the coercivity is ∼922 (769) Oe corresponding to applied field perpendicular (parallel) to the c axis, which is the highest value reported in this system. For Mn:ZnO ferromagnetism disappears beyond Mn concentrations of 15 at.%. © 2013 Elsevier Ltd. All rights reserved.