This paper presents the development of a framework for establishment of virtual environment for testing and tuning of attitude controller for rotary wing Unmanned Aerial Vehicles (UAVs). A flybarless mini-helicopter UAV is used as the platform for exposition of the proposed framework. A hardware-in-the-loop simulation (HILS) framework is established using a physics based flight dynamics simulation to enable controller design for rotary wing UAVs. The HILS setup includes the flight dynamics model, physical servo actuators and actual UAV autopilot. A computationally light real-time flight dynamics simulation is developed by using the properties estimated using series of simple ground-based experiments to simulate the small unmanned helicopter. The simulation is validated by performing flight tests on the actual UAV. It is demonstrated that accurate physics based simulations can be done without performing system-identification experiments, which can be an issue for an unstable rotary-wing vehicles with unknown dynamics. The utility of the HILS setup has been established by using the tuned PI attitude controller developed in the virtual environment for stabilization of the actual UAV under hovering condition. The validated HILS setup obviated the need for carrying out flight testing for system-identification, as all the relevant parameters required for the real-time simulation could be estimated using ground based tests. © 2017 by the American Helicopter Society International, Inc.