This paper presents an analytical study of the aeroelastic behavior of composite rotor blades with straight and swept tips. The blade is modeled by beam type finite elements. A single finite element is used to model the swept tip. The nonlinear equations of motion for the finite element model are derived using Hamilton's principle and based on a moderate deflection theory and accounts for: arbitrary cross-sectional shape, pretwist, generally anisotropic material behavior, transverse shears and out-of-plane warping. Numerical results illustrating the effects of tip sweep, anhedral and composite ply orientation on blade aeroelastic behavior are presented. It is shown that composite ply orientation has a substantial effect on blade stability. At low thrust conditions, certain ply orientations can cause instability in the lag mode. The flap-torsion coupling associated with tip sweep can also induce aeroelastic instability in the blade. This instability can be removed by appropriate ply orientation in the composite construction.