This article presents a compact substrate integrated coaxial line (SICL) based quadrature hybrid operating at 10 GHz specifically for X-band communication with controllable harmonic rejection. The proposed SICL based compact branch-line coupler utilizes four open-ended coaxial stubs. Design equations are analytically derived using even-odd mode analysis and the range of realizable impedance is graphically determined from the derived equations. Variation in position of transmission zero as a function of open-ended stub's electrical length is illustrated through simulation and a 15 dB harmonic rejection up to 31.8 GHz observed. The fabricated SICL based quadrature hybrid demonstrates a measured fractional bandwidth of 12.24% and occupies only 0.10λ2garea. Use of compact size and electromagnetically robust passive components designed in SICL technology provides exceptional front-end communication system. © 2019 IEEE.