This letter presents a substrate-integrated-waveguide-based compact, dual-mode self-diplexing antenna. A single square cavity with a rectangular ring slot is used for both dual-mode excitation and radiation. A prototype diplexing antenna is fabricated that radiates linear, orthogonal polarization at 9.5 and 10.5 GHz. The measured peak gains at the two frequencies are 5.75 and 5.95 dBi, respectively. Intermodal coupling between the modes provides two transmission zeros, which are utilized to achieve port isolation better than 29 dB. The proposed technique can be used for closely spaced radiating frequencies. As no external filter is used, the structure is compact. Furthermore, because of the slot loading effect, the antenna is smaller than a recently published antenna claimed as the most compact antenna. Its working principle is discussed in detail. Finally, design guidelines are suggested. © 2018 IEEE.