Abstract: Recently, orbital angular momentum (OAM) based vorticose communication has attracted much attention because of its potential to significantly increase the spectrum efficiency (SE) of wireless communications. However, the multiple radio frequency (RF) chains used for multiple OAM modes lead to an unexpected cost for wireless vorticose communications. In this paper, we first propose the mode modulation (MM) based OAM system to allow multiple OAM modes sharing a common RF chain. The proposed MM cannot only reduce the hardware cost, but also boost the SE by introducing the mode as an additional dimension for data transmission. Moreover, without the channel state information (CSI) feedback, we develop an equal-probability mode modulation (EMM) scheme to maximize the SE of MM-based OAM systems with the limited RF chains. In addition, with the CSI feedback, we develop a Huffman coding based adaptive mode modulation (AMM) scheme, which can adaptively choose the OAM modes to further increase the SEs of OAM-based wireless communications. We also develop the joint power and probability allocation policy for the AMM scheme to achieve the maximum SEs for OAM-based vorticose communications. Numerical results demonstrate that the MM can offer a new mode dimension for vorticose communications and the AMM scheme can achieve larger SE than the EMM scheme.