The IEEE 802.11ad standard operates in the mmWave frequency band and supports data transfer speeds of up to 7 Gbps. However, the signal propagation characteristics in the mmWave band are adverse, and the achievable coverage area is less (around 6m). Hence, the coverage area around the access point (AP) is partitioned into virtual sectors and, AP communicates with stations in each sector by employing directional beamforming for improved coverage. Therefore, at any given instance, the AP serves only one sector, and in general, AP chooses the sectors in a round-robin policy. However, this round-robin based sector selection results in inefficient channel utilization when the traffic generated across the sectors is non-uniform. In this paper, a short-term memory based online learning framework is developed for efficient sector selection by the AP for improving throughput while balancing medium access delay and average queue size at the STAs. The performance of the proposed optimal sector selection policy is compared with the traditional round-robin based sector selection and random sector selection policies. From the performance analysis, it is observed that the proposed sector allocation framework improves the throughput by 51% and 112% when compared with round-robin and random sector allocation policies, respectively. © 2020 IEEE.