High dimensional datasets expose a critical obstacle in machine learning. Feature selection overcomes this obstacle by eliminating duplicated and unimportant features from the dataset to increase the robustness of learning algorithms. This paper introduces a binary version of a hybrid swarm intelligence approach as a wrapper method for feature selection that gathers between the strengths of both the grey wolf and particle swarm optimizers. This approach is named Improved Binary Grey Wolf Optimization (IBGWO). The original version of this hybrid approach was proposed in the literature with a continuous search space as a high-level hybrid form, which runs the optimizers one after the other. Two different types of transfer functions, named S-Shaped and V-Shaped, are applied in this work to turn continuous data into binary. Nine of high-dimensional small-instance medical datasets are employed to assess the proposed approach. The experimental results demonstrate that IBGWO based on S-Shaped (IBGWO-S) outperforms the binary particle swarm and the binary grey wolf optimizers on six out of nine datasets according to the classification accuracy and fitness values. IBGWO-S selects the fewest features on 100% of the datasets. The results show IBGWO based on V-Shaped (IBGWO-V) outperforms the binary particle swarm and binary grey wolf optimizers on five datasets based on the classification accuracy and fitness values. The results indicate that IBGWO-V outperforms IBGWO-S in terms of all studied evaluation metrics. The results also show that IBGWO-S and IBGWO-V outperform eight meta-heuristics known in the literature in selecting the relevant features with acceptable classification accuracy.