In order to estimate and compare the genetic diversity and population structure between Abbottina rivularis and Hemiculter leucisculus in the Yellow River and Weihe River in Henan province, we used cytb sequences to estimate population genetic parameters and tested for demographic fluctuations. Our results showed that the population polymorphism of the two species was low. The results from phylogenetic tree, as well as haplotype network structure, showed that A. rivularis and H. leucisculus had three distinct lineages, however, there was no phylogeographic structure. The gene flow with moderate to low among populations explained that the two species had lower genetic diversity and distinct population differentiation. Then we analyzed the Demographic History of A. rivularis and H. leucisculus. Bayesian skyline plot revealed that the stock of the two species declined dramatically 0.019 and 0.026 million years ago, respectively. In view of the population polymorphism, population differentiation and demographic history, we concluded that the genetic diversity of A. rivularis and H. leucisculus was low in this study, and it was necessary to strengthen the protection of their resources.