Zinc (Zn) deficiency, threatening billions of people’s health, has attracted worldwide concerns. Wheat is one of the most important staple food crops, and the major source for human Zn nutrition. Dryland is almost the main wheat production area in the world, but usually suffers from the low soil Zn availability due to the high soil pH, the high soil carbonate contents, and the frequent drought stress. Therefore, optimizing cultivar, fertilizer and soil nutrient management is urgently needed for stabilizing and increasing the wheat grain yield and simultaneously the grain Zn concentration on calcareous soil in dryland. In a three year field studies, cultivars with high grain yield were selected from 123 wheat cultivars, and the variation of their grain Zn concentration, reasons for high grain Zn concentration at high yield levels and the potential for increasing the grain Zn concentration were evaluated and explored. An in farm investigation together with soil and plant sampling was carried out in 9 different sites of three provinces in dryland wheat production areas in China from 2015-2018, to study the farmer’s wheat yield and the grain micronutrient concentration differences, and explore the grain Zn concentration difference at higher yield levels. Also, samples were collected from a long term location-fixed field experiment initiated in 2004, to explore the change of wheat grain yield and its Zn concentration and soil available macro- and micro-nutrients affected by nitrogen (N) and phosphorus (P) application rates, for the purpose to understand the right N and P application rates, corresponding soil available N, P and Zn nutrients for a higher grain yield and higher grain Zn concentration. Obtained results showed that, for high yielding cultivars, those with high grain Zn concentration also showed high shoot and grain Zn uptakes and high Zn harvest indexes. Analysis of straw Zn concentration showed that it varied from 1.2 to 10.5 mg kg^-1. Based on the established model between grain Zn and straw Zn, and assumed that the straw Zn of the high-yielding cultivars could be decreased to a lower level around 1.5 mg kg^-1, their grain Zn concentrations would be able to increase to 40.0 mg kg^-1. Results from the in farm investigation showed that, wheat grain Zn concentration varied from 12.2 to 48.0 mg kg^-1 for high-yielding farmers, and the soil available N was higher and available P was lower for the high grain Zn samples. Higher soil available potassium (K) and lower soil available ferrum (Fe) were also found beneficial for improving grain Zn concentration. The long term field experiment showed that each 100 kg P₂O₅ ha^-1 increase of P application rate could led to a decrease of wheat grain Zn by 9.2 mg kg^-1, and for balancing the wheat grain yield and its Zn concentration in dryland with calcareous soil and soil available Zn of 4.6 mg kg^-1, the N and P fertilizer rates should be maintained respectively at 160.0 kg N ha^-1 and 95.0 kg P₂O₅ ha^-1, when the grain yield is around 6000 kg ha^-1, its Zn concentration is at a relatively high level of 32.0 mg kg^-1, and soil available N and P are around 6.5 mg kg^-1 and 12.0 mg kg^-1, respectively at sowing. Our results showed the possibility for improving wheat grain Zn concentration by enhancing the Zn uptake from soil and its translocation from straw to grain by new cultivar breeding and selection, and by optimizing the soil N, P, Fe and Zn as well as the N and P fertilizer management in similar areas of calcareous soil and dryland climate in the world.