It’s not like most plants can simultaneously have both male and female reproductive organs, or that oak, birch, linden etc. i.e. trees placed in urban area, won‘t bear mangos.
Not to mention that in order to get the fruits you see and eat, there need to have the trees well tended. Leaving it be would’ve meant chances of infestations, „mini-fruits“, and whatnot.
Humans are also not the only thing waiting to eat the fruits.