Yes.  When fruit/nut trees are stressed, they drop all the extra fruit they won't be able to carry to maturity under those conditions.