Problem with Apricot Tree
It sounds to me as if your apricot tree has done quite well by itself, despite not being in the best position for most of its life and possibly neglected.
I would firstly recommend pruning out any dead wood during the next few months. Then in spring prepare the soil around the tree to a couple of meters, digging it over and mixing with it some compost. Apply an appropriate fertiliser through its growing period (spring and summer). Keep it watered regularly, but be careful not to over-water as this could cause other problems in the long run.
This should help with the fruit forming more fully and ripening, although you may not see this until the following year.
If you do find any pests that you suspect are causing the holes in the leaves then i would be interested to see them, so please do forward any images you can get and hopefully i can tell you what they are and if they pose any problems. But for now it sounds to me as if your tree needs revitalising from the roots.
Hope this helps.