What they are potted up in might have something to do with it. Garden soil and compost are not recommended ingredients for growing a plant in a container, even short term. The soil particles are too small, adversely impacting good drainage and aeration; and compost is excessively water retentive. Combined with the limited drainage afforded by growing in a container and the negative effects of the garden soil and compost, it is not surprising the trees are struggling.
Poor or slow drainage, lack of proper aeration and potentially overwet soils inhibit healthy root development and that can lead to reduced growth and stunting. In severe cases, it can generate root rots which lead to plant death.
Repot with a high quality bespoke container potting soil mix that is of adequate durability and sufficiently large particle size and you should see marked improvement.
Q