From that picture it looks like what attacks my lemons, Queensland fruit fly. It's been devastating this year. Other than covering the tree with a net, I don't think there's much you can do about it. From the look of leaves, I'd suggest the tree also needs a good prune back (according to Peter Cundell cut back by about a third and open the tree up a bit to let more air circulate) and add a bit of garden lime
along with some blood and bone meal.
B B.

Thank you I will try all that!

I find it very strange if they r fruit flies because my neighbours lemon tree is only 2m away and has beautiful yellow fruits! If my tree had fruit flies wouldn't they jump and ruin their tree too?!

As I said, that's what it looks like, whether it actually is might depend on where you are growing the tree and/or if you spot any fruit fly in and about the vicinity. My neighbour, too, has a lemon - Meyer lemon, the same as mine - and oddly enough, his tree is much less affected than mine. Perhaps it might be an idea to take one or two of the fruit to your local nursery and seek their advice. Also if the tree is growing in, say, Victoria, or any cool temperate region, it's likely not to be fruit fly. By the way, where is your second picture?