Seems to be too much water and not enough night.

If you're in the northern hemisphere, right up in front of a south facing window is best.

Water when the soil feels dry at the drain hole.

dont water in sips, thoroughly flush the soil when you water. Never let water sit in the saucer.

What do you mean by you added more drainage?

Thanks Dave. We are in Southern California. Both our Southern windows are sliding glass doors and there is no space in front of them that wouldn't be in the way.

The pot had 3 very small drainage holes, we made them bigger using a drill.

North facing Windows usually aren't a good choice for ficus.

Direct sun is sunlight that is outdoors.

Even sun shining through a window isn't direct, it would be filtered sun.

Ficus like as much light as you can give them. I personally grow all of mine outside in direct sun all summer.

That said, you can keep it where it is, but it probably won't be getting enough light.

a window that recieves full sunlight all day will be your best bet. Ambient outdoor light through a window just isn't much.

Ok I will move again for sure then. Thank you!

No it is not in the original growers pot. I replanted into a slightly larger one. The drainage was blocked it's much better now and the roots look good I did trim a few. I was just reading about the grit. Should I move and repot again? I feel like I now may be fussing with it too much but also don't want it to continue to decline.