I like a panel on both sides -- then you're not looking at the ugly side of the fridge, and it really completes the built-in look.

(from my own kitchen renovation)

I probably wouldn't use a panel against the wall in this case. You can google "how to frame a refrigerator against a wall" for pictures of how it will look. (acm, the side of the fridge will not be visible; only the doors will extend past the 25 inch deep wall). Make sure there is enough room to open the door fully.

I may have to rescind my previous opinion as I kind of agree with Kitchen West as far as "looks" are concerned. But the OP's wall is only 25 inches deep so the fridge doors will extend past the wall and the door opening wouldn't be an issue.

Oh, I assumed that the cabinets would be on the corner side, so that the fridge would have clearance. Well, you'll want a filler panel on that side anyway, for door swing, so I'd still do the panel there, to have something to attach your cabinet above to. Look at my photos: the cabinet is normal depth, but has been pulled forward to finish the "built in" feel of the fridge, so it needs good attachements on both sides. If your fridge is genuinely counter depth (not that common, honestly), then maybe you won't have a pull, but if it's even a couple of inches deeper, you might want both panels.

Kitchen West and acm have it covered. Mention the uneven wall to your kd so she can make sure she has the right materials on order to address that.

Be smart and build out both side panels to accommodate a full size fridge. Counter depth and built in are two different fridges.

You will pay $800-1000 for a counter depth and loose size it makes zero sense unless it is a must have for design to work. Build out the sides and upper cabinet.

Is that an electrical socket on the left adjacent wall?

If so, how often will you need to use it?

Here is a visual.

24" deep if using a side panel

2 tall sides; nice if you have the space for visual symmetry

This is another way, especially if you need access to that switch on the left adajcent wall.

And since it's 24" out from wall, the fridge side clearance on left side may decrease, but watch out for door frames,etc.

Btw, fridge will be passing 24" deep wall & will be seen from side view.

I had my side panels cut deeper to avoid the sides of the fridge showing that is depicted in that last picture above. The above-the-fridge cabinet was pulled out to be flush with the side panels. The deeper panels also avoided having the countertop overhang sticking out past the side panels. Not quite sure how that would be addressed with the wall on one side. Having the 'fridge sides stick out past the opening kind of ruins any "built-in" look.

The fridge against an adjacent wall changes things.

A lot of other things to take into account.

For visual comparison to my above post:

Side view : If panels were deeper than wall.

Thanks so much to all who have commented! Super helpful. Just to answer a couple of questions. The counter-depth refrigerator is standard width but the box, allowing approximately one inch for the plug (it is fairly flat) is roughly 241/4 inches deep. The doors do not swing flat out to the side (open to roughly 80 degrees) so we should not run into the wall or panel. I'm sold on installing on both sides. The refrigerator will open to a very open space in the kitchen so I think the handles and doors coming past the wall edge into that area should be OK (a neighbour did something similar in their kitchen and it looks fine). Really appreciate everyone! Thanks so much.

If you want a built in, buy a built in. A counter depth is NOT the same animal and you will have clearance issues when trying to make it behave similarly in a design. You have issues here with the overall design.