Above all else, I'd start with plans for some well sited and well chosen trees for shade and additional privacy. I know nothing about plants suited for California so I can't suggest anything.

Regarding the 3D plan, is there a reason you're placing the seating and dining areas away from the sliding door that I assume is the access from house to yard? You indicated the left/south side of the house as the entry to the yard; will guests enter there rather than through the house?

Here are a couple of options I played with (differing only in pool placement), but they put the dining and seating areas near the slider, which might not appeal to you. They include grass for the kids to run from one side of the yard to the other, the playset and pool as well as a play tent, and a separate-feeling place to explore (the hideaway spot near the north side entrance with a shady bench under a tree). The two trees near the patios (citrus?) and the trees along the back fence could be fruit trees (consult a local supplier for low-chill-requirement varieties for your area), and the ones in the raised bed could be underplanted with strawberries. Grapes and/or kiwi could grow on the overhead structures (and the trellis if built strong enough). I only heard fruit and privacy as wishes regarding plants, but shrubs/flowers could be added too, if you wish. Some thoughts, anyway . . .

Edited photos:

And a plan view of the second option (pool away from patios):

The shady bench (imagine shrubs/flowers around it; didn't add them because the SketchUp model was already too big!):

Thanks so much simplynatural! I really appreciate the sketch up designs. The reason for having the patio away from the house is because we don’t want the patio gazebo/pergola to darken the interior of the house. Our previous house had an attached gazebo and my dinning room was always dark.
Regarding the guests entry, per experience we’ve had friends coming in to the backyard directly (instead of going through the house) many many times. I do love other types of plants and love your placement of planters. That’s even the style I prefer, linear and clean.
If you have a pergola, what are your thoughts on adding a liner? My husband is considering this option but I think it will trap debris and dirt and need periodical cleaning

Thanks Dig Doug! The additional vegetation improves the final concept tremendously. The lounge chairs are a great idea to rest while the kids play.

I have a rectangular backyard too. When I first moved into my new house, I designed my front yard and drew it out for the builder to build my stepping stones, front entry porch, side hedges, dwarf woodboxes, weeping cherry, shape of my front lawn, even the lamp post. They were accommodating but left the backyard, and I didn't even know what to put in my backyard at the time. Forward years later and lots of dollars later with experimenting different plants and stuff, I finally had evergreen hedges along the long backside of my fence. I didn't do that privacy hedges at the beginning cause the area behind my fence, the other side was vacant, at the time I thought they could never build on it, cause during that time I believed the land didn't have the square feet for it, and it was supposed to be conserve land. Anyways, that was changed, so I put up evergreen hedges along the whole backside. To cut down the amount of grass space, I have a little fire pit surrounded by pavers connected to my backyard patio. Along the right side fence I have flowering shrubs and hostas and flowering perennials, with a little water fountain. My patio is a 10 x 12 typical norm, since the rectangular yard is so wide, I had huge pavers connecting the front of my paved walkway, all the way to the back, connecting to backyard patio, with distance out as far as my patio. It's taken me years but I think I know what my next steps are to add on to the aesthetic of the yard. I have cut down the grass area size tremendously as my kids are grown now. I also have a shed on the left hand side and paved walkway leading up to the shed with round hedges both sides framing the walkway. I like low maintenance hands off plants cause I'm in zone 4. My evergreen plants in a container in my front yard even dies from heat in the summer. I think you should build the bare bone structure, dividing up the space how you want it, and add on to it as you go. .