For the climber, have you considered Bougainvillea? I realize they're very common in your area and you might be wanting something a little different, but my guess is that it would do well there, and they are gorgeous plants! Alternatively, Wisteria might work well, but I'd go for the American kind unless you want your fence and shed demolished over the next few years.
For the main part, how about hibiscus? Or pampas grass and some contrasting lower-growing grasses, maybe?

Hi Moleman,
Thank you for your suggestion.
I was also thinking of bougainvillea for the fence (the neighbor has it against his fence too :)). If I pick a red color flowers for the bougainvillea, what would be some nice contrasting plants/shrubs/ornamental grass to go with it?

How about an orange marmalade bush/Streptosolem jamesonii to go with a Barbara Karst or San Diego Red Bougainvillea. A big mass of Phlomis fruticoas could also give a lot of color there over a long season. I could also see adding some Aloe cameronii, Aloe striata or Aloe buhrii for seasonal color and extremely easy care.

What was it you didn't like about the plants you removed?

If you like the red form bougainvillea and it's already on the fence, just tell your neighbor to let it grow over to your side. Put some hooks in the fence, and tie a branch or two across your fence; in a couple of years you'll have all the bougainvillea you could wish for. The red forms can get quite rampant, no need for two! In front, plant agave attenuata, a spineless agave in a soft grey-green color which will make large rosettes (36") and is a nice combo with the red bougainvillea.

If you want something smaller, check out Senecia confusus. It has red-orange flowers with yellow, blooms continuously, and won't eat your fence. To set the whole area on fire, plant in front with canna tropicanna (vividly striped leaves, orange flowers) and more of your gazanias. If you're not a canna fan, try a large-leaved colocasia, or a dwarf form of cut-leaf philodendron, such as 'Xanadu' for a tropical effect.

Or you could repeat the vine on your garage trellis -- or move it to the more roomy home on your fence. Plant some mexican sage in front of it, salvia leucantha.

You have a gazillion choices, maybe you could describe a look you trying to achieve?

Hi everyone,

Thank you for all the suggestion! I appreciate them very much. What we finally decided are bougainvillea in the back (Barrakast red), some fortnight lilies to cover its feet (green and white flowers), then some heavenly bamboo for folliage and color(red, yellow and berries), and the front of the patch is covered with day lilies (yellow).
M