IMO, you must remove all of the existing shrubbery and replace it with things more promising. The existing shrubbery is wounded, downtrodden and has little chance of recovering. Plus, it was not chosen for low maintenance or excitement. Just start over. You stand a much better chance of making something good out of new plants ... planted a little farther from the wall so they have room to grow. A retaining wall is only necessary if you need to fix the grade for functional purposes, which doesn't seem to be the case. Instead, compensate for the falling grade by using a larger shrub at the low end and lesser height shrubs as the grade rises. In the area around the porch and farther left, you could get away with a single type of plant. (Any hedge that is part of a foundation planting should be trimmed level, not parallel to the grade ... so as to make the house appear stable.) Some color are around the entrance could help cheer up the overall scene. You really need to learn the proper way to trim shrubs as it will make an astounding difference. In order to keep the lower foliage, shrubs must be trimmed wider at the bottom then they are at the top. Of course, select varieties that can stand the existing light conditions as well.

The first question..."has the tree outlived its usefulness?" begs another, HOW is the tree useful? Like a giant parasol, it provides protection from strong summer sun. It even provides a little protection from beginning rains. Visually, it adds interest as a sculptural object. And it frames and enhances (or obliterates depending on how it's controlled) the view of the house/property. I cannot see that it has outlived these functions. But I can see that it could be controlled a little differently so it does a better job of framing the view. Here's a little sketch that illustrates my point. I offer it not to convince you, but in order to let you know what to look for as you observe trees in others' front yards. As you travel about during the day, notice the difference in effect between trees that have been limbed up and those with low-hanging limbs. In general, large older trees that are limbed up present a cleaner, more stately appearance. If they have been limbed up properly (with balance and without protruding stubs) the character of the trunk may be much more impressive than one which has the competition of low limbs and foliage. The tree on the left competes and obstructs; the one on the right frames and shelters. Take this suggestion with a grain of salt as it's based solely on the view shown in the next photo. I've marked red Xs to show which limbs I'd probably remove, but this is something that would really depend on what the tree looks like from all angles of view. The goal would be to have the tree be balanced from all views and get the limbs high enough to visually clear the house. Remember that limbs left on the tree will get LOWER in the future as they grow longer and heavier. (It is hardly ideal to "organize" the branching structure of a tree after trees have grown for many years without guidance. It's far preferable to prune frequently along the way. But we're working with what we have.) Keep in mind that limbing up the tree also has the benefit of allowing more light to enter below while still retaining the tree's ability to shade from the high sun. This means that one's ability to grow other landscape plants below it is improved. "Is the tree too big for the house?" Imagine a little log cabin at the base of giant redwoods. Could there not be a more charming picture? In my view there's no problem whatsoever with a tree towering over a house. But they need to work together, not be in competition. The tree should shelter the house, not obliterate it visually. The tree roots should not interfere with the house foundation. One must desire the shade. When those requirements are met, a tree can be welcome. Though big old trees sometimes do fall over in severe storms, the likelihood of this happening is small enough that one can find thousands of danger-causing conditions that can be worried about before it. I just measured the trunk diameter of an oak tree that stands 14' from my house. It's 62". But before I would cut it down (It's benefits to me are tremendous even though it can also be a PITA,) I'd quit driving or riding with others as that's far more dangerous and likely to cause me injury. Keep the new walkway WELL clear of the tree trunk. I think you must pay much more attention to the geometry (shape and layout) of the new path than what is shown in either of the proposed schemes. It is a built object that is an extension of the architecture. It should demonstrate this. Because of the tree, it would almost certainly have a curving component to it. You might need professional assistance here to come up with the right scheme. (Though it MIGHT work out that way, in my mind, it is not a "given" that best location for the porch steps is at the center of the porch. Design is often a balancing act and you'd be considering more than just the porch and the steps.) Regarding "what to plant in front of the house"...the first thing to do here is work out the shape & form of the plant masses before thinking about the species of anything. You have to do this before you can figure out what plants are capable of making these forms. Consider that a very attractive custom built section of decorative fence could be the basis of screening the AC unit. It could be an opportunity to further enhance the architecture of the house and work in unison with plants. For some brainstorming ideas, Google images for: air conditioner screen.

Judging by the (not-sunburned in full sun) archonotophoenix in the picture, I'm going to guess that if you're in the US, you're in coastal Southern California or coastal central California. The fence looks very modern. Is the house in keeping with that look also? If this space were mine to work with, I'd grow a bright red colored bougainvillea on the far right (San Diego Red or Purple Robe would really stand out against the white wall) and train it across the top of the fence. Then I'd use terra cotta pots and grow dramatic agaves like Agave attenuata, Agave blue flame or Agave desmettiana Joe Hoak. If my guess was correct and you are in fact in coastal California, the agave will not need irrigation. The bougainvillea might need water once a month once it's established.

" Nothing should be planted within 6-9 feet of a house foundation." " The tree is too close to the house. It blocks the windows and may be damaging your foundation" I'm not sure where these notions come from - maybe it's just a regional viewpoint - but I think they can be taken with a large grain of salt !! There are no hard and fast rules for this.....just common sense. If your foundation is structurally sound - no cracking, crumbling or settling - then you can plant whatever you want next to it. Plant roots do not damage foundations.....they can only take advantage of those that are compromised already. Ideally, trees close to a structure should be sited so they are at least no closer than one half the distance of the mature canopy spread. And that is more for the tree's benefit than it is for the structure. Smaller shrubs can be planted as close as one likes, provided they too have a enough room to grow without obstruction and are accessible for maintenance to both them and the house. As to the 'tree' in question, it is too far away to be able to determine the type, but based on the maturity of the other plantings, we could assume that it has been there for a while so not overly fast growing. And the perspective of the photo is misleading....it may actually be sited further from the side of the house than it looks. People plant trees in front of windows all the time - they can be a visual privacy screen as well as offering some direct exterior shading - the window it is covering looks like it would bake in the high degree of sunlight showing in that photo. The point of my comments is that for the most part, these are very pesonal landscaping choices, not anything that is dictated by any hard and fast 'rules' either design-wise or horticuturally. To the OP: live in your new house for a few months, see how the landscape affects the interior and make your plans to remove or improve as YOU see fit.