If you live in New York as suggested in your name, you might want to search on cliffandjoann. They built a stunningly beautiful pond and have explained the details. They say that some of their plants are in pots. I use impatiens and am under the impression that they will grow without much actual soil. It is certainly true for water cress (from the grocery store) anchored directly in flowing water. For something larger, sweet flag will certainly work. You can also use vines or other plants growing beside or near the area. You can also add logs or other natural materials for interest and coverage.

in theory it would be bushy, mid size, clumping perenneials like Alchemilla mollis or Geranium macrorrhizum... My dad used to have those kind of plants along the lawn i our tiny garden when I was a kid, seemed to work fast forward several decades, I keep trying this approach and regularly spend large amount of time weeding out runner-grasses and weeds out of my perennials whilest bit by bit adding some kind of edging stones (I guess, my late dad did some regular, low key, permanent weeding around the edges)

Yep, anagallis arvensis, an annual cornfield weed. I would not be in a massive rush to pull it out unless you had something else to fill the space. They are easy to remove and, as weeds go, fairly innocuous. Its blue cousin, anagallis monelli is an underused little charmer (not unlike nemophila, phacelia, platystemon and a host of other little Cali annual).

Stagger prune. Trying to explain simply: Cut half of the long parts growing over the walk about 50% to the bed edge, and the rest back to the bed edge. Roughly alternate the long and short cuts. When the edged-length cuts regrow out to the point where the 50% were originally trimmed, trim only the original longer stems back to the edge. The deeper cut stems will be covered by the longer growth, then the regrowth of the deeper cuts will cover the new to-edge pruned stems. This will prevent that ugly sheared look, with bare stems, that would result from straight hedge pruning. It does not have to be 50%--you can do three different lengths, for example. Nor does it have to be precise in length to the edge. But if you stagger prune whenever you trim it, it should never have that butchered look. The technique works to contain height with a more natural appearance, too.