You should use a flexible acrylic or vinyl based grout, around here its often sold in tubes but you may be other options. If you use regular grout - it will not flex and might end up cracking and popping out, especially in a older house where your foundation does a little shifting and settling. Which is the same reason people are telling you ceramic is a bad idea.
As for the subfloor- you can leave whats there and go right over - as long as its level and straight with no gaps or damage, but with 3 layers already being there its not recommended and would void any warranty on the tile.
Ripping it up is a option - but depending on the layers of subfloor - it might not be logical , maybe you should investigate at a edge of the room - probley at the doorway into the room - you might be able to see what type of subfloor is under the tile. If the tile is on plywood and the plywood is laying over your original subfloor - then it should be easy enough to get up the plywood and all vinyl layers. If you take it up plywood and all - you should put down another player of plywood underlayment.
On the other hand if the tile is laid down right on top of the original sub floor- then its probley not something you want to get into removing - its very sticky and time consuming to strip off- a nightmare in my opinion.
The other option is just go over the entire existing floor with plywood underlayment, if your leaving the first layer intact - you can go with a thinner luan type plywood thats slightly cheaper - last i checked it was around $10 per 8'x4' sheet, but then again i live on the east coast and prices can vary greatly.