are you going with a high contrast grout? If so - the misalignment will stick out more. (and speaking of that - DO NOT do a contrasting grout with that beveled tile - getting the grout consistent would be a NIGHTMARE)
If you are blending, you will be fine and if they are set well, I'd certainly leave them alone. Judge the job in total - as millworkman said - not 3-4" away
Q