Naah. Mine are all poly. Just when I did the new car, I left the rubber ones in the trailing links. Don't really notice much, if any, difference between the two.
Hitting a pothole is more "sudden" and imparts more front-to-rear shock to the chassis than acceleration forces - even strong ones. Since the trailing link only locates the wheel front-to-rear, the alignment will hardly be affected at all.