If you press Stop once, it actually pauses playback (pressing Play now will resume playback from where it was paused). Pressing Stop a second time will stop playback. When playback is stopped, pressing Play will now start playing from the beginning of the track again.
So to stop and re-start a song, you need to press Stop twice and then press Play.