The double-drift stimulus (a drifting Gabor with orthogonal internal motion) generates a large discrepancy between its physical and perceived path. Surprisingly, saccades directed to the double-drift stimulus land along the physical, and not perceived, path (Lisi & Cavanagh, 2015). Here we asked whether memory-guided saccades exhibited the same dissociation from perception. Participants were asked to keep their gaze centered on a fixation dot while the double-drift stimulus moved back and forth on a linear path in the periphery. The offset of the fixation was the go-signal to make a saccade to the target. In the visually-guided saccade condition, the Gabor kept moving on its trajectory after the go-signal but was removed once the saccade began. In the memory conditions, the Gabor disappeared before or at the same time as the go-signal (0 to 1000 ms delay) and participants made a saccade to its remembered location. The results showed that visually-guided saccades again targeted the physical rather than the perceived location. However, memory saccades, even with 0 ms delay, had landing positions shifted toward the perceived location. Our result shows that memory- and visually-guided saccades are based on different spatial information.