to say it's "pulled in a direction where the big mass no longer is" seems incorrect, because above we have "whenever a gravitating object moves inertially, the gravitational acceleration vector at a point removed actually points at where the object actually is at a given instant", due to the cancellaction and abberation effects.

"Indeed, the vector (2.5) does not point toward the “instantaneous” position of the source, but only toward its position extrapolated from this retarded data" (which is pretty much the same location)