I should have been clearer, sorry. You're talking about a curved path in three dimensions. I'm talking about a straight line in three dimensions, but with curvature applied through an inaccessible fourth dimension.
We can only conceive of three spatial dimensions, so have to proceed by analogy using one dimension fewer. If you travel direct - the shortest distance - between two points P and Q on the surface of a sphere, you are travelling in a straight line, d... but that line curves through three dimensional space. The equivalent straight line in three dimensional space, C, cuts through the surface of the Earth. You can see below that if your straight-line travel 'd' continues (the dotted black line) you end up approaching your starting point from the opposite direction.
This is what I mean - if (disregarding expansion) you travel directly 'up' into the air and then into space from the Earth's north pole, then if you continue in this straight line, eventually you would see the Earth's south pole coming into view above you. Same as if you travelled due East on Earth - eventually you'd arrive back at your starting point, from the West... same thing but one dimension higher.
Imagine the universe as the three dimensional surface of a four dimensional hypersphere. A straight line across a three dimensional volume returns to its starting point from the opposite direction, if that three dimensional volume is the 'surface' of a 4 dimensional hypersphere. We can do the 'd' journey, but we can't do the 'C' journey because our travel within the universe is restricted to the three dimensional 'surface', we can't tunnel through the fourth dimension.
Gravity, yes, can be conceived of as a higher dimension, in the sense that it can be explained not as an attractive force, but as a curvature of spacetime. For simplicity I think we should consider this a separate topic, because it is distinct from the 'straight line across a thing that is one dimension higher' argument, which holds regardless of gravity.