What is division. Divide 6 by 2 to get 3. It is nothing more than a shortcut to determine how many times to add 2 to itself to get 6. Same for multiplication. If we can add, then we can divide.
I will dig into the concepts of Group, Field, etc. I do solicit suggestions of where to look.
I think you are taking things way too literal, instead of first reading up on what's a group and a field.
You are thinking too much in terms of real numbers, or rather rational numbers, which has nothing to do with
modular arithmetics.
Let me give you another example: the natural numbers set: 1, 2, 3, 4, ... infinity
Now, how exactly would you divide 3 by 7? It is undefined, because no value can ever have an inverse in regards to multiplication.
Similarly, you won't be able to subtract 2 from 6, because there is no such thing as "- 2" to add to 6, since no value has an opposite in regards to addition.
So the point is: Natural numbers are not even a group, even though you may be able to define addition and multiplication. Why? Because a group requires every element to have an inverse in regards to the group's operation definition.
Let's "fix" the issue: take integral numbers: -inf, ... -2, -1, 0, 1, 2, ... inf
Wow! Now we have a group, because we every element has an inverse (- x)
Now we can do 6 - 2, progress!
But we're still stuck at 3 / 7, because not every element has a multiplicative inverse, so there is no element in our group that can be multiplied by 7 so that it ends up as the value 1.
Now, we can "fix" this by taking rational numbers: every pair of a / b such that a and b are integral numbers.
Great, progress, now every fraction a / b that you can think of has an inverse = b / a
Now you can do 3 / 7, and it's in the field because a = 3 and b = 7 so this is the simplest form in which it exists already.
So now we have a field. Unfortunately, this is an infinite field of fractional numbers, which has nothing to do with modular arithmetic, because we took things way too far simply when we introduced negative numbers.
To be even more cruel, we took things too far simply by assuming there is a natural order of 1, 2, 3, 4 ....
A modular group is built on other principles, and the elements are not even sequential, they are just unique in such a way to respect some properties, mainly a prime order such that every element has a guaranteed additive inverse. Similarly, a field will also guarantee a multiplicative inverse. The sequence will usually look random, but it has clear build-up rules, some of them old of 400 years.
Those are "eaten" using number theory concepts, prime numbers theorems, and so on, not with real numbers algebra / arithmetic.
So, next time when you read "group" or "field" remember that they may be defined with totally different operations than "normal" addition or multiplication.
Points on a ECC curve are not "added" by their coordinates, it is an "geometric addition" using intersections between lines and the curve's shape, not by adding x or y together and calling it a day.
If I give you a group of emoji elements, my rules for "addition" might involve complex facial characteristics of the colors and symbols of the two emojis, to end up as the addition result. Same way for ECC points.