Yes, yes, I know that that is being said, but the logical error in this reasoning is that what you put forward as a desired goal of a system is taken as a logical technical consequence of said system. This leaves the logical possibility that the system is not doing what was the desired goal, aside.
The technical fact is that if there is only one block chain out there, made by miners who agree mutually on a given protocol, and hence, don't produce a competing chain, but only build upon each-other's blocks, then other nodes can only decide to download, or not download that chain, but that's all there is around.
This is like as if there was a single web server, making HTML pages with a certain protocol, and you deciding that your browser is going to read pages that satisfy another protocol. Ok, you can point your browser to that server, and your browser will refuse to load the page. And you get a blanc page. Because there's nothing else your browser can download that could satisfy him.
A non-mining node can download a chain, and verify whether it agrees to the protocol rules on which it is built ; and if it is not satisfied, it stops loading that chain. But there's no other chain around. So it just gives the user a "blank page" as wallet.
The kind of reasoning is like "as our technology has as an aim that this rocket flies to the moon, and as our rocket is using propellers, propellers can fly rockets to the moon". We've left out the logical possibility that our rocket will not fly as designed. This is the logical equivalent of "as we want full nodes to be in control of the protocol, they are in control of the protocol". However, as gmaxwell pointed out, the protocol is not what we write on paper, but what *actually happens in nature*. And what *actually happens in nature* is about 20 mining pools making a unique chain, and nobody else does. So that chain is what defines the protocol, and hence, whatever software those 20 pools use, is what defines the protocol, because that's the only one really out there in nature.
Only miners can cause a fork. A non-mining full node can accept the only chain in town, or not accept it and stop.