As opposed to what? A register based language? Note that what bitcoin implements is just a bytecode interpreter, not a language. You could chose to implement a more user friendly language that compiles to bitcoin bytecode if you chose to, but the benefits would be marginal at best given that the scripts are very small. It's not like you have to wade through pages and pages of the stuff when talking about your typical script. People have already invented a defacto language when talking about bitcoin scripts (using "OP_CHECKSIG" instead of its bytecode…or {} to denote stuff pushed on the stack).
That bytecode is also a formal language. I'm not talking about performance improvements, so it doesn't matter much if the interpreter interprets bytecodes or strings containing operators names such as "OP_CHECKSIG". Both would be equivalent in what they can do (but the second notably worse for storage).
My point is, why have such a potent language?
Why not just a list of bytecodes that express conditions to be validated?
Separating from the beginning the operator scripts from the data?
You have a list of data and a stack of operators that can refer to that data as variables or use constants for the parameters of the operators.
You wouldn't have flow control or stack words, for example.
I guess the reason is to allow use cases that aren't thought of yet, but new words can be added later, as is the case for p2sh.
Well, kind of answered myself now: a more potent language needs to be extended less times and extending the language is really an issue with a blockchain to agree.