If you take that into consideration, then it's clear that smarter utilization of block space is what lead to lower fees.
Indeed, this is a really important aspect of the fee market that people often don't account for. It's one of the reasons why limiting block size is really important for scalability.
Fee pressure naturally causes services and users to make their use of transactions more efficient. This not only frees up block space and lowers fees as transaction batching becomes the norm and Segwit (and later Schnorr) adoption increases, but it's a way of organically encouraging a more scalable blockchain. If fees were perpetually cheap, then users would have no incentive to optimize for scalability.