I think you misunderstand that consensus still comes from everyone on the network. The reason there are delegates is because specialization and centralization lead to greater efficiency as we can see given that block production goes from 10min with bitcoin mining (dpow) to 10-30sec with dpos. Think of it through the lens of more traditional governing structures. we could have a fully democratic government in the united states but that would be terribly inefficient since it would take way to long to collect the votes of all citizens on any given issue. not to mention not everyone knows enough or cares enough about each issue, so we would never be able to get 100% of ppl to vote any way. These are the problems that plague peercoin. as the dpos whitepaper elucidates:
peercoin has the same centralization as bitcoin due to pow mining pools (dpow) and it has low participation rates in pos mining which causes it to be to inefficient to provide security/consensus to the network. this is the underlying reason for this hybrid model, proof of stake mining essentially constitutes a fully democratic governing structure that requires human participation. just as with any democracy the incentives are not there to have 100% participation. furthermore the incentives are even more skewed in this model because people can make disproportionately more with pow mining than with pow stake mining (i believe at the current rate inflation it is something like 9:1) .
the problem with the representative democracy is that we vote every four years so the representatives are not directly accountable to their constituents. of course if you want to get reelected then you will attempt to act in your constituents best interests, but there lies another problem in the fact that the representatives job is so complicated and misunderstood by people that they do not always know if a representative has fully acted in their best interests.
in delegated proof of stake you vote for your own representative (it is important to understand that this still use transactions as proof of stake - you cast your vote through transactions). if your representative is in the top 100 delegates that means that he is one of the top 100 most trusted nodes on the network. the delegates job is simple, he must produce blocks in a timely manner and include all valid transactions in those blocks. if a delegate does not do these things everyone on the network can easily detect that the representatives is not acting in everyones best interest. the client will automatically adjust your vote setting to another more trustworthy delegate. you do not necessarily have to vote ( make a transaction) to change the delegates status. assuming that this is a healthy network and there are several hundred transactions per block if the delegate does not do his job those transactions will automatically have down votes for this delegate so that he is effectively fired on the spot.
dpos puts together the best aspects of decentralization (collectivism) with the best aspects of centralization (specialization). decentralization for the purpose of decentralization is not a way to organize society or a network. the truth is that an organization is most decentralized when there are few if any barriers to access for all positions that govern it. competition decides whether or not someone holds a position. the delegates term in dpos is limited not by arbitrary term limits but by his own conduct and whether or not that conduct serves that better good.