How does the decentralized masternode payment system work?
Each time a block is generated on the network, there is a voting process that begins which all masternodes take part in. All masternodes begin the process by deterministically figuring out if they're part of the sub-quorum for the next block that needs to be voted on. Sub-quorums within the network are 10 masternodes and randomly pick masternodes for each block (in a deterministic, tamper proof way).
Next if a masternode is selected it goes through the following very complex process to pick the winner. First it will look at every masternode on the network and get the last paid time, according to the masternode list. "LastPaid" according to this process means a node has at least 2 votes for a given block. Notice there are ten votes total, so it's actually possible for up to 5 nodes to be flagged as "Paid" from a given block. This allows the network to quickly solve disagreements about the order in which nodes should get paid, then establish a permanent payment cycle. However, the downside is, it's possible to have a payment skipped when the cycle is getting established.
Second, the masternodes are then added to a list by last paid, then 10% of that list that was paid the furthest in the past is eligible for payment each block. At this point, the masternode with the highest score (deterministically again) is the one that nodes will vote for. This means there's really no order of payment each block, each masternode will simply remain in this winning queue until it randomly gets picked. Currently on average it would take 4.68 days to get into the eligible queue, then you have a 1 in 300 chance of being selected each block. This means on average the nodes should stay eligible for 3/4 a day before finally getting paid, but that could actually take longer if unlucky.
To determine if you were skipped you can do a "masternode winners 2000 | grep XAddr" from a linux console to find every time you were seen in the list. If you see >=2 votes and are not the winner, then you were super unlucky. Otherwise if you're in the list and it's going past 6 days, it would be the later situation, where you're eligible and just temporarily unlucky. There's also a third option, if you issue a "masternode start" from the cold node, you will reset your queue position.
When a miner pays a masternode, it will select the entry with the highest votes from the network. That's also the node that will get enforced for payment. This was quite difficult to write, but has worked great (although is somewhat resource intensive).
I guess I continue to be unlucky... My node @ Xb7qiTRzJJewfby9eXGeAcYMN7RNGgPnBw hasn't been paid in 12 days.
)
