The best way I can explain it is with an example.
Notice how none of those shares are actually difficulty 16.
Even if you are sending more shares, you won't actually make anything until you get a share that is equal to or higher then P2Pool's difficulty. That share in the middle over a thousand was one I think.
You aren't loosing anything though, don't misunderstand.
Lets say it took you an hour, but you finally got a P2Pool share. It wouldn't have mattered if your difficulty was 16, or set by the pool, because you still had a share worth enough to be a P2Pool share. The setting basically tells cgminer "Don't send any shares lower then this difficulty".
It's hard to explain, if you read up on the bitcoin wiki about how shares and pools work you may understand better. The point is, wait until you start receiving money in your wallet before you stop mining.