1. Coordinating the market price of the currency pair.
2. Safely transferring ownership of the traded currencies.
The first problem can be solved completely within a p2p system using only bitcoins as the currency. See Prediction Markets.
The second problem is tricky because it involves transferring ownership of fiat money, e.g. USD. But, if the market price is known to all via (1), it becomes a lot easier to just meet somebody on the street and swap dollar bills for bitcoins, or do it via #bitcoin-otc. So I think the hard problem here is the prediction market.
I agree. The cash in hand for Bitcoin amount would have to be negotiated up front.
I did find 3 places (In the Wiki) currently doing this:
http://btcnearme.com/
https://ubitex.org/
http://tradebitcoin.com/about