Maybe it should be concerning that NSA had an unprecedented presence at Def Con, which also had large Bitcoin-user attendance, and AFAIK, the only USG agency to ask for Gavin was the CIA. Doesn't sound like particularly welcoming behavior.
NSA is perfectly free to attend DEFCON and they've never been discouraged from doing so in the past. One of the tenets of DEFCON is that individuals from all perspectives and positions -- government, big business or otherwise -- should be able to open up a dialogue with the security community. We only benefit from increased speech and participation.
They might be a bit creepy (and people are justified being wary of them) but their strategic interests are aligned with the passions of DEFCON attendees. No need to feel intimidated. DEFCON has been on the government's radar for a long time. Now they're actually treating it like a constructive resource and handing it some legitimacy.
