Education, or lack thereof, is the prime factor in my opinion. Everything else is beating around the bush in comparison.
No one is even teaching economics in school, so unless someone studies it in college or on their own, they won't know even the basics of economics. Add the fact that the current mainstream school of thought is all about fiat money, and it's easy to see why Bitcoin isn't getting more traction. People see it as a risk and a liability, and unless they are attracted to the "get rich quick" side of it, they have little reasons to adopt Bitcoin.