FATCA forces non-US financial institutions to follow US laws in two ways. Firstly, they are busy signing agreements with different governments that will mean financial institutions are forced to provide customer data to the IRS. These are called "inter-governmental agreements", and yes France is signing one:
http://www.stepjournal.org/journal_archive/2012/step_journal_june_2012/fatca_in_the_frame.aspx
The second way is far nastier and more insidious. They are planning to impose a recursive "passthru" 30% withholding tax. That means if you do business with the USA you must follow US law and if you don't, your US assets will be taxed at 30% (essentially bankrupting you or making you uncompetitive). But any business that comes into line because of that must also impose the 30% tax on any other financial institution they deal with, even if that institution does not interact with the US in any way shape or form! Because the financial system is a fully connected graph, this essentially means that US law spreads through it like an infection and eventually every financial institution will be forced to comply or face crippling sanctions from other financial institutions they interact with. This happens even in cases where US law conflicts with local law.
Dave, as you're the one exposed to this stuff, please make super sure you have fully researched this topic. The US will force you to follow its rules whether you like it or not and the French government will not save you - this is not a paranoid hypothetical but a phenomenon that is already happening. Your best bet is to do what all other financial institutions are doing and deny access to "US persons" (citizens, residents and people who have held a green card in the last 10 years, iirc). This doesn't exempt you from FATCA but it minimizes the amount of paperwork and reporting to the IRS you'll have to do.
I would just do as Mike says and not get yourself into more trouble that needs be here in the beginning. You will grow, just dont grow too fast to follow the growth and for some weird reason see yourself in a courtroom in New York in 5 years for money laundry. All the best!