The problem is that, without these nuclear power plants Japan will be facing blackouts. There are no significant coal or petroleum deposits in Japan, and they are forced to import these commodities from other nations (coal from Australia and Indonesia, Petroleum from GCC, and LNG from Qatar). This makes electricity generated from thermal power-plants very expensive, when compared to that from the nuclear power plants.
A few years back, there was a proposal for constructing a gas pipeline from Sakhalin (Russia) to Hokkaido (Japan), which would have delivered cheaper pipeline natural gas. But the proposal was rejected by the Japanese government under pressure from the United States.