As you rightly noted, the restrictions introduced in the pandemic have caused devastating damage to the economy and people's lives, I think much more than the pandemic itself. And this also does not help citizens trust the authorities and rush to use hastily made vaccines.
There is no "massive" refusal. Even here in India, you won't find more than 5% or 10% of the population who hold that view. For the third world nations, the major issue is with the unavailability of the vaccine and not the refusal from certain sections to get vaccinated. But for the government, this 5%-10% figure is a very big number. Because it hinders the required coverage to attain herd immunity. And I don't think that there was a better option other than imposing the restrictions. Countries such as Brazil and Sweden (which refused to impose restrictions) paid a massive price in terms of loss of lives.