That should be enough unless you are really unlucky and you send a low-fee transaction at a time where we have to wait for 30-40 minutes for the next block, while in the meantime a huge influx of new higher-paying transaction enters the mempools. Not to mention a sudden price surge or drop of value and the congestions they usually bring to the table.
It only works assuming that the both the blocks and the rate of transactions are approximately constant. Most fees estimation takes into the fee rates for the previous blocks and use the data points to build out their own fee estimation. This is by far the most accurate way to eliminate volatility of the block intervals as it takes into account a much larger history. If the block were to be mined 10 seconds before and you choose a 1vMB from tip, it would at the very least take 2 blocks to confirm or perhaps even longer.If you think estimatesmartfees has too conservative of an estimate, then you can change the parameters if you're using Bitcoin Core for a more economical one. While it is by no means perfect, it is a far better estimation than what we can do by looking at the mempool distribution.
