That being said, I don't really care if historical figures like Jefferson owned slaves. That's just the times they lived in. We should be able to take our history warts and all, just like if a great writer was a murderer or a rapist that wouldn't make his/her books any less valuable. Same with a great scientist. People are not either good or bad, they're more complicated than that.
This is quite true. Statues are complicated as well, in that they provide that direct link between the past and the present. Jefferson owing slaves in the time in which he lived is one thing. We might argue that were Jefferson alive today he would be highly unlikely to be in favour of slavery. What about a statue of Jefferson that is still standing today, as an object suitable for pride and veneration? Is a statue of Jefferson a monument to his achievements, or to the totality of his existence?
Let's assume for simplicity that Jefferson's only achievement was the Declaration of Independence, and that the only other thing we know about him is that he was a slave owner. The creation of a statue of the man would then be a celebration of the author of the Declaration - it would not be a celebration of him as a slave owner, there were plenty of slave owners, he was nothing special or statue-worthy in that regard. Then let's assume we erect a second statue, not depicting the human, but rather the Declaration itself. Presumably that second statue can stand for all time, or at least until we find the Declaration offensive. The difference is that the statue of the man has the potential to be construed as celebrating the man rather than his achievements. If it is a statue of the man alone, then should it to be removed because of his links to slavery, which we now find offensive? What about if the statue was of him holding the Declaration? Does that specification of context work to remove objections?
Tearing down statues of men who became rich and famous solely on the back of the slave trade is one thing, and is eminently justifiable; the issue of statues of figures such as Jefferson becomes clouded with nuance. Is his ownership of slaves incidental or is it sufficiently important to outweigh his achievements? What about
the Act Prohibiting the Importation of Slaves? What about establishing a nation and freeing it from the yoke of British tyranny? What about treatment of indigenous Americans?