So if statues of Robert E. Lee must come down even though Lee owned no slaves - but merely because Lee fought for a particular cause - then what about Ulysses S. Grant? Grant participated in the Mexican War, the first of this Republic's wars of imperial acquisition.*
I mean, Grant himself thought the Mexican war "wicked" but went because he felt it was his duty. Lee thought slavery wicked but thought it was his duty to wear the grey uniform.
So help me understand - why do one set of statues come down but not the other?
* The United States took quite a large amount of territory from Mexico after that war.