Cite your sources regarding "only 2-3% of southerners owned slaves," please.
I don't necessarily doubt you, I'd just like to read more about that assertion if there's any data to back that up.
http://mapserver.lib.virginia.edu/php/start.php?year=V1860#11
If you divide total number of slaveholders by the entire population, that percentage is technically accurate.
But entire population includes the slaves themselves, which is just a bit misleading.
If you change it to 'free population' it becomes a little larger; but even that isn't fair, as one owner of many slaves can also have many members of their family that don't 'count' as slaveowners.
So if you pull how many families owned slaves by state, you get this (I added the %s):
MISSISSIPPI 63,015 30,943 49.10%
SOUTH CAROLINA 58,642 26,701 45.53%
GEORGIA 109,919 41,084 37.38%
ALABAMA 96,603 33,730 34.92%
FLORIDA 15,090 5,152 34.14%
LOUISIANA 74,725 22,033 29.49%
TEXAS 76,781 21,878 28.49%
NORTH CAROLINA 125,090 34,658 27.71%
VIRGINIA 201,523 52,128 25.87%
TENNESSEE 149,335 36,844 24.67%
KENTUCKY 166,321 38,645 23.24%
ARKANSAS 57,244 11,481 20.06%
MISSOURI 192,073 24,320 12.66%
MARYLAND 110,278 13,783 12.50%
DELAWARE 18,966 587 3.10%
NEBRASKA (territory) 5,931 6 0.10%
KANSAS (territory) 21,912 2 0.01%