Brown was born on May 3, 1933, in Barnwell, South Carolina, to 16-year-old Susie (née Behling; 1917–2004) and 21-year-old Joseph Gardner Brown (1912–1993) in a small wooden shack.
On Christmas Day 2006, Brown died at approximately 1:45 a.m. EST (05:45 UTC), at age 73, from congestive heart failure, resulting from complications of pneumonia.