One of the most grievous aspects of slavery was that it stripped human beings of their own stories. Regular records weren’t kept on enslaved individuals’ births, deaths, or marriages with nearly the regularity as those of white Americans, and the discouragement or prevention of literacy among slaves (in many cases, it was even illegal to teach a slave to read) meant that far fewer letters or other written documents remain today to give us insight.

Paul Jennings, an enslaved African American who served the Madison family both at Montpelier and in Washington, D.C., made the incredible journey from slavery to freedom to memoirist. His brief volume, entitled A Colored Man's Reminiscences of James Madison, is considered the first memoir about life at the White House. It’s also a rich firsthand account of the relationship between slave and slaveholder—even more valuable for its insight into a system that was at odds with its perpetrators’ values.

Jennings was born in 1799 at Montpelier to an enslaved woman about whom we know very little except that she was granddaughter of a Native American. His father was Benjamin Jennings, a white British merchant, but, as was the rule of the day, Jennings’s mother’s status kept him firmly under the yoke of slavery.

As a household slave, his fate was better than many–Paul Jennings’s duties were confined to the domestic. After 1817, he likely lived in Montpelier’s South Yard, close enough to the house to be available to the family at a moment’s notice. 

Jennings would become Madison’s personal manservant, learn to read and write, achieve some degree of autonomy, and find himself in a unique position to observe the goings-on of influential statesmen. At age 10, he accompanied the Madisons to Washington after James was elected president. He later described the nation’s new capital as “a dreary place.” At the time, Pennsylvania Avenue was, depending on the weather, a mess of mud or dust, and the East Room of the President’s House was unfinished. Indeed, the whole city of Washington was nowhere near as established as Philadelphia, Boston, or New York, and it must have felt like chaos to one used to an orderly plantation life. 

Over the course of Madison’s presidency, Jennings would see Dolley Madison establish a sense of order, a precedence of graceful hospitality, and a signature presidential style that still characterizes the White House to this day. As young Jennings went about his duties—laying the table, placing wine and ale in the coolers for formal dinners—he was helping to set the stage where political history would unfold. 

Reminiscences primarily gives us Paul Jennings’s perspective of the Madisons during the War of 1812. It also includes a few paragraphs of character studies of both James and Dolley, a poignant description of James Madison’s death, and some insight into Dolley’s widowhood. Jennings depicts himself as relatively independent in his comings and goings. On describing the night of Dolley Madison’s flight from the White House: "About sundown I walked over to the Georgetown ferry, and found the President and all hands...waiting for the boat.” James Madison comes in and out of that evening’s narrative ("Just then we came up with Mr. Madison and his friends, who had been wandering about for some hours, consulting what to do”), but Jennings seems to choose his own path, walking “on to a Methodist minister’s,” where he would witness the burning of the Navy Yard from afar.

The Madisons took Jennings back to Montpelier from Washington at the age of 18, and he would serve as Madison’s manservant until the former’s death in 1836. We know from Dolley’s correspondence that Jennings was much praised for his skills when waiting at table—we even know that Dolley’s friends and acquaintances requested to borrow him from time to time. In 1844, after Madison had died and Jennings and Dolley had returned to Washington, Dolley allowed him to return to Virginia to be with his dying wife, Fanny Gordon Jennings. A year later, he was allowed another family visit, although Dolley wrote to her son peevishly at Jennings’s delay in returning. 

Paul and Fanny Jennings had five children—Felix, Frances, John, Franklin, and William. The latter three would go on to fight on the side of the Union in the Civil War. His second wife, Desdemona Brooks, died as well, and Jennings married Amelia Dorsey in 1870, spending his final four years with her.

Though he chooses to speak only very highly of both James and Dolley in his 1863 memoir, it’s probable from other records that Jennings supported an 1848 mass slave escape attempt on the schooner Pearl. Two years before, he had negotiated his own freedom—not, as he had hoped, via Dolley Madison directly, but with statesman Daniel Webster. Dolley, who apparently considered freeing Jennings or selling him to her son Payne Todd, finally sold Jennings for $200 to an insurance agent named Pollard Webb. Webster then bought Jennings for $120, allowing him to purchase his own freedom at a rate of $8 per month.

Paul Jennings firmly established himself and his family in Washington’s free black community, which was at that time three times as large as its enslaved community. He began working at the Pension Office as a “laborer” (a term that encompassed many clerk-like duties) in 1853, eventually bought property on L Street, and lived there with his third wife, Amelia Dorsey, with his daughter and grandchildren next door. During his tenure as free man, he would occasionally visit the now-impoverished Dolley Madison and even provide “small sums of money from [his] own pocket” if he found her wanting. 

Throughout his remarkable life, Paul Jennings witnessed firsthand the paradox that’s so startling for today’s Americans. How could one of our greatest advocates of liberty take such an active part in the institution of slavery? And why would a once-enslaved human being act so loyally to his former mistress while at the same time helping slaves escape?

In 1874, at age 75, Paul Jennings departed this world, leaving behind him property in Washington, an increasing number of descendants, and a story that would become legend in the decades and even centuries to come.