On a hot night in August 1841, fugitive slave Frederick Douglass stood before a thousand white people inside a rickety wooden building in Nantucket, Mass. A handful of Black people appeared in the crowd, but the group looked like a sea of white to Douglass. “Accustomed to consider white men as my bitterest enemies,” he later recalled, he trembled as he prepared to address them.
Not three years had passed since he had escaped from enslavement in Maryland. “There is no spot on the vast domains over which waves the star-spangled banner where the slave is secure,” Douglass would later explain. “Go east, go west, go north, go south, he is still exposed to the blood hounds that may be let loose against him.” No fugitive slave was safe in the United States—not even at an abolitionist convention. And yet Douglass felt he had no choice about speaking up. He was already part of the movement that ran on words. His host, Quaker William Coffin, had brought him to this meeting of the Massachusetts Anti-Slavery Society after hearing Douglass speak at a Black church in New Bedford. “Tell your story, Frederick,” Coffin urged Douglass now, as the abolitionists waited. The strikingly handsome man—strong chin, chiseled mouth, and wide-set eyes—usually dressed in a waistcoat, formal jacket, and high-collared white shirt, rose reluctantly to his feet. Douglass never could remember what it was he said that evening.
[time-brightcove not-tgx=”true”]
Although journalists from various antislavery publications attended the conference, there is no record of Douglass’s remarks. But the reporters in the room agreed on one thing: Douglass brought down the house. “Flinty hearts were pierced,” Lydia Maria China reported for the abolitionist newspaper National Anti-Slavery Standard, “and cold ones melted by his eloquence. Our best pleaders for the slave held their breath for fear of interrupting him.”
As soon as the meeting ended, John A. Collins, theology school dropout and then-vice president and general agent for abolitionist William Lloyd Garrison’s Massachusetts Anti-Slavery Society, invited the newcomer to become an antislavery agent. Douglass would travel and tell his story. Although she was not present at Douglass’s maiden appearance on Nantucket, Maria Weston Chapman, the beautiful, wealthy Bostonian from a prominent abolitionist family, would manage much of Douglass’s new career. Weston Chapman, nicknamed “the Contessa” by one of her admirers, was perhaps Garrison’s closest comrade. Since the gorgeously dressed socialite had shocked people in the modest abolition world by walking into a meeting in 1834, her fashionable Boston town house had become the beating heart of the Society that fueled the movement.
Read more: We Must Hear the Warning in Frederick Douglass’ ‘Sources of Danger to the Republic’ Today
At the center of the meeting on that remote Massachusetts island in August 1841—a kind of snapshot of the movement for the abolition of slavery—was the fugitive, with his indelible story of life in the slave South, the inexcusable wrongdoing at the heart of the American republic. The historian Manisha Sinha would later call these stories “the movement literature of abolition.” Also in the picture was the audience of white northerners, who had been gathering for over a decade to argue for the immediate, unambiguous abolition of slavery. And finally, here and there, an embarrassingly few, but crucial, Black abolitionists, who had formed the backbone of the movement from the beginning. The action centered on Douglass’s “heart-piercing” speech, reflecting the outsized power of rhetoric for the abolition movement. The scene opened up the possibility of an alliance that was, at its very apex, interracial and sex-integrated—the first such major public movement in the history of the nation. For 12 years this alliance worked to change the nation. But, as with all alliances, sooner or later the question would arise: Who gets what from the deal? By 1853, the partnership of Garrison, Douglass, and Weston Chapman was done.
The strains on the interracial aspect of the enterprise of Douglass and the mostly white New England abolitionists were visible already in Nantucket. Later, when Douglass later became the most popular and renowned speaker in a movement that lived on words, his appearance that night in 1841 became a legend. At the time, however, Garrison took only passing notice of the slave’s debut in his abolition newspaper The Liberator, not even giving Douglass the dignity of using his proper name. “Messrs. Bradburn, Collins, Quincy, Pillsbury, Whiting, and other speakers were present, (among them several talented colored young men from New-Bedford, one of them formerly a slave),” Garrison reported, “whose addresses were listened to by large and attentive audiences with deep interest.” Nevertheless, in the summer of 1841, Douglass joined Garrison and Weston Chapman in the anti-government Massachusetts Anti-Slavery Society faction. Their relationship raises all the questions of whether an alliance across race, sex, and class can survive.
The answer is unsurprisingly yes. And no. The alliance fueled a crucial decade for the rise of the movement. Yet there were strains from the beginning. Douglass, almost immediately a rising star, would not tolerate the patronizing and racist treatment of Weston Chapman and some of the other Boston abolitionists. Two short years later, he got into a faceoff with the tour manager, who was using abolition to advance other causes. When Weston Chapman scolded Douglass for the fracas and threatened to dock his pay. Douglass warned Weston Chapman and his other managers: “I trust I have as far as one can have, a just sense of their claims to my gratitude and respect.”
Get your history fix in one place: sign up for the weekly TIME History newsletter
Four years later, when he found out Weston Chapman was secretly directing his hosts in Ireland to manage him so he wouldn’t try to capitalize on his fame, or, worse, assist a competing British faction, he wrote more plainly “If you wish to drive me from the [Massachusetts] Anti-slavery Society, put me under overseership and the work is done.” As it turned out, the breakup went more slowly than Douglass anticipated. First he moved to Rochester, New York and began working with the New York faction. Then, in 1851, he acknowledged openly that he had adopted their turn to politics as the way to end slavery.
When Garrison discovered that Douglass had gone over to the political New York faction of abolitionists, he shamefully retailed gossip about Douglass’ personal life, and the breach was sealed. The two never really spoke again.
But the story has a happy ending. The breakup, sending Douglass to the New York abolitionists, more driven to political engagement than the anti-government Bostonian branch, perversely led to triumph—for the abolitionist and the movement. Twelve years after the Garrisonians drove Douglass away, Douglass went to the White House, for the inauguration of the author of Emancipation, elected by the political rise of abolition.
On March 4, 1865, Abraham Lincoln was inaugurated for the second time. Douglass, who was by now the most influential Black man in America, had delivered the strategic and political wartime support requested of him by the president.
In a symbol of how the world had changed, abolition lawyer and politician Salmon P. Chase had replaced the author of Dred Scott v. Sandford, Roger Taney, on the Supreme Court. Douglass had met Chase years before on the abolition-lecture circuit. And so Douglass, a Black man who Taney said had no rights that white America needed to respect, went to Chase’s chambers to help him try on the robe he would wear to swear in Lincoln. Then he went to the Capitol to watch the ceremony.
Read more: This Little-Known Abolitionist Dared to Speak in Public Against Slavery
After the ceremony, Douglass determined to “present himself” at the White House reception, “though no colored person had ever ventured” to do so. After all, he thought, freedom had become the law of the land, and “colored men were on the battle-field mingling their blood with that of white men in one common effort to save the country.” No one else “of his own color” would go. They were unwilling to risk the unhappiness of rejection. As usual Douglass would have to lead the way. As usual too, he held up his decision to his lifelong standard: Did going to the White House for the inaugural reception fit his self-image as a man? “I had for some time looked upon myself as a man,” he reminisced later, “but now in this multitude of the elite of the land, I felt myself a man among men.”
Two policemen at the door roughly turned him away. Someone recognized him and took the word to Lincoln. Next thing Douglass knew, he was standing in the East Room with the president. Lincoln wanted to know what the other great orator thought about his speech. When Douglass demurred at taking so much of the president’s time, Lincoln answered: “There is no man in the country whose opinion I value more than yours. I want to know what you think of it?”
Douglass lived another quarter-century after the abolitionists dissolved their society in 1870. With Lincoln dead, the work of undoing his great work of emancipation commenced. The fragile hope of a Black and white movement—embodied for a time in the alliance between Douglass and the Boston abolitionists, and then in the friendship of Douglass and Lincoln—would have to wait. Nevertheless, Douglass persisted. When he died, on Feb. 20, 1895, he was in the middle of his campaign against lynching. From the time he stood up in Nantucket until the last day of his life he never stopped fighting the poison of American slavery.