Long before Everett disposed of the tripartite offer, Fillmore faced the question of his reelection. Unfortunately his own Whig party was rapidly disintegrating as a national organization. As early as the election of 1850, it had become evident that the Whig party would pay a heavy price for any compromise settlement. In several key Massachusetts, New York, and Ohio elections, antislavery politicians replaced well-known Unionists, while much of the South moved toward moderation on the sectional issues. Only in Mississippi and South Carolina did key leaders persist in the belief that the Compromise of 1850 was a betrayal of southern interests. Southern Whigs accepted the compromise but warned the North that any infringement of this settlement would terminate in disunion. Thus, the Compromise of 1850 was the only platform on which the Whig party could remain united. Yet so unpopular was the compromise among northern Whigs that no one even remotely associated with the Fugitive Slave Act could win northern support. What was to southern Whigs the final measure of forbearance was for the North totally unacceptable. The mass of southern Whigs nevertheless maintained their party allegiance in the caucus of 1852 and secured a campaign platform affirming the compromise.
Fillmore had long regarded his accession to the presidency sufficient reward for his political endeavors; he had little desire for another term. He remained silent on the matter of his candidacy and engaged in no political maneuvering to assure his nomination in 1852. When Webster announced his candidacy, Fillmore decided to withdraw formally from the race. Yet his friends prevailed on him to sustain his candidacy, at least passively, until by campaign time his public support rendered a withdrawal of his name almost impossible. Shortly before his death in 1852, Clay endorsed Fillmore. The Whig convention opened in Baltimore on 16 June. From the outset it was at an impasse as Fillmore, Mexican War hero General Winfield Scott, and Webster, supported only by New England, divided the vote. Finally, on the fifty-third ballot, Pennsylvania bolted to Scott, permitting him to win the Whig nomination. Fillmore accepted the decision with magnanimity; the party did not. The widespread distrust of Scott among the party faithful left the Whig standard in shreds. No longer was the national Whig party capable of fulfilling the political ambitions of its adherents. Weed admitted gravely, "There may be no political future for us." In the subsequent election Scott carried four states—Massachusetts, Vermont, Tennessee, and Kentucky. The Whig party had entered its last presidential campaign.
Fillmore's conservatism on sectional issues had alienated the northern Whigs, but his reputation for earnestness and integrity remained high among the party moderates. Franklin Pierce, who succeeded Fillmore as president, could not prevent the further sectionalization of American politics. The Kansas-Nebraska Act of 1854, which opened Kansas to slavery expansion, sent northern antislavery forces into open rebellion; most joined the new Republican party. By 1856 the Whig party was dead. Its moderates entered the Know-Nothing party, known in 1856 as the American party. Meeting in Philadelphia in February 1856, the American party endorsed the Kansas-Nebraska Act and nominated Fillmore, then traveling abroad, for the presidency. The northern delegates withdrew, limiting the party largely to conservative southern Unionists. The Republicans now dominated the North. The Democrats, in control of the South, gained enough northern votes to elect James Buchanan. Fillmore carried only one state, Maryland.
Fillmore never ran for public office again. He returned to Buffalo to become that city's leading citizen, devoting himself to educational and charitable affairs. He became the first chancellor of the University of Buffalo and the first president of the Buffalo Historical Society. In 1858 he married Caroline McIntosh, the widow of Ezekiel McIntosh of Albany. (His first wife, Abigail, had died in March 1853 as he was preparing to leave the White House.) Fillmore died on 8 March 1874 and was buried in Forest Lawn Cemetery, Buffalo.