L'Enfant, ultimately dismissed for insubordination in 1792, left the layout of the federal city complete on paper, but it is generally believed that there were no building plans for the proposed structures, only lines of demarcation that indicated the buildings on their sites. Washington, acting on the suggestion of his secretary of state, Thomas Jefferson, announced a national competition for plans for the design of the Capitol and President's House (as it was then typically called). While the competition was being advertised, Washington contacted James Hoban, a builder he had met in Charleston, South Carolina, and consulted with him about the type of design Washington envisioned for the presidential residence. Hoban, an Irish American schooled in monumental building arts in Dublin, had followed his trade in Philadelphia and Charleston. He ultimately won the competition as Washington's man. Other entries were for structures that suggested palaces. Hoban's entry shows how Washington's ideas had changed in only a few years in favor of a building less palatial, yet by American standards still very grand.
Hoban almost certainly held up models before his presidential patron. The chosen design was based on a house that had been familiar to Hoban in Dublin, the palace of the Duke of Leinster. It was about fifty years old at the time, a broad stone mansion rising two stories above a rustic base. More the house of a squire than a lord, the image appealed to Washington. The commissioners charged with overseeing construction of the city attempted to modify the plan by changing the material to brick; Washington disallowed this. When they respectfully protested the scale, Washington omitted the raised rustic base story and increased the volume of the house by twenty percent. He insisted that the stonework be elaborately rendered in the grand Anglo-Palladian manner.
The commissioners, unpaid government appointees, were left with the responsibility of getting so ambitious a house built. A sandstone quarry was purchased downriver on the Potomac at Aquia Creek. Stonemasons were engaged in Scotland, a party from the Highlands and one from Edinburgh. Both were expert; both had worked to plans of the celebrated Scottish architects Robert and James Adam. They put their best efforts into the President's House, which must have seemed very out of style to them, having recently worked on the restrained neoclassical buildings of New Town in Edinburgh.
By 1798 the building stood much as we know it today. Washington never lived there. He set the stakes for it, establishing the north wall, in cellars already dug out for L'Enfant's "palace." In moving the building north, Washington took it out of view in the axis down Pennsylvania Avenue to the Capitol, a vista filled by the Treasury portico today. He violated the eighty-two acre preserve, called from the outset "the President's Park," by mandating that the government offices be built there, flanking the executive mansion. Thus Washington started the pattern of change with respect to the White House that would be followed by presidents for two centuries to come.