The fort played a crucial role in the Battle of Williamsburg, which was fought on May 5, 1862, during the American Civil War. The battle was part of General George B. McClellan's Peninsula Campaign and was fought between Union forces commanded by Major General Joseph Hooker and Confederate forces commanded by General James Longstreet. The battle lasted for several hours, and the Union forces eventually withdrew, giving the Confederates a strategic victory.

During the American Revolutionary War, the Rosegill estate was occupied by British forces as they sought to gain control over the colonies. The estate's strategic location, near the main road between Yorktown and Williamsburg, made it an ideal location for a fortification. The British constructed a redoubt at Rosegill, which they named Fort Magruder, in honor of their commander, General John Magruder.