Crailo State Historic Site is a unique little piece of Capital Region history situated just across the river from downtown Albany in Rensselaer. Part of the foundation of the house dates to the 1640's when a house was constructed on the site for Domine Johannes Megapolensis, the first Dutch Reformed minister hired by Kiliaen Van Rensselaer to tend to the people of Rensselaerswyck in New Netherland. Rensselaerswyck was Kiliaen Van Rensselaer's private colony of about 700,000 acres that he had purchased in 1630. Evidence in the archaeological record of the site indicates this building was destroyed, most likely by fire, in the 17th century. Jeremias Van Rensselaer then constructed another building on the site of the earlier house.

In 1707 Hendrick Van Rensselaer, Kiliaen's grandson, built a new two room house on the site, incorporating the extant foundation. The new Dutch style house consisted of two rooms, one story high. A large barn was also built close to the house. The estate attached to Crailo at that time was 1,500 acres. In 1746, during King George's War, nearby raids by the French and their allies convinced John Van Rensselaer, Hendrick's son, to fortify the home with gun ports and a palisade.

The farm land around Crailo was used to camp soldiers during the wars the English fought with the French during the 18th century. The barn was used as a hospital for sick soldiers. It has been claimed that the poem that became the well-known song "Yankee Doodle" was written at Crailo in 1758. According to the story Richard Shuckburgh, a British surgeon, attached to General James Abercromby's ill-fated expedition against the French at Fort Carillon, now called Fort Ticonderoga, had come to visit the Van Rensselaer home. While there he observed some New England militia men sloppily performing military drill. He found the image ridiculous and penned a poem mocking them. Put to music and given time the insult became the patriotic tune known today

In 1762, as the French and Indian War was winding down, a two story addition was added to the east side of the house and the roof of the original house was raised to match. The new structures made the house appear to be Georgian style especially when viewed from Albany.

At some point around 1790 the house was given a federal style makeover. Within a few years Alexander Hamilton may have stayed at the house when he and his wife fled a Yellow Fever epidemic in Philadelphia. The Hamilton's were forced to wait overnight on the east side of the river for a team of doctors to certify there health. Eliza Hamilton's mother had been born at Crailo and it belonged, at that point to her cousin.

By 1806 the 1,500 acres that had once made up the Crailo estate had been divided up and sold as building lots. In 1825 the house was rented for the first time. In 1834 the house became a boarding school for boys under 14, the Mansion School. The school closed in 1839 only to reopen in 1843 as Mansion Hall School. This school operated until 1850. Also during the nineteenth century several small wooden additions were made to the back of the house.

In 1852 Dr. Jeremiah Van Rensselaer moved his large family into the house. They resided in the house until about 1870. In large dike was built in front of the house and filled to create the existing shore line. Shortly thereafter many members of the Van Rensselaer family became ill and died. In 1874, Anna Van Rensselaer, the Doctor's widow, sold the house to the Callendar family.

The Callendar family attempted to profit from owning the house. On a stone in the basement they carved the letters "KVR" and the date "1642" to date the house. In 1880 they published a booklet on the house claiming the inscription gave the true age of the house. They then began offering tours of the house at twenty five cents a person.

The Callendars abandoned the house in 1894 despite ongoing efforts to have the state take possession of it. In 1897 it was sold at auction to an ice dealer who planned to tear the house down and sell it off brick by brick for souvenirs. He had torn the wooden additions off the house before a Van Rensselaer family descendent convinced him to sell the house to her. The house was saved by Susan de Lancey Van Rensselaer Strong. Strong spent several years trying to restore the house herself. Originally she intended to give the house to the Daughters of the American Revolution for use as a museum but the internal politics of the organization prevented the donation. In 1924 Strong donated the house to the State of New York on the condition that it be maintained as a museum.

Between 1931 and 1933 New York embarked on what can only loosely be called a restoration. It was decided to make the building appear to be a house built in Holland in 1663, not one that would have appeared in New Netherland or New York. The interior was in need of extensive repair and most of the original material was replaced. The style chosen seems to have been a Dutch house as seen through an English Colonial Revival lense, which led to New England style windows and varnished wood paneling throughout the house. It was furnished with "old timey" things from the 17th, 18th and 19th century as well as reproductions.

In 1986 the traditional displays were replaced with an exhibit that focused on the recent important archaeological discoveries from the Dutch period in the area such as Fort Orange. The house was now much more of a museum, the museum of the colonial Dutch in the Hudson River Valley, rather than a historic house. In 2009 this exhibit was once again updated with a new exhibit entitled "A Sweet and Alien Land" which included new text, period artwork, interactive components and audio visual elements in addition to the archaeological evidence. One room of the house was also designed to look like the interior of a typical Dutch colonial house so that visitors could get a better sense of how their colonial predecessors had lived.