The significance of the Athenaeum is threefold: it is a fine example of classical revival architecture, one of only two in the City of Alexandria; it has a long and colorful history of occupation; and it is home to the Northern Virginia Fine Arts Association.

The land the Athenaeum was built on on was originally owned by Lord Fairfax and surveyed by George Washington. As such, it is located in one of the oldest parts of Alexandria. Constructed between 1851 and 1852, it began as the Bank of the Old Dominion. The bank was the source of capital and financing of various businesses—being strategically situated at the head of "Captain's Row," a block of 18th century houses still facing the original cobblestone street. It is believed the massive vault of the bank sat atop a huge base of stone and rock—which is under the current floor of the main gallery, and visible from the kitchen and main meeting room below. The Bank of the Old Dominion occupied the building from its completion until the Civil War.

Following the occupation of Alexandria by Union forces, the building was taken over and became the Chief Commissary Office of the U.S. Commissary Quartermaster. The NVFAA has a copy of a photograph taken by Matthew Brady showing Army officers outside the building during the period. The bank eventually closed its doors in 1862. From 1870 to 1907, the building served as the home of the First Virginia Bank.

From 1907 to 1925, Leadbeater and Sons, one of the oldest Alexandria firms, utilized the building for their wholesale pharmacy business. During the 2007 renovation of the building, small corks and product packaging from Leadbeater were discovered in the rubble under the main gallery subfloor. In 1925 the Free Methodist Church of North America used the building as their house of worship. They were the first society of that faith to be formed in the Metropolitan region.

The Northern Virginia Fine Arts Association bought the building in 1964, repaired it, restored it to its current state and renamed it the Athenaeum. The Athenaeum became the first branch of the Virginia Museum of Fine Arts and remains a showcase for all kinds of art. The Athenaeum is on both the Virginia Trust and National Register of Historic Places, so conserving the building is a foremost goal. Over $300,000 has been raised over the last 40 years to restore and preserve the Athenaeum.

As a 501c(3) nonprofit organization, the NVFAA survives financially with income from membership dues, grants, private donations, art and ticket sales, and facility rentals. The Athenaeum is one of the few privately owned buildings in Alexandria open to the public and depends largely on the community for its survival.