Born in Boston, Massachusetts. Married Eliza Ann Blackman April 30, 1846. Their only child died in infancy. He was employed as a clerk in the hat store of Seth J. Thomas. He took over the business in 1840 and continued there until 1852. On March 8, 1852, he was appointed by President Franklin Pierce as United States Appraiser in the Custom House for the Port of Boston. He retired in June 1860.

Colburn began collecting at age 15. He collected autographs, books, manuscripts, minerals, portraits and shells. Much of his collection was sold at auction by Woodward in 1863. He was one of the founders and original members of the Boston Numismatic Society in 1860. He served as president of that group from 1865 until his death. He was elected an honorary member of the ANS December 23, 1867. Received an honorary A.M. degree from Williams College in 1869. In July 1870 the Boston Numismatic Society took over publication of the AJN. He was one of the editors. He served on the committee of publication from 1870 until his death in 1891. He died in Boston. In 2022 he was inducted into the ANA Hall of Fame.

bio: ApCAB; Attinelli; TCBDA; CW/NM 10/6/76 obit: AJN 26 Jan 1892 pages 49-50