Richardson was at first apprenticed to an engraver and afterwards to a cabinet-maker, whom he left to set up in business for himself. After five years of cabinet-making, he became a teacher, and from 1806 to 1813 filled the post which his father had held at the grammar school. Then he decided to adopt an artistic career, and soon acquired a reputation as a painter of landscape. He worked chiefly in watercolour, and found most of his subjects in the scenery of the Borders and the Scottish Highlands, though in later life he went as far afield as Italy and Switzerland.
He died at Newcastle on 7 March 1848, leaving a widow and a large family, six of whom (George Richardson, Edward Richardson 1810-1874, Thomas Miles Richardson Jr.1813-1890, Henry Burdon Richardson, Charles Richardson and John Isaac Richardson) followed the father's profession.