A fire in James City County damaged a residence, and a man was taken to the hospital with smoke inhalation.