The official who spoke on condition of anonymity said that two gunmen on a motorbike opened fire on the American and killed him on Sunday, AFP reported.
The US citizen was the deputy director of a Swedish language center in Taizz.
The official also said that the attackers fled the scene after the shooting.
Yemen is in turmoil after US-backed dictator Ali Abdullah Saleh was forced to leave power after one year-long anti-regime protests in the country.
Saleh handed power to the then-vice president Abdrabuh Mansur Hadi.
The power transfer was under a Saudi-backed deal brokered by the (Persian) Gulf Cooperation Council in April last year and signed by Saleh in the Saudi capital, Riyadh, on November 23, 2011.
Hadi, who is a UK-trained field marshal, was sworn in as the country’s new president following a single-candidate presidential election on February 21 that was backed by the US and Saudi Arabia. He will serve for an interim two-year period as stipulated by the power transfer deal.