It happened around five a.m. Friday morning at 1600 Pacific Avenue.
Firefighters say the flames spread to an out building and threatened the Speedway's Grand Stand.
Damage is estimated at $80,000.
Homeowner Stan Blaine is a grounds-keeper for the Speedway. He and his wife Carol got out with minor injuries.
A fund has been setup for the Blaines at U.S Bank in Yakima.
Yakima Speedway has also setup a donation center at Kastl Tree lot at 20th and Lincoln Avenue.
The Red Cross is working with the family. Investigators are still trying to figure out what caused the fire.