After speculation of elder abuse or kidnapping, the fitness guru told Entertainment Tonight that he's just "been taking it easy."
