According to The Mirror, in a surprise move, Richmond was forced out from his role as an unpaid consultant to the board because of opposition from two of the four main directors.
Simon Morris and Melvyn Levi were unhappy with the former Bradford City chairman's influence at Leeds United.
They were uneasy after Leeds last week sent out a fax to all 92 league clubs and agents, stating Richmond had been given authority to handle transfers and contracts.
It was felt that Richmond should not be given such wide-ranging powers when he had not invested into the club and they were annoyed Richmond had been projecting himself as the club's face and voice when he was officially not even an employee.
John Boocock, chairman of Leeds United Supporters Trust, said: "For fans there will be some relief he is leaving. He was tainted and could have tainted Leeds United if he had stayed much longer."