"Birmingham City Football Club expresses its sincere gratitude to Tom Brady for his involvement with and unwavering support for the Club over the last three years," the Championship side said in a statement.
"It was truly amazing to have someone with Tom's experience and passion associated with the Club, and the Club looks forward to continuing to build on that foundation from the training ground to the Birmingham community."
The club gave no reason for Brady's departure and did not specify what would happen to his stake.
The former NFL quarterback joined Birmingham's ownership group in August 2023 through a partnership with Knighthead Capital Management, becoming a minority investor and chairman of a new advisory board.
Brady, 49, also holds minority stakes in the NFL's Las Vegas Raiders and the WNBA's Las Vegas Aces.
His departure comes as Birmingham enjoy a strong Championship campaign, sitting top of the standings on goal difference with 29 points after 15 matches.