Gozaburo was the original founder and CEO of Kaiba Corporation, which at that time was a successful arms manufacturer. He was also a world famous chess champion, which is how Seto and Mokuba heard of him at their orphanage. When Gozaburo visited, Seto challenged him to a game of chess, telling Gozaburo that if he refused, he would tell the press he was scared of a little kid. Gozaburo, confident he would not be beaten by a child, agreed to a single game, with Seto stating the stipulation that if he wins, Gozaburo will agree to adopt him and his brother. In the manga, Seto cheated. In the anime, Seto had spent time studying Gozaburo's strategies and how to overcome them, and thus defeated him. Gozaburo had to keep his end of the bargain.
However, he was a cruel father to Seto, forcing him to spend all his time studying and forgoing any of the joy or fun usually associated with childhood. At this point his biological son Noah died as well. With his intended Kaiba Corp heir digitized, Gozaburo decided to breed Seto as his heir, pushing him even further. His fatal mistake was when he gave Seto 2% of Kaiba Corp shares as part of a mission to earn 100 times the amount within a year- something Seto does easily. Later, Seto, along with the board of directors, aka, The Big Five, buys 49% of the company. Gozaburo, also with 49%, is ousted from power by Seto's 51%. He is reported to have committed suicide by jumping from the top floor of the Kaiba Corporation building. (in the English anime, he ran away and disappeared.)