Another option is Chris Malachowsky, who founded Nvidia alongside Huang and Curtis Priem, and is the only executive that shares the same level of seniority.

Malachowsky has played a key part in developing the company's computing technology.

Ian Buck, who oversees Nvidia's high-performance computing business, has also emerged as a potential candidate if the company is keen to put an engineer in the top job.

Buck was among the early pioneers who helped popularise the use of graphic processors beyond gaming applications.

"He could protect Nvidia's technological edge, but his ability to manage the wider commercial relationships and geopolitical challenges would face scrutiny," said Charu Chanana, investment strategist at investment bank Saxo.

Today, Nvidia's leading processors are widely regarded as the gold standard for AI development.

Huang's children also work at Nvidia in executive positions but are relatively young and inexperienced.

Read More