Near plans to build world’s largest 1.4T parameter open-source AI model

11/11/2024 17:47
Near plans to build world’s largest 1.4T parameter open-source AI model

Near Protocol unveils plans for the world’s largest open-source AI model, using crowdsourced research and aiming to decentralize AI tech.

Update Nov. 11 at 6:00 am UTC: Two quotes previously attributed to Edward Snowden in the final paragraphs have been corrected to attribute to Illia Polosukhin.

Near Protocol has unveiled an ambitious plan to build the world’s largest open-source artificial intelligence model on the opening day of its Redacted conference in Bangkok, Thailand. The 1.4 trillion parameter model would be 3.5 times bigger than Meta’s current open-source Llama model.

It will be created through competitive crowdsourced research and development from thousands of contributors on the new Near AI Research hub, with participants able to join the training of a small 500 million parameter model from today, Nov. 10.

Near Protocol’s ambitious AI model

The project will grow in size and sophistication across seven models, with only the best contributors making the leap to working on progressively more complex and larger models. The models will be monetized and privacy will be preserved via the use of encrypted Trusted Execution Environments to reward contributors and encourage constant updating as the technology progresses.

Related: Near patches critical bug that could crash every node on the network

The expensive training and compute will be funded with token sales, Near Protocol co-founder Illia Polosukhin told Cointelegraph at the Redacted conference in Bangkok.

Cryptocurrencies, Near Protocol, ChatGPT

Near co-founder Illia Polosukhin. Source: Cointelegraph

“It costs about $160 million, so it’s a lot, obviously, but actually, in crypto, it is raiseable money,” he said. Polosukhin added:

“Then the tokenholders get repaid from all the inferences that happen when this model is used. So we have a business model, we have a way to monetize it, we have a way to raise money, and we have a way to put this in a loop. And so people can actually reinvest back into the next model as well.”

Near is one of the few crypto projects with the ability to realize such an ambitious undertaking: Polosukhin was one of the authors of the groundbreaking transformer research paper that led to ChatGPT, and co-founder Alex Skidanov worked at OpenAI in the lead-up to the era-defining model’s release in late 2022.

Skidanov, who now heads up Near AI, conceded that it’s a massive undertaking with a big hurdle to overcome. 

Decentralized AI tackles privacy issues

To train such a large model, the project would need “tens of thousands of GPUs in one place,” which wouldn’t be ideal. But to use a decentralized network of compute “you’d need a new technology that doesn’t exist today because all the distributed training techniques we have require very fast interconnect.” However, he added that emerging research from Deep Mind suggests it is possible.

Cryptocurrencies, Near Protocol, ChatGPT

Near co-founder Alex Skidanov. Source: Cointelegraph

Polosukhin said he hasn’t spoken with existing projects like the Artificial Superintelligence Alliance but would be happy to see if there are synergies. Whatever happens, he said decentralized AI technology must win for all our sake.

“This is probably the most important technology right now and probably in the future. And the reality is, if AI is controlled by one company, we effectively are going to do whatever that company says,” he explained, adding:

“If all AI and effectively, all of the economy is being done by one company, there’s no decentralization at that point. So it is effectively like the only way that Web3 is still relevant, philosophically, if we have AI that also follows the same principles.”

Conference guest speaker Edward Snowden drove the message home with a chilling depiction of centralized AI transforming the world into a vast surveillance state.

He also spoke of the need for civil rights on the internet and for the need to recognize “there are legitimate limits on their authority to regulate, and the only way to preserve our digital sovereignty is to create our own systems that are enforced through math.”

Magazine: Asian crypto traders profit from Trump’s win, China’s 2025 CBDC deadline: Asia Express

Read more --->