California-based startup Nucleus AI, comprised of talented individuals from Amazon and Samsung Research, has recently announced the launch of their first product – a revolutionary 22-billion-parameter large language model (LLM). This general-purpose model, which can be fine-tuned for various tasks and applications, is available under an open-source MIT license as well as a commercial license. Nucleus promises that their model surpasses other models of a similar size and aims to utilize AI to transform the field of agriculture.
While the LLM market has plenty of contenders such as OpenAI, Anthropic, and Cohere, Nucleus AI sets itself apart by focusing on building an intelligent operating system for agriculture instead of creating chatbots. By leveraging their 22-billion-parameter transformer model, Nucleus AI aims to optimize supply and demand, mitigate uncertainties, and revolutionize the farming landscape much like Uber revolutionized the taxi industry.
Nucleus began training their 22B model approximately three and a half months ago with the help of compute resources provided by an early investor. The training process involved pre-training the model on a context length of 2,048 tokens, utilizing a trillion tokens of data from sources such as Wikipedia, Stack Exchange, arXiv, and code. This comprehensive approach ensured that the model had a well-rounded knowledge base, spanning from general information to academic research and coding insights.
Moving forward, Nucleus plans to release additional versions of their 22B model, trained on 350 billion tokens and 700 billion tokens respectively. Additionally, they intend to introduce two RetNet models – one with 3 billion parameters and another with 11 billion parameters – pre-trained on a larger context length of 4,096 tokens. These smaller-sized models hold the potential for significant improvements in terms of speed and costs, as preliminary experiments have shown that they are 15 times faster and require only a quarter of the GPU memory compared to equivalent transformer models.
Nucleus AI’s groundbreaking model marks a milestone in the AI research community. While previous research has only explored the possibility of such models, Nucleus AI is the first to build and publicly release a practical application. The company’s ambition extends beyond enterprise software; their true goal is to develop an AI-based marketplace for farmers. This intelligent operating system aims to optimize the supply and demand for agricultural products, leading to increased efficiency and profitability for farmers.
The integration of AI into the agricultural industry presents a unique opportunity to tackle various challenges faced by farmers. Nucleus AI’s operating system has the potential to address issues related to climate change, lack of knowledge, supply optimization, and distribution management. By hyper-optimizing the demand and supply chain for farmers, Nucleus AI seeks to revolutionize the way agriculture is conducted, providing farmers with tools and insights that can transform their success in the industry.
Nucleus AI’s emergence from stealth with their 22-billion-parameter large language model is a significant milestone for the AI community. This model, which outperforms other models of a similar size, is just the beginning of Nucleus AI’s journey to revolutionize the agricultural industry. By offering not just chatbots but an intelligent operating system, Nucleus AI aims to optimize supply and demand, mitigate uncertainties, and create a more efficient and profitable farming landscape. As more details about their farming-centric OS and the RetNet models are revealed later this month, the industry eagerly awaits the impact that Nucleus AI will have on the future of agriculture.
Leave a Reply