xAI opens source its powerful language model Grok
Elon Musk-owned xAI has published the source code of its large language model Grok-1 with 314 billion parameters on GitHub under the open source Apache 2.0 license.
The open release includes basic model weights, network architecture, and other components. As noted, Grok-1 has not been specially optimized or censored for specific tasks such as dialog systems.
This week, @xAI will open source Grok
– Elon Musk (@elonmusk) March 11, 2024
The model was trained for four months until October 2023 and was intended to generate code, texts, and answers to questions. Opening the code will allow third-party researchers and developers to work with the model.
Despite the openness of the scales and other components, Grok-1 comes without training code and reproducible data sets.
Grok weights are out under Apache 2.0: https://t.co/9K4IfarqXK
It is more open source than other open weights models, which usually come with usage restrictions.
It’s less open source than Pythia, Bloom, and OLMo, which come with training code and reproducible datasets. https://t.co/kxu2anrNiP pic.twitter.com/UeNew30Lzn
– Sebastian Raschka (@rasbt) March 17, 2024
After purchasing Twitter (X), Elon Musk repeatedly criticized the closed source code of competitors’ AI models such as OpenAI. Now he is suing OpenAI, accusing the company of violating the open source agreement.
Publishing source code is a common practice to get feedback and improve models. However, the largest players prefer to keep the code closed or issue limited licenses, as is the case with Meta’s Llama 2.
After the launch of Grok chatbot based on this model, a paid subscription to X was required to access it earlier. But in preliminary testing, it failed to stand out against ChatGPT, Gemini, and other more advanced systems.
