The tech world is abuzz with the latest innovation from Stability AI.
Known for its remarkable Stable Diffusion text-to-image generation model, the generative AI pioneer is now venturing into the realm of code generation. Brace yourselves, as Stability AI rolls out StableCode, a cutting-edge open Large Language Model (LLM) designed to help users generate programming language code.
StableCode is being launched in three different flavours: a base model for general use, an instruction model, and a long-context-window model that can support up to a whopping 16,000 tokens. The model is built on the foundation of programming language data from the open-source BigCode project, with additional fine-tuning by Stability AI. Initially, StableCode will support Python, Go, Java, JavaScript, C, markdown, and C++ programming languages.
Christian Laforte, head of research at Stability AI, shared the company's vision, "We'd like to do the same thing with the StableCode model: basically allow anyone that has good ideas [and] maybe has a problem, to be able to write a programme that would just fix that problem."
StableCode's training involved significant filtering and cleaning of the BigCode data. Nathan Cooper, lead research scientist at Stability AI, revealed that the training for StableCode included successive training on specific programming languages, following a similar approach to what's done in the natural language domain.
What sets StableCode apart is its long-context-window version, which offers a context window of 16,000 tokens, claimed to be larger than any other model. This longer context window enables the use of more complex code generation prompts and allows StableCode to examine a medium-sized code base that includes multiple files, to help understand and generate new code.
Unlike other generative AI models, StableCode uses an approach known as rotary position embedding (RoPE), which doesn't weigh current tokens more than past ones, making it more suitable for coding.
The launch of StableCode marks a significant milestone in the field of generative AI and code generation. As developers begin to explore and utilise this new tool, the future of coding looks set to be reshaped by this innovative technology.
Made with TRUST_AI - see the Charter: https://www.modelprop.co.uk/trust-ai
Comments