Here's how to get started with the open source tool Continue
Code assistants have gained considerable attention as an early use case for generative AI – especially following the launch of Microsoft's GitHub Copilot. But, if you don't relish the idea of letting Microsoft loose on your code or paying $10/month for the privilege, you can always build your own.
an AI code assistant and integrate it into an IDE, it's far from the only option out there. In fact, there are numerous large language models trained specifically with code generation in mind. What's more, there's a good chance the computer you're sitting in front of right now is capable of running these models. The trick is integrating them into an IDE in a way that's actually useful.is designed to plug into popular IDEs like JetBrains or Visual Studio Code and connect to popular LLM runners you might already be familiar with – like Ollama, Llama.cpp, and LM Studio. Like other popular code assistants, Continue supports code completion and generation, as well as the ability to optimize, comment, or refactor your code for different use cases. Additionally, Continue also sports an integrated chatbot with RAG functionality, which effectively allows you to talk to your codebase. We'll be looking at using Continue with Ollama in this guide, but the app also works with several proprietary models – including OpenAI and Anthropic – via their respective APIs, if you'd rather pay per token than a fixed monthly price.A machine capable of running modest LLMs. A system with a relatively recent processor will work, but for best performance, we recommend a Nvidia, AMD, or Intel GPU with at least 6GB of vRAM. If you're more of a Mac person, any Apple Silicon system, including the original M1, should work just fine – though we do recommend at least 16GB of memory for best results. This guide also assumes you have the Ollama model runner set up and running on your machine. If you don't, you can find our guide, which should have you up in running in less than ten minutes. For those with Intel Integrated or Arc graphics, you can find a guide for deploying Ollama with IPEX-LLMFor this guide, we'll be deploying Continue in VSCodium. To get started, launch the IDE and open the extensions panel. From there, search for and install"Continue." After a few seconds, Continue's initial setup wizard should launch, directing you to choose whether you'd like to host your models locally or tap into another provider's API. In this case, we're going to host our models locally via Ollama, so we'll select"Local models." This will configure Continue to use the following models out of the box. We'll discuss how to change these out for alternative ones in a bit, but for now these offer a good starting place:A general-purpose LLM from Meta, which is used to comment, optimize, and/or refactor code. You can learn more about Llama 3 in our launch-day coverageAn embedding model used to index your codebase locally enabling you to reference your codebase when prompting the integrated chatbot.This is a code generation model by BigCode that powers Continue's tab-autocomplete functionality. If for whatever reason Continue skips past the launch wizard, don't worry, you can pull these models manually using Ollama by running the following in your terminal:Before we continue, it's worth noting that by default, Continue collects anonymized telemetry data including:The number of tokens generated;file located in your home directory or by unticking the"Continue: Telemetry Enabled" box in VS Code settings.With the installation out of the way, we can start digging into the various ways to integrate Continue into your workflow. The first of these is arguably the most obvious: generating code snippets from scratch.In this case, our prompt was"Generate a simple landing page in HTML with inline CSS." Upon submitting our prompt, Continue loads the relevant model – this can take a few seconds depending on your hardware – and presents us with a code snippet to accept or reject.Continue can also be used to refactor, comment, optimize, or otherwise edit your existing code. For example, let's say you've got a Python script for running an LLM in PyTorch that you want to refactor to run on an Apple Silicon Mac. You'd start by selecting your document, hittingAfter a few seconds, Continue passes along the model's recommendations for what changes it thinks you should make – with new code highlighted in green and code marked for removal marked with red. In addition to refactoring existing code, this functionality can also be useful for generating comments and/or docstrings after the fact. These functions can be found under"Continue" in the right-click context menu.While code generation can be useful for quickly mocking up proof of concepts or refactoring existing code, it can still be a little hit and miss depending on what model you're using. Anyone who's ever asked ChatGPT to generate a block of code will know that sometimes it just starts hallucinating packages or functions. These hallucinations do become pretty obvious, since bad code tends to fail rather spectacularly. But, as we've previouslyIf letting an AI model write your code for you is a bridge too far, Continue also supports code completion functionality. That at least gives you more control over what edits or changes the model does or doesn't make. This functionality works a bit like tab completion in the terminal. As you type, Continue will automatically feed your code into a model – like Starcoder2 or Codestral – and offer suggestions for how to complete a string or function. The suggestions appear in gray and are updated with each keystroke. If Continue guesses correctly, you can accept the suggestion by pressing theAlong with code generation and prediction, Continue features an integrated chatbot with RAG-style functionality. You can learn more about RAG in our hands-on guide, but in the case of Continue, it uses a combination of Llama 3 8B and the nomic-embed-text embedding model to make your codebase searchable.This functionality is admittedly a bit of a rabbit hole, but here are a couple of examples of how it can be used to speed up your workflow:and selecting the file you'd like to add to the drop-down.How reliably Continue actually is in practice really depends on what models you're using, as the plug-in itself is really more of a framework for integrating LLMs and code models into your IDE. While it dictates how you interact with these models, it has no control over the actual quality of the generated code. The good news is Continue isn't married to any one model or technology. As we mentioned earlier it plugs into all manner of LLM runners and APIs. If a new model is released that's optimized for your go-to programming language, there's nothing stopping you – other than your hardware of course – from taking advantage of it. And since we're using Ollama as our model server, swapping out models is, for the most part, a relatively straightforward task. For example, if you'd like to swap out Llama 3 for Google's Gemma 2 9B and Starcoder2 for Codestral you'd run the following commands:Note: At 22 billion parameters and with a context window of 32,000 tokens, Codestral is a pretty hefty model to run at home even when quantized to 4-bit precision. If you're having trouble with it crashing, you may want to look at something smaller like To swap out the model used for the chatbot and code generator you can select it from Continue's selection menu. Alternatively, you can cycle through downloaded models usingChanging out the model used for the tab autocomplete functionality is a little trickier and requires tweaking the plug-in's config file. After pulling down your model of choice , click on the gear icon in the lower right corner of the Continue sidebar and modify"title" and"model" entries under"tabAutocompleteModel" section . If you're using Codestral, that section should look something like this:By default, Continue automatically collects data on how you build your software. The data can be used to fine-tune custom models based on your particular style and workflows.in your home directory, and, from what we understand, isn't included in the telemetry data Continue gathers by default. But, if you're concerned, we recommend turning that off. The specifics of fine-tuning large language models are beyond the scope of this article, but you can find out more about the kind of data collected by the app and how it can be utilized in We hope to explore fine-tuning in more detail in a future hands-on, so be sure to share your thoughts on local AI tools like Continue as well as what you'd like to see us try next in the comments section. ®The Register was provided an RTX 6000 Ada Generation graphics card by Nvidia and an Arc A770 GPU by Intel to support stories like this. Neither supplier had any input as to the contents of this and other articles.EU AI Act still in infancy, but those with 'intelligent' HR apps better watch out7California trims AI safety bill amid fears of tech exodus And as Anthropic boss reckons there's 'a good chance ... we'll be able to get models that are better than most humans at most things'
United Kingdom Latest News, United Kingdom Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Police funding in Lancashire 'an area of increasing concern''Government funding needs to be based on the unique community needs faced by individual forces'
Read more »
FYI: Data from deleted GitHub repos may not actually be deletedAnd the forking Microsoft-owned code warehouse doesn't see this as much of a problem
Read more »
Malware crew Stargazers Goblin used 3,000 GitHub accounts to make bankMay even have targeted other malware gangs, and infosec researchers
Read more »
Your copilot for improved cyber protectionWatch this video to learn how Palo Alto Networks is using GenAI to automate and simplify cybersecurity
Read more »
If you give Copilot the reins, don't be surprised when it spills your secrets'All of the defaults are insecure' Zenity CTO warns
Read more »
Creating your own Microsoft Copilot chatbot is easy but making it safe and secure is pretty much impossible says security expertJeremy has been writing about technology and PCs since the 90nm Netburst era (Google it!) and enjoys nothing more than a serious dissertation on the finer points of monitor input lag and overshoot followed by a forensic examination of advanced lithography. Or maybe he just likes machines that go “ping!” He also has a thing for tennis and cars.
Read more »