It is almost time to code in spoken English, but not yet time to architect in spoken English

There is a bit of history about how English became the most prominent language for programming. However, the wanton variations in programming language syntax and the poor manner in which various software frameworks are built, makes one wish for being able to do build software using spoken English.

Models are still unable to visualize and understand what we want it to do

Recently I tried building a simple chatbot using ChatGPT to generate all the code for me. During that effort, I noticed how the LLM (ChatGPT) did not actually understand what I meant by the left side or right side of the page. Also, when it re-generated code based on corrections I asked it to do, it would re-generate even older errors which I had corrected a few prompts earlier. This showed a critical flaw in the use of LLM's for coding. The LLM is unable to visualize what we want it to do. Even though ChatGPT-4o has got much better at code generation, these issues continue to plague even modern models.

Techniques being used to build better AI copilots

From Andrew Ng's "The Batch" newsletter, I came across the information about the techniques used: 

• Chain of thought (the model is made to think step by step)
• Self-consistency (the model generates many responses and picks the one that's more consistent with the other responses)
• ReAct (reasoning and action steps are interleaved to achieve the task)
• Self-refine (the agent reflects on its own output)
• Reflexion (allows a model to act, reflect, evaluate and repeat)
• Test time compute (increases the amount of processing power allotted to inference)

Coding assistants

WindSurf IDE's CoPilot

I recently tried using extensions which either connected to LLM's on a server or used a local LLM via Ollama. Today, there are better options:

1. Cursor: It promises to keep code private. I'm yet to try it, but it has a free version for limited use, so I'm likely to try it, although a bit reluctant since it needs me to download their IDE. However, I'm always skeptical about privacy, because many of these websites which make tall claims of keeping data private and encrypted, actually might have lax security, as muah.ai users found out recently.

2. WindSurf:  This is an IDE based on VS Code which you need to download and setup to make it automatically connect to their server. I tried this and found to be reasonably good. There are some glaring usability issues related to using undo on the generated changes, but in terms of what it provides, there are actions (perhaps self-refine or reflexion) it does to make changes to the code and see if it makes sense and then it automatically re-attempts changes until it gets to what it thinks is correct. With prompts where you give it many instructions, it tends to really mess up the code. Anyway, having to use a separate IDE is a no-go for me, so I'll be uninstalling it soon. Since it can be used to prompt the LLM to access my filesystem, my trust in it is even lesser now (possibility of hacks).

3. Supermaven: Not yet used this, but since it's an extension which works with VS Code, NeoVIM and JetBrains, it's better placed than options like WindSurf, and has a free tier too, which might not be as restrictive in terms of how long you can use it. Their 7 day data retention policy was also a relief.

4. GitHub CoPilot: I'm more likely to try this since it does not need me to install a separate IDE, and now they have a free plan too.

5. ChatGPT's canvas: When you first generate code in ChatGPT, it switches to a canvas mode where you can do more edits. As of today it's useless, since it ended up replacing my HTML code entirely with JavaScript when I prompted it to make some small changes.

6. Sourcegraph: Their unlimited free autocomplete was the first thing I noticed. Might try it later.

7. Amazon Q: Has not only a free option, it also has plugins for Eclipse, VS Code and Jetbrains.

8. Double: This has only a 30 day free version, but seems to have some features that rival GitHub Copilot.

9. Gemini in IDX: I'm not surprised not many people are talking about this. It's in beta stage now. It's basically VS Code in your browser, and activate Gemini coding assistant or install any other extension as a CoPilot. Since it's in beta it's not working fully yet. Once it works, it might be far better to use than VS Code or Android Studio.

10. Tabnine: They seem to have even integration with Jira built in, which is a cool feature. Only 30 days for a free trial though. They support a lot more IDE's like Jetbrains, Eclipse, VS Code, NeoVIM, and more.

There are a few other stray examples like PotPie, Pieces, a PR agent, copilot.live and Tabby. So far, comments on Reddit don't seem to be too happy with GitHub Copilot, but they seem to like SuperMaven, WindSurf and Cursor. There's more coming, with agentic software like Devin, De and LangGraph studio

To pay or not to pay

From what I've seen so far, it'll take some time for LLM's to be able to become capable enough to design an overall architecture and plan for efficiency of code reuse. So programmers really need to know their concepts of the framework and the language well to be able to build reliable software. Due to this, I believe it's not yet worth it (at least for me) to pay for these services. It's more than enough for me to design the architecture myself and prompt the free LLM's only when I need it to generate any code. But that being said, these coding assistants are truly effective at slicing down development time of a few weeks or months, to a matter of minutes or hours. All that drudgery of figuring out errors and their solutions, and finding better or alternate ways of doing things are getting done so much faster with the LLM.

On a side note

I wonder how well LLM's will adapt to people who prompt it with grammatical mistakes in their English. Although this also means that now people will be able to chat or prompt LLM's in their native language, and the LLM will actually be able to generate code using non-English languages. I still feel we are only scratching the surface of how software will be built in the future. There are already people trying to get rid of API's. I'd like to see a future where we plug in our minds to a computer and simply think to generate software solutions from what we imagine.