10 March 2024

Websites offering ChatGPT-like functionality without needing to login

Being able to use ChatGPT for free is cool. It does get annoying that one has to login to use it. When you are in a hurry, and want to ask something to an LLM, there are some websites you can quickly access by keeping them bookmarked.

Vercel's AI playground

https://sdk.vercel.ai/

This is my favourite. It not only uses ChatGPT, it simultaneously generates an output with Llama too.

 

HuggingChat and HuggingFaces Spaces

 

https://huggingface.co/chat/ and https://huggingface.co/spaces/olivierdehaene/chat-llm-streaming

HuggingChat is a page that opens without much of a hassle, but it allows only 3 prompts. In settings you can choose from various models.

HuggingFaces spaces allows typing more prompts and even allows you to easily change the "p mass", which appears to be the temperature, but might be top p.

 

Talk AI

 

https://talkai.info/chat/

This begins with a slightly irritating human verification that needs a manual input, and has a 30 prompt limit, but the chat interface is ok.


Chat AI GPT

https://chataigpt.org/

Even this begins with a human verification, but it automatically disappears. The chat interface is ok.


LM Sys

https://chat.lmsys.org/

This site has the annoying human verification delay and needs you to click on a usage terms agreement. The good part is that it allows you to compare models and allows changing various parameters like temperature, top P and max output tokens.

28 February 2024

Use a free, local LLM instead of GitHub CoPilot

There was a time when Google and StackOverflow replaced the need to buy books to refer documentation. Now we have entered the era of LLM's (Large Language Models) replacing Google and StackOverflow for coding. We've also entered the era of frameworks like SkyPilot assisting with batch jobs on the cloud. Choosing an LLM itself is like choosing an ice cream in a buffet of ice creams. To ease my programming, I decided to try a few VS Code extensions similar to GitHub CoPilot. The objective was to find extensions that worked locally without sending any of my data to an external server, and were free to use. 

CAUTION: VSCode extensions aren't checked by anyone for safety, so install it only if you really trust the creator of the extension. Even for the ones mentioned below, check first.

There's a leaderboard of how well models can write code. 

Instruct model versus base model

  • Base models: These aren't designed for answering questions. These are meant to provide completions. If you prompt it with "What is the capital of India", it'll respond with "?indiacapitalisdelhi#code ends here". Notice how it outputs the question mark. It only generates a completion based on the data it was trained with. Base models are used to generate code while you type.
  • Instruct models: The instruction tuned models are designed to answer questions. Asking it for the capital of India would return something like "The capital of India is New Delhi. It serves as the center of government for the country and...". The instruct models are used for question-answer type of interaction with the LLM.

 

Using Ollama for the local LLM instead of OpenAI's API

Download and install Ollama using `curl -fsSL https://ollama.com/install.sh | sh`. It installs to `/usr/local/bin` and creates a systemd service. You can also build ollama and take advantage of ROCm for AMD inbuilt GPUs. There are some ways to stop the service. Downloaded models get stored in `usr/share/ollama/.ollama/models/blobs`, and you can change the directory by changing the OLLAMA_MODELS environment variable. You can access it on localhost via port 11434. Some other commands:

  • systemctl list-unit-files | grep ollama will show you if it's running.
  • systemctl status ollama will show you more info about the process.
  • sudo systemctl stop ollama.service will stop the process.
  • sudo systemctl disable ollama.service will disable auto-startup.
  • sudo systemctl start ollama.service to start the process.
  • sudo systemctl enable ollama.service will enable auto-startup.
  • ollama list will show a list of the downloaded models.
  • ollama pull <modelname> will download any model listed here. Remember to use a smaller model for getting a faster response. This is especially important when you are using only CPU.
  • ollama rm <modelname> will delete a downloaded model.

 

Twinny 

So far, Twinny has been the only local LLM that has worked fine, is well designed and supports multiple models. You can open Twinny's sidebar by pressing Ctrl+Shift+t. You can use the Alt+\ key combo to generate code completions. You can stop the code generation using Ctrl+Shift+/. The code completions are called FIM (Fill In Middle). Do take note that to generate code completions, you need to use models called "base models". Don't use "instruct models" for code completions. To make the suggested code in grey become actual code, you need to press Tab.

Here, the stable-code model is used for fill-in-middle code suggestions

You can also use the Twinny sidebar to chat with it, similar to how you chat with ChatGPT. For this, you need to use "instruct models". You can even highlight code in the main editor and ask Twinny to explain the code or generate code based on the highlighted code. For this functionality, the model specified in Chat is used. Chat can sometimes not understand the User or it produces incomplete outputs. This depends on how the model is prompted, which model is used and how much processing power is available. I'll update this section if I find out more.

For this chat, the `codellama:latest` model was used.

 

There is also a way to use models from HuggingFace.


Continue

The other almost-good extension I found was this. It can work by connecting to one of Continue's servers, or you can configure it to work with a local LLM too. Continue sends telemetry data to its server, so you need to go into the settings to disable it.

Continue didn't account for the entire code when adding new code

As shown in the screenshot above, I selected a function and asked Continue to add an error message if a file was not found. It did a poor job of adding new code. The entire code barely had 30 lines. This showed me that Continue was poorly programmed. There is of course a sidebar where you can type prompts and have code explained or generated, but I uninstalled Continue.

It connects to this server:

Running `nslookup node-proxy-server-blue-l6vsfbzhba-uw.a.run.app` showed the server it connected to


Continue still has many bugs that need to be resolved. Also, I couldn't fully trust it to run entirely locally, so I uninstalled it.


Other local LLM extensions

  • Wingman-AI: It's good, but it didn't support the `stable-code` model, so I uninstalled it.
  • Backseat pilot: Poorly explained on how to use. Meant to use llama cpp python. Uninstalled it immediately
  • Open copilot: Needs cody and llama cpp. Uninstalled it.
  • Local pilot: Requires a Github copilot account.
  • llama coder: Didn't generate anything. No interface to work with.
  • Tabby: Needs Docker Tabby server. Works on CPU and CUDA.
  • Your copilot: Doesn't work out-of-the-box. Meant for OpenAI API. Giving it Ollama's URLs didn't work.
  • Ollama autocoder: No interface. Not straightforward to use. Auto completion didn't work with the default space+pause or ctrl+space.
  • Wingman: Uses LM Studio or Openai APIs. Can't use at work without their permission. Likely uses telemetery. Has a nice set of prompt UIs for mode, prompt, placeholders. caters to general programming, creative writing and technical writing. The appimage didn't work on Linux.
  • Ollama copilot: Incompatible with latest vscode version.
  • Refact.ai: The local version needs docker with NVidia GPU.


If you don't want to pay for an OpenAI API but still want to use ChatGPT3.5's free chat, a person created Headless ChatGPT.


01 January 2024

Get rid of Google's annoying sign in popup

 


It's like a horror movie ghost giving you a jump-scare each time you open a website. Google somehow thinks it'd be a great idea to take away focus from whatever we are doing, and present us with the most annoying thing on the internet: The Popup!

Thankfully, it's possible to get rid of this. 

On Firefox:

1. Install the Adguard extension.

2. Enable Adguard to run even in private tabs (if you use private tabs).

3. In Adguard's preferences, select "Filters" and then enable the "Annoyances" toggle button.

4. Now click on the "Annoyances" section to view all the annoyances that are to be blocked, and enable any of the annoyances you want to get rid of (for example, the "AdGuard Popups filter").

Some details on using uBlockOrigin instead, are given here.

On Google Chrome:

This of course works only if you are signed in. Direct link.

1. Go to "My Account" : "Security" ( https://myaccount.google.com/security ).
2. Click on "See all connections" link in "Your connections to third party apps & services".
3. Click on the gear at the top of the "Third party apps & services" page.
4. Disable the "Signing in with Google" prompt.

 

That's it. Enjoy your popup-free browsing! Some other people have given the same solutions here and here too.

08 November 2023

Ten minute email id

Ever been to a website that required you to enter your email id to sign up for even a short trial, and doing so, exposed you to their spammy newsletters and promotional emails? Well, 10minutemail is here to rescue you from that.

I came across this interesting concept https://10minutemail.com/ (there's also https://10minutemail.net/), which allows you to keep extending the 10 minute limit. If you refresh the page, you won't lose that email id until the 10 minutes expires. 

I tried sending an email from my real email id to the address shown, and sure enough, within a minute, 10minutemail showed the test email in its inbox. You'll have to scroll down for it.

It's interesting that people actually created such a concept.

Tips: 

  • Did you know that GMail ignores the dot in an email id? For example, if you have an email id as brad.pitt@gmail.com, you can write it as bradpitt@gmail.com, and the email will still reach the same person.
  • Did you know that you can add anything after a plus symbol, and those parts get ignored by GMail? For example, if you subscribe to a newsletter from vox, you can provide your email id to them as bradpitt+vox@gmail.com. The "+vox" will be ignored, but when you receive the email, the email id will be shown as "bradpitt+vox@gmail.com". This makes it much easier to filter emails and recognize where it's coming from.