An Imprecise, Inaccurate but Offline-First Search Index

As with any cross continent flight, I had no internet connectivity on my way to DEF CON. I couldn't even launch my dev containers in VS Code because they depend on a startup script that requires connectivity.

This double whammy set me back to a time without any dependency IntelliSense or documentation access. It made me realize how much I leverage editor features that provide hints for parameter ordering, types, and sometimes documentation.

At least I remembered the hono type definition for setCookie.

While I couldn't iterate on the flight, I could chisel away and rough out the project I am working on.

It's the app to receive and decide on applications submitted by merchants wishing to vend at FurSquared!

Without any connectivity, I roughed out yet another authentication integration with sign in with Google, a local developer tool to create accounts, a basic page template and a substantial set of CSS.

As a backend developer, who happens to understand CSS mechanisms thoroughly, I don't always have every attribute memorized. Over the last two years I would open tailwind CSS documentation, look for the visual effect I'm after, and then use the underlying CSS properties myself.

Except, on this flight, I didn't have tailwind to refer to.

I have heard about people using ChatGPT over Google to search things. It so happens that ChatGPT will read web pages now, but this rumor predates that capability. I happened to have Ollama and Llama 3.1 8b downloaded on my MacBook. I figured "what did I have to lose?"

After starting 3.1:8b, I said "how to hide browser default styling for checkbox" It gave a long winded answer, then hallucinated aria attributes in CSS, but in there I found the tidbit I was looking for: `appearance: none`.

To hide the browser's default styling for a checkbox, you need to use the following properties in your CSS:

input[type="checkbox"] {
  appearance: none;
  -moz-appearance: none; /* For Firefox */
}

The appearance property is used to control how an element looks. By setting it to none, we're telling the browser not to apply its default styling.

It should have ended there. Instead, it then hallucinated and said that removing the default styling will harm accessibility and then repeated the CSS above but with another non-existent attribute aria-label: "Check me";. When in actuality, about every article will say use the built in html to represent a checkbox and style it as you will, with appearance: none to reset it. This attribute, aria-label, does exist for HTML but not CSS.

I continued this flow of occasionally asking for a specific property or effect, or an example of a property being used such as transition.

Dissatisfaction with Llama

Out of morbid curiosity, I decided to ask "What is 10 days in milliseconds".

There are 1000 milliseconds in 1 second.

So, there are:

• 86400 seconds in 1 day (24 hours x 60 minutes x 60 seconds)
• 864000 milliseconds in 1 day
• 8640000 milliseconds in 10 days (864000 milliseconds/day x 10 days)

Therefore, 10 days is equal to 8,640,000 milliseconds.

The true answer to how many milliseconds are in ten days is: 864,000,000 milliseconds.

I think we need to give these models a literal calculator escape hatch. And maybe, just maybe, a datalog reasoning engine.

For a "not found" page, I wanted it to say, randomly "The (animal) you're looking for is not here". I asked for a list of one hundred animals. It gave a list of twenty and then commented that there could be ninety more. 20 = 100 = 110, according to the technology meant to replace humanity.

// If you want to generate an array of 100 random animal names:
const allAnimals = [
  "Lion", "Tiger", "Bear", "Monkey", "Giraffe", "Zebra", "Elephant", "Kangaroo", "Penguin", "Seal",
  "Tapir", "Meerkat", "Fennec Fox", "Armadillo", "Axolotl", "Cheetah", "Hippopotamus", "Okapi", "Snow Leopard", "Wolverine",
  // ... 90 more animals ...
];

Along the many prompts I gave while disconnected from the earth, I noticed a pattern of hedging at the end. I suspect that alignment is responsible for this encouraging this behavior.

Here's one example at the end of asking for some drop zone HTML and CSS.

Please note that this is just an example code and may need modifications to suit your specific requirements. You'll also need to handle server-side processing for file uploads.

The code it generated was not great. In fact, it generated functions with unused parameters, duplicate functions with different names, and pretended that uploading was a synchronous function of looping over an array.

The imprecise and incorrect output had enough hints to where I could see that the DragEvent existed. Visual Studio at least had TypeScript definitions for DOM functions, which then revealed the DataTransfer object inside.

By matching these hallucinations with the resources I had, I was able to learn about something I've always delegated to a library, without internet connectivity.

Google's Monopolistic position has allowed them to comfortably optimize for ad revenue by sacrificing search result quality. I can see why folks are frustrated enough to ditch Google and try an LLM as a search engine. It isn't perfect, but it gives enough specific details to find out more… without any ads.

That said, the MDN documentation is great. Consider supporting MDN. I have no affiliation with them.