Experimental

My home automation dashboard broke down in two ways this year. Hardware, then software. My wife and I really liked this dashboard. It saved us from pulling out our phones and getting sucked into emails or social media while we were supposed to be living our lives.

Both breakdowns ended up in rebuilds. The hardware obviously solved by 3D modeling and printing and the software part became another problem I would vibe code - no - vibe engineer - my way out of.

After rebuilding, it is totally different. But cooler perhaps? And everything it uses should last many decades this time, instead of just one decade.

Warning: This article has a lot of embedded code, so the ball machine is slow unless you have a REALLY fast computer. Play at your own risk.

Seriously, though.

GPT-5, as a simple text completion model, is not a revelation.

This isn’t so surprising. It was becoming clearer with every new raw LLM release that the fundamental improvements from scaling solely the performance of the core text predictor were starting to show diminishing returns. But I’m going to make an argument today that, although the LLM itself is not nearly as much of a leap from GPT-4 as GPT-4 was from GPT-3, we have still seen at least a whole-version-number of real improvement between the release of GPT-4 and 5 as we did between 3 and 4. The reasons for that are mostly what exists around that LLM core.

The advent of general coding AI assistants almost immediately changed how I think about hiring and interviews.

In the software engineering world, this mindset shift was psychologically easy for me, because I’ve always had a bias against the types of coding questions that AI can now answer near-perfectly. And they also happen to be the kind of questions I personally do badly at - the ones requiring troves of knowledge or rote memory of specific language capabilities, libraries, and syntax. It is not so psychologically easy for everyone, especially those who have developed a core skill set of running or passing “leetcode-style” interviews. Even before AI, the only types of coding questions I would personally ask were things that simply evaluate whether a candidate is lying or not about whether they can code at all, which was and still is surprisingly common. I have interviewed people that list bullet points like 7 years of Java experience but can’t pass a fizz-buzz like question, and this was a question I gave out on paper with a closed door and no significant time pressure.

“The Flapper” is a project that spawned out of a simple VR movement mechanic test that I had in my head for a while, which turned out to be surprisingly fun! The idea is to flap your arms to fly - wrapped up as a multiplayer battle to really get people moving.

In order to start working on this game, because there was so much standard VR code that I had to write for Treekeepers, I decided to make a sort of engine out of the Treekeepers codebase and work off of that rather than start from scratch. That let me tie in some of the nice associated graphics, music and sound effects I had made, and a bunch of other helper functions and tools I use for things like the camera following around the character, how I deal with collisions, a bunch of netcode, etc.

I have been occasionally challenging GPT to create models using OpenSCAD, a “programming language for 3D models”

Both struggle, but GPT-4 has been a massive improvement. Here are both models’ outputs after asking for an acorn and 3 messages of me giving feedback:

For the record, it is impressive that these LLMs can get anything right with no visual input or training on shapes like these. Imagine looking at the programming reference for openSCAD and trying to do this blind. The fact that the 3.5 version has a bunch of strangely intersecting primitives and some union issues has been normal in my experience. It takes quite a bit of spatial logic to get a model not to look like that.

I’m writing this post retrospectively as I never published it at the time of creation. It will live here as a “stake in the ground” of AI software capabilities as of March 2023. Note- if you’re reading on substack, this post won’t work. Go to hockenworks.com/gpt-4-solar-system.

The interactive solar system below was created with minimal help from me, by the very first version of GPT-4, before even function calling was a feature. It was the first of an ongoing series of experiments to see what frontier models could do by themselves - and I’m posting it here because it was the earliest example I saved.

Here’s a link to the chat where it was created, though it’s not possible to continue this conversation directly since the model involved has long since been deprecated: https://chatgpt.com/share/683b5680-8ac8-8006-9493-37add8749387

I’ve been playing around with neural radiance fields (NeRFs) lately and thought a fun way to explore them would be flying through them in the Treekeepers “Puddle Jumper” in true scale.

The “Human Joystick” is an experimental VR movement system in which the player moves through the virtual environment by changing their physical location within their VR “playspace”.

The Answering Machine is a proof-of-concept system that I built using pre-LLM natural language processing (NLP), specifically NLTK, to produce answers to questions asked about data in plain English.

Looking back, this project was a great insight into what LLMs immediately allowed that was incredibly difficult before. This project was several months of work that the openAI sdk would probably have allowed in a few weeks - and that few weeks would have been mostly frontend design and a bit of prompting.

Try it here: http://voicequery-dev.s3-website-us-west-2.amazonaws.com/ Github: https://github.com/hockenmaier/voicequery

The system uses natural language processing to produce answers to questions asked about data in plain English.

Raspberry Pi Control Panel is a hardware project I designed in 2016 to manage home automation systems. The project involved designing a custom 3D-printed case for a Raspberry Pi microcomputer with a touchscreen interface.

Links:

• GitHub
• Thingiverse