Stable Diffusion

I’ve played with Stable Diffusion a few months back, but the learning curve is steep, and easy access to DallE & MidJourney made it not worth it to press on. Until… I got excited about a super cool extension:

https://huggingface.co/monster-labs/control_v1p_sd15_qrcode_monster

I just love the idea of having the same image have one analog meaning to a human and discrete meaning to a machine. And the same extension can be used for general masking other than just QR codes.

Without further ado, here’s the eye candy:

A benign QR code

General masking for an almost subliminal effect

There’s much to be refined in my incantation, but I really like how the mask isn’t just overlayed on top of a drawing in a subtle way, rather it’s the basis for growth of AI hallucinations.

Mindstorm Gondola Plotter

So out of nowhere Robin designed a gondola plotter with his Lego Mindstorms. There are many crazy pants projects I hear about every day. A lot aren’t attempted, most scratch an itch and finish half baked, few make it to top notch level. When Robin talked to me about his plotter, I was excited but skeptical it would turn into much. I also expressed some of the issues he might have with the early model. Lo and behold he finished the day with a very well made machine he threw together in 1 shot. We both got every excited when it moved for the first time.

I was on a mission to try and explain to him the double pythagorean theorem that gives these machine a cartesian coordinate system. We launched into it the next day so that he could translate it into Scratch, it was a good back and forth where we both managed to bridge each other’s understanding. I was very impressed by his ability to understand what was going on and problem solve bugs, he totally got it.

We ended up with this ugly code block, I have no idea if there’s a better way to code this in Scratch:

The machine became perfectly addressable on a cartesian system. Seeing this many first try successes, we got all the more excited and I told him I’d find a way to import SVGs in there. He didn’t believe that I could, and I tooted my own horn a bit saying I used to be a very good hacker. I had to deliver now… I couldn’t find a way to import anything in the neutered version of Scratch that Mindstorms run so I’d have to figure out a way to programmatically edit the *.lms file that is the project to inject the code blocks. Turns out, it’s just a bunch of files zipped up, including one called scratch.sb3, itself a bunch of files zipped up, including project.json. Browsing around at the data structure in that file, I found the doubly linked lists of objects which represent sequences of commands.

Trying to inject my own

I write some code to create these JSON blocks so that I can inject as many go_to function calls as there are instructions in my GCode converted SVG. Zip everything back up and we are in business. Robin totally thinks I can hack the FBI now.

So I convert an SVG into 32,000+ commands, and well, the Mindstorm app crapped its pants. I went a little easier with about 1800 and it had to think a while but it did it. Resulting in a hardly manageable project with a large go_to vomit in the middle of it. But it worked!

If you stream the commands… There isn’t enough room on the brick otherwise.

Finally, again with few obstacles, the Mindstorm Gondola Plotter gets to work on a simple Millenium Falcon.

It doesn’t make the calls to pen_up and pen_down yet, that’ll be a formality. It also has the very classic issue with moving on motor after another. Robin understands that we can simply break down the paths into more coordinates, but he may try other approaches.

For posterity, the first actual drawing (it looks a lot better than my first plotter drawings)

I absolutely love the design where all the logic is contained in the drawing apparatus, and the plotter is hanging on 2 fixed point. This was a bit forced by Lego in that you can’t have your motors too far away, but it’s really slick.

tel père tel fils

He was very proud of himself and I was too. I always prime him for the engineering process where a merciless series of failures precede any progress, but this project was very friendly at every step. Robin did spend a LOT of time working on various lego guns. I thought it was educational because of how long he had to spend on the fine mechanisms for moving, spring loading and releasing missiles. Clearly he learned from it given how easily he threw a complex machine together.

Warming Up

I’ve recently gotten the ok to deploy a large gondola plotter in a public place. Theoretically, there is not limit to how large a gondola plotter can be. Practically, there are considerations to how large they get. I’ve been working on solving the various issues that arise from having a 10′ deployment. Of course with the ~20% margin to avoid extreme positions, it’s not 10’^2 drawable. Still though, there are many challenges due to the scale. Finding paper big enough is a challenge, so is moving it without damaging it. I figured out other quirks, it’s boring, let’s skip to the eye candy:

I have several public facing experiments lined up for it over the next few months or years. I’m not in a hurry. This is just a warm up to figure out what it means to deploy it in a public spot. What’s fairly clear though, is that there isn’t much I can do to make this better, and so this may be the culmination of a 5 years development effort.

That’s 3 days of the machine working non-stop, I’m watching the ink level closely and hope it’ll be done in a couple more days.