Thread by @reidscarboro

Using Unity to make an interactive art thing for an elementary school STEAM (science, tech, engineering, arts, math) event - progress thread! 👇

Context: My wife is a K-5 art teacher, I ocassionally help out with events to show kids how games and art overlap. For the STEAM event, she asked me to run a "tech" booth. Usually the booths are some sort of craft/interactive thing, and I have a big screen to work with!

Project requirements: Create a cool *thing* that showcases technology. Any input/activity needs to be simple enough for a 5 year old. We're expecting a few hundred kids to come through the booth over 3 hours. I have a big screen and a lot of table space.

The idea: "Put a Fish in the Aquarium" - we'll have worksheets with fish outlines that kids can color in, then I'll scan them onto laptop running a Unity game on the big screen. Unity game hot-loads images and puts them into an aquarium scene.

I did a similar booth a few years ago, with bugs instead of fish - the kids had a great time seeing their art jumping around on the screen, hoping to replicate and elevate that experience!

Loading images: Wrote some code that would keep track of all the png files in a directory. When it finds a new image, delete the background to isolate the fish using a flood-fill algorithm and Texture2D.GetPixel/SetPixel.

Once the fish graphic is isolated, I create a new Fish that's essentially just a square mesh with the new texture on it. I found "Flock Box DOTS" by Cloud Fine on the Asset Store to make the fish swim around - super easy to work with!

To give the fish a swimming animation, I used Shader Graph (for the first time, very cool!) to do some vertex deforming.

(Here's the Shader Graph)

Putting it together, feeling good so far!

Some Asset Store models (using the same vertex deform shader), terrain, fog, lighting/shadows

Stress test, and a decal to emulate light caustics from water ripples

Success in turning physical fish drawing into digital fish!

Was having some blur issues, but this Logitech c270 has a manually adjustable focus if you open it up 👍

The capture setup

Spawn bubbles! Because it looks cool, and to help the kids identify their fish as it spawns

Various fish for the kids to color - traced these outlines from the 3d models in our fishing game Tides

Final setup at school, kids are here in 30

Overall a success! Kids loved it - ended with about 260 fish. Everyone seemed to really enjoy seeing their fish swim around with the others, though it did get kinda crowded towards the end😁

And a final recording of all the fish!

#gamedev #madewithunity #unity3d