Thread by @everestpipkin

i have a new project launching today at thephotographersgallery.org.uk/whats-on/digit…

its a browser-based cycle of thousands of videos curated and altered from a machine-learning dataset. each is slowed down, interpolated, and upscaled immensely, one flowing into another

(it plays in an iframe over a companion essay from Adam Milner and the right of the site, so you can leave them playing as you browse & read, or full-screen it on a second monitor)

i proposed Lacework last summer, after becoming a bit stuck on MIT's "moments in time" dataset, which like so many others is intended to train AI to recognize action in video. it contains one million 3 second videos, each tagged with a single verb like asking, snowing or praying

there are a few things that are particular about moments in time. it tries to break down actions into just 339 doing verbs. it also doesn't clarify the /subjects/ of videos- it is more interested that something is flying than if that thing is a bee, person, satellite, or bird

the logic of the dataset is poetic at times. from the whitepaper: "three seconds is a temporal envelope which holds meaningful actions between people, objects and phenomena"; "events can be symmetrical in time ("opening" is "closing" in reverse)" moments.csail.mit.edu/TPAMI.2019.290…

i've honestly got a lot to unpack from making this work. this is because i watched pretty much the whole dang dataset - i curated the videos by hand.

it was .. brutal.

i did the bulk of this watching after quarantine started. as the world entombed, I spent hundreds of hours watching these tiny moments. two lovebirds beak to beak, a tiger putting paws around a keeper, the contact of a fist to a face, a kiss, a crowd, a touch

much of the archive is hard to watch- there are moments of extreme emotion or personal vulnerability- tears, screaming, and pain. bad jokes, lots of pornography. and much worse; some truly horrible images, which i did not curate into the work and i will not describe here

even the parts of moments in time that have real joy in them do suffer somewhat from the context- they have been scrapped, aggregated, condensed, algorithmically cut to 3 seconds, and deemed a verb

for some reason, i had expected the act of watching the dataset to be calming or exploratory, like watching the world out of a window

but this is not an archive of beauty, or even an archive of ambiently being in the world. it is an archive with purpose, an archive of actions

there is a condensed functionality to it. here are things that are done in the world. it is not entertaining or poetic or beautiful, even though things that are that are contained within it, even though those were what i was looking for. watching it felt raw

around hour 250 of watching the dataset, sometime in april, i started having The Dream

in The Dream i am living on- or perhaps am- a satellite, orbiting a planet and monitoring planetary broadcasts directly under me for the brief moment i am in that place. just a fleeting, tiny vision of <something happening><something happening><something happening>. for years.

even awake, i've come to see patterns in camera quality, in shadows, in the color of paint and clothes and the types of trees, in movement, in texture that could unfold into ever more detail

i see patterns in everything, patterns that unify all of the videos, which have nothing to do with their subjects and everything to do with the lens sweeping to capture the running dog or the light flaring or the compression running up the edges or the logos superimposed on top

i wonder if this is how a sorting algorithm feels

i’m using topaz lab’s proprietary software ai gigapixel to do the bulk of the upscaling, which has a super-easy-to-use command-line interface. it was trained on another dataset of pictures at various resolutions. it only sees patterns too

this process has been described as hallucinatory, which is an accurate marker - it is a recurrent looking, a push in and in for ever more detail which then spirals into something else entirely

like looking at images of the earth via satellite

zooming in so far that you can see the lines in the dirt left by the tractor, by thousands of tractors, the ways in which they are interrupted by the landscape, the boundaries of jagged creeks, the wind, the dunes, the oil wells, the ocean

all these millions of lives, this infinite detail, this lacy intricacy that grows ever more granular the closer you get, then grows again, and again