Unraveling the Code Part 1: A Journey between Abstract Language and Visual Abstraction
March 25, 2023Writers, in my eyes, are the most amazing artists. With a mere arrangement of words, they can transport us to entirely new universes. Our imaginations are vast, but without a means to manifest them, they remain fleeting memories. Words provide that direct path to tangibility.
Over the past few years, I’ve been captivated by words - the letters that form them and the power they hold. I wanted to create a series of works centered around words as either the medium or the resulting art itself. Since I’m not a wordsmith, writing a novel or anything long-form was out of the question. Instead, I sought to design a translation map that could transform letters into geometric shapes. This would provide me a way to either conceal messages via code or imbue a canvas with literary significance.
Last May, I embarked on my first attempt, handcrafting unique shapes and color combinations from cardstocks to represent each letter of the English alphabet. I then explored various arrangements, hoping to create something visually striking. The result? An image representing the phrase “LOVE IS A DRUG”, marking the first iteration of my new series: CODED LANGUAGE.
Initial versions of my work tend to be raw - like seeds germinating in Styrofoam cups before being transplanted into a garden. As a visual artist, my next step was to refine the concept into something more appealing. Using Photoshop, I reimagined the shapes and their corresponding letter assignments. I also took the liberty to arrange them in a way that created a balanced composition on the canvas. This experimentation gave birth to Version 2.
By the time I reached the second iteration of this project, I’d discovered Disco Diffusion, the precursor to Stable Diffusion. I spent days creating AI-assisted art with DD and soon began exploring how it could enhance my project’s visual quality. Artwork crafted through Disco Diffusion often exudes a psychedelic aura, perfectly fitting for a piece called “LOVE IS A DRUG.” Below are the results of running the first two pieces through Disco Diffusion.
Upon embarking on this project, I knew there was much to learn in order for me to reach a level of satisfaction. I also recognized the project’s nature demanded a generative approach. With this in mind, I delved into the world of generative art and its underlying processes. My research led me to P5.js, a popular JavaScript library used by many artists in the field. As a software engineer, programming is second nature to me. So, I seized the opportunity to learn P5.js through this project.
At this point, I also changed the name of the project to ABSTRACT LANGUAGE, as I felt CODED LANGUAGE was too on the nose. After a couple of weeks honing my JavaScript skills and figuring out the P5.js library, Version 4 came to life. Though visually underwhelming, I consider it the first seed planted in the garden. My goal, from there, was to develop it further, eventually sharing it with the world as a way to fully commit to its evolution.
For Version 5, my aim was to refine the algorithm, hoping to generate visually striking art from any inputted words. However, this proved challenging, as longer strings like “WE MUST LEARN TO ACCEPT IMPERMANENCE WHILE STILL ENJOYING EVERY MOMENT” caused shapes to overlap and create a massive, indistinguishable blob. To address this issue, I experimented with cutting out the insides of some shapes, but the improvement was minimal. Nevertheless, this iteration still represented a significant step forward from the previous version.
Unsatisfied with the project’s visual presentation at this stage, I looked into alternative ways to enhance it beyond coding. This pursuit led me back to AI, specifically Stable Diffusion. By then, I had already fine-tuned a diffusion model using works from my previous explorations in both traditional and digital art. I’ll delve into this part of the process in another blog post. With my model, I began generating AI-assisted pieces adjacent to each generative art piece from my algorithm. I used the generative pieces as a base, running them through the image-to-image process to maintain some of the original piece’s fidelity while pushing the concept further into abstraction.
This phase also prompted me to putting more consideration into the text fed through the P5.js code, as I wanted to incorporate it into my Stable Diffusion prompt. So, I decided to experiment with haikus. I asked my girlfriend to compose one for me to use, and she crafted the following:
Her world was her own
To mold like clay in her hands
She would mold chaos
With everything in place, this project bore its first truly satisfying fruit.
Despite my significant progress, I remained haunted by the fact that any text beyond the length of a haiku resulted in a chaotic mess of colors for the generative piece. After spending a few more weeks diving more into P5.js, I opted to make some substantial modifications to the code. I altered the shape functions to follow a more uniform pattern and arranged them on a grid. This approach provided an efficient solution for addressing the overlapping issue, as the shapes were no longer haphazardly placed. I was also able to resize the shapes based on the canvas size and text input length, resolving the problem of shapes spilling out of the canvas in previous iterations. These adjustments and others paved the way for Version 6.
Upon completing Version 6, I decided to take a break and shift my focus to other creative endeavors, allowing this project some room to breathe. During this break, I began exploring ChatGPT’s potential as a coding assistant. I experimented with creating a couple of Javascript projects, harnessing the power of ChatGPT to streamline the process. I knew this newfound expertise would prove invaluable when I returned to this project… In Part II of this blog series, I will delve into how ChatGPT played a crucial role in helping me reconstruct this project from the ground up at a remarkable pace, ultimately bringing it closer to my original vision. In Part III, I will share insights on my approach to fine-tuning diffusion models for use with Stable Diffusion, enabling the creation of artworks inspired by my previous creative explorations.