From Vector-space to the Fluidity of the Written Signs: the Aesthetics of Chinese Visual Embeddings

As machine learning permeates every facet of our lives, the vector space is no longer alien to digital media artists as well as the electronic literature community. Due to the existence of a wide range of easy-to-learn tools dedicated to word vectors in popular programming languages, word vectors as a tool has become approachable for artists and writers. Creative projects are made by either utilizing word embeddings or synthesizing them to generate vectors for a sentence or even an entire novel. However, most research and creative attentions are still focused on the semantics, while other aspects of language, such as phonetics and visuality are rarely investigated. Further, among the few artistic attempts using vector space in terms of the non-semantic features of language, Alison Parrish has generated poetic phonetic similarity vectors using the feature engineering method. On the other hand, Gene Kogan has explored the latent space of Chinese handwriting using the deep convolutional generative adversarial network (DCGAN). It is with this background that I began to turn my focus to the visual similarities of the written language and trained a set of Chinese visual embeddings with Convolutional Neural Network (CNN), a method that is usually used by the machine learning community for image classification. By treating each Chinese character as an image, this approach attempts to find visual similarities among the written signs that lie outside the traditionally accepted rules of the Chinese writing system.With the major contributions of the talk being Chinese visual embeddings, this presentation is going to engage with the project based on technical details, aesthetic concerns and conceptual discussions. The technical component of the talk will address how and why the Chinese visual embeddings were trained using the proposed data set and the training process that makes it different from previous attempts. Following the technical details, demonstrations of the Chinese visual embeddings will be shown through multiple computational sketches made as part of my work, Graphein. Finally, the conclusion will encourage reflection on our own relationship with language as an imperfect system as sometimes, we are so familiar with the language we use that we take granted of the grammar and rules. Thus, with graphological and critical attention focused on the inner visual structure of Chinese characters, both Chinese visual embeddings and the project Graphein are attempts to bring out the visual poetic patterns embedded within the written system and seeking new connections and fluidity through the fresh ‘eyes’ of the algorithm. The last section will engage the project with critical discussions of language through the lens of digital technology. Although language is discrete on most scales, especially in the written form, the project attempts to explore the visual fluidity of written language by focusing on the graphological structure of Chinese characters with the help of machine learning methods. The aesthetics of smooth visual transition can then be used to defamiliarize a writing system and bring new insights to the materiality of the written language. Embracing the subtle space between reading and seeing, it encourages reflections on language as a socially created imperfect system through the practice of writing (graphein) collaborated with algorithms.