Can you tell us a bit about how your project started? Why a robot artist, exactly?
Ivan Berman: It all started in the summer of 2017. We needed to showcase the use of robonomics, which is the intersection of robotics and finance. In short, several years ago our colleagues Sergei Lonshakov and Alexander Kapitonov launched a project aimed at integrating blockchain and cryptocurrency into intelligent devices – robots, sensors, or any other devices that can be connected to the internet and given tasks, all the while earning cryptocurrency for their work. We searched for different ways to prove the viability of this concept; one such example was the robot artist.
Alexei Ovcharov: First and foremost, we wanted the project to use neural networks in training so as to introduce an element of chance and thus “humanize” the process as much as possible.
The initial idea was to collect a data pool – popular YouTube videos, tweets, Facebook posts, Instagram photos – and analyze it by running it through a neural network and forming a general picture of what’s happening in the world. Unfortunately, problems quickly emerged – we couldn’t gain access to the necessary data. So we decided to do things by hand using Microsoft Azure (a cloud computing service for building, testing, deploying, and managing applications and services – Ed.). For instance, we would take a text or a picture and extract from them a single word that describes them in general terms.
We came to a point where we’d have words that contained quite a lot of information – ones that could be used, say, to describe everything that’s happening all over the world right now.
IB: At first, we used tweets, literally collecting the first batch for testing by hand. Recently, a colleague of ours has developed software to do the same with Instagram: it analyzes a number of posts using the same hashtag and then uses Microsoft cognitive services to make up a list of keywords. Then, it picks one word from the pool and translates it into a Japanese hieroglyph. Then, it’s converted into an image and sent to the robot to be drawn.
AO: Hieroglyphs are very popular in Russia; most people have no idea what they mean, but they find them appealing. From a technical standpoint, we found it interesting to learn how to draw them properly. There’s a whole science, an art form, about how to do it right. To us, it was a fascinating task from both a technical and an artistic point of view.
At first, the robot would draw using individual square-shaped strokes, but the initial sketches and tests didn’t look very presentable. We decided to pick a specific brush type and teach the robot to draw at the proper angle – and it learned to draw the entire hieroglyph at once.
Does the robot only know how to draw hieroglyphs or are there other subjects? Can it create a painting in the traditional sense of the word?
AO: Some two to three months ago, our robot did a special drawing for an event in the US. It was the logo of Techstars, an acceleration program our bosses take part in. It was a proper drawing, but not an artistic one; it didn’t have any gradients or fades, for instance.
Vadim Manaenko: The robot is in Tolyatti at the moment, but it will soon be shipped here, to ITMO University’s building on Birzhevaya Line in St. Petersburg. We’ll assemble a developer team to work on it. Using the knowledge we’ve already got, we’ll further improve on its learning algorithm and drawing skills.
One of our ideas is to make it fully autonomous: it will paint independently, earn money, cover its expenses, and even order canvases and paint. Basically, it’ll be its own independent entity. Our engineers will help hone its skills – perhaps it’ll start doing paintings, too.
How exactly does the robot make money? Does it draw hieroglyphs to order or sell existing images? Where do you sell its works and how do people find them?
IB: In late 2018, we held an auction to test the system. It was done using a decentralized blockchain app and cryptocurrency as the payment method. In essence, it was a chatroom where we’d post the lots, and payments were made through the app. Then, we’d get in touch with the winners to find out where to ship their purchase. For now, this is still a part of the process where we have to help the robot, as there is no automated delivery system yet. But we can imagine a future scenario where the paintings are shipped by, say, a delivery drone.
VM: With our robot artist, we wanted to examine a real-life scenario: there are artists who draw according to their clients’ wishes, and there are those who create from their imagination. We wanted the robot to create its own works and then auction them off. The winning bidders are publicly given the rights to whichever painting they purchase. One of the advantages of blockchain is that the data stored within it can’t be changed or forged. When a deal is made between the robot and the human, both parties sign their messages with individual keys. Later, we can access the public blockchain and find out who owns the painting. This way, the buyer receives the physical object as well as a record of ownership that may not have legal power right now, but will probably be recognized as such in the near future.
What’s your answer to the ethical question of whether this is an exploitation of the robot’s artistic abilities? What about the copyright? Does it belong to the robot, the AI, or their creators?
IB: The very idea of making a self-dependent robot artist is about how people interested in such matters have to pay for its work. One of the project’s goals is to demonstrate the possibility of an autonomous future where capital is generated by machines in closed cycle: the robot draws, the robot sells, it receives payments, ships orders, purchases paints and canvases.
Of course, there are ethical points of contention. Regulators and adjacent scientific communities are quite skeptical. For instance, there’s a book titled Ethically Aligned Design, created by the IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems, the core message of which is that robots cannot be given freedom and autonomy. But history shows us that you can’t just up and forbid people from doing something. Someone at some point will eventually do these things out of moral conviction or simple desire for profit. Sooner or later, things will get out of control. And it’s better for us all to be ready for that.
Our research advisors and our colleagues from MIT Media Lab believe that such an approach – involving cryptocurrency, security, and, most importantly, transparent human-robot relations – can improve the situation. It would be way worse if these new relations were to emerge suddenly and people would have to literally fight for their place in the new technological world. I’m for taking early steps towards such a synthesis of humanity and machines. It’s for our own benefit.
Right now, robots aren’t “smart” enough for us to empathize with them, so to say. But sooner or later, they will learn to imitate human behavior, at the very least. And human nature is such that some of us will feel compassion and even pity for them.
AO: There’s already some dissonance in the field. There are many questions of ethics in robotics. The same questions surface in regards to neural networks; there’s even a saying: “I'm not scared of a computer passing the Turing test. I’m terrified of one that intentionally fails it.”
For instance, Elon Musk, who founded OpenAI (a non-profit research company that aims to develop an open and friendly AI), often says in interviews that he considers neural networks unsafe, that they can somehow harm humanity. That’s why he’s trying to get into that field not out of business interest, but in an attempt to regulate neural networks.
Our task is the same: we want to build a system capable of self-regulation and independence that would also be safe for humans. This is largely beneficial to society. If a company goes automated, such as setting up an autonomous accounting system, they could save on expenses related to all sorts of financial activities and also reduce paperwork. At the same time, the system wouldn’t be completely independent from humans; its own welfare will depend on us, and its actions will be transparent to outside observers since a robot doesn’t have a personal bank account. But it’ll be able, for instance, to withdraw from the company’s account to conduct its own repairs.
Tell us a bit about your collaboration with the MIT Media Lab.
IB: A few years ago, our boss Alexander Kapitonov was at a conference where he met Eduardo Ferrer, a researcher at that lab who works with blockchain-based swarm robotics. Because when you’re dealing with a lot of robots, security really becomes a major issue. And one solution is to use blockchain. Alexander had an invention, and Dr. Ferrer had interest in it. Since then, Alexander has been going there at least once a year for the Symposium on Blockchain to promote our ideas and find partners.
Why were the MIT Media Lab interested in us? Because they are the kind of lab that keeps an eye on trends that may change our lives in the future. They take the different fields of science and art and try to create projects that will one day become part of our lives in the most peculiar ways. They call themselves an “antidisciplinary” laboratory. In a famous example, they helped with the development of the LEGO Mindstorms range of programmable robotic toys. It has since become popular not only as a learning material, but also as an experimental medium. It might have seemed like a regular toy, but it turned out to be very convenient for testing out robotic technologies.
All in all, they became interested in our project because such technologies – self-dependent robots capable of replacing human labor – can truly have a major impact on the future of humanity.
As part of our collaboration with the MIT Media Lab, we’ve already published one joint article and are preparing a second one. The former is about new robotics services that could be made using robonomics and blockchain; the other is about the robot artist specifically. We’re also planning to invite MIT staff to ITMO University to expand on our international collaboration.
And aside from research papers, what other plans do you have for the future?
IB: We’ve long wanted to make a robot sculptor, and a couple of months ago we thought that it could make not just sculptures, but two-dimensional reliefs too. We don’t know yet what it’ll be, which material we’ll use for sculptures, or whether it’ll involve 3D printing. The only thing we need are people who are interested in making such projects.
