Making AI contact: Can artificial intelligence help aspiring cartoonist achieve his vision?

Making AI contact

CAN GENERATIVE ARTIFICIAL INTELLIGENCE
HELP A CARTOONIST ACHIEVE HIS VISION?

By Joe Dworetzky • Bay City News

Like all cartoonists, I suppose, my fondest hope was that a cartoon of mine would one day appear in The New Yorker.

In my case, it was all fantasy. I was in the bottom quartile — probably decile — of cartoonists. Gracing the pages of the New Yorker was as realistic as being drafted by the Golden State Warriors because Steph Curry saw me playing horse at the playground in the Presidio.

There were many reasons I was not going to the Bigs, but at the core there was one that could not be overcome.

… I couldn’t draw.

And for all of the 99 years the magazine has been around, that would have been the end of it.

But just in time for its 100^th year, there was a cosmic shift in the universe. Generative AI had arrived and with it came a new and startling possibility.

Could a cartoonist with middling skills in the drawing department, transform his game to play in the biggest of leagues, just by typing a prompt into the program and seeing what it generated?

Accidental cartooning

For me cartooning was something of an accident. In 2014, I wrote a Young Adult novel and the publisher agreed that illustrations would be a good addition. I approached a friend who is an artist and asked if she would illustrate the book. She did, and both the publisher and I loved the result.

But in the process of working with the artist, I was struck by how hard it was to explain in words — even for one who has spent his life working in that medium — how something should look.

I began to wonder if I could do it myself.

I couldn’t draw — I had never done more than doodle during long meetings in the 35 years when I was practicing law — but how hard could it be? Why not just teach myself to draw?

I decided I would draw faces at first because, well, they looked like they would be easier than the other possibilities, particularly hands. Hands looked brutal.

I began on Jan. 2, 2015, and vowed to post something every day.

I used water colors and ink to make the drawings.

I liked the colorful water-coloring part, but my actual drawings were dreadful, so far from what I had in my head, that I couldn’t just post them on their own. I decided to add a little slogan to each drawing — the sort of banal maxim that you might find on a T-shirt — just to distract from the hideousness of my artwork.

I kept the process going day after day, constantly wondering what on earth I had been thinking, why my work was so crappy, would it ever get better, and how could I gracefully abandon this lame project.

Cartoonish

I had been at it for about three months when it occurred to me that what I was doing was cartooning.

I had always loved cartoons — editorial cartoons, social cartoons, comic strips, comics — what a great tradition and ecosystem. And even better, cartoonists don’t need to be great artists. Many are of course, but with a good idea you could offset a weak drawing, or at least that’s what I told myself.

I began to try to become a cartoonist. Not a legit cartoonist, not for pay. But good enough that I could answer if I was ever on a plane in an emergency and the attendant shouted “is there a cartoonist in the house?

By the start of 2016 I had gotten into the daily challenge and looked forward to seeing what I would come up with. I experimented with different styles, though my drawing had not improved very much.

I kept up the daily posting and along the way I got a couple of breaks. Trump came along and I branched from social cartooning into political cartooning. An editor at SFWeekly liked my political views and my cartoons ran there for several months. I also found several homes for my social cartooning. I began to really enjoy it.

The only problem was the same problem: my drawing.

I thought about collaborating with a bona fide artist, but there was no money in it. How to find a creative person who could actually draw and convince him or her to join in the daily sport just for the thrill of working with me? Pretty unlikely.

But I kept at it.

For three years I posted a cartoon on my Instagram feed every day. I never missed — 1,096 consecutive daily posts — despite many trips and travels and all the drama of day-to-day life.

I stopped daily cartooning at the end of 2018 when I began a three-year stay as a very old student at Stanford. But I did not give up cartooning altogether. I was quite proud to cartoon for the Stanford Daily while I was a student.

My study at Stanford lead me to journalism. I got a master’s degree in 2020 and began work at Bay City News as a journalist in 2020.

Most of my time was spent as a reporter, but I continued cartooning at BCN and my work found a generous home on the Local News Matters website under the title “Bay City Sketchbook.”

And everything seemed great until the arrival of generative AI.

The artificiality of intelligence

ChatGPT, by itself, was a big deal in 2022 and as soon as it was available I began to play with it. It only generated text at that point, and while it was great for parlor games, I would never do more than fool with it. I was going to write my stories — journalism and fiction — on my own.

But one day, I learned that Adobe had something called “Firefly” that generated images from textual prompts and was integrated with Adobe’s Photoshop and Illustrator programs.

I was familiar with Adobe products (disclosure — my wife is on Adobe’s Board) and Firefly had a feature that allowed me to upload one of my own drawings and direct Firefly to use that drawing as a reference for the style it should use in generating an image. That seemed a good place to start.

I began to see if I could use Firefly to make a cartoon.

I fussed for a while, figuring out how to use the software. I found one of my drawings that seemed okay as a reference. I did not want Firefly to generate a title or caption — I could do that — I just wanted to see what sort of an image it could create.

For some reason I had snakes on my mind that day and I used variations of that visual in the textual prompts I fed into Firefly.

Within two hours I had a dozen images that could be used to create cartoons. I dropped them into Photoshop and added titling and captioning and soon had a gallery of AI generated cartoons.

I slithered from snakes to other topics. I experimented with the many style tweaks that the software offered me.

This time around my prompts used chess as an organizing principle.

I was blown away by the output.

It is charitable — to me — to say that the drawings were in my style, but for a minute it made the process feel more legitimate than if it was just Firefly riffing from the work of others. What astounded me was the fact that in the space of an afternoon I had learned how to use the software and created a dozen cartoons, any one of which I would have been proud to have drawn, if I even could have.

A few weeks later, I tried the same thing on OpenAI’s platform using DALL-E as a plugin for ChatGPT. At that time, DALL-E wouldn’t allow me to use one of my drawings as a reference, so I needed to write my prompts with the care I used in crafting a story to explain the style that I wanted to use.

As with Firefly, I quickly figured out how to use the platform to generate images more or less in accordance with my instructions, and after a few hours I had created several cartoons with drawings far beyond what I could have drawn even if given a month to work on them.

For one of the first prompts, I gave DALL-E a scene to envision.

I had written an academic piece that compared how T.S. Elliot and Einstein conceived of time, and I thought it would be interesting to see how generative AI would depict a luncheon meeting between the two geniuses. Again, I did the titling and the text in the speech bubbles, but DALL-E did all the work of generating the image. It even drew speech bubbles for me to use for the text.

Over the next few weeks, I experimented with a variety of cartoons and illustrations:

I put a disclosure/disclaimer on the cartoons: “Portions of this cartoon were created with generative AI,” which I later made more direct: “This cartoon was created using generative AI.”

I started sending AI-created cartoons to friends as novelties. I also created our family holiday card with the technology. This is the cover of an 8-page booklet we sent to our people:

As I worked with the platforms I found it easier and easier to make something that worked for a particular need.

Generative AI had essentially solved the great problem with my cartooning; now I could produce a good-looking cartoon without needing to draw very well, or indeed at all. And I could do it in minutes. I should have been delighted, but that was not how I felt at all.

I was thoroughly depressed.

Imposter Syndrome and friends

I worked through my feelings and I found there a number of different emotive threads, all twisted together.

One thread was guilt. I felt like I was passing off someone else’s work as mine. The facts didn’t actually support that feeling, for I was always clear that the work was created by AI, not by JD, but still, it felt vaguely fraudulent to get credit for a cartoon spun up by a platform.

Another and closely related thread was the feeling that I was plagiarizing from the work of another artist or artists. My giving credit for the drawings to the AI platform didn’t credit the actual artist or artists who created the work that the platform trained on. I had no idea who they were or how I could give them credit, but it still felt that they were entitled to be recognized.

There was also a significant legal issue embedded in that thread.

As a lawyer, I had studied copyright law and when generative AI surfaced I attended a copyright law seminar to see what I could learn about the application of the copyright laws to the new medium. The issue of using copyrighted training data is far from settled — there are a dozen cases in the federal courts, including a blockbuster case filed by the New York Times — where creatives who have had their work hoovered up to train the large language models that are at the core of generative AI have sued the platforms for the unauthorized use of their copyrighted work.

Generative AI had essentially solved the great problem with my cartooning. I should have been delighted, but I was thoroughly depressed.

The seminar presenters were skeptical that those cases would ultimately succeed because — in their view — the generative output isn’t a copy of any work, rather the underlying work is just what the algorithm has used to learn the rules about how new images can be designed. And whether or not that is the winning argument, most of the lecturers thought that platforms would either limit their training data to work in the public domain (and therefore not be subject to copyright) or more likely would sign large scale licensing arrangements with the copyright owners, eliminating the issue altogether. (Adobe’s Firefly was trained on images that Adobe owned or licensed. In one ad Adobe says Firefly is “lawyer-proof.”)

A further thread of feeling was that using drawings I had not drawn cheapened the work and made my role in creating it far less worthwhile, sort of like the difference between creating a cartoon and just forwarding it to someone else.

Wrapping all those threads of emotion was a feeling of embarrassment at the excitement I felt in being able to create quickly something that could do many of the things a good cartoon could do.

Prompt master

In the weeks that followed, I kept thinking about the legitimacy of using the new tools for cartooning. Then one afternoon I was out walking and listening to a book called “Dilla Time” by Dan Charnas and Jeff Peretz that recounts the life of the legendary hip-hop producer J. Dilla who, according to the authors, “reinvented rhythm.”

As the book warms up, it goes through the history of modern music, pausing for a long time on hip-hop and going deep on how it differed from funk and rap and other emergent strands of music. The authors described hip-hop as musicianship by artists who aren’t playing actual instruments but are using machines to make sounds at the direction of programs and algorithms.

They described the process of sampling a sound or beat from a recording and then going to work on that snippet with an electronic toolkit, blending, stretching, looping, layering, combining, trimming, all to the end of creating a new work that has been built in some part from samples of sound.

A person listening to the final work might not even be able to identify the sampled sounds, anymore than a builder could identify a single board in a finished house, though it is integral to the construct.

The authors made the compelling case that hip-hop producers are musical artists and hip-hop is music, even if the artists do not sing or play any musical instrument. Their musicianship is in the work that they do on sounds. The authors describe the output as “sonic collage.”

It struck me that using generative AI for cartoon images had something in common with hip-hop producers. The AI cartoonist need not draw at all, but uses a program to create a fresh piece of work. The platform is trained on original works of art and so the program must in some way use bits and pieces of the training data to create the new images, but there is an argument that the new cartoon made its own way into the world and would have legitimacy in the same way that a piece of hip-hop is a work of musical artistry.

Once that thought planted, I found supporting arguments. I heard people saying that the magic and artistry of the AI artist was found in writing the prompt. Sure, anybody could write a prompt, but there was a world of difference between a good prompt and a bad one. A good prompt created a rich detailed engaging drawing. A shitty one created confusion and nonsense.

In other words, writing the prompt was something that took artistry. The lawyers were even saying you could copyright your prompts. Maybe you could actually tell yourself that when generating images from text, you were still a cartoonist, just one who worked in the magic land of prompts.

But as seductive as that argument was, it proved too much.

Hip-hop used machines to make hip-hop, but that didn’t mean that simply using machines assured that the output was music, any more than xeroxing an oil painting made the copy machine operator a painter.

Back to the feelings

I marinated on all those threads for a while, but I knew I hadn’t really put everything on the table from an emotional perspective.

And so I went back and thought it through again and now I realized what it was that I wasn’t admitting to myself. There was indeed another emotive strand, and this one more powerful than any of the others.

I didn’t like this one at all, but there it was.

I was depressed by generative AI because now anyone with the program could do what I had struggled to do in those early days of figuring out how to be a cartoonist. You didn’t need to draw, all you needed was the platform. What had been kind of a cool, unique thing for me — even if at my level it was mostly a parlor trick — was gone overnight.

I remembered a cartoon I had drawn years ago, looking at it now I realized it was mocking me.

But as painful as it was to admit that it was my vanity that fueled much of my depression with generative AI, that wasn’t the worst of it.

The worst followed when I remembered how in all of the discussion about how autonomous vehicles were going to take away the jobs of truckers and all the others who make a living by driving, I was not particularly troubled. I didn’t want anybody to lose their job, but I didn’t think driverless vehicles should be barred any more than I thought that wizened old men in gold uniforms and braided hats should be required to operate elevators, as they had been when I was a boy.

I hadn’t shed a tear for the people who lost their jobs and their livelihood to technology; it was only when the technology came for cartoonists — for me! — that I felt the pain.

What a poseur.

More collaborative. Less sleazy

I could have stopped the inquiry there but a crucial question remained to answer: would I continue cartooning with generative AI? Would it be the magic bean that would take my work to the New Yorker?

For all the emotional baggage, there was no question I would keep experimenting in the short term; under all circumstances I wanted to understand the technology.

Then I saw a post that touted another generative AI platform — this one called Leonardo AI — and the post said it allowed a user to upload their own training data.

I tracked Leonardo down and sorted out its interface. One could indeed train the model. Instead of the single image I had uploaded in Firefly for reference, in Leonardo I was allowed to input 40 drawings. I found a gallery of profiles of modern writers I had drawn a few years earlier for a book on writing. The drawings were all in black ink and watercolor in the same general style:

I inputted the training images and waited a half an hour as the algorithm munched them up — I visualized a blender or maybe more accurately a garbage disposal. When it was done, I wrote out a detailed prompt and it spit out this drawing:

The app created something that was not actually my work but it looked at the same skill level. I will go further; it looked like I drew it.

I kept adjusting inputs and outputs. Another one, and this in color:

I loved it. I wouldn’t claim that it was my work, but it was generated from my work. On a really good day I could have drawn it. And it felt collaborative. I could even tweak it like any digital drawing.

I don’t think I would feel sleazy using this drawing in a cartoon.

And so the question remains, will I incorporate generative AI into my cartooning going forward?

I don’t know yet.

Struggling with hand drawn images has been a big part of what has given me joy as a cartoonist. But if I can find a way to really train the algorithm on my own work — not on 40 drawings but on 1,000 or 2,000 — I might be able to silence the voice in the back of my head that says that this is sleazy.

But does it even matter? Will there be cartoonists after generative AI? Even before AI, the days of cartoonists seemed numbered. Cartoons seemed creatures of newspapers and magazines and they were on the Endangered Species List.

If everyone could create a cartoon — AI was the democratization of cartooning — wouldn’t that just mean that the cartoons would be like GIFs or memes, made by millions of people all over the world and given away free to Instagram and the other platforms where all their value would be sucked out and owned by the platforms rather than by the creative people who brought them into the world? That’s what platforms did. At the beginning they created cool tools to help people create but, in the end, it was the platform — not the creatives — that made the money.

I choose to hope that won’t be the case. What I have learned from my years as a cartoonist is that while journalism can shine a powerful spotlight, a good cartoon is the flash of a single strobe light, bringing to the reader — for one brilliant instant — the essential core of the issue.

We should never lose that.