At MDandMe, we often joke that art is my main edge over AI. As a Harvard-commissioned portrait artist, Canadian Portrait Society national champion, and guest instructor at the Harvard Art Museums, I must admit I have looked down on AI's capability to produce "real" art and in a sense dismissed it all as "AI slop". Last week, this all changed.

While writing a series of over-the-counter guides to help MDandMe users manage simple medical issues at home, I turned to DALL·E 3 via our enterprise ChatGPT plan to generate cover images because of the sheer volume of images needed. After initially little success, our front-end software lead Whitman suggested I try using a “line drawing” style. Here’s some samples of what we ended up settling on:

Sketches generated with DALL·E 3 for our recently started series of over-the-counter treatment guides.

Where to Start

Suppose you want to draw a picture representing “insomnia.” Telling DALL·E to just “draw insomnia” is like taking the whole internet of photos related to insomnia and combining a random sample of them. The result looks like the typical eerie, cartoonish, unnaturally smooth images that has become the typical expectation for AI “art” (which many would argue isn’t true art); however, it only takes a quick search on Google images to see where the inspiration is coming from.

Google Images search results for "insomnia" vs ChatGPT DALLE 3 rendering of insomnia.

Step 1: Specify the medium

If you want predictability or consistency, the first step is to pick a medium so that we get into the desired corner of the internet. Are we talking about a cartoon, a sketch, a painting, a photograph, a sculpture, an ink print, or abstract art? I chose “line drawing sketch pen and ink style with minimal shading”, with the combination of 4 descriptors to guide the AI away from its smooth photo-but-not-quite tendencies. See the difference below.

Prompt: Line drawing sketch pen and ink style with minimal shading of insomnia

Step 2: Specify the style or the audience

Beauty is in the eye of the beholder, so if you know what type of specific style you are looking for, specify the type of place that style is likely to appear. Is it a fashion magazine, an art catalog, a newspaper cover, or a scientific journal? People have also found that mimicking specific artists’ styles can work, although obviously this is controversial due to copyright concerns. Avoid 3D terms like “art gallery” or “mural” if you want a 2D image without other objects like frames, buildings, desks around your artwork. I chose “in the style of an art elegant magazine.”

Prompt: Line drawing sketch pen and ink style with minimal shading of insomnia in the style of an elegant magazine sketch

Step 3: Tell it to be more human

I found that the easiest way to bring out imperfections, rougher lines, and more disorganized shading was just to say “make it look like a human artist did it” or even better, a talented human artist. Again, I thought of this technique as forcing the AI to only draw inspiration from a small higher-quality subset of its training data. In fact, if the first image generated had good form but poor art quality, I would just say "Great! Keep this form but make it look like a talented human artist did it” as a follow-up prompt. 

Prompt: Compose an artistic black and white line drawing sketch of insomnia trouble sleeping in the style of an elegant magazine sketch, with minimal shading. Make it a human-drawn pen and ink style with some cross-hatching.

Step 4: List what you don't want

A lot of images used to train the models might contain text, which can degrade the quality of an otherwise good generative image. Specify “never use text” and “background needs to be white.” For my drawings, I also specified “don’t show face” although it often didn’t listen. 

Step 5: Repeat until satisfied

Quality with DALL·E 3 is highly variable, in part because ChatGPT first auto-decides on a more specific prompt based on your initial prompt. I found that I would need to try 2-30 times before I got an image I was happy with. This can take a fair amount of time (10+ minutes) but is much, much faster than drawing from scratch would be. Due to the level of stochasticity is no need to change your prompt unless your images are nowhere close to what is desired after 3 consecutive tries. Start a new chat if you do not like the generated image at all.

The insomnia drawing I chose (which was attempt 8 using more or less the same prompt) had a pen on top of it, probably mimicking photos that artists take of their artwork along with the tools used.

Step 6: Touch up with AI spot editing

You finally got an image that you are impressed with. Congrats! You are happy until you realize that your beautiful drawing has an extra finger, a physically impossible detail, or a frame and a little bit of wall. Fortunately you can touch up an individual area to edit. In the case of my insomnia drawing, I needed to remove the pen that was in the original generated image (probably mimicking photos where artists include their tools along with their art).

Backup step: Touch up manually. 

There are some details that DALL·E 3 is unable to get right, even after an obscene number of tries. These include:

• Internal anatomy. Organs diagrams are very much off the table right now, unless you want it to look like a xenomorph from Alien.

• Rashes and external abnormalities. These tend to be highly exaggerated and medically inaccurate.

• Generating an missing finger when it initially generated too few. It tends to be much easier to just add it manually on Procreate or another drawing app.

• Specific parasites, bugs, microbes: Anatomy of non-animal critters tends to be quite inaccurate and unrealistic, looking a bit like an old horror movie.

• Blood pressure monitors: Interestingly, DALL·E 3 consistently generated a mix of an electronic BP monitor (which doesn't require a stethoscope or inflation bulb) and a manual cuff with stethoscope and inflation bulb all in one image. This probably reflects online pictures showing both types as "blood pressure monitors" and the AI not realizing that they indeed both exist but don't co-occur.

• Numbers and letters: Remain a challenge for DALL·E 3 but it is improving

Template prompt: 

Compose a [describe the style of art you desire in at least 3 ways] of a [brief description of subject] in the style of [type of publication you would expect to see the art in]. Make it look like a talented human artist did it in [repeat style with 1-2 more details]. Never use text and don’t [something else you don’t want]. Background needs to be white.

Two Takeaways

1. A key principle in art that I like to remember is: what the eye cannot see clearly, the mind fills in more vividly than reality. You will notice that most artwork, close up, contains purposeful imperfections and color choices you don’t see in reality. That is the essential quality of art that can allow it to capture a moment in time in a more authentic way than reality itself. The same applies to AI. Aim for less detail and you might get something more authentic.

2. The second theme of AI art that I learned was that, for now, if you want to use it as a base for your own artwork, you need to be prepared to “let it do its thing.” People who commission artists may understand this feeling well. When you commission an artist, you don’t necessarily know exactly what you want. You trust that a good artist will be able to use their judgement to figure out what looks best. As an artist, I pride myself in being able to produce exactly what I envision. With AI, I almost never get exactly what I want, but once in a while (usually after 5-25 attempts) I can generate something impressive, better than what I envisioned, using a simple model like DALLE on ChatGPT. Humbling and a little scary, but also inspiring and useful. 
   
   

   ---

William co-founded MDandMe while an MD Candidate at Harvard Medical School. In his spare time he paints for Harvard facilities and teaches for a few museums. Please email william@mdme.ai with any questions or comments you may have.