The deep learning behind robotic opera singing.

This is Shimi. Shimi sings opera.

To develop Shimi’s behaviors, a deep neural network ingested the following: 10,000 files from 15 improvisational musicians playing responses to different emotional queues; 300,000 samples of musical instruments playing different musical notes, to add musical expressivity to the spoken word; one of the rarest languages in existence—a nearly extinct Australian aboriginal vernacular made up of 28 phonemes.

But what is the goal of it all?

In a world where we are surrounded by robots who need to communicate their state of mind, mood and emotion, prosody, and gestures seem like a great subtle back channel to do so, since humans cannot really process multiple linguistic channels effectively. So our long term goal for this research would be to scale our system to large groups of robots.

Mar 6, 2019

More Headlines

It was exciting. So I guess I didn't feel anything negative. And afterwards I definitely got a lot of messages from people that I've known for a long time—catching up, saying that's pretty cool.

Interview with the Uber engineer who recently faced Mark Zuckerberg in a jiu-jitsu tournament.

↩︎ GQ

Greenpeace says recycling actually makes plastics more hazardous to human health.

↩︎ dezeen 2d

An investigation finds unreported attacks that killed or wounded hundreds of Cambodian civilians during Kissinger's tenure.

↩︎ The Intercept 2d

It’s the hyper-feminine romanticization of your wardrobe, with extra lace, frills, pearls, and silk pieces that represent the idea of a European vacation more than they reflect actual European fashion trends.

American TikTok users experience “Europecore” by turning off their air conditioning.

↩︎ Nylon

A cosmetics company and an influencer get knocked for using a school shooting to sell a beauty product.

↩︎ High Snobiety 3d

Somes notes on how the ProPublica reporters dug up and broke the Clarence Thomas story.

↩︎ Slate 3d

Some variations of incomplete cubes by Sol LeWitt (and others). That is to say, "all the ways of the cube not being complete."

↩︎ Flashbak 6d

“Ever breathe a frequency?” Report from a recent sound experience in France where alphorn players met Catholic protesters.

↩︎ Pitchfork 6d

“Personally, I think ‘eat the rich’ is quite acceptable right now.” Recent signals from Rosecrans Baldwin’s travel diary.

↩︎ Meditations in an Emergency 6d

Doxa means common belief or popular opinion and comes from verb δοκεῖν dokein, “to appear”, “to seem”, “to think” and “to accept.”

A dozen terms from Ancient Greek that are perhaps worthy of a comeback.

↩︎ Classical Wisdom

A man holding up a sign on the street saw “greatness” in the Los Angeles Lakers when no one else did.

↩︎ The New York Times 2w

The secrets of an obituary writer. "For a newspaper beat, there's nothing like it.”

↩︎ Meditations in an Emergency 2w

A video inquiry as to why so many hair and nail salons feature “Nagel women.”

Maya Bernstein-Schalet recalls the pandemic through dishes cooked.

↩︎ Waxing & Waning 3w

A selection of paintings and extremely detailed preparatory drawings by artist Nathan Walsh.

↩︎ Booooooom 3w

Photographs of “daredevils posed at Yosemite’s cliff edges in days of yore.”

↩︎ California Sun 3w

How to make people appreciate you more during Zoom meetings. "Loudness, it turns out, isn't as good a metric as intensity."

↩︎ Business Insider 3w

How to preserve a medieval Italian ghost town. "Members also take on low-cost maintenance activities such as weeding."

↩︎ Atlas Obscura 3w

Maybe the butterfly is the current “it” animal, taking on the title from owls and llamas and sloths and unicorns. In 2021, birds captured our attention as we all felt caged. Now, newly hatched from our cocoons, we have space to fly.

Some musings on why so many people, celebrities included, have a butterfly tattooed on their sternum.

↩︎ The Trend Report

The cutting edge of coffee drinks features "freeze-distilled" and "cryo-desiccated" milk.

↩︎ Sprudge 3w

See More From TMN