14.6 C
New York
Wednesday, April 10, 2024

'Klara and the Sun' Imagines a Social Schism Driven by AI

logoThe AI Database →



Face recognition

Human-computer interaction





Machine learning

Machine vision

Kazuo Ishiguro’s latest novel, Klara and the Sun, presents us with a world in which not one but two kinds of artificial intelligence have arrived.


In the book’s strangely familiar near-future, AI has upended the social order, the world of work, and human relationships all at once. Intelligent machines toil in place of office workers and serve as dutiful companions, or “Artificial Friends.” Some children have themselves become another form of AI, having had their intelligence upgraded via genetic engineering. These enhanced, or “lifted,” humans create a social schism, dividing people into an elite ruling order and an underclass of the unmodified and grudgingly idle.

Klara and the Sun, Ishiguro’s first book since he won the Nobel Prize in Literature in 2017, builds upon themes that recur in his previous work—loss and regret, sacrifice and longing, a sense of reality unmoored. But technology takes a more central role, and Ishiguro uses artificial intelligence, both biological and mechanized, to reflect on what it is to be human.

This vision of the future also speaks to Ishiguro’s feelings about the present. As he tells WIRED, the novel was inspired by recent technological advances, and a desire to understand where these advances could lead humanity.

Ishiguro spoke to WIRED from his home in London via Zoom. The following transcript has been edited for length and clarity.

WIRED: Klara and the Sun looks at what happens when some people can be enhanced genetically—something that may soon be possible thanks to the Crispr technology. How did you become interested in that?

Kazuo Ishiguro: I first heard about it when my New York literary agent sent me a clipping. This was when the first real breakthrough had been made by Jennifer Doudna, and I immediately thought, wow, this is going to do interesting things to our society.

If you’re of my generation—I'm 66—a lot of what's happened in the world has, in one way or the other, been to do with struggles against unjust hierarchies, class systems, colonial systems, castes according to skin color. When I heard about this, I thought, well, actually, it's going to make a meritocracy something quite savage.

But I was excited by it as well. I was lucky enough to meet Doudna, in 2017, at a conference in London, and I got very interested in the whole thing.

Crispr is an absolute breakthrough because it's so accurate, it's relatively cheap, and it's relatively easy to do. This means that its benefits can be with us very, very quickly.

Already, I understand, there are people who have been cured of sickle cell and other blood related illnesses. The possibilities, in terms of medicines and also in terms of producing food, are enormous.

But by the very fact that it’s relatively cheap and relatively easy to do, it’s going to be very hard to regulate. And I can see that a lot of the energy behind Crispr is in the private sector. It’s not under the traditional government or university auspices, so oversight is going to be difficult.

My question is, how do you build the platforms for debate and discussion in our society so that everyone can participate in the discussion? I think it's kind of odd that people are not more aware of it. People seem to be much more aware of AI, and people like to talk about that.

I was about to ask about AI actually, since Klara and the Sun is set in a time of intelligent machines. Are you similarly excited—and also troubled—by recent progress in artificial intelligence, things like AlphaGo?

Well, AlphaGo has been superseded several times. But what was interesting about AlphaGo’s success against Lee Sedol, the Korean Go champion, a few years ago, was the manner in which it won. It played in a completely different style. It made moves that made people fall about laughing, but the really hilarious, idiotic move proved to be the sensational one. And I think that opened the possibility of all kinds of things.

I remember raising the question with a very leading AI expert about whether there could be a program that could write novels. Not just novels that would pass some sort of Turing test, but novels that would really move people or make people cry. I thought that was interesting.

What did the expert say?

Well, OK, I was talking to Demis Hassabis [cofounder of DeepMind], and he was quite interested in this idea. We talked about it over a number of conversations, and I think the key question here is: Can AI actually get to that empathy, by understanding human emotions, controlling them through something like a work of art?

Once it gets to the point where an AI program, AlphaTolstoy or whatever, can actually make me laugh and cry, and see the world in a different way, I think we've reached an interesting point, if not quite a dangerous point. Nevermind Cambridge Analytica. If it can do that to me, then it understands human emotions well enough to be able to run a political campaign. It can identify the frustrations and angers and emotions in the nation, or in the world at large, and know what to do with that.


The novel also considers how a person’s personality might be captured and re-created algorithmically. Why are you interested in that?

Klara and the Sun just accepts a world in which big data, algorithms, these things have become so much part of our lives. And in that world, human beings are starting to look at each other in a different way. Our assumption about what a human individual is and what's inside each unique human individual—what makes them unique—these things are a little bit different because we live in a world where we see all these possibilities of being able to excavate and map out people's personalities.

Is that going to change our feelings toward each other, particularly when we’re under pressure? When you actually face the prospect of losing somebody you love, I think then you really, really start to ask that question, not just intellectually but emotionally. What does this person mean? What is this loss? What kind of strategies can I put up to defend myself from the pain?

I think the question becomes something very very real then. It's not just an abstract philosophical question about, you know, the ghost in the machine, whether you have some sort of traditional religious idea of a soul or a more modern idea of a set of things that can be reduced to algorithms, albeit vast and complicated one.

So it becomes a very human and very emotional question. What the hell is a human being, what's inside their mind and how irreplaceable is any one human? Those are the questions that, as a novelist, I'm interested in.

Artificial intelligence isn’t yet close to this. Should we still worry about what it can do?

In general, that question, about human oversight, is one that we need to be thinking about right now. In the popular discourse, the thing seems to revolve around whether the robots are going to kind of take us over, a kind of crazy zombie vampire kind of scenario, except featuring kind of sophisticated AI robots. That might be a serious concern, I don't know, but it's not one of the things that I'm particularly worried about. I think there are other things that are much more on our doorstep that we have to worry about.

The nature of this generation of machine learning, which I understand is called reinforcement learning, is quite different to the old forms [of AI], never mind just programming a computer. It's just about giving an AI program a goal, and then we kind of lose control of what it does thereafter. And so I think there is this issue about how we could really hardwire the prejudices and biases of our age into black boxes, and won't be able to unpack them. What seemed like perfectly decent normal ideas a few years ago, now we object to as grossly unjust or worse. But we can go back on them because we can actually see how they're made. What about when we become very dependent on recommendations, advice, and decisions made by AI?

A lot of people who know far, far more than I do about these things have expressed skepticism about the whole idea of having a human in the loop—allowing a human being to kind of supervise that process. It's just completely fanciful because we're just gonna be so way, way behind, there's no way that we're gonna be able to keep up.

There’s been a lot of talk about AI ethics recently. Do you think big tech companies should lead that discussion?

Maybe you can tell me—what is the consensus as far as you can see about the wish to open up the discussion? I can't quite figure out whether the people who are our most heavily invested in developing AI, whether their real position is one of secrecy? That they don't want oversight and they don't really want people talking about it very much. Or is it the reverse, that they're saying, “We're doing all this but it's up to you to think very carefully?”

I think there are people who have those motives, but the momentum of these big companies tends to drive toward controlling the discussion, making sure that it doesn't go in directions that affect the business.

I have noticed that even in the last three or four years AI companies do seem to be much more cautious about publicity. But I don't know if it's just some sort of PR exercise, this idea that it could be a partnership between human beings and machines, in the way that we've always had partnerships between human beings and machines. I don't know if that's a sincere belief, or whether it's just a way of trying to head off alarm in the general public.

I guess I'm asking fundamentally if the ultimate goal is to always have this partnership between human beings and machines, or is the ultimate goal that, well, we just come to rely on the machines because the human mind is just going to be contaminating, it’s going to be so, so behind that it is absurd. It is going to be a nominal, token presence, like one night watchman in a stadium full of machines.

The AI researchers that I talk to are trying to build things that are, in every way, as smart as us. It’s unclear what happens after that.

This will be a major challenge for us, I think. If it is the case that this isn't like the traditional rounds of automation where you know one set of jobs disappear but new sets of jobs appear—and a lot of people think it isn't—we will have to rethink the way we've run our societies for centuries. That old system where each of us contributes to a larger enterprise, and we get paid, and then we use that money to fund our private lives and feed our families and so on, that system is going to have to be rethought.

And not just in terms of the material way we distribute money and stuff but also in terms of prestige and a sense of self-respect.

I think one of the things that we do seem to have very hardwired into us as human beings is this need to contribute to the larger community around us. If I don't do it, I feel bad in some kind of way and I'm also inclined to think badly of people who don't.

The big question I would have about all of this is how do we build forums in which one can have this discussion in a meaningful way.

Do you have any ideas for how to do that?

Ha, no, I don’t. Because, you know, out of necessity we're talking about something that is global and international. It's not a good time for international institutions at the moment. Also, to what extent are the people who really know willing to encourage meaningful discussion, or to give out information so we can have a meaningful discussion?

I don't think the people in Silicon Valley, or wherever they are, are a monolithic group that would all hold the same view on this. Different people have different views and possibly they have different interests, but what I would ask is what do they want from the rest of us?

Returning to the book, Klara is quite a likable character, for an AI. Should technologists try to make sure their creations are as nice?

One of the things is that it may be difficult to predict whether the AI program or the robot is going to be what we would define as nice or not. As you know, reinforcement learning just relies on that central reward function, and I understand there is this term called overfitting, which is that it behaves in a way that we wouldn't quite predict. It goes about trying to fulfill its goal in a way that actually is very destructive to us.

Maybe you can give AI these kinds of Isaac Asimov type rules, maybe you could do something like that, but I can't see it. I mean, and we can't decide amongst ourselves, without robots, what is desirable, what is nice and what isn't. In the country you're living in, even the question of what freedom, or what democracy, means [isn’t clear]. A lot of people think freedom means that you riot in the Capitol building to reinstate democracy.

The big thing I'm trying to say is, how do we get the conversation going? Different parts of our culture have roles to play. I think people who write books, people who make these blockbuster television series, have quite a big role to play. And I think there have been some very interesting TV series, like Westworld, that kind of raise these questions. But you know we need the conversation to get more urgent and more serious.

The past year has, of course, been incredibly difficult and different. Has it been an inspiration to you as a novelist?

One thing I have to say about the pandemic is that a staggering number of people have died, and this is something that I don't think we've quite kind of woken up to. We're talking about a scale of death that is just extraordinary. Right now there are millions of people around the world kind of shocked and bereaved having lost somebody close to them. And I think the emotional damage of this is going to be colossal.

But we’re focusing, and it's understandable, on how the high street might change and Zoom and working from home. But the big, big thing about this is this level of death. In Britain, you know over 130,000 people died in the past year, which is more than twice the civilian death toll of the Second World War. I heard one commentator say recently that the half a million dead mark in the United States means that it’s more dead than the combination of the two world wars and Vietnam.

I don't think we can go through a situation like this where so many people have died, and so many people are bereaved, and there is a sense that things were not done correctly, without something. Without actually there being a major consequence of that. And I'm not quite sure what that is.

The other thing, I wouldn't say it's an inspiration, it's just what I've observed, is that there is this kind of strange contradictory thing happening—these two different versions of the truth or how you look for the truth seem to come to the fore with a vengeance in the last few months.

On the one hand, we've come to rely desperately on the scientific method where people say, show me the evidence, peer review the evidence. And at the same time we have a situation where half of the people in the United States believe that Donald Trump won the election but had it stolen from him because they wish to believe it. The idea is that the truth is what you wish to believe. Feel it emotionally strongly enough in your heart.

People like me have placed so much emphasis on the importance of emotional truth; I create things like novels that are supposed to kind of move people. And it has made me kind of pause a moment. Looking at these two completely opposed attitudes to coexisting in a massive way in our lives at the moment, I kind of wonder if I actually contributed to this idea of what you feel is the truth.

I'm conscious that we've gone a bit dark in this interview.

Yes! Klara and the Sun is supposed to be a cheerful, optimistic book!

Well, finally then, let me ask how optimistic you are about the future.

Well, I think these areas [of science and technology] that we've talked about can bring us enormous benefits. It could happen. If we can rise to the challenge of using these incredible tools in a positive way, I think we’ve got a lot to look forward to.

In terms of liberal democracy and how it's been shown to be very fragile, I don’t know, I do feel rather shaken about that. I think there are very strong competitive models now to liberal democracy that weren't around 30 years ago. And I'm not saying that, you know, things like artificial intelligence will actually help regimes that are not liberal democratic regimes but, I mean, that's not inconceivable.

It might take away the advantage that liberal democratic societies had over authoritarian ones or centrally planned ones. With sophisticated artificial intelligence perhaps the Soviet Union would have thrived, perhaps they would have produced more luxury goods than the West, and we would be envying their supermarket choices.

But if I have cause for optimism, it’s the optimism I tried to put into Klara and the Sun. I think there are positive aspects of human beings, and some of that I think is hardwired into us, almost in the way that things are programmed into a creature like Klara. She is kind of a reflection of human nature, and she mirrors back human society that she learns from. I picked up the parental thing—nurture and protect—but there are many things about human beings that do seem to be hardwired into us, and I think that continues to give us hope.

All these breakthroughs in science, I think they could be great. But, you know, we've got to be ready for them.

Related Articles

Latest Articles