Technology, AI and ethics.

“We learned how to shoot a gun better, but not if we should pull the trigger.”

“We learned how to shoot a gun better, but not if we should pull the trigger.”


Conditio Humana Editor-in-chief Alexander Görlach conversed with Khalid Kadir, lecturer at UC Berkeley.

We are presenting this conversation as a special audio podcast in parallel to its transcription. Thank you for listening.

This interview has been edited for length and clarity.

Alexander Görlach: We have no real overlaps, but we do have overlaps. You work on engineering social justice, which sounds very buzzworthy. But literally, when we look into the problem with many societies and many democracies nowadays, it has something to do with rising inequalities. I am aware you do not work on “high-developed” countries but I still find it very interesting to talk about how we could reestablish or erect an engineered social justice; this is what I would like to talk to you about.

Khalid Kadir: I started working locally, more recently here in California on these issues in the field of environmental justice, thinking about the role of engineers in that process. So I have my feet in both spaces, in that way. I agree that inequality definitely lies at the heart of some of these things. I think about, what are the structures that have produced that inequality? It is a question of power and who has it, and how they have maintained it.

Ideology has a big part to do with it though. Science is an ideology; the idea that we can know everything by counting and measuring is part of the problem. We privilege certain ways of knowing in the world today, and by privileging those ways we silence and marginalize things that can’t be known that way, or through other ways of knowing. If I can’t prove it to you through numbers, then it can’t be proven, it’s just impossible. On top of that, if I can’t justify it through finances, then it’s not justifiable. The sort of supremacy of economics as the discipline through which all things have to pass is part of the problem too.

Alexander Görlach: To challenge environmental issues, we have 99 out of 100 studies regarding climate change layout and what we may have to do in order to prevent the globe from a rising temperature above 22 Celsius, and yet we don’t do anything. So there must other things that play into that.

Khalid Kadir: Totally. I think part of it is because it’s not about the overall truth of a thing; it’s about who benefits and doesn’t benefit from the truth of that thing. This is where I think power plays a role in this. The title of that film was really appropriate: An Inconvenient Truth. The idea that even if we do know something it is not convenient to those who benefit. Climate change benefits many people. It benefits us. We can fly around and by flying we contribute to climate change.

So we are in this complicated space. Driving our cars, we benefit from the ongoing extraction of fossil fuels. Yet we also see it as a problem. So, I think that there is a tension there and behind that, a lack of accountability. We are able to externalize the cost of what we do to spaces, or other people, or future generations, and that lack of accountability is part of what lies at the problem here. It is wrapped up in part of what enables us to keep doing this is a techno-utopianism that we will be able to fix it with technology. We always have. We have a very short understanding of history and think that the way we live on the earth today is completely different than anyone in history.

The things that we understood societally for all of the history of humanity are very different now because we are doing things that we have never been able to do and can’t understand. I think that lies at the root of it. But our ability to do that lies in engineering and science; the industrial revolution and burning carbon fuels have enabled all this process. We have this modern belief in a sense of progress, that we are going somewhere, whereas traditional beliefs are more circular and cyclical and that we are not socially evolving, going forward or going backward. Every new person born has to go through some of the same learnings. It’s not like my children are suddenly going to be much smarter than me because they were born from me. They still have to learn math!

I think about, what are the structures that have produced that inequality? It is a question of power and who has it, and how they have maintained it.

Alexander Görlach: That’s very interesting about the circular model because that’s actually true about let’s say tolerance or other values that every generation has to learn. That is not par with other sorts of knowledge that we pile upon. Would you say that’s the biggest challenge of reconciling the circular way we have been living?

Khalid Kadir: Absolutely, without question. This is something I’m struggling with right now myself. At the root of that, I think being someone who is in higher education, I immediately think, well the problem is we don’t have a good liberal arts education and we don’t understand the basic human sciences or basic humanistic fields anymore and have devalued those. We only take the fields that lead ourselves to this kind of build and you have to learn to not be greedy, and so does the next person. We’ve got a bigger muscle that enables us to do more with our greed, which is a problem. We are out of balance in that way and in some sense we double down on the technical and we see that getting better and stronger. We learned how to shoot a gun better but we haven’t learned where to point it or even if we should pull the trigger.

Alexander Görlach: After the Facebook heyday and Cambridge Analytica and other companies who have taken stances such as Google, they realized that ethical behavior or ethical products and ethical assumptions which come from a liberal arts education, they include that in new products. It’s not too alien anymore for these big companies. They are rich, and now they have more time to think about these impacts, but that may be a little cynical.

Khalid Kadir: I am a little cynical. I am cynical about their approach to ethics. There is a lot of this “ethics-washing”. Look at who Google put on their ethics board and how that got shut down right away. They have their finger on the pulse and they realize that they can’t be complete jerks. I don’t believe that they want to be these complete utilitarians. But they are utilitarian still.

Alexander Görlach: If you are a corporation, you have a business model, it is your core and that is how you can finance all the other stuff. 

Khalid Kadir: We’ve seen this in history. The church wins a war battle and they say that it proves God is on their side. Now Google gets bigger and it proves that they must be doing something right. It doesn’t prove that they are right; you can do things wrong and still win. We have a sense that success is the evidence that we must also be in the moral right and that is not true. We have taken that on in a capitalistic framework in some ways.

Alexander Görlach: Then as a consequence, it sounds like we would have to implement some severe and serene, honest ethics in a corporate structure.

Khalid Kadir: And they have to be based in justice, not just ethics. They have to take into account history and what it means to enter into a space when there are historical and structural things that have created the space we are in. There is no tabula rasa. We are dealing with histories of injustice. What does it require of us? That’s a much more complicated conversation.

Alexander Görlach: We have companies implementing it into structures, but on the other hand we have biases in algorithms, which is a severe source of inequalities and injustice. It comes down to a single technician and what data they want to implement and what components this data may have or imply. This seems to be too broad to find a solution to the problem.

Khalid Kadir: It’s not simply the efforts of the technicians, but the technicians are operating in structures that started long before they began their technical work. When we see algorithms used to decide who does or doesn’t belong on welfare, they are inheriting the already unequal marginalizing structures from before so that the technician can’t even undo that. There is a larger sociopolitical conversation that has to happen that exists beyond the technocrats.

On the other side of this, as an individual human moving through space, one has the moral responsibility to understand the things that we are part of. The challenge I have with my students is that do you decide that you just cannot take these jobs? At what point does it become just a veto? When do you decide you cannot work in an industry that is too corrupt? I am not too extreme on that, but sometimes it comes up. Can I work for Palantir, which is deporting people in horrible ways? Maybe not. What about Google? Or Amazon or Microsoft? What about a startup? There are some veto points but we need to ask questions about whether we can do just work in these spaces or are they constituted in such a way that they make it impossible to do just work? It is not just in the making of the work you are doing, but in the institution you are working in.

The other problem that comes out of this is the trickiness with algorithms. They are not single-purpose. You design an algorithm to figure out the best way to hold a computer mouse, the same technology can be used to figure out the best way to hold a drone or shoot bullets. This becomes a real ethical challenge for us. I did something to do X, and now it is being used to do Y. Do I have any moral responsibility?

There is no tabula rasa. We are dealing with histories of injustice. What does it require of us?

Alexander Görlach: That goes back to what you said about accountability. If you assemble data for an insurance premium, you just add layer over layer, and you may not be sure about what data you are deploying. 

Khalid Kadir: 100%. When the East German soldiers would shoot people who were trying to go over the wall, they would claim that they were just doing what they were told. That is not a moral excuse. They are still guilty of killing people. If you work for Google, you do not decide what Google does. That is a simplistic sense of removing all moral power that we have.

I think we need to take ownership over where we work and bearing responsibility for that choice. Now, granted, not everybody has that kind of choice. When we talk about tech workers, there is a lot of leeway in choice.

Alexander Görlach: We all use Google’s products. I do enjoy them, I am not going to lie. I do also enjoy a number of aspects of Facebook and Twitter. That means I am part of the narrative as a user. You mentioned the Google ethics board that got shut down after a few days. In contrary to your argument, there is already empowerment of the employees within these companies. 

Khalid Kadir: Power is contested, never held. Employees are challenging this and just recently, some of them were fired for organizing. There is a tension going on. Up until those employees got fired, I was very optimistic, but now I am a little nervous. I’m curious to see what this lawsuit manifests, but I think this is part of that struggle. Establishing a balance of power and checks and balances is a really useful way of ensuring that there is accountability, rather than setting up a bunch of technocratic rules. Establishing a balance of power allows for democratic debate.

I wonder where users are in this conversation. Users should have some say in that, not just employees and the owners. If we look at the history of unions in the US, the last 40 years have not been very good. It might be a turning point. Are we going to see a rise in different kinds of unionism that can challenge employers and ask this? The thing that concerns me about this is that it is not just the US we have to think about, but other places such as China and Russia. That complicates the story.

When we see algorithms used to decide who does or doesn’t belong on welfare, they are inheriting the already unequal marginalizing structures from before so that the technician can’t even undo that.

Alexander Görlach: I would argue that to make a product better, product teams should have people from the humanities or from different cultures. Would that be the first step that we could recommend these big conglomerates do in order to make their products more sustainable and ethical?

Khalid Kadir: That is absolutely important, however, it is not just about getting people in the room. It is making sure they actually have a real say in the conversation. You could imagine the anthropologist or the humanities philosopher being sidelined in this conversation.

I was once judging a big ideas contest and they had all these metrics and a rating scale. One of the rating scales was ethics and they had a 5% weight in the final value. But ethics is a veto question. I am worried about the technical methods they use to integrate these other ideas and they need to consider the possibility that there are things we cannot do, or there are things we can do but not make a profit off of. What is the structural institutional framework that incentivizes what doesn’t happen? And as long as that structural institutional framework is the maximization of the quarterly bottom line, we have a problem, that is always going to push out these other ideas.

Alexander Görlach: That goes back to what you said about making the assumption that everything can be expressed in numbers. We have this dichotomy and both work in the disfavor of social justice. 

Khalid Kadir: There’s this post-enlightenment thinking that we can understand the world through a mix of empirics plus logic. What that does if effectively shuts out a lot of other ways of knowing that may be metaphysical or brings in other faculties. It empowers those that can speak through empirics. If I can speak for the environment through numbers, I’m the one who gets to represent the environment, as opposed to the person who says there are values that I have associated with the environment that I don’t represent in numbers, but I represent in other ways.

Alexander Görlach: Both sides are important. How do we get both sides talk to one another?

Khalid Kadir: My take is that the empirical side dominates. Everybody understands that you need to know numbers. I think the push has to go the other way, that we need to push on our curricula, and on people who do empirics to slow their roll and to start engaging with other ways of knowing. But they have to do in a way that is real. It needs to be an honest opening of these spaces that people open their eyes to other ways of understanding and valuing the world. This is the classical goal of pluralistic liberal democracy. But without its structural incentives that reward that and sanction those that don’t, I don’t know if it is possible. People are not going to do it out of the goodness of their hearts or because they were told to do it.

Then we need to think about what are the institutional structures that will force that upon us and force us to open up the way we make decisions and understand the world through other faculties. Again, my focus is in higher education primarily. So I’m thinking, how do we feed this stuff into higher education? At Berkeley right now we are battling if this is engineering or sociology. But that means that what people are saying is that engineering does not include anything outside of traditional empirical stuff. The problem is that until you say, “This is our family too”, you are never going to take it seriously. We need to reconstitute what technical practice includes and doesn’t include. Technical practice has to include these other things and say you are not an engineer until you have read Aristotle and this is part of what it means to be an engineer.

The other problem that comes out of this is the trickiness with algorithms. They are not single-purpose. You design an algorithm to figure out the best way to hold a computer mouse, the same technology can be used to figure out the best way to hold a drone or shoot bullets.

Alexander Görlach: It sounds a little like what I know about deliberative democracy where you use mathematic skills in order to find an assembly that represents every party involved. People in these contexts of representation are keen on also making a decision that is not 100% in their own interests, which helps to navigate faster solutions. Is that something that you dream of also in a corporate context?

Khalid Kadir: I was with you all the way up to the last two words! But totally. I’ll admit that in the recent past, I’ve started to become nervous about the limits and possibilities of liberal democracy.

John Maynard Keynes. He wasn’t trying to say capitalism. He was trying to save civilization. He wrote all this stuff about civilization being a thin and precarious crust. We need to separate out the economic questions from the political because there’s no reason for poverty, we have enough we can do this technically and solve the problem. There was an elite attitude of managing the world for everyone else. He was part of the elite. He believed that small groups of people who are really intelligent need to make sure that society doesn’t fall apart, otherwise democracy would lead to demagogues and populace.

Given what I see happening in some places in the world, including here, I’m a little nervous. Is that is that maybe right? Might there be some truth? Plato didn’t think democracy was the best form of government. He thought that was tyranny in some ways, so I’m starting to become suspicious of democracy in this way. I am still a proponent. One of my favorite books is Undoing the Demos by Wendy Brown, where she says that the destruction of our democratic sensibilities is because of a neoliberal framework where we don’t understand ourselves as also being sort of democratic agents in the world. We don’t see that as part of what it means to be a person anymore. But democracy doesn’t work without that. Democracy is a total failure if people don’t understand how power works.

Alexander Görlach: Like were we discussing, if you are a consumer that’s something different than a citizen and I feel a lot of our engagements nowadays are in the framework of consumerism. That is something that Hannah Arendt already foresaw in mass society evolving into something more tyrant.

Khalid Kadir: Chris Hedges writes about the triumph of spectacle and Neil Postman writes about amusing ourselves to death. The way we are politically engaged is through entertainment forms of TV news and now Internet news. That’s superficial and reductive, not critical-thinking oriented and passive. You just take it, you don’t interact with it. I worry about these things.

The Luddite tendencies in me are saying, turn off the screens! Be in company with people! That might help us learn to be better democratic citizens, but there is no turning off the screens. We’re all in this.

Alexander Görlach: So I feel like all humans if they are in China, or the United States or in Germany, for that matter, they want good governance. In Singapore and China, they say that by restricting some of what is known as civic values or rights in the West, we enable ourselves to have better governance. Hong Kong, for instance, has a huge problem with housing, whereas Singapore does not. Both are not full democracies. So it’s not about saying all non-democratic societies deliver good governance. This is an argument often portrayed in disfavor of liberal democracy, that it has, in fact, has leadership limits.

Khalid Kadir: There are two things about the question of scale that come to mind. I’m not sure that as you scale to larger and larger contexts that democratic governance works as well. In a smaller context, it works better. But when our worlds and climates become so different, the ties become thinner and it becomes harder to constitute that. Inversely, the premise behind liberal democracy is the nation-state and I am a little unsure about the value of the nation-state in a world where capital, goods, and people cross borders with different fluidity. We don’t allow people but we do allow capital without controls.

This creates an imbalance of power too. One radical proposal is to get rid of borders and let people cross where they want. If we are really talking about liberalism that might be a model, right?

Alexander Görlach: The first argument is very intuitive but it’s also not true empirically. So you and I, even though we live in different time zones, and in different climates, we may share like more values than people just outside the Metropolitan hub. Back in the day, people would identify as Americans, but now we can see that is not anymore the case for social justice reasons. It is a very heartfelt notion about borders and nations, but the nation-state nowadays is the guarantor of human rights and that they are upheld. If you look into the European Union, they formed a union of sovereign states. As a citizen, I have the same constitutional rights in Spain as I have in Germany. I can live in Spain and the Spaniards can live in Germany. The nation-state is a defender of human rights.

Khalid Kadir: I would say they are, potentially. If you ask a Polish person living in Germany, they are not going to say that they have the same rights as a German. In the United States, we see the differential access to justice depending on your race and your class.

Alexander Görlach: On the other hand, when I talk to people who want to abolish biases in algorithms, they say it’s completely impossible and we should go to constitutional standards and erase data regarding gender or race. That’s quite interesting, where the nation-state is the vessel where constitutional human rights are acknowledged. If we have to get rid of biases, which is very difficult, the technicians are saying we should go with those that are prohibited in specific legal frameworks. If that were to succeed that would be a nice iteration.

Khalid Kadir: France does not collect data on race, because this is their way of saying we’re gonna exclude racial bias by not collecting it. And yet France has all sorts of racial issues. So there is a challenge there.

Might we want to think about scales of justice that go outside of nation-states? What prompts me to think this way is there’s an article by Nancy Fraser, Injustice at Intersecting ScalesOn ‘Social Exclusion’ and the ‘Global Poor’. She argues that if you and I are looking to seek justice, we go to the institutions in our own nation-states to demand justice for something that happened to us. But, the person who made this shirt cannot do that because they are in another country. She argues we should think about global supply chains. Everybody that touches that supply chain should have access to justice along that supply chain.

500 years ago supply chains were all local and regional. You could get justice in the space of supply chains. But that is no longer the case and our institutional legal frameworks haven’t kept up with the speed at which our technological ability has changed.

Alexander Görlach: That may work if they are all like-minded democratic countries who subscribe to the same legal foundations. 

Khalid Kadir: What if the parents of Pakistani children making soccer balls could sue the teams that bought the soccer ball?

Democracy is a total failure if people don’t understand how power works.

Alexander Görlach: The problem is they are not protected by Pakistani law in the way that I am by German law. But on a general note, this is a very appealing thought. We have established lots of international institutions, which I feel come from a time with another framework also ideologically, like the Cold War. I know politicians and I agree with them that we should create a new entity for like-minded countries because the United Nations are blocked by Russia and China. Why don’t we create another body that actually combines all like-minded countries and work on similar problems? Potential solutions could be applied to other countries and it would foster the understanding of our mutual similarities.

Khalid Kadir: The problem is, how do we do that without becoming the next hegemonic entity? The challenges are, whether it’s multiculturalism or pluralism, how do we do that? In the end, we’re only having this conversation because someone pulled rare earth minerals out of the Congo into our devices. I think it is really important that we understand that marginalization and poverty are relational. They are not just relative, but our wealth is only in direct relation to other people’s poverty. The janitor that cleans classrooms is what enables me to have a clean classroom. If that janitor’s salary goes up, either tuition goes up or my salary goes down. These things are connected, they are not separate.

And so when we think about our world as deeply relational, then we can understand that when someone is poor it is in relation to someone else being wealthy. How do we understand how that relationship gets constructed? Most of the advanced post-industrial world is benefiting from the poverty of everyone else. Jason Hickel says that Africa has developed the West, not the other way around. He looks at net dollar flow and says we have talked about developing African nations, but it’s the extraction from Africa that has developed the West.

That is where there is an ethical challenge for us. If my comfort is only based on other people’s discomfort, there is a problem. We might even turn this into a math problem and say, what is the level of wealth that is fair given the number of people in the world and how many resources there are? We have forgotten that it is a zero-sum game. There is only so much on the planet, there is only so much nickel or gold on the planet. We have lived under the illusion that we can grow without limits and that has enabled us to get wealthier without feeling guilty for taking from others. But when we realize that we live in a world with limited resources and that getting wealthier is in relation to someone else getting poorer, that changes the conversation.

Pulling from Timothy Mitchell’s fantastic book Carbon Democracy, this is because from 1920 to 1970 all prices went down and so we use more and more and more. This is when Keynesian macroeconomics came into existence and he put forth this idea that the economy can grow and grow. It was never held into account because whenever you ran out of a resource, you could synthesize something. Don’t have enough wood? Use plastic. But that is coming from petrochemicals and as we used more and more petrochemicals, they become cheaper and cheaper. So we are able to live in this illusion of endless resources. That all came crashing to a halt in the 1970s when oil prices spiked. Then, we saw a transition to marketspace governance and neoliberalism because we ran into the limits of Keynesian economic modeling. We have all forgotten because we are a generation or two removed from thinking about the world and life having resource limits that we don’t engage our lives that way.

Alexander Görlach: I would agree and I would say we are still in that mindset because economics says growth stimulates ideas. These old models of the economy do not work anymore. How do we break it to the world?

Khalid Kadir: I think we are all a bunch of addicts. We are all addicted to our consumption and our things. John Galbraith discussed this in the 1950s and he was an important figure for promoting capitalism in Harvard. But we said we live in an affluent society addicted to consumption.

It’s a spiritual problem but there’s also a material environmental problem. We need to learn to recover our addictions and take care of each other so that we don’t need to shop in order to feel good. These things are all interconnected. One way to break it to humans is to help them feel safe without having to consume more.

Share Post
Khalid Kadir

Khalid is a Continuing Lecturer at UC Berkeley, teaching courses in the Global Poverty & Practice (GPP) program, Political Economy, and the College of Engineering. After completing his PhD in Environmental Engineering at Berkeley, Khalid focused his research on the complex role that engineering expertise plays in the politics of international development and poverty alleviation, and his current work focuses on the intersection of poverty, expertise, and politics. He is a recipient of the 2017 Distinguished Teaching Award, UC Berkeley’s most prestigious honor for teaching.

Show Comments