AI

“Systems built without us will embed assumptions about who matters and who doesn’t”

Jamila Rizvi is nervous about AI. Here’s why she’s not opting out - and hopes you won’t either.

By Melanie Dimmitt

Published 17 March, 2026

AI

“Systems built without us will embed assumptions about who matters and who doesn’t”

Jamila Rizvi is nervous about AI. Here’s why she’s not opting out - and hopes you won’t either.

By Melanie Dimmitt

Published 17 March, 2026

The results are in. AI Magazine has named its top 10 global leaders in artificial intelligence – the people leading a market projected to hit US$4.8 trillion by 2033. 

And they’re all men. 

“The economic architecture of the next century is being constructed right now. And the people doing the constructing are, overwhelmingly, the people who aren’t in this room.”

Jamila Rizvi is in the final moments of her keynote at the 2026 FW Leadership Summit, which explores AI’s role in shaping inclusive, equitable leadership. The 10 men pictured behind her demonstrate that, here, there is much work to be done. 

“I say that, as someone who’s spent a regrettable amount of time looking at the founding team photos of major AI companies, there’s a lot of hoodies,” says Rizvi. 

For the past 30 minutes, the bestselling author, political commentator and FW Deputy Managing Director has explored some of the real-world impacts of technology that’s been built, consciously or not, in the service of a single gender. 

Rizvi referred to healthcare – specifically, liver health – where AI diagnostic tools trained primarily on data from male patients are producing worse outcomes for women. 

“Researchers from the University College London examined AI models used to screen for liver disease from routine blood tests. The models were twice as likely – twice as likely – to miss liver disease in women as they were in men,” she says. 

Recruitment is another example, with Rizvi raising Amazon’s revelation that, back in 2018, it was using an AI tool to screen job applicants.  

“The tool had been trained on a couple of decades of successful Amazon hires,” she says. “A decade of successful Amazon hires told you something about who got hired in those junior jobs and who got in those senior jobs and who got hired at all. So the tool learned very quickly that it should preference male candidates.”

Amazon, like many major companies, removes gender from applications. AI got around this by seeking out tell-tale keywords – “women’s leadership scholarship” or “women’s soccer team”, for example.  

“And by doing that, the tool was able to actively downgrade applications from women, penalising CVs that contained those terms,” says Rizvi.

“Systems built without us will embed assumptions about who matters and who doesn’t.”

Amazon scrapped this tech but, almost a decade later, the memo is yet to circulate. 

“Variations of this exact problem are happening across our industries, often without anyone realising that they’re even happening,” says Rizvi. 

“Across the world, AI is being used in hiring platforms, in healthcare diagnostics, in credit and lending decisions, in housing applications, in child welfare systems, in immigration assessments, in the sentencing tools used by courts, and in law enforcement technology that is being deployed on our streets. 

“These tools are built on datasets that under-represent women – particularly women of colour – and they perform worst for the very people who are most likely to be harmed by the wrong answer.” 

Rizvi paints an alarming picture – but she’s quick to remind us that no robot is to blame.

“The AI is not at fault here,” she says. “It’s what we’re giving it. And the data we already have is biased. You input that in and you’re gonna get biased results.” 

This “quiet crisis”, as Rizvi calls it, runs alongside the louder, more dramatic, science-fiction version of AI replacing humanity. 

“[It’s the] slow, structural version where biases we’ve spent decades trying to dismantle get automated and encoded into what we’re doing,” she says. 

But, as Jamila points out, we’re only at the beginning of this gen AI era. Now is the time to recalibrate – and reshape what’s next.

Audience members at the FW Leadership Summit, Sydney

After presenting the 10 so-called top global leaders in artificial intelligence – all men, all likely owners of more than one hoodie – Rizvi leaves us with a message of hope. She clicks to the next slides, and scrolls through a list of women at the forefront of AI.

“They might not be getting the same recognition but there are many of them – and they are arguably doing some of the most consequential work of all,” she says. 

Rizvi highlights former OpenAI CTO Mira Murati, now founder of AI safety research startup Thinking Machines Lab. She also points to Daniela Amodei, President of and co-founder of Anthropic. And Jaime Teevan, Chief Scientist and Technical Fellow at Microsoft, among several others. 

“They are building the intellectual and institutional architecture that will govern AI usage in some of our biggest companies into the future. Their work matters. And we need more women,” says Rizvi. 

And the stakes, she argues, couldn’t be higher. 

“Like many of you, I am nervous and unsure about AI. [But] systems built without us will embed assumptions about who matters and who doesn’t… We have been here before. We have watched transformations happen without us and spent years trying to retrofit our way back in again. We can’t do that again,” she says. 

“The conclusion to this AI story is very much not predetermined. It is being written right now, as we sit in this room. The question is not whether or not you are going to be affected by it. The question is whether you are going to help write it.” 

Image credit: Vienna Marie Creative

Not an FW member? Don’t career alone. Join the network that’s in your corner here.