Welcome to the first public session of the Committee’s inquiry into algorithms, social media and the summer riots. We are here today to talk about algorithms used by social media platforms and search engines, and how they can spread misinformation. Our inquiry was sparked in part by the riots last summer, which followed the horrendous killing of three young girls in Southport on 29 July. I remind the members of the Committee and our witnesses that, despite yesterday’s developments, legal proceedings relating to this case are not complete. Under the sub judice rule, the House and its Committees should not discuss, in public, matters awaiting judgment in UK courts, in order to avoid prejudicing legal proceedings. In this session, Members and witnesses must not speculate or express views as to sentencing in the case. Wider discussion of events around the case, such as misinformation and how it spread online about the person who has been prosecuted, is acceptable. This morning, the Prime Minister announced an inquiry into the Southport killings and noted that it is frustrating that different rules currently appear to apply to online content, where different information is able to spread. I welcome to the Committee Paul Waugh, who is guesting in our evidence session this morning. He is a member of the Culture, Media and Sports Committee. With that, let me welcome our first panel, which is made up of three witnesses. We have Kelly Chequer, a councillor on Sunderland city council; Zara Mohammed, secretary general of the Muslim Council of Britain, and Ravishaan Muthiah, director of communications at the Joint Council for the Welfare of Immigrants. I would like to kick off by asking you each in turn to answer a question. We have very little time for this session and it is a large subject to discuss, so I am going to ask you each to be brief in your responses, with a maximum of a minute. My question is quite general. What did you feel was the impact of the summer 2024 unrest on the communities that you represent? I will start with Councillor Chequer. What intelligence did you have about what was happening on social media in order to help you to prepare for it?

Thank you, Chair, for the opportunity to come and speak to you today. I am the deputy leader of Sunderland city council. I was the political lead during our troubles on 2 August last year. I do not think we can overstate the impact on our communities, particularly our Muslim communities who were targeted throughout the violence, including a specific attack on a mosque. The impact of that on our communities has been extreme. I know that the communities welcome the work that is happening both locally and nationally to try to address that. The emotion that people feel, particularly around the spread of online information and hate as an underlying cause for this, is of extreme concern across all of the communities. In terms of the intelligence, I was the acting leader of the council at the time and we worked very closely together with the police. We had the same challenge, in that we were getting a lot of misinformation and, we believe, almost intentionally false information via social media about planned unrest, including incorrect convening dates, times and venues. That meant that both the police and I had very little intelligence on how we could manage the situation, what the scale of it was going to be and what the intention of it was going to be. That made it very challenging to police and manage in our communities accordingly.

Thank you very much. It seems as if there was almost organised misinformation that you feel you were suffering from. Ms Mohammed, I turn to you.

Chair and Members of Parliament, thank you so much for having me. I am Zara Mohammed, secretary general. I was also leading and co-ordinating at the time. For us, there were three levels. In terms of impact, it was completely devastating. Nationally, Muslim communities had collective fear and trauma, even if they were not in areas that were being attacked. As the councillor rightly said, they were receiving information that their mosque was going to be targeted. The second perspective we look at is how quickly misinformation was able to attribute very specific mosques and very specific localities. It felt very real. They knew the names of the mosques. They knew the locations. Not even some Muslims knew where those mosques were. They were very organised in their time specificity. We were watching in live time, particularly on X, what was happening through other platforms. The algorithms were boosting that. You were seeing more of what you hated, as well as more of what was targeting you. Communities were viewing in real time in Hull the refugees being attacked, cars being stopped or arson attempts on mosques in Sunderland. We were in real-time contact, because we are in a massive network of associations and mosques, with those institutions and asking, “Do you know if anything is happening? Are you okay?” We spoke with Southport and Sunderland. We spoke to more or less every mosque impacted. The third side was around action and response. There were figures like Tommy Robinson, for example, and others who were being boosted on the platform. Messages of what they were saying were then enticing others. Any time we posted safety guidance, there would be a stream of hate on our comments and posts. The point I want to make is that that made it challenging for us to support locally with the police and with other associations. We put on lots of safety webinars, but where there were strong relationships with local communities and mosques, that was a lot more effective, although not where there were weaknesses and gaps and also smaller police forces that were not well equipped to manage. Finally, in an atmosphere of heightened fear and anxiety people were really very scared. In some ways we were amplifying fear, even when we were trying not to.

That is an important point. Mr Muthiah?

Thank you to the Committee for having me. At the Joint Council for the Welfare of Immigrants we represent migrants across the country. Whether it is people who are in accommodation that we represent, or people who are very near asylum accommodations, these attacks totally devastated and traumatised our communities. I want to give one clear example of someone who was traumatised, and tell you her story. Our client Alia was afraid to leave her asylum accommodation during the riots. She is a former professional volleyball player who fled Iraq in May 2022, when she faced death threats. She was detained for deportation to Rwanda in 2023. She is someone who already had a lot of trauma and had come to this country for safety. She is very vulnerable. She had made multiple suicide attempts. After the riots, she told us in her own words, “I am so scared that this is going to happen to me. I cry every night. The Home Office didn’t protect me. They didn’t understand my situation. I came here to be safe, not belittled or threatened. It’s not too much to ask. I think English people hate me.” If we need any example of what these riots did to our communities, especially migrant communities, that is one. I think the environment that was created by politicians, by social media companies, by the police, by the Home Office and by the far right enabled the riots. That is one example of the traumatic impact that this violence had on our communities. The good news is that Alia has now been granted refugee status, but on that journey she was re-traumatised on our shores. I think we, and this Committee, need to take a hard look at why that happened. In terms of the information, I echo what Zara said. I won’t go into more detail now, because we do not have much time, but maybe later on.

Thank you; that is a very powerful testimony. I thank all three of the witnesses.

You have touched on the role of social media already, but can you expand on what role you think the social media and private messaging platforms, particularly the latter, played in driving the unrest over the summer?

I was particularly worried about the way that misinformation was circulated on closed platforms that we didn’t have access to. We can easily see what happens on those open platforms, so we are at least able to identify when there is misinformation. When things are spread on closed platforms, combating that and being able to challenge information that is not true or is not accurate becomes even more difficult. Along with our partners, we always try to encourage people to take their information from trusted sources, and that becomes impossible in these situations. Closed systems and closed platforms are something that have been a major concern. I don’t believe that the open platforms are easier for us to address, but at least we have more information. The information that we could not access was what caused the damage.

What is very interesting is the use of WhatsApp, Telegram, Facebook groups and discussion forums, closed and open. Obviously, X was a huge force in this. It is really important to note that it was people of all ages. We had youngsters on the street; we had parents; we had families; we had people who had businesses; and people who were going to riot and attack mosques. We travelled to Belfast as well and we saw the damage. What we found was that people were able to use Islamophobic material and racist materials that were all anti-refugee. They were using slogans and passing that misinformation very quickly through platforms to instigate hate. Some of my colleagues, imams in Liverpool, went and asked the rioters, “Why are you doing this?” None of them mentioned Southport. In some ways the use of the platform was propaganda of hate that incited them, “This is where we need to go—this is why we are doing it”, with shared slogans of hate and bigotry, particularly referencing anti-Muslim hatred and Islamophobia. That was the strong sentiment. Those group chats were being circulated among Muslims, with hateful, derogatory images of Muslims and of mosques that referenced, over a period of 10 years, different criminal experiences. The use of the platform, again, was co-ordinating very quickly, and to all audiences and all ages. That showed it was not a very specific audience. It may or may not have been ideologically driven, but they were sharing that information very quickly and very widely.

Both Kelly and Zara have talked about the instant messaging. I can shine a bit more light on the social media platforms because we had a lot of hate during the riots. It is really clear that social media platforms enabled the spread of anti-migrant rhetoric and racist sentiments. It emboldened those who had been radicalised by years of anti-migrant policies and rhetoric to take to the streets. It is really important for this Committee to think about the algorithms that promoted engagement and content with the most impression. This led to algorithms platforming posts that encouraged violence against migrant communities. That viral content and viral spread of information acts like a virus would; for example, there are super-spreaders. We know what super-spreaders are from covid-19. There are super-spreaders such as Tommy Robinson, such as politicians and such as tabloid newspapers. They use digital platforms to spread the virus of misinformation. On top of that, like a virus, the information then mutates. It changes and amplifies. It becomes more dangerous and has more cut-through with people as it gets amplified by more and more people. That creates what we had in the riots, which was extreme information disorder. That is what we saw. We know that social media companies and search engines that rely on engagement and impressions to boost their own advertising revenue are less likely to moderate this high engagement content. That ultimately leads to the loudest and most polarising views receiving the most digital airtime. We have seen more and more digital content that targets communities like ours. We have seen it much more on X, or Twitter, since Elon Musk’s takeover. It has become a really dangerous place for our communities. There has been a massive spike in racist and anti-migrant posts on the platform, but that content generates revenue and engagement for the companies, so they willingly turn a blind eye. It is clear that the business model of social media creates the worst environment for people. I want to say one last thing. I am sorry, because I know you want to end this.

And I just have one follow-up.

In this case, X was the message carrier. If pigeons, like in the olden days, were the message carrier, would we blame the pigeons? I don’t think we would. I don’t think we would be blaming pigeons and having a Committee about pigeons. We need to look at society as a whole. Of course we need to look at X, but these messages were created offline. They were created in the media. They were created, sadly, by some of our politicians. We need to really deal with the far-right radicalisation that is happening online, in digital communities.

Linked to that, having identified the issue, what should be done to prevent this from happening again? Who should be held responsible for the spread of disinformation?

I am only going to give you a few seconds to answer.

The platforms must be controlled, as I said. It is really clear that we need to control platforms. We need to have some strong controls on politicians and the language that they use, which is then used on the platforms, and on traditional media and other sources of disinformation and hate speech. We have done some studies and research at JCWI, which has found that the language used by traditional media or politicians is echoed on digital media, and then back again. It is a loop of hate. It is not just one thing; we have to deal with hatred wherever it comes up.

It is multi-pronged. Obviously, there are the social media platforms. There is the level of regulation and accountability. There are political figures. Ultimately, it is about empowering the user. As you say, the hate itself is already there, but how do we empower young people to understand, “Don’t share”? Even me sharing it made it worse because I was sharing it to my colleagues and saying, “Have you seen this?” That is boosting the platform. Then Sky News and other channels are sharing it. We are all sharing the content, which is saying to the algorithm, “Oh, this must be really good content; let’s share it more.” We have to look at the user, the young people and the communities we are trying to protect and empower, but we are also part of the story. How are we managing the story? There is only a certain level of regulation you can really go to. Even in political speech, there is that polarisation. We are losing the middle ground of dialogue, conversation and debate.

Thank you. How we should respond to algorithms, when we don’t actually know how the algorithm works, is a really important question that we will pursue. Councillor Chequer?

Thank you. I echo my colleagues’ statements that social media and the algorithms sharing the information facilitated the organisation of the unrest, particularly in our communities. The circulation of misinformation fomented the unrest, but it would be wrong to think it actually caused the unrest. There need to be two approaches. We need to target how this information is shared and should be challenged and there needs to be more opportunity for people to remove factually incorrect information. We also need to try to understand the causes of the unrest in our communities because they are complex and, in our case particularly, they are not recent.

Who do you think should be held accountable for this misinformation spread?

The social media platforms that are enabling it, without a doubt.

It is worth saying in this context that the LinkedIn post that falsely claimed the alleged attacker was a migrant was shared more than 2 million times before being taken down. Lauren, do you want to come in briefly?

It is on the question we touched on about the business models of algorithms, and what social media would like people to look at. Is that part of promoting the harmful, misleading content?

Yes. When a platform has a business model that is all about engagement that boosts its advertising revenue, no matter what the content is, if it is getting high engagement it will continue to leave that content on and turn a blind eye, until—as the Chair said, it got 2 million views before it was taken down—there is wider action or stronger repercussions for those businesses. At this moment in time there are not strong repercussions for businesses that promote this sort of hateful content.

Zara, many of my constituents of south Asian heritage in Rochdale felt very scared last summer, and vulnerable. They were very grateful to the Home Office for swiftly providing security in many mosques. What is the link between social media hate and violence against Muslims in particular and how different is it from ordinary Islamophobia, as we call it, among the mainstream media and maybe newspapers?

Thank you, Paul, for your leadership, particularly at that time. It is the power of the anonymous. You can really hate as much as you like without accountability. Islamophobia gets lots of clicks and is perpetuated at lots of levels; 40% of all religiously motivated hate crime, which is physical stuff that we can see, is against Muslim communities, but online, how do we evidence it? There is so much hate and vitriol that I personally get, and colleagues and parliamentary colleagues get, but no accountability. Politicians fear for their life, but Muslim women in particular fear even more gendered Islamophobia, so for us it is the wild west in some ways. Also, Islamophobia sells. The worst thing about the riots, which really showed, is how much people were galvanised in hatred towards Muslims. It was perpetuating biases and stereotypes that already existed, and not in a vacuum. There is a context. You could say it is a post-9/11 era. You could talk about even before that. There is a kind of global phenomenon of the idea of normalisation of Islamophobia and anti-Muslim hatred. There are two quick points I want to make. There was a lack of narrative of belonging. People could not call it Islamophobic hate. They did not want to call the riots out for what they actually were, but they were coming for mosques and coming for Muslims. Then there was the Government action. We really welcome the protective security. Many mosques where elderly people go or where women go for after-school classes for kids are not fortresses of protection, so when that protection finally came in—great. But it showed the weakness in the system that nobody even thought Muslims could be attacked. We had had so much vandalism and lots of things happening on the back of any bad news story, so we feel that the latent issue of Islamophobia has not adequately been addressed. In pockets of northern towns, things were bubbling and unfortunately that came out and spilled out on the streets. There is something much bigger we have to take home. For British Muslims, unfortunately, the riots have left quite a big scar. There is a lack of real action and long-term strategy on what we are going to do to tackle Islamophobia and not make a security issue an immigration issue, which is what the riots are really pointing at.

Thank you.

There is a clear link between years of hate speech towards migrants and migrant communities and the violence we saw towards the same people in the riots of August 2024. While organisations like our own have seen that and known it, those riots showed clear evidence to all of us of how dangerous this hate speech is to all of us in our communities. In terms of the question directly how social media affects people on the ground and moves from the digital to the street space, we have done some recent studies and found that on Facebook the mere mention of immigration leads to many comments and responses. That can be generally a lot of hateful comments. For example, we found that there is a massive amount of misinformation and disinformation on social media platforms about immigration and that it is a leading contributor to the hate and intolerance towards migrants. We have seen that the language of right-wing influence is, as I mentioned, parroted online and then parroted back in the tabloid press. For example, some of the comments we have seen talk about mass deportations, and about people who come to our country as illegal invaders, scroungers and parasites. Those are just a few of the comments that come up. We have also seen a sharp rise in AI content across social media. That is a massive worry and I think the Committee should look at it, because it has meant the ability for people to create false images that spread far and wide and convince many people of all persuasions that the image on the internet is correct. Without clearly defining AI content as AI-generated, and having clear checks on it, we are going to see proliferation of that content.

Are you asking for that content, or phrases such as “alien invaders” or “parasites”, to be made illegal speech in some sense?

That sort of speech should not be allowed in society generally, I would say. There is a certain level of hate speech that appears online that we would not accept if it appeared on the street. There is a certain amount of discriminatory language, again, that appears online, that we would not accept. The way that we deal with that speech online needs to be much firmer. Coming to the second part of the question, about the Online Safety Act, we would say that at the moment the blunt answer to “Are migrants provided with enough safety?” is no. They are not provided with enough safety online, nor are our communities. Is the Online Safety Act a good move in the right direction? Potentially, yes, but there is a difference between the Act and the enforcement of the Act. What will Ofcom’s role be, with the Act? Will it be able to have greater restrictions on social media companies? At the moment, the context is quite worrying. Mark Zuckerberg is moving Facebook away from fact checkers. X has moved away from that. Our own Prime Minister has been attacked by the head of a social media company, and social media companies in America are very close to the Government, as we saw from the presidential inauguration yesterday. It is a really worrying time. I think there needs to be clear action from our Government and politicians, urgently.

We have to do both at the same time. If we are to tackle racism or anti-migrant hatred, we have to tackle it online and in the general public via traditional media and any other ways. If we close one angle, which is the digital, it will just appear somewhere else, so we have to do it everywhere.

We need to move on.

Thank you very much. I echo my colleagues’ comments about the concerning nature of this and the challenges that were posed nationally and locally. In local government we have no more powers than any other individual to get content removed from an online platform. That in itself presents a real challenge. Our ask is that local government be given more power to enable us to counter, challenge and remove misinformation and harmful content when it appears online. I would also like to see us being able to work with communities. Things like the Online Safety Act are very welcome, but, as my colleagues have said, we have concerns about the enforcement of these things at national and local level. The powers need to be there to enable us to challenge these things promptly and to ensure that our communities are engaged and kept up to date with these things, because often it is our communities who hear about them first. They can be a very useful source of information, and we should welcome that.

Thank you. We are running late.

This is a question to all three of you, but let’s be as brief as possible because of the time constraints. Specifically, were you satisfied with the Government’s response to social media around the unrest last summer?

The Government’s response to social media was very difficult. I think they felt the same frustrations that we did. I was very satisfied with the Government’s response to the unrest and their support for us as a city. We had excellent support throughout the entire scenario, but they were in no better position than we were to challenge the social media and clearly were not able to act to help us to prevent the violence that we saw in Sunderland on 2 August.

In brief, no. I don’t know what the response was, if I am honest. Narrative is really important. Obviously, there was the justice deterrent, which we saw, and that had to be done physically, but online it was very hostile, very Islamophobic. The dehumanisation of Muslims was carrying on and groups were allowed quickly to organise. The police did not know if it was real or not. There were so many weaknesses around the chain. In live time, Muslim communities were watching and saying, “Why isn’t anybody doing anything about this?” There were some really big shortcomings and the Government were very unprepared. Where is the team? Nobody knows, even in policing, who is responsible for making sure, when communities are at risk, as they were, in live time, that they are able to respond. Most of those police forces were very small. What could they really do? That is a huge area that needs to be tackled.

From our perspective and that of our communities it is impossible to be satisfied with the Government’s response to the social media activity that led to the riots in August 2024. The damage, physically and emotionally, to our communities, was exacerbated by the Government’s inaction. In fact the Government’s response to social media activity that caused the summer 2024 riots was utterly woeful and we would even say that it verges on a dereliction of duty to the communities we serve. It felt to many of us in the migrant community we serve that the Government sat by and allowed the violence to intensify day on day. Many were practically begging for support from the Government as our safety was put at risk. The statements were slow. How can it be possible that the riots rumbled on for so many days? I don’t think you need me to say that it was a failure. The evidence speaks for itself. Social media activity started the fire that led to the riots, and the large accounts that spread the misinformation and disinformation were allowed to continue spreading hate speech and incite violence throughout the far-right riots, so there was no meaningful social media enforcement against the hate speech.

Thank you all very much for your candour. Given Kelly’s point about how the Government do not have a huge amount of control over what the social media companies can do, although I completely accept the points you make, Zara and Ravishaan, about not being happy with the Government’s response, what would you have wanted to see on top of what was done?

Thirty seconds.

All it needed was a counter-narrative. There was no narrative of belonging or to say, “This does not represent our Britain.” There was no narrative about Islamophobia.

I second that. The Government could have been much clearer and louder, earlier. There could be more enforcement and more accountability of social media companies, very early, and of hateful platforms and individuals on those platforms. The accounts need to be shut down very quickly. If they were, there could have been a much quicker end to the violence and to the misinformation, disinformation and hate speech online.

I would like to offer a slightly different view. Where we saw, in the days following the unrest in Southport, positive action from police forces, that was shared on social media. We had a significant threat of further violence in the days that followed. There was a specific threat on an asylum-seeking processing centre in our city. That did not occur, and I believe that that was in part by positive messaging locally and nationally through social media. There was at that stage a narrative of, “If you do this there will be consequences,” which spread well across social media.

Thank you very much to the panel for sharing their experiences of the impact of the spread of misinformation over the summer and the unrest in their communities. It is clear that the impact was significant, that people are still traumatised by that impact and that the damage, both in personal and economic terms, was significant. There are questions for this Committee and for the Government with regard to how the misinformation was allowed to spread so quickly and what speedy response there could have been. Councillor Chequer, I am very impressed by what you said about the co-operation with the police, Government and local authority, which helped to reduce the ongoing impact, but I also recognise, Ms Mohammed and Mr Muthiah, what you said about the way in which we were all part of spreading the misinformation to some extent, and we also need to address that. Thank you very much for sharing your experience. Witnesses: Marianna Spring, Imran Ahmed and Dr Whittaker.

Welcome to our second panel in this morning’s session on algorithms, social media and the summer riots. Earlier, we heard from representatives of those impacted by the summer unrest. Now we welcome a panel of experts on some of the reasons, how the misinformation spread, and the challenges of addressing it. I will start by addressing a question to all the panel, so I shall briefly introduce them: Imran Ahmed is the CEO of the Centre for Countering Digital Hate; Dr Joe Whittaker is a lecturer at the School of Social Sciences Cyber Threats Research Centre, Swansea University; and, joining us exceptionally early from the United States, is Marianna Spring, the disinformation and social media correspondent of the BBC. As I said, we heard earlier of the profound impact that the unrest over the summer had on communities in this country. I would like each of you as briefly as possible—we have a lot of interest in this subject—to answer the question of how far the business models of social media platforms promote the spread of harmful or misleading content and, as specifically as you can, in a short period of time, how they do it. How do they promote that spread?

Good morning, panel. My name is Imran Ahmed. I am the founder and CEO of the Centre for Countering Digital Hate. By way of background, CCDH was set up because the fundamental physics of how information is shared in our society, in how we negotiate our social mores and values, and even how we negotiate the corpus of information that we call facts, has shifted to online spaces. The unrest this summer was profoundly shocking, but predictable and unsurprising. An incident in which violence was inculcated through the spread of acute disinformation was an entirely predictable feature of the design of social media platforms, which are not designed to be spaces in which free speech occurs. They are not designed to be spaces in which we find information that is good, or the truth; they are designed to be spaces in which we spend as much time as possible, in order to keep watching adverts. Truth is a really bad business model for social media platforms.

Could you think about how the business model of social media platforms promotes the spread of harmful or misleading content?

I am coming to that.

We don’t have very much time.

For example, if you went on a social media platform and found the truth immediately, you would go off and do something else, so what they want to do instead is provide as broad an array of information as possible, prioritising that which is the most emotionally engaging—that which most predictably would lead you to respond or spend more time trying to work out what truth there is to it. There is actually an incentive for them to pick the most engaging content. There is evidence provided by Mark Zuckerberg himself. Zuckerberg, a few years ago, presented a chart that is crucial and that cuts to the absolute core of how these platforms are designed. On the x axis he had shown how violative the content was, or how close it was to breaking the rules of the platform, and on the y axis he plotted engagement levels. You find that there is a very flat line on engagement for low-violative or medium-violative content, and right at the cusp of being violative, breaking the rules of the platform. Then it shoots up with an exponential curve. So you find the most near-violative content. Of course, we know that engagement drives amplification. Those two facts are why we see the churning of fringe violative content into the mainstream and that the normalising effect is quite substantial.

Thank you. I would like Ms Spring to respond.

Thank you very much. My answer will focus very much on how the algorithms—the recommendation systems across all the main platforms—are designed to promote engagement over safety. What that means is that they are very much running on emotions—content that will trigger a reaction. It is that content that will get more eyeballs, and as a consequence the social media companies can sell more ads and make more money. It is core to how they work. Almost every investigation or podcast series or anything I have done over the past year has come back to the question of prioritising engagement over safety. I was listening to the previous panel and there was a lot of discussion about the riots and the impact of misleading, false or hateful information that was recommended or promoted during that time, and again it very much comes down to the question of emotional content that triggers a reaction. It is important to talk about what can incentivise users to post in that way: not only the attention, views and likes, but also the monetary gain. I can talk in a little more detail about the way people, not just the social media companies, can make money from this type of content themselves. Beyond the specific situation of the riots in the summer and the shocking attacks that unfolded in Southport, there are so many examples, such as those I investigated for “Panorama”, of young teenagers and young men in particular being recommended hate and violence that they don’t want to see, on social media platforms. That, again, comes back to the algorithms prioritising engagement over safety, pushing them content they think they will react to. A lot of the social media insiders at companies I have spoken to across the board have corroborated that idea and spoken more about the way algorithmic pathways can lead people to different places. Last year was a huge election year, and we saw something similar in the proliferation, sometimes with AI-generated content, of unfounded conspiracy theories that played into people’s existing political biases or views and the often legitimate distrust, questions and concerns that they had, but that could lead them down rabbit holes on particular social media platforms. I am speaking to you from the United States because I am out here for Donald Trump’s inauguration, for the BBC’s “Americast” podcast, and I spent a lot of time yesterday talking about the prominence of social media bosses at that inauguration and the relationship between those tech bosses and Donald Trump. That, in some ways possibly marks a new era in social media, where there is a lot of conversation about freedom of expression and the importance of people being able to say what they want to say on social media. There are positives to that discussion, but a concern that I have heard from people affected by that kind of content is the way that it can be seen by so many people and so many eyeballs. That comes back every time to the algorithms and the recommendation systems that are essential for the social media companies to work in the way they work right now.

I agree firmly with the two previous panellists’ points. All the big social media platforms have spent billions, if not trillions, on optimising recommendation algorithms—their search feeds—to understand their users as accurately as possible, with the aim, exactly as Imran said, of user retention, for more time on advertising. The most alarming thing is that, because they are all AI-driven, nobody knows how they work exactly. We have bits and pieces, but they are completely opaque. I will talk a little later about some of the research; even researchers have a very limited idea of how these things work. When we talk about the business model—yes, they spend billions, because of advertising revenue—one extremely interesting point is that on one of the biggest platforms, Twitter/X, users can buy algorithmic amplification. You pay a certain amount every month for a blue tick and you immediately go to the top of the feed. You are boosted. When we look at big disinformation incidents—I assume we will talk about Southport and the aftermath of that, with the riots—we can see that the blue ticks are responsible for a huge amount of the misinformation that is out there. One final point that I think is very important is that we talked about Mark Zuckerberg’s blog post in which he correctly made the point that as content gets more and more fringe it increases. If someone from a tech platform were here, they would say they were taking active countermeasures, as part of their business models, to try to reduce that. They all say in their site rules and terms of service that they are going about trying to de-rank content and make it more difficult for people to find, as it gets closer to that point. I am very sceptical about that: I think that events over the past few years suggest that platforms are marking their own homework when it comes to reporting those kinds of result.

That is something that the Online Safety Act is meant to change.

We are legislators, trying to work out how to legislate and control on behalf of the public. What should we think is the crime in this ecosystem? What is the act that we need to try to legislate against?

Great question.

Crime is the wrong word. The best part of the Online Safety Act is that it is what is called a systems-and-processes piece of legislation. It is about making sure that platforms are compliant with a certain set of rules. In the UK, with the Online Safety Act, there is a very sharp divide between legal and illegal content, at least as far as adults are concerned; children have their own measure. When we look at Europe and the Digital Services Act for the very large platforms, it is slightly blurrier. We can talk about that as well. The fundamental principle when it comes to speech acts is that whatever is illegal in the offline realm should be policed in the same way in the online realm. That is the fundamental principle that fairly balances human rights as well as the harms that can come as a result of online speech.

Thank you. Imran or Marianna, would you like to add anything briefly to that?

I will let Marianna go first, if she would like.

The key is to focus, as was being said, on the real-world consequences of this. I always caveat these answers with “I’m an investigative reporter, not a campaigner,” so lots of my job is about exposing the problems rather than coming up with the solutions necessarily, but I think it is crucial that people listen to the people who are harmed and affected by disinformation, hate and other kinds of content on social media, whether they are teenagers or people across the country and the world who are affected and feel they are not being listened to. For me as a journalist, it is about accountability for the social media companies, transparency and feeling as though when you send them a right-of-reply email you are able to get some kind of answer about what they are doing and what is going on. A lot of the questions for me around the Online Safety Act come down to that.

Shortly after the riots, CCDH convened a roundtable of DSIT, the Home Office, Ofcom, ISBA, which is the representative body of the British advertising industry, the Met police counter-terrorism command, antisemitism campaigners and Islamophobia campaigners. We looked at the powers we already had prior to the Online Safety Act, and the powers we will have under it, and we did a gap analysis on the powers we would need in the future. We had five recommendations. The first one was to reintroduce the misinformation risk assessment and reporting requirements under the OSA. That can be done through statutory instruments. The second was to increase the capacity to identify emerging harms by giving us better access to data. One of my great frustrations with the Online Safety Act is that we do not have any transparency requirements under the Act. They were introduced under the Data Act subsequently. Those powers need to be further enhanced. It is still one of the weakest regimes when it comes to transparency of the algorithm, the enforcement mechanisms and the advertising. Those are the three crucial components of how the models work. The third recommendation is to have a crisis response mechanism for the regulator. The fourth is to close the loopholes, the small high-risk platforms, which the Secretary of State has refused to do. The final one is to address the advertising and commercial incentives. Every advertiser I have ever spoken to is outraged to know that they are essentially funding content that leads to riots in the UK. They want more transparency and control over the way their adverts reward certain types of content, and by helping them to do that we can help ourselves too.

That is a very important point, which addresses some of the questions that are coming.

Marianna, I would like to understand this. When misinformation goes from online portals into mainstream media, what is the responsibility of the mainstream media not to give air and credence to misinformation? We saw this in Southport, where mainstream media news sites were picking up from social media platforms certain mosques that did not end up being targets. What is the role of mainstream media in not giving air and credence to some of the misinformation online?

The best way to answer this question is to focus on my own experience in the journalism that I do. When it comes to covering myths and disinformation that are spreading on social media, there are two principles I stick to in the reporting I do. The first is understanding how viral or prevalent a piece of misinformation or disinformation is. Is it reaching tens of thousands, hundreds of thousands, or millions of people? Has it crossed the threshold of us needing to understand the impact it has had because a significant number of people have been reached by, recommended or exposed to that bit of disinformation? The second thing I always assess is the real-world consequences of that. Has that specific bit of disinformation or misinformation led to violence, or has it contributed to harming an individual or a group of people? When either of those lines is crossed, it is important that we look at it and investigate what has happened. It is also important to think about how we do that and the approach we take. As I mentioned in a previous answer, I focus a lot on the real-world consequences and understanding who is affected and harmed, and also how it happened, helping the audiences at the BBC, whether that is across podcasts, online or TV.

Marianna, that wasn’t my question. My question is what the responsibility of mainstream media is to fact-check before they spread the misinformation further, as we saw during the riots.

I appreciate that, but I can only talk about my journalism; I am not a BBC spokesperson. I am talking to you about my experience of covering these kinds of stories. I am sure everyone sitting in this room agrees it is important that the media across the board get stuff right, that they get the facts right and that they carry out all the necessary journalistic checks, whether that is speaking to people on the ground, or speaking to relevant sources and experts and everyone else about what is going on. I focus specifically on the spread of harmful, hateful content and disinformation on social media. In particular, during the riots I saw how the trust people afford to credible outlets was weaponised, inasmuch as we saw—I am sure we will talk about this in a later answer—pseudo-news sites that were sharing false allegations about the attacker in Southport, for example, which contributed to the spread of disinformation, but which people seemed ready and inclined to believe because they thought they were real. That is more the realm I spend my time looking at.

I appreciate you are not a BBC spokesperson. Chair, I think this is an issue we need to pick up because, as we keep hearing from witnesses, one of the things that happened during the riots is that essentially it gave credence to misinformation online. I have a question for Joe and Imran. All Governments have some kind of disinformation monitoring of networks. I would like to understand what you think of the UK Government’s efforts on that. How do they compare with a country like the US in terms of how the Government are monitoring and taking action on disinformation and online platforms?

The US is an interesting comparison. Like almost every other country, we are not doing all that much. As I alluded to in a previous answer, the UK’s approach in legislation has been to demarcate very clearly the difference between legal and illegal. There is a lot of value in that; it just leads to some unpleasant consequences. When we talk about these sorts of things in the US, there is absolutely no attempt whatsoever to stem any kind of speech, except very immediate violent speech and all of the usual sorts of things like libel. The first amendment provides hugely more protection. More importantly, when I speak to American colleagues, particularly in the US Government, they tend to take a more philosophical view about not trying to change other people’s minds about stuff. Compared with America, we are doing a decent amount, but if we look eastwards to Europe and what the Digital Services Act is doing and the way that seeks to counter disinformation, at least on the very large platforms, it is considerably less. For example, from the incident in Romania with the election and the European Commission invoking the Digital Services Act to seize data from TikTok and things like that, we can see that real legislative things are being done as a result. Clearly, we are doing less than the European Union, although that obviously comes with issues around free speech as well.

No Governments are doing it well. All Governments are to some extent reliant on civil society and organisations like mine to provide them with data and insight into what is happening on the platforms, and our ability to generate that data has been limited. Let me give two examples from the last year of why it is so difficult to get the data. We produced a report on X a year and a half ago that showed a rise in the prevalence of hate speech on the platform after Elon Musk took over. He subsequently sued us, not for defamation or for being wrong, but for the act of doing research itself. We won that case in California early last year. Nevertheless, it had a chilling effect on people even using the site to do research. His claim was essentially that the terms of service of X prohibit people from doing research on that platform. You can read it and consume that. The second example is the purchase of CrowdTangle by Meta. CrowdTangle was the primary tool used by us to study Meta platforms. They bought it for $1 billion only so that they could shut it down before the elections. We therefore could not check what was happening during the US elections last year on their platforms. It is becoming quite difficult. The US has quite a sophisticated apparatus for the study of overseas disinformation networks. For example, the GEC, situated in the State Department, is a relatively sophisticated outfit. We have worked with the UK Government’s counter-disinformation unit. I have spoken to colleagues from the Home Office and other spaces in their counter-disinformation unit. They are woefully underfunded. The reliance on civil society is becoming difficult, because a war is being waged by the platforms themselves on civil society—CCDH is at the forefront of that—in seeking to defund and close the spaces available for us both to research and to communicate our evidence.

Those are very important points. The monitoring of misinformation raises an important point.

I have a quick question for Marianna. We have talked about misinformation and disinformation, but is there a role to look at repetition and atmospherics? For example, if you look back at the 2011 riots, particularly on something like X, repeated exposure often to the same incident from different angles or perspectives can give the impression that it is much more long-standing or severe than it actually is. Certainly, in the 2011 riots the role of 24-hour rolling news repeatedly showing the same footage of buildings burning in Croydon contributed to a wider kind of contagion across London. If you are repeatedly exposed to violent incidents on X, for example, you get the impression that it is happening all the time everywhere. If you look at the Southport riots, the camera footage from within the body of the riot makes it seem like a large and unpleasant incident. If you look at the drone footage from above, it is quite a small incident along basically one or two crossroads, and two streets away there is no one and nothing is happening. It is not as if the whole of Southport was consumed in the conflagration. What is your view of the responsibility of platforms to give a sense of perspective about scale and longevity, and is that another form of misinformation or disinformation?

That goes to an important point about the way algorithms work. Essentially, they do what you have just described. They repeatedly push you content that you express an interest in and often that content will evolve or take you down a particular path or, in some cases, it becomes more extreme. I have seen that first hand with the kind of journalism I have been doing. I have many old iPhones that I run called undercover voters. I ran them around the UK election and in the US election. They are fictional characters that I create based on data. They are private and they don’t have friends, but they have social media profiles across all the main sites. What they allow me to do is test and see what kinds of content are being recommended to different types of users by algorithms based on preferences, location, the kind of content they show an interest in, their behaviour and so on. I have to do that because of the absence of information that is shared with us by the social media platforms. It is the only way of being able to interrogate what people are being pushed. What I have found from running those accounts over the past year, or the American ones that I have been running for longer, is what you describe. On particular issues, you see how a user will be recommended the same set of accounts on a site posting content that sometimes becomes increasingly extreme, so their perspective is distorted. It is important to think about that in a number of ways. When people are repeatedly recommended the kind of content that is being pushed directly to their feed, and I have spoken particularly to younger users who feel this to be the case, they feel as though it is intended for them. That can make them feel as though it is something they should believe, be interested in or like because it is based on their preferences. When I did a BBC “Panorama” in September, a teenager called Kai in Manchester told me it made him feel as though some violence and misogyny he had been recommended was intended for him and he should believe it. He felt that was distorting the perception of some of his peers and he would have to speak to them about that, because he didn’t really want to be seeing that sort of stuff. There is a question about social cues for people like that. He said to me that it distorts your sense of what is and is not okay. You see a video or post that has had thousands of views—millions, in some cases. It is where you take your social cues from, so you start to think that a particular narrative or rhetoric is normal and widespread, even if that is not necessarily the case. The final way of thinking about it—we talk about it a lot—is the rabbit holes people end up going down, the new kind of content and places they are exposed to and the way someone can go from having no interest in a topic at all to finding themselves believing something they never thought they would. During the election in the US, I spoke to two women who believed that the assassination attempt on Donald Trump was staged, without any evidence to support that idea. One of them was someone who had never believed this stuff, but she had been repeatedly exposed to that idea and the dislike of Donald Trump, and that led her to the point where she thought something that had happened had not happened.

That is a full answer, but it is a very important point. Kit, it sounds as if the answer to your question is that we don’t have any right to have a perspective.

One of the key points that has been made is what is happening in the US and how different countries are trying to regulate and get to grips with the issue. Given the fact that it is an international issue and we are seeing foreign actors and states using misinformation and disinformation as a tool in the new web warfare that is done online, do you think it is possible for the UK effectively to regulate, manage and mitigate this, particularly in the light of what we see with people like Elon Musk and with Donald Trump now back in the White House and the changes we are seeing in other countries? We can do all we want in our country, but how do we insulate ourselves and collaborate with partners around the world to make sure stuff is not spread from further afield?

CCDH has offices in Brussels, London and DC. We opened one in Los Angeles two days before the fires started, so it was not ideal timing. We speak to the European Union on the DSA. I am heading to Cape Town in February to speak to their online safety folks. We have been working with Australia and New Zealand, which were ahead of the curve because of what happened in Christchurch. Jacinda Ardern was incredibly forward-thinking in the Christchurch call. Australia has an e-safety commissioner, Julie Inman Grant, who is doing extraordinary work out there as well. There are countries seeking to legislate and regulate. For the main part, those people focus on systems, transparency and meaningful accountability. One of the problems with the Committee’s questions today is that if you put them to the platforms, they could gaslight you by just saying, “Look, we take down 99%, guys. We’re great.” [Laughter.] Oh, you have had the gaslight. I could smell the residual aroma in the room—I wasn’t sure if that was the mice or Meta. Of course, we have the serious problem that they mark their own homework; they provide their own data. We have had no transparency. Meaningful accountability requires that transparency, so there are things you can do.

What is the answer to Tom’s question?

The specific answer is to look at what Brazil did with Elon Musk recently. Elon Musk shut down X in Brazil because he was unwilling to comply with the law. It took days before he turned around and had to comply with the law as laid down in Brazil. I don’t think the Brazilian law is a particularly good one, because it allows them to take down individual bits of content. As someone who believes in freedom of speech, which is a profoundly British concept and a core value of our democracy, I am worried about something that says that an individual bit of speech can be made illegal by the Government, unless it would be illegal offline as well. We can see that platforms are willing to bend, but you are absolutely right that the pandemic of disinformation spreading across our world, corrupting our information ecosystems and leading to the degradation of science, truth, democracy and the values of our societies, is American. There is a reason I moved to DC in 2020, six months after opening CCDH. The worst and most sophisticated bad actors are weaponising the platforms to spread disinformation, particularly race and identity-based hate disinformation, and lawmakers have singularly failed to deal with the issue since 1996, which was the last time the United States Congress promulgated laws relating to online content: the Communications Decency Act 1996, which introduced section 230. It is their problem. We have to apply pressure as well on our friends in the United States to be doing the right thing.

I have very little to add; I completely agree. The only point I would make is that there are friends to try to co-operate with, but, as you raised in your question, there are clearly adversarial states that are explicitly creating armies of people to try to sow this. I don’t think there is any easy way out. There is probably an acceptance that this a problem that will exist. The idea of co‑operating with Russia, North Korea or Iran is for the birds. I completely agree with those points. There will not be a silver bullet.

Can we deal with this unless the US pulls its finger out, to put it bluntly?

You can do some things. You cannot fix the problem, but you can certainly mitigate it. Just because you cannot fully deal with something does not mean that you should not take mitigation measures. In some respects we are at an early stage of regulation of what is a new problem in our society. We should not be ashamed of the fact that we have made a noble attempt with the Online Safety Act, on a multi-partisan basis, to try to do the right thing, but we should be ready immediately to start thinking about what we have missed, where the technology is moving, whether or not it is working effectively and to iterate and iterate again. We will have to do that with what is new and interesting, with all the positives that come with it. Let’s not forget that these are great platforms. The ability to shrink the world and make Timbuktu seem as close as Tulsa or Tyneside is a wonderful thing.

Thank you for bringing Tyneside into this. We appreciate your enthusiasm.

It is about all of you in front of us. Brazil demonstrated that there is power to rein it in. It is one thing to have an Online Safety Act, but another thing to enforce it.

The point I was going to make is that this Committee is looking exactly at how we can ensure the UK can make a difference, particularly for the communities we heard from earlier this morning.

What exactly is the Online Safety Act missing that we should be bringing in?

Can you answer that in 10 seconds, Joe? Imran answered it earlier. Do you have something to add?

I think my views are pretty complex. Feel free to move on.

We have talked an awful lot about the business model of these platforms causing the behaviour we are trying to examine. I find it is always a good rule to follow the money. Mr Ahmed, you are the first person to talk about advertisers. I find it slightly strange that no one has yet identified the behaviour of the paymasters in funding the behaviour of the platforms. Is there anything we should be looking at in terms of either giving the advertisers more control over the way their adverts are placed or making them responsible for what they are funding? Is there any evidence that events like the riots make money for the people who are advertising?

Let me speak to the first bit, which is whether or not advertisers are happy about the situation. Absolutely not. We work with ISBA, which is the UK advertising industry’s lobbying body. It is emphatic that it wants to reduce the prevalence of advertising on malignant content, hate content, disinformation and the sort of stuff that led to Southport. We were working with the Global Alliance for Responsible Media, a sub-unit of the World Federation of Advertisers based in New York, which was building assessments of what sort of content they would allow their ads to appear next to. The Global Alliance for Responsible Media—GARM—shut down last year after Elon Musk sued it for acting as a cartel that was stopping him making money. He also sued advertisers for not advertising on his platform.

He didn’t get very far with that.

Exactly. It is like suing someone for not being your mate. People have the right to decide. Bizarrely, the first amendment right is with the advertisers; freedom of association is protected by the first amendment. Absolutely, they want this. What we could do is open up transparency in advertising. At the moment, advertisers do not know where their adverts appear. If there were an open database of what content was monetised by whom, we would have more clarity and accountability moments. Civil society could ask a question. For example, the last report we did was on eating disorder content on YouTube. We found adverts for Nike appearing on content encouraging young girls to go on a 500-calorie-a-day diet, which would kill them over time. Would Nike be happy about that? Absolutely not. Did they know until that moment? Probably not, but once they know, there is a considerable responsibility on them to do something about it. The irony is that people call us a censorship organisation and say we are studying censorship. We are not. We want a more informed discourse through greater transparency.

That question on advertising is really important, especially as it drives business models.

Yes. It highlights something that we should be talking about. It is not just about the advertisers, which is obviously an important conversation about how the platforms themselves are funded, but why people make the kind of content you are describing. It is because it will get eyeballs and clicks, and as a consequence they are often able to make money from that. A lot of that comes down to the way social media platforms often have incentives for creators and influencers. They have allowed people to make money. In some cases that has been a positive thing. We have seen how people have been able to build entire careers on the basis of their content, but you can also see the risk that that poses. I have certainly seen that at first hand, not least around the riots but also around the various elections. For example, there was a pseudo-news site called Channel3Now which was one of the first to share false claims about the identity of the attacker following the attack in Southport. Because it looked credible and like a news site, lots of people were sharing it. There had been various accusations levelled at the site, including suggestions that it was connected to Russia in some way, or it was part of a foreign influence operation. When I tracked down the people behind it, I didn’t see any evidence of that. I spoke to one of the people involved in the site. There was one based in America and one in Pakistan; one was a hockey player in Nova Scotia. From the conversations I had, it appeared to be about clicks and money. They knew that the quicker they put something out, whether or not it was true, the more money they could make The same applies to some of the big accounts on X right now that I have tracked down and spoken to. Some are newly created and were made in October, yet they are making thousands of dollars, if not more, from the content they are posting, including content that is not true or has been accused of promoting hate as well. What you are describing is symptomatic of the way the social media sites operate. If we talk about X specifically, it has made changes to how accounts can make money. If you are a premium user, you sign up and buy a blue tick and you are able to make money from other premium users interacting with your content. The more controversial, exciting or upsetting that content is, the more clicks and views you will get and therefore the more money you can make. You can see how that motivates people and has explicitly motivated some of the people I have spoken to who also appear to have ideological aims. I have contacted X quite a lot about the question of monetisation. They have responded to one right-of-reply email I have sent since Elon Musk took over. I have sent dozens of emails and it is hard to get answers out of them, but they continue to say that they are defending users’ right to freedom of expression.

It is the transparency of that freedom of expression that is not being defended.

I want to pick up something Dr Whittaker said earlier, and it has been touched on by Ms Spring and Mr Ahmed. I was struck by the iPhones dotted around the country and the example of Nike. Dr Whittaker, you suggested that in effect we do not have enough evidence of social media algorithms recommending harmful content. Why is that? What are the gaps, and what would be helpful to us?

As with the Online Safety Act, the key distinction is to draw the line between illegal and what you might call legal but harmful content, which has actually been most of what we have been talking about today. Looking at the whole basis of academic and think-tank research, there is not much evidence to suggest that illegal content is being amplified, but there is a bucketload of evidence to suggest that legal but harmful content is being amplified. Starting with illegal, one might take the optimistic view that social media platforms are so good at getting rid of illegal content that it is not there to be amplified. You might be able to detect a certain amount of sarcasm in my voice. There is a really big knowledge gap when we talk about things like terrorism and suicide content. With all of these things, there is not a lot of research. When we talk about legal but harmful content, there are about 50 empirical studies, and most of them point to the fact that it is promoting in some way. However, as I said earlier, there are some really big gaps that limit my ability as a researcher and all other researchers to do this. We have heard one already: social media platforms have made it as difficult as possible for us to research internally. That basically means that almost every study looks at YouTube, because for years YouTube—it is a bit technical—had an API that means you can log on and look at those things. There are a whole host of other issues. Almost every study is done in the English language. It is good when we are talking about the UK context, but when we are talking about the genocide in Myanmar and the things that the Facebook whistleblower evidence spoke about, guess what? It turned out that there were only two or three Burmese speakers doing content moderation on Facebook. We talk about how things might be amplified and the ability to stem that flow when it becomes a problem. English language is another big problem. Finally, almost every study is what you might call a black box study, and this includes the ones that I have done. You try to change some things on social media and see what the algorithm spits out at the end. It is interesting and informative, but you are not able to understand what that algorithm is doing. There was one study under the previous ownership on Twitter where they used internal data, and it revealed some really interesting things. Despite all the studies pointing in a certain direction, there are major limitations that mean we should be cautious about the conclusions.

Thank you very much. Adam, do you want to come back in on that point?

Thank you, Chair. In the discussions we have been having, which have been very positive generally, we have talked lots about the Online Safety Act, but—for us to take away as legislators, picking out some of the holes that we may or may not have identified in the Online Safety Act—where do you see the future of legislation to combat some of the issues that we are talking about?

We have been doing some thinking about what we have called Project Online Safety Act 2.0. What does the next iteration look like? There are three key aspects that we need to deal with. There are algorithms and more transparency around those—the way the algorithms work. People call them a black box. Even for the platforms themselves, they are unpredictable because they are so complex. It is not just one equation; it is lots of nested algorithms—

One under the other.

Exactly, underneath each other. When Meta studied QAnon and how someone could be radicalised into QAnon, they created a profile for someone called Carol, and then worked out how many steps it took for her to be radicalised. It took a week before her page was full of stuff that they said broke their own rules. That was revealed by Frances Haugen.

Sorry, could you answer the question?

AI is the second part: AI and having more clarity and rules around that. Finally, there is the advertising as well. If you did those three aspects and enhanced the current transparency regime, the statutory data access pathways for researchers and for organisations like ours, those are things that are obvious right now and that absolutely with alacrity need to be fixed.

Great—thank you. Joe, would you add anything to that, very briefly?

It won’t surprise you to learn that I am also in favour of researcher access to data. It is what we do. The future of legislation is first and foremost finding ways to enforce the legislation that exists. Countries around the world are passing Online Safety Act-type things. If they end up being toothless, it is not going to matter. In response to the earlier question, if America doesn’t pull its finger out, are we doomed? Yes, if the legislation that has been passed around the world does not end up having teeth, we will be doomed.

I would like to press you, but we are running out of time and we have a number of questions to cover. Marianna, is there anything that you would like to add to what has been said?

A lot has been said already, but there are a couple of things that we have not mentioned that are worth thinking about, in terms of where we go in online safety. Something I spend quite a lot of time talking and thinking about is social media literacy. For me, it keeps coming back to the algorithms, as I have been saying, and a lot of the solutions or ideas that are discussed sometimes feel like they are band-aids on a gaping wound. A lot of it is about accountability for the social media companies, and feeling, as journalists, as though we can ask them difficult questions and they can answer those difficult questions, as well as feeling that the public are equipped, in the absence of that, to deal with the kinds of content they are being pushed and recommended. I hope that in some ways some of the work I do functions a little bit like social media literacy. That is something that is really valuable for politicians to think about, because it is one of the best ways of guarding against some of the harms we have been discussing.

Thank you for that. We are aware that, after the terrible killings in Southport, Twitter, or X, was recommending the false name of the supposed assailant. It was trending; it was being recommended to all X users. In the light of your answer to that last question on future legislation, I want to raise the issue of generative AI affecting the production and spread of misleading and harmful content online. You touched on it briefly, Imran. I am thinking of a Google search responding with discredited racial ethnic theories when asked about the IQ of different countries. I am also thinking of Apple trying to synthesise BBC headlines in a way that led to misinformation. In future legislation, would you add any other points specifically to address generative AI?

The core problem with all of this is the friction, the speed and the cost of producing and distributing disinformation. Social media reduces the cost of spreading disinformation for every marginal user and every marginal message to zero. The current cost of spreading disinformation on social media is the production side. Generative AI reduces the marginal cost of producing every new message to zero. A very basic script that you could write for producing disinformation is to get generative AI to produce 1,000 different iterations of a message to disinform people, to put that on social media, see the engagement levels, and use that data to continue to refine through the generative AI algorithm the disinformation itself. You have a perpetually improving disinformation machine. Should we be dealing with that? Clearly, having just described how we are essentially in a thermonuclear age of disinformation, it is time that we had something in place for generative AI. The two problems are guardrails in terms of their output and curation of what is put into those models. Transparency helps it. Transparency on how they curate the content they put in, and transparency on the guardrails that they have on what is outputted, allows for a rich and informed dialogue with civil society and lawmakers on how those platforms should be designed and what the limits on them should be. One very good example I can give you is the study that we did on generative AI and eating disorders. We told a bunch of generative AI platforms that we were a young girl and we wanted to know how we could lose weight. One of the platforms said, “Why don’t you try heroin?” Another said, “Why don’t you try ingesting a tapeworm egg?” Another wrote us a plan for an 800-calorie-a-day diet, which will kill someone over a period of time.

I am a terrorism/extremism researcher first and foremost. When we look at the bad actors operating in this space, they are clearly all starting to adopt generative AI. Some of the biggest concerns are perhaps not the best founded. There were concerns about jihadists resurrecting bin Laden to try to radicalise people. We see headlines and things like that, but that is not the concern. It is exactly as Imran suggests: the ability to do things at a much greater pace. When you have transnational terrorist organisations, the ability to translate your propaganda into two dozen languages in a matter of seconds provides opportunities that you would not necessarily have had before. The creation of deepfakes is something we are seeing as well, specifically from a terrorism/extremism perspective, and that is something that will need to be dealt with. The guardrails are currently all operated under a self-regulatory method. We trust ChatGPT, and it has got better at stopping things from bad places, but as a citizen I want a Government to have signed it off and for it to be conforming to a regulatory scheme.

Marianna, would you add anything?

Yes. It is interesting thinking about the arc of what Joe was saying about generative AI and the real fears people have about the impact it could have on the spread of disinformation and how that has panned out. Most of the cases and examples that I investigated over the past year that involved, in particular, deepfake images or audio being created were only effective because this kind of content was recommended by the algorithms to a widespread audience—to the question before about the distortion of perspective—and people were primed and ready to see this kind of content and it perhaps confirmed their biases. I looked at AI-generated images of Donald Trump alongside black voters created by his supporters, which, when I spoke to some voters who had seen these kinds of content, was effective only when it seemed to play into their existing biases and they were being pushed it, and then they believed it to be real and to confirm their perception of the individual. The most worrying example that I have seen was a clip I investigated of London Mayor Sadiq Khan making comments that he had never made about a pro-Palestinian march being moved to accommodate Armistice Day celebrations. That was effective because it was recommended, pushed and spread in places where people were primed to believe it and it played into their existing perceptions. It had a real-world consequence; it appeared to exacerbate tensions offline. That is just another thing that has been flung into what, as we have already discussed, is a pretty complicated and messy social media ecosystem.

Thank you. There are some lessons and important suggestions for the Committee to look at when it comes to generative AI specifically. I turn to Paul, a previously practising journalist, to raise issues around freedom of expression and misinformation.

You rightly referred, Imran, to the fact that freedom of speech is a very British value, and you also referred to the American constitutional protections. How can a UK Government tackle misinformation and disinformation without infringing on freedom of expression? I am thinking in particular of some free speech absolutists who say that the Prime Minister was wrong the other week when he said that freedom of expression does not include the freedom to spread lies. Some absolutists would say it includes the freedom to spread lies as long as it does not have a causal link to harm, harassment or violence. What is your answer to that?

I am going to ask Marianna to start, and then Joe and Imran, briefly if you can. This is obviously a particularly important issue for the Committee. I should say that we are still in discussions with X as to who will appear on its behalf before the Committee.

Thank you for the question. Perhaps it has been widely accepted that stuff that is not true and that is harmful in different ways will exist on social media because of the scale of social media, because of how many people can use it and because of how it works, in that it does not rely so much on proactive moderation as on reactive moderation of content. The question is about the right for those lies to be seen by as many people as they are. It comes back to the algorithms. If a post is untrue or false and it is seen by 10 people, that is a completely different kettle of fish from it being seen by 10 million people. What we see right now on some social media platforms—X is an example—is that whether content is true or not it can be amplified and recommended. I have seen ample evidence to support that. I have even spoken to people behind accounts who acknowledge it is the case; whether they are true or not, they know that their posts can be seen by millions of people, they can make money from them, and there are no real repercussions. There might be a community note that is added to a post—other users fact-checking that post—but the problem is that often when a post has already been seen that many times the impact has already been felt. There are real questions about the extent to which any labelling or fact-checking succeeds in correcting the narrative.

Part of what you are saying is often referred to as people having the right to freedom of speech but not freedom of reach. I suppose the question is how that non-right to freedom of reach should be mediated. Imran, would you like to add anything?

Paul, I think the debate on this issue has become really convoluted and complex. I blame this on the invention of the terms “misinformation” and “disinformation”. Misinformation means being wrong; it is an untruth. Disinformation adds in mens rea; it is a lie, an intention to deceive. People have a fundamental right to express both by law. They have the right to lie about whether an asylum seeker was behind the events in Southport. Algorithms lie. Specifically, when they select misinformation and when they know that the most violative content is the stuff that gets the most engagement, and when they then choose to reward that engagement with amplification, it is the design of the systems that adds the mens rea aspect. Perhaps people can be wrong. They can be stupid. People have the right to be dumb. Platforms know better. They already know that they are complicit in the promotion of that misinformation, and in doing so it becomes disinformation. These are disinformation machines. The framework that CCDH has always had is “Don’t criminalise speech.” That is a bananas thing to do. Safety is created by transparency, which allows for meaningful accountability, and then when there is clear evidence of negligence, which is a core concept under British civil law as well, or of knowing indifference to where disinformation may lead to human harm, there should be costs attached to that. One mechanism for imposing them is regulatory cost. The second mechanism is a market cost, which is that by creating transparency for the advertisers too they can withdraw their revenue. It is a double way of impacting the bottom line and of changing the system. Right now, the problem is that Southport was profitable for X and the other platforms. We need to make sure that in future it is costly, whether that is through the advertising or through a transparency-enabled system that allows for an open and informed discourse on these issues, or if there is sufficient evidence of mens rea, as there may be in the case of X, through regulatory costs, too.

What we are hearing is that there is no cost to lying but there are revenues associated with lying. Joe, would you like to add anything briefly?

The internet has fundamentally changed how we deal with issues of free speech. We have 2,500 years of philosophy and law and trying to work out that here is the state and here are the people, and people occasionally say something that is over the line and get punished. Since about 1995, there has been a third part of the triangle with the tech companies. For about 25 years, we have been outsourcing our free speech public square discussions to people situated in San Francisco and silicon valley to make those kinds of decision for us. We are now, within the last three or four years, in the EU, the UK and Australia devising laws that fundamentally change that calculation. I am quite distrustful of the phrase “Freedom of speech doesn’t equal freedom of reach.” It is a healthy sign of a functioning democracy not only that all views are heard, and there is some kind of equality in the way different views are heard, but there are some instances in which certain kinds of fringe views are actively promoted in some way. At the BBC, they always do their best to try to get a range of different types of views on any one issue. That same sort of principle can extend to algorithms. We have a common-law system that deals with these sorts of things quite well. If speech is so reckless that it is putting people in danger, that is the sort of speech that should be outlawed, and there is no reason why platforms should be immune from that. It is the century-old adage of falsely shouting “Fire” in a crowded theatre. There is so much speech that took place and that people have been sentenced for around the riots that was exactly that. It is in some way just about enforcing the long-standing laws that have been built up over several hundreds of years.

Or finding new ways for that.

I have a very brief final question for Marianna. We are talking a lot about the business models and the algorithms, and the computerisation of that process, but what about the human element like the Facebook community groups? There are many tens of thousands of them in every town and city up and down Britain, and they are often controlled by two or three individuals who are the admins for the groups. What is the role that they should have in any kind of regulation or legislation? What threat have you seen of how they filter these messages, rather than just the algorithm?

Yes, you are right. People can be recommended specific groups that might be relevant to them—particularly, like you say, local groups or places where they might actually know some of the people involved. They can be a really positive community resource in lots of ways, but there is a risk posed by them. This extends beyond any one particular social media platform. It is anywhere you can join a large channel or group where you might have stuff in common with people and you might share either views or a local area. It all comes back to the question of trust. You are more inclined to trust people you know or who you feel are perhaps more similar to you or people you agree with. I have spent quite a lot of time looking into frenzies that occur on social media around disappearances or when something bad happens and when a family is affected, particularly when there is a disappearance where there are no answers. You find a new genre of sleuth or online detective getting involved. That has happened not just in groups but across the board; people are making videos and their own kinds of content. People, particularly in these kinds of environments, feel as though they can put forward ideas, which is their right to freedom of expression, but they often end up not being true and can actually cause harm to people who are experiencing something terrible. Whenever you are thinking about all the different social media platforms and the kinds of content on them and the spaces that exist, it is important to understand how each of them functions and how each of them permits the proliferation of stuff that might not be true or might be harmful or unhelpful. Groups definitely fall into that category, particularly those that are harder to enter and are closed or private in some way.

Thank you very much, Marianna. We are past time, but I want to give each of you an opportunity to tell the Committee—it should be a maximum of five words each—which organisations and bodies you think the Committee should be holding accountable for the spread of misinformation. Identify Government organisations, private sector organisations, community organisations or groups. Could you tell the Committee which you think we should be holding accountable?

Start with the social media companies. They are the primary offender in the violence that we saw last summer and in the increasing spread of disinformation online. Secondly, you have to treat the mainstream broadcasters as complicit, too. They have been just as complicit in reporting that reifies trends on social media into facts in the mainstream media, too—what people are saying on Twitter. The third people are yourselves, the politicians. There are people who have opportunistically weaponised trends on social media to spread disinformation and to tell other people that they believe in it too and therefore they are on their side. We have seen that, whether it is antisemitism, Islamophobia, anti-vax disinformation or all sorts of other malignancies. We need to hold ourselves to a different set of standards.

Okay, that is great. Politicians, social media companies and the third one.

The only thing to add is the people who are out there committing violence. It is easy to get lost in that when we are talking about big social media platforms. There were thousands of people out there committing violence last summer.

Good point. Finally, Marianna.

It is important to be able to hold the social media companies accountable for the way the algorithms and systems work. It is really important to hold accountable the people who are spreading this kind of content, pushing it and promoting it. It is also important to understand why this is happening, and that approach is something that I really favour. It is important to ask questions of politicians and everyone else who has a part to play in this. As someone who spends a lot of time meeting the real people who are affected by this stuff, whether or not they believe it or are harmed by it, my final plea to you is about the families affected, the communities affected. They often feel like they are becoming ping-pong balls in a bit of a political game, whether it is the debate about freedom of expression or anything else. It is important to stick up for them. They feel that nobody is listening to them, particularly not the social media companies. It would be good if someone listened to them.

Thank you, Marianna, and I thank all members of our panel. On that last point, all the Committee members feel very strongly that it is part of our job to stand up for those who are impacted by the spread of misinformation on social media. We very much value your evidence. You have raised a number of important issues and points that I think had not been raised before: how we monitor misinformation in the UK and globally, the right to assess and research on social media, what duty we have as individuals, transparency, particularly the transparency of algorithms, and the connection to advertising revenues. They have been very powerful testimonies. We could go on a lot longer. Thank you so much for your contribution.