In Castelfidardo, in Italy’s Marche region, a nine-year-old boy used an artificial intelligence application to add a snake to a photograph of his bathroom. The photograph even shows a trace of a finger over the camera lens. According to newspaper accounts, subsequently reported by Tgcom on 15 November, the image was shown to adults as evidence of a real encounter. Firefighters attended, investigations by the Carabinieri forestry police’s CITES unit began, and the photograph started circulating. Later, during the checks, the boy admitted that the snake had been added digitally.

Corriere Adriatico also published my comment on the incident. But I believe this story calls for a broader reflection than a news report allows.

Because we risk dismissing it too quickly as a prank that went wrong. Or focusing mainly on yet another demonstration of what artificial intelligence can do.

What strikes me most is the disparity: a nine-year-old child was able to set adults, emergency responders, investigations and news networks in motion around something that did not exist.

In this sense, the incident shows how a community can be thrown off balance. An action within a child’s reach can have consequences that require many people’s involvement to understand and manage.

The snake was imaginary. The time, worry and resources involved were real.

The AI-manipulated photograph: a snake digitally added to a bathroom
The photograph of the supposed snake, later found to have been manipulated with artificial intelligence, that triggered the alert in Castelfidardo. Reproduced from Paolo Clini’s LinkedIn article.

The ease of doing something has outpaced the ability to understand its consequences.

We do not know how familiar the child was with the application, how he had gained access to it, or what rules he had been given. We cannot turn these gaps in our knowledge into a public trial of his family. But his parents, and all parents, must ask themselves about their responsibility and the freedom of action afforded to a nine-year-old child.

We can, however, acknowledge the question this case poses to us all: what capabilities are we giving children, and what preparation accompanies them?

Knowing how to use an interface does not mean understanding the implications of what one is doing. Achieving a convincing result does not demonstrate maturity, awareness or critical competence.

This misconception affects adults too. We see a child navigating a screen quickly and conclude that they have mastered the tool. But dexterity concerns the immediate action. Responsibility means imagining what will happen afterwards: who will believe that image, who will be frightened, who will have to respond, who might suffer harm.

The Castelfidardo case makes this gap visible. And it forces us to take it seriously.

An image also becomes dangerous through the credibility we lend it.

A fake does not act alone. It is shown, described, forwarded and interpreted. It enters a conversation between people who know one another, reaches a chat, passes to a page and arrives at a news outlet.

At each step it may acquire authority. Those receiving it see that others have already shared it; those reading the story may assume that someone has already checked it.

A chain can thus develop in which everyone relies on the verification they imagine someone else has carried out.

This is a collective vulnerability. The Castelfidardo incident demonstrates it. The technical quality of the image matters, but so do the story accompanying it, trust in the person showing it and the emotion it provokes.

An imperfect photograph can be persuasive if it arrives at the right moment, through a trusted relationship, in a situation of fear.

For this reason, our response must address the entire process that turns an image into news and news into a decision.

Those who protect us must be able to verify without giving up responding.

We must be clear on this point. When a possible danger is reported, emergency responders must assess it and protect people. The fact that an alarm turns out to be unfounded does not automatically make the response wrong. This episode also occurred around the time of school bomb alerts involving implausible emails. It therefore raises questions about the tools and skills available to those managing such situations.

Il Resto del Carlino had already reported the firefighters’ doubts and the possibility that the photograph had been created with artificial intelligence. It would be wrong to describe the incident as though nobody had raised questions.

The issue is how to strengthen verification while managing a possible risk.

When I was asked for an opinion on the image, I pointed out some suspicious details, including blurred edges. Of course, blurring does not automatically establish that an image is fake, but it is a sign that should not be overlooked. Such signs call on us to take greater responsibility.

We need questions about provenance, the original file, direct witnesses and the evidence available.

For those reporting the news, this also means distinguishing precisely between a report and an established fact. And giving corrections appropriate visibility by updating the initial account.

The most troubling question concerns what could happen deliberately.

According to the accounts, this incident began as a prank. But it forces us to consider scenarios in which harm is deliberately intended.

Think of a fake image used to discredit a business, attribute behaviour to a person or suggest a danger in a public place. The damage can begin before a correction is available.

I am not saying that one incident demonstrates how frequent these phenomena are, or justifies predicting a society in permanent panic. I am saying that it reveals a concrete possibility: producing misleading content can be far simpler than checking it, correcting it and repairing the consequences of its spread.

There is a further risk. If we become accustomed to considering every image potentially false, we may end up rejecting authentic evidence too.

We must build trust that knows how to verify. A community that believes everything is vulnerable; a community that no longer believes anything is equally so. Then there is the risk of imitation. Perhaps the greatest danger.

Laws are necessary. Education must make them effective in everyday life.

The European AI Act provides for transparency obligations for certain systems and content: these include technical marking of synthetic outputs by providers and disclosure obligations for certain uses of manipulated content. These are important measures, with specific scopes and exceptions.

It is not correct, however, to sum this up by saying that ‘every AI-generated image must always have a visible label’. The regulation distinguishes between actors and activities and provides an exclusion concerning purely personal, non-professional activities for deployer obligations. This distinction does not remove other responsibilities that may apply.

Above all, a rule cannot replace understanding the consequences of an action.

We need rules, technical safeguards and producer responsibility. And we need people capable of understanding why an invented photograph presented as real can harm someone.

We must begin at five years old. This is the real challenge.

Until recently, I thought mainly of secondary education. This incident convinces me that we must radically bring forward the start of this education.

My proposal is to begin as early as five, with activities appropriate to children’s development. It is an educational proposal: I am not identifying a scientifically established threshold or proposing independent access to generative tools for young children.

We can work without screens too. Through a drawing, a story, a photograph or a guided game, we can learn to distinguish what we imagine from what we describe as having really happened. We can ask how someone frightened by false news feels. We can explain why, when something worries us, we should speak to an adult, and why forwarding content is already an action with consequences.

Education also means protecting the freedom of imagination: invention is an extraordinary resource. The crucial step is understanding that presenting an invention as evidence of a fact changes our relationship with others.

In its guidance on generative AI in education, UNESCO highlights the need to protect human agency, train teachers and adopt age-appropriate approaches. Within this framework, I believe that education in judgement should precede independent use of these tools.

Adults must also start learning again.

Families need to know the applications available to their children, set understandable limits, accompany their experiences and make dialogue possible when something gets out of hand.

Schools need continuing teacher training and time for education. An occasional session on online dangers cannot, on its own, support a change of this magnitude.

Technology companies must make features understandable and provide safeguards appropriate for minors and supervision tools that people can actually use.

Newsrooms, public authorities and organisations must prepare to handle content of uncertain provenance, using procedures proportionate to the possible harm.

None of these actors can solve the problem alone. But each must take a recognisable share of responsibility.

Humanities education belongs here, at the heart of the issue.

I have long argued that governing artificial intelligence requires stronger human education.

History teaches us to question sources. Philosophy helps distinguish a claim from the reasons supporting it. Art teaches us to read images and their power. Literature enables us to imagine life and the consequences of our actions from other people’s perspectives.

These abilities must develop alongside scientific and technological knowledge.

We can teach someone to produce an image with a few instructions. Educating someone capable of deciding how to use it takes time, relationships, examples and responsibility.

This is the investment on which we must focus our energies.

I work with artificial intelligence every day, in research and in the enhancement of cultural heritage. I know its extraordinary possibilities. That is precisely why I consider it urgent to address what this incident reveals.

I would like Castelfidardo to become a starting point: for education from early childhood, accessible learning for families, better safeguards and a shared culture of verification.

The child is nine years old. The responsibility for building a world capable of guiding him is ours.

Back to all articles ↗