MoD’s ‘ChatGPT’-style software to speed up military review could be attacked, critics fear

Prof Mariarosaria Taddeo, an Oxford University academic specialising in the ethics of defence technologies, told Politico: “It’s not a problem if AI is used to support decision-making, but the question is if it is used to support decision-making or if the AI is adopted without any critical thinking.”

She added: “Even if the AI is internal to an organisation, are we basically creating a huge Trojan horse? AI is really fragile, it can be attacked.”

The Telegraph understands that the Government considers such an attack to be a low level threat because the software will be stored on an MoD system, which is monitored.

A defence source familiar with the plans said the “sophisticated language model” will be able to answer substantial questions by the user and use the submitted evidence to formulate its response.

The source said: “It sounds like Chat GPT. Fundamentally, it is a summarising tool. It is not producing data for a decision, it’s providing millions of words and summarising them.”

As part of the SDR participants were invited to respond to 24 questions on topics, which ranged from the future of defence to the culture of the services and how to best support Nato.

The responses, which were between 2,000 to 4,000 words per question for those directly involved in defence and 500 words per question for the general public, will be inputted into the AI system so that they can be discussed over the next two months.

Those sifting through the answers can ask the AI questions such as “what is the general opinion on the state of the Army”, to which the software will give an overarching response based on all the information it has received.

The software, which has been purpose built by Palantir, the US data giant, which secured a contract with defence business Babcock to help build the Royal Navy’s new fleet of warships, will crunch the responses into a more digestible format.

The software marks the first time the Government has used AI to assist with a significant review.

It is understood that the MoD believes AI technologies will transform every aspect of defence so that large datasets can be navigated easier.

An MoD spokesman said: “We have been transparent about our ambition to use AI for a wide range of defence applications, and the team are utilising this technology to help review and analyse high volume of submissions received to the Strategic Defence Review (SDR).”