Security professional Bruce Schneier argues that large language models have the same vulnerability as phones in the 1970s exploited by John Draper.
"Data and control used the same channel," Schneier writes in Communications of the ACM. "That is, the commands that told the phone switch what to do were sent along the same path as voices."
Other forms of prompt injection involve the LLM receiving malicious instructions in its training data. Another example hides secret commands in Web pages. Any LLM application that processes emails or Web pages is vulnerable. Attackers can embed malicious commands in images and videos, so any system that processes those is vulnerable. Any LLM application that interacts with untrusted users — think of a chatbot embedded in a website — will be vulnerable to attack. It's hard to think of an LLM application that isn't vulnerable in some way.
Individual attacks are easy to prevent once discovered and publicized, but there are an infinite number of them and no way to block them as a class. The real problem here is the same one that plagued the pre-SS7 phone network: the commingling of data and commands. As long as the data — whether it be training data, text prompts, or other input into the LLM — is mixed up with the commands that tell the LLM what to do, the system will be vulnerable. But unlike the phone system, we can't separate an LLM's data from its commands. One of the enormously powerful features of an LLM is that the data affects the code. We want the system to modify its operation when it gets new training data. We want it to change the way it works based on the commands we give it. The fact that LLMs self-modify based on their input data is a feature, not a bug. And it's the very thing that enables prompt injection.
Like the old phone system, defenses are likely to be piecemeal. We're getting better at creating LLMs that are resistant to these attacks. We're building systems that clean up inputs, both by recognizing known prompt-injection attacks and training other LLMs to try to recognize what those attacks look like. (Although now you have to secure that other LLM from prompt-injection attacks.) In some cases, we can use access-control mechanisms and other Internet security systems to limit who can access the LLM and what the LLM can do. This will limit how much we can trust them. Can you ever trust an LLM email assistant if it can be tricked into doing something it shouldn't do? Can you ever trust a generative-AI traffic-detection video system if someone can hold up a carefully worded sign and convince it to not notice a particular license plate — and then forget that it ever saw the sign...?
Someday, some AI researcher will figure out how to separate the data and control paths. Until then, though, we're going to have to think carefully about using LLMs in potentially adversarial situations...like, say, on the Internet.
Schneier urges engineers to balance the risks of generative AI with the powers it brings. "Using them for everything is easier than taking the time to figure out what sort of specialized AI is optimized for the task.
"But generative AI comes with a lot of security baggage — in the form of prompt-injection attacks and other security risks. We need to take a more nuanced view of AI systems, their uses, their own particular risks, and their costs vs. benefits."
Brighton confirm that head coach Roberto de Zerbi will leave the club after their final Premier League match of the season at home to Manchester United on Sunday.
Brighton confirm that head coach Roberto de Zerbi will leave the club after their final Premier League match of the season at home to Manchester United on Sunday.
A blind worker for the National Federation of the Blind said Sonos had a reputation for making products usable for people with disabilities, but that "Overnight they broke that trust," according to the Washington Post.
They're not the only angry customers about the latest update to Sonos's wireless speaker system. The newspaper notes that nonprofit worker Charles Knight is "among the Sonos die-hards who are furious at the new app that crippled their options to stream music, listen to an album all the way through or set a morning alarm clock."
After Sonos updated its app last week, Knight could no longer set or change his wake-up music alarm. Timers to turn off music were also missing. "Something as basic as an alarm is part of the feature set that users have had for 15 years," said Knight, who has spent thousands of dollars on six Sonos speakers for his bedroom, home office and kitchen. "It was just really badly thought out from start to finish." Some people who are blind also complained that the app omitted voice-control features they need.
What's happening to Sonos speaker owners is a cautionary tale. As more of your possessions rely on software — including your car, phone, TV, home thermostat or tractor — the manufacturer can ruin them with one shoddy update...
Sonos now says it's fixing problems and adding back missing features within days or weeks. Sonos CEO Patrick Spence acknowledged the company made some mistakes and said Sonos plans to earn back people's trust. "There are clearly people who are having an experience that is subpar," Spence said. "I would ask them to give us a chance to deliver the actions to address the concerns they've raised." Spence said that for years, customers' top complaint was the Sonos app was clunky and slow to connect to their speakers. Spence said the new app is zippier and easier for Sonos to update. (Some customers disputed that the new app is faster.)
He said some problems like Knight's missing alarms were flaws that Sonos found only once the app was about to roll out. (Sonos updated the alarm feature this week.) Sonos did remove but planned to add back some lesser-used features. Spence said the company should have told people upfront about the planned timeline to return any missing functions.
In a blog post Sonos thanked customers for "valuable feedback," saying they're "working to address them as quickly as possible" and promising to reintroduce features, fix bugs, and address performance issues. ("Adding and editing alarms" is available now, as well as VoiceOver fixes for the home screen on iOS.)
The Washington Post adds that Sonos "said it initially missed some software flaws and will restore more voice-reader functions next week."
An anonymous reader quotes a report from The New York Times: There's a big debate in the tech world over whether artificial intelligence models should be "open source." Elon Musk, who helped found OpenAI in 2015, sued the startup and its chief executive, Sam Altman, on claims that the company had diverged from its mission of openness. The Biden administration is investigating the risks and benefits of open source models. Proponents of open source A.I. models say they're more equitable and safer for society, while detractors say they are more likely to be abused for malicious intent. One big hiccup in the debate? There's no agreed-upon definition of what open source A.I. actually means. And some are accusing A.I. companies of "openwashing" -- using the "open source" term disingenuously to make themselves look good. (Accusations of openwashing have previously been aimed at coding projects that used the open source label too loosely.)
In a blog post on Open Future, a European think tank supporting open sourcing, Alek Tarkowski wrote, "As the rules get written, one challenge is building sufficient guardrails against corporations' attempts at 'openwashing.'" Last month the Linux Foundation, a nonprofit that supports open-source software projects, cautioned that "this 'openwashing' trend threatens to undermine the very premise of openness -- the free sharing of knowledge to enable inspection, replication and collective advancement." Organizations that apply the label to their models may be taking very different approaches to openness. [...]
The main reason is that while open source software allows anyone to replicate or modify it, building an A.I. model requires much more than code. Only a handful of companies can fund the computing power and data curation required. That's why some experts say labeling any A.I. as "open source" is at best misleading and at worst a marketing tool. "Even maximally open A.I. systems do not allow open access to the resources necessary to 'democratize' access to A.I., or enable full scrutiny," said David Gray Widder, a postdoctoral fellow at Cornell Tech who has studied use of the "open source" label by A.I. companies.
Chelsea face Manchester United at Old Trafford in another huge match this afternoon, also kicking off at 3pm BST. Sarah Rendell is keeping tabs on that one in our other WSL live blog.
Leonid Volkov, who was brutally attacked in March, says he shares his late friend’s belief in ‘beautiful Russia of the future’
Leonid Volkov, a close ally of the late Russian opposition leader Alexei Navalny, has vowed to “never give up” fighting against Vladimir Putin despite recently being attacked outside his home.
Myrteza Hilaj and Kreshnik Kadena convicted after NCA operation into Albanian crime group involved in illegal migration
Two men who used a plane to smuggle people from northern France to an aerodrome in Essex have been jailed.
Myrteza Hilaj and Kreshnik Kadena, both from Leyton in east London, were found guilty at Southwark crown court in March of facilitating the commission of a breach of immigration law.
Luis Palma’s 86th minute goal secures 3-2 comeback win
Tagawa strikes as Hearts share 3-3 draw with Rangers
Luis Palma scored a late winner as Celtic twice came from behind to round off another successful Premiership campaign with a 3-2 victory over St Mirren on trophy presentation day at Parkhead.
St Mirren’s captain, Mark O’Hara, twice gave the visitors the lead but goals from Matt O’Riley and Kyogo Furuhashi had the champions level at the break. The second half was a quieter affair but Palma converted Anthony Ralston’s cross from close range in the 86th minute to give the home fans more reason to cheer before the presentation of the silverware.
Source: go.theregister.com – Author: Team Register Kettle It’s been a fairly troubling week in terms of the relationship between China and the Western world. Chiefly, America announced stiff import tariffs on Chinese-made tech, Microsoft gave key engineering and cloud staff the opportunity to get out of China while they still can, and the UK signaled […]
Source: go.theregister.com – Author: Team Register The US Department of Justice has booked two brothers on allegations that they exploited open source software used in the Ethereum blockchain world to bag $25 million (£20 million). The pair – computer scientists Anton, 24, of Boston, and James Pepaire-Bueno, 28, of New York – are accused of […]
Source: go.theregister.com – Author: Team Register Australian prescriptions provider MediSecure is the latest healthcare org to fall victim to a ransomware attack, with crooks apparently stealing patients’ personal and health data. “While we continue to gather more information, early indicators suggest the incident originated from one of our third-party vendors,” the e-script provider said in […]
Source: go.theregister.com – Author: Team Register Three individuals accused of helping North Korea fund its weapons programs using US money are now in handcuffs. All three are said by Uncle Sam’s prosecutors to have used different methods to evade sanctions against the hermit nation and extract money from America’s economy to benefit the Kim Jong-Un […]
Source: go.theregister.com – Author: Team Register Interview On Wednesday the FBI and international cops celebrated yet another cybercrime takedown – of ransomware brokerage site BreachForums – just a week after doxing and imposing sanctions on the LockBit ransomware crew’s kingpin, and two months after compromising the gang’s website. While the BreachForums shutdown didn’t have quite […]
Source: go.theregister.com – Author: Team Register A cybercrime gang has been abusing Microsoft’s Quick Assist application in social engineering attacks that ultimately allow the crew to infect victims with Black Basta ransomware. This, according to Redmond, which said the campaign has been ongoing since mid-April, and blamed a financially motivated group it tracks as Storm-1811 […]
When on holiday in Berwick the artist often gave his work away. Now a new exhibition reveals the value of drawings that survived in a shoebox
A 1958 drawing of a family with their dogs by LS Lowry from one of his many holidays in Berwick-upon-Tweed is to go on public display for the first time. But the sketch is lucky to have survived: it was kept in a shoe box for 43 years, emerging somewhat creased because its recipient had little idea of Lowry’s significance.
The signed and dated drawing on headed notepaper from the Castle Hotel, where the artist stayed for most summers from the 1930s until the 1970s, was given to hotel receptionist, Anne Mather. “I didn’t think much about it, and only after he died did I remember it,” Mather told the Berwick Advertiser in 2001 when she put the sketch up for auction. “He was quiet and reclusive, but I can still visualise him in the lounge. He would sit and doodle, with his glasses at the end of his nose.”
Ex-president’s legal team sure to make hay out of Juan Merchan’s $35 gift to Biden for President and anti-Republican groups
The judge overseeing Donald Trump’s hush-money campaign finance trial in New York has been cautioned by a state ethics panel over two small donations made to Democrat-aligned groups in 2020.
The caution is likely to be seized on by Trump and his lawyers as evidence of his claims that the New York trial, now entering its fourth week, has been unfairly adjudicated by Judge Juan Merchan along partisan political lines.
From haggis to spicy chorizo, offbeat crisp flavours tasted and rated
£1.50 (150g), sainsburys.co.uk Quite a heavy crisp, you feel like you’re going to get your money’s worth. Texturally excellent; strong ridges. It tastes pure paprika-y. Very satisfying. ★★★★
The military alliance turns 75 soon. But there’s little to celebrate in Kyiv, as Putin’s forces continue their bloody advance
Nato’s grand 75th birthday celebration in Washington in July will ring hollow in Kyiv. The alliance has miserably failed its biggest post-cold war test – the battle for Ukraine. Sadly, there’s no denying it: Vladimir Putin is on a roll.
Advancing Russian forces in Kharkiv profit from the west’s culpably slow drip-feed of weaponry to Kyiv and its leaders’ chronic fear of escalation. Ukraine receives just enough support to survive, never to prevail. Now even bare survival is in doubt.
The novelist on the continuing relevance of Ibsen, the joyful quilt art of Faith Ringgold and where to find British scotch eggs in New York
Born in Greenwich, Connecticut in 1966, author Claire Messud studied at Yale University and the University of Cambridge. Her first novel, 1995’s When the World Was Steady, and her book of novellas, The Hunters, were finalists for the PEN/Faulkner award; her 2006 novel The Emperor’s Children was longlisted for the Booker prize. Messud is a senior lecturer on fiction at Harvard University and has been awarded Guggenheim and Radcliffe fellowships. She lives in Cambridge, Massachusetts with her husband, literary critic James Wood; they have two children. Her latest novel, This Strange Eventful History, is published on 23 May by Fleet.
Second operation to remove dead tissue has ‘contributed to a positive prognosis’ for Robert Fico, health minister says
Slovakia’s prime minister, Robert Fico, remained in a stable but serious condition as the man accused of trying to assassinate him made his first court appearance.
The Slovakian health minister, Zuzana Dolinková, said on Saturday that a two-hour surgery to remove dead tissue from multiple gunshot wounds had “contributed to a positive prognosis” for Fico.
Nelson Shardey, 74, became tearful on hearing of support for effort to gain settled status after 50 years in UK
A retired 74-year-old newsagent who has lived in the UK for nearly 50 years said “tears were running” from his eyes after strangers fundraised more than £30,000 to support his legal fight to remain in the country.
Nelson Shardey, who has been described as a Merseyside “local legend”, is pursuing a legal challenge against the Home Office after he was refused indefinite leave to remain, despite living and working in the UK since 1977.
Low-price deals in UK mean consumers are eating less-nourishing food more frequently, say experts
Office workers looking for a cheap lunch on the high street might struggle. With inflation pushing up prices in recent years, a sandwich, snack and drink at popular coffee chains can now cost upwards of £10, while even the average supermarket meal deal has risen by more than 21% in price since before the pandemic.
But now fast-food chains have moved to fill the gap in the market. In March, KFC introduced a new lunch deal for £5.49, offering a fried chicken wrap with a drink and side – either crisps or a cookie – and available from Monday to Friday until 3pm. “KFC is now workplace appropriate, for when finger lickin’ is not,” the chain said in its promotional material.
Patrick Grant says rise of low-cost retailers means new clothes ‘haven’t got cheaper, they’ve just got worse’
While filming The Great British Sewing Bee, the presenter and clothing entrepreneur Patrick Grant found himself in need of a pair of black socks.
The production team bought a pair from the Marks & Spencer shop close to where the popular BBC show was being filmed. Grant said: “They went to everybody’s favourite high street store, that used to sell on the basis of quality and value, and they bought me their Autograph socks, which are supposed to be their best socks.
Victoria Atkins says she has asked officials to look into claims doctors and nurses who have spoken up were mistreated
The NHS must listen to whistleblowers and investigate their concerns in the interests of patient safety, the health secretary has said.
Victoria Atkins said she had asked officials to look into cases where there were claims of mistreatment of people who had spoken up about the issues they had experienced.
Source: securityaffairs.com – Author: Pierluigi Paganini Turla APT used two new backdoors to infiltrate a European ministry of foreign affairs Russia-linked Turla APT allegedly used two new backdoors, named Lunar malware and LunarMail, to target European government agencies. ESET researchers discovered two previously unknown backdoors named LunarWeb and LunarMail that were exploited to breach European […]
Source: securityaffairs.com – Author: Pierluigi Paganini City of Wichita disclosed a data breach after the recent ransomware attack The City of Wichita disclosed a data breach after the ransomware attack that hit the Kansas’s city earlier this month. On May 5th, 2024, the City of Wichita, Kansas, was the victim of a ransomware attack and […]
The dog-killing South Dakota governor’s VP hopes are in tatters. But she’s not the first politician to flame out with an own goal
She could have been a contender. But then she wrote a book. And suddenly Kristi Noem was caught like a rabbit – or a rambunctious puppy – in the headlights.
The governor of South Dakota found herself insisting that a false claim she met the North Korean dictator Kim Jong-un had been put in her book by accident. Wait, said Elizabeth Vargas of NewsNation, you recorded the whole audiobook version and read this passage out loud. Why didn’t you take it out then?
Last year’s Barbenheimer was hailed as saving cinema. Now takings are down and even franchises are falling flat. Can Hollywood manoeuvre itself out of this disaster zone?
In Hollywood, the first weekend of May is traditionally seen as the official kick-off of the summer movie season: an auspicious blockbuster date that has, of late, become rather a boring one.
Since 2007, when Spider-Man 3 (three full cycles ago in that deathless franchise) topped the box office – and barring two years where the global pandemic threw the mainstream release schedule into disarray – that weekend has been the exclusive domain of Marvel superhero adaptations, through to Guardians of the Galaxy Vol. 3 claiming the No 1 spot last May. That stranglehold was set to continue this year, with the legacy-milking superhero mash-up comedy Deadpool & Wolverine scheduled for a 3 May release. It doubtless would have creamed the competition, too, had last year’s Hollywood strikes not delayed it to July.
We’re going backwards: a football player can say hateful things from a university podium while students are being arrested
Imagine for a moment that that you are a young woman who has spent more than $100,000 on your university degree. After four years of hard work it’s your graduation and Harrison Butker, a kicker with the Kansas City Chiefs, is the commencement speaker. During his speech the NFL star, who has made millions by kicking a ball, kindly informs you that your hard-earned degree was a waste of time and that your true role in life is supporting your husband. Imagine what that would be like.
“IVF, surrogacy, euthanasia, as well as a growing support for degenerate cultural values in media, all stem from the pervasiveness of disorder.” (This doesn’t make any sense, I know, but the entire speech is incoherent.)
Joe Biden has been “vocal in his support for the murder of innocent babies”. (Which is certainly true if we’re talking about Gaza – but Butker was talking about abortion rights.)
Pride month is a “deadly sin”.
You can’t spread the antisemitic talking point that Jews are responsible for the death of Jesus any more or you’ll end up “in jail”.
Women have been subject to “the most diabolical lies” and, while some women in the audience may go on to have successful careers the majority should be “most excited about your marriage and the children you will bring into this world”.
His wife would be “the first to say that her life truly started when she began living her vocation as a wife and as a mother”. (This sparked long applause from the audience.)
We should fight against “the cultural emasculation of men”.
The fashion designer and Biba founder, 87, on the best business advice she’s been given, why she wears black and the reason she never cries
As a child living in Jerusalem, we used to visit churches and convents. I always wanted to redesign the nuns’ outfits – they just weren’t dressed correctly. Jesus wasn’t a very good designer.
I have no temper. I like people too much. I really love them. I think it’s from living in a big family.
I had a mum who was so beautiful it was embarrassing. I used to stand outside the front door and give away all her clothes from Paris to anyone who needed them, which was not very popular.
There’s nothing I’m scared of. When you’re younger you’re scared of dying. As you get older, it seems rather fun. I can’t wait – there must be a big party going on on the other side.
Finding the person you are going to spend the rest of your life with is so instant, it’s so obvious, it’s absolutely amazing. It doesn’t often come. You’re very lucky if you get one such person in your lifetime and I had one. It was so wonderful.
“Slowly, slowly, catchy monkey!” Our first accountant at Biba gave me that advice and it’s the best advice I’ve ever been given. We should have put that on a T-shirt.
When I was very, very small I used to follow my father around like a little dog. He was the biggest person in my life and I get very angry that I didn’t have longer with him.
The thing to do with grief is to bottle it and it just keeps. You don’t want to be a bore to people. They’ve got their own problems.
New York actor was taken to hospital after a stranger punched him in the face while he was walking in Manhattan on 8 May
A man wanted in connection with the random attack on actor Steve Buscemi on a New York City street earlier this month was arrested on an assault charge on Friday, police said.
The 66-year-old star of Boardwalk Empire and Fargo was walking in midtown Manhattan on 8 May when a stranger punched him in the face, city police said. He was taken to a hospital with bruising, swelling and bleeding to his left eye, but was otherwise OK, his publicist said at the time.
South West Water identifies ‘damaged valve’ as possible cause of cryptosporidium contamination in Brixham area
Health officials are expecting more cases of a waterborne disease in Devon, as an MP said “heads are going to roll” over the outbreak and that the anger among residents was “palpable”.
The UK Health Security Agency has confirmed 46 cases of cryptosporidium infection in the Brixham area, while more than 100 other people have reported symptoms, including diarrhoea, stomach pains and dehydration.
Login
CISO2CISO.COM & CYBER SECURITY GROUP
The Guardian
