AI failures in healthcare

Oh my word:

On Saturday, an Associated Press investigation revealed that OpenAI's Whisper transcription tool creates fabricated text in medical and business settings despite warnings against such use. The AP interviewed more than 12 software engineers, developers, and researchers who found the model regularly invents text that speakers never said, a phenomenon often called a "confabulation" or "hallucination" in the AI field.

Upon its release in 2022, OpenAI claimed that Whisper approached "human level robustness" in audio transcription accuracy. However, a University of Michigan researcher told the AP that Whisper created false text in 80 percent of public meeting transcripts examined. Another developer, unnamed in the AP report, claimed to have found invented content in almost all of his 26,000 test transcriptions.

Of course, they use it because it's cheaper than paying a human transcriber.  So riddle me this, Healthcare Administrator: what do you call yet another AI that lies all the time?  A day that ends in "-day".

And people have started noticing:

While the vast majority of people over 50 look for health information on the internet, a new poll shows 74% would have very little or no trust in such information if it were generated by artificial intelligence.

Meanwhile, 20% of older adults have little or no confidence that they could spot misinformation about a health topic if they came across it.

That percentage was even higher among older adults who say their mental health, physical health or memory is fair or poor, and among those who report having a disability that limits their activities. In other words, those who might need trustworthy health information the most were more likely to say they had little or no confidence they could spot false information.

People are smart enough to catch a whiff of marketing Bravo Sierra.

From now on I will start asking all of my healthcare providers if they do transcription, and if so whether they use AI for the transcription.  If they do I will demand to review the transcript.  If they won't, I'll get a different provider.

Meta fined for storing user passwords with no encryption

Holy cow, I've been in this industry for decades and can't remember a time when everyone knew that you encrypted the damn passwords*:

Officials in Ireland have fined Meta $101 million for storing hundreds of millions of user passwords in plaintext and making them broadly available to company employees.

Meta disclosed the lapse in early 2019. The company said that apps for connecting to various Meta-owned social networks had logged user passwords in plaintext and stored them in a database that had been searched by roughly 2,000 company engineers, who collectively queried the stash more than 9 million times.

This is such a rookie mistake that it makes you wonder what those 9 million queries were looking for.  Meta has such a horrible reputation for abusing its users privacy that the suspicion is that this was just one more wring on that rag.  That's only a suspicion, but Meta has certainly earned that suspicion over the years.

* Yeah, yeah I know - one-way hash.  I try not to use too much tech jargon.

KIA cars can be hacked with a smartphone

I hope you don't drive a KIA.  This is actually a failure of post manufacturing security processes, not that it makes things any better:

Sam Curry, who previously demonstrated remote takeover vulnerabilities in a range of brands – from Toyota to Rolls Royce – found this vulnerability in vehicles as old as model year 2014. The mess means the cars can be geolocated, turned on or off, locked or unlocked, have their horns honked and lights activated, and even have their cameras accessed – all remotely.

...

The issue originated in one of the Kia web portals used by dealerships. Long story short and a hefty bit of API abuse later, Curry and his band of far-more-capable Kia Boyz managed to register a fake dealer account to get a valid access token, which they were then able to use to call any backend dealer API command they wanted.

"From the victim's side, there was no notification that their vehicle had been accessed nor their access permissions modified," Curry noted in his writeup. "An attacker could resolve someone's license plate, enter their VIN through the API, then track them passively and send active commands like unlock, start, or honk."

Security wags have long called this sort of architecture "broken by design" - it was intentionally set up to allow privileged access via a poorly authenticated system that has to scale through a big organization.  I don't have much confidence that KIA can fix this, or that they will likely want to.

And oh yeah - there's a smartphone app to help the Bad Guys.

All I can say is that 1968 Goat isn't vulnerable to this attack, and will never be.

 

US bans Chinese "Connected Car" tech

They say it's a security concern.  They're right:

Now, the US Commerce Department is set to enact a de facto ban on most Chinese vehicles, by prohibiting Chinese connected car software and hardware from operating on US roads, according to Reuters.

The rationale? National security concerns. "When foreign adversaries build software to make a vehicle [connected], that means it can be used for surveillance, can be remotely controlled, which threatens the privacy and safety of Americans on the road," said Commerce Secretary Gina Raimondo.

"In an extreme situation, a foreign adversary could shut down or take control of all their vehicles operating in the United States all at the same time, causing crashes, blocking roads," said Secretary Raimondo, a scenario we saw depicted in Fate of the Furious (where it caused me a headache), as well as more recently (and to better effect) in Leave the World Behind.

Yup.

Now I expect there's a whole lot more behind this and the security risks are just nice window dressing, but it's pretty hard to argue with this.

Well, that's one way to improve the Internet coverage on a Navy ship

Navy finds hidden Starlink dish on ship:

Still, the ambassador had nothing on senior enlisted crew members of the littoral combat ship USS Manchester, who didn't like the Navy's restriction of onboard Internet access. In 2023, they decided that the best way to deal with the problem was to secretly bolt a Starlink terminal to the "O-5 level weatherdeck" of a US warship.

They called the resulting Wi-Fi network "STINKY"—and when officers on the ship heard rumors and began asking questions, the leader of the scheme brazenly lied about it. Then, when exposed, she went so far as to make up fake Starlink usage reports suggesting that the system had only been accessed while in port, where cybersecurity and espionage concerns were lower.

Well, it is a pain in the rear end to get hooked up to SIPRnet ... 

Of course, there's been a general helping of Courts Martials to everyone involved.

And the funniest bit?  Elon Musk had Starlink change the default WiFi SSID to "Stinky" to encourage customers to change the damn defaults.

So it's Price Controls now, eh?

So Kamala doesn't know much about history, it seems.  Or economics.

Disney+ Terms of Service does not give blanket immunity

Sanity breaks out at Disney:

Disney said it is abandoning its motion to compel arbitration in a case filed by a man who alleges his wife died from anaphylaxis after a restaurant at a Disney complex failed to honor requests for allergen-free food.

Disney's motion to compel arbitration controversially cited the Disney+ streaming service's subscriber agreement, which includes a binding arbitration clause. The plaintiff's lawyer called the argument "absurd."

Disney confirmed this week that it will withdraw the motion, which it filed on May 31.

Good.  It was a stupid argument anyway.  Man, they generated a lot of ill will with that bone-headed move, though.

 

 

The buzz from Black Hat this year

Every year in the heat of the Las Vegas desert is the Black Hat Briefings, the premier computer security conference.  There's always interesting news from the briefings (and from the much less buttoned down conference, DEFCON, which runs immediately afterwards).

So what's the buzz from Black Hat this year?  It seems that Palo Alto Networks had Booth Bunnies at their display booth:

[blink] [blink]

Now I did my share of manning the booths (yes, I was a Booth Bunny, thank you for asking) back in the '90s and the '00s.  But even in the '90s we were considerably more buttoned down than this, and for good marketing reasons.  Sure, some of the attendees might like the scenery, but some will not - and some of them will very much not like the scenery.  This has been known to be bad conference marketing juju for literally decades.

Of course, the Palo Alto Networks' Chief Marketing Officer had to go full frontal groveling* in his apology:

PAN's chief marketing officer Unnikrishnan KP, or Unni as he's often called, issued his apology earlier this week calling it "tone deaf."

"Last week at Black Hat in Las Vegas, an unfortunate decision was made at a Palo Alto Networks event to have hostesses wear branded lampshades on their heads," he said. "It was tone-deaf, in poor taste, and not aligned with our company values or brand campaign. 

"I take full responsibility for this misjudgment and have addressed it with my team and am taking steps to prevent such misguided actions in the future.

"Please accept my heartfelt apologies for this regrettable incident."

Nikesh Arora, PAN's chairman and CEO, doubled down on the apologies on Tuesday, echoing the points made by Unni, adding that what happened was "unacceptable."

I expect the headcount at Palo Alto Networks' marketing department has gotten a spin.  We apologize again for the fault in the subtitles. Those responsible for sacking the people who have just been sacked have been sacked.

* See what I did there?  I crack myself up.

Crowdstrike threatens Delta Airlines

Wow:

CrowdStrike says it is "highly disappointed" and rejects the claims made by Delta and its lawyers that the vendor exhibited gross negligence in the events that led to the global IT outage a little over two weeks ago.

That's according to a letter, seen by The Reg and sent to David Boies, partner at the law firm Delta hired to investigate the airline's legal options after it struggled more than most to bring its systems back online, leading to a sprawling list of flight cancellations.

The Falcon vendor reiterated its apology to Delta and the wider customer base. It then went on to remind Boies, known for his work as special counsel during the 1990s US antitrust trial against Microsoft, that it had been proactive in reaching out to Delta, offering support to the airline "within hours" of the incident unfolding.

...

CrowdStrike's lawyer, Michael B. Carlinsky, then poked the bear further. He said that among other things, in this hypothetical trial Delta would also need to explain why it took so much longer than competitors to recover from the same issue, why it refused the free on-site help CrowdStrike offered – the support that led to faster recovery times than Delta's, and the operational resiliency of its IT infrastructure.

This is hands down the biggest screw up - ever - by any security vendor.  I guess that a screw up this big is a potential extinction-level event for Crowdstrike but this sure doesn't sound like it will calm down their customer base.  OK, so they offered some help when they took down Delta, and Delta didn't jump on this.  That sounds like it's 1% on Delta and 99% on Crowdstrike.

But that's not what's going on here - it's explicitly telling a customer that they will drag them through the mud if the customer sues them for their monumental screw up.

Holy moley.

Just how bad is the illegal immigration problem?

Libertarians have ditched their support for "free markets and free people":

I would prefer not to lose my Libertarian purity certificate. I want a political philosophy that is simple and has universal application. That way I don’t have to think too hard. For the last 30 years libertarianism has been that philosophy. Name an issue of the day and I can give you the answer. Sluggish growth? Privatise, lower taxes and de-regulate. Busy roads? Privatise. Inflation? Abolish the Bank of England or re-introduce the Gold Standard, or, er… privatise the Bank of England. OK, some issues are not quite that easy but usually they are. Until we get to immigration. Because if libertarianism means open borders then libertarianism is wrong because open borders are a disaster.

Read the whole thing, and the comments.  And remember that this is Libertarian Central.  If the Open Borders crowd has lost Samizdata, they've lost everybody.

Adobe updates License terms to be less douchy

The key word here is "less":

Adobe has promised to update its terms of service to make it "abundantly clear" that the company will "never" train generative AI on creators' content after days of customer backlash, with some saying they would cancel Adobe subscriptions over its vague terms.

Users got upset last week when an Adobe pop-up informed them of updates to terms of use that seemed to give Adobe broad permissions to access user content, take ownership of that content, or train AI on that content. The pop-up forced users to agree to these terms to access Adobe apps, disrupting access to creatives' projects unless they immediately accepted them.

...

On X (formerly Twitter), YouTuber Sasha Yanshin wrote that he canceled his Adobe license "after many years as a customer," arguing that "no creator in their right mind can accept" Adobe's terms that seemed to seize a "worldwide royalty-free license to reproduce, display, distribute" or "do whatever they want with any content" produced using their software.

...

Adobe's design leader Scott Belsky replied, telling Yanshin that Adobe had clarified the update in a blog post and noting that Adobe's terms for licensing content are typical for every cloud content company. But he acknowledged that those terms were written about 11 years ago and that the language could be plainer, writing that "modern terms of service in the current climate of customer concerns should evolve to address modern day concerns directly."

...

"You forced people to sign new Terms," Yanshin told Belsky on X. "Legally, they are the only thing that matters."

The original story is here.

I'm not sure this brouhaha is over.

It's time to opt out of Windows Recall

Holy cow, what a nightmare:

Microsoft is not giving up on its controversial Windows Recall, though says it will give customers an option to opt in instead of having it on by default, and will beef up the security of any data the software stores.

Recall, for those who missed the dumpster fire, was announced on May 20 as a "feature" on forthcoming Copilot+ Windows PCs. It takes a snapshot of whatever is on the user's screen every few seconds. These images are stored on-device and analyzed locally by an AI model, using OCR to extract text from the screen, to make past work searchable and more accessible.

The ultimate goal for Recall is to record nearly everything the user does on their Windows PC, including conversations and app usage, as well as screenshots, and present that archive in a way that allows the user to remind themselves what they were doing at some point in the past and pull up relevant files and web pages to interact with again. The archive can be searched using text, or the user can drag a control along a timeline bar to recall activities.

But security testers have raised doubts about the safety of recorded information and have developed tools that can extract these snapshots and whatever sensitive information they contain. The data is for now stored as an easy to access non-encrypted SQLite database in the local file system.

"Dumpster fire" doesn't even begin to describe it.  It's easy to imagine all sorts of ways that this would violate laws (e.g. storing healthcare PII unencrypted is a HIPAA violation).

Never mind what sort of reindeer games hackers might get up to - after all, Windows has historically been so difficult for viruses and malware to invade, amirite? 

If you're still using Windows, you should configure it to opt out of Recall.  Or upgrade to Linux.  All the cool kids are.

GE Medical Ultrasound imager critical security vulnerabilites

"Vulnerabilities" meaning plural: remote code execution, ransomware danger, other cool stuff.  

The good news: you need physical access to the device (supposedly; of course these would *never* be put on the network ...).  The bad news: it's unlikely in the extreme that these devices will ever get patched.

If only someone had been warning them of this problem ...

More info emerges on the UnitedHealth cyber incident

None of it is good for UnitedHealth.  Multiple rookie security failures - including no use of multi-factor authentication for remote login, no network segmentation, and no internal security threat hunting. 

I don't know if there will be lawsuits over this, but this is all basically indefensible.  After all, they are a healthcare provider, and HIPAA/HITECH mandates all of this.

Kaiser Permanente shares user data with Google, Microsoft, and others

Well, well, well:

Millions of Kaiser Permanente patients' data was likely handed over to Google, Microsoft Bing, X/Twitter, and other third-parties, according to the American healthcare giant.

Kaiser told The Register it has started notifying 13.4 million current and former members and patients that "certain online technologies, previously installed on its websites and mobile applications, may have transmitted personal information to third-party vendors," when customers used its websites and mobile applications.

Kaiser has since removed that tech from its websites and apps, and said it is not aware of "any misuse of any member's or patient's personal information."

Yeah, I'll bet.

If you get Kaiser Permanente insurance at work, you might want to ask your HR department for an assessment of whether your data was included in this data sharing scheme.  It's hard to see how at the minimum HIPAA-adjacent data was not shared here.

 

Ring doorbell company fined millions of dollars for privacy violations

Well knock me over with a feather:

The FTC today announced it would be sending refunds totaling $5.6 million to Ring customers, paid from the Amazon subsidiary's coffers.

The windfall stems from allegations made by the US watchdog that folks could have been, and were, spied upon by cybercriminals and rogue Ring workers via their Ring home security cameras.

The regulator last year accused Ring of sloppy privacy protections that allowed the aforementioned spying to occur or potentially occur.

...
 

In the most egregious case, one employee went out of his way to view "thousands of video recordings belonging to at least 81 unique female users," according to the FTC. A coworker reported this behavior to her supervisor, who it's alleged initially said this snooping wasn't that strange until he realized the rogue employee was only reviewing videos of "pretty girls."

The fines work out to $50 per effected Ring customer.  Don't spend it all in one place.

The dangerous side of IT security

Security researcher fined for revealing insecure system:

After discovering and reporting a vulnerability in an e-commerce database that was putting customers and their personal information at risk, a security researcher in Germany was fined €3,000 for doing so.

In 2021, a contractor, known as Hendrik H., said he was troubleshooting software for Modern Solution GmbH when he realized that password access to the remote server was stored in plain text in MSConnext.exe. This easy access would make the password simple for many to find, and a threat actor could access data to everything stored on the database server, including customer information.

There is a lot of back and forth on this between the company and the researcher, with court appeals (and more planned).  But this seems odd to me.  If the researcher was working for the company (as stated) then why did he not have a "get out of jail free" card from company management for what he was doing?  This is basically a letter (typically from the Chief Information Security Officer) saying the researcher is authorized to poke around and that the company will hold him harmless.  It also will have non-disclosure and other restrictions so that the researcher won't up and publish embarrassing info.

 It doesn't seem that any of this was in place, so I'm wondering what sort of "research" this guy was up to.

When the White Hats are actually Black Hats

Not cool, dude:

An Atlanta tech company's former COO has pleaded guilty to a 2018 incident in which he deliberately launched online attacks on two hospitals, later citing the incidents in sales pitches.

Under a plea deal he signed last week, Vikas Singla, a former business leader at network security vendor Securolytics – a provider to healthcare institutions, among others – admitted that in September 2018 he rendered the Ascom phone system of Gwinnett Medical Center inoperable.

...

After all of the events had transpired, Securolytics began emailing potential clients regarding new business opportunities, citing the publicized attacks.

 Quis custodiet ipsoes custodes, indeed.

 

The terms "Software Engineering" and "Military Intelligence" are strangely related

It is said that Engineering is "Science that works", so we have to relegate "Software Engineering" to the same bucket as "Military Intelligence" and "Jumbo Shrimp".  Exhibit A for the prosecution is this month's Microsoft Patch Tuesday, which fixes a data leakage vulnerability caused by a divide by zero condition:

CVE-2023-20588 is a “division-by-zero” vulnerability affecting specific AMD processors that can “potentially return speculative data resulting in loss of confidentiality.”

Microsoft addressed the vulnerability in its Patch Tuesday update round, as the latest Windows versions enable mitigation and protection.

[blink] [blink]

Oooooh kaaaaay.  Maybe I'm old fashioned but aren't folks taught that divide by zero is no bueno?  Like taught that in Coding 101?

All I can think is, well, bless their little hearts.  Wow.

Vandals cut down most famous tree in UK

This tree:

Voted Tree of the Year in 2016 by the conservation charity Woodland Trust, the Sycamore Gap Tree was one of the most photographed trees in the United Kingdom. It’s also known as the “Robin Hood Tree” because it appeared in the 1991 film Robin Hood: Prince of Thieves, despite Hadrian’s Wall being some 130 miles north of Sherwood Forest.

According to the National Trust, this iconic sycamore tree was planted in the late 1800s by John Clayton, the saviour of Hadrian’s Wall, to be a feature in the landscape. 

So why would someone cut it down?  Tik-tok views:

Why would anyone do this? A question we are all confused over. Police officers are looking into claims that the tree was felled to be posted online and carried out as part of a TikTok stunt

Dumbass Tik-tockers.  Hang em.  From a tree.  As a warning to others.