I know mistakes may happen. I wish you will be able to fix that, and any other security issues fast and definitely.
Yet, as someone, who was IT consultant, Senior Architect at Siemens, designing systems for Pharma, banks and European Commission, I have to ask: who stores customer’s passwords in plain text, and even more - sends them with DEBUG information to an email mailbox?
You are intelligent, and well educated people. Please, stop storing sensitive data in plain text (use checksums instead, simplest solution), and please, stop sending customers’ data along with device debug information. Definitely you do not need that to debug software.
I wish you, at NDSP, and us (your customers) all the best.
(I am kindly asking also everyone, to not reply to this post. I wrote it as I needed to drink and react, but my intention is not to start a shit hurricane here)
The OP @4TuneMan did not expose this bug. The bug/vulnerability was shared by Dan from Neural DSP in an announcement today on the forum as well as an email sent out to Neural users. It is also public knowledge out on other forums.
Ahh I though he had found it and decided this was the best way to report it.
I haven’t had an email and haven’t seen a post about it.
Edit: found the post, read it.
Sure, it’s a big mistake, but they found it and are fixing it straight away. Sounds like it was something that was overlooked very early on and never changed afterwards.
I appreciate their honesty and the way they have reacted.
I really think it’s the second part in this case, sending that information in an email.
Why did you email plaintext wifi passwords with crash/debug logs? First, I can’t imagine having a wifi password in debug logs provide any benefit to support staff to debug anything. Second, using email to store logs are a bad idea. You potentially have PII in there, it’s hard to provide ACL or audit controls against who on your support team or malicious actors access that email box and email is generally publicly accessible. Just setup an debug log API, with no methods to access that data and don’t send wifi passwords.
The storing of the plaintext password seems like a red herring. Here the password is a wifi password. The QC needs to be able to retrieve the plaintext password to send it to a router. An encrypted wifi password isn’t usable by a router. Sure you could encrypt it locally and decrypt it when you need to pass it to a router, but in this case the QC itself sent the password in debug logs. So even if it were encrypted, that wouldn’t have prevented the password leak. Encrypting the wifi password would prevent someone who had physical access to the QC from maybe pulling out the SD card, and assuming if they were able to read the data on the card and access the database on it, then it would prevent them from getting the plaintext password. That’s a pretty extreme attacker to get a single wifi password from a single device.
I 100% agree. What possible purpose is served by sending a local Wi-Fi password to Neural? Even sending the Neural account password is of dubious utility when as you say a debug API or other methods are available that use much less risky and invasive methods for troubleshooting password mismatches as the cause of connectivity problems.
Also correct that employees working in support should as a rule never be privy to passwords of any sort. With the possible exception of when they handout “change on first login” temporary passwords. And even that approach has its vulnerabilities and is best avoided. All that aside though, everyone can probably agree they should NEVER EVER be sent via email in plain text where they can potentially be “sniffed” or just straight up viewed by the bad actors you referred to. They are also infinitely more, if perhaps not totally, secure when stored encrypted on corporate servers. Good points!
I don’t know if it’s true, but someone mentioned in the NDSP Discord that the Wifi password in the debug logs was part of a memory dump included in that report. So if true, the password was just in memory at the time of the dump, presumably to connect to your router. That would explain why it got sent. They didn’t think to sanitize that password from the report before sending it.
Understood, thanks for the info. So there is a good chance that the passwords were sent completely inadvertently and were never intended to be part of the troubleshooting process.
As you said, the QC needs to be able to send unencrypted passwords to connect to the router, so it will need a way of sending it in plaintext at some point.
The showing of it in logs and the subsequent sending if those logs via email will all have been an accidental oversight.
I’ve worked on projects before where we had to go through and deliberately hide passwords in logs because the default is just to show everything. So I assume this was something similar.
I’d suggest that WiFi is the red herring, really. Email addresses of all users were available, paired with WiFi passwords. The hack would be that some (very few) people would use the same password for WiFi as they do for all their accounts, at which point the breach is a lot more dangerous.
Mistakes happen. I think it’s equally fair to be tolerant of the error or to flip out over it. If I were using the same password for WiFi and email, I’d be very annoyed, even though my choice to do so would have been poor. My poor security would be my fault, but the breach wouldn’t.
I’m feeling tolerant of this one only because it wasn’t an executed hack and my email address / password isn’t currently in Russia / China (probably), rather it was a reported vulnerability. Luck.
Discussion on the NDSP Discord suggests that this security hole was reported to Neural three months ago. What’s the truth?
From your job history it didn’t sound as though this was your first day on the internet. Welcome!
I saw this claim as well and would have to see some more proof before accepting this as being true. Until then I am viewing this as nothing more than a rumor. If it is true, then obviously Neural needs to be much more proactive than they appear to have been at the moment.
It’s sad but true (no pun intented) I’ve been in contact with NDSP be looking into solutions for handling and rewarding stuff like this in the future. They haven’t been honest about that part which I find a little disappointing since it’s the customers right to know this stuff imho.
And yes, I got an answer by a dev confirming they where going to look into this about a week later. But for that, someone else had to demand through Instagram DM’s to the NDSP page to look into the mail, before that I didn’t get any response on it.
According to NDSP this was going to be fixed in the 2.1.0 update but my gut says no. And even if that were true, that would’ve been problematic since that update is probably months out. So that would mean almost half a year needed to patch something very simple.
Only thing I can say is that this was handled very poorly, and I really hope this will change for the better.
Hard to sort through exactly how long it took Neural respond when posts like this pop up. Precisely why many companies implement corporate security policies to address this sort of thing. That way you don’t have to go hunting for a fall guy when there is a problem, and the company is not making knee-jerk responses on an ad-hoc basis.
Formally codifying an improved security policy to help avoid this sort of thing in the future might be extremely timely and helpful. Nobody can guarantee impenetrable security, but measures can be taken to reduce risk and improve response time when a problem is discovered, as well as testing regularly for vulnerabilities - white-hat hacking.
They’re not the only company that has or will screw up. They just need to be upfront about it and show us that they will take security more seriously in the future. I am waiting for an official statement regarding the handling of this incident. If they don’t acknowledge this serious mistake, then that will certainly affect my trust in them.
They need actually to explain why they didn’t do anything about it months ago. This whole thing has me looking at other products in case I decide that Neural isn’t trustworthy. It’s not the breach, it’s the choice to not fix the vulnerability immediately.
@ThomasVanIseghem , do you have a link to the mentioned discussion ? (I searched NDSP Discord server and found nothing related to this posts… )
thanks.
)