I use a Dexcom Continuous Blood Glucose Monitor that pairs with this pump (I don't use the pump). And honestly, this is not surprising. The software quality in this space seems really low. I don't understand why some single-developer Mastodon client app can absolutely reliably get notifications to my watch but Dexcom can't. Even on the phone, the app cannot get alerts right at all, sometimes stacking them over an hour or more, but getting the current value for the alert, so you can get like five in a row (ALERT YOUR BLOOD SUGAR IS IN RANGE).
I don't understand why some single-developer Mastodon client app can absolutely reliably get notifications to my watch but Dexcom can't.
The same reason why software developed by individuals or small teams can be very high quality, while "enterprise" software is universally horrible; the former work in more self-structured and effective ways, but the latter usually involves tons of bureaucracy. This tends to introduction a selection bias to the types of developers who work in those environments.
I’ve never met a dev who is pleased to be working on a shitty enterprise app.
In my experience working in enterprise software, we as the devs are far removed from any customer problems.
Those problems, when reported, go through some or all the layers of support engineers, product owners, and engineering managers and eventually get put on some every growing list of stuff to do.
The devs, being so far removed from customers, choose to focus on what they can see, which is usually just whatever feature was prioritized because big customer A was promised it would be ready by some date.
A single dev has no such barriers, as you were saying.
I was here to say exactly the same thing but about another insulin pump system.
A few months ago, an update to Roche's DiabetLoop (DBLG1) created a thousand problems with emails recommending temporary tricks, subsequent updates, etc.
It’s unclear how such a serious issue didn’t surface during QA testing for a system that is, ultimately, fully under the control of the manufacturer (hardware/software).
I see a lot of effort to ensure the ecosystem is increasingly closed, touting the need for security of devices that can be fatal, but, in reality, there's little attention given. (And I say this because bugs and issues that everyone reports are always present, they're known, and they’re not being addressed.)
I’d argue that the regulation for ‘alternative control enabled’ (ACE) pumps [1] is too vague. These devices should be ‘designed to reliably and securely communicate with external devices’. These incidents were caused by the app crashing, resulting in the connection re-establishing itself frequently enough to drain the batteries [2]. How reliable should it be? Shouldn’t it include some stability measures?
Manufacturers should have to submit their devices and software for certification of the technology before itself. Bluetooth and coding standards are mature enough at this point, and it’s time customers start demanding this from their products.
Reliability to the degree that medical devices need to be when you're sharecropping on someone else's shifting-sand platform is a game I'm glad I'm not paid to play. It in no way excuses the bad behavior, they chose to go this route and committed to the work it would take, but the results of the such certification would be outdated by the time you finish it.
I think a "sacrificial device" is the right way to make reliable the unreliable parts. Big battery for running the pump, small battery for the smart device connection. If something happens and you drain the auxiliary device it's annoying but not life critical. Your phone and pump can both alert you indicator light/notification that the auxiliary device died.
It's because one app is made by an engineering enthusiast. The other app is made by a company whose primary field isn't software, the executives are most likely wholely inexperienced with software development and/or sometimes nepo hires who then proceed to hire yesman middle managers or even outsource app development.
What faang allows remote work? What faang has average devs making 400k?
(Also it remains weird to me that Netflix is considered a tech company)
The reality is companies like this often use minimum cost contract companies to develop the software rather than actually employing developers directly. They seeing software engineers (or any non managerial position) as purely an expense not the source of value in their products so you only get penny pinching on development and QA.
People working in their spare time make better apps than these. I don't think you need great developers, just ones that follow specs and use native development tools would probably do.
In any case, I was responding to the notion that the FDA was quality-controlling these things. They aren't doing a very good job.
I use a Freestyle Libre and think the software is junk too. Lot of proprietary lock-in and trying to push you towards certain vendors in this space.
Should all be about people and not profit. These aren't difficult problems to solve. Nightscout is great, as is xDrip+, but pairing your device with xDrip+ gets you locked out of the vendor app and invalidates your warranty, which is a big deal as sensor quality is hit-and-miss and may come out of your arm before the 15 day window is up, so you won't get a free replacement if that happens.
I would love to figure out DIYing CGMs, but the enzyme-dipped probe is the challenging part.
Also, I don't think smartphones should play an active role in this space. As a means of reading data and perhaps modifying routines on a dedicated device? Maybe. But these things should all function independently of phones.
My phone does an update or runs out of juice? I stop getting CGM data until I unlock with PIN and use the NFC to reconnect. Happened to me last night in my sleep, so missed going hypo in my sleep.
Complete junk in this space, IMO. It's very annoying.
I recently did a writeup [1] on the 510(k) FDA clearance process (which cleared this device for market).
Basically this device was cleared through a DAG of other devices that were "substantially equivalent". I made a website to visualize these relationships, and any recalls that occurred in the parent devices. If anyone is curious about this particular device, see here [2]
The “substantially equivalent” approval loophole is a textbook example of regulatory capture:
- approvals for version N + 1 are basically skipped, as is safety testing
- manufacturers get a liability shield
- for many classes of devices, ladder pulling implies that no one will ever get another v1 approved. (The current rules for that are impossible to meet, but the incumbent company is grandfathered in under the old rules).
The testing is NOT skipped. This is a misunderstanding of 510(k).
The device still has to go through testing, validation and verification. It is a very detailed process and follows IEC spec. The device also needs IEC 60601 testing by a third party lab. In this case at least 60601-2-24.
The 510(k) means it is not doing something new therapeutically therefore it doesn't need to go through PMA process which so much longer and more complex. In this instance they're saying a pump that uses electronics to monitor and deliver glucose already exists. Our device does the same thing therapeutically but we do it differently and here is all out docs showing the device is safe.
If they came up with a magical patch that used quantum chemtrail energy to align the shakras and thereby affect the patient's insulin levels, then they would need to go through PMA and show the therapy is safe and works in small clinical trials followed by larger trials before they can make a device that can be marketed.
It isnt capture. IT is the opposite. Capture is how first movers and established players raise the bar for others. the 510k process does the opposite. it means that the FDA has already reviewed devices of that nature for safety and efficacy, and you dont need to do a clinical trial, you just need to do the testing
to correct your points:
- No testing is skipped (just trials
- There is no liability shield.
I have no idea what you are talking about with ladder pulling. Do you have examples?
The testing to show "substantial equivalence" are not generally trivial. The "original" device needs to show safety, and effectiveness versus clinical endpoints. The follow up devices need to show safety and effectiveness versus appropriate measurements. Some degree of safety testing and/or analysis is generally always required for medical devices, regardless of if its a PMA or 510k.
For the specific case of blood glucose monitors, the 510k submission will require testing to show that your monitor is safe, and provides "at least as good" blood glucose monitoring compared to predicate devices. This allows the device manufacturer to avoid having to prove that blood glucose monitoring will improve clinical outcomes for patients with diabetes. Testing to show performance typically requires some degree of real world (ie, with real operators and patients) testing.
There are certainly be some cases where this gets abused, but overall I think it's a logical system.
Wow, awesome work! It's crazy to think about how the FDA itself probably can't answer a question like "how many predicates does device X have", but I guess also on brand with the idea that government is incompetent more often than not.
Trying to enact a policy that a limits the number/age of predicates in a 501(k) application is impossible if you have no way to find them!
I reported a bug with LibreLink to Abbott about 18 months ago and they still haven't fixed it.
The app doesn't respect Android's Do Not Disturb override list which essentially means I can't use DND on my phone. It's been the same on two phones, there are loads of reports on the Play Store reviews and on Reddit, so it's definitely a bug and not just me.
They haven't even acknowledged it's a bug, every time I'm directed to reinstall the app, clear the cache, factory reset the phone etc.
Is the override set in the system, or does the app somehow know about it? I can imagine some developer misinterpreting how DND detection works and not sending a notification if it's enabled, though I can also imagine android's implementation being bugged in some specific way.
Kind of surprises me that something this easy to replicate and this pervasive (200+ instances is a lot of instances when you consider the size of the user pool) got through the testing. Would be interested to see what the FDA makes them do as corrective action and a breakdown of how this got into production code.
It simply says "Software Design Change" as FDA determined root cause... so super vague.
EDIT: Although the root cause might be software, it's possible that they did detect the crashing during testing, but did not characterize it fully, and/or missed a step in their risk assessment analysis to determine that it may impact battery life and therefore have risk of harm to life.
At least the Tandem doesn't require using the phone app; it's not an option with the Omnipod, which has no other way of communicating with the device. (you either use an app on a small handful of approved Android phones, or use a stripped down Android device they provide you; iOS approval is in the works, but not here yet)
Apple already has tons of custom rules for different types of applications. Does anyone know which restrictions they impose on medical apps? especially ones that control medical grade devices?
Perhaps but Apple has always lagged behind Google on approving diabetes device apps (or severely limit the functionality). I know the Android version of the app for the Tandem had the ability to deliver bolus well before the iPhone version, and the iPhone version of the Omnipod controller app has FDA approval but still isn't approved by Apple
> Does anyone know which restrictions they impose on medical apps? especially ones that control medical grade devices?
Zero. It's not Apple's responsibility. Google Play Store just asked developers to provide more information on apps in the categories "Health" and "Medical" [1], which is more than Apple did so far.
The tech is really hit or miss sometimes. I have a pump that used to panic error out on a misused bluetooth connection, forcing you to set it up again.
Eventually the app developers fixed it but, for something that’s supposed to run 24/7/365 without fail it really rubbed me the wrong way.
On a slightly tangential note, I am interested in working on something better although not hardware inclined, so please reach out to me at my username @tutamail.com if you are interested.
I was personally affected by this, but it did not harm me. The battery drain was obnoxiously high (having to charge 2x a day instead of once every 2-3 days). I am always near a power plug so I was never in much danger of shutdown. However, I can easily imagine how others would be.
I ended up realizing something was odd with the app and tried uninstalling/reinstalling the app to fix it. Which did in fact fix it (at least temporarily).
Readit News
It's kinda terrifying to be honest.
The same reason why software developed by individuals or small teams can be very high quality, while "enterprise" software is universally horrible; the former work in more self-structured and effective ways, but the latter usually involves tons of bureaucracy. This tends to introduction a selection bias to the types of developers who work in those environments.
In my experience working in enterprise software, we as the devs are far removed from any customer problems.
Those problems, when reported, go through some or all the layers of support engineers, product owners, and engineering managers and eventually get put on some every growing list of stuff to do.
The devs, being so far removed from customers, choose to focus on what they can see, which is usually just whatever feature was prioritized because big customer A was promised it would be ready by some date.
A single dev has no such barriers, as you were saying.
Or is your point that only lower quality devs have jobs at those places?
A few months ago, an update to Roche's DiabetLoop (DBLG1) created a thousand problems with emails recommending temporary tricks, subsequent updates, etc.
It’s unclear how such a serious issue didn’t surface during QA testing for a system that is, ultimately, fully under the control of the manufacturer (hardware/software).
I see a lot of effort to ensure the ecosystem is increasingly closed, touting the need for security of devices that can be fatal, but, in reality, there's little attention given. (And I say this because bugs and issues that everyone reports are always present, they're known, and they’re not being addressed.)
Manufacturers should have to submit their devices and software for certification of the technology before itself. Bluetooth and coding standards are mature enough at this point, and it’s time customers start demanding this from their products.
[1] https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPCD/cla...
[2] https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfRES/res...
I think a "sacrificial device" is the right way to make reliable the unreliable parts. Big battery for running the pump, small battery for the smart device connection. If something happens and you drain the auxiliary device it's annoying but not life critical. Your phone and pump can both alert you indicator light/notification that the auxiliary device died.
Why is this surprising? No one is willing to spend any money on it.
A developer can make $400K+ at a FAANG and work remotely. Or they can work on-site for a quarter that salary in a medical enterprise company.
FAANGs may suck at hiring good developers, but good developers are excellent at leaving shitty companies.
Want better experiences on this stuff? Start paying embedded developers non-crap amounts of money.
(Also it remains weird to me that Netflix is considered a tech company)
The reality is companies like this often use minimum cost contract companies to develop the software rather than actually employing developers directly. They seeing software engineers (or any non managerial position) as purely an expense not the source of value in their products so you only get penny pinching on development and QA.
In any case, I was responding to the notion that the FDA was quality-controlling these things. They aren't doing a very good job.
Should all be about people and not profit. These aren't difficult problems to solve. Nightscout is great, as is xDrip+, but pairing your device with xDrip+ gets you locked out of the vendor app and invalidates your warranty, which is a big deal as sensor quality is hit-and-miss and may come out of your arm before the 15 day window is up, so you won't get a free replacement if that happens.
I would love to figure out DIYing CGMs, but the enzyme-dipped probe is the challenging part.
Also, I don't think smartphones should play an active role in this space. As a means of reading data and perhaps modifying routines on a dedicated device? Maybe. But these things should all function independently of phones.
My phone does an update or runs out of juice? I stop getting CGM data until I unlock with PIN and use the NFC to reconnect. Happened to me last night in my sleep, so missed going hypo in my sleep.
Complete junk in this space, IMO. It's very annoying.
Basically this device was cleared through a DAG of other devices that were "substantially equivalent". I made a website to visualize these relationships, and any recalls that occurred in the parent devices. If anyone is curious about this particular device, see here [2]
[1] https://wcedmisten.fyi/post/medical-device-analysis/
[2] https://www.510k.fyi/devices/?id=K232380 (click on "Predicate ancestry graph")
- approvals for version N + 1 are basically skipped, as is safety testing
- manufacturers get a liability shield
- for many classes of devices, ladder pulling implies that no one will ever get another v1 approved. (The current rules for that are impossible to meet, but the incumbent company is grandfathered in under the old rules).
The device still has to go through testing, validation and verification. It is a very detailed process and follows IEC spec. The device also needs IEC 60601 testing by a third party lab. In this case at least 60601-2-24.
The 510(k) means it is not doing something new therapeutically therefore it doesn't need to go through PMA process which so much longer and more complex. In this instance they're saying a pump that uses electronics to monitor and deliver glucose already exists. Our device does the same thing therapeutically but we do it differently and here is all out docs showing the device is safe.
If they came up with a magical patch that used quantum chemtrail energy to align the shakras and thereby affect the patient's insulin levels, then they would need to go through PMA and show the therapy is safe and works in small clinical trials followed by larger trials before they can make a device that can be marketed.
to correct your points:
- No testing is skipped (just trials
- There is no liability shield.
I have no idea what you are talking about with ladder pulling. Do you have examples?
For the specific case of blood glucose monitors, the 510k submission will require testing to show that your monitor is safe, and provides "at least as good" blood glucose monitoring compared to predicate devices. This allows the device manufacturer to avoid having to prove that blood glucose monitoring will improve clinical outcomes for patients with diabetes. Testing to show performance typically requires some degree of real world (ie, with real operators and patients) testing.
There are certainly be some cases where this gets abused, but overall I think it's a logical system.
Trying to enact a policy that a limits the number/age of predicates in a 501(k) application is impossible if you have no way to find them!
Im not sure why you would think that. The answer would take 5 minutes to answer with the PDFs (which is where the parent post got the info).
Its well known you can infer the safety of a specification by assessing a small subset of its features. Quod erat demonstrandum. /s
But seriously, this is how backdoors are trivially integrated.
The app doesn't respect Android's Do Not Disturb override list which essentially means I can't use DND on my phone. It's been the same on two phones, there are loads of reports on the Play Store reviews and on Reddit, so it's definitely a bug and not just me.
They haven't even acknowledged it's a bug, every time I'm directed to reinstall the app, clear the cache, factory reset the phone etc.
Enabling DND makes LibreLink instantly flash up a warning saying "Alarms aren't available."
It's the preventative action that would be interesting, but we probably won't see that.
The specific recall action in the FDA database is here: https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfRES/res...
It simply says "Software Design Change" as FDA determined root cause... so super vague.
EDIT: Although the root cause might be software, it's possible that they did detect the crashing during testing, but did not characterize it fully, and/or missed a step in their risk assessment analysis to determine that it may impact battery life and therefore have risk of harm to life.
for context, there are about 200,000 formally reported events per year for pumps with alternate controllers in the US.[1]
https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfTPLC/tp...
https://www.fda.gov/medical-devices/medical-device-recalls/i...
At least the Tandem doesn't require using the phone app; it's not an option with the Omnipod, which has no other way of communicating with the device. (you either use an app on a small handful of approved Android phones, or use a stripped down Android device they provide you; iOS approval is in the works, but not here yet)
https://www.omnipod.com/innovation/iphone
Zero. It's not Apple's responsibility. Google Play Store just asked developers to provide more information on apps in the categories "Health" and "Medical" [1], which is more than Apple did so far.
[1]: https://support.google.com/googleplay/android-developer/answ...
[1] https://developer.apple.com/app-store/review/guidelines/#phy...
Eventually the app developers fixed it but, for something that’s supposed to run 24/7/365 without fail it really rubbed me the wrong way.
On a slightly tangential note, I am interested in working on something better although not hardware inclined, so please reach out to me at my username @tutamail.com if you are interested.
I ended up realizing something was odd with the app and tried uninstalling/reinstalling the app to fix it. Which did in fact fix it (at least temporarily).