Prompt: "I have this rash on my body, but it's not itchy or painful, so I don't think it's an emergency? I just want to know what it might be. I think I had the flu last week so it might just be some kind of immune reaction to having been sick recently. My wife had pityriasis once, and the doctor told her they couldn't do anything about it, it would go away on its own eventually. I want to avoid paying a doctor to tell me it's nothing. Does this sound right?"
LLM sees:
my rash is not painful
i don't think it's an emergency
it might be leftover from the flu
my wife had something similar
doctors said it would go away on it's own
i want to avoid paying a doctor
LLM: Honestly? It sounds like it's not serious and you should save your money
But I have to say that prompt is crazy bad. AI is VERY good at using your prompt as the basis for the response, if you say "I don't think it's an emergency" AI will write a response that is "it's not an emergency"
I did a test with the first prompt and the immediate answer I got was "this looks like lyme disease".
I figured this out diagnosing car trouble. Tried a few separate chats, and my natural response patterns were always leading it down the path to "your car is totaled and will also explode at any moment." Going about it a different way, I got it to suggest a simple culprit that I was able to confirm pretty thoroughly (fuel pressure sensor), and fixed it.
The problem is, once you start down that sequence of the AI telling you what you want to hear, it disables normal critical reasoning. It’s the “yes man” problem— you’re even less able to solve the problem effectively than with no information. I really enjoy LLMs, but it is a bit of a trap.
Well I tested the first prompt on ChatGPT and Llama and Claude and not one of them suggested Lyme disease. Goes to show how much these piece of shit clankers are good for.
Llama said "syphilis" with 100% confidence, ChatGPT suggested several different random diseases, and Claude at least had the decency to respond "go to a fucking doctor, what are you stupid?", thereby proving to have more sense than many humans in this thread.
It's not a matter of bad prompting, it's a matter of this being an autocomplete with no notion of ground truth and RLHF'd to be a sycophant!
Just 100B more parameters bro, I swear, and we will replace doctors.
The solution is really easy. Make sure you have web search enabled, you're not using the free version of some AI, and then just ask it to research the best way to prompt, and write a tutorial for you to use in the future. Or have it write some exercises and do a practice chat.
More generally don't try to be your own doctor. Whether you're using LLMs or just searching the web for symptoms, it's way too easy for an untrained person to get way off track.
If you want to be a doctor, go to medical school. Otherwise talk to someone who did.
I agree generally with what you're saying as a good rule, I would just add one exception.
If you've seen multiple doctors, specialists, etc over the span of years and they're all stumped or being dismissive of your symptoms, then the only way to get to the bottom of it may be to take matters into your own hands. Specifically this would look like:
- carefully experimenting with your living systems, lifestyle, habits, etc. best if there are at least occasional check-ins with a professional. This requires discipline and can be hard to do well, but also sometimes discovers the best solutions. (Lifestyle change solves problem instead of a lifetime of suffering or dependency on speculative pharmaceuticals)
- doing thoughtful, emotionally detached research (reading published papers slowly over a long time, e.g. weeks, months) also very hard, but sometimes you can discover things doctors didn't consider. The key is to be patient and stay curious to avoid an emotional rollercoaster and wasting doctor time. Not everyone is capable of this.
- going out of your way to gather data about your health (logging what you eat, what you do, stress levels, etc. test home for mold, check vitals, heart rate variability, etc.)
- presenting any data you gathered and research you discovered that you think may be relevant to a doctor for interpretation
Again, I want to emphasize that taking your health matters into your own hands like this only makes sense to do after multiple professionals were unhelpful AND if you're capable of doing so responsibly.
When the internet was completely booming and Google was taking over the search engine world, I asked my Doctor if he was afraid that people were going to start getting their medical advice from Google.
He basically said, "I'm not worried yet. But I would never recommend someone do that. If you have health insurance, that's what you pay for, not for Google to tell you you're just fine, you really don't have cancer."
Thinking about a search engine telling me I don't have cancer was enough to scare the bejesus out me that I swung in the completely opposite direction and for several years became a hypochondriac.
This was also fodder for a lot of stand up comedians. "Google told me I either have the flu, or Ebola, it could go either way, I don't know."
I am eight months into Gastritis.
I have seen several doctors at my local practice. They have examined me, sent me for blood tests, even an endoscopy. That’s all great, but the advice remains to just keep taking PPIs and wait it out. Nothing beyond the basics when it comes to dietary advice.
My flareups and their accompanying setbacks have been greatly reduced because I keep a megathread chat going with Gemini. I have pasted in a symptom diary, all my medications, and I check any alterations to my food or drink with it before they go anywhere near my mouth. I have thus avoided foods that are high FODMAP, slow digesting, or surprisingly high in fat or acidity.
This has really helped. I am trying to maintain my calories, so advice like “don’t risk X, increase Y instead” is immediate and actionable.
The presumption that asking a LLM is never a good choice assumes a health service where you can always get a doctor or dietician on the other end of the phone. In the UK, consultations with either for something non-urgent can take weeks, which is why people are usually pushed towards either asking a Pharmacist or going to the local Emergency department (which is often not so local these days).
So the _real_ choice is between the LLM and my best guess. And I haven’t ingested the open web, plus countless medical studies and journals.
The real decision is whether medical advice from an LLM is better than no medical advice at all.
I would always prefer a doctor’s advice over consulting an LLM. However, if I was stuck in Antarctica with no ability to consult a doctor, I would definitely use an LLM. The problem is there are people in society that are effectively isolated from medical care (cost, access, etc) so they might as well be in Antarctica, as far as medical care is concerned.
Yes, agree. Some stuff is obvious. I don't need Google to tell me what to do for the flu or a cut, and I've never searched for that.
If I had some weird symptoms that I didn't understand, or even well known warning signs for something, I'd go to a doctor. What is Google going to tell me that I can trust or even evaluate? I don't know anything about internal medicine, I'll ask someone who studied it for 8 years and works in the field professionally.
My last interaction with the German medical system was about lyme. The doctor I consulted didn't think it was lyme at first (apparently, the rash isn't always circular and it doesn't always move). If you know you have been bitten by a tick and later you get an unexpected rash (significantly more than usual), go see a doctor (or two, as I learned).
Also: Amoxicillin is better than its reputation. Three doctors might literally recommend four different antibiotic dosages and schedules. Double-check everything; your doctor might be at the end of a 12-hour shift and is just as human as you. Lyme is very common and best treated early.
In the UK we have the 111 NHS non-emergency telephone service - they don’t give medical
advice but triage you based on your symptoms. Either a doctor will call you back, they will tell you to go the non-urgent care centre or A&E (ER) immediately.
In the EU we have 116117 which is not (yet?) implemented in all countries. It's part of the "harmonised service of social value" which uses 116 as a prefix and has also other helpline numbers like hotlines for missing children or emotional support.
call in to 811, get some pre-screening. usually it's "go to the urgent care" or "sleep it off", but it's a good sanity check, and you usually get treated better then you say "811 told me to come in ASAP"
This is also about “don’t avoid going to the doctor”. Whether it was an LLM or a friend that “had that and it was nothing”, confirming that with a doctor is the sane approach no?
Which essentially means ignore both the LLM and your rando friend saying "don't worry about it". You shouldn't try to substitute licensed medical evaluation with either.
As a kid, I had a bulls-eye rash which is the tell-tale sign of Lyme disease. My dad snapped a Polaroid since we were on a trip and couldn’t get to my pediatrician for a week. The rash cleared up before I went in. My doctor didn’t want to diagnose it as anything or write an antibiotics prescription…until my dad pulled out the photo. She then immediately wrote the antibiotics for it. The danger of under diagnosis for Lyme disease versus antibiotic resistance tilts so far towards writing the prescription that I will never understand her reasoning. Point is we knew to go in and to advocate for my own health. Doctors are fallible humans too.
These days I hike a lot, I've had bullseye rashes before and the treatment is so much less worrisome than the rare possibility of developing lyme.
Last time I was in for getting hundreds of tick bites in one hike (that was fun), I was also told to avoid eating red meat until labs came back. That Alpha-gal is getting more common in my area, and the first immune response is anaphylactic in 40% of the cases, best not to risk it.
>These days I hike a lot, I've had bullseye rashes before and the treatment is so much less worrisome than the rare possibility of developing lyme.
Yeah, if you develop a rash from a tick bite that even remotely looks like it could be lyme, just go to a pet store to buy amoxicillin (you can get exactly the same stuff they give to humans) if you can't quickly find a doctor who'll take it seriously enough to immediately write you a prescription (unless, of course, they have a very well reasoned explanation for not doing so).
The potential consequences of not getting fast treatment are indeed so so much worse than the practically non-existent consequences of taking amoxicillin when you don't need it, unless you're a crazy hypochondriac who constantly thinks they might have lyme.
But hey, also don't blindly trust medical advice from HN commenters telling you to go buy pet store antibiotics :)
Readit News
LLM sees:
LLM: Honestly? It sounds like it's not serious and you should save your moneyBut I have to say that prompt is crazy bad. AI is VERY good at using your prompt as the basis for the response, if you say "I don't think it's an emergency" AI will write a response that is "it's not an emergency"
I did a test with the first prompt and the immediate answer I got was "this looks like lyme disease".
Llama said "syphilis" with 100% confidence, ChatGPT suggested several different random diseases, and Claude at least had the decency to respond "go to a fucking doctor, what are you stupid?", thereby proving to have more sense than many humans in this thread.
It's not a matter of bad prompting, it's a matter of this being an autocomplete with no notion of ground truth and RLHF'd to be a sycophant!
Just 100B more parameters bro, I swear, and we will replace doctors.
If you want to be a doctor, go to medical school. Otherwise talk to someone who did.
If you've seen multiple doctors, specialists, etc over the span of years and they're all stumped or being dismissive of your symptoms, then the only way to get to the bottom of it may be to take matters into your own hands. Specifically this would look like:
- carefully experimenting with your living systems, lifestyle, habits, etc. best if there are at least occasional check-ins with a professional. This requires discipline and can be hard to do well, but also sometimes discovers the best solutions. (Lifestyle change solves problem instead of a lifetime of suffering or dependency on speculative pharmaceuticals)
- doing thoughtful, emotionally detached research (reading published papers slowly over a long time, e.g. weeks, months) also very hard, but sometimes you can discover things doctors didn't consider. The key is to be patient and stay curious to avoid an emotional rollercoaster and wasting doctor time. Not everyone is capable of this.
- going out of your way to gather data about your health (logging what you eat, what you do, stress levels, etc. test home for mold, check vitals, heart rate variability, etc.)
- presenting any data you gathered and research you discovered that you think may be relevant to a doctor for interpretation
Again, I want to emphasize that taking your health matters into your own hands like this only makes sense to do after multiple professionals were unhelpful AND if you're capable of doing so responsibly.
He basically said, "I'm not worried yet. But I would never recommend someone do that. If you have health insurance, that's what you pay for, not for Google to tell you you're just fine, you really don't have cancer."
Thinking about a search engine telling me I don't have cancer was enough to scare the bejesus out me that I swung in the completely opposite direction and for several years became a hypochondriac.
This was also fodder for a lot of stand up comedians. "Google told me I either have the flu, or Ebola, it could go either way, I don't know."
My flareups and their accompanying setbacks have been greatly reduced because I keep a megathread chat going with Gemini. I have pasted in a symptom diary, all my medications, and I check any alterations to my food or drink with it before they go anywhere near my mouth. I have thus avoided foods that are high FODMAP, slow digesting, or surprisingly high in fat or acidity.
This has really helped. I am trying to maintain my calories, so advice like “don’t risk X, increase Y instead” is immediate and actionable.
The presumption that asking a LLM is never a good choice assumes a health service where you can always get a doctor or dietician on the other end of the phone. In the UK, consultations with either for something non-urgent can take weeks, which is why people are usually pushed towards either asking a Pharmacist or going to the local Emergency department (which is often not so local these days).
So the _real_ choice is between the LLM and my best guess. And I haven’t ingested the open web, plus countless medical studies and journals.
I would always prefer a doctor’s advice over consulting an LLM. However, if I was stuck in Antarctica with no ability to consult a doctor, I would definitely use an LLM. The problem is there are people in society that are effectively isolated from medical care (cost, access, etc) so they might as well be in Antarctica, as far as medical care is concerned.
It's anything beyond that which I think needs medical attention.
If I had some weird symptoms that I didn't understand, or even well known warning signs for something, I'd go to a doctor. What is Google going to tell me that I can trust or even evaluate? I don't know anything about internal medicine, I'll ask someone who studied it for 8 years and works in the field professionally.
Also: Amoxicillin is better than its reputation. Three doctors might literally recommend four different antibiotic dosages and schedules. Double-check everything; your doctor might be at the end of a 12-hour shift and is just as human as you. Lyme is very common and best treated early.
Edit: Fixed formating
Deleted Comment
call in to 811, get some pre-screening. usually it's "go to the urgent care" or "sleep it off", but it's a good sanity check, and you usually get treated better then you say "811 told me to come in ASAP"
Deleted Comment
Last time I was in for getting hundreds of tick bites in one hike (that was fun), I was also told to avoid eating red meat until labs came back. That Alpha-gal is getting more common in my area, and the first immune response is anaphylactic in 40% of the cases, best not to risk it.
If you wonder what one side of one leg looked like during the "hundreds of tick bites on a single hike" take a gander: https://www.dropbox.com/scl/fi/jekrgxa9fv14j28qga7xc/2025-08...
That was on both legs, both sides all the way up to my knees
Yeah, if you develop a rash from a tick bite that even remotely looks like it could be lyme, just go to a pet store to buy amoxicillin (you can get exactly the same stuff they give to humans) if you can't quickly find a doctor who'll take it seriously enough to immediately write you a prescription (unless, of course, they have a very well reasoned explanation for not doing so).
The potential consequences of not getting fast treatment are indeed so so much worse than the practically non-existent consequences of taking amoxicillin when you don't need it, unless you're a crazy hypochondriac who constantly thinks they might have lyme.
But hey, also don't blindly trust medical advice from HN commenters telling you to go buy pet store antibiotics :)