The takeaways at the very bottom of the page are valuable:
> Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.
The whole "quantified self" movement might be more about OCD and perfectionism than anything else.
I've been wearing an Apple Watch for close to 10 years. I've tracked my weight as well along those years but nothing crazy like OP. The Apple watch tracked plenty.
I had some strange symptoms and two doctors insisted I had a weak heart and potential heart failure. This was shocking! Turns out I do have a really "weak" rhythm, but heart failure is when your heart is progressively getting worse in it's pumping. I don't even remember which metric he looked at in my Apple health - but basically my heart has always been this way. A doctor looking at a single data point might think I have abnormally low blood pressure/heart rate, but if I've had this for 10 years with no change, the medical assessment is very different - it means nothing. Sometimes boring data is exactly what you need. For this reason, I will probably always wear an Apple watch (or equivalent) moving forward.
Data can feel useless for 10 years until one day it becomes critical. The benefit is spiky and uneven.
But you didn’t spend hundreds of hours on it, so when it did happen to be useful it seemed like an outsized benefit.
I would wager that for most people, most data about themselves will be useless and not worth collecting.
Of course you can’t know what data will be useless or not, so unless the cost of collecting it is minimal or nil (wearing a smart watch, writing down your weight each day/week), it’s probably not worth it.
Spending hundreds of hours to build a solution to capture all data about yourself to find interesting patterns has a huge assumption baked into it: that there are interesting patterns to find.
> Data can feel useless for 10 years until one day it becomes critical. The benefit is spiky and uneven.
Not sure if in your case the data was critical, since the doctor likely would have just had you wear a monitor for a while after to come to the same conclusion.
My first car had a broken speedometer. It was a "hand" style like a clock. Instead of moving to the speed, it would spin 365 degrees. The faster the car went, the faster it would spin. Turns out, I acclimated to it and generally knew what speed i was going (relatively speaking).
The lesson, I think, is everything is relative. Even a dashboard with flawed data that is "consistent" can highlight anomalies. And often, that's all you really need out of them. (Or the lack of anomaly)
This is the obvious "benefit" of hindsight. Yes, you accidentally had access to data that provided historical patterns you exactly could use.
But, for anyone who does, there is another 1000 who do not when something hits them: many illnesses develop gradually, and all of our tests (thousands of blood tests, scans and imaging tech...) would benefit from having historical data when we were "ok".
Similarly, you probably did not have more data than what Apple provided to help narrow the problem you still had, right?
And if everyone was put under so many tests, we'd actually be "solving" a bunch of non-issues for people over-reacting to small deviations from "normal" range.
Apple watch helps you with a few parameters — not to be discounted — but I don't really see it as a counter.
But I see people start min-maxing these numbers as a replacement for big picture health goals.
From the outside, I see someone spending a lot of time focusing on numbers while they are actually regularly stressed, who doesn’t get good sleep, and has somewhat bare minimum exercise.
Collecting data is great but don’t sink so much effort into it until you have a problem.
I had a similar epiphany a few years back when I started wearing a step-tracker/sleep monitor.
It was kinda interesting to see how many times I woke up, or track hours, but to be honest I realised after a few months that when my tracker said "You had good sleep", or "You had bad sleep" I was already aware - I woke up smiling, or grumpy depending on how I'd done.
I didn't ever look at the data and think "I want to go to bed now to catch up on the four hours I missed yesterday". I continued to have mostly consistent hours, but if I was doing something interesting I'd stay awake, and if I was tired I'd go to bed earlier naturally. The graphs and data wasn't providing anything of value, or encouraging me to change my behaviour in any significant way.
I've found one very interesting thing from these trackers - namely how even small amount of alcohol destroy sleep quality metrics. One beer is enough for my sleep scores to drop by like 20-30% and it's consistent and reproducible for me every time. Whether it actually matters - I don't know, but it made me drink much less (from maybe once a week to maybe once every two months) which is good outcome I guess.
Eh, I found several interesting things from various tracking tools. Take a nap? Sleep is destroyed this night. Exercise in the evening? Same. Not something I’d pay attention to without noticing the chart afterwards.
There’s also the motivation factor. I’m not sure of the total %, but I certainly did some exercising just to fill the daily goal. Nothing life-changing, but for the price of a cheapo apple watch se once every 5 years or so, more than worth it.
It’s not unlike simplistic time tracking on my iphone. I spent a lot of time on bullshit websites. Obviously I knew it was happening, but the sheer magnitude was surprising. It’s akin to acute pain letting you know there’s a health problem vs something brewing in the background that you are vaguely aware of, but have no motivation to truly care about - one is far more noticeable than the other
Being aware, and being aware that you are aware are very similar things that are subtly different.
I was aware that alcohol affects your next day, even a little. That's because people always say that alcohol is bad for you (surprise surprise). I heard this, so you could say that I was aware. I generally thought about this as "a hangover is bad for you." and was somewhat dismissive of the "even a single drink has a bad effect" mantra.
I did some experimenting, and slowly realized that even a single drink can indeed have an impact on the next day. It's not a hangover, but an impact that I could feel nonetheless. I needed to do some light stats and a lot more journaling to build this awareness. I am now aware that I am aware.
Same. I had a Garmin for about 6 months and I eventually just stopped wearing it and sold it. Knowing how many steps I took today, checking it several times a day to see if I was meeting my goal, knowing how many vertical feet I skied.. none of this data ended up meaning anything to me.
> it did not justify the hundreds of hours I invested in this project.
I agree with this but minimizing the cost changes the ROI.
Personally, I've discovered useful insights tracking various life metrics. But I also found quickly diminishing returns after a few weeks or months -- if an association isn't obvious within that timeframe it's either too much effort to isolate or too slow or small to matter.
At various points I've tracked calories, macronutrients, weight, allergens, supplements, sleep, exercise volume, exercise timing, nighttime screen use, spending/budget, air quality, and mood. Now I know what kind of cooking wrecks the air quality in my house, what foods I don't digest well, what various protein/carb/fat ratios look like on my plate, how much effort it takes to improve fitness, that exercise in the morning or early afternoon improves my sleep while exercise in the evening harms it, and that any alcohol or caffeine wreck my sleep while screens at night have no measurable effect. But once I understand the associations I can alter my behavior and move on.
> The whole "quantified self" movement might be more about OCD and perfectionism than anything else.
I would agree that continuing to track metrics every day long after they've stopped yielding new insights is often compulsive behavior. But I think that's an argument for time-boxing experiments, not necessarily avoiding them altogether.
I did something similar - I put 18 years of comments on reddit, HN, slashdot, and 3 years of LLM chats in the system. I ended with a similar conclusion - it was less useful than I expected. My intent was to do RAG over my corpus, have a LLM get direct access to what I commented over the years, but unfortunately this much information has a negative effect on LLM creativity. Its responses started to fall in line too much with my ideas and it lost its spark. In the end my conclusion was that all that data was facing towards the past while I desire LLMs to improve in the other temporal direction.
I did the same but with GPT embeddings. My primary problem was different though. I wanted to find when I talked about a related subject somewhere. Search works really well.
I have built a much more limited version of this that combines my journals, sketches, photos, geolocation etc. I found it useful for noticing patterns in my behaviour and in my irrationality. Journaling helped the most, because recording my feelings means acknowledging them and reviewing them.
Above all, it's just interesting. I enjoy reading about the day-by-day progression of a crush or my brutally honest feelings about a trip that produced stunning pictures. It weaves nuance into my history.
For me it's about managing ADHD-like (never diagnosed, and I don't care to assume) patterns, coupled with self-accountability. I have a self-improving dashboard (it kicks off Claude to propose additions based my positive/negative feedback on past attempts, and then builds them) that feels quite helpful. E.g. one of the things it added was fitbit integration that shoves my step count and resting heart rate in my face every day, and it's helped me drive my step count back up towards where I want it to be.
I do think it's not worth spending a whole lot of time on, though - hence why the first thing I did was add that mechanism to have Claude build it for me, with me mostly glancing at a plan and saying yes/no. It's the perfect thing to vibe-code - if it breaks, I revert a commit and it doesn't matter because nobody depends on it but me.
It's important to tell the readers how long you've been doing this - especially to those that also manage ADHD or ADHD-like symptoms.
Why? Because those individuals tend to spin something up, tell everyone about it (online, and offline) and then stop doing it few days later.
The result then ends up being a false signal for others in the same boat. People who read it, feel a spark of recognition ("someone like me actually figured this out"), and then invest real time, energy, maybe money, into replicating something the author themselves quietly abandoned two weeks later.
Just a small heads up from someone who used to get burned in the past :)
It's pure armchair psychology, but this type of project always makes me think about anxiety. Who really needs this level of self observation and control? At the same time, I really enjoy reading about it and I find the window into somebody else's world intriguing.
> Who really needs this level of self observation and control
I liked doing similar things in the past. There's no anxiety in the equation, just pure curiosity. How many times have I done a thing a month/year? I was always curious about stuff like this, much like the OP. There's also the hacker spirit in play - designing the apps for tracking stuff.
You're equating two different things here: "building a custom web application" with "quantified self". These are two entirely different things.
Felix's statement isn't a condemnation of quantified self. I think that sometimes, when you're applying algorithms that aren't well-studied, you get pointless or bogus data. I feel much this way about sleep tracking algorithms from the likes of Apple and Whoop. Vitals, too, although it seems remarkably good at detecting when I'm about to get sick.
As a person in the older demographic of HN, having an Apple Watch quietly collect useful data over years and store it in a database has been immensely useful in helping resolve medical issues.
I used to really be into QS, and if I'm training for a marathon, I'm still studying graphs and drawing conclusions. I'm still into QS, but now I just silently log data to Apple Health and use it for medical histories or to identify certain trends (vo2max, cardio fitness, blood pressure, etc) a couple times a year.
I never spent as much time as OP on this but I did collect a lot of data during peak quantified self. I liked having automatic data collection and being able to see trends, for example. I stopped because of a few issues, not related to time:
* Hardware companies went out of business, stopped supporting devices, etc. It became obvious that there was no long term commitment to make good quality hardware that lasted a long time.
* Many devices and/or data collection was consolidated big, data hungry companies Google and Apple. Competitors have similar anti-consumer uses of data. I don't want any of these companies to have my data.
* Related to the last one, limited to no offline or local only data collection.
It is very hard to gather most of this data with off the shelf hardware and keep your data private.
This provides some insight into startup ideas around privacy, local first, offline only self data collection. I agree this kind of personal life data is something you don't want to contribute unknowingly to big data. I could see wanting to share it particularly around a health problem where only massive compute has a chance at providing answers.
> however retrospectively, it did not justify the hundreds of hours I invested in this project.
Trying to extrapolate this conclusion to the entire "quantified self" movement is not correct. The issue is the time cost, not the act itself. If a trusted company came along (as if...) that sucked up this much data to allow you to answer these questions with minimal effort, I'm sure this would be a different story.
Anything at the fringes of tech with no tried and true solution requires hundreds of hours of effort. The author's conclusions are also personal, there are other styles of living and conclusions to be drawn that change the calculus on whether to do it or not.
I ended up doing something similar to this project a year or so ago: for nearly an entire year, I tracked every single thing I ate or drank.
Who knows how many hours I spent scanning nutritional facts on the backs of boxes, estimating amounts of liquids, and even tracking sips of water. And weighing myself! Thank goodness I used a "smart" scale at least, and I didn't have to worry about carefully inputting my weight to an app each time.
But the whole project was an exercise in perfectionism. "I have to remember this sandwich and log it the next time I'm alone" made me anxious, but once I logged it, I felt a sense of completion. The database and my personal history are now at nirvana. Everything is complete.
All for me to learn things every human alive knows today: eat more food and you'll gain weight; eat less food and you'll lose weight. Yes, I can now tell you the exact average difference in calories I'll eat, statistically speaking, on a day that I have adderall in the morning vs a day that I don't. Yes, there's a similar (but much smaller) difference in average calories per day if I have caffeine in the morning as well. And I can tell you that I generally eat an additional 200-400 calories per day on a Fri-Sun than on a Mon-Thu. Wow, groundbreaking.
I've always had a lot of water, but matching foodanddrink.csv to my HealthKit data showed that I have more water on days that I walk more steps. Mildly interesting to see it written out for me? I guess. But was it worth cataloguing every cup of water? Absolutely not.
Was any of [gestures broadly at me pulling out my phone and cataloguing each item I ate] necessary to learn that? Of course not. It gave me a chance to look back at the database and say "Wowee! I did that! Every day for a year, wow, I'm so cool!" and not much else.
extensive tracking of self-related metrics to improve ones health is the equivalent to reading tons of self-help books to improve ones life/social skills/...
We already mostly know what makes people happy/healthy: personal connections, physical activity, healthy diet and some sort of purpose/goal in life that goes beyond day-to-day activities.
The problem is that these things generally require (hard) work and can be unpleasant sometimes, so humans do what humans do and spend unreasonable amounts of time doing the more pleasant things such as reading and gathering info rather than applying these and what they already know.
(That's not to say that a project like this can't be fun or lead to insights, especially across longer time spans, but i feel like all of the questions in the first paragraph have fairly obvious answers if you know yourself at all, that don't require extensive tracking of stats to get)
Agreed, I have tendency to do this to “fix” something in my life, after doing it for 2 weeks I already know, just the consciously pay attention to something even without recording make it very obvious what’s going on.
I did something similar to pull data from my Garmin watch. This meant writing all manner of code to pull data out of FIT files (interesting and often infuriating self-describing file format), coming up with schemas to hold that data to make it queryable, adding visualisations, performing analysis, pattern matching, etc.
The end result is nothing really useful, I had a bunch of scripts that semi-automated some jobs that would have taken 1 minute to do manually and only ever needed to be done a max of five times a day, but I learned a load of things along the way. Often these were useful lessons that can be applied to many other things when developing software.
In a similar vein I've gone to lots of trouble to build a cooling system for my homelab rack (ESP32 to control PWM fans, Dallas 1-wire for reading temp/humidity, exposing measurements as metrics for scraping/observability, designing things to deal with the different voltages involved, etc). I could have just gone and bought an off-the-shelf solution from AC Infinity and installed it in minutes but where would the fun in that be.
yep, I do a simple version of this in Google Sheets. Very useful to be able to "Ctrl-F" your life, especially when combined with Google Maps location history.
> The whole "quantified self" movement might be more about OCD and perfectionism than anything else.
As someone who has dealt with OCD and perfectionism, I think that could be the case for a lot of people. And the urge to obsessively track everything can be debilitating.
I too learned this the hard way. I still haven't figured out why this is the case - like, is the inherent incidental irreducible complexity of human life too high, so it dominates majority of our actions?
> The whole "quantified self" movement might be more about OCD and perfectionism than anything else.
Quantified Self, at least in the intended form, is focused on testing specific theories, not on collecting large amounts of data and trying to find something interesting in them.
If you are not sick, tracking your life like this is, like he said, is useless. Enjoy your life!
On the other hand, if you are sick like me, charting your long term heath data from doctors visits and photographing skin issues can lead to great discoveries. I have been diagnosed with Erythrocytosis and a susceptibility to mycobacteria infection which caused pulmonary nodules and skin lesions. Only after showing my data collection to my doctor. Since I have mental illness they constantly over looked my physical issues so I needed hard data to convince them of my ideas.
For those curious, I have an minor IL12B deficiency and a partial immune deficiency leading to mildly elevated levels of DexoyATP which is partially corrected with zinc supplements.
This is disappointing.... Last year i noticed large chunks of my life were being monitored via many spreadsheets and i endeavoured to bring them all together in to one Oracle DB. My plan was to eventually put some ui and graphing on top very similar to OP but seeing this is making me think twice.
I kinda disagree. Doing something about open questions which are bothering you can be valuable in itself, even if the result of the process is useless in the end. And usually you are also learning some valuable things along the way, or just push your life into a necessary direction. That isn't related to OCD or perfectionism, it's just gaining more control and insight about some aspects of your life.
A simple back of the envelope calculation shows that Felix causes between 70 and 110 tonnes of CO2 emissions per year just from flying.
Paris accord says 1.5t per person per year, from all activities, Felix's flying alonre is ~10-15x current European yearly per person emissions and ~50-75x those compatible with +1.5C.
If you want to reduce air travel for environmental reasons, then tax it more.
Shaming individuals doesn't seem to be productive or helpful.
Air travel works for people if the benefits outweigh the costs. The only thing that changes behavior is to change the costs.
And even if costs were 10x there are still plenty of people who will fly tons, because it would still be economically productive. There are always going to be people who fly 10x more than others, because certain jobs and roles simply require it.
> If you want to reduce air travel for environmental reasons, then tax it more.
> Shaming individuals doesn't seem to be productive or helpful.
First, none of us have any power to "tax it more" so this is a dead end of discussion. Second, people have agency and we can hold them accountable socially for negative actions even if they are abiding by the current laws (or tax regime). This happens all the time, because laws don't fully align with morality in a culture. Suggesting that we should leave such things to the sole discretion of the economy and taxes describes a strange unhuman-like society that we don't live in.
I think a broad tax would just make it more difficult for middle and lower class to fly. Tax the business/first class and frequent flier, but don't push people who can already barely afford to fly out.
You wrote one of the solutions as if it conflicts with the other one.
Let's raise the tax on an activity according to its negative side effects, while pointing out individuals that do a lot of it and dont take personal responsibility.
> Shaming individuals doesn't seem to be productive or helpful.
I don't see how much support from history for that viewpoint. Some examples of positive societal change driven in part by shaming individuals: drink-driving, civil rights, sexual harassment, automobile safety, the slave trade, McCarthyism.
Most people don't spend a lot of time wistfully considering their CO2 usage, myself included. The religious zeal by which people feel the need to tell me about the ever-warming planet is honestly more off-putting than most actual religions.
Of the very few "f*cks" I can give in my life, I prefer to spend mine as I choose rather than being scolded for not giving mine to the pressing issues that others deem important.
I would totally do this without any shame if I had the need/desire. CO2 isn't going to be solved by well intentioned individuals making absolutely no impact. It will be a generic solution that solves it for everyone, or it won't be solved at all.
I'm also not going to take shorter showers when people are farming in a desert and shipping the crops to China.
You might think this makes me a terrible person. That's probably good. Because it will help people understand what we're up against and what needs to happen to actually solve the problem.
I refuse to worsen my life for an absolutely minuscule abstract benefit to someone, eventually, maybe. I am going to continue eating meat and flying when I want to, and I don't feel the least bit bad about it. It's unfortunate that climate change is happening, but my personal actions are not meaningfully contributing. My choices do not have cascading effects on others; whatever I do, the climate will change in the same manner as it otherwise would have. In that light, I am not going to reduce my QOL just to feel self righteous for no actual benefit to anyone.
You may wonder how this is consistent with my propensity to recycle and follow traffic laws and not do crimes and other socially beneficial minor things to which a comparison could be made to CO2 output: because those all have a greater benefit:cost ratio than flying and eating meat, by far. I am including the benefits of living in a high-trust society in that analysis.
I work as an airline pilot, and I just recently calculated mine to be 65,658 tonnes in my 10-year career so far (not quite so bad when calculated per seat-mile, but still eye opening!).
https://jameshard.ing/pilot/#statistics
There is a section comparing flight emissions to US citizen average total emissions. This might make him feel good, but only 30% of an average American's emissions come from transportation and just a sliver from flying, so it's very likely his total emissions are much higher.
I believe you misunderstand what "average" is: Americans (or people in general), on average, do not fly much.
Some — eg. those who've moved for work — probably fly 10-15x the average, and 10-15x of people fly 1/100th of average. Flying cost and availability correlate with average needs, but that presupposes that some people have higher needs altogether.
Exactly, I stopped reading when I saw the flying stats. There are people who still haven't clocked where our climate is headed.
I get that you may have to see family abroad or maybe indulge for a holiday, but this is "I'm using an airplane to commute" kind of level.
And here I am trying to book my train tickets to go to London instead of flying even though it costs three times as much just to avoid a few kg of CO2 (among other things), it's making me angry.
Depends where you take the train to London, it is a much nicer experience anyhow than going to airports and people should consider that as well (ignoring climate stuff)!
On the price, the very annoying thing is that fuel for planes is not taxed! Changing this would require quite some effort (falls under some specific laws, that are old and nobody wants to touch, etc.) but I think everybody should just ask "honest tax on fuels!" as this will make less people say (or thin) "but climate change is a hoax". Planes are just unfair competition to other transport due to taxes!
Isn't this a drop in the ocean? Why would any 'normal' person forgo flying? How much CO2 emissions have 'world leaders' produced going to summits, or Taylor Swift and her fans flaying to concerts or war flights?
Couple of things: 1) NO ONE is suggesting any one forego flying altogether, or skipping their once-a-year overseas vacation or periodic family visit. 2) THIS level of flying is not normal and is exactly the kind of harmful behavior people have in mind when they complain about frequent flyers. 3) Whinging about summits and Taylor Swift is just a bad faith red herring argument. Obviously less flying is better, no matter by who. To the extent it's related to the topic at all, it bolsters the case for less air travel.
Everything any one person does is a 'drop in the ocean'. Thankfully, we organize and do things collectively very well - it's in our fundamental nature going back to non-human ancestors, and there is a long, rich history of how much we accomplish. Alsmost nothing that has ever been accomplished has been done without a lot of people doing it together.
The general concern around Taylor Swift's emissions has always struck me as shortsighted. Her Eras tour is estimated to have generated around $5bn in economic uplift in the US, at an estimated 10,000 tonnes CO2e for her personal travel. Even if the total footprint is higher, that is thousands of times lower than the emissions intensity of an industry like fast fashion. From an environmental point of view, attending a Taylor Swift show is a much less carbon-intensive way to spend your money than ordering from Temu.
In my experience, tracking objective things like "nutrition" and "sleep hours" is immensely useful to reflect on what went wrong, and tracking subjective things like "mood" or "stress" is useless given hedonic adaptation or heavy swings that make problems obvious, and not need tracking.
What's key is be able to visualize metrics easily on the data and frictionless data entry, I've got a decent setup with iPhone Action + Obsidian + QuickAdd scripts on Obsidian Sync (mobile + laptop). for visualization I use Obsidian Bases and Obsidian notes that run Dataview code blocks and Chart.js, couldn't be happier.
I could track things that are not interesting to reflect on like vitamin D supplementation for accountability but I've never bothered, especially if it's taken ~daily.
The language you use to describe this is fun, I make an app for self tracking called Reflect and would love your opinion of it, even if it doesnt suit your needs exactly.
Strongly agree with this. I’ve been using Apple’s “mood” log for about two years now, and it is extremely helpful for me to have a concrete view of the history of my general affect.
“This entire month I’ve been feeling good, I want to pinpoint why,” or “it’s clear since stressor X entered my life, my affect is lower; how can I resolve this?”
These long term trends are harder for me to track without data. It might be easy for others, but not me!
As someone with Schizoaffective Disorder Bipolar Type, if you are not diagnosed with a mood disorder, tracking "swings in you mood" when you have no clinical disorder seems like a disorder of its own.
I have had people tell me they were "manic". Then I showed them videos I took when I was manic and they see what I mean when I tell them they are not manic.
We have come to a place where we do not want even normal fluctuation in mood, and that is a illness of its own, but it is a cultural illness.
I get that everyone wants to be cynical about this, but you really can't deny that both the visualization and sheer scale of data is impressive. The way the "my life in weeks" is done is also very cool, I'll be stealing that for myself.
Bravo. This is my dream and also my nightmare. I was super into the quantified self movement a while back, before Apple Watch, writing Withings Scale API wrappers in Ruby and Fitbit days.
Every time I try to seriously track metrics of my life, the excitement of the insight gets worn away by the friction of recording and managing. I expect LLMs can help reduce the cost of this by an order of magnitude but then, as you mention, the question is, what do you do / change / learn because of the data?
I recently started tracking nutrition macros with an iOS app MacroFactor which I really like. This is the first time taking my weight doesn't feel like a IDK SHRUG moment and I can actually map my food intake to my weight.
Finances is probably the other highly actionable data source that is such high friction to manage (downloading CSVs, OFXs, monthly...) that it has always been a false start for me. I finally wrote a service to talk to Plaid directly and I successfully used it to categorize my business expenses at tax time. I finally have programmatic access to my bank account data!
You conclusion is definitely a cautionary take:
> the main conclusion is that it is not worth building your own solution, and investing this much time.
But, perhaps a subset of that data you find useful.
For the few times that I have tracked myself doing various activities, I have found that I didn't even need to do anything with the data, I was actively changing habits whilst I knew I was being monitored. For example, I wore a GCM for a few weeks and I found it difficult to live my "normal" day to day lifestyle. I wanted to do the things I felt were "right" and "healthy". Whether or not that is a good thing or not is a different topic, but just my 2c.
The value rarely shows up where you expected it to.
I kept a rough log of my sleep and mood for about a year with no specific goal. Mostly forgot about it. Then I had a weirdly bad few months and went back to look — turns out there was a pretty clear pattern I would've never noticed in the moment.
Maybe the framing of "was it worth it" is the wrong question. It's less like an investment with a return and more like keeping receipts. Useless 99% of the time, then suddenly you really need one.
As someone who dislikes being tracked, I find it disconcerting to have this level of data stored in 3rd party services, but joining up multiple services to give one coherent picture is cool. How cool is questionable, trying to correlate health data with location data is going to give a strange picture, and I question how relating health data to weather is useful. (Do I have lower blood pressure on rainy days?)
Forgive me for I am being sceptical. The might be some insight here I have not considered, but I'd feel a lot more comfortable if it was all self-hosted / self-collected data.
Readit News
> Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.
The whole "quantified self" movement might be more about OCD and perfectionism than anything else.
/edit: quantified, not qualified
I've been wearing an Apple Watch for close to 10 years. I've tracked my weight as well along those years but nothing crazy like OP. The Apple watch tracked plenty.
I had some strange symptoms and two doctors insisted I had a weak heart and potential heart failure. This was shocking! Turns out I do have a really "weak" rhythm, but heart failure is when your heart is progressively getting worse in it's pumping. I don't even remember which metric he looked at in my Apple health - but basically my heart has always been this way. A doctor looking at a single data point might think I have abnormally low blood pressure/heart rate, but if I've had this for 10 years with no change, the medical assessment is very different - it means nothing. Sometimes boring data is exactly what you need. For this reason, I will probably always wear an Apple watch (or equivalent) moving forward.
Data can feel useless for 10 years until one day it becomes critical. The benefit is spiky and uneven.
I would wager that for most people, most data about themselves will be useless and not worth collecting.
Of course you can’t know what data will be useless or not, so unless the cost of collecting it is minimal or nil (wearing a smart watch, writing down your weight each day/week), it’s probably not worth it.
Spending hundreds of hours to build a solution to capture all data about yourself to find interesting patterns has a huge assumption baked into it: that there are interesting patterns to find.
Not sure if in your case the data was critical, since the doctor likely would have just had you wear a monitor for a while after to come to the same conclusion.
The lesson, I think, is everything is relative. Even a dashboard with flawed data that is "consistent" can highlight anomalies. And often, that's all you really need out of them. (Or the lack of anomaly)
But, for anyone who does, there is another 1000 who do not when something hits them: many illnesses develop gradually, and all of our tests (thousands of blood tests, scans and imaging tech...) would benefit from having historical data when we were "ok".
Similarly, you probably did not have more data than what Apple provided to help narrow the problem you still had, right?
And if everyone was put under so many tests, we'd actually be "solving" a bunch of non-issues for people over-reacting to small deviations from "normal" range.
Apple watch helps you with a few parameters — not to be discounted — but I don't really see it as a counter.
But I see people start min-maxing these numbers as a replacement for big picture health goals.
From the outside, I see someone spending a lot of time focusing on numbers while they are actually regularly stressed, who doesn’t get good sleep, and has somewhat bare minimum exercise.
Collecting data is great but don’t sink so much effort into it until you have a problem.
It was kinda interesting to see how many times I woke up, or track hours, but to be honest I realised after a few months that when my tracker said "You had good sleep", or "You had bad sleep" I was already aware - I woke up smiling, or grumpy depending on how I'd done.
I didn't ever look at the data and think "I want to go to bed now to catch up on the four hours I missed yesterday". I continued to have mostly consistent hours, but if I was doing something interesting I'd stay awake, and if I was tired I'd go to bed earlier naturally. The graphs and data wasn't providing anything of value, or encouraging me to change my behaviour in any significant way.
There’s also the motivation factor. I’m not sure of the total %, but I certainly did some exercising just to fill the daily goal. Nothing life-changing, but for the price of a cheapo apple watch se once every 5 years or so, more than worth it.
It’s not unlike simplistic time tracking on my iphone. I spent a lot of time on bullshit websites. Obviously I knew it was happening, but the sheer magnitude was surprising. It’s akin to acute pain letting you know there’s a health problem vs something brewing in the background that you are vaguely aware of, but have no motivation to truly care about - one is far more noticeable than the other
I was aware that alcohol affects your next day, even a little. That's because people always say that alcohol is bad for you (surprise surprise). I heard this, so you could say that I was aware. I generally thought about this as "a hangover is bad for you." and was somewhat dismissive of the "even a single drink has a bad effect" mantra.
I did some experimenting, and slowly realized that even a single drink can indeed have an impact on the next day. It's not a hangover, but an impact that I could feel nonetheless. I needed to do some light stats and a lot more journaling to build this awareness. I am now aware that I am aware.
I agree with this but minimizing the cost changes the ROI.
Personally, I've discovered useful insights tracking various life metrics. But I also found quickly diminishing returns after a few weeks or months -- if an association isn't obvious within that timeframe it's either too much effort to isolate or too slow or small to matter.
At various points I've tracked calories, macronutrients, weight, allergens, supplements, sleep, exercise volume, exercise timing, nighttime screen use, spending/budget, air quality, and mood. Now I know what kind of cooking wrecks the air quality in my house, what foods I don't digest well, what various protein/carb/fat ratios look like on my plate, how much effort it takes to improve fitness, that exercise in the morning or early afternoon improves my sleep while exercise in the evening harms it, and that any alcohol or caffeine wreck my sleep while screens at night have no measurable effect. But once I understand the associations I can alter my behavior and move on.
> The whole "quantified self" movement might be more about OCD and perfectionism than anything else.
I would agree that continuing to track metrics every day long after they've stopped yielding new insights is often compulsive behavior. But I think that's an argument for time-boxing experiments, not necessarily avoiding them altogether.
Above all, it's just interesting. I enjoy reading about the day-by-day progression of a crush or my brutally honest feelings about a trip that produced stunning pictures. It weaves nuance into my history.
A good thing.
I do think it's not worth spending a whole lot of time on, though - hence why the first thing I did was add that mechanism to have Claude build it for me, with me mostly glancing at a plan and saying yes/no. It's the perfect thing to vibe-code - if it breaks, I revert a commit and it doesn't matter because nobody depends on it but me.
Why? Because those individuals tend to spin something up, tell everyone about it (online, and offline) and then stop doing it few days later.
The result then ends up being a false signal for others in the same boat. People who read it, feel a spark of recognition ("someone like me actually figured this out"), and then invest real time, energy, maybe money, into replicating something the author themselves quietly abandoned two weeks later.
Just a small heads up from someone who used to get burned in the past :)
What's the difference here between assuming and saying 'ADHD-like' instead?
I liked doing similar things in the past. There's no anxiety in the equation, just pure curiosity. How many times have I done a thing a month/year? I was always curious about stuff like this, much like the OP. There's also the hacker spirit in play - designing the apps for tracking stuff.
Felix's statement isn't a condemnation of quantified self. I think that sometimes, when you're applying algorithms that aren't well-studied, you get pointless or bogus data. I feel much this way about sleep tracking algorithms from the likes of Apple and Whoop. Vitals, too, although it seems remarkably good at detecting when I'm about to get sick.
As a person in the older demographic of HN, having an Apple Watch quietly collect useful data over years and store it in a database has been immensely useful in helping resolve medical issues.
I used to really be into QS, and if I'm training for a marathon, I'm still studying graphs and drawing conclusions. I'm still into QS, but now I just silently log data to Apple Health and use it for medical histories or to identify certain trends (vo2max, cardio fitness, blood pressure, etc) a couple times a year.
* Hardware companies went out of business, stopped supporting devices, etc. It became obvious that there was no long term commitment to make good quality hardware that lasted a long time.
* Many devices and/or data collection was consolidated big, data hungry companies Google and Apple. Competitors have similar anti-consumer uses of data. I don't want any of these companies to have my data.
* Related to the last one, limited to no offline or local only data collection.
It is very hard to gather most of this data with off the shelf hardware and keep your data private.
Trying to extrapolate this conclusion to the entire "quantified self" movement is not correct. The issue is the time cost, not the act itself. If a trusted company came along (as if...) that sucked up this much data to allow you to answer these questions with minimal effort, I'm sure this would be a different story.
Anything at the fringes of tech with no tried and true solution requires hundreds of hours of effort. The author's conclusions are also personal, there are other styles of living and conclusions to be drawn that change the calculus on whether to do it or not.
Who knows how many hours I spent scanning nutritional facts on the backs of boxes, estimating amounts of liquids, and even tracking sips of water. And weighing myself! Thank goodness I used a "smart" scale at least, and I didn't have to worry about carefully inputting my weight to an app each time.
But the whole project was an exercise in perfectionism. "I have to remember this sandwich and log it the next time I'm alone" made me anxious, but once I logged it, I felt a sense of completion. The database and my personal history are now at nirvana. Everything is complete.
All for me to learn things every human alive knows today: eat more food and you'll gain weight; eat less food and you'll lose weight. Yes, I can now tell you the exact average difference in calories I'll eat, statistically speaking, on a day that I have adderall in the morning vs a day that I don't. Yes, there's a similar (but much smaller) difference in average calories per day if I have caffeine in the morning as well. And I can tell you that I generally eat an additional 200-400 calories per day on a Fri-Sun than on a Mon-Thu. Wow, groundbreaking.
I've always had a lot of water, but matching foodanddrink.csv to my HealthKit data showed that I have more water on days that I walk more steps. Mildly interesting to see it written out for me? I guess. But was it worth cataloguing every cup of water? Absolutely not.
Was any of [gestures broadly at me pulling out my phone and cataloguing each item I ate] necessary to learn that? Of course not. It gave me a chance to look back at the database and say "Wowee! I did that! Every day for a year, wow, I'm so cool!" and not much else.
We already mostly know what makes people happy/healthy: personal connections, physical activity, healthy diet and some sort of purpose/goal in life that goes beyond day-to-day activities. The problem is that these things generally require (hard) work and can be unpleasant sometimes, so humans do what humans do and spend unreasonable amounts of time doing the more pleasant things such as reading and gathering info rather than applying these and what they already know. (That's not to say that a project like this can't be fun or lead to insights, especially across longer time spans, but i feel like all of the questions in the first paragraph have fairly obvious answers if you know yourself at all, that don't require extensive tracking of stats to get)
I did something similar to pull data from my Garmin watch. This meant writing all manner of code to pull data out of FIT files (interesting and often infuriating self-describing file format), coming up with schemas to hold that data to make it queryable, adding visualisations, performing analysis, pattern matching, etc.
The end result is nothing really useful, I had a bunch of scripts that semi-automated some jobs that would have taken 1 minute to do manually and only ever needed to be done a max of five times a day, but I learned a load of things along the way. Often these were useful lessons that can be applied to many other things when developing software.
In a similar vein I've gone to lots of trouble to build a cooling system for my homelab rack (ESP32 to control PWM fans, Dallas 1-wire for reading temp/humidity, exposing measurements as metrics for scraping/observability, designing things to deal with the different voltages involved, etc). I could have just gone and bought an off-the-shelf solution from AC Infinity and installed it in minutes but where would the fun in that be.
As someone who has dealt with OCD and perfectionism, I think that could be the case for a lot of people. And the urge to obsessively track everything can be debilitating.
Quantified Self, at least in the intended form, is focused on testing specific theories, not on collecting large amounts of data and trying to find something interesting in them.
See, for example:
https://gwern.net/zeo/zeo#what-qs-is-not-just-data-gathering
On the other hand, if you are sick like me, charting your long term heath data from doctors visits and photographing skin issues can lead to great discoveries. I have been diagnosed with Erythrocytosis and a susceptibility to mycobacteria infection which caused pulmonary nodules and skin lesions. Only after showing my data collection to my doctor. Since I have mental illness they constantly over looked my physical issues so I needed hard data to convince them of my ideas.
For those curious, I have an minor IL12B deficiency and a partial immune deficiency leading to mildly elevated levels of DexoyATP which is partially corrected with zinc supplements.
Yes it is! And if you control everything, you won‘t make mistakes.
Paris accord says 1.5t per person per year, from all activities, Felix's flying alonre is ~10-15x current European yearly per person emissions and ~50-75x those compatible with +1.5C.
Shaming individuals doesn't seem to be productive or helpful.
Air travel works for people if the benefits outweigh the costs. The only thing that changes behavior is to change the costs.
And even if costs were 10x there are still plenty of people who will fly tons, because it would still be economically productive. There are always going to be people who fly 10x more than others, because certain jobs and roles simply require it.
> Shaming individuals doesn't seem to be productive or helpful.
First, none of us have any power to "tax it more" so this is a dead end of discussion. Second, people have agency and we can hold them accountable socially for negative actions even if they are abiding by the current laws (or tax regime). This happens all the time, because laws don't fully align with morality in a culture. Suggesting that we should leave such things to the sole discretion of the economy and taxes describes a strange unhuman-like society that we don't live in.
I don't see any shaming. It was all matter of fact, free from judgement.
Let's raise the tax on an activity according to its negative side effects, while pointing out individuals that do a lot of it and dont take personal responsibility.
I don't see how much support from history for that viewpoint. Some examples of positive societal change driven in part by shaming individuals: drink-driving, civil rights, sexual harassment, automobile safety, the slave trade, McCarthyism.
Of the very few "f*cks" I can give in my life, I prefer to spend mine as I choose rather than being scolded for not giving mine to the pressing issues that others deem important.
I'm also not going to take shorter showers when people are farming in a desert and shipping the crops to China.
You might think this makes me a terrible person. That's probably good. Because it will help people understand what we're up against and what needs to happen to actually solve the problem.
"Take less flights" isn't the solution.
You may wonder how this is consistent with my propensity to recycle and follow traffic laws and not do crimes and other socially beneficial minor things to which a comparison could be made to CO2 output: because those all have a greater benefit:cost ratio than flying and eating meat, by far. I am including the benefits of living in a high-trust society in that analysis.
Some — eg. those who've moved for work — probably fly 10-15x the average, and 10-15x of people fly 1/100th of average. Flying cost and availability correlate with average needs, but that presupposes that some people have higher needs altogether.
Deleted Comment
I get that you may have to see family abroad or maybe indulge for a holiday, but this is "I'm using an airplane to commute" kind of level.
And here I am trying to book my train tickets to go to London instead of flying even though it costs three times as much just to avoid a few kg of CO2 (among other things), it's making me angry.
On the price, the very annoying thing is that fuel for planes is not taxed! Changing this would require quite some effort (falls under some specific laws, that are old and nobody wants to touch, etc.) but I think everybody should just ask "honest tax on fuels!" as this will make less people say (or thin) "but climate change is a hoax". Planes are just unfair competition to other transport due to taxes!
Reminds me of the soggy straw memes floating around now. I've been having those why bother? thoughts as well.
Dead Comment
Dead Comment
What does this have to do with Felix?
Unrelated link: https://xcancel.com/Ryanair/status/776292730179682304
Deleted Comment
What's key is be able to visualize metrics easily on the data and frictionless data entry, I've got a decent setup with iPhone Action + Obsidian + QuickAdd scripts on Obsidian Sync (mobile + laptop). for visualization I use Obsidian Bases and Obsidian notes that run Dataview code blocks and Chart.js, couldn't be happier.
I could track things that are not interesting to reflect on like vitamin D supplementation for accountability but I've never bothered, especially if it's taken ~daily.
https://apps.apple.com/us/app/reflect-track-anything/id64638...
“This entire month I’ve been feeling good, I want to pinpoint why,” or “it’s clear since stressor X entered my life, my affect is lower; how can I resolve this?”
These long term trends are harder for me to track without data. It might be easy for others, but not me!
I have had people tell me they were "manic". Then I showed them videos I took when I was manic and they see what I mean when I tell them they are not manic.
We have come to a place where we do not want even normal fluctuation in mood, and that is a illness of its own, but it is a cultural illness.
Every time I try to seriously track metrics of my life, the excitement of the insight gets worn away by the friction of recording and managing. I expect LLMs can help reduce the cost of this by an order of magnitude but then, as you mention, the question is, what do you do / change / learn because of the data?
I recently started tracking nutrition macros with an iOS app MacroFactor which I really like. This is the first time taking my weight doesn't feel like a IDK SHRUG moment and I can actually map my food intake to my weight.
Finances is probably the other highly actionable data source that is such high friction to manage (downloading CSVs, OFXs, monthly...) that it has always been a false start for me. I finally wrote a service to talk to Plaid directly and I successfully used it to categorize my business expenses at tax time. I finally have programmatic access to my bank account data!
You conclusion is definitely a cautionary take: > the main conclusion is that it is not worth building your own solution, and investing this much time.
But, perhaps a subset of that data you find useful.
I kept a rough log of my sleep and mood for about a year with no specific goal. Mostly forgot about it. Then I had a weirdly bad few months and went back to look — turns out there was a pretty clear pattern I would've never noticed in the moment.
Maybe the framing of "was it worth it" is the wrong question. It's less like an investment with a return and more like keeping receipts. Useless 99% of the time, then suddenly you really need one.
Forgive me for I am being sceptical. The might be some insight here I have not considered, but I'd feel a lot more comfortable if it was all self-hosted / self-collected data.