I want your API to not return 500 errors for hours at a time, multiple times a day...
Today it was down from 9AM to 2PM, and then from 9:30PM to 11PM.
Yesterday was even worse.
It isn't rate throttling because I only try to retrieve current conditions every 30 minutes.
Also, the load of the API might be a lot lower if it were possible to retrieve just the current measurements, but the response you get back for observations is insanely bloated with 12 hours back of each measurement, plus a bunch of station information that is heavily duplicative of information provided about the station.
Nothing about the station should be returned when you ask for the station's measurements should be returned except a link to API call for the station's information.
Huh? This is kind of an odd take for a few reasons. For starters, NOAA isn't "extremely underfunded"; with the possible exception of the current budgeting cycle, NOAA generally does pretty well and has strong bipartisan support. It could always use more money, but I wouldn't call it "underfunded.
The reason NOAA doesn't buy more data is because most of the available data has limited value. Personal weather stations have substantial quality issues and add almost no value in areas where we already have high-quality surface observations. We thin out and throw away a ton of surface observations already during the data assimilation process to initialize our forecast models anyways - data from aloft is far more valuable and impactful from a forecast impact perspective.
For what it's worth, few if any companies use proprietary observations to improve their forecasts. It's an open secret that the vast majority of companies out there are just applying proprietary statistical modeling / bias correction on top of publicly available data. Only a handful of companies actually have novel observations, and there's limited evidence it makes a significant difference in the forecast. At best, it can result in the way that those statistical corrections are applied to existing forecasts and ensembles - you can count on one hand the number of companies that actually run a vertically-integrated stack including data assimilation of proprietary observations and end-to-end numerical modeling.
That isn't to say there isn't unique value in the observations. It's just that the industry flagrantly misleads about how they use them.
I'm very interested to see how the ML modeling revolution changes this. The ability to perform global forecasts on a single GPU should make it cost competitive for more companies. I know several companies are already deriving their own weights for the forecasting component so that they can sell them. Google appears to be working on the next piece of the puzzle too with using ML for the data assimilation step, or skipping that altogether and using observations to go directly to forecasts.