Bartweiss (u/Bartweiss)

Bartweiss commented on How well do cars do in crash tests they're not optimized for? danluu.com/car-safety/... · Posted by u/weinzierl

ebg13 · 6 years ago

So they _intentionally_ set themselves up to lie to regulatory agencies and consumers about real world efficiency. That honestly sounds basically the same to me. In both cases the tests are poor approximations, and in both cases someone could accidentally optimize the test, and in both cases someone did it intentionally to deceive people.

Bartweiss · 6 years ago

> in both cases someone could accidentally optimize the test

I think this is what I disagree with.

The water heater story is about a viable-for-market design which also optimized for the test. The equivalent for a car emissions test might be optimizing the transmission to reduce emissions at the specific speeds which will be tested. Those speeds could be sweet spots of the engine curve by accident, or they could be planned that way. I don't think that's necessarily right, but it's within the bounds of "natural" design for the product.

Instead of doing that, VW submitted something for testing which was fundamentally different from what went to market. Rather than being misleading, the test results were fundamentally irrelevant. Creating two completely different modes of behavior isn't something you could do by chance, and it means there's no real limit on how badly they could cheat.

Bartweiss commented on How well do cars do in crash tests they're not optimized for? danluu.com/car-safety/... · Posted by u/weinzierl

im3w1l · 6 years ago

Many important differences.

1. Actively detecting test and behaving differently. It's like stealing a test vs teaching to the test.

2. Lower stakes. Health issues are much more serious than inefficiency.

3. It affects the buyer. It's more acceptable for the buyer to be cheated than everyone around them.

4. People could have created these layers by accident. Favouring those who got lucky is unfair.

Honestly I think basically all my gadgets exaggerate how energy efficient they are, by tuning parameters for tests that don't correspond to the real world. My dishwasher has an energy efficient mode, the manual literally says it's just for compliance and recommends other modes. It's just a fact of life.

Bartweiss · 6 years ago

This is omnipresent even where regulators aren't involved: every graphics card benchmark out there is 'manipulated' relative to real world performance. At this point it's so universal that I don't think anyone is even fighting it - as long as everyone games benchmarks roughly the same amount, the relative scores stay usable.

Your point about fairness and passive design is the one that makes me view these cases differently also. In the anecdote, the product being tested was the same one being sold, and there's no sign the heater was worsened to improve test performance. The designers just picked the best-scoring option among some reasonable configurations. (Frankly, once they noticed that issue, what were they supposed to do? Pick the worst-scoring, or pick the spec out of a hat?)

In the VW story, the test-bench vehicle was fundamentally different from the market vehicle, and the road version was designed to behave worse on the metrics to get other gains. I happen to know someone who bought a diesel Jetta specifically because it was more eco-friendly than other options, and I think he'd draw a clear line between tuning for test metrics and VW consciously lying to their buyers.

Bartweiss commented on How well do cars do in crash tests they're not optimized for? danluu.com/car-safety/... · Posted by u/weinzierl

ebg13 · 6 years ago

But the efficiency ratings also exist for public health and environmental reasons. Why does it matter to you that one is a mandate? They're both intentionally deceiving regulators and consumers (vs accidentally acing the test without cheating). Both are fraud.

Bartweiss · 6 years ago

I do think that manipulating a purely instructive measure is less extreme than manipulating a compliance test; consumers can seek alternate tests and reviews, but the state emissions test has special status even if a dozen other tests give a different result. That said, I believe Energy Star ratings affect tax rebates and electric bills, and they're required to be printed on products - so that's not really an arbitrary test.

There are other differences here too, I think. The water heater trick is passive manipulation that stays in place at all times, which limits how far from "real" performance it can get. And per the story, it seems more like "teaching to the test" than "cheating". That is, Volkswagen consciously moved away from the mandate outside of testing. The water heater was (potentially) as energy-efficient as they could design, with the test score manipulated on top of that.

None of that makes it harmless - if "as good as you can make" doesn't hit standards without manipulating them, that's still a problem. But I do find it less galling than "intentionally worsens emissions outside the test bench".

Bartweiss commented on How well do cars do in crash tests they're not optimized for? danluu.com/car-safety/... · Posted by u/weinzierl

dharmab · 6 years ago

This phenomenon also appears in the testing of crash helmets. The DOT (US government) helmet standard is easy to game since it's very strict on how and where the helmets are tested. The Snell Foundation standard is a bit better since the humans testing the helmets are allowed to look for weak points to target for anvil drops. The new FIM standard adds more variability to the tests to better simulate the varying angles of crashes.

https://youtu.be/uLj9WfoWPSQ?t=413

Bartweiss · 6 years ago

Crash test dummies have basically this problem also. They're designed for realism in certain very narrow ways, and then the very small number of approved dummies are used for testing car safety.

The industry has made a bit of progress, surprisingly unprompted by regulations - female and child dummies came into circulation before they were required in tests. But overall, testing is still run against a tiny handful of body types which move 'realistically' in only a few regulation-guided respects.

Bartweiss commented on Greatest Java apps blogs.oracle.com/javamaga... · Posted by u/miked85

manuelabeledo · 6 years ago

I guess that they wanted to go down the route of "hardly controversial and highly popular" applications, that's why they didn't include Maven.

Granted, Maven is an essential piece in the Java ecosystem, yet it seems that many developers are highly opinionated about it.

Bartweiss · 6 years ago

Citing Maven also feels a bit circular. It's an important Java application, but being a build tool it's only because there's lots of Java out there to build.

Minecraft and a lot of the other apps are terminally impressive, so it's easier to justify the ecosystem that produced them.

Bartweiss commented on How South Korea Reined In The Outbreak Without Shutting Everything Down npr.org/sections/goatsand... · Posted by u/mmhsieh

kstenerud · 6 years ago

If you look at the data here [1] (desktop edition because the mobile edition doesn't have drill down), you can see that all countries (including Asian) are following the same pattern in the "daily increase" graph (click a country on the left to see its data). The only outliers are China and South Korea (flatline), and Japan (flattened curve).

So while masks may have some effect, it hasn't been enough to give different graphs to Asian nations in general.

[1] https://www.arcgis.com/apps/opsdashboard/index.html#/bda7594...

Bartweiss · 6 years ago

Wait, which countries are we referencing outside of those three?

Thailand looks straightforwardly exponential so far and has fairly heavy mask use, agreed. But Singapore, Taiwan, and arguably Malaysia seem too early to call: they're still plausibly on either of a European curve or South Korea's ramp-then-flatline.

Vietnam, Cambodia, Laos, Mongolia, and Burma all seem to be below the line for meaningful data. And Hong Kong isn't broken out. So I guess my questions are: do Indonesia and the Philipines have "mask cultures" to a level comparable to South Korea and Japan, and are their testing regimes wide enough to rely on those curves?

I don't know the answer to that. And I agree that the "masks work" graph/meme circulating is questionable. But unless I'm missing something/somewhere, this data just looks like "too soon to call"?

Bartweiss commented on A detailed look at the router provided by my ISP 0x90.psaux.io/2020/03/01/... · Posted by u/paddlesteamer

non-entity · 6 years ago

A while back, I was playing around with the cable modem / router the ISP gave me because I was curious and an idiot. After screwing around a bit, I managed to find a vulnerability that exposed technician credentials plaintext and they actually worked. Had no idea where to report it though, because the manufacturers contact page could be summed up as fuck you we don't talk directly to consumers. I dont think the vulnerability was that bad, as you had to be logged in to the web interface already with another account, but still.

I don't really trust ISP provided hardware / software now though.

Bartweiss · 6 years ago

> you had to be logged in to the web interface already with another account

Obviously I don't know specifics, but if this applies to any router which has multiple tiers of login then it could be a pretty serious problem. I suspect that might be true for routers designed specifically broadcast multiple networks (e.g. school or shared apartment-building routers)?

Bartweiss commented on Social distancing slowing not only Covid-19, but other diseases too qz.com/1824020/social-dis... · Posted by u/lxm

impendia · 6 years ago

I agree with your original point, even if you no longer stand by it.

>Historically, Kinsa’s methods have been able to account for a sharp rise in the number of people taking tests

What historical precedent is Kinsa referring to? There has never before been a pandemic of this magnitude, in a time and place where many people owned internet-connected thermometers.

Perhaps I am being uncharitable, but it seems that they are leaping to conclusions here. I'm sure that they have done their best to account for this, but the claim that "Dalziel has ensured that a spike in testing didn’t lead to inaccurate predictions" is one I find difficult to take at face value. Ensured?

You said that "there could be some selection bias here." Could. Sorry to cause you to cringe again, but +1.

Bartweiss · 6 years ago

I don't think this is uncharitable at all. I'm sure Kinsa has made a good effort at controlling for testing frequency, and I'm sure it's helped. But there's no reason to think the dynamics of COVID-motivated testing are the same as for flu-season, or new-buyer novelty, or anything else.

And more importantly, how could we know if it is? That's not just a Kinsa problem; we see this over and over again with peer-reviewed studies that "control for" certain factors like socioeconimics or health history. They're inherently limited to controlling for what they know about, and it's never perfect. Often, the entire effect is from an undiscovered variable. Take, say, the widely-promoted study finding that visiting a museum, opera, or concert just once a year is tied to a 14% decline in early death risk. The researchers tried to control for health and economic status, then concluded "over half the association is independent of all the factors we identified that could explain the link." [1]

Now, what seems more likely: that the unexplained half is from the profound, persistent social impact of dropping by a museum or concert once a year? Or that some of the explained factors like "civic engagement" can't be defined clearly, others are undercounted (e.g. mental health issues), and some were missed entirely?

I suspect Kinsa did much better than that, because they're not trying to control for such vague terms. But I think "even after controlling for" should basically never rule out asking "what if it's a confounder"?

[1] https://www.cnn.com/style/article/art-longevity-wellness/ind...

Bartweiss commented on For Years WallStreet Spent More on Buybacks Than It Earned-Now They Want Bailout thesoundingline.com/for-y... · Posted by u/baronmunchausen

ntsplnkv2 · 6 years ago

Privatization also stops making sense when private companies constantly need bailouts.

Bartweiss · 6 years ago

Good point.

The TARP bailout in 2008 involved buying a ton of stock from troubled companies, but it was sold back to them as soon as they could buy the money back. And this will be the second bailout for a bunch of airlines.

So one of the most interesting ideas I've heard is that we shouldn't nationalize things by fiat, but when TARP-style bailouts happen, the government should just keep the stock, at least for a while. If it really was a one-off crisis, the shares are a good investment. But if it's a failing business, or one paying dividends and then looking for handouts, it's not just a money sink.

Bartweiss commented on For Years WallStreet Spent More on Buybacks Than It Earned-Now They Want Bailout thesoundingline.com/for-y... · Posted by u/baronmunchausen

_opc6 · 6 years ago

We could also nationalize the airline industry. We've already seen how capitalism alone is a poor predictor of future demand.

And while we're at it, let's make the internet a utility.

Bartweiss · 6 years ago

There's also a fairly good argument for this in the line of trains and highways. Planes aren't physically trapped on one course, but pretty much every nation heavily regulates who can fly where, when. Airports are often state-controlled, and even private ones need state approval to add new runways or flights.

What we have now is one of the ridiculous "private non-market" arrangements. When airlines in Europe fly empty planes to stop the government from taking their flight slots away, that's not the fault of the companies, but it's also not a functional market we should expect efficiencies from.

I'm not a fan of "regulate markets into dysfunction then nationalize them", but if the fundamental restraints on travel are too severe to let the market function freely, privatization stops making much sense.