jameslevy (u/jameslevy)

jameslevy commented on Do AI detectors work? Students face false cheating accusations bloomberg.com/news/featur... · Posted by u/JumpCrisscross

gradus_ad · 10 months ago

Seems like the easy fix here is move all evaluation in-class. Are schools really that reliant on internet/computer based assignments? Actually, this could be a great opportunity to dial back unnecessary and wasteful edu-tech creep.

jameslevy · 10 months ago

The only longterm solution that makes sense is to allow students to use AI tools and to require a log provided by the AI tool to be provided. Adjust the assignment accordingly and use custom system prompts for the AI tools so that the students are both learning about the underlying subject and also learning how to effectively use AI tools.

jameslevy commented on Drasi: Microsoft's open source data processing platform for event-driven systems github.com/drasi-project/... · Posted by u/benocodes

otterley · 10 months ago

Looks very Azure-centric. Both installation guides (https://drasi.io/how-to-guides/install-sample-applications/b... and https://drasi.io/how-to-guides/install-sample-applications/c...) require Azure to work.

And then there's this:

> Installing Drasi in an EKS cluster can be significantly more complex than a standard installation on other platforms. Instead of downloading a CLI binary using the provided installation scripts, this approach requires modifying the source code of the Drasi CLI and building a local version of the CLI.

Is this an actual requirement or just the current easy path?

jameslevy · 10 months ago

Does it require Azure to work? Or could the Azure steps be relatively easily be swapped out for AWS/GCP/etc?

jameslevy commented on Squiggle: One line code change to add insights to the OpenAI Realtime API squiggle.ai/introduction... · Posted by u/jameslevy

jameslevy · a year ago

Squiggle is an extension of the OpenAI Realtime API that generates insights (summarization, highlights, structured data, and content flags) in realtime as the conversation occurs. You can use Squiggle with a one line code change, no account setup needed.

The website has a link to a simple demo and an embedded video of the demo in action.

I originally built this for something I'm working on, but figured I should make it available as an independent service that anyone can use.

The insight-generation features do incur additional costs utilizing the OpenAI API, with the same OpenAI API key you are using for the Realtime API, and you can configure how often the insights are generated and thus how many extra API calls are being made.

Any feedback is very appreciated!

jameslevy commented on Why you should not apply to YC twitter.com/dvassallo/sta... · Posted by u/georgehill

holoduke · a year ago

Good? My two startups were 2m for 5% shares. Pretty standard.

jameslevy · a year ago

First money in at a 40M valuation? Assuming this is pre-seed, that seems like the investor is taking on a lot of risk based on the distribution of outcomes.

jameslevy commented on Tesla recalls all cybertrucks for faulty accelerator pedals techcrunch.com/2024/04/19... · Posted by u/sarimkx

chollida1 · a year ago

> . After performing a series of tests, it decided on April 12 to issue a recall after determining that an “[a]n unapproved change introduced lubricant (soap) to aid in the component assembly of the pad onto the accelerator pedal,” and that “[r]esidual lubricant reduced the retention of the pad to the pedal.”

How do you have so little quality control and insight into your manufacturing process that someone on your own production line can introduce a new step to your truck manufacturing process that no one noticed?

I guess when analysts said the incumbent auto manufacturers would have a large advantage over Tesla in manufacturing, this is what they meant?

Because this looks like a very unprofessional error to have made for a company that has done well up until now.

jameslevy · a year ago

"Show me the incentive, I'll show you the outcome". I do not know anything of the culture of Tesla and am not commenting on it specifically, and that quote could apply just as much to a company like Boeing. But at companies working on safety-critical products, the problems usually arise from employees acting in their own self interest to do what will get them rewarded for shipping on time, or to avoid punishment for causing delays.

jameslevy commented on Summers: Inflation Reached 18% in 2022 Using the Government's Previous Formula forbes.com/sites/theapoth... · Posted by u/robertn702

jameslevy · a year ago

Goodhart's law applies to how inflation measures have ceased to be an accurate measure: "when a measure becomes a target, it ceases to be a good measure."

jameslevy commented on Yes, social media is a cause of the epidemic of teenage mental illness afterbabel.com/p/phone-ba... · Posted by u/throwup238

jameslevy · a year ago

It's surprising how many parents I know who are in denial about this. It must be because they themselves are constantly using social media and don't want to accept what it is doing to their own mental health.

jameslevy commented on The lifecycle of a code AI completion sourcegraph.com/blog/the-... · Posted by u/tosh

Lucasoato · a year ago

One thing I really miss is a standard way to block any copilot/ai code completion tool from reaching specific files. That’s particularly important for .env files, containing sensitive info. We don’t want leaking secrets outside our machine, imagine the risks if they become part of the next training dataset. That’d be really easy to standardize, it’s just another .gitignore-like file.

jameslevy · a year ago

These types of tools should exclude all files and directories in the .gitignore as standard operating procedure, unless those files are specifically included. Not just because of secrets, but also because these files are not considered to be part of the repository source code and it would be unusual to need to access them for most tasks.

jameslevy commented on The curious side effects of medical transparency newyorker.com/news/essay/... · Posted by u/pseudolus

doodlesdev · 2 years ago

   > Furthermore, simplified metrics frequently distort incentives. If graduation rates are the metric by which funding is determined, then a school might do whatever it takes to bolster them. Although some of these efforts might add value to students’ learning, it’s also possible to game the system in ways that are counterproductive to actual education.

I thought this part of the article particularly interesting. The problem is not just the access to data, but the obsession to optimize the numbers shown. The solution is to understand that:

   whenever a metric becomes a target, it ceases to be a good measure

Which is known as Goodhart's Law [0]. The problem with transparency and any kind of data is that you should be very careful when choosing your metrics and targets, as statistics can be deceiving. Unfortunately, this is probably too complicated to institutionalize in large groups of people who might not get the point, so ultimately a balance has to be chosen between transparency and confidentiality.

In the case of medical records, I believe doctors should have the option to write two notes: One accessible to the patients and the other for medical staff, without the option of sharing it. I fear the consequences of their work being so transparent could ultimately lead to bad communication and in the worst case leading to the wrong treatment.

[0]: https://en.wikipedia.org/wiki/Goodhart's_law

jameslevy · 2 years ago

It's amazing to me how often Goodhart's Law can be observed in the wild, and I've come to believe it is an important enough concept that it should probably be something (along with logical fallacies and other concepts related to critical thinking) that should be prioritized above a lot of other things taught in high schools.