I think I have almost the opposite intuition. The fact that attention models are capable of making sophisticated logical constructions within a recursive grammar, even for a simple DSL like SQL, is kind of surprising. I think it’s likely that this property does depend on training on a very large and more general corpus, and hence demands the full parameter space that we need for conversational writing.
Readit Newssiddboots commented on Why are your models so big? (2023) pawa.lt/braindump/tiny-mo... · Posted by u/jxmorris12
Since this is still on HN frontpage. Does anyone suggest any math books to help solve these problems. I am pretty sure you can't solve problems above 50 without strong maths background.
Concrete Mathematics is probably the best single book that you could read to prepare you for some the problems beyond the first 50. It’s extremely fun, and also mathematically serious. A large portion of PE problems are exactly in the cross sections of number theory, combinatorics, and computation that is covered in this book.
siddboots commented on Category Theory Illustrated – Natural Transformations abuseofnotation.github.io... · Posted by u/boris_m
What's the thing with category theory? I see this topic discussed quite frequently here but I don't get it why people are so into it
It's just a good set of models to use to think about all sorts of different mathematical systems, kind of like a unified vocabulary. Beyond undergraduate level, category theory these days plays a huge role within many vast fields - e.g., algebraic geometry, algebraic topology, or representation theory.
+100000 to
A hybrid of Strong (the lifting app) and ChatGPT where the model has access to my workouts, can suggest improvements, and coach me. I mainly just want to be able to chat with the model knowing it has detailed context for each of my workouts (down to the time in between each set).
Strong really transformed my gym progression, I feel like its autopilot for the gym. BUT I have 4x routines I rotate through (I'll often switch it up based on equipment availability), but I'm sure an integrated AI coach could optimize.
I do this at the moment in my hand rolled personal assistant experiment built out of Claude code agents and hooks. I describe my workouts to Claude (among other things) and they are logged to a csv table. Then it reads the recent workouts and makes recommendations on exercises when I plan my next session etc. It also helps me manage projects, todos, and time blocked schedules using a similar system. I think the calorie counter that the OP describes would be very easy to add to this sort of set up.
siddboots commented on Crimes with Python's Pattern Matching (2022) hillelwayne.com/post/pyth... · Posted by u/agluszak
While the article is very entertaining, I'm not a fan of the pattern matching in Python.
I wish for some linter rule that can forbid the usage of pattern matching.
Can you explain why? Genuinely curious as a lover of case/match. My only complaint is that it is not general enough.
siddboots commented on Why LLMs can't really build software zed.dev/blog/why-llms-can... · Posted by u/srid
- When we have a report of a failing test before fixing it, identify the component under test. Think deeply about the component and describe its purpose, the control flows and state changes that occur within the component and assumptions the component makes about context. Write that analysis in file called component-name-mental-model.md.
- When ever you address a failing test, always bring your component mental model into the context.
Paste that into your Claude prompt and see if you get better results. You'll even be able to read and correct the LLM's mental model.
In my experience, complicated rules like this are extremely unreliable. Claude just ignores it much of the time. The problem is that when Claude sees a failing test it is usually just an obstacle to completing some other task at hand - it essentially never chooses to branch out into some new complicated workflow and instead will find some other low friction solution. This is exactly why subagents are effective: if Claude knows to always run tests via a testing subagent, then the specific testing workflow can become that subagent’s whole objective.
siddboots commented on I tried to replace myself with ChatGPT in my English class lithub.com/what-happened-... · Posted by u/lapcat
A lot of the purposes in education for which the use of AI would be considered "cheating" involve writing assignments of one sort or another, so I don't know why most of these education scenarios don't simply redirect the incentive.
For example, in an English class with a lot of essay-writing assignments, the assignments could simply be worth 0% of the final mark. There would still be deadlines as usual, and they would be marked as usual, but the students would be free to do them or not as they pleased. The catch would be that the *proctored, for-credit* exams would demand that they write similar essays, which would then be graded based on the knowledge/skills the students would have been expected to gain if they'd done the assignments.
Advantages:
- No more issues with cheating.
- Students get to manage (or learn to manage) their own time and priorities, as is expected of adults, without being whipped as much with the cane of class grades.
- The advanced students who can already write clearly, concisely and convincingly (or whatever the objectives are of the writing exercises) don't have to waste time with unneeded assignments.
- If students skip the assignments, learn to write on their own time using ChatGPT and friends, and can demonstrate their skills in exam conditions, then it's a win-win.
This all requires that whoever is in charge of the class have clear and testable learning goals in mind -- which, alas, they all-too-often do not.
This would mean moving to 100% weighted exams, and there's good reasons why there has been a general trend away from that over recent decades. For one thing, some students simply perform better under pressure than others, independent of their preparedness and knowledge of the material.
Mind you, I don't really have any alternative suggestions.
siddboots commented on Logical implication is a comparison operator btdmaster.bearblog.dev/lo... · Posted by u/btdmaster
This perspective is the inspiration for much of lattice theory. When you consider implication as an ordering, then "x and y" becomes max(x, y), "x or y" becomes min(x, y). True becomes the top term, False becomes the bottom. One of the neat implications is that much of what we think of as being propositions in boolean algebra also work in the wider setting of Heyting algebras i.e., any lattice that also has implication.
I typically have a discussion about how I want the architecture to be and my exact desired end state. I make the model repeat back to me what I want and have it produce the plan to the degree I am happy with. I typically do not have it work in building large amorphous systems, I work with and have it plan subsystems of the overall system I'm building.
A lot of my discussion with the model is tradeoffs on the structure I'm imagining and methods it might know. My favorite sentence to send Claude right now "Is go google this." because I almost never take its first suggested response at face value.
I also watch every change and cancel and redirect ones I do not like. I read code very fast and like the oversight, because even small stupidities stack up.
The workflow is highly iterative and I make changes frequently, my non AI workflow was like this too. Write, compile, test, tweak and repeat.
I like this workflow a lot because I feel I am able to express my designs succinctly and get to a place I'm happy with with much less writing than a lot of the actual code itself which in many cases is not an interesting problem, but work that needs to happen for a working system at all.
I do wind up taking over, but feel less than I used to, in edges where its clear there is not a lot of training data or I'm working on something fairly novel or lower level.
I work in Python, Rust and Typescript (Rust by far most often) and the majority of my work is technically challenging but at the systems design level maybe not low level systems programming challenging. Think high concurrency systems and data processing, training models, and some web dev.