Readit News logoReadit News
imsh4yy commented on SSE sucks for transporting LLM tokens   zknill.io/posts/sse-sucks... · Posted by u/zknill
ivan_gammel · 3 months ago
I don’t get it. Client generates UUID for prompt, PUTs the prompt with this UUID on server. Server caches the generated output for reasonable time, so that subsequent PUTs get 200 instead of 201. Transport protocol failures then do not matter. If response isn’t 4x, just retry.
imsh4yy · 3 months ago
Yep, came here expecting to read an interesting take on why SSE sucks or a better alternative, but this just reads like "skill issue." A term I very much dislike but seems appropriate here.
imsh4yy commented on Show HN: StepKit, an open and cross-platform durable execution standard    · Posted by u/tonyhb
imsh4yy · 4 months ago
I've had early access to StepKit and this kind of sane, explicit API with pluggable backends feels like the right direction. Kudos to the team here!
imsh4yy commented on Best Practices for Building Agentic AI Systems   userjot.com/blog/best-pra... · Posted by u/vinhnx
AndyNemmity · 7 months ago
Neat idea!
imsh4yy · 7 months ago
Thank you :)
imsh4yy commented on Best Practices for Building Agentic AI Systems   userjot.com/blog/best-pra... · Posted by u/vinhnx
AndyNemmity · 7 months ago
Love how you experimented, you are a creative thinker.
imsh4yy · 7 months ago
Haha, thank you! I just like to build stuff.
imsh4yy commented on Best Practices for Building Agentic AI Systems   userjot.com/blog/best-pra... · Posted by u/vinhnx
AndyNemmity · 7 months ago
I agreed with much of this, but I started looking into the enterprise ai systems that large companies are making, and they use agent control via software.

So I tried it. It's much better.

Software is just better at handling discrete tasks that you understand, like mapping agent pathing. There's no point giving that to an AI to do.

The Cordinator "Main Agent" should just call the software to manage the agents.

It works really well in comparison.

You can have the software call claude code via command line, sending the prompt in. You have it create full detail logs of what it's doing, and done, and created.

Maybe I'll change my mind, everything is moving so fast, and we're all in the dark searching around for ideas, but so far it's been working pretty well.

You do lose in the middle visibility to stop it.

I also have it evaluating the outputs to determine how well the agents followed their instructions. That seems key to understanding if more context adds value when comparing agents.

imsh4yy · 7 months ago
Anyway I can learn more about this?
imsh4yy commented on Best Practices for Building Agentic AI Systems   userjot.com/blog/best-pra... · Posted by u/vinhnx
tlarkworthy · 7 months ago
These subagents look like tools
imsh4yy · 7 months ago
Yes they are tools.
imsh4yy commented on Best Practices for Building Agentic AI Systems   userjot.com/blog/best-pra... · Posted by u/vinhnx
canterburry · 7 months ago
"The “Smart Agent” Trap: I tried making agents that could “figure out” what to do. They couldn’t. Be explicit."

So what about this solution is actually agentic?

Overall, it sounds like you sat down and did a proper business process analysis and automated it.

Your subagents for sure have no autonomy and are just execution steps in a classic workflow except you happen to be calling an LLM.

Does the orchestrating agent adapt the process between invocations depending on the data and does it do so in any way more complex than a simple if then branch?

imsh4yy · 7 months ago
You're right that this isn't the "autonomous agent" fantasy that keeps getting hyped.

The agentic part here is more modest but real. The primary agent does make runtime decisions about task decomposition based on the data and calls the subagents (tools) to do the actual work.

So yeah, it's closer to "intelligent workflow orchestration." That's probably a more honest description.

imsh4yy commented on Best Practices for Building Agentic AI Systems   userjot.com/blog/best-pra... · Posted by u/vinhnx
adastra22 · 7 months ago
It seems very doubtful to me that every query would be literally the same (e.g. same hash), if these are plain text descriptions of the subset task.
imsh4yy · 7 months ago
I mean that depends on how you define the "input" for the tool. Some can be very deterministic like an enum, boolean, number, etc.
imsh4yy commented on Best Practices for Building Agentic AI Systems   userjot.com/blog/best-pra... · Posted by u/vinhnx
jasonriddle · 7 months ago
Sure, but to clarify, so you are probably setting temperature to close to 0 in order to try to get as consistent output as possible based on the input? Have you made any changes to top k and/or top p that you have found makes agents output more consistent/deterministic?
imsh4yy · 7 months ago
Yes, temp is close to 0 for most models. For top k and top p, I've been using the default values set in OpenRouter.

u/imsh4yy

KarmaCake day28August 18, 2022View Original