* will you store my code + train on workflows that Devin does for me? * are you piping data to other third party providers (i.e. anthropic, openAI)?
* Looks like for gpt-4 turbo (https://artificialanalysis.ai/models/gpt-4-turbo-1106-previe...), there was a huge latency spike on December 28, which is causing the avg. latency to be very high. Perhaps dropping top and bottom 10% of requests will help with avg (or switch over to median + include variance)
* Adding latency variance would be truly awesome, I've run into issues with some LLM API providers where they've had incredibly high variance, but I haven't seen concrete data across providers
Deleted Comment
edit: I re-ran the same prompt on perplexity llama-2-70b and getting 59 tokens per sec there
This + the Music ControlNet post from yesterday gives me some hope that audio AI will go the direction of creative tools, rather than dystopian full song generation.
Just go left to right on each bottle, and keep track of how often each prefix has appeared (i.e. on the first bottle, if you get 1, 0, 0, 1), we'd keep track of: {"1": 1, "10": 1, "100": 1}. Now, if a prefix of length 1 appears 7 times, or a prefix of length 2 appears 3 times, we stop measuring (because there's only 1 left).
In all cases, for 8 bottles you will need 4 measurements, for 4 bottles you will need 3 measurements, 2 bottles will require 2 measurements, and 2 bottles will require 1 measurement. (4 * 8) + (4 * 3) + (2 * 2) + (2 * 1) = 32 + 12 + 4 + 2 = 50. But for the very last bottle, you can just do 0 measurements, by way of process of elimination. so 50 - 1 = 49.