Built out the demo on my M1 Max Macbook and it was absolutely terrible. Around 10 seconds for each reply, and even then it was saying something totally unrelated.
what is your context size?
Readit Newsjayavanth commented on Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift blog.ivan.digital/nvidia-... · Posted by u/ipotapov
jayavanth commented on When Fine-Tuning Makes Sense: A Developer's Guide getkiln.ai/blog/why_fine_... · Posted by u/scosman
jayavanth commented on NASA Observes First Visible-Light Auroras at Mars jpl.nasa.gov/news/nasa-ob... · Posted by u/pseudolus
Why is the quality so bad compared to visible light daytime photos?
jayavanth commented on Elizabeth Holmes's partner raises millions for blood-testing startup theguardian.com/us-news/2... · Posted by u/elorm
This time it's different
jayavanth commented on Gemma 3 QAT Models: Bringing AI to Consumer GPUs developers.googleblog.com... · Posted by u/emrah
This is my first time trying to locally host a model - gave both the 12B and 27B QAT models a shot.
I was both impressed and disappointed. Setup was piss easy, and the models are great conversationalists. I have a 12 gig card available and the 12B model ran very nice and swift.
However, they're seemingly terrible at actually assisting with stuff. Tried something very basic: asked for a powershell one liner to get the native blocksize of my disks. Ended up hallucinating fields, then telling me to go off into the deep end, first elevating to admin, then using WMI, then bringing up IOCTL. Pretty unfortunate. Not sure I'll be able to put it to actual meaningful use as a result.
you should set a lower temperature
Not true - from the makers :)
do you store the recordings on your server?
jayavanth commented on Intel announces retirement of Pat Gelsinger intel.com/content/www/us/... · Posted by u/tybulewicz
Sad to see Gelsinger go, but despite the churn, I don't think Intel is doomed. At worst, per their most recent Q3 results, I see $150Bn in assets and $4Bn in outstanding stock, and I see the US Gov (both incoming and outgoing) gearing up for a long war against China where Intel would be an asset of national importance.
My back of envelope calculation says Intel should be about $35 a share (150/4). If they stumble when they report Q4, I think the US Gov will twist some arms to make sure that fresh ideas make it onto the board of directors, and perhaps even make Qualcomm buy them.
I do think that Intel need to ditch some of their experiments like Mobileye. Its great (essential) to try and "rebalance their portfolio" away from server/pc chips by trying new businesses, but Mobileye hasnt grown very much.
how do you calculate share prices? curious
jayavanth commented on KlongPy: High-Performance Array Programming in Python github.com/briangu/klongp... · Posted by u/cl3misch
Deleted Comment
They use BFV, which is an FHE scheme allowing a limited number of fast additions and multiplications (enough for their use case).
Zama uses TFHE, which allows any operation (eg comparisons) with unlimited depth.
So if you only need add/mul, BFV, BGV and CKKS are good options. For anything else, you better use TFHE
I was curious about that choice as well. I guess they also just wanted to operate on integers and not floats