leach (u/leach) - Readit News

leach commented on GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM old.reddit.com/r/LocalLLa... · Posted by u/zigzag312

zozbot234 · 7 months ago

It seems you'll have to offload more and more layers to system RAM as your maximum context size increases. llama.cpp has an option to set the number of layers that should be computed on the GPU, whereas ollama tries to tune this automatically. Ideally though, it would be nice if the system ram/vram split could simply be readjusted dynamically as the context grows throughout the session. After all, some sessions may not even reach maximum size so trying to allow for a higher maximum ends up leaving valuable VRAM space unused during shorter sessions.

leach · 7 months ago

Ah I see interesting, I'll have to play around with this more. I switched from Nvidia to AMD and have found AMD support to still be rolling out for these new cards. I could only get LM studio working so far but I'd like to try out more front ends.

Not a major setback because for long context I'd just use GPT or claude, but it would be cool to have 128k context locally on my machine. When I get a new CPU I'll upgrade RAM to 64, my GPU is more than capable of what I need for a while and a 5090 or 4090 is the next step up in VRAM but I don't want to shell out 2k for a card.

leach commented on GPT-OSS-120B runs on just 8GB VRAM & 64GB+ system RAM old.reddit.com/r/LocalLLa... · Posted by u/zigzag312

leach · 7 months ago

I'm a little confused how these models run/fit onto VRAM. I have 32gb system RAM and 16gb VRAM. I can fit the 20b model all within vram, but then I can't increase the context window size past 8k tokens or so. Trying to max the context size leads to running out of VRAM. Can't it use my system ram as backup though?

Yet I see other people with less resources like 10GB of vram and 32gb system ram fitting the 120b model onto their hardware.

Perhaps its because ROCm isn't really supported by ollama for RDN4 architecture yet? I believe I'm using vulkan to currently run and it seems to use my CPU more than my GPU at the moment. Maybe I should just ask it all this.

I'm not complaining too much because it's still amazing I can run these models. I just like pushing the hardware to its limit.

leach commented on The dating app paradox npr.org/sections/money/20... · Posted by u/zwieback

throwaway2037 · 2 years ago

> That's a recipe for women to feel creeped out.

Countless surveys have shown that women do want to be approached. And don't forget about the "Brad Pitt vs Stalker" duality that exists for women and dating: They either view you as handsome who can do no wrong (including approaching them at Meetups), or some kind of creep. There is little in-between. Also, women view about 80% of men as unattractive. It is not a normal distribution, as men rate women's attractiveness. The open secret is that you need to approach lots of women on a regular basis in all sorts of different settings. Eventually, you will find luck.

leach · 2 years ago

Basically this, the bookstore was a stand in for any type of place that you may frequent and see others frequent.

People react differently to being approached, just like anyone would. If they are just into you it works. If not some are polite and see it as a compliment and just say no. Others will be offended and scoff. Either way no one gets hurt and you just move on.

Eventually you just get lucky with someone who is interested in you back. This is kinda how it was for most of history, so I find it odd people are so against it now. We are social creatures! go out and meet people, if they happen to be mean oh well, that reflects entirely on them.

leach commented on The dating app paradox npr.org/sections/money/20... · Posted by u/zwieback

Qem · 2 years ago

> It's much easier to naturally meet people in real life through work/school.

It was. Nowadays people including the office in their dating pool face a high risk of harassment claims.

leach · 2 years ago

If you don't date at work you still make friends at work and grow your social circle. Leverage that to meet new people through work people.

It can be risky dating at work but some find the trade off worth it. I suppose it depends on how comfortable you are at your job too. I've definitely seen relationships blossom in my workplace more than once. When you spend so much time with people it's only natural.

leach commented on The dating app paradox npr.org/sections/money/20... · Posted by u/zwieback

herbst · 2 years ago

You are talking specifically about the male experience.

As 'female' it doesn't matter how often I use the app, if my profile attracts enough males I get matches and ice breakers all day long. If I accidentally open the app after 2 months it just gets more.

I don't need to match or look out. I get nice and stupid messages in mailbox and can choose from them.

If I go to match 80% (made up but realistic number) of the profiles shown already matched with me.

The apps don't want me to buy anything, they nag me for my time.

I could go on. By design I will only see the most successful or 'aggresive' profiles and nothing else.

leach · 2 years ago

This is very true, I've never used a dating app as the opposite sex so I'm not sure what their experience is like. This definitely sounds about right though.

Makes sense that attention is what they want from you, and how the experience compares to that of an average dude on it.

I suppose I'm forgetting other experiences too, I guess I follow the two "rules" of dating apps because as a dude I get a decent amount of matches. Still I don't like the dating apps, maybe I just yearn for something more real I'm not sure.

leach commented on The dating app paradox npr.org/sections/money/20... · Posted by u/zwieback

leach · 2 years ago

Dating apps by and far are quite useless. If you ever want to know how insidious they are, just download one, finish your profile, and swipe for 10 minutes a week.

Since you are not an "active user" they will give you the most attractive people to swipe on. Every couple of days they will give you a "limited time" discount on gold or platinum or whatever. The push notifications are my favorite part, "you could be missing out on the love of your life!!!".

Not to mention the interactions with the UI are littered with casino like visuals. The whole purpose of the app is to get you addicted and spending time and money on it.

It's much easier to naturally meet people in real life through work/school. If you can't there, go hang out at coffee shops or bookstores or something and just hang. Strike up conversation with people, just live. You'll get rejected and some people will be rude but it's all real. You could also always pick up hobbies and meet people there. Just be social, don't spend time and money on these machines of misery.

leach commented on Noiszy: A browser plugin that creates meaningless web data – digital “noise.” noiszy.com... · Posted by u/throwaway154

elitepleb · 3 years ago

https://adnauseam.io - clicking ads, so you don't have to.

leach · 3 years ago

This looks very interesting

leach commented on Fear of AI just killed a useful tool techdirt.com/2023/08/08/t... · Posted by u/donohoe

iamflimflam1 · 3 years ago

Something that we need to learn is that these brief outbreaks on social media burn themselves out pretty quickly. Everyone shouts for a bit and then moves onto to the next bit of manufactured outrage.

leach · 3 years ago

I agree, people get angry at something for a day and then the timeline tells them to be angry at the next thing.

Always feel bad for people who cave to the mob, usually if the mob is yelling at you you’re on the right track

leach commented on Turns Out There's Another Problem with AI-It's Environmental Toll theguardian.com/technolog... · Posted by u/ikue

leach · 3 years ago

These people would rather stunt humanities growth than come up with alternative energy solutions.

Notice how they always complain but never actually suggest solutions, like setting up more modern nuclear power plants or funding fusion R&D.

No, they are just looking to sell doom and clicks so to them solving the problem would actually put them out of work.

leach commented on IRC is the only viable chat protocol (2022) koshka.love/babel/irc-for... · Posted by u/CHB0403085482

leach · 3 years ago

I’d love to use open source solutions but my friends would never switch over to that because they are lazy and don’t care.

I assume there are many such cases