Readit News logoReadit News
emikulic commented on Mistral releases ‘unmoderated’ chatbot via torrent   404media.co/260-million-a... · Posted by u/cainxinth
jddj · 2 years ago
The philosophical arguments are definitely real and valid, but I still find the clans and labels funny.
emikulic · 2 years ago
It's over for decels.
emikulic commented on TinyLlama project aims to pretrain a 1.1B Llama model on 3T tokens   github.com/jzhang38/TinyL... · Posted by u/cmitsakis
e12e · 2 years ago
Are they upsampling - whatever that means in the context of datasets?

AFAIU slim pajama is about 627B tokens, and Starcoder:

> approximately 250 Billion tokens.

Ed: I see TFA says:

> Combined Dataset Size - Around 950B tokens

> Total Tokens During Training - 3 trillion (slightly more than 3 epochs/1430k steps)

... but I'm not seeing how one becomes three? That's more like 1 trillion than 3 trillion tokens?

emikulic · 2 years ago
Three epochs means it sees each token three times. The dataset is ~1T like you said.
emikulic commented on Cyberpunk in the Nineties (1998)   streettech.com/bcp/BCPtex... · Posted by u/gnoll_of_gozag
Calamitous · 2 years ago
I want the dystopia I was promised in my youth, not the one I got.
emikulic · 2 years ago
I want flying cars and moving sidewalks like I was promised by The Jetsons. :(
emikulic commented on Meta's new AI is being used to create sex chatbots   washingtonpost.com/techno... · Posted by u/elorant
anonzzzies · 3 years ago
Excellent. Any people running these somewhere? Not really interested in sex chats but chatbots that weren’t neutered for any topic/words.
emikulic · 3 years ago
Try https://huggingface.co/ehartford/WizardLM-7B-Uncensored and related models. They're not even trained on smut, just the neutered responses were removed before the RLHF stage (IIUC)
emikulic commented on Scores decline again for 13-year-old students in reading and mathematics   nationsreportcard.gov/hig... · Posted by u/alach11
BeFlatXIII · 3 years ago
I went from terrible at memorizing basic arithmetic to top of class at doing real math.
emikulic · 3 years ago
I used to be bad at math but then I did a 360° on that.
emikulic commented on Stability AI Launches Stable Diffusion XL 0.9   stability.ai/blog/sdxl-09... · Posted by u/seydor
Klaus4 · 3 years ago
There were test/testp models that were based on SD, but v4 and v5 are created from scratch.
emikulic · 3 years ago
Emad said SD cost $600,000 to train. I wonder if Midjourney also had to pay that to train from scratch.
emikulic commented on Stability AI Launches Stable Diffusion XL 0.9   stability.ai/blog/sdxl-09... · Posted by u/seydor
brucethemoose2 · 3 years ago
Its not that simple, as A1111 uses the old stability AI implementation while pretty much everything else uses HF diffusers code.

I worked trying to add torch.compile support to A1111 for a bit, fixing some graph breaks locally, but... It was too much. Some other things, like ML compilation backends, are also basically impossible.

emikulic · 3 years ago
What benefits does the Huggingface diffusers(?) implementation have over A1111?
emikulic commented on Stability AI Launches Stable Diffusion XL 0.9   stability.ai/blog/sdxl-09... · Posted by u/seydor
buffington · 3 years ago
Easy Diffusion is, uh, easily my favorite UI.

While it has a fraction of the features found in stable-diffusion-webui, it has the best out of the box UI I've tried so far.The way it enqueues tasks and renders the generated images beats anything I've seen in the various UIs I've played with.

I also like that you can easily write plugins in Javascript, both for the UI and for server-side tweaks.

emikulic · 3 years ago
I like to run A1111 in --api mode and write my own script to drive it over HTTP.
emikulic commented on Reddit Doubles Down   platformer.news/p/reddit-... · Posted by u/stanislavb
candiddevmike · 3 years ago
A lot of the Google search results for reddit lead to a error page or subreddit is now private, not the content Google originally indexed.
emikulic · 3 years ago
Wow, Reddit is going to lose a lot of pagerank.
emikulic commented on CIA 2010 covert communication websites   cirosantilli.com/cia-2010... · Posted by u/hosteur
boomboomsubban · 3 years ago
I apparently misunderstood how the wayback machine worked. I thought it only archived pages that a user requested, and most pages end up archived due to people with the browser add-on installed to archive every page they visit.

Thanks to both people that cleared up my mistake, it has always seemed they had much stronger coverage than they should for my mistaken view of how it worked.

emikulic · 3 years ago
You're thinking of archive.is

u/emikulic

KarmaCake day72January 20, 2013View Original