Readit News logoReadit News
patrickas commented on What every software developer must know about Unicode in 2023   tonsky.me/blog/unicode/... · Posted by u/mrzool
dathinab · 2 years ago
> The only modern language that gets it right is Swift:

I disagree.

What is the "right" things is use-case dependent.

For UI it's glyph bases, kinda, more precise some good enough abstraction over render width. For which glyphs are not always good enough but also the best you can get without adding a ton of complexity.

But for pretty much every other use-case you want storage byte size.

I mean in the UI you care about the length of a string because there is limited width to render a strings.

But everywhere else you care about it because of (memory) resource limitations and costs in various ways. Weather that is for bandwidth cost, storage cost, number of network packages, efficient index-ability, etc. etc. In rare cases being able to type it, but then it's often us-ascii only, too.

patrickas · 2 years ago
That is why I like the way Raku handles it.

It has distinct .chars .codes and .bytes that you can specify depending on the use case. And if you try to use .length is complains asking you to use one of the other options to clarify your intent.

  my \emoji = "\c[FACE PALM]\c[EMOJI MODIFIER FITZPATRICK TYPE-3]\c[ZERO WIDTH JOINER]\c[MALE SIGN]\c[VARIATION SELECTOR-16]";
  say emoji; #Will print the character
  say emoji.chars; # 1 because on character
  say emoji.codes; # 5 because five code points
  say emoji.encode('UTF8').bytes; # 17 because encoded utf8
  say emoji.encode('UTF16').bytes; # 14 because encoded utf16

patrickas commented on US Senator Uses ChatGPT for Opening Remarks at a Hearing on AI   bloomberg.com/news/articl... · Posted by u/makaimc
hammyhavoc · 3 years ago
ChatGPT itself tells me it has no explicit world model.
patrickas · 3 years ago
It is not about what the model tells you.

This paper shows an emergent world model in an LLM that was taught to play otello moves https://ar5iv.labs.arxiv.org/html/2210.13382

https://arxiv.org/pdf/2303.12712.pdf This paper discusses (among other things) how a GPT4 model navigated between rooms in a text adventure game and was able to create a map afterward. Literally building a model of the world as it was navigating and drawing a map of that afterwards

patrickas commented on US Senator Uses ChatGPT for Opening Remarks at a Hearing on AI   bloomberg.com/news/articl... · Posted by u/makaimc
hammyhavoc · 3 years ago
Unlike human beings, LLMs have no explicit world model, they're an LLM. Why would a politician be replaced by an LLM? It seems like a highly inappropriate use.
patrickas · 3 years ago
LLMs do have explicit world models that can be even manipulated. There are many recent papers on the subject.
patrickas commented on Google DeepMind CEO says some form of AGI possible in a few years   wsj.com/articles/google-d... · Posted by u/type4
gwoolhurme · 3 years ago
That is actually a really interesting find. Some of the verbiage is parallel to the hype we have around AGI right now too "fusion by 1990 instead of 2000..." does that not sound exactly like AGI in X year up from Y year that we read now?
patrickas · 3 years ago
Please see my reply to parent.

> "fusion by 1990 instead of 2000..." Those three dots are omitting the most important part of the issue if we spend that much extra money on R&D

In the fusion case no one was willing to spend the money, in AGI's case it looks like everyone seems to be willing to spend the money.

I personally hope they won't, but that is a crutial point not to be overlooked.

patrickas commented on Google DeepMind CEO says some form of AGI possible in a few years   wsj.com/articles/google-d... · Posted by u/type4
resource0x · 3 years ago
Fusion by 1990 with a bit of extra investment promised in 1972 https://www.science.org/doi/10.1126/science.175.4027.1194.b

I remember tons of articles from that time arguing the same. Then the attention shifted to "fifth-generation computers".

patrickas · 3 years ago
I don't think this contradicts the AGI prediction though (nor the "Fusion by 1990 with a bit of extra investment" prediction for that matters)

https://external-preview.redd.it/LkKBNe1NW51Wh-8nLSTRdQtTha2...

This chart shows how much people in the 1970s estimated should be invested to have fusion by 1990, to have it by 2000s and to "never" have it. We ended up spending below the "never" amount for research over four decades so of course fusion never happened exactly as predicted.

I think the main difference is that no one was interested in investing in fusion back then, while everyone is interested in investing in AGI now.

patrickas commented on Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models   cerebras.net/blog/cerebra... · Posted by u/asb
leereeves · 3 years ago
That emergence is precisely what I'm looking for evidence of.

Human beings evolved to eat and reproduce and yet here we are, building computers and inventing complex mathematical models of language and debating whether they're intelligent.

We're so far from the environment we evolved to solve that we've clearly demonstrated the ability to adapt.

ChatGPT doing well at a language task isn't demonstrating that same ability to adapt because that's the task it was designed and trained to do. ChatGPT doing something completely different would be the impressive example.

In short: I don't categorically reject the possibility that LLMs might become capable of more than being "statistics-based text generators", I simply require evidence.

patrickas · 3 years ago
This paper from late last year shows that LLMs are not "just" stochastic parrots, but they actually build an internal model of the "world" that is not programmed in, just from trying to predict the next token.

https://ar5iv.labs.arxiv.org/html/2210.13382

PS: More research has been done since that confirmed and strengthened the conclusion.

patrickas commented on GPT-4   openai.com/research/gpt-4... · Posted by u/e0m
LesZedCB · 3 years ago
this here is why it's not fair to criticize GPT-4 so quickly on this question.

for the record, I made the same mistake as nonfamous at first, i almost commented "but it's correct" before going back to double check what i was missing.

i simply skimmed the problem, recognized it as a common word problem and totally missed the unusual constraints from the question. i just didn't pay attention to the whole question.

patrickas · 3 years ago
Your comment reminded me of this article:

Humans Who Are Not Concentrating Are Not General Intelligences

https://www.lesswrong.com/posts/4AHXDwcGab5PhKhHT/humans-who...

patrickas commented on String length functions for single emoji characters evaluate to greater than 1   hsivonen.fi/string-length... · Posted by u/kevincox
patrickas · 5 years ago
Raku seems to be more correct (DWIM) in this regard than all the examples given in the post...

  my \emoji = "\c[FACE PALM]\c[EMOJI MODIFIER FITZPATRICK TYPE-3]\c[ZERO WIDTH JOINER]\c[MALE SIGN]\c[VARIATION SELECTOR-16]";

  #one character
  say emoji.chars; # 1 
  #Five code points
  say emoji.codes; # 5

  #If I want to know how many bytes that takes up in various encodings...
  say emoji.encode('UTF8').bytes; # 17 bytes 
  say emoji.encode('UTF16').bytes; # 14 bytes 

Edit: Updated to use the names of each code point since HN cannot display the emoji

patrickas commented on Welcome to the Next Level of Bullshit   nautil.us/issue/89/the-da... · Posted by u/NoRagrets
warent · 6 years ago
"GPT-3 is a marvel of engineering due to its breathtaking scale. It contains 175 billion parameters (the weights in the connections between the “neurons” or units of the network) distributed over 96 layers. It produces embeddings in a vector space with 12,288 dimensions."

I don't know much about AI, though I do know about programming, and to me this vaguely smells like "our program is so great because it has 1 million lines of code!"

Does the number of parameters, dimensions, etc, really have anything to do with how breathtaking and marvelous something like this is?

patrickas · 6 years ago
As far as I understand in this specific case yes.

The whole schtick of GPT-3 is the insight that we do not need to come up with a better algorithm than GPT-2. If we dramatically increase the number of parameters without changing the architecture/algorithm its capabilities will actually dramatically increase instead of reaching a plateau like it was expected by some.

Edit: Source https://www.gwern.net/newsletter/2020/05#gpt-3

"To the surprise of most (including myself), this vast increase in size did not run into diminishing or negative returns, as many expected, but the benefits of scale continued to happen as forecasted by OpenAI."

patrickas commented on Welders set off Beirut blast while securing explosives   maritime-executive.com/ar... · Posted by u/tafda
yodon · 6 years ago
There is no need to hunt for a conspiracy theory. Once the decision was made by their judiciary to store thousands of tons of explosives near a major population center, the important question ceases to be about what random unexpected event happens to be the one that sets it off. The probability of disaster was clearly going to integrate to one over some timescale. The failure was systemic. The tragedy was inevitable. It's actually kind of amazing it took this long to happen.
patrickas · 6 years ago
There is no indication that it was "the judiciary's decision" to store it "near a major population center".

That's a story floated by the head of customs to try and shift the blame to the judiciary. But there is no evidence for it, and there is much evidence against it.

Source: The court documents released by journalists Riad Kobeisyi and Dima Sadek.

u/patrickas

KarmaCake day464January 24, 2009View Original