fantispug (u/fantispug)

fantispug commented on Batch Mode in the Gemini API: Process More for Less developers.googleblog.com... · Posted by u/xnx

tripplyons · 2 months ago

For those who aren't aware, OpenAI has a very similar batch mode (50% discount if you wait up to 24 hours): https://platform.openai.com/docs/api-reference/batch

It's nice to see competition in this space. AI is getting cheaper and cheaper!

fantispug · 2 months ago

Yes, this seems to be a common capability - Anthropic and Mistral have something very similar as do resellers like AWS Bedrock.

I guess it lets them better utilise their hardware in quiet times throughout the day. It's interesting they all picked 50% discount.

fantispug commented on Sam Altman Slams Meta’s AI Talent Poaching: 'Missionaries Will Beat Mercenaries' wired.com/story/sam-altma... · Posted by u/spenvo

moralestapia · 2 months ago

>he's just commoditizing the complement

That's a cool smaht phrase but help me understand, for which Meta products are LLMs a complement?

fantispug · 2 months ago

Meta's primary business is capturing attention and selling some of that attention to advertisers. They do this by distributing content to users in a way that maximizes attention. Content is a complement to their content distribution system.

LLMs, along with image and video generation models, are generators of very dynamic, engaging and personalised content. If Open AI or anyone else wins a monopoly there it could be terrible for Meta's business. Commoditizing it with Llama, and at the same time building internal capability and a community for their LLMs, was solid strategy from Meta.

fantispug commented on Programming languages that blew my mind (2023) yoric.github.io/post/prog... · Posted by u/todsacerdoti

vunderba · 10 months ago

While I wouldn't say "mind blown", I really like F#'s built-in support for fluent interface style programming using the pipeline operator, e.g.

  X |> one |> two |> three

  versus the more conventional

  three(two(one(X)))

fantispug · 10 months ago

I find this style changes the way I think about and write code transformations. It's also in shell pipelines, R's magrittr, and Clojure's thread macros, and can be emulated in some OO languages with methods that return the transformed object itself.

fantispug commented on Finding near-duplicates with Jaccard similarity and MinHash blog.nelhage.com/post/fuz... · Posted by u/brianyu8

derefr · a year ago

As a document clustering / dataset deduplication technique, how well does "throwing ML at the problem" (e.g. using a pre-trained LLM encoder to generate vector embeddings of your documents, and then sticking those vectors into a vector DB and doing k-means to cluster the vectors) compare, quality-wise and performance-wise, to these simpler discrete-algorithm methods?

fantispug · a year ago

I have seen it work better than LSH.

Each time you embed a document you search for approximate nearest neighbours before adding it, so it is O(N) like MinHash. Vector indexes like HNSW and PQ have better performance/quality tradeoffs than SimHash LSH which is the analogue of MinHash for cosine distance.

The quality depends on what you mean by near duplicate and the embedding model you use. Current models work well, and if you have labelled data you can fine tune them to be better.

The main drawback is the additional cost of embedding all the documents, especially for longer documents. But this cost has dropped really quickly with smaller models, better optimisations, and faster hardware.

fantispug commented on NLP Course – For You lena-voita.github.io/nlp_... · Posted by u/mjakl

light_hue_1 · 2 years ago

As an ML researcher I'm sad to say that this painfully out of date. It's definitely a course from 5 years ago. Totally irrelevant as an intro to NLP today.

fantispug · 2 years ago

It covers a lot of the fundamentals in some detail (attention and transformers, decoding, transfer learning) that are underneath current cutting edge NLP; this is still a very good foundation likely to be good for several more years.

What might be missing is in-context learning, prompt engineering, novel forms of attention, RLHF, and LoRA (though it covers adaptors), but this is still changing rapidly and the details may be irrelevant in another year. If you have a look at a recent course like Stanford CS224N 2023 there's a lot of overlap.

fantispug commented on Coping with Copilot sigarch.org/coping-with-c... · Posted by u/lameda

alain94040 · 3 years ago

Copilot doesn't change the fact that trivial questions (such as 'write a Fibonacci function') can already be googled.

What I don't know (I can't try the 'free' Copilot without providing a credit card), is how well it understands unusual constraints and separate pieces of C++ classes that make the real programs we write.

For instance, a simple test would be: "write a fibonacci function but skip the number 5". Does it do the right thing?

Another challenge would be a C++ class with several fields. That's what our real programs look like: composite data objects, mixing several different algorithms together to implement complex behavior.

Maybe my data structure has a hash table of items, as well as a direct link to the largest item. When I say: "write the function to insert a new item in the list, and remember to update the largest item if it is larger than the current one", would Copilot do the right thing? Each step is easy in itself (adding an element to a hash, comparing an item to another one).

fantispug · 3 years ago

> Maybe my data structure has a hash table of items, as well as a direct link to the largest item. When I say: "write the function to insert a new item in the list, and remember to update the largest item if it is larger than the current one", would Copilot do the right thing? Each step is easy in itself (adding an element to a hash, comparing an item to another one).

In general it won't solve all your problems, but it's helpful for automating simple things like this (but you still need to test edge cases). With this prompt in Python (which I'm more familiar with):

  from dataclasses import dataclass
  from typing import TypeVar
  
  T = TypeVar('T')
  
  @dataclass
  class MaxDict:
      items: dict[T, float]
      max_value_item: T

      def add_item(

It completed:

    def add_item(self, item: T, value: float):
        if value > self.items[self.max_value_item]:
            self.max_value_item = item
        self.items[item] = value

This was my second attempt; first I called it `max_item` and the completion did something about comparing the key.

fantispug commented on Coping with Copilot sigarch.org/coping-with-c... · Posted by u/lameda

wilg · 3 years ago

I (surprisingly) could not get it to generate a Fibonacci function that skips 5.

fantispug · 3 years ago

With the prompt

  // Print the fibonnaci sequence, except the number 5.
  int main() {

I got the result:

    int a = 0;
    int b = 1;
    int c = 0;
    while (c < 100) {
        c = a + b;
        a = b;
        b = c;
        if (c != 5) {
            cout << c << endl;
        }
    }
    return 0;

fantispug commented on Coping with Copilot sigarch.org/coping-with-c... · Posted by u/lameda

cmrdporcupine · 3 years ago

I understand the problem here and I sympathize with professors in these circumstances. Learning the fundamentals will remain important. But as others have pointed out: if students are unwilling to learn, or are taking shortcuts, it will ultimately hurt them in the long run.

One thing tho is that things like Copilot put the lie to the hypothesis (propagated mostly by the Google-style job interview) that intensely coding clever for-loops to perform algorithmic magic (for things usually already in the standard library) is the best measure of the competence of a software developer. If it can literally be done by a machine, maybe we should be measuring based on something else. Especially since this kind of thing is best done on the job by looking in a good textbook or using the standard library. Probably grading at the university level needs to consider this also.

I haven't used Copilot. I doubt I'm its intended audience. After 30 years, actually writing code is perhaps the easiest part of my job. The mechanics is the easy part. The big picture thinking and figuring how to get it all together into a system is the hard part (and honestly there are plenty of people far better at it than I am). Now, if we get a Copilot for that ... then our profession is in trouble.

fantispug · 3 years ago

This happened in mathematics about a decade ago when Wolfram Alpha came out. Lecturers started complaining that Wolfram Alpha could solve assignment problems with worked steps.

In both cases I think this is a real opportunity; we can let students get more quickly to bigger problems and systems thinking by leveraging these tools. It requires professors to start thinking innovatively about how to teach and assess these subjects.

fantispug commented on A vision of a multi-threaded Emacs coredumped.dev/2022/05/19... · Posted by u/pxc

yakubin · 3 years ago

For me the most frequent source of freezes was Tramp. It led me to ditch Emacs for VSCode (with the remote editing extension) at least for my dayjob (I still use Emacs for my own projects). Tramp is an official built-in package, so it's not as if I downloaded a random package from GitHub.

fantispug · 3 years ago

These issues are often interactions between packages. I use TRAMP daily without freezing, but only when I disable company mode in shell modes. VSCode certainly requires much less of this tweaking, even with a variety of extensions installed.

fantispug commented on Racket Compiler and Runtime Status blog.racket-lang.org/2021... · Posted by u/gleb_the_human

DC1350 · 5 years ago

I used Racket in an intro to CS course a few years ago and was told it’s designed to be a teaching language. Is there any reason to use racket over a more popular functional language? Does anyone use it in production?

fantispug · 5 years ago

The killer feature of racket is it is very easy to make Domain Specific Languages (including the teaching language) and related tooling. However last time I looked the library ecosystem didn't seem great; there were many libraries but few that were actively maintained.