Readit NewsWhen 4-bit quantization comes out, I would expect a GPU with 12GB VRAM to be able to run it.
Disclaimer: I work at Hugging Face
- They benchmarked against general models like GPT-3 but not well-established specific models that have been trained for specific tasks like SPECTER[0] or SciBert[1]. Specter outperformed GPT-3 on tasks like citation prediction two years ago. Nobody seriously uses general LLMs on science tasks, so nobody who actually wants to use this cares about your benchmarks. I want to see task-specific models compared to your general model, otherwise whats going to happen is I either need to run my own benchmarks or, much more likely, I shelve your paper and never read it again. If you underperform some that's fine! If you don't compare to science-specific models all you're claiming is that training on science data gives better science results... thats not exactly an impressive finding. Fine-tuning is a separate thing, I get it, but pleeeeeease just give the people what they want.
- Not released on huggingface. No clue why not. On the back-end this appears to be based on OPT and huggingface compatible, so I'm really confused.
- Flashy website. Combine 1&2 with a well designed website talking about how great you are and most of my warning lights got set off. Not a fan.
@authors, if you're lurking, please release more relevant benchmarks for citation prediction etc. Thanks.
[0] - https://arxiv.org/abs/2004.07180 [1] - https://arxiv.org/abs/1903.10676
Central city locations in Lisbon and Porto have exploded in price, but this is due to AirBnB rather than rich expats. Portugal's tourism industry has boomed, which has put money in a lot of people's pockets, but has distorted local economies somewhat. Better management is required, for example cracking down on AirBnB and NOT building that new cruise liner terminal.
Unfortunately the upper middle class idyll (alluded to in the article) of living a dilapidated (but in a cool way) apartment ten paces from your favourite coffee shop, while doing a socially respectable play job is now no more viable here than in SF, London or Berlin. Indeed, my own favourite 'coffee shop' got turned into a chain tequila bar.
Outside the hot centres, things aren't as bad. Public transport (at least in the AML - Lisbon Metropolitan Area) is excellent by any standard, and decent apartments are available for sensible prices well within one hour public transport commute of the downtown. Compare with London, where people are fighting over each rental.
> family factor weighs a lot here
Yes, and this is a problem, given that Portugal only became democratic 48 years ago and has a large amount of cultural baggage from quite frankly medieval times. Also, the implication of this statement is that family doesn't count elsewhere - and in case you missed it it was spelled out a few lines later, which should give you an idea of the residual naivety and insularity of (old) Portuguese culture. Agency - the idea that you can do or think something independently, and then take responsibility for it - is somewhat lacking too.
But! There is much that is very good about Portugal and the Portuguese people, I feel that here it is ok to be kind to each other. Furthermore, there are good changes, new opportunities are arising (perhaps in a very small part to some of the more productive visa holders), and I see a great future for Portugal - even if many Portuguese often don't.
I also see a bright future in Portugal. Many young people that moved out after university want to come back, after making a killing in their career. They might earn less with the move, but you don't need a six digit salary to live very comfortably here.
Expecting software to be open source is nice when there is an army of 10s of thousands of FAANG employees to constantly keep it up to date, but less so when there's limited people. Sure, it hypothetically could be kept up to date by the generous and capable people of the city after the fact, but that's farfetched. It isn't realistic or practical for a budget-conscious software company to open them selves up to scrutiny, participate in the open source community, accept bug fixes, do code reviews from strangers, etc. It's more expensive to do OSS, not less.
(As an example, the Linux Kernel is mainly made by large companies with lots of expensive employees. Pick your 10 favorite GitHub project with more than 10k stars and see who the primary contributors are.)
Hugging Face transformers has >60k stars and fewer than 30 employed maintainers, many sharing other responsibilities. Arguably, part its success comes external model contributions from FAANG companies (among others), but the key ingredient was the creation of an open platform.
Disclaimer: I work there
But to address both: is it very relevant what LLM you use right now? Local or hosted, openAI or other?
It seems like the interface has converged around chat-based prompts.
New ideas for tuning or improving the efficiency of foundational models are published almost every week.
If one wants to build a product on top of of generative AI, why not simply start with what’s free or works with one’s dev environment?
Presumably, the interaction with or API to text-based gen AI will be very similar no matter what engine is best for your use case at any given time.
This would imply these backends will be swappable, the way web services are that copy AWS S3 APIs.
So, to return to the point, can’t people just build their product with openAI or other and plan to move away based on the cost and fit for their circumstances?
Couldn’t someone say prototype the entire product on some lower-quality LLM and occasionally pass requests to GPT4 to validate behavior?
It seems far-fetched to believe this tech can be constrained by legislation.
OpenAI can lobby all they want, it won’t necessarily buy them anything. Look what happened with FTX.
Since LLMs can be run locally and the engines be black boxes to the user, how could a legislative act really prevent them from being everywhere—-especially given the public utility.
It can be done -- it is the basis for assisted generation and related work. It does require full access to the model, to be time and money-efficient. See https://huggingface.co/blog/assisted-generation
Disclaimer: I'm the author of the blog post linked above.