Readit News logoReadit News
snippyhollow commented on Code Llama, a state-of-the-art large language model for coding   ai.meta.com/blog/code-lla... · Posted by u/marcopicentini
brucethemoose2 · 2 years ago
Did Meta add scalable rope to the official implementation?
snippyhollow · 2 years ago
We changed RoPE's theta from 10k to 1m and fine-tuned with 16k tokens long sequences.
snippyhollow commented on HiPlot: High-dimensional interactive plots made easy   ai.facebook.com/blog/hipl... · Posted by u/snippyhollow
mbostock · 6 years ago
Based on a comment buried in the source, this library seems to be heavily based on work by Kai Chang:

http://bl.ocks.org/syntagmatic/3150059

It’s a shame Kai isn’t created in the README, LICENSE, or announcement.

snippyhollow · 6 years ago
Thanks for the feedback, we'll fix this.
snippyhollow commented on Online speech recognition with wav2letter anywhere   ai.facebook.com/blog/onli... · Posted by u/moneil971
throwawayhhakdl · 6 years ago
> Trained models: ...

I think I remember a similar thing happening with previous wav2letter releases.

I would love for a simple tutorial on just using a pretrained model but that feels unlikely to ever happen

snippyhollow commented on Everything I googled in a week as a professional software engineer   localghost.dev/2019/09/ev... · Posted by u/_quhg
aloknnikhil · 6 years ago
On the list of things I always Google, is how to create a symbolic link under Linux. I just can't figure out a way to remember what comes first; the source or the destination. The man pages add to the confusion by calling the "source" the target. So, the rule of thumb I now follow is cp or mv semantics.
snippyhollow · 6 years ago
It is like `cp`.
snippyhollow commented on PyTorch 1.0 is out   github.com/pytorch/pytorc... · Posted by u/kashifr
htfy96 · 7 years ago
What surprised me most is the elegance of C++ API. Compared to its equivalence in Python, the C++ version is almost the same if we discard the "auto" keyword [0]. As mentioned in the doc, they put user-friendliness over micro-optimizations, which also proves the expressiveness of modern C++ (at least when they want to prioritize user-friendliness!) [0]: https://pytorch.org/cppdocs/frontend.html#end-to-end-example
snippyhollow · 7 years ago
Quite usable, all of TorchCraftAI uses it https://torchcraft.github.io/TorchCraftAI/ :)

u/snippyhollow

KarmaCake day1498February 25, 2009
About
Research scientist at Facebook AI Research.

PhD in Bayesian modeling applied to Real-Time Strategy Games. Post-doc on models of language acquisition in babies.

http://github.com/syhw

View Original