DotSlash is a new, open source command line tool that makes large executables available in source control with a negligible impact on repository size, thus avoiding expensive clone operations.
Readit Newsbolinfest commented on DotSlash: Simplified Executable Deployment engineering.fb.com/2024/0... · Posted by u/bolinfest
If you want to skip the paper and watch the video: https://youtu.be/ANDJ0TKjyWw
Disclaimer: I am the person in the video.
bolinfest commented on Sapling: A new source control system with Git-compatible client engineering.fb.com/2022/1... · Posted by u/bolinfest
Originally started as an extension to Mercurial, but grew into its own SCM with a cross-platform virtual filesystem in C++ and a distributed server in Rust.
bolinfest commented on Scaling Mercurial at Facebook (2014) engineering.fb.com/core-d... · Posted by u/based2
As an engineer that worked both at google and Facebook, I vastly prefer googles monorepo on perforce. Combining that with citc was pretty solid way to develop your project.
Hg is a bit of a nightmare in the wfh situation. Really slow, hangs for a long time if you haven't synced in a few days. Yes, im sure there are ways to tweak, but not sure if you can tweak them enough!
As a fellow Facebook employee, FYI, Hacker News is not a great place to get tech support.
For the benefit of everyone else on this thread, note that Facebook uses https://github.com/facebookexperimental/eden. From the README:
> Despite having originally evolved from Mercurial, EdenSCM is not a distributed source control system.
i.e., this is not "stock" Mercurial. For example, the Eden server (Mononoke) is written in Rust and the complementary virtual filesystem is written in C++. This has very different performance characteristics than real Mercurial.
EdenSCM works with both EdenFS (the custom filesystem) and a traditional filesystem. If you use EdenFS, pulls will be much cheaper because you only fetch what you use. If you use a traditional filesystem, EdenSCM supports the same "sparse checkouts" feature as stock Mercurial (https://firefox-source-docs.mozilla.org/build/buildsystem/sp...), which can also be used to reduce the size of the slice of the monorepo you pull down.
Last I checked, Perforce (and Google's "implementation" of Perforce, Piper) did not provide nearly the same level of support for stacked diffs as Eden. As both Google and Facebook have cultures of pre-commit code review, working with stacked diffs makes it much easier to make progress while waiting for approvals on earlier diffs.
I believe there are relative advantages/disadvantages of Eden vs. Piper+CitC and that both projects aspire to have the best of each in the limit.
bolinfest commented on Buck – A build system developed and used by Facebook buckbuild.com/... · Posted by u/Nimsical
My name is Michael Bolin and I created Buck.
When I started the project, Buck had one specific goal: to make Android builds faster (https://youtu.be/CdNw6mRpsDI). At the time, the recommended way of building Android (from Google) was to use Ant. So when someone points to Buck as an example of "creating competing tools more often than persisting with and improving existing ones," I'd like to point out that you can't fix Ant if these are your issues with Ant:
* It is unsound. * Because it is unsound, it is irreparably slow. * It uses XML as a build language.
Yes, in July 2012, there were a number build systems on the market (though Bazel was not one of them, but Pants was), and none of them focused on building Android. And even if they did, few (if any) software companies were building an Android app as large as Facebook, so it was unlikely that anyone else was going to design for our scale.
It also wasn't just about build times, but about how I wanted to see us organize code in our repository. At the time, there was a flat list of folders in the Android repo, each called lib-something. This drives me insane because you inevitably end up with two (or more!) people creating com.facebook.common.StringUtils, each in their own lib-something. (It's also annoying to `ls` this "lib-" directory over time.)
In contrast, Buck/Bazel encourage the use of a unified tree, but still encourage fine-grained modularization (which is key as your build graph gets very large). This has been shown to scale to extremely large monorepos at both Facebook and Google.
Finally, by having total control of the build system, we were able to build in all sorts of cool tricks to build Android very fast, both in the large and in the small: https://youtu.be/Y9MfGS3qfoM. I don't think there is any other build system we could have decided to work with at the time to achieve these gains.
Buck has since evolved to build everything else at Facebook. This is not because the Buck team set out to conquer the world, but because people internally wanted the benefits of Buck for their builds. Building an alternative toolchain to xcodebuild was a mammoth effort (and one for which I take no credit). Having one build language for a heterogeneous collection of programming languages in a monorepo is no small feat.
Finally, to the people who believe "The big guys want developers locked into their ecosystems," I have news for you: the Buck team is not offended if you use Bazel, Gradle, Make, or anything else. Buck is open source because we wanted to share it with the community, not dominate it. Like many of you, people are excited to show their work and learn from others.
bolinfest commented on Buck – A build system developed and used by Facebook buckbuild.com/... · Posted by u/Nimsical
Can somebody compare this to Gradle?
I mean somebody who has actually used both of them (and not read some versus blog posts).
I explained that here: http://stackoverflow.com/a/19111982/396304