Readit News logoReadit News
droptablemain · 2 years ago
My repos were coming up as 404s. I was wondering if I had been canned...
__MatrixMan__ · 2 years ago
The "nothing to see here" approach to access control has a lot of weird culture-consequences. I wish software would just address me like the peasant that I am, rather than trying to gaslight me into believing that my artificially limited world is the whole one.
openthc · 2 years ago
If there's nothing wrong with me, maybe there's something wrong with the universe...The universe is a spheroid region, 705 meters in diameter.
dilap · 2 years ago
I think you don't want to reveal the presence of projects by name or not -- just the name of repo could be sensitive info, potentially.
kodapoda · 2 years ago
I think if you are in a corporate account and have correct access permissions to the account (i.e. URL namespace) it should not show 404. It's just super confusing.

Deleted Comment

ellisv · 2 years ago
Not mutually exclusive - GitHub could be down AND you could be canned!
APhoenixRises · 2 years ago
No joke, but this is exactly my situation.
ignoramous · 2 years ago
We were warned multi-deploys with big changes were incoming: "For lack of a better term, some big shit is coming at GitHub Universe." - Thomas Dohmke, CEO

https://twitter.com/ashtom/status/1720319071567421679

Deleted Comment

dekhn · 2 years ago
Well, that explains a problem in an interview I was just administering (the interview code sample downloads a file from github).

Guess I can't treat github like a CDN.

shepherdjerred · 2 years ago
Hah I had the same thought

Dead Comment

taf2 · 2 years ago
Ah I was thinking it was southwest wifi!

Deleted Comment

Dead Comment

SoftTalker · 2 years ago
I like how "Every request returns a 404" == "degraded performance"
agilob · 2 years ago
It should load faster if there's nothing to load
flukus · 2 years ago
If nothing loads nothing gets cached and you can get in a cycle of very slow 404s.

The current abomination I'm working on avoids this by caching the errors and serving them for several hours...

methodical · 2 years ago
They should have called it upgraded performance. The 404 page loads blisteringly quick!
CodeCompost · 2 years ago
Degraded in relation to 99.90÷ spread across the year.
akdor1154 · 2 years ago
How on earth did you make that typo? I'm honestly intrigued, do you use an ime?
layer8 · 2 years ago
At least it did return a response.
WolfeReader · 2 years ago
I cannot wait for Gitea, Forgejo, and GitLab to start federating with each other via ActivityPub. Then we can all take one more step away from a corporate-controlled internet.
perihelions · 2 years ago
- 'one more step away from a corporate-controlled internet"

Downvote me to grey-world if you like, but I think everyone's crazy to put all their code infrastructure in the hands of fucking Microsoft. Especially literal free open-source software. Who do you think Microsoft is? What do you know of Microsoft's history and their core values (they're "embrace, extinguish & exsanguinate"). It's like giving fucking Sauron safekeeping of your power-rings in Mordor, oh we have great infrastructure for safe ring storage here, very secure, the orcs are really expert guards.

BHSPitMonkey · 2 years ago
What exactly is the risk? That they'll stop providing the services they sell today? The design of git makes switching to another primary remote very easy (granted, most users probably don't have good habits around backing up data from Issues/Wiki/Releases and risk losing that data if it's taken away suddenly -- but the repo itself is durable and portable on a whim.
rsuelzer · 2 years ago
But, I like the stock.
rglullis · 2 years ago
That would surely be nice and helpful, but why do we need to wait for it?

Even my open source projects in github are just mirrors from the "real place" of work: gitlab or my own gitea instance. If github is down, it is a minor inconvenience but I can still work.

asdff · 2 years ago
Or just host your own git repo and cut out the dependency entirely.
dgivney · 2 years ago
I'm a little confused, Gitlab is a publically traded corporation?

https://www.google.com/finance/quote/GTLB:NASDAQ

WolfeReader · 2 years ago
GitLab, Gitea, and Forgejo are applications that can be easily self-hosted. One of those also has a corporation associated with it, but that has minimal effect in this case.
ironmagma · 2 years ago
Does that matter if it federates openly? Anyone can.
Eji1700 · 2 years ago
Once again proving that Monday is "whenever you push to production"
Animats · 2 years ago
Github is becoming too important. A routine backup system is needed, something like archive.org, that backs up important Github projects.
swatcoder · 2 years ago
I know this a greybeard's fantasy and that most people working today were trained not to bother, but: important things should not have GitHub as a failure point.

Hobby projects and today's work? Sure. Point straight at GitHub and hack away. And when it goes down, get yourself a coffee.

But everything that's anywhere near production should have already pointed those github requests to a mirror or other tool in your own controlled ecosystem. The status of GitHub should have nothing to do with whether you're customers are getting what they're paying you for. Same goes for Docker containers and every other kind of remotely distributed dependency.

willsmith72 · 2 years ago
It's not that black and white. Where do you draw the line on what can and can't be a failure point?

My cloud provider is probably an acceptable point. If every AWS region goes down I'm not going to have a spare cloud provider.

What about an auth provider? Do I need a backup there?

What about CI, do I need multiple CI systems?

3rd party search services, realtime messaging services, the list goes on.

For 1% of systems, you need backups for all of these (or to not use anything external). The other 99%, building backups for every one of these systems is a losing business strategy.

Some of them sure, but which those are will vary based on the context. It's not as simple as having a backup for "every other kind of remotely distributed dependency."

daxaxelrod · 2 years ago
Interesting perspective, disagree on one central tenant. Even though Github holds production code, it is not production. Built artifacts and the machines running those artifacts is production. When Github goes down, which it rarely does, it just means developers cant sync for a couple hours, no different than if someone works offline. The temptation to increase internal devops complexity should not be an automatic immune response when a service goes down, it comes with all sorts of hidden costs.
andrewaylett · 2 years ago
It's not entirely clear to me whether you're talking about using GitHub for your own production tooling, or as a source for some arbitraty third party component. If it's the latter, then I completely agree with you. Use a read-through proxying package repository. I don't care if you run it yourself or if you pay a provider, but don't pull stuff from the origin every time you build.

In the general case, adoping an external system will bring with it greater reliability than trying to run stuff oneself. The differences are that you don't get to choose your maintenance windows, and you can't do anything to fix it yourself.

Take care about who you pick, and own the depencency, because you've put a part of your own reputation in the hands of your provider.

Now, if you pick GitHub as a part of your controlled ecosystem -- which is totally reasonable, if it fits your use-case -- then you still shouldn't be pulling arbitrary stuff from places outwith your control. GitHub has package repository tooling that you can use :). Although it's not entirely clear to me that it's as suitable for third-party dependencies as tools like Artifactory or Nexus.

hipadev23 · 2 years ago
That exists on every dev’s machine.
Animats · 2 years ago
In theory, the code is there, but putting a project back together in a hurry after trouble at Github is non-trivial. Especially if the build process uses proprietary stuff such as "GitHub actions". The issues and discussions are all Github-only, too.
bogota · 2 years ago
Host a backup of your own code? It’s easy and can be done on a rpi. I wrote a go program in 1000 lines that automatically does this for me. And then I actually started using that as the main source and pushing the backup to github.

It also pulls down anything i star into a different folder which get a sync one a day. The rest get a sync every hour.

369548684892826 · 2 years ago
There's also a tool called myrepos[0] that can be cronjobbed to keep local repos up to date

0: https://myrepos.branchable.com/

antoineMoPa · 2 years ago
If only there was a decentralized system to allow version management of code.
dleslie · 2 years ago
Sadly, GitHub doesn't store its value-add assets within the repository itself; so all of the PR conversations, Gists, Issues, and so forth aren't within git itself.
__MatrixMan__ · 2 years ago
I realize you're talking about git, but I think it's also pretty important to have a place where users can submit issues and dev's can say "it's fixed in version XYZ", these are not features of git.

I've been meaning to give radicle a try.

fragmede · 2 years ago
You'd still need some sort of centralized... hub to communicate and manage things.
lmarcos · 2 years ago
In the context of companies: if you don't have your own cache of github,npm, etc., you're doing it wrong.
ajsfoux234 · 2 years ago
https://archive.softwareheritage.org/browse/ has some repositories archived.
shepherdjerred · 2 years ago
I assume that there are some archives of public GitHub repositories somewhere.

Dead Comment

paulddraper · 2 years ago
My hard drive.
rglullis · 2 years ago
I need a macro template for those memes with Bart Simpson on the blackboard and make it say "I will always have a backup plan for third-party services".

Seriously people: gitea exists. Gitlab self-hosted exists. Drone/Woodpecker CI exists. It's not that difficult to set up a project that does not depend on Github. I spent less time setting these up than the amount of down time that Github has had this year.

waihtis · 2 years ago
Problem is, this is one of 50 things that can go wrong for which you should have a robust backup plan for
asdff · 2 years ago
It's amazing how many of these issues can be obviated by taking a step back at a given SAAS, asking can I self host this, then if you can and need redundancy to just buy two desktops and stick one in your place and another in your friends apartment in another town. With a lightweight static site, a modern desktop is probably more than powerful enough to deal with most any load you might realistically see for your given project. You are also extremely unlikely to have both of these desktops go down at once if they are on different local power grids and internet service providers, short of invasion of the continental U.S. perhaps.
layer8 · 2 years ago
If you’re using 50 things requiring separate backup plans, and you aren’t large enough that it’s also no problem to organize backup plans for those 50 things, you’re doing something wrong, I’d say.
verve_rat · 2 years ago
Or, just take the rest of the day off? The world will still be there after the weekend.
rglullis · 2 years ago
yesterday I couldn't set up a bunch of servers that I needed provisioned and cloudflare's API had an outage.

Today if I were using github, my day would be wasted again.

For all the talk about companies trying to cut out on meetings by putting a sticker price to it (this 30 minute meeting could've been an email and cost $2000), at what point do we start saying "this outage could've been avoided and cost us $5k"?

andrelaszlo · 2 years ago

    ⮕ git push
    ERROR: Repository not found.
    fatal: Could not read from remote repository.
    
    Please make sure you have the correct access rights
    and the repository exists.
That scared me for a minute. ':D

misnome · 2 years ago
Yeah, retriggered our deploy pipeline a few times, checked downdetector and no reports, logged in to machine and verified external connection.

Sighed, assumed I was on the first wave of an incident on top of their current slack incident, then logged off for the day.

methodical · 2 years ago
PR really spun gold when they decided to label everything from every single one of their databases getting deleted and backups nuked to intermittent connectivity issues as "degraded". Who exactly are they making feel better by not calling a spade a spade?
brookst · 2 years ago
People will take issue with any wording. Why try to find the exact right words each time when people will have exactly the same complaints?
methodical · 2 years ago
I take issue with clear PR-speak trying to make the issue lesser than it actually is. When you're having an outage- call it an outage. Having a feature completely unusable and labeling it as "degraded performance" is clearly twisting your words to lessen the outwardly perception of the scale of the problem.
skeaker · 2 years ago
Shareholders, generally.