baron_harkonnen (u/baron_harkonnen)

baron_harkonnen commented on Ask HN: How can I get a tech job that's more social? · Posted by u/98Windows

sophacles · 4 years ago

What's tone deaf and patronizing about your responses is that you assume that your personal working style is the best and only one for every person. Imagine not understanding that different people are different, and as a result may have working styles that are different too.

baron_harkonnen · 4 years ago

Where are you getting that I'm assuming that "personal working style is the best and only one for every person"? I am recommending what worked for me. It's a bit different than the question the OP ask's but, given a similar situation, it's what has worked.

I suspected you my be projecting a bit of your own insecurities here.

Unfortunately HN has become an increasing hostile place, and I think rather extreme reactionary response to a simple recommendation is becoming the norm.

In a weird way, thanks for finally waking me up to the realization that is community is not what it was and not one where I should be spending time anymore.

baron_harkonnen commented on Ask HN: How can I get a tech job that's more social? · Posted by u/98Windows

LindyTalker · 4 years ago

Tone deaf. There's still ~6 hours a day of just sitting in a room alone. Glad you've found success in life but this might not be for others

baron_harkonnen · 4 years ago

Call it what you want but that advice comes from experience. I worked remote for many years before pandemic and had plenty of issues with isolation.

Sometimes in life socialization comes easy and requires no work, but that's not always the case. This is a common problem people face once they leave high school/college. The truth is that real social relationships, the ones that keeps us more fulfilled take work.

I realized that to not feel lonely all the time when working at home I had to put some effort into making sure I reached out to friends, scheduled lunches and calls (which can happen during that 6 hour period, that's one of the benefits of WFH), make sure I went to local meetups etc. The end result was that from that effort I made some fantastic friendships that were much more fulfilling.

Even in the OP's current work environment a little more deliberate effort can go a long a way. I'm sure other coworkers feel the same way, if they pair programmed in the office why not reach out and schedule some pairing sessions? I've know many remote places that do this. Schedule virtual coffees during the day where two people can just chat, again something I've seen at many remote places.

Yes, for decades people could get a basic level of socialization for no effort, but like most no effort things that socialization goes as easy as it comes.

Having been in a similar situation to OP, that was the lesson I learned so that's the advice I'm giving, which I hardly think is tone deaf.

baron_harkonnen commented on Ask HN: How can I get a tech job that's more social? · Posted by u/98Windows

baron_harkonnen · 4 years ago

I would recommend looking into decoupling your social needs from your job.

Even in the time before wide spread WFH this was an issue, during good times it feels great to go into work and get all your social needs met but when layoffs come or companies collapse suddenly that great work friend everyone loves gets let go and in a few more weeks they effectively don't exist anymore.

It's great if you can meet people at work and create a real friendship (I've certainly done that). Now that you're remote you can put more time into keeping up with those people. Schedule lunches, video calls etc.

Get to know your neighbors better, join local interest groups, schedule video chats with friends you haven't chatted with in a while that live far away, and make sure to get lunch with local friends whenever you can.

It will be a bit of transition but ultimately you'll have a much richer social life and honestly enjoy work more as well since you have more outlets in the day that have nothing to do with your 9-5.

baron_harkonnen commented on The planet survived six hours without Facebook. Let's make it longer next time theregister.com/2021/10/1... · Posted by u/samizdis

AndrewDucker · 4 years ago

It's a big deal when Facebook goes down. This is why: https://www.stonekettle.com/2021/10/recap-october-4-2021.htm...

People want to talk to their friends. They want to keep up with what their friends are up to. Sometimes they want to do that in ways that you, specifically, aren't a fan of, but that's fine - different people are different. But they want a place they can go and see "Oh, Dave had a kid. Oh, Agnes got a promotion. Oh, Cameron went to that new pizza place and it looks really tasty!"

This means that either we design something decentralised that's as easy as Facebook to get onto use, or we're going to just end up replacing Facebook with something with essentially all of the same characteristics as Facebook.

It's that or regulate it. In many many different ways across many different countries. That won't be much fun either!

baron_harkonnen · 4 years ago

A variation of this comment shows up in every FB related post, and it seems completely absurd to me that anyone holds this view seriously.

The idea that interacting on Facebook is somehow "keeping up with friends" is itself perhaps the greatest victory of Facebook's own marketing. The idea that 'liking' a picture and leaving a public comment is a sincere human interaction is ridiculous. It's social noise used to replace interpersonal connections and is being pitched that this is somehow better.

If FB suddenly disappeared people would likely go back to texting each other images and calling each other more often, in private where they could more openly admit their struggles and non-public views. People would stop pretending that they had more than 30 "friends". Daily likes and emojis would be replaced with actual video calls, less frequent but longer, more intimate conversations.

This isn't speculation as it's how everyone I know who isn't active on facebook communicates. When I call my distant friends and family the interactions are entirely different than the public facing, image maintaining, completely non-intimate communication that happens on any "social" media.

You're not keeping up with what's happening when you interaction on facebook, because what's really happening is struggles and concerns that you don't necessarily want to share in public with everyone you know. Real human connections involve being vulnerable around someone you trust, which is fundamentally in opposition to the foundations of how something like facebook works.

baron_harkonnen commented on Neural Networks from Scratch aegeorge42.github.io/... · Posted by u/bane

medo-bear · 4 years ago

i am not sure what you are disagreeing with. chain rule is basic calculus that precedes understanding hessians. my argument is, if you can not understand what the chain rule is, you will not understand more complicated mathematics in ML. do you think i am wrong ?

EDIT: also uncertainty estimation is the stuff of probabalistic approach to ML. i would say that people who do probabalistic ML are quite mathematically capable (at least to my experience)

baron_harkonnen · 4 years ago

> chain rule is basic calculus that precedes understanding hessians.

It doesn't have to be that way. The hessian is an abstract idea and the chain rule and more specifically backpropagation are methods of computing the results for an abstract idea. When I want the hessian I want a matrix of second order partial derivatives, I'm not interested in how those are computed.

For a more concrete example, would you say that using the quantile function for the normal distribution requires you to be able to implement it from scratch?

There are many, very smart, very knowledgeable people that correctly use the normal quantile function (inverse CDF) every day for essential quantitative computation that have absolutely no idea how to implement the inverse error function (an essential part of the normal quantile). Would you say that you don't really know statistics if you can't do this? That a beginner must understand the implementation details of the inverse error function before making any claims about normal quantiles? I myself would absolutely need to pull up a copy of Numerical Recipes to do this. It would be, in my opinion, ludicrous to say that anyone wanting to write statistical code should understand and be able to implement the normal quantile function. Maybe in 1970 that was true, but we have software to abstract that out for us.

The same is becoming true of backprop. I can simply call jax.grad on my implementation of loss of the forward pass of the NN I'm interested in and get the gradient of that function, the same way I can call scipy.stats.norm.ppf to get that quantile for a normal. All that is important is that you understand what the quantile function of the normal distribution means for you to use it correctly, and again I suspect there are many practicing statisticians that don't know how to implement this.

And to give you a bit of context, my view on this has developed from working with many people who can pass a calculus exam and perform the necessarily steps to compute a derivative, but yet have almost no intuition about what a derivative means and how to use it and reason about it. Calculus historically focused on computation over intuition because that was what was needed to do practical work with calculus. Today the computation can take second place to the intuition because we have powerful tools that can take care of all the computation for you.

baron_harkonnen commented on Neural Networks from Scratch aegeorge42.github.io/... · Posted by u/bane

medo-bear · 4 years ago

if you don't understand chain rule then you dont understand backprop, which means you do not really understand how deep learning works. at most you can follow recipes cook book style. it is kind of how one can make a website without a deep understanding of networking

baron_harkonnen · 4 years ago

> at most you can follow recipes cook book style.

Here I disagree with you pretty strongly. Once someone is comfortable with differentiable programming it's much more obvious how to build and optimize any type of model.

People should be more concerned about when to use derivatives, gradients, hessians, Laplace approximation etc rather than worry about the implementation details of these tools.

Abstraction can also aid depth of understanding. I know plenty of people who can implement backprop, but then don't understand how to estimate parameter uncertainty from the Hessian. The latter is much more important for general model building.

baron_harkonnen commented on Neural Networks from Scratch aegeorge42.github.io/... · Posted by u/bane

medo-bear · 4 years ago

Backpropagation is a particular implementation of reverse mode auto-differentiation, and it is the basis for all implementaions of DL models. It is very strange for me to read this as though it is very obvious and commonly accepted fact, which I don't think it is.

baron_harkonnen · 4 years ago

> to read this as though it is very obvious and commonly accepted fact

I'm not entirely sure what you're referring to by "this" but assuming you mean my comment, I think what I'm saying is very much up for debate and not an "obvious and commonly accepted fact". Karpathy has a very reasonably argument that directly disagrees with what I'm suggesting [0]. Of course he also agrees that in practice nobody will every use backprop directly.

Whether it's JAX, TF, PyTorch, etc the chain rule will be applied for you. I'm arguing that I think it's helpful to not have to worry about the details of how your derivative is being computed, and rather build an intuition about using derivatives as an abstraction. To be fair I think Karpathy is correct for people who are going to be learning to explicitly be experts in Neural Networks.

My point is more that given how powerful our tools today are for computing derivatives (I think JAX/Autograd have improved since Karpathy wrote that article), it's better to teach programmers to learn think of derivatives, gradients, hessians etc as high level abstractions. Worrying less about how to compute them and more about how to use them. In this way thinking about modeling doesn't need to be restricted to strictly NNs, but rather use NNs and example and then demonstrate to the student that they are free to build any model by defining how the model predicts, scoring the prediction and using the tools of calculus to answer other common questions you might have.

edit: a good analogy is logic programming and backtracking/unification. The entire point of logic programming is to abstract away backtracking. Sure experts in Prolog do need to understand backtracking, but it's more helpful to get beginners understanding how Prolog behaves than understand the details of backtracking.

[0] https://karpathy.medium.com/yes-you-should-understand-backpr...

baron_harkonnen commented on Neural Networks from Scratch aegeorge42.github.io/... · Posted by u/bane

minihat · 4 years ago

Each time I teach neural nets to an engineer, there's only a 50% chance they can write down the chain rule. Colah's blog on backprop used to be my favorite resource to leave them with (https://colah.github.io/posts/2015-08-Backprop).

The explanation of the calculus in this tool is equally fantastic. And the art is very cute.

There are many ways to skin a cat, of course, but this is as good a tutorial as I've seen for getting you through backprop as fast as possible.

baron_harkonnen · 4 years ago

Given the current state of automatic differentiation I'm not so sure it's even necessary or particularly useful to focus on backpropagation any more.

While backprop has major historic significance, in the end it's essentially just a pure calculation which no longer needs to be done by hand.

Don't get me wrong, I still believe that understanding the gradient is hugely important, and conceptually it will always be essential to understand that one is optimizing a neural network by taking the derivative of the loss function, but backprop is not necessary nor is it particularly useful for modern neural networks (nobody is computing gradients by hand for transformers).

IMHO a better approach is to focus on a tool like JAX where taking a derivative is abstracted away cleanly enough, but at the same time you remain fully aware of all the calculus that is being done.

Especially for programmers, it's better to look at Neural Networks as just a specific application of Differentiable Programing. This makes them both easier to understand and also enables the learner to open a much broader class of problems they can solve with the same tools.

baron_harkonnen commented on Norm Macdonald's story about the first time he met Bob Dylan twitter.com/smalwigwamlig... · Posted by u/apophatic

baron_harkonnen · 4 years ago

I'm a bit surprised that nobody has mentioned how much we have failed as technologists that a story is being shared by png images of text... brave new world indeed.

baron_harkonnen commented on “Kids who grew up on smartphones do not know how to do anything on a computer” reddit.com/r/apple/commen... · Posted by u/kossTKR

slindsey · 4 years ago

Is this any different from any new tech? The first car owners had to know a lot about taking care of that car. When I was young, you were expected to know how to do many regular maintenance tasks on your car and fix the simple things. Today most people couldn't do much of anything on a new car. Same trajectory.

baron_harkonnen · 4 years ago

Just because this is an older problem doesn't mean it's not a problem.

Yes, people that loved fixing up cars would have complained about "kids these days" not knowing how to fix a car. I was certainly one of those kids.

However seeing this happen with a field I know makes me realize those people complaining about people not knowing how to take care of their cars were right.

And while this is true of "any new tech", it's a relatively recent phenomenon as tech has only been advancing this rapidly in last 100 years or so.

It is a bad thing to be increasingly alienated from the tools you rely on for your day-to-day existence. Of course it's in part because cars and computers have both been increasingly designed by their manufactures to be difficult to take apart and understand. Tools that you don't understand increasingly control you rather than the other way around and I personally think this is a trend worth resisting.

It's good to point out that this is an issue and even better to encourage more people to be curious, and learn how things work. Another example that I'm surprised of is people's homes. I know a shocking number of home owners that cannot fix a single thing in their own home without calling a "professional" (and it's increasingly surprising how many professionals also don't understand what they're doing!) This was extremely evident during the freezing temperatures in Texas where many people didn't know how to shut off the water to their homes.

One of the best parts of the "hacker" mentality is to encourage people to not be scared of their tools and the things they own. While things have gotten more complicated, you can do a large amount of repair and modification on your own for almost everything. It honestly feels very liberating (not to mention saving you a lot of money) to snake a clog 12 ft deep in your drain, replace your car's serpentine belt, restore old hardwood floors, repair broken refrigerators from parts, etc.