Readit News logoReadit News
Posted by u/rafiki6 3 years ago
Ask HN: What is the most impactful thing you've built?
I'll start. For me, I think the most impactful thing I've ever built was an internal application for a FX trading desk that eventually went on to handle billions in daily trades.

It didn't use any fancy frameworks, just plain old CRUD on Java.

jiggawatts · 3 years ago
During a centralisation of public school local servers to a data centre, I created a consolidated library enquiry system. It served over 2,000 libraries, had 330 million titles, and had about a million users. It was efficient enough to run off my laptop, if need be.

AFAIK it was one of the top five biggest library systems in the world at the time.

I was asked to add some features that would have been too difficult in the old distributed system. Things like reading competitions, recommended reading lists by age, etc…

I watched the effect of these changes — which took me mere days of effort to implement — and the combined result was that students read about a million additional books they would not have otherwise.

I’ve had a far greater effect on the literacy of our state than any educator by orders of magnitude and hardly anyone in the department of education even knows my name!

This was the project that made realise how huge the effort-to-effect ratio that can be when computers are involved…

_e4c8 · 3 years ago
Cool story! what languages, frameworks, etc did you use? Or are you about to tell me COBOL? :P
jiggawatts · 3 years ago
The legacy back-end system being migrated was Clipper + dBase III on DOS, which is reminiscent of COBOL.

The part I added was built with ASP.NET 2.0 on top of Microsoft SQL Server 2005, and was eventually upgraded to 4.0 and 2008 respectively.

The only magic sauce was the use of SQLCLR to embed a few small snippets of C# code into the SQL Server database engine. This allowed the full-text indexing to be specialised for the high level data partitioning. Without this, searches would have taken up to ten seconds. With this custom search the p90 response time was about 15 milliseconds! I believe PostgreSQL is the only other popular database engine out there that allows this level of fine-tuned custom indexing.

joshxyz · 3 years ago
> hardly anyone in the department of education even knows my name!

it's okay sir, we now know you as jiggawatts

silasb · 3 years ago
... and this is how OCLC was created?
sideshowb · 3 years ago
> had a far greater effect on the literacy of our state than any educator by orders of magnitude

Nice work, but check your ego mate. Seems your growth hacking would have had zero result if those kids couldn't read to start with, so you could share some credit ;-)

dang · 3 years ago
"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."

https://news.ycombinator.com/newsguidelines.html

silasdavis · 3 years ago
Maybe it wasn't meant that way. If they hadn't had been there then somone else would have been. You can be on the crest of a wave and not be responsible for its power.
hsbauauvhabzb · 3 years ago
By that logic, the people who farm the trees that make the books have more impact than anyone before them, unless you want to consider the people that make the tools, or feed the farmers, etc etc.
domlebo70 · 3 years ago
Any more info? This is fascinating.
nicbou · 3 years ago
> This was the project that made realise how huge the effort-to-effect ratio that can be when computers are involved

I love Steve Jobs' metaphor: computers as a bicycle of the mind [0]. Unfortunately, a lot of effort is concentrated on problems that scale to billions of people. There's a lack of attention to problems that would have a big effect for a relatively small number of people. It's a shame, because they're a blast to work on.

[0] https://www.youtube.com/watch?v=L40B08nWoMk

OkayPhysicist · 3 years ago
They really are. I think the most rewarding software I ever wrote was my first paid gig, where I automated lap swim scheduling for my local swim club. Took me maybe an hour, got paid more money than I'd make in two days as a lifeguard, and they were thanking ME for it. Turned out I had saved a volunteer upwards of an hour every week. With a shitty little JavaScript program.
triyambakam · 3 years ago
The first time I heard that metaphor, I thought that he meant it in the way that bicycles are really fun to ride. I agree with both interpretations.
2muchcoffeeman · 3 years ago
I have a bunch of small scale ideas that I want to implement. Not necessarily for profit. Any ideas on how to execute?
egberts1 · 3 years ago
Impactful?

Designed and deployed credit card readers used in gas pumps back in 1979. (Sold to Gasboy)

Wrote a fine tuner to allow communication between satellites (precursor to TDRSS days). Still used to this day.

Failover of IP in ATM switches (VVRP, PXE, secondary DHCP, secondary DNS, secondary LDAP, secondary NFS). While not invented here, it is still used today as this is a Common setup to this day.

Printer drivers for big, big high-speed Xerox printers on BSD. Still used to this day by big, big high-speed printers.

Also, early IDS products (pre-Snort) at line-speed. Sold to Netscreen.

Easy zero-setup of DSL modem before some BellCore decided to complicate things (thus exploding their field deployment budgets; Southwestern Bell/Qwest enjoyed our profitable zero-setup). Sold to Siemens.

1Gps IDS/IPS before selling it to 3Com/Hewlett-Packard Packard.

Now, I'm dabbling in a few startups (JavaScript HIDS, Silent Connections, replacing the systemd-temp).

Impact? It is more about personal pride but its impacts are still being felt today.

tuyguntn · 3 years ago
OMG, Who are you?
ynniv · 3 years ago

  user: egberts1
  created: May 5, 2015
  karma: 1337
chefkiss

Deleted Comment

keepquestioning · 3 years ago
How did you find all these product market fits?

Have you made more than a typical SWE?

egberts1 · 3 years ago
It was actually a wandering hyperactive/ADHD mind that often said "why isn't there one" and follows through doggedly to the very end.

It is one of those traits where a mind clicks and said "this is it and how" and surprisingly gets into the most illusive hyperfocus/high-energy mode (without using any drug).

Slow-path network processing (arguably me) was commercially made in Ascom Timeplex in 1982 and someone else leaked it to Cisco (or ripping AT's patent off). I got that from observing how different river bends (re)connect year-after-year while doing trout fishing trips.

Money-wise, I am disabled, got abled, disabled again in different way, re-enabled, now just coasting with my own ideas: JavaScript Host-Based Intrusion Detection/Protection System, being one of them. And an portable AirPod detector (for home/auto/travel) is another idea. And DNSSEC for within private enterprise is almost done.

Money is not my thing but it does help greatly in the pursuit of my ideals (so many hardwares, so many test equips).

rglover · 3 years ago
kloch · 3 years ago
At work: the CDN for Megaupload. I was also the guy who had to shut it down when the FBI seized it.

Personal non-code project: The first adult LEGO fan conference in 2000. While I got out of that business years ago it has been replicated by dozens of other annual cons around the world. Back then the LEGO group didn't really understand and was very weary of adult fans. Now there's a whole reality tv show about them with LEGO designers as the judges, and LEGO actively supports cons and clubs.

Open source project: A project I released anonymously ~2010. Several github repos (unrelated to me) keep this project alive (the main one has ~600 stars and ~200 forks) and it's apparently used in several commercial products too.

Website: ip4.me/ip6.me serves 3-5M queries per day. I want to find a good non-profit to take this over to keep it ad and javascript free forever.

MivLives · 3 years ago
Thank you for the Lego thing.

My mom got into adult lego when she took apart my child hood lego and reassembled them to resell.

Now we mail each other sets that the other is done with, and it gives us a great opportunity to connect. We're both anxious people and there's something relaxing about just assembling something where everything has a place.

When she found out there's a lego con in my town, she made plans to come visit me so we can go together and I can show her around the city I just moved to.

agotterer · 3 years ago
That’s a wonderful story!

My 3 and 6 year old love lego kits. Historically I found myself sitting with them and helping when they got stuck or directing them when I saw they made a mistake. More recently I decided to pick up my own kit and build along side them. I’m currently working on the Saturn V rocket. It’s been a lot more fun for me and a way to bond with my kids.

robswc · 3 years ago
>Website: ip4.me/ip6.me

>At work: the CDN for Megaupload. I was also the guy who had to shut it down when the FBI seized it.

>adult LEGO fan conference

Wow, what a small world. That's what I love about HN. The people that make things you use are on it :)

I wish I had something nearly as impressive. I just have open source stuff that people use. Nothing recognizable though.

jmt_ · 3 years ago
Please tell me you're legally allowed to talk more about Megaupload and the work you did - sounds like an absolutely amazing blog post, would love to hear as much as you're able to discuss.

Also, I have a project in production at work where a device needs to grab its public IP address. My code has a list of sites that provide that info and I have ip4.me as a fallback in that list, so thank you for building it!

nanidin · 3 years ago
Legoland in my city still requires adults to be accompanied by children to enter. Kind of bizarre.
agotterer · 3 years ago
That doesn’t surprise me. It’s a very child focused park and I’m guessing they want to control the experience and environment as best they can. A bunch of high schoolers running around might change the dynamic.

Deleted Comment

scrollaway · 3 years ago
> I want to find a good non-profit to take this over to keep it ad and javascript free forever.

Maybe worth reaching out to Mozilla. That's the only actual non-profit I can think of who I think would have both the ability and the incentive to keep it online.

ignoramous · 3 years ago
> I think would have both the ability and the incentive to keep it online.

Ability? 5M/day for "what's my ip" is not much, and I'd wager most of us on this site would be able to keep it up and alive just fine. As for incentive... in addition to the Mozilla Foundation, orgs like Calyx, NLNet, Quad9 come to mind.

scrame · 3 years ago
or maybe the internet archive?

Dead Comment

powerpurple · 3 years ago
what a chad
simonw · 3 years ago
Probably this JavaScript function I posted on my blog in 2003 https://simonwillison.net/2003/Mar/25/getElementsBySelector/
cookie_monsta · 3 years ago
Wow. You were the original querySelector. It's funny how you forget that somebody actually sat down and wrote these things into existence at some point. Thanks!
hn_throwaway_99 · 3 years ago
Even more impressive to me is writing things into existence without the benefit of being able to dig in to the underlying browser tech, and only being able to use the public (at the time) DOM APIs like getElementById, etc.
iamwil · 3 years ago
He's understating, perhaps on purpose.

Datasette, Django, and Lanyrd.

hnfong · 3 years ago
To be fair the original question was "most" impactful thing...
s1291 · 3 years ago
Ten years ago I was reading [0] and I remember your name was mentioned somewhere. Here is a quote:

> Locating elements by their class name is a widespread technique popularized by Simon Willison (http://simon.incutio.com) in 2003 and originally written by Andrew Hayward (http://www.mooncalf.me.uk)

[0] Page 91 from "Pro JavaScript Techniques" by John Resig.

simonw · 3 years ago
Yeah here's Andy's getElementsByClassName post (via the Internet Archive): https://web.archive.org/web/20030402172546/http://blog.moonc...
jmt_ · 3 years ago
Hey Simon, thanks for creating Django with Adrian. I was deeply interested in programming from a young age but learning Django in my teens sparked a passion for web development that has yet to feign so many years later! Appreciate all your contributions to this space.
hnfong · 3 years ago
OMG.

My most impactful thing I've done outside of paid work is a website running on Django. I could live without queryBySelector or their descendants, but not without Django.

Thank you, Simon.

yuuu · 3 years ago
wane
swyx · 3 years ago

   /* That revolting regular expression explained 
   /^(\w+)\[(\w+)([=~\|\^\$\*]?)=?"?([^\]"]*)"?\]$/
     \---/  \---/\-------------/    \-------/
       |      |         |               |
       |      |         |           The value
       |      |    ~,|,^,$,* or =
       |   Attribute 
      Tag
   */
all regexes should have ascii art explainers!

(from https://static.simonwillison.net/static/2003/getElementsBySe...)

Beta-7 · 3 years ago
What a coincidence. Just yesterday i've used getElementsBySelector for the first time while making a greasemonkey script.
kmoser · 3 years ago
Wow, 10 years before document.querySelectorAll()!
dmitriid · 3 years ago
> Wow, 10 years before document.querySelectorAll()

querySelectorAll wouldn't ever appear without jQuery which got its idea from Simon's idea.

And even then querySelectorAll was so poorly implemented that it didn't even have any useful helper methods.

Aeolun · 3 years ago
I like seeing this. At the time I remember thinking we needed something like this, and why doesn’t the browser have it already?

Then thinking, I suppose you could do it by (exactly the method you used), but never actually doing it because if it were that simple, someone would have already done it.

Actually, seeing the date, I realize this predates me even leaving high-school, which makes it even more atrocious that I never knew of it!

karaterobot · 3 years ago
IOU 1 beer.
mabbo · 3 years ago
A pipeline approval tool (internal at Amazon) that counts metrics.

I was a fairly fresh college-hire SDE1 at Amazon. And I was annoyed, because I'm lazy. Every time I was oncall, I had to manage the deployment pipeline for my teams software- the UI for the tool used by Pickers inside Amazon Warehouses. On Monday, deploy the latest changes to the China stack (small). On Tuesday, check if anything bad happened, and then deploy to the Japan stack (small-ish). On Wednesday, Europe (big). Thursday, North America (biggest). Repeat each week.

And I thought "why am I doing this? There are APIs for all of this stuff!". So I made an automated workflow that hooked into the pipeline system. You gave a metric to look for, a count of how many times the thing should have happened, and an alarm to monitor. If everything looks good, it approves. I hooked it up for my pipeline, and then it usually finished the entire weekly push before Tuesday afternoon. I made it in about 2 weekends on my own time.

And I left it open for anyone in the company to configure for their own pipelines. A few weeks later I was checking if it was still operating normally and realized there were something like 50 teams using it. Then 100. Then a lot more.

The last I heard, it's considered a best practice for all teams within the company to use it on their pipelines. Before I left in 2021, it was running something like 10,000 approval workflows per day.

I named it after the BBQ/grilling meat thermometer in my kitchen drawer- "RediFork". Given the overlap of "people who read HN" and "devs who worked at Amazon", I probably saved someone reading this an aggregate hour or two of work.

ignoramous · 3 years ago
I had always wondered why it was called "RediFork"... thought it might have been using Redis or something.

Thank you for creating it!

mabbo · 3 years ago
Literally stole it from this: https://www.amazon.com/Maverick-RediFork-Rapid-Matrix-Thermo...

Eg: Stick a fork in it and see if it's done yet

scubbo · 3 years ago
Holy shit. Forget counting the hours (which are easily into the tens, if not hundreds) you saved me and my teams - more importantly, RediFork was a great "on-ramp" for easily introducing observability in existing services without instrumenting anything new, and a great way to demonstrate the power of automation to newbies.

From one "engineer whose irritation at inefficiency spawned a whole tool" to another (I got sick of staying up overnight to run load tests, so wrote myself an automation and monitoring tool - which got picked up, spun off to its own team, and now is used by >300 teams) - thank you!

otikik · 3 years ago
You saved Amazon a lot of wasted man hours. I hope they compensated you well
ornornor · 3 years ago
Hope so too but

> I made it in about 2 weekends on my own time.

femto · 3 years ago
In 1995 I (and a few others) designed and built the first WiFi node [1]. At the time there was only one WiFi unit in the world, and it was the one on our bench. It now has about 20 billion descendants.

[1] https://sci-hub.se/https://doi.org/10.1109/40.566198

comboy · 3 years ago
Awesome. What made it win? Were there any close competitors? Was infrared close to being the winner? I'm also surprised big enough FPGA was already around. Thanks.
femto · 3 years ago
> What made it win?

Initially, ignoring the wisdom of the time that said OFDM was no good for indoor channels. The research project was eventually shut down due to lack of commercial interest, but the research leaders had enough faith to immediately start their own company (Radiata). Later, commercial success for Radiata came from being in the right place at the right time.

> Were there any close competitors?

In the research phase, not that I was aware of. In the commercial phase, Atheros. The story I was told after the event was that Cisco had decided to buy whichever company came to market first. Radiata came to market 2 weeks before Atheros and so Radiata was acquired.

> Was infrared close to being the winner?

It could have been, but specular reflection in IR channels causes inter-symbol interference, which limits the data rate. If someone could have solved that problem then IR might have happened instead of WiFi.

> I'm also surprised big enough FPGA was already around.

At the start of the project FPGAs were not big enough, so we had to partition across multiple 3000 series Xilinx parts. Bigger FPGAs had been released by the end of the project, so the transmitter fitted on a single XC4025 FPGA, using manual placement. The 4025s were brand new and Xilinx (as always) were difficult to deal with, so we had to beg for devices and they magnanimously granted us 3 or 4 chips.

At the time there wasn't much sense of occasion, as we were busy doing the work and none of us knew how big it would get.

anakaine · 3 years ago
CSIRO ftw. Now if only in my part of the world we could get CSIRO scientitists onto the same actual page as scientists from another 'industry' council, and academia, instead of them just paying lip service to each other in the public forum but actively undermining each other behind closed doors.
femto · 3 years ago
Most of the team was Macquarie University.

At the time the collaboration with CSIRO worked quite well, as there were no business development types involved. In 1995 CSIRO was more concerned about science than IP. Since then they have become more money/IP focused. Maybe they got gold fever from the $1 billion in royalties they made from their WiFi patent?

justsocrateasin · 3 years ago
First job out of college, I was at a consulting firm doing software development for DHS (Homeland Security). I got a lot of flack from my friends and family for "working for the devil", but the work was actually objectively good for society - basically there was a big data problem where when an immigrant trying to illegally cross into the US was apprehended, and if they were sick, their custody would be transferred from US Customs and Border Protection (CBP) to Health and Human Services (HHS) so they could receive medical attention. There was zero data transparency between these two orgs, so when that transfer happened it usually caused families to be separated (Sick dad, healthy mom and child, sick dad gets brought in for care and never finds his family again). Since HHS and CBP don't have data communication and everything is siloed, the handoff was really poor and they often wouldn't find each other for months afterwards.

There was a lot of talk about this in the news, and although the software I was working on didn't entirely fix the problem, it allowed the agencies to communicate better. Their data wasn't siloed, and families got separated for only a few days rather than (sometimes) permanently.

I really miss that job. The pay was atrocious and zero WLB, but everyone agreed it was an important problem to solve, and I think the tool we had built really was helping.

616c · 3 years ago
Sadly true and relatable. Thanks for fighting that good fight.
atlasunshrugged · 3 years ago
Wow, that's fantastic work! Hopefully you haven't been turned off of gov work forever, there are now more and more programs to bring in tech talent (e.g. the Presidential Innovation Fellowship, TechCongress, USDS, U.S. Digital Corps) at more reasonable pay scales for impactful roles. May be worth checking out if you want to do it again
schoen · 3 years ago
Let's Encrypt (along with the coauthors of https://abetterinternet.net/documents/letsencryptCCS2019.pdf and many other contributors). Now the world's largest public certificate authority!
jeff_carr · 3 years ago
You win. I'm only the co-founder of Digitalocean. I tip my hat to one of the giants we all stand on the shoulders of.
schoen · 3 years ago
The Let's Encrypt team was a large group effort across three organizations, plus a fourth organization that was spun off to develop and run it, and I in turn tip my hat to all of my colleagues. :-)

(Including Peter Eckersley https://en.wikipedia.org/wiki/Peter_Eckersley_(computer_scie... who passed away earlier this fall at just 43.)

satvikpendem · 3 years ago
And yours is what I run my business on. Thank you!
mysterydip · 3 years ago
I refused to use certificates for my own projects as it was too complicated and expensive, until LE came along. Thank you!
Simon_O_Rourke · 3 years ago
Just to say thank you for Let's Encrypt - it's saved my bacon on at least a dozen times in the past six months alone.
arthuredelstein · 3 years ago
Let's Encrypt is incredible. Thank you!
sagarpatil · 3 years ago
I built a WordPress plugin that helps you to generate free SSL certificate using Lets Encrypt. At it’s peak, it was being actively used by 50,000+ [https://wordpress.org/plugins/ssl-zen/]
Anon4Now · 3 years ago
I think we all owe you, your co-authors, and the sponsors a big thanks.
Rietty · 3 years ago
Thank you so much for this! I use it on my personal site and it was as simple as configuring a few cron jobs!
brianvoe · 3 years ago
You are doing the lords work. Great job!!!