A couple of fun behind the scenes stories which I think at this point are past the statute of limitations.
Source: I was the Service Engineer on pipes for around a year or so.
1) When you went to the pipes landing page, there were a few demo pipes to show people what was possible. One of them combined search results from say ebay/craiglist/amazon to show prices for things.
One day I was looking through the source of these, and noticed there were affiliate ids in the ebay/amazon links. They all belonged to some early team member who had long since left.
I showed a couple people on the team, we all guessed at how much they were making from this, said good on em and went about our day. I still wonder how much they ended up making from it.
2) It was my first on-call, and all of a sudden the west coast pipes cluster just went bananas. After ~5 minutes, east coast started to go nuts and the west coast subsided.
Someone, despite numerous defensive measures, had found a way to create a pipe-bomb that would recursively call multiple versions of itself. Once the west coast load balancer failed over, one of these requests would hit the east coast and the 'virus' would jump over there. This flipped flopped back and forth until I figured out how to blacklist pipe ids (and eventually got a code fix).
As a fresh Yahoo! Developer Network employee I was the person who blabbed about the top-secret _callback parameter the weekend Pipes launched, which caused the service to be taken offline for capacity upgrade.
I had an interesting series of voicemails, ranging from "dude, WTF?" to "okay, we need to talk to you right away" to "we're pretty sure you're fired" but ending with "hey, Tim O'Reilly picked it up and ALL IS FORGIVEN THANK YOU!!!"
And now, Pipes can be mobilized and viewed on any mobile phone using SMS and WAP! Simply get the RSS feed of the Pipe and mobilize it using the 411Sync Mobile API. Within seconds, view it on your cell phone
Oh wow. There is so much in this paragraph that was best left forgotten.
At that point in history APIs that returned a JSON object wrapped in a callback were few and far between (and Yahoo! had almost all of them). Pipes would give you anything you wanted in a callback, allowing for nothing-but-front-end mashups of anything that would hold still while being scraped.
Craigslist banned Pipes and un-banned it after YDN employee Jeremy Zawodny went to work there:
I used Yahoo! Pipes in a somewhat greyhat way. I started pulling in data from Reddit's RSS feeds for certain subreddits and using that to create and post content for fairly spammy Twitter, Tumblr and Facebook pages. I'd parse the titles (for example, change things like "cake day" to "birthday") and adding hashtags at the end.
You could ensure only top content made it by setting a minimum karma limit. It was a way of producing content 24/7.
I only did a couple of trial pages to test out the concept -- I never made any money from it nor had any intention to monetize -- but I'm almost completely certain that it was a method used by dozens of blackhats to make autoblogs for the purposes of ad revenue. You could even fake a comment history or have a "unique" title by selecting the top comments from the Reddit posts. Once you have one recipe down, you can just copy it for other subreddits. And there was an endless amount of hyper-focused niche subreddits that you can instantly plug into.
I used to use pipes for a network of real estate listing websites. I would combine Craigslist, Ooodle, Trulia and some others and then place Adsense ads around the feeds.
My sites were getting a lot og organic search traffic, but were ultimately dropped for not having unique content
I was at the O'Reilly Emerging Tech conference session that functioned as (what I remember to have been) the Pipes launch party. Everyone involved with the project seemed young and full of optimism, and while they didn't know exactly how Pipes would make money, I remember being as impressed with the UI as I was the forward-thinking ideas that motivated the project. Keep in mind that in 2006, IE 6 was still omnipresent so the fact that they had a usable drag and drop interface running put it in rarified air along with other UX trailblazers like Flickr and DabbleDB.
To this day, I am still bewildered and not just a little angry about how Yahoo's entire M&A machine was really just a catch-and-kill trap for taking the most promising ideas out of the ecosystem and slowly resource starving them into irrelevance.
I remember that ETech. Back when Yahoo was doing a bunch of things that were cool and not profit-driven, like FireEagle.
In a lot of ways that tech was all ahead of its time. It's a shame the monetization schemes weren't similarly evolved. If it were, Yahoo may still be more relevant today.
I met one of the FireEagle devs there, at the same ETech event, and he was a really great person. We proceeded to bump into each other a few more times over the years, specifically at the Ajax Experience conferences. He really tried to convince me that Yahoo wasn't all that bad, although he eventually did have to move on.
As evidenced by the previous paragraph, I've forgotten his name which makes it exceptionally difficult to reconnect.
Consider this a HN "missed connections" attempt. ;)
It feels like Zapier and IFTTT are the successors to Pipes, but I never quite got the same feeling from them (maybe I haven't checked Zapier in a while). But Pipes was ahead of its time and I'm still not sure of anything that really comes close to what I could do with pipes even back then. Thank you.
Integromat [0] somewhat approximates Pipes. Haven't heard as much about it as IFTTT and Zapier. I found it because they've got one of the few remaining active API integrations with RunKeeper, which since closed the door to new application registrations.
Node-RED seems to be the best alternative which is successful and good enough these days. It's selfhosted, and it seems combining it with home assistent seems also very popular to get the best of it.
Sorry, one last note. I was a broke college student at the time. I made tens of thousands of dollars with Pipes doing ethical arbitrage. I still miss it, where's our Kickstarter?
Was Caterina Fake involved in any way with Yahoo Pipes? I believe she headed Yahoo Brickhouse at the time, but the timelines are foggy for me. Pipes is so unlike any Yahoo product.
Pipes preceded Brickhouse and Caterina Fake's involvement. Brickhouse was the first step in encrusting the Pipes project in enough layers of management to ensure it would die.
Yahoo pipes was really fun and great, and I was happy that someone besides Google was still innovating in an exciting way. I remember being very sad when Yahoo Pipes was finally unplugged. That day I realised that the web is not about users, not about building fun and useful things, not about building lego-like structures, it's just filter bubbles, echo chambers and troll farms.
Hey Dan, Feedity's founder here. Thank you for using our feeds service! Just so you know, besides HTML webpages you can also create feeds for dynamic webpages with client-scripts, as well as social media profiles.
Oh boy do I remember yahoo pipes. I used it heavily for some time. I always suspected their servers were under massive load running all their jobs so sometimes things were slow.
Pipes made you think the future of the web was going to be technical and brilliant.
Readit News
Source: I was the Service Engineer on pipes for around a year or so.
1) When you went to the pipes landing page, there were a few demo pipes to show people what was possible. One of them combined search results from say ebay/craiglist/amazon to show prices for things.
One day I was looking through the source of these, and noticed there were affiliate ids in the ebay/amazon links. They all belonged to some early team member who had long since left.
I showed a couple people on the team, we all guessed at how much they were making from this, said good on em and went about our day. I still wonder how much they ended up making from it.
2) It was my first on-call, and all of a sudden the west coast pipes cluster just went bananas. After ~5 minutes, east coast started to go nuts and the west coast subsided.
Someone, despite numerous defensive measures, had found a way to create a pipe-bomb that would recursively call multiple versions of itself. Once the west coast load balancer failed over, one of these requests would hit the east coast and the 'virus' would jump over there. This flipped flopped back and forth until I figured out how to blacklist pipe ids (and eventually got a code fix).
Deleted Comment
I had an interesting series of voicemails, ranging from "dude, WTF?" to "okay, we need to talk to you right away" to "we're pretty sure you're fired" but ending with "hey, Tim O'Reilly picked it up and ALL IS FORGIVEN THANK YOU!!!"
https://web.archive.org/web/20070302201929/http://developer....
Oh wow. There is so much in this paragraph that was best left forgotten.
If you look at some janky JS code[1] I had on my website at the time, you can see what you can do with it.
That would basically source
://pipes.yahoo.com/pipes/pipe.run?_id=7CTRtbtL3BGX_AHbjknRlg&_render=json&_callback=load_daily_show
as a static asset in my page, which would load the JS and call load_daily_show(<json>) as a result.
Now, EVERYONE who hits my page is invoking the pipe as a backend API call, with no caching and unfortunately with the entire Y/T cookies intact.
[1] - https://web.archive.org/web/20081007043923/http://t3.dotgnu....
Craigslist banned Pipes and un-banned it after YDN employee Jeremy Zawodny went to work there:
https://techcrunch.com/2009/12/01/craigslist-yahoo-pipes-fli...https://techcrunch.com/2009/12/16/craigslist-yahoo-pipes/
You could ensure only top content made it by setting a minimum karma limit. It was a way of producing content 24/7.
I only did a couple of trial pages to test out the concept -- I never made any money from it nor had any intention to monetize -- but I'm almost completely certain that it was a method used by dozens of blackhats to make autoblogs for the purposes of ad revenue. You could even fake a comment history or have a "unique" title by selecting the top comments from the Reddit posts. Once you have one recipe down, you can just copy it for other subreddits. And there was an endless amount of hyper-focused niche subreddits that you can instantly plug into.
Fun times.
My sites were getting a lot og organic search traffic, but were ultimately dropped for not having unique content
To this day, I am still bewildered and not just a little angry about how Yahoo's entire M&A machine was really just a catch-and-kill trap for taking the most promising ideas out of the ecosystem and slowly resource starving them into irrelevance.
In a lot of ways that tech was all ahead of its time. It's a shame the monetization schemes weren't similarly evolved. If it were, Yahoo may still be more relevant today.
As evidenced by the previous paragraph, I've forgotten his name which makes it exceptionally difficult to reconnect.
Consider this a HN "missed connections" attempt. ;)
And Meebo! They build an entire windowing system in the browser...and supported IE6.
[0] https://www.integromat.com/
I have now found that https://feedity.com/ does a good enough job scraping pages to produce feeds. I then run that through https://fivefilters.org/content-only/ to fetch the full content of the page and read it in https://www.inoreader.com/, my favorite Google Reader clone.
Some people are amazed that I am always on top of new content they produce.
Pipes made you think the future of the web was going to be technical and brilliant.
Oh well, that was at least half right.