I worked on the censorship and government reporting (sending all logs) infrastructure for Akamai China CDN. I'm glad to see it get shut down. Happy to answer questions.
Previous discussion about it: https://news.ycombinator.com/item?id=33678019
Readit News052c7028e commented on Akamai to shut down its CDN operations in China content.akamai.com/index.... · Posted by u/nunodio
052c7028e commented on Ask HN: What is the thing you've built that you regret the most? · Posted by u/Octabrain
052c7028e commented on Ask HN: What is the thing you've built that you regret the most? · Posted by u/Octabrain
Glad you regret it. Not trying to rub it in as I don't think anything productive will come from self-flagellation, but this is truly awful and I think the US should have laws that make it a crime for any US corporation to participate in this sort of thing.
I was powerless to stop it. I was just a junior engineer, and it was decided by the CEO to do the project. So, actually, I feel I made the right choice -- I participated in the project but worked hard on making sure it was as limited as possible. I successfully advocated for several categories of logs to not be sent because they were not required by law.
So, yes, I regret I couldn't do more, but I don't regret the choices I made with the information I had and the position I was in.
052c7028e commented on Ask HN: What is the thing you've built that you regret the most? · Posted by u/Octabrain
While I wouldn't put any authoritarian moves beyond China's reach, the ICP recordal mechanism already requires government approval.
In that case, isn't it better for user privacy (not that anyone cares about it in China) to receive an ICP recordal but then wait for an actual request from law enforcement to turn over the logs?
Also, while you wouldn't see anyone from Amazon or Cloudflare comment on your thread, both have the ability to stream logs to a destination, and that is also exposed to customers, so I don't think they needed to build anything else.
All of the sites served had an ICP license. This is separate, and the CDNs in China have regulations specific to CDNs they need to comply with.
At the time, Akamai also had the capability to stream logs, but the ministry of technology required a specific, custom interface to receive them, which required engineering work, especially to do it for an entire country without the customers configuring it themselves. I would be extremely surprised if it required no engineering work at Amazon or Cloudflare to deliver the logs in the way they requested.
052c7028e commented on Ask HN: What is the thing you've built that you regret the most? · Posted by u/Octabrain
When I was at Akamai about 5 years ago, I was involved in building the system for making their CDN compliant in China. There were two main features, and they were activated on all servers running inside mainland china (not HK, macau or Taiwan)
1. Logs of the CDN were sent in real time to the ministry of technology -- there was about a 15 minute delay if I remember correctly, and they could impose fines if they were delayed. The log included the url visited, the IP address of the visitor, and a few other things. Perhaps the user agent? I forget.
2. The ministry of technology had a special API to block URLs on the CDN. Basically, they provided a list of URLs that would return a 451, and of course those logs also went to the government.
No other country had this kind of access at the time, but it was considered critical for the business to continue to operate in China. As I understand it, these are required to comply with chinese government regulations, and other CDNs like Cloudflare and Cloudfront have also built similar capabilities. Perhaps jgrahamc can comment on what cloudflare did?
I feel quite guilty about being involved with that project, but the business was set on building it, so I did what I could to limit the blast radius. I would not be surprised if someone got arrested or was killed because of it.
Traceroute is fine for casual analysis but it is inaccurate and/or incomplete for anything serious because it misses routers that use load balancing on packet headers. For research purposes, it's better to use Paris Traceroute: https://paris-traceroute.net
> Why should you use Paris traceroute?
> Because traceroute fails in the presence of routers that employ load balancing on packet header fields. The failures lead to the discovery of inaccurate and incomplete paths, that may mislead operators during problem diagnosis and result in erroneous internet maps. Paris traceroute, by controling packet header contents, obtains a more precise picture of the actual routes that packets follow.
Akamai uses something similar to perform trace routes in-band of HTTP TCP sessions to clients: https://datatracker.ietf.org/meeting/94/materials/slides-94-...
1. How was working with China requests and logging, differing from working with other nation states?
2. Was there full services brought up only for China specific needs? What would they take care or?
3. How would any blocks work? allowlist or denylist? Was takedown immediate, or was it working with the customer/client and getting them to take it down within SLA?
2. There were specific infrastructure changes made for blocking and sending logs inside mainland china.
3. The CDN node would deny access to specific urls uploaded by the Chinese partner company. I don't remember the SLA. The SLA for reporting visited URLs was 15m IIRC