alexflint (u/alexflint)

alexflint commented on Httptap: View HTTP/HTTPS requests made by any Linux program github.com/monasticacadem... · Posted by u/alexflint

jumploops · 7 months ago

This is clever! I've been playing around with netns and TUN devices lately for a work project, and this idea is just so simple and clean.

alexflint · 7 months ago

Thanks! Yeah linux network namespaces are a powerhouse that we're only just starting to fully utilize (outside of containerization).

alexflint commented on Httptap: View HTTP/HTTPS requests made by any Linux program github.com/monasticacadem... · Posted by u/alexflint

mhils · 7 months ago

This is really cool, thank you for sharing! We've built a similar feature for mitmproxy lately, but with different tradeoffs. Our approach does require root and we don't have automated certificate install (yet), but we don't require apps to run in a dedicated namespace (so you can capture already-running processes). Super awesome to see this now, excited to dive into the code and see how you do TCP reassembly etc. :)

alexflint · 7 months ago

Thank you! mitmproxy is fantastic - thanks for all the work that's gone into that project. Maybe we can get in touch and chat about all this stuff.

alexflint commented on Httptap: View HTTP/HTTPS requests made by any Linux program github.com/monasticacadem... · Posted by u/alexflint

concerndc1tizen · 7 months ago

Which privileges are required? CAP_NET_ADMIN? Or nothing at all?

alexflint · 7 months ago

Nothing at all!

You do need write access to /dev/net/tun. This is standard for all users for the distros that I've looked into, but it is ultimately a distro-specific thing.

alexflint commented on Httptap: View HTTP/HTTPS requests made by any Linux program github.com/monasticacadem... · Posted by u/alexflint

ranger_danger · 7 months ago

Why not use eBPF instead? Then you could see all http requests from all processes at once, including ones that are already running. Plus you wouldn't need to bother with TLS at all, just hook on e.g. write(2).

alexflint · 7 months ago

Unfortunately TLS happens inside the the application, not in the kernel, so using eBPF to hook syscalls to write won't help with TLS decryption.

alexflint commented on Httptap: View HTTP/HTTPS requests made by any Linux program github.com/monasticacadem... · Posted by u/alexflint

abracadaniel · 7 months ago

I see it supports .har, but .warc support would be amazing. It's the iso standard for web archives, which would give you the ability to replay archived websites via multiple tools.

alexflint · 7 months ago

Fascinating, I hadn't heard of WARC - thanks for the pointer and I'll consider it.

alexflint commented on Httptap: View HTTP/HTTPS requests made by any Linux program github.com/monasticacadem... · Posted by u/alexflint

2030ai · 7 months ago

I sadly assumed the first countryside photo was generated but I assume now it is real!

The mix of tech and meditation would appeal to me. Maybe the idea does (actually doing it is probably hard!).

It seems like a "Buddhist Recurse"

alexflint · 7 months ago

Yeah that photo is real! That's where I live!

Yes, it's true, actually doing it is hard, but to be honest not as hard as a lot of other stuff (getting a phd for example, or goodness gracious buying a house in San Francisco). I love getting up early. I love living out in nature. I love chanting and eating meals together and making a version of Buddhism for AI systems!

If you're interested in what it's like, we have written a bunch of very short few-paragraph stories about our time at MAPLE here: https://tales.monasticacademy.org/

alexflint commented on Httptap: View HTTP/HTTPS requests made by any Linux program github.com/monasticacadem... · Posted by u/alexflint

wutwutwat · 7 months ago

Did everyone forget about wireshark, which can totally be ran as non-root?

https://blog.wireshark.org/2010/02/running-wireshark-as-you/

alexflint · 7 months ago

Wireshark is awesome but yeah as others mentioned it's the TLS decryption piece that is difficult in that workflow

alexflint commented on Httptap: View HTTP/HTTPS requests made by any Linux program github.com/monasticacadem... · Posted by u/alexflint

jshier · 7 months ago

No network namespaces, but the various Network Extension APIs might be able to do this, though it's difficult. RocketSim (I'm unaffiliated, just an example) recently added a simulator-specific network throttle (to replace the system-wide Network Link Conditioner Apple ships) using a content filter extension. Even though this is a system-wide API, it seems you can limit its impact to a single app. And it seemed to properly compose with Proxyman and Cloudflare's Warp VPN at the same time, so perhaps it could be a general solution.

alexflint · 7 months ago

Woah, this is super helpful info. Thanks. That sounds like a real possibility for a macOS port actually.

alexflint commented on Httptap: View HTTP/HTTPS requests made by any Linux program github.com/monasticacadem... · Posted by u/alexflint

wslh · 7 months ago

That's a great DX! I wonder if an alternative way is to just hook functions like read, and write, and other functions linked to the SSL libs. It is true that you should be aware of the SSL libs in place but OpenSSL is the most popular.

alexflint · 7 months ago

This is a very interesting possibility actually. Not 100% sure what the implication would be but maybe there is something here.

alexflint commented on Httptap: View HTTP/HTTPS requests made by any Linux program github.com/monasticacadem... · Posted by u/alexflint

q2dg · 7 months ago

Mitmproxy v11.1 can do a similar thing

alexflint · 7 months ago

Yeah mitmproxy is great. The main difference with httptap is that it's an HTTP proxy server, so you have to configure your program to use a proxy server. When I wrote httptap I wanted to be able to run `httptap <command>` and see the httptraces right there in standard output. There is an absolute ton of cool things that mitmproxy can do that httptap is not even close to, like interactively modifying HTTP requests and such. Very cool project.

u/alexflint

KarmaCake day483October 5, 2013View Original