Show HN: Tunnelling TCP through a file

That’s cool. A question that immediately comes to my mind is this: how is this different than using the Windows SMB native support for named pipes over the IPC$ share? From your example it would seem the users on both sides of the share would require the same AAA for your implementation as would be required to use IPC$ via SMB.

If that is indeed the case, (I am not a Windows pr SMB expert, so perhaps you will tell me it is not the case and to be clear, I am not asserting that it is or trying to say anything negative; I am just curious and you seem like a pretty likely source to be able to answer the question, since something motivated you to write this tool)

One difference would seem at first brush is that a named pipe used for this would not need to be periodically truncated in 10 minute intervals to prevent runaway file growth. Again, not knocking what you’ve made at all, I’m genuinely curious for the answer.

mr_mitm · 2 years ago

There are many other protocols besides SMB that can be used to share a file. Webdav, ftp, nfs, two systems accessing a third file share, shared folders in a VM, ... Or a file share provided by RDP like in the example, which does not include the IPC share as far as I know.

ycombiredd · 2 years ago

Hah! You answered my question re how is it different. It’s strange… I re-read your comment twice a with a furrowed brow: “provided by RDP like in the example” and kept thinking to myself “what? There was a screenshot clearly showing…” yada yada SMB. So I went back to the look at the image in the readme. /smh. You know how they say eyewitness testimony is often among the least accurate, even immediately after the incident in question, yet the eyewitness is always so sure, “but I just saw it and…” Wow. Mind playing tricks on me.

Anyway, you’re right, there was no SMB involved in that readme. :-) So strange when you can’t trust your own eyes or short-term memory. Crazy.

adolph · 2 years ago

A classic is dd over nc

https://www.ndchost.com/wiki/server-administration/netcat-ov...

The main trick is to do it over UDP to get speed like Aspera:

https://www.ibm.com/products/aspera

_flux · 2 years ago

I wish unix domain sockets worked over NFS. It would be such a nice way to assign user-level permissions on networked services. ssh /services/gw etc. Maybe even file:/// -urls for the web browser, though the cross domain logic would need updates.

It would presume trusting the permission system over NFS, though.

Neat idea. For better performance, you might want to use native filesystem APIs, for example on windows use CreateFile with the FILE_WRITE_THROUGH flag to prevent caching and and extra flushing.

fiddyschmitt · 2 years ago

Thanks J :) Absolutely! I did try various flags to optimise performance. For CreateFile (which in .NET is wrapped by the FileStream class) I tried FILE_WRITE_THROUGH (which is FileOptions.WriteThrough in .NET), and found it impacted performance quite a lot.

The key to high performance as you rightly pointed out was preventing flushing. In the end, what worked best was reducing the number of writes to disk (which is the bottleneck). I did that by buffering 10-50 ms worth of TCP data, coupled with only flushing explicitly (using a large buffer so that neither BinaryWriter or FileStream flush automatically).

jclarkcom · 2 years ago

I think you can use named pipes if you are using SMB. Here is a example code in python (not tested).

import socket import win32pipe import win32file import sys

BUFFER_SIZE = 4096

def create_pipe(pipe_name): return win32pipe.CreateNamedPipe(pipe_name, win32pipe.PIPE_ACCESS_DUPLEX, win32pipe.PIPE_TYPE_BYTE | win32pipe.PIPE_READMODE_BYTE | win32pipe.PIPE_WAIT, 1, BUFFER_SIZE, BUFFER_SIZE, 0, None)

def open_pipe(pipe_name): return win32file.CreateFile(pipe_name, win32file.GENERIC_READ | win32file.GENERIC_WRITE, 0, None, win32file.OPEN_EXISTING, 0, None)

def proxy_server(machine_name): server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server_socket.bind(('0.0.0.0', 8080)) server_socket.listen(1)

    pipe_name_1 = r'\\' + machine_name + r'\pipe\proxy_pipe_1'
    pipe_name_2 = r'\\' + machine_name + r'\pipe\proxy_pipe_2'

    while True:
        client_socket, _ = server_socket.accept()
        pipe_1, pipe_2 = create_pipe(pipe_name_1), create_pipe(pipe_name_2)
        win32pipe.ConnectNamedPipe(pipe_1, None)
        win32pipe.ConnectNamedPipe(pipe_2, None)

        while True:
            data = client_socket.recv(BUFFER_SIZE)
            if not data:
                break
            win32file.WriteFile(pipe_1, data)
            response = win32file.ReadFile(pipe_2, BUFFER_SIZE)[1]
            client_socket.send(response)

        client_socket.close()
        win32file.CloseHandle(pipe_1)
        win32file.CloseHandle(pipe_2)

def proxy_client(server_name, machine_name): target_host, target_port = 'cnn.com', 80

    pipe_name_1 = r'\\' + server_name + r'\pipe\proxy_pipe_1_' + machine_name
    pipe_name_2 = r'\\' + server_name + r'\pipe\proxy_pipe_2_' + machine_name

    pipe_1, pipe_2 = open_pipe(pipe_name_1), open_pipe(pipe_name_2)
    target_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    target_socket.connect((target_host, target_port))

    while True:
        data = win32file.ReadFile(pipe_1, BUFFER_SIZE)[1]
        if not data:
            break
        target_socket.send(data)
        response = target_socket.recv(BUFFER_SIZE)
        win32file.WriteFile(pipe_2, response)

    target_socket.close()
    win32file.CloseHandle(pipe_1)
    win32file.CloseHandle(pipe_2)

if __name__ == '__main__': if len(sys.argv) < 3: print("Usage: python script.py [server|client] [machine_name] [server_name (for client only)]") sys.exit(1)

    mode, machine_name = sys.argv[1], sys.argv[2]

    if mode == 'server':
        proxy_server(machine_name)
    elif mode == 'client':
        if len(sys.argv) < 4:
            print("Please provide the server name or IP address for the client mode.")
            sys.exit(1)
        server_name = sys.argv[3]
        proxy_client(server_name, machine_name)
    else:
        print("Invalid mode. Please use 'server' or 'client'.")
        sys.exit(1)

naikrovek · 2 years ago

You wrote all that without testing anything? Oh you used ChatGPT probably

fiddyschmitt · 2 years ago

That's awesome!

rakoo · 2 years ago

There's a more complete version of this with nncp: https://www.complete.org/nncp/

aeonik · 2 years ago

This is utterly mind blowing.

Woah!

rnmmrnm · 2 years ago

reminds me how the other day I circumvented disabled SSH port forwarding by using https://github.com/nwtgck/yamux-cli + ncat already on the machine to do something like: yamux -l lport | ssh server -- <script: ncat --proxy-type=http -klp proxyport & and yamux localhost proxyport>.sh

good thing I needed HTTP access or else i had to find that socks5 server that's actually working again.

I've been trying to read up on TCP/UDP multiplexing and I'm having a hard time understanding what the use cases are, or advantages over just using regular connections, sockets, and application logic.

Do you have a good resource recommendation to learn this more?

Well here I tunneled multiple TCP connections traffic over a single input/output streams (the SSH shell IO stream).

Basically when you got a single communication channel and it's expensive to create more channels to tunnel multiple TCP streams where each connection get's its on channel, this is when you reach for the muxer.

i.e. here I probably could make it without yamux if instead I used SSH control port(socket? master?) and created many shell IO streams. but using yamux makes it cleaner. Also the server I'm working on is very buggy and you really get like 50% success rate logging in.

I suggest you download this `yamux` utility and play around with it really :)

Nice! It's so good when you can cobble together various tools to achieve something like that.

dheera · 2 years ago

Feature request: TCP over Facebook Messenger, TCP over Whatsapp

For use on flights where they give you "messenger only" internet access.

Feature request: TCP over JPEG cat pictures on HTTP port 80

It would likely do a better job at circumventing the China firewall than existing VPNs that can be flagged as VPN protocols.

mike_d · 2 years ago

> For use on flights where they give you "messenger only" internet access.

They still rate limit these flows to a very low speed.

ranger_danger · 2 years ago

> on flights

You could just sniff the traffic for an already-logged-in MAC and clone it.

simpsond · 2 years ago

But that will negatively impact the person who paid. That’s not cool.

DonHopkins · 2 years ago

Will this work with a pigeon and a USB stick? (Given a pigeon strong and fast enough to fly back and forth thousands of times.)

imhoguy · 2 years ago

Ping-eon would be very high.

HocusLocus · 2 years ago

> Will this work with a pigeon and a USB stick? (Given a pigeon strong and fast enough to fly back and forth thousands of times.)

> Ping-eon would be very high.

The last 10 years of the Internet has been one long setup for this joke. Resetting the clock for 10 years.

Haha!

bawolff · 2 years ago

What's the benefit of this over netcat + mkfifo?

Hi Bawolff! Yes that's a great combo. I guess File Tunnel provides a couple of things that makes the tunnel maintenance a bit easier.

1. It gracefully supports each side of the tunnel turning on and off.

2. It accepts any number of clients, and forwards them through the tunnel.

3. It recycles the shared file.

lambdaxyzw · 2 years ago

netcat+mkfifo does the actual transfer with TCP - so there's no tunneling involved. This solution tunnels tcp through file abstraction, so it works significantly differently.

As for the uses, see the three author listed. For example, by using a file share as transport you may evade firewall.

Deleted Comment

zamadatix · 2 years ago

  mkfifo /mnt/shared/connection
  nc localhost 12345 > /mnt/shared/connection < /mnt/shared/connection

Seems the same?

Thorrez · 2 years ago

That only works on Linux, but it looks like this program works on Windows too.

andrewstuart · 2 years ago

So there’s multiple writers to the file? How is this arbitrated?

Also how does either side know when the file has been updated?

jandrese · 2 years ago

It looks like one file per direction, and this is a point to point connection. It looks like each side just reads to the end of the file and then tries to read more to see if any new data is available.

Yes exactly. One file in each direction made arbitration easier.

In the future I will implement a single file to handle both directions.

Hi Andrew, there's just one writer (which can tunnel multiple TCP connections).

Arbitration was indeed one of the trickiest bits. Originally I pre-reallocated the full file size (10 MB). Then used an integer at the beginning of the file to signal to the other side that a block was ready. The other side repeatedly read that int, and read the corresponding part of the file. But writing twice (once for the data, once for the int) had a significant performance impact.

In the end, what worked best was not pre-allocating the file. Rather letting the file grow whenever the writer writes to it. The reader knows when data is available by doing a PeekChar() in a tight loop. It's surprisingly fast, and accurately reflects the state of the file.