tubes middleware

[shiona]

Hi, I think pwntools tubes could use some sort of "middleware" solution, that would process the data between
the underlying communication layer and the recvX functions.

I tried to look if the pwntools already has such a thing, but did not find one, if I just missed it, please let me know and close this ticket. The system that connects tubes together looked somewhat promising, but I don't think it does what I'm envisioning.

It is also very possible something like this cannot be implemented.

For a very simple example, say a service communicates with a rot13. Currently I can do:

p = Remote(IP, PORT)
p.recvuntil(b'fryrpgvba >')
p.sendline(b'ybtva')
...

but it gets a little bit tedious to do the cipher by hand for each line. And if the service uses some actual encryption scheme, then it just gets worse.

What if you could do:

def rot13(input):
  ...

p = Remote(IP, PORT)
p.addmiddleware(rot13, rot13)
p.recvuntil(b'selection >')
p.sendline(b'login')
...

I'm not sure about the syntax (mainly should the middleware be a class with in and out methods, or just separate functions), but I hope this gets the idea across.

Things that I've thought this should be able to do is

• decrypt / encrypt data without state
• decrypt / encrypt data with state (know the last message, use running block/stream cipher for subsequent messages)
  • I think this can be done by user if the middleware is a class, or with capturing lambdas if not
• keep track of message count to add running sequence numbers to messages
• add / check signatures/checksums on messages.
  • either silently skip, warn or fail if some message fails these
• maybe it would be nice to be able to disable/enable the middleware for some messages?
• might be useful to layer multiple middlewares together,
  • one to add / strip sequence number closest to the user code, and one to encrypt/decrypt above that.

Of course the things mentioned would be actually done by the middlewares themselves, but the tubes layer would need to provide API to connect them.

I looked into creating this, but when I found the unrecv function and realized sometimes the data needs to flow back, I wasn't sure I knew how to to implement any of this. Or if it would be possible.

[peace-maker]

You can create your own tubes by overwriting the recv_raw and send_raw functions and applying any transformations there. Maybe that's already enough for your usecase. Adding such an example to the docs as a starting point to implement such middleware could be enough? I'm not sure such a middleware API really adds to the experience instead of making exploit scripts more confusing. How often do you encounter such use cases? I usually just create my own assemble_packet function that does the encoding and just does tube.flat({...}) instead of transparently expecting recvline to already do the work for me.

from pwn import *
from pwnlib.util.packing import _encode, _decode
import codecs

class rot13tube(remote):
    def recv_raw(self, *a, **kw):
        raw_data = super().recv_raw(*a, **kw)
        if raw_data and self.isEnabledFor(logging.DEBUG):
            self.debug('Received raw %#x bytes:', len(raw_data))
            self.maybe_hexdump(raw_data, level=logging.DEBUG)
        if raw_data is None:
            return None
        return _encode(codecs.decode(_decode(raw_data), 'rot_13'))

    def send_raw(self, data, *a, **kw):
        data = _encode(codecs.encode(_decode(data), 'rot_13'))
        if self.isEnabledFor(logging.DEBUG):
            self.debug('Sent raw %#x bytes:', len(data))
            self.maybe_hexdump(data, level=logging.DEBUG)
        super().send_raw(data, *a, **kw)

s = listen(1337)
io = rot13tube(s.lhost, s.lport)
s.wait_for_connection()
io.sendline(b'Hello, World!')
log.info('sent: %r', s.recvline())
s.sendline(b'Uryyb, Jbeyq!')
log.info('received: %r', io.recvline())
io.close()

Output:

$ python rot13tube.py DEBUG
[+] Trying to bind to :: on port 1337: Done
[+] Waiting for connections on :::1337: Got connection from ::1 on port 48898
[+] Opening connection to :: on port 1337: Done
[DEBUG] Sent 0xe bytes:
    b'Hello, World!\n'
[DEBUG] Sent raw 0xe bytes:
    b'Uryyb, Jbeyq!\n'
[DEBUG] Received 0xe bytes:
    b'Uryyb, Jbeyq!\n'
[*] sent: b'Uryyb, Jbeyq!\n'
[DEBUG] Sent 0xe bytes:
    b'Uryyb, Jbeyq!\n'
[DEBUG] Received raw 0xe bytes:
    b'Uryyb, Jbeyq!\n'
[DEBUG] Received 0xe bytes:
    b'Hello, World!\n'
[*] received: b'Hello, World!\n'
[*] Closed connection to :: port 1337
[*] Closed connection to ::1 port 48898

[shiona]

Thank you for the example. I think it covers most of what I was looking for. I think that adding an explanation of this to the documentation would be a good thing.

The reasons I wish this to happen transparently are:

• I could use recvuntil
  • maybe the server sends an unknown amount of data before a prompt. I can currently recv and loop over it to transform the data and test if it matches, but that's extra work that recvuntil is designed to do
• I could use interactive
  • the tubes would just magically mangle the data to and from whatever protocol the other end uses, I as a user would have just simple text interface.

I also found that I'm not the only one who has thought about this scenario: in the response of #962 zachriggle introduces the idea of a filter to do something to the user input before it gets sent in the interactive mode.

I'll test some things with your example code and return with comments at some later point.