PageKite: why Python?

Bjarni Rúnar Einarsson • PageKite.net

What is PageKite?

Enabling (web)servers everywhere

PageKite is a tool for enabling (web)servers anywhere:

• PCs, laptops, mobile phones, embedded hardware ...

Problems to solve:

• Most devices don't have visible IP addresses
• Most devices are behind firewalls
• Many devices move around a lot
• Many "networks" only have one public IP for many computers

IPv6?

Diagram: A PageKite-enabled website

pagekite.py + pagekite.net

Introducing pagekite.py

pagekite.py implements a tunneled reverse proxy.

• General features ...
  • Open source / free software - AGPL license
  • Designed to be easy to use and deploy
• • Is an HTTP and HTTPS reverse proxy (and SSH!)
  • Protocol-agnostic tunneling:
    • Tunnels through firewalls and NAT (outgoing TCP)
    • Uses adaptive compression (zlib)
  • Implements both ends: front & back
• • Built-in HTTP server for diagnostics

A trivial example

This is one way to make a web server on localhost (ports 8000 and 8443) visible as http://bar.foo.net/, ...

foo.net $ sudo pagekite.py --runas=nobody \
 --isfrontend --ports=80,443 \
 --domain=raw,http,https:*.foo.net:s3cr37

laptop $ pagekite.py \
 --frontend=foo.net:443 \
 --backend=http:bar.foo.net:localhost:8000:s3cr37 \
 --backend=https:bar.foo.net:localhost:8443:s3cr37

Introducing pagekite.net

pagekite.net is a front-end service provider, a "FOSS start-up"

• Goals
  • Fund pagekite.py and related development
  • Make the tech. available to non-technical users
• Features   (italics = works in progress)
  • Geographically distributed, managed front-ends
  • Privacy friendly: minimal records, anonymized logs, simple ToS, IMMI
  • Use name.pagekite.me or your own domain
  • Unlimited subdomains & active connections
  • Free SSL for everyone! - wildcard on *.pagekite.me
  • We charge for bandwidth ...so no ads!

Use-cases I've seen...

Some of the use-cases I have seen so far:

• A web-server on my Android phone, just for fun.
• 4 Linux VMs on a gaming rig at home: all visible, my own cloud.
• My web guy demoing designs directly from his laptop
• Teachers publishing class material off recycled hardware
• Arduino hackers sending data to Pachube.com
• Pagekite+SSH for remote administration of embedded HW in factories

Basically any time a router or firewall is in the way.

... so why Python?

Choosing a programming language

Language requirements:

1. Must not suck!
2. Same language for program and service
3. Ease of program deployment
4. Good support for the WWW

Things we don't care about (yet):

1. Raw speed
2. Footprint (RAM, disk)

So how is it doing? - good

The good:

• I can use Python 2.2 as a baseline
• Installation is indeed easy
• Exceptions let me adapt to my environment
• Python comes with Linux, Mac OS X!
• Py2exe helps with Windows
• Django is awesome!

So how is it doing? - bad

The bad:

• I use Python 2.2 as a baseline.
• Multiple almost-compatible SSL libraries, limited APIs
• Adding a GUI will fragment the code
• Django assumes an SQL backend (I use Redis)
• Memory bloat ...

So how is it doing? - ugly

Things moved around during Python's development ...

try:
  from urlparse import parse_qs, urlparse

except ImportError, e:
  from cgi import parse_qs
  from urlparse import urlparse

So how is it doing? - ugly

Argh, different OSes throw different errors!

  HARMLESS_ERRNOS = (errno.EINTR, errno.EAGAIN, errno.ENOMEM, errno.EBUSY,
                     errno.EDEADLK, errno.EWOULDBLOCK, errno.ENOBUFS,
                     errno.EALREADY)
...
      try:
        sent_bytes = self.fd.send(sending)
      except IOError, err:
        if err.errno not in self.HARMLESS_ERRNOS:
          self.LogError('Error sending: %s' % err)
          return False
...

So how is it doing? - ugly

The SSL APIs are a bit inconsistent, so ...

def Shutdown(self, direction):
  try:
    if self.fd:
      if 'sock_shutdown' in dir(self.fd):
        # This is a pyOpenSSL socket, which has incompatible shutdown.
        if direction == socket.SHUT_RD: self.fd.shutdown()
      else:
        self.fd.shutdown(direction)

  except Exception, e:
    self.LogDebug('Shutdown (%s/%s) error: %s' % (direction, self.fd, e))

Thank you!

Questions?

Links:

• Code, company: PageKite.net
• Slides: pagekite.net/slides/rpug2011/
• Github: github.com/pagekite/PyPagekite

Bonus FAQ slide!

• But my $device isn't on-line all the time!
  • Do all websites need to be highly available?
• Isn't proxying everything really slow?
  • Reduced RTT and compression make up for the detour
• Isn't pagekite.net a centralized service too?
  • Pagekite is a router, not a data store
  • It's FOSS, you have choice & can run your own
• What about backups?
  • Yes, you still need backups. :-)