Making the Web 1000x Bigger
Bjarni Rúnar Einarsson • pageKite.net
Motives
Why make the web bigger?
Look, it fits on one slide!
No, seriously. Why?
- The web is centralized, in the cloud
- Most devices are mere consumers - browsers
- Everything has to be "uploaded"
- 3rd party hosting is the norm
- ... so the power is centralized ...
- Terms and Conditions
- Privacy Policies
- Censorship, DMCA, data retention laws, ...
- ... and the web serves companies first, people second.
When I imply that the web is too small, what I'm really getting at is that the vast majority of the devices connected to the Internet, participate in the WWW merely as consumers of data - publishing is centralized and generally takes place on 3rd party hardware.
This has certain fundamental implications for how we can use the web, both technical (we have to upload everything before sharing), and social, in the form of Terms and Conditions and of course Privacy Policies - which are barely ever read by anyone.
If anything, this trend is accelerating with the rise of SaaS, PaaS, "the cloud" and behomoths like Google and Facebook.
Can we...
Host our own sites?
Why not? A bit of history.
Once upon a time...
- Web-servers were "high tech"
- Servers were expensive and "high tech"
- Desktops were insecure.
- We ran out of IPs! ... on 03.02.2011.
- My radical argument: only 4. is a major issue today
Enabling (web)servers everywhere
With a server on every computer, how would we make them all reachable without more IP addresses?
- Protocol-layer routing: tunnels and reverse proxies
- ... existing solutions:
- {Squid,Apache,lighttpd,Varnish} & {ssh tunnel,ssltunnel,VPNs}
- Tor hidden services & www.tor2web.com
- Opera Unite
- pagekite.py & pageKite.net - that's us!
- IPv6?
Diagram: reverse proxied
|
pagekite.py + pagekite.net
Introducing pagekite.py
pagekite.py implements a tunneled reverse proxy.
- General features ...
- Open source / free software - AGPL license
- Designed to be easy to use and deploy
- Is an HTTP and HTTPS reverse proxy (and SSH!)
- Protocol-agnostic tunneling:
- Tunnels through firewalls and NAT (outgoing TCP)
- Uses adaptive compression (zlib)
- Implements both ends: front & back
- Built-in HTTP server for diagnostics
Limitations
There are some limitations to this approach...
- HTTP
HTTP/1.0 and 0.9 might not send Host: headers. - HTTPS
Relies on SNI (variable browser support) or ...
TLS cert at front-end (so not end-to-end) - SSH
Relies on IP tracking hack (HTTP-before-SSH) or ...
HTTP proxy support (openssh & putty both work well)
Needing a front-end is a bit lame. But it surprisingly pretty well!
A trivial example
This is one way to make a web server on localhost (ports 8000 and 8443) visible as http://bar.foo.net/, ...
foo.net $ sudo pagekite.py --runas=nobody \ --isfrontend --ports=80,443 \ --domain=raw,http,https:*.foo.net:s3cr37
laptop $ pagekite.py \ --frontend=foo.net:443 \ --backend=http:bar.foo.net:localhost:8000:s3cr37 \ --backend=https:bar.foo.net:localhost:8443:s3cr37
Introducing pagekite.net
pagekite.net is a front-end service provider, a "FOSS start-up"
- Goals
- Fund pagekite.py and related development
- Make the tech. available to non-technical users
- Features
(italics = works in progress)
- Geographically distributed, managed front-ends
- Privacy friendly: minimal records, anonymized logs, simple ToS, IMMI
- Use name.pagekite.me or your own domain
- Unlimited subdomains & active connections
- Free SSL for everyone! - wildcard on *.pagekite.me
- We charge for bandwidth ...so no ads!
Using pageKite.net
Another example, this time using the pageKite.net managed front-end service and enabling SSH and the HTTP user interface ...
laptop $ pagekite.py \ --defaults \ --httpd=localhost:9999 \ --backend=http:bar.foo.net:localhost:8000:s3cr37 \ --backend=https:bar.foo.net:localhost:8443:s3cr37 \ --backend=raw/22:bar.foo.net:localhost:22:s3cr37
So simple!
Why not put it in Diaspora?
or Status.net?
Use-cases I've seen...
Some of the use-cases I have seen so far:
- A web-server on my Android phone, just for fun.
- 4 Linux VMs on a gaming rig at home: all visible, my own cloud.
- My web guy demoing designs directly from his laptop
- Teachers publishing class material off recycled hardware
- Arduino hackers sending data to Pachube.com
- Pagekite+SSH for remote administration of embedded HW in factories
Basically any time a router or firewall is in the way.
Thank you!
Questions?
Links:
- Code, company: pageKite.net
- Slides: pagekite.net/slides/fosdem2011/
- Github: github.com/pagekite/PyPagekite
Bonus FAQ slide!
- But my $device isn't on-line all the time!
- Do all websites need to be highly available?
- Isn't proxying everything really slow?
- Reduced RTT and compression make up for the detour
- Isn't pagekite.net a centralized service too?
- Pagekite is a router, not a data store
- It's FOSS, you have choice & can run your own
- What about backups?
- Yes, you still need backups. :-)