This page last updated: Sun Jan 14 21:00:00 2007
What: A HTTP proxy is a piece of software that acts as an intermediary between HTTP client software (i.e. browser) and HTTP server software. The proxy receives all requests from the browser, and relays them (possibly modified) onto the server. Likewise, it receives all responses from the server, and relays them (possibly modified) to the client. HTTP Proxies can be used for a wide variety of tasks, including filtering, logging, caching, etc, etc, etc. At some stage, most web programmers will make use of some form of proxy.
Why: The difficulty sometimes arises that it's not easy to find a proxy which implements exactly the function which is required. There may be one or more proxies that implement something close, but not exactly. In these cases, it is often necessary to take an existing proxy that is as close as possible, and modify its source code to meet requirements.
The function of this page is to make this searching and comparison process easier for python programmers. I have listed here all of the open source python HTTP proxies I could find. Listing them together should make the job of comparing features a little easier, at least on a high level basis.
Please feel free to email me if
| Name (click for details) | Primary features | Minimum python version | Design | License | Last updated YYYY-MM-DD | HTTP | HTTPS | Windows NTLM |
| NTLM Authorization Proxy Server | Windows NTLM authentication, HTTPS | 1.5.2 | Threaded | GPL | 2006-06-16 | 1.1 | yes | yes |
| WebCleaner | Range of filtering/blocking techniques, html parsing, javascript engine (SpiderMonkey), recognizes known browser attacks, XML DTD for describing filters and rewrite rules | 2.4 | Asyncore | GPL | 2006-12-16 | 1.1 | yes | yes |
| SPIKE Proxy | SPIKE Proxy is a tool for looking for application-level vulnerabilities in web applications. | 2.3 (requires pyOpenSSL) | Threaded | GPL (soon to be LGPL) | 2004-11-05 | 1.1 | yes | yes |
| Willow | Bayesian content filtering, browser-based interface, caching, Windows NTLM authentication | 2.2.2 or greater | Asyncore | LGPL | 2003-06-06 | 1.1 | yes | yes |
| Amit's Web Proxy Project | Filtering/blocking, compression, experimental architectural approaches, range of loadable modules | all | Asyncore | MIT | 2003-11-16 | 1.1 | no | no |
| HTTP Replicator | Caching. Designed for caching Debian packages. Good asyncore design pattern. | 2.3 | Asyncore | GPL | 2004-11-29 | 1.1 | no | no |
| MindRetrieve | Indexing proxy | 2.4 | Threaded | BSD | 2005-03-28 | 1.0 | no | no |
| Personal Proxy Server | Indexing proxy | 2.3 | Threaded | LGPL | 2005-05-24 | 1.0 | no | no |
| Twisted Proxy | Twisted Proxy | 2.3 | Twisted | LGPL | 2005-05-24 | 1.0 | no | no |
| WebDebug | Request and Resource logging, Browser interface, Statistics | 2.0 (older versions support 1.5.x) | Threaded | GPL | 2002 | 1.1 | no | no |
| httpMonitor | Logging, header transformation | 1.5 | Threaded | GPL | 2002-03-15 | 1.1 | no | no |
| AdZapper | Elimination of Banner Advertisements. Programmable through XML "zapplets": a wizard is provided. Controllable through a web/browser interface. | 1.5.2 | Medusa | Python | 2001-09-02 | 1.1 | no | no |
| Archiver Proxy | "An http proxy server which archives all your HTTP traffic." | 2.0 | Asyncore | "This program is free software." | 2001-08-29 | 1.0 | yes | no |
| Cut the Crap | Content filtering, filters extensible through python "plug-ins" and "zapplets language", ACL (Access Control Lists) | 2.0 | Asyncore | GPL | 2001-08-20 | 1.1 | no | no |
| Alfajor | Cookie filtering, basic content filtering, optional GUI | 1.5.2 | Threaded | GPL | 2000-02-28 | 1.0 | no | no |
| Tiny HTTP Proxy | Solid basic design pattern | 2.1 and greater | Threaded | Unknown | 2003-06-17 | 1.1 | yes | no |
| TCPWatch | TCP and HTTP protocol debugging, optional GUI Interface | 2.1 | Unknown | Zope ZPL | 2003-06-17 | 1.1 | yes | no |
| Mediator | Mediator support, content filtering | ?? | Threaded | GPL | Unknown | 1.1 | no | no |
| Munchy | Design pattern for basic content filtering and blocking | ?? | Threaded | Unknown | 2000-04-25 | 1.0 | no | no |
| HTTP Debugging Proxy | Basic debugging proxy | all | Threaded | Unknown | Unknown | 1.1 | yes | no |
| Name | NTLM Authorization Proxy Server |
| Last updated YYYY-MM-DD | 2006-06-16 |
| HTTP version | 1.1 |
| HTTPS Connect | yes |
| Windows NTLM support | yes |
| Minimum python version | 1.5.2 |
| Platform | Win32 |
| Author | Darryl Dixon, who succeeded Dmitry Rozmanov |
| Home page | http://ntlmaps.sourceforge.net/ |
| Primary features | Windows NTLM authentication, HTTPS |
| Features |
From the product page 'NTLM Authorization Proxy Server' (APS) is a proxy software that allows you to authenticate via an MS Proxy Server using the proprietary NTLM protocol. Since version 0.9.5 APS has an ability to behave as a standalone proxy server and authenticate http clients at web servers using NTLM method. It can change arbitrary values in your client's request header so that those requests will look like they were created by MS IE. Main features:
|
| License | GPL |
| Design Architecture | Threaded |
| Notes | Of note is that Dmitry has built his own HTTP server/client from the ground up, rather than using the HTTP support in the python standard library. |
| Name | WebCleaner |
| Last updated YYYY-MM-DD | 2006-12-16 |
| HTTP version | 1.1 |
| HTTPS Connect | yes |
| Windows NTLM support | yes |
| Minimum python version | 2.4 |
| Platform | all |
| Author | Calvin (aka Bastian Kleineidam) |
| Home page | http://webcleaner.sourceforge.net/ |
| Primary features | Range of filtering/blocking techniques, html parsing, javascript engine (SpiderMonkey), recognizes known browser attacks, XML DTD for describing filters and rewrite rules |
| Features |
From the project page
|
| License | GPL |
| Design Architecture | Asyncore |
| Notes | Derived from Amit's Proxy 4. |
| Name | SPIKE Proxy |
| Last updated YYYY-MM-DD | 2004-11-05 |
| HTTP version | 1.1 |
| HTTPS Connect | yes |
| Windows NTLM support | yes |
| Minimum python version | 2.3 (requires pyOpenSSL) |
| Platform | Linux/Windows |
| Author | Dave Aitel |
| Home page | http://www.immunitysec.com/resources-freesoftware.shtml |
| Primary features | SPIKE Proxy is a tool for looking for application-level vulnerabilities in web applications. |
| Features |
From the product web-site: SPIKE Proxy is a professional-grade tool for looking for application-level vulnerabilities in web applications. SPIKE Proxy covers the basics, such as SQL Injection and cross-site-scripting, but it's completely open Python infrastructure allows advanced users to customize it for web applications that other tools fall apart on. Features include
|
| License | GPL (soon to be LGPL) |
| Design Architecture | Threaded |
| Notes | HTTPS is not tunneled - it's MITMed. |
| Name | Willow |
| Last updated YYYY-MM-DD | 2003-06-06 |
| HTTP version | 1.1 |
| HTTPS Connect | yes |
| Windows NTLM support | yes |
| Minimum python version | 2.2.2 or greater |
| Platform | Linux only. From the product page: "There is no windows suport at this time. It is being worked on and an should be release shortly." |
| Author | Digital Lumber |
| Home page | http://www.digitallumber.com/willow |
| Primary features | Bayesian content filtering, browser-based interface, caching, Windows NTLM authentication |
| Features |
From the product page Willow is a content-filtering proxy server. It bears one similarity to the many other pieces of software available for web filtering in that it is designed to filter web content. That, however, is where the similarities end. The differences between Willow and other solutions are significant, and these differences make Willow the first really usable internet filter. In addition to being the first web filter to really work, Willow was also designed to make life easy on network administrators. To this end Willow supports the following:
|
| License | LGPL |
| Design Architecture | Asyncore |
| Notes | Willow is interesting because it uses Bayesian filtering to recognise the bad content that you don't want to see. Which means that it has a Bayesian network that has to be trained on good and bad content in order to be able to tell the difference between them. Which means that you need some bad content in order to train it: a problem which the authors have solved by including a corpus of pornography in the download! |
| Name | Amit's Web Proxy Project |
| Last updated YYYY-MM-DD | 2003-11-16 |
| HTTP version | 1.1 |
| HTTPS Connect | no |
| Windows NTLM support | no |
| Minimum python version | all |
| Platform | Mostly *nix |
| Author | Amit Patel |
| Home page | http://theory.stanford.edu/~amitp/proxy.html |
| Primary features | Filtering/blocking, compression, experimental architectural approaches, range of loadable modules |
| Features |
Amit has written a variety of proxies, with differing features. Among the features of the varying proxies are
Among the loadable modules that come with Proxy 3 are
|
| License | MIT |
| Design Architecture | Asyncore |
| Notes | Amit has some good notes on the various implementations he has created, including discussion of techniques. Amit put a lot of work in transforming content as it passed through the proxy: e.g. eliminating javascript that popped-up windows, eliminating requests to ad-servers, etc. |
| Name | HTTP Replicator |
| Last updated YYYY-MM-DD | 2004-11-29 |
| HTTP version | 1.1 |
| HTTPS Connect | no |
| Windows NTLM support | no |
| Minimum python version | 2.3 |
| Platform | Linux |
| Author | Gertjan |
| Home page | http://freshmeat.net/projects/http-replicator |
| Primary features | Caching. Designed for caching Debian packages. Good asyncore design pattern. |
| Features |
Replicator is a replicating HTTP proxy server. Files that are downloaded through the proxy are transparently stored in a private cache, so an exact copy of accessed remote files is created on the local machine. It is in essence a general purpose proxy server, but especially suited for maintaining a cache of Debian packages. |
| License | GPL |
| Design Architecture | Asyncore |
| Notes | A debian package is available. The python code is discussed on the project homepage. The code is clear and concise enough to make HTTP Replicator a good design pattern for writing an asynchronous (asyncore) proxy. |
| Name | MindRetrieve |
| Last updated YYYY-MM-DD | 2005-03-28 |
| HTTP version | 1.0 |
| HTTPS Connect | no |
| Windows NTLM support | no |
| Minimum python version | 2.4 |
| Platform | Windows and Linux |
| Author | Wai Yip Tung |
| Home page | http://www.mindretrieve.net/ |
| Primary features | Indexing proxy |
| Features | From the product page: "MindRetrieve is a personal desktop search engine. Unlike general purpose search engines that intent to index and search every web page in the world, MindRetrieve focus on a small but very special area. That is the web that you have seen. " |
| License | BSD |
| Design Architecture | Threaded |
| Notes | This proxy depends on PyLucene, a python port of Jakarta Lucene |
| Name | Personal Proxy Server |
| Last updated YYYY-MM-DD | 2005-05-24 |
| HTTP version | 1.0 |
| HTTPS Connect | no |
| Windows NTLM support | no |
| Minimum python version | 2.3 |
| Platform | all |
| Author | Duncan Gough |
| Home page | http://www.suttree.com/code/pps/ |
| Primary features | Indexing proxy |
| Features | PPS is a local web proxy that indexes the URL, page title, date and text of every page you visit. It feeds that data into a database that can be queried later, providing you with a searchable browser history. |
| License | LGPL |
| Design Architecture | Threaded |
| Notes | This proxy is based on Tiny HTTP Proxy. This proxy depends on Lupy, a python port of Jakarta Lucene 1.2 |
| Name | Twisted Proxy |
| Last updated YYYY-MM-DD | 2005-05-24 |
| HTTP version | 1.0 |
| HTTPS Connect | no |
| Windows NTLM support | no |
| Minimum python version | 2.3 |
| Platform | all |
| Author | Duncan Gough |
| Home page | http://www.suttree.com/code/proxy/ |
| Primary features | Twisted Proxy |
| Features | "This is an extremely simple proxy server that I created since there were no examples of how to use the Twisted proxy code that I could find." |
| License | LGPL |
| Design Architecture | Twisted |
| Notes | This proxy depends on Twisted |
| Name | WebDebug |
| Last updated YYYY-MM-DD | 2002 |
| HTTP version | 1.1 |
| HTTPS Connect | no |
| Windows NTLM support | no |
| Minimum python version | 2.0 (older versions support 1.5.x) |
| Platform | all |
| Author | Paul Clip |
| Home page | http://www.cyberclip.com/webdebug/index.html |
| Primary features | Request and Resource logging, Browser interface, Statistics |
| Features |
From the product page: Here is a quick summary of WebDebug's features:
|
| License | GPL |
| Design Architecture | Threaded |
| Notes |
| Name | httpMonitor |
| Last updated YYYY-MM-DD | 2002-03-15 |
| HTTP version | 1.1 |
| HTTPS Connect | no |
| Windows NTLM support | no |
| Minimum python version | 1.5 |
| Platform | all |
| Author | Volker Stampa |
| Home page | http://home.wtal.de/stampa/httpMonitor/ |
| Primary features | Logging, header transformation |
| Features |
From the product page In a XML-configuration file you can specify python-functions as processors for the HTTP-messages you are interested in (requests or responses, which match certain criteria). You may have as many processors as you want, without modifying the actual program. The only thing you need to do, is configure the httpMonitor and write your processors. The package comes with three sample processors. One for logging header-information of the HTTP messages (httpMonitor.filters.logger), one for modifying header fields (httpMonitor.filters.headermodifier) and a third one for accessing (and modifiying) parsed html-text. The processors can be configured to tell them exactly what to log or what to modify. |
| License | GPL |
| Design Architecture | Threaded |
| Notes |
| Name | AdZapper |
| Last updated YYYY-MM-DD | 2001-09-02 |
| HTTP version | 1.1 |
| HTTPS Connect | no |
| Windows NTLM support | no |
| Minimum python version | 1.5.2 |
| Platform | all |
| Author | Adam Feuer |
| Home page | http://www.zaplet.org/adzapper/ |
| Primary features | Elimination of Banner Advertisements. Programmable through XML "zapplets": a wizard is provided. Controllable through a web/browser interface. |
| Features |
From the product page: adzapper is a filtering proxy that can block ads from being displayed on your web browser. instead of ad banners, you see blank spaces: adzapper transforms the ads into transparent gifs. The rules that describe what are ads and what are not ads are called "zaplets", and are configurable on a per-website basis. This way, when websites change their ads or their graphic design, it is easy to build and share new zaplets that block the ads. adzapper is based on Sam Rushing's Medusa, a very fast asychronous-sockets web-server written in Python. Medusa is single threaded, but this doesn't mean it is slow! in my experience it is one of the fastest, lightest webservers out there. |
| License | Python |
| Design Architecture | Medusa |
| Notes | From the product's About Page: "adzapper was inspired by the Muffin filtering proxy written by Mark Boyns, and additional inspiration and the impetus to start coding was provided by Constantinos Kotsokalis and his CTC filtering proxy". |
| Name | Archiver Proxy |
| Last updated YYYY-MM-DD | 2001-08-29 |
| HTTP version | 1.0 |
| HTTPS Connect | yes |
| Windows NTLM support | yes |
| Minimum python version | 2.0 |
| Platform | all |
| Author | Aaron Swartz |
| Home page | http://logicerror.com/archiverProxy |
| Primary features | "An http proxy server which archives all your HTTP traffic." |
| Features |
|
| License | "This program is free software." |
| Design Architecture | Asyncore |
| Notes | Based on code from The Medusa Tutorial and Neil Schemenauer's Munchy. |
| Name | Cut the Crap |
| Last updated YYYY-MM-DD | 2001-08-20 |
| HTTP version | 1.1 |
| HTTPS Connect | no |
| Windows NTLM support | no |
| Minimum python version | 2.0 |
| Platform | all |
| Author | Constantinos A. Kotsokalis |
| Home page | http://www.softlab.ece.ntua.gr/~ckotso/CTC/ |
| Primary features | Content filtering, filters extensible through python "plug-ins" and "zapplets language", ACL (Access Control Lists) |
| Features | From the product page: "Cut The Crap" is a proxy-like server, that will keep advertisement banners out of your sight |
| License | GPL |
| Design Architecture | Asyncore |
| Notes |
| Name | Alfajor |
| Last updated YYYY-MM-DD | 2000-02-28 |
| HTTP version | 1.0 |
| HTTPS Connect | no |
| Windows NTLM support | no |
| Minimum python version | 1.5.2 |
| Platform | all |
| Author | Andrew Cooke |
| Home page | http://www.acooke.org/jara/alfajor/index.html |
| Primary features | Cookie filtering, basic content filtering, optional GUI |
| Features |
From the product page Alfajor is an http cookie filter, written in Python with an optional GUI. It acts as an http proxy (you must configure your browser to use it) and can either contact sites directly or work with a second proxy (eg. a cache). The program allows cookies to be sent from your browser to any address which matches a list of regular expressions. The GUI allows the filter to be turned on and off, sites to be added or removed as they are accessed, and the list of regular expressions to be edited. From version 1.3, Alfajor can block all data from named sites. This allows filtering of adverts. By running two instances of the program in series, cookies can be controlled and adverts blocked. For more information, see here. Please note that Alfajor does not fully conform to any HTTP version. However, in practice, it works with the vast majority of sites. |
| License | GPL |
| Design Architecture | Threaded |
| Notes |
| Name | Tiny HTTP Proxy |
| Last updated YYYY-MM-DD | 2003-06-17 |
| HTTP version | 1.1 |
| HTTPS Connect | yes |
| Windows NTLM support | yes |
| Minimum python version | 2.1 and greater |
| Platform | Beos/Windows |
| Author | Suzuki Hisao |
| Home page | http://mail.python.org/pipermail/python-list/2003-June/210343.html |
| Primary features | Solid basic design pattern |
| Features | |
| License | Unknown |
| Design Architecture | Threaded |
| Notes | This proxy is so small, there is no home page for it. The link given above points to the archive of Suzuki's post of the code to comp.lang.python. |
| Name | TCPWatch |
| Last updated YYYY-MM-DD | 2003-06-17 |
| HTTP version | 1.1 |
| HTTPS Connect | yes |
| Windows NTLM support | yes |
| Minimum python version | 2.1 |
| Platform | all |
| Author | Shane Hathaway |
| Home page | http://hathaway.freezope.org/Software/TCPWatch |
| Primary features | TCP and HTTP protocol debugging, optional GUI Interface |
| Features |
From the product page: TCPWatch is a utility written in Python that lets you monitor forwarded TCP connections or HTTP proxy connections. It displays the sessions in a window with a history of past connections. It is useful for developing and debugging protocol implementations and web services. The latest version, 1.2, adds support for recording sessions to a directory. |
| License | Zope ZPL |
| Design Architecture | Unknown |
| Notes |
| Name | Mediator |
| Last updated YYYY-MM-DD | Unknown |
| HTTP version | 1.1 |
| HTTPS Connect | no |
| Windows NTLM support | no |
| Minimum python version | ?? |
| Platform | all |
| Author | Itamar Shtull-Trauring |
| Home page | http://www.itamarst.org/software/ |
| Primary features | Mediator support, content filtering |
| Features | Apart from its ability to filter content, stripping ads, javascript, etc, this proxy has one unique feature: It can forward all requests through a given anonymiser site or other "mediator" site. Best to visit the page and see it in action. |
| License | GPL |
| Design Architecture | Threaded |
| Notes | This is derived from Neil Schemenauer's Munchy proxy. Here is a direct link to the single python source file. |
| Name | Munchy |
| Last updated YYYY-MM-DD | 2000-04-25 |
| HTTP version | 1.0 |
| HTTPS Connect | no |
| Windows NTLM support | no |
| Minimum python version | ?? |
| Platform | all |
| Author | Neil Schemenauer |
| Home page | http://arctrix.com/nas/python/munchy.py |
| Primary features | Design pattern for basic content filtering and blocking |
| Features | Munchy is designed to strip ads and other time/bandwidth consuming and privacy invading stuff. |
| License | Unknown |
| Design Architecture | Threaded |
| Notes | The content filtering database is well out-of-date. |
| Name | HTTP Debugging Proxy |
| Last updated YYYY-MM-DD | Unknown |
| HTTP version | 1.1 |
| HTTPS Connect | yes |
| Windows NTLM support | yes |
| Minimum python version | all |
| Platform | all |
| Author | Xavier Defrang |
| Home page | http://defrang.com/python/ |
| Primary features | Basic debugging proxy |
| Features | This small HTTP proxy prints the headers of all HTTP requests and responses |
| License | Unknown |
| Design Architecture | Threaded |
| Notes | This proxy is based on Tiny HTTP Proxy. Here is a direct link to the single python source file. |
source http://www.xhaus.com/alan/python/proxies.html page by Alan Kennedy