Feb
27
2008
The ability to transform and inspect HTTP data as it flows in and out of a web application has many practical uses (both inside and outside of security). On IIS, this capability was historically restricted to ISAPI filters. Http Modules written in ASP.NET have always allowed processing of requests and responses to and from an ASP.NET application, but with the advent of IIS7 and the integrated ASP.NET pipeline, Http Modules written in ASP.NET now have access to virtually all stages of request processing (including those not handled by ASP.NET).
Transformer.NET is an Http Module designed for on-the-fly inbound and outbound url rewriting. Apache’s mod_rewrite, used to manipulate inbound request urls, is arguably one of the most popular Apache modules around. While there have been several ports of mod_rewrite to IIS (with implementations ranging from Http Modules to ISAPIs), they all share one shortcoming in common with their Apache predecessor: They only rewrite requests and not urls within outbound responses (such as links that are generated within an HTML page).
This has long been a pet peeve of mine. If you want to use mod_rewrite, you typically need to update the underlying website source code so that the hyperlinks within the application point to the “rewritten” urls. This can be a major effort and inconvenience if the site is already written, and even worse, it may not be possible for 3rd party or COTS web applications.
The initial beta release of Transformer.NET differs from previous rewrite modules because it supports bi-directional (inbound and outbound) url rewriting. Bi-directional rewriting eliminates the need to modify the underlying website code, which is great for legacy or third party web sites and applications. In addition to the ability to parse response content (such as HTML), Transformer.NET also includes the following two key internal mechanisms:
| |
Normalization Engine
Normalizing all urls into their absolute representation quickly became essential for two reasons. First, the module needs to be able to apply configured rules to a given url in any form. So a rule for “/foo/bar.htm” might need to be applied to “bar.htm”, “../bar.htm”, or any other number of relative url variants depending on the path of the rendering page. Second, if “/foo/bar.htm” is rewritten to “/fake/bar.htm”, then suddenly all of the relative links on the page (images, css, etc) will be broken. Replacing a relative link on a rewritten page with its absolute counterpart is essential.
Internal Url Cache
Like anything in the software world, performance is important. Inspecting and transforming very large responses when lots of rules have been defined can be a real performance killer. To help minimize performance impact, Transformer.NET maintains an internal cache of all rewrites that are performed. This eliminates un-needed processing the next time the url is rendered on a page. The net result is that as more requests are parsed by the module, performance impact continually decreases. To avoid stale cache entries, the cache gets cleared any time a change is made to a rewrite rule.
|
The bi-directional rewriting capability of Transformer.NET was really an initial “proof-of-concept” for us to start building bi-directional HTTP inspection and transformation solutions to solve some very interesting web application security problems.
Unlike Apache’s mod_rewrite, Transformer.NET does not implement conditional rewrites (ala mod_rewrite’s RewriteCond) so it is not intended to be a total port. The current beta version can be downloaded from our tools page. Transformer works on IIS6 (limited to ASP.NET applications) and with any site running on IIS7. A detailed user guide is included with the download.
Feb
19
2008
I wrote a plug-in for Burp Proxy that decompresses HTTP response content in the ZLIB (RFC1950) and DEFLATE (RFC1951) compression data formats. This arose out of an immediate need on a recent web application security assessment.
Inspecting the HTTP traffic between client and server of the application under review, it appeared that most of the response bodies were compressed and unfortunately not being decoded by Burp (despite the “unpack gzip” option being enabled). The client, a Java applet, relied on response data for a lot of interesting functionality (including access control) and having the ability to easily view and manipulate the contents in plaintext before being received by the applet was clearly beneficial (let’s ignore the obvious client-side security issue here ‘ this is a topic for another discussion).
As I mentioned earlier, it appeared the response content was compressed; however the expected Content-Encoding HTTP response header was not present. Inspection of the de-compiled Java applet code confirmed that compression was being performed with the java.util.zip.Deflater and java.util.zip.Inflater classes. At present, Burp Proxy does not support the ZLIB and DEFLATE compression formats (only GZIP compression is supported).
Burp is an essential tool in any web app testing toolkit and extending its functionality to inflate “deflate” compressed response content via the handy IBurpExtender interface seemed a worthwhile contribution. I hope others find the plug-in useful as well; at a minimum, it will be useful when the application returns for a round of regression testing.
The Burp plug-in can be downloaded here.
Also included with the download is an example servlet called “DeflateTestServlet” for generating HTTP responses bodies in the RFC1950 and RFC1951 compressed formats for testing the plug-in.
Also, here’s a good link that may help clarify your understanding of the compression formats used with HTTP.
