Sunday, September 13, 2009

Compressing ASP.NET Session and Cache State

A lot of ASP.NET web applications out there rely on Session and Cache. Session is used for storing user and session specific information (including logged in credentials, user's products, etc). Cache normally include application wide specific data. As mentioned in my previous article all this requires ViewState.

If you do require Session and Cache to a large degree in your web applications (and its a valid implementation) then I suggest looking at another Scott Hanselman article. He shows you how to implement ZIP compression on your Session and Cache data. He provided an extension (.NET 2 and higher) to compress and decompress data stored in these containers.

How to use it? Here is a snippet from his article on how easy it is:

1.Session["foo"] = "Session: this is a test of the emergency broadcast system.";
2.Zip.Session["foo"] = "ZipSession this is a test of the emergency broadcast system.";
3.string zipsession = Zip.Session["foo"];
4.Cache["foo"] = "Cache: this is a test of the emergency broadcast system.";
5.Zip.Cache["foo"] = "ZipCache: this is a test of the emergency broadcast system.";
6.string zipfoo = Zip.Cache["foo"];


Just by doing this you can store 5 to 10 times the amount of data for the same amount of ViewState. Another huge saving.

Compressing ViewState in ASP.NET

Since the days of ASP.NET 1.1 ViewState has been a sore point for many developers. ASP.NET has allowed many windows forms programmers to transition to web. They have included design time support and drag and drop functionality in ASP.NET that many junior and mid level developers use. The big downside of ASP.NET is the amount of ViewState that goes back and forth to the web server on EVERY postback.

One way is to rewrite your site using AJAX. And please not MS Ajax with their UpdatePanels. This may look like AJAX but you are still posting the entire ViewState to the server. Using AJAX will unfortunately require you to rewrite a lot of your web application.

If this is not an option you can try compressing the ViewState that ASP.NET uses. I found a nice article by Scott Hanselman that describes exactly this. He even went as far to provide a base page for us to implement. Unfortunately its in VB.NET but not everyone is perfect :). The beauty of .NET is that you can just extract this base class in different assembly and reference the DLL.

By using this method you can typically compress around 50% of your ViewState. That is a huge saving.

HTTP Compression IIS 6.0

HTTP compression is an easy way to save bandwidth and improve performance on your sites. The most common way is the enable GZIP on your web server. Using HTTP compression in IIS 6 is probably the easiest way to implement HTTP compression on your sites. You will need an administrator on the web server to perform these steps.

A good article can be found at http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/502ef631-3695-4616-b268-cbe7cf1351ce.mspx?mfr=true

You will need to install the IIS 6 Resource Toolkit in order to edit the IIS meta base. By default the GZIP compression ratio is set very low in IIS. To update it you will need to edit the IIS meta base. No additional tooling is currently provided to help you on this.

From listening to the guys on StackOverflow the magical Gzip number that you need to configure in IIS meta base is 4. This will depend on the files you are compressing and how much CPU you want to assign to the compression procedure. Naturally the higher the number the better compression results but the more intensive CPU cycles are required.

HTTP Compression Module

After my previous post I decided to spend some time researching the various ways to optimize ASP.NET applications. I found some interesting articles.

HTTP Compression Module:

I found this article on CodeProject. Compressing files before they are sent to the browser has many benefits. The page will load faster and less bandwidth is used, resulting in better user experience and reduced cost. There are a few different HTTP Compression module implementations out there, but this one adds to those by compressing WebResource.axd.

This implementation has three parts to it. One is the compression of Web pages. It uses a custom filter to replace JavaScript and stylesheet links. To use the code is pretty easy. Once you've compiled the HTTPCompress module, you can add a reference to any .NET 2.0 site. Next, you should set a few parameters in the web.config and you're good to go. If you want to see whether the content is really filtered, you will need something that logs the HTTP pipeline.

In the module, you need to hook up an event to the PostReleaseRequeststate. Before, I used to hook it up to the BeginRequest event but this executes way too early for the module to execute. At PostReleaseRequestState, the whole page has executed and its response content has been generated.

   41         public void Init(HttpApplication context)
42 {
43 context.PostReleaseRequestState+=
new EventHandler(context_PostReleaseRequestState);
44 }


Next is combining multiple JavaScript and CSS files into one file and compressing them. It is better to have one JavaScript file and one CSS file for the browser to download. This is because the browser can't use parallel downloading for these types of files. Each one must be downloaded before another can begin. But it is also difficult to maintain large JavaScript and CSS files. The solution is to keep them separate, and then combine them into one file dynamically. And then on top of that, compress them.

The HttpModule was updated to parse through the HTML of the page and replace the links to JavaScript and CSS files, combining wherever possible. It is now possible to write your pages like you normally would (in the past, to compress a script you would have to link to js.axd?files=jsfiletocompress.js) and the HttpModule will replace the link.

Imagine, instead of including five JavaScript files equaling 177 KB, you can include one file equaling 52 KB. For example, in the sample Web site, there are 5 JavaScript files: 2 ScriptResource.axd, 2 WebResource.axd, and 1 *.js file that combines 5 *.js files in the JavaScript folder. Uncompressed, these files would equal 329 KB, but compressed they equal 89 KB. Those are huge savings.

The third is the compression of WebResource.axd. All files are cached on the server and on the browser (including WebResource.axd). This way also enables you to include JavaScript or CSS from another domain and still compress and cache the files.

This method also supports MS Ajax UpdatePanels.

Using the code:

Add a reference to the DLL or drag and drop the DLL into the bin folder of the Web application. Add the following to the web.config. It should work transparently, meaning you can develop your site without the compression module, then add the compression later; or develop with the compression module and then remove it later without having to make any changes besides deleting the web.config items and the DLL.

<configSections>
<sectionGroup name="DCWeb">
<section name="HttpCompress" type="DC.Web.HttpCompress.Configuration,
DC.Web.HttpCompress"/>
</sectionGroup>
</configSections>

<DCWeb>
<HttpCompress compressionType="GZip">
<IncludedMimeTypes>
<add mime="text/html" />

</IncludedMimeTypes>
<!--<ExcludedMimeTypes>
<add mime="text/html" />
</ExcludedMimeTypes>-->
<ExcludedPaths>

<add path="~/NoCompress/Default.aspx" />
</ExcludedPaths>
</HttpCompress>
</DCWeb>

<!-- The js.axd and css.axd must be enabled to allow javascript and css
compression -->
<httpHandlers>
<add verb="*" path="js.axd,css.axd"
type="DC.Web.HttpCompress.CompressionHandler,DC.Web.HttpCompress"/>
</httpHandlers>
<!-- The compression module must be enabled for the WebResource.axd to be
compressed -->
<httpModules>
<add name="HttpCompressModule"
type="DC.Web.HttpCompress.HttpModule,DC.Web.HttpCompress"/>
</httpModules>

Thursday, September 3, 2009

Google Page Speed

I was listening to a HanselMinutes podcast who was interviewing the guys from StackOverflow. Apparently the guys at StackOverflow use an OpenSource tool from Google called Page Speed to better their web site. So naturally I decided to download and try it myself.

Wow. And double wow. This is an excellent tool. It works as an extension to Firebug in Firefox. Once you open Firebug on a web page there are 2 new tabs for Page Speed. You can analyze the performance of your web site. This will give you an overall performance summary of the web page and allow you to drill down into various facets.


Not only does this show you the problem areas of the web page but they also give you the solutions to solve them. With images they give you compressed image versions you can use. With Javascript they give you compressed versions to use. With other things they direct you to best practices pages that give you insight into how to improve the page.

The Show Resources link shows you a complete break down of which elements are being downloaded to the browser and their corresponding sizes with how long it took to download them. The Page Speed Activity tab lets you record activities on your page and it will give you a graph on which elements were executed at which time intervals.

This is a very nice tool that we will be incorporating in our development efforts and to ensure that best practices are being followed.

Development Processes

We have been working on implementing our new development processes in our company for the last 6 months. This has been a gradual process, starting with new projects. This was done to help align business and IT processes and strategies and put some structure in place to help better our development efforts and improve the probability for project success.

This new process is very agile and iterative. This allows business to be very involved and manage their expectations better. Those who know me know that I like to follow a DDD approach to software design and we apply this is our process. This allows us to work with the domain experts and establish a good conceptual model and an ubiquitous language that all parties understand.

Basic overview of the process:
  1. Project Kick off - Present the problem space and appoint the key role players
  2. Feature Spec and Requirement Analysis - Determine deliverables and user stories. Architects works closely with domain experts to establish a good conceptual model.
  3. Feasibility and Sign off - Determine if project is feasible and profitable. Time estimates, work break down and solution design.
  4. Development
  5. Testing
  6. Deployment and Sign off
  7. Review and Feed back into the process - very important to take what has been learned and adjust our processes if need be.
We have also introduced a research project where the outcome is feasiblity. This is done in large projects where the domain is a bit uncertain. This allows us to implement a proof of concept or at the very least give us the opportunity to see if the project will be feasible. The business stake holders will then decide if the project will go forward or not. With a relative small expense we can determine which projects will most likely not be successful.

Unfortunately our legacy systems did not follow this approach and thus there are still quite a bit bugs and enhancements that get requested and are sometimes unmanaged. We are trying to implement some processes regarding this as well but has not been too successful up until recently.

Over the last month we have started implementing SCRUM standup meetings in the mornings and things have already started going much better. We are identifying potential problems much sooner and have a complete record of what everyone is doing. This also helps knowledge sharing between problems. From these SCRUM sessions additional meetings are arranged if necessary to help solve problems identified.

In addition we have also introduced weekly post-motrem meetings where we evaluate the past week's progress and also the coming week's work. This allows us to determine what the pit falls were and help us avoid them in the future. This also allows us to schedule our resources more efficiently.

I believe IT processes are very important and I am happy working in a team that believes in this as well.