Software Books

I have been doing software development for about 4 and half years now. Before that I did a 4 year Engineering degree. Over the years I have read good books and I have read bad books. Books that have helped me and books that have wasted my time.

I believe books regarding technology, like C#, VB, Visual Studio, ASP, Java and so on, are very important for young developers to get started and up and running. After a year or so, these books become repetitive and redundant. After you have solidified your basics, any new technology is just a matter of reading a bit online and discussions with co-workers on the pros and cons of the specific technology.

Books that I believe that are extremely important are those books that are completely technology agnostic. These books cover the core principles around software development and best practices around using them. The following list are books that I believe are must-reads for any software developer that takes his career seriously:

• Domain Driven Design by Eric Evans
• Applying Domain Driven Design by Jimmy Nilson
• Patterns of Enterprise Application Architecture by Martin Fowler
• Refactoring to Patterns by Martin Fowler
• UML destilled by Martin Fowler
• xUnit test patterns by Martin Fowler
  
• Enterprise Integration Patterns by Gregor Hohpe and Bobby Wolfe
• Code Complete by Steve McConnell

These books have jump started my career. This book will teach you patterns and best practices to apply to any industry. These books give us a common language and vocabulary to use when solving problems and communicating to a team.

Take the time, add them to your library and help improve the software industry. The more people learn this, the better quality systems we will provide. Perhaps this list will help some people that have taken me a while to come to grips with.

Modern Software Failure

I was listening to a great HerdingCode podcast by Greg Young. He discusses how modern software is failing us. He says that these days we base our success on past failures.

He goes on to talk about how modern software has evolved to become a basic CRUD (Create/Read/Update/Delete) system. He proposes that if this is the type of systems we require then we might as well create an add-on for Enterprise Manager for SQL Server and let the users work directly on the tables. Sort of what Access gave us....

He argues that our software should not consist of how to store data but rather on the behavior that the client needs. A great example of the difference between CRUD system and DDD system:

A DDD person will chat to a domain expert on what the requirements are. The domain expert will reply that he needs the system to update an employee's salary. Greg Young argues that this is the wrong approach. We have trained clients over the years to speak in CRUD. What we should ask is what is the behavior associated with the salary increase? Did the employee get a promotion? Did the employee transfer between departments? Did the employee get a pay increase? All of these options have different rules associated with it.

So as a software engineer we should understand what the business requires and model of software according to that. Doing so we will avoid pit falls and make our software more descriptive and re-usable. We should not just tackle the domain logic with this approach but the UI as well. Design screens around tasks and not CRUD operations.

Some very nice things to think about. Check out the podcast, it could help you...

WCF, delving deeper...

Windows Communication Foundation (WCF)... This has been a buzz word for quite a while now, ever since .net 3.0 beta was released. I wonder if many people actually realize what WCF provides and what the actual benefits are.

I think most people are like me and have used it in the traditional web services (asmx and SOAP) sense, which is basically a glorified RPC framework. The reason I say this is that once you add WCF as a reference it actually generates a proxy (which is a copy of the contracts) on the client. It will use this proxy to make any calls. This is fine in most normal situations but what else does WCF provide?

Well, in short it has abstracted the way we look at services. We are no longer bound to the infrastructure code of writing over a channel. Thus the service is ambiguous to what channel it uses (SOAP, TCP, MSMQ, REST, etc). It also provides a more elegant way of dealing with cross cutting concerns like security, exception handling, end point addressing, etc. Thus with WCF you can concentrate on writing the required business processes and worry about the hosting details at a later stage. Basically taking the "Junk" out of the code. That is great, but is there more?

In short, Yes! My colleague and I were creating our own framework to support generic Integration Pattern concepts like Channels, EndPoints, Routers, Pipe And Filters, Channel Dispatchers, etc. This framework would allow us to easily scale and re-use common components.

We asked the question: Can WCF solve our infrastructure and messaging concerns in a enterprise integration system? On the surface the answer was No. Like most people we saw it as a RPC framework. But after looking a bit deeper we realized that there was so much more to WCF than originally thought. It supports numerous Messaging concepts, built into WCF, that could help us extremely much.

It supports the following:

• Channels (built in support for TCP, MSMQ, SOAP, REST and named pipe)
  
• End Points (synchronous, asynchronous, and dual binding for some channels)
• Channel Dispatchers
• Poison Queues
• Input and Output Queues
• Routers
• Pipe and Filters
• Message Transformation
  
• Transactions
  
• Cross cutting concerns: security, exception handling, etc
• Etc

These are just a few concepts of Enterprise Messaging Integration that WCF supports built-in. Unfortunately most books do not cover these kinds of topics. And most integration concepts are covered in Microsoft by technologies like BizTalk. It took some time to find relevant articles, but once we found it we were sold.

Here are two very nice articles to get you started which we found on MSDN magazine. These articles show you how WCF addressing works and how to build a simple pass through router, public subscribe router and even a content based router using a dynamic recipient list.

Part 1:
http://msdn.microsoft.com/en-us/magazine/cc500646.aspx

Part 2:
http://msdn.microsoft.com/en-us/magazine/cc546553.aspx

We are very pattern centric at our company and we are very pleasantly surprised that Microsoft have developed a framework which is so closely modeled to the Integration Patterns.

CodeRush Refactor! Pro

From hearing and watching the guys from DNR (dot net rocks) and DRNTV I have decided to try out the DevExpress CodeRush tools. This comes with Refactor! Pro. I must say I am quite impressed. It has all the refactor features from Resharper but with some added benefits.

It has some nice visuals with the refactor capabilities which helps a developer during his coding phases. One thing that stands out is the CodeRush tool panel. I normally dock this on the left hand side (luckily I have a wide screen). This allows you to see what short cut keys and refactor commands are available at any given time. As you type it will also filter the results. This is especially helpful when learning to use CodeRush.

They say that after about 5 days of using this you can increase productivity to about 25%. Must say, its worth a shot.

CodeRush supports refactoring for a large number of technologies including:

• C#, VB.NET, C++
• XML (especially nice for config files and WCF configurations)
• JavaScript
• ASP.NET

Check it out at
http://www.devexpress.com/Products/Visual_Studio_Add-in/Coding_Assistance/

It comes with a 30 day trial.

Gaming... a hobby

I have a couple of hobbies that I pursue. For those of you that like games (PC or Xbox360) please check out my other blog at:

http://musthavegame.blogspot.com/

Hope you guys have some fun with it.

WPF User Experiences

Wow. Just had a look at a DNR TV show (http://www.dnrtv.com/default.aspx?showNum=115) by Bill Hollis where he illustrates how to build great user experiences using WPF.

This is quite a mind shift from my usual black and white screens (and some will know me for my orange ones :)) using Microsoft Sans Serif fonts. The combination of layout, animations, design and patterns is mind blowing. You need to see it to realize how far technology has come from traditoiinal windows forms and ASP.NET forms.

You simply cannot purely focus on integration and business logic anymore. We will need to consider the UI integration and patterns as well. I think the MVVM patterns will be a good fit here as well as the powerful WPF routing capabilities and the power of layouts (especially list boxes).

Will seriously need to consider these kind of principles in all new projects going forward using Silverlight or WPF.

MEF

I might have become a MEF head.... and I know what you are thinking and the answer is no... I have not resorted to drugs to fix my coding problems. MEF stands for Microsoft Extensibility Framework.

I was watching an interesting DNR TV show yesterday (http://www.dnrtv.com/default.aspx?showNum=130). This show was on MEF. Unfortunately Glen Block did not provide a very good code example on how to implement MEF, however he did get the concepts across. I must say, I like the idea.

Basically, MEF allows you to "extend" existing sealed applications by dropping in DLLs. How this works is when an application needs a piece of functionality, it will signal it needs an assembly that does 'X', and MEF will go and find all assemblies that can fullfill that requirement.

This is how it works: Lets say you have an application that needs to perform some sort of comparison (could be String, Int32, Business object, etc). Instead of hard coding those comparisons into your application you can use the Specification pattern and inject an interface (lets say IComparison) into the class that needs to use that comparison. This can also be done using a Service Locator.

Thus, a shared assembly can be used to store these common interfaces that you need to use in your applications. There could be one or many assemblies that use this interface assembly to provide implementations. Lets call one of these assemblies 'Extensions'. This assembly provides two implementations of IComparison - StringComparison and Int32Comparison. These two classes implement the IComparison interface and implement the necessary methods (like Compare(object1, object 2)).

MEF resides in the System.ComponentModel namespace. In the Extensions assembly you add a reference to it, and mark the classes you want MEF to discover with an [Export] attribute. This attribute has various overloads:

• Just [Export()] allows you to make this specific class discoverable.
• [Export(typeof(IComparison))] will make this class discoverable through the IComparison interface
• [Export("SomeCategory")] will make this class discoverable using the custom string type.

The class is not limited to one category however, you can assign multiple [Export] attributes to each class if you so require and thus giving them different categories.

Now, the calling assembly (that needs to use the comparison) will have a reference to the shared assembly (where IComparison resides). It DOES NOT need a reference to the Extensions assembly.

MEF uses Catalogs to define where your assemblies reside - thus you can add it manually, use DirectoryLocator, etc - all these are built in - thus you can just tell MEF that my assemblies reside in some directory and it will do the loading for you. Then you instruct MEF that you require an implementation of IComparison to implement this comparison. MEF will then load those implementations (in our case two of them) into a variable which you have marked with [Import] attribute. You can then use these references as you would any class.

As you can see from this example, it can be very useful in Dependency Injection/Inversion of Control where you want to use Service Locators to fetch an implementation of interface for you. MEF allows you to do this and not worry about the details of dynamically loading DLLs, using refection, etc to get those implementations.

Check it out at http://www.codeplex.com/MEF

Event Sourcing

I read a very interesting article yesterday. We are busy with a POC on this and will probably be implementing this on one of our big projects coming up.

Marting fowler describes a new pattern called event sourcing (http://www.martinfowler.com/eaaDev/EventSourcing.html) which is based on the Domain Event pattern. This pattern challenges your thinking around traditional business application development.

The idea is that you have a domain model that is based on state. This state comes in from one or more input sources. These input sources generate events that come into your system. These events get routed to your EventLog.

The EventLog has three purposes:

• It persists all events it receives (in order) to some repository (in memory, database, etc).
• It generates events based on the incoming events and forward the events to the EventProcessor.
• On startup, it will get all available events persisted from the repository and reconstitute the state of the domain.

The EventProcessor propagates the events throughout your domain model.Once the domain receives the events, additional events can be triggered based on domain change. These events get routed to one or more output sources.

What this gives us is that the domain model is completed driven around events. This is especially usefull when working in a publish/subscribe model where your system is receiving broadcast messages (like order price change) from a broadcaster. Only the relevant data is pushed to your system. Because we have a complete log (in order) of events in a repository we can provide rollback or playback functionality. If the system dies for some reason, on startup the domain will reload itself from the repository and be in the correct state because the order of events were persisted.

A very interesting model to adapt when working in a messaging system.

Code Generation in .Net

When developers think of code generation they think generally of ORM tools like LinqToSql or Entity Framework. My feelings on these technologies are putting it midly, "crap".

I was listening to an interesting podcast this morning on DotNetRocks with Peter Vogel (http://www.dotnetrocks.com/default.aspx?showNum=453) in which he explains other uses of code generation in .Net.

An interesting point he made (and which I think is very applicable) was avoiding to write redundant code. He was not referring to ORM but rather other functions like reading information from a web.config or app.config file. The example he gave was generating code for connection string settings in app.config file. Every time he saves an app.config file, his code generation tool kicks off and generates a ConfigurationManager class which generated code to the settings in the config file.

An example is having multiple connection strings in the config file. For instance, DevConnectionString and StagingConnectionString. His code generation tool will then allow him to access these properties using type safety and early binding. Thus, in code, he can call ConfigurationManager.ConnectionStrings.DevConnectionString or ConnectionStrings.StagingConnectionString and thus avoiding the annoying problems of miss spelling the connection string.

Maybe something to take a look at and invest in. He mentioned a variety of code generation tools in .Net including T4.

Enterprise Integration Patterns: Overview

I would like to cover an overview of the general concepts in Enterprise Integration Patterns in messaging systems. I will cover each topic in more detail in further posts, but lets start at the basics.

Figure 1: Message Router routes messages from input channel to one or more output channels

Firstly, where would you use these Integration Patterns? Well anytime you want to move information from one system to another you can use these patterns. These patterns are extremely useful if you have to access data that is contained in a legacy system or if one system is "sealed" and you cannot easily access it's data (through API, database, etc).

The first challenge and the major selling point of Enterprise Integration is that you want the 2 or more systems to be as loosely coupled as possible i.e you don't want one system having a direct reference to the other system. Why do you ask? Any change in business rules or the addition of additional recipients will result in code changes at numerous places. This makes it very inflexible.

There are two main flavors of messaging:

• Request/Reply: An example is you needing customer information that resides in different CRM system. A request message will be sent to the CRM requesting information. The CRM system will send a reply containing the information.
  
• One way: An example is that you have an event that occurred in your system that you want to broadcast to other systems. These systems do not have to acknowledge or reply to your broadcast.

Messaging consists out of a couple of main concepts:

• Channels: Messaging systems transmit data through a Message Channel - a virtual pipe that connects a sender and a receiver.
  
• Messages: Message is an atomic packet of data that can be transmitted on a channel.
  
• Pipes and Filters: Break up various operations between sender and receiver by chainging operations using pipes and filters.
  
• Routing: Router receives a message on input channel and determines how to navigate the channel topology and directs the message to the final receiver.
  
• Transformation: When applications do not agree on the format for piece of data, use a transformer to map one message type to another.
  
• Endpoints: An endpoint bridges the gap between how the application works and how the messaging system works

Thus to illustrate we can use the request reply example. An ordering system needs information stored in CRM system when placing orders. The ordering system would create a request message (CustomerId=1 request) and place this message on a channel (MSMQ as an example). The router will receive this message of the queue, determine the type of message and route it to the CRM system endpoint. The endpoint will receive the message and use a Channel Adapter (discussed in future post) to act as anti corruption layer and transform the message into an object the CRM system understands. The CRM system will process the message, retrieve the valid customer information and send that information via a response message to the Reply Address specified in the original request message. The route will pick up the reponse message, perform any needed operations on it and send it off to the ordering system endpoint.

In further posts I will go into detail on each of these topics by illustrating why they are important and when and where you will use them. I will also show a couple of different flavors on each of them.