Monday 29 December 2014

Theory Of Constraints And Software Engineering (Improving the throughput)

About one year ago or so I was introduced to the Theory Of Constraints (TOC). It was one of the best things that have ever happened to me as a manager. It changed my view on everything,  It was all thanks to one of my colleagues coming back from a conference and ordering in a book called the "Phoenix Project", the only reason why I was excited about reading it was because of its funky cover (Figure 1).

Figure 1. Phoenix Project Book Cover
As soon as I started to read this book I was instantly hooked, I didn't know at the time that I was going to discover something very important.

In this article I will briefly talk about my understanding of TOC and how I have applied it at work.

Right, let's jump in.

Imagine for a second the following situation. You are an owner of a digital agency, your agency builds web applications only for e-commerce/marketing companies. The following disciplines work for your agency: 10 business analysts, 7 designers,  10 testers, 15 software engineers and 2 web developers. Due to type of work that this agency does (front-end heavy) it should be obvious to us that our constraint in this case will be our web developers. This means no matter how hard other disciplines will work the only work that will be actually shipped to our clients will be what web developers had time to work on. All other work that was done independently by other disciplines will be stored away, waiting for web developers to catch up. This means that your overall throughput is completely constrained by your web developers. 

Let's resolve above situation by using "TOC 5 Focusing Steps" (5FS):
  1. Identify - Find your constraint (you can use Kanban, Utilisation charts, etc), in our case web development is our constraint.
  2. Exploit - Now we need to find out exactly what process this discipline is following, what is making this disciplines life painful, is it a build server? Is IDE not working correctly? Are web developers too involved in the backend development and backend developers should be doing more? In other words offer all the help that you possibly can. Make sure that web developers have always some work to do, after all they are the constraint. 
  3. Subordinate - Make it all about your constraint, only do as much work as a constraint can handle and avoid at all cost the pile up of inventory, this means that disciplines before and after the constraint will be idle. This is not a problem, there will be a much bigger problem if these people are not idle! Now you will need to start to set work in progress limits with buffers, in TOC world it's known as "Drum-Buffer-Rope". 
  4. Elevate - Hire more people in, change the process, transfer some of the existing staff in to the constrained discipline, do what ever you can to break this constraint.
  5. Repeat - By now your constraint should have been broken, so you need to find your next constraint. Don't allow inertia to become a system constraint. 
This video does a great job demonstrating TOC. TOC also focuses on inventory and throughput accounting, I am not going to talk about throughput accounting but I am going to talk about inventory.

Inventory in software engineering is the following:
  • Work In Progress (anything that is being developed in the current iteration)
  • Unassembled or partially assembled work (POCs, Shelved code, Designs, Wireframes, Business requirements, etc)
  • Backlog items / Requirements / Bugs


Figure 2. Lots Of Inventory = Lots Of Waste
This is what happens when you have lots of inventory in the system:
  1. What was produced is no longer needed.
  2. What was produced was wrong.
  3. Need to juggle lots of work at the same  time (context switching / multi-tasking).
  4. What was produced needs to be relearnt again.
  5. It becomes extremely hard to keep track of versions, projects and roadmaps. More processes and bureaucracy will be added to keep everything under "control".
As inventory builds up, waste builds up and it compounds the problem.

One of the ways to remove inventory build up is by cutting out "hand-shake / handover" periods between different disciplines. For this to happen teams need to work together as one unit delivering shippable software every iteration. In the Agile world these teams are also known as feature teams (multidisciplinary teams) they produce inventory just-in-time.

After reading lots about TOC I as a Product Owner have decided to give it a try with one of my feature teams at work.

Here is how we have applied TOC and some other good practices:
  • We have acknowledged testers as our constraint, these guys had to do manual and automation testing (Identify).
  • We have helped them by identifying what problems they were facing most often, in our case it was a build server was taking too long to build and to publish the latest application (Exploit).
  • We have knowingly created a bit of inventory for the next iteration each time, this way testing had something to work on straight away as soon as iteration started, this ensured that they are busy at all times (Exploit).
  • To deliver to testing team faster we have broken our requirements down, this ensured continuous flow (Exploit).
  • Testing was in the loop at all times so there were no surprises with requirements (Communication).
  • We never undertook any work that was vague or unknown (Risk reduction).
  • If we couldn't estimate work or write down a good acceptance criteria for it (because it was unknown) we would create a time boxed investigation task (Risk reduction).
  • If there was a quality issue or something unknown came up during development, team would stop the line and get an answer immediately (Communication).
  • Certain type of work would be done in different parts of iteration, for example at the end of iteration testers would be regression testing, this was an ideal time for other disciplines to do some infrastructure work or training (Subordinate).
  • We did only as much work as testing team could handle (Subordinate).
  • We were looking to hire more testers (Elevate).

Figure 3. Stop The Line.

Above steps have really helped us to improve our throughput, but it also did way more than that. This team was:
  • One of the most efficient teams in the department.
  • Everyone always left on time / no one ever had to do overtime.
  • They have finished ahead of the schedule at worst they have finished on time.
  • As they have finished ahead or on time they had time to groom requirements and get ready for the next iteration (maybe even start work ahead of the schedule).
  • Team has gelled incredibly.
Of course none of this would not have been possible if I didn't have incredible people in the team, their personalities have played the key role.

Summary: 
  • Stop doing work upfront per discipline (creating inventory) and star to work together as one multi-disciplined team (feature team) by using continuous flow / just-in-time delivery.
  • Have your team deliver production ready software every iteration.
  • Break down your requirements in to small deliverables (continuous flow) to achieve full multi-disciplined team utilisation. 
  • Use Kanban board / Utilisation charts to visualise your constraints. 
  • Use TOC 5FS to increase your teams throughput.
  • Ensure that your constraint is never starved of work and is protected from distraction, inefficient processes and bureaucracy. 

Monday 1 December 2014

Applied Domain-Driven Design (DDD), Part 0 - Requirements and Modelling

About a year ago I have written a series of articles about Domain-driven design, you can find the main article here. Looking back I've realised that I've committed the most typical mistake and started to code my business domain without requirements or any draft designs!

I am going to try and fix this, this is what I am going to do:
  • Identify User Stories 
  • Identify the Nouns in the user stories 
  • Identify the Verbs in the user stories 
  • Put together object interaction diagram
  • Put together object responsibilities diagram 
  • Put together class digram UML showing only interesting interactions 

So here are made up user stories:
  • As a customer I want to be able to put products that I want to purchase in to the shopping cart so that I can check out quickly later on
  • As a customer I want to see the total cost all for all of the items that are in my cart so that I see if I can afford to buy everything 
  • As a customer I want to see the total cost of each item in the shopping cart so that I can re-check the price for items
  • As a customer I want to see the total cost for all of the items in the shopping cart with total tax 
  • As a customer I want to be able to specify the address of where all of the products are going to be sent to
  • As a customer I want to be able to add a note to the delivery address so that I can provide special instructions to the postman
  • As a customer I want to be able to specify my credit card information during check out so that I can pay for the items
  • As a customer I want system to tell me how many items are in stock so that I know how many items I can purchase 
  • As a customer I want shopping cart to check that items are still available for purchase during a check out so that I can still purchase items that are in the cart
  • As a customer I want to receive order confirmation email with order number so that I have proof of purchase 
  • As a customer I want to specify invoice address for the order so that I can receive invoice for the order

Now I am going extract nouns and verbs from the stories above. I am looking for the nouns that will become my main objects and not the attributes.

Nouns: 
  • Customer
  • Item
  • Order
  • Shopping Cart
  • Address
  • Invoice
  • Delivery
  • Tax
  • Credit Card Information 

*Note: I've removed duplicates for better, more official names, for example Item = Product, Order = Purchase, etc.

Verbs:
  • Put products in to the shopping cart 
  • See total cost for all of the items
  • See total cost for each item
  • See total tax for my country 
  • Specify delivery address
  • Specify delivery note for delivery address
  • Specify invoice address
  • Receive invoice for the order
  • Sent invoice
  • Specify credit card information 
  • Pay for the items
  • Tell me how many items are in stock
  • Check that items are still available during check out
  • Receive order confirmation email 

By using above nouns and verbs we can put together a diagram such as this:
[Figure 1] Object Interaction Diagram 

Once we have object interaction diagram we can start thinking about object responsibilities. One of the most common mistakes is to push responsibilities on to the actor object i.e. Customer. We need to remember that objects must take care of themselves and objects need to be closed for direct communication and that you need go through the functions to communicate with them.

So let's follow above approach and assign responsibilities:

[Figure 2] Object Responsibilities Diagram 

Now that we have object interaction and responsibilities diagram in place we can start thinking about lower level UML class diagram:


[Figure 3] UML Diagram

Figure 3 shows methods, class names, dependencies, interfaces and composition. I've took a bit of time and reflected only on the most complex / interesting parts of the model. I will worry about attributes and other details later on, detail will naturally emerge when I start coding. Figure 3 is suppose to be a rough sketch, that is all, teams can whiteboard Figure 3 during a meeting, take a picture and distribute it to everyone in the team and get on with the actual coding. After a week or so picture will be forgotten and the parts of the above model (that have been useful) will live and breath in the actual code.

Now my made up user stories can be modelled in my many different ways and Figure 3 is just my interpretation of it. Key thing is to think about what you are building first, don't just jump in and start coding and don't get carried away with detail either (attributes, constructors, etc) focus on interesting and complex parts first.

Summary:
  • Don't start doing anything until you have requirements, if you don't have a BA in the company that's fine, you will have to do BA's job and identify requirements first.
  • Don't just jump in to the code soon as you have requirements, put together object interaction and responsibilities diagrams first. 
  • When you have identified your objects, interactions and responsibilities use UML class diagrams to put together a draft model (whiteboard sketch will do).
  • Don't try to model the reality of the world, model the reality of your organisation. Different companies will have different objects, in one company "address" might be an object and you might have "address type" coming of it (invoice, shipping, etc), in another company there will be "invoice address", "shipping address" and "seller address" object, that company might need these objects as these objects will inherit from the base "address" object. Remember it is all about your business domain and not the actual "reality".

Useful links:

Tuesday 14 January 2014

Solution for MVC Ajax.BeginForm throwing SecurityPermission due to medium trust Level

I've written a very simple MVC 5 application and deployed it to my 1and1 hosting provider. Immediately I was struck by an error. I've forgotten that 1and1 like many other hosting providers enforce medium trust level.

I've gone to my local project and modified the config to simulate this problem locally. Line of code that threw an error was:
@using (Ajax.BeginForm("GetInTouch", "Home", new AjaxOptions { HttpMethod = "POST", InsertionMode = InsertionMode.Replace, UpdateTargetId = "makeContact"}))
{

}

Stack trace:
 
Request for the permission of type 'System.Security.Permissions.SecurityPermission, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed

at System.Security.CodeAccessSecurityEngine.Check(Object demand, StackCrawlMark& stackMark, Boolean isPermSet)
    at System.Security.CodeAccessSecurityEngine.Check(CodeAccessPermission cap, StackCrawlMark& stackMark)
    at System.Security.CodeAccessPermission.Demand()
    at System.Web.HttpContext.System.IServiceProvider.GetService(Type service)
    at System.Web.HttpContextWrapper.GetService(Type serviceType)
    at System.Web.WebPages.UrlRewriterHelper.IsUrlRewriterTurnedOn(HttpContextBase httpContext)
    at System.Web.WebPages.UrlRewriterHelper.WasRequestRewritten(HttpContextBase httpContext)
    at System.Web.WebPages.UrlUtil.GenerateClientUrlInternal(HttpContextBase httpContext, String contentPath)
    at System.Web.WebPages.UrlUtil.GenerateClientUrl(HttpContextBase httpContext, String contentPath)
    at System.Web.Mvc.UrlHelper.GenerateUrl(String routeName, String actionName, String controllerName, RouteValueDictionary routeValues, RouteCollection routeCollection, RequestContext requestContext, Boolean includeImplicitMvcValues)
    at System.Web.Mvc.Ajax.AjaxExtensions.BeginForm(AjaxHelper ajaxHelper, String actionName, String controllerName, RouteValueDictionary routeValues, AjaxOptions ajaxOptions, IDictionary`2 htmlAttributes)

I was shocked 1and1 hosting provider said that they support MVC 3 and 4, .NET Framework 3.5 and 4. All I did was build a super simple website with a contact form. What's happening?

Line System.Web.HttpContext.System.IServiceProvider.GetService(Type service) indicates that it's using some kind of a internal dependency injection. Dependency injection normally uses reflection to build objects, reflection is not supported in medium trust environments.

Taking a closer look at the stack trace we see that it's using System.Web.WebPages.UrlUtil.GenerateClientUrl(HttpContextBase httpContext, String contentPath). It's using Html.BeginForm(method, controller) to build the URL for the form.

Now, we identified the problem, where and why it's happening. So what's the solution? Not to use helpers? Create your own helpers? These are all possible solutions. Luckily, I am using Ajax helper so I can get away with the following for now:

@using (Ajax.BeginForm(new AjaxOptions { HttpMethod = "POST", InsertionMode = InsertionMode.Replace, UpdateTargetId = "makeContact", Url = "Home/GetInTouch"}))
{

}

As URL is specified manually (Url = "Home/GetInTouch") it's not going to try and generate it. This still leaves us with other problems.

You can't use Html.BeginForm(action, controller) as it's going to try and resolve these URL's. If you go with Ajax solution above and client has no JavaScript nothing will happen, form action will point to "/". This is not great, at least now you have an answer and a partial solution.

If you have found a better way around this, please do comment.