Thursday, 9 March 2017

Cloud Architecture & Engineering, B2B Revolution and B2C Evolution


This year I will be posting series of articles about Cloud architecture, it will be my attempt to make multitenancy, cloud and SaaS less theoretical. Articles that have been / being written:

Few years ago I was having a conversation with one of my friends about cloud, SaaS and multitenancy architecture. I was telling him how different it was to normal “hosted” app architecture. That we can setup new business tenant with custom url in seconds. He has said, well, instant tenant provisioning and custom URLs are not new. So what’s so special about it? Blogger and WordPress have been doing it for years. At that point in time I have struggled to answer his question, there was too much to say, ideas in my head were too raw. In this article I am going to explain why cloud, SaaS and multitenancy is revolutionary.



Before we start, let’s get few definitions out of the way:

Model
Description
Example
B2B, Business Applications Business sells to businesses Oracle, Visual Studio Team Services, Sage & Salesforce
B2C, Consumer Applications Business sells to consumers Spotify, Netflix & Blogger.

Your solution’s technical requirements will differ a lot depending on what business model you follow, we will be compare typical B2C and B2B requirements shortly.

Term
Somewhat My Definition
Software as a Service (SaaS) This is where you don’t care about software versioning, upgrades, deployments, where it sits, etc. You just pay for it and use it. If you are using Spotify, you are using SaaS.
Cloud Before cloud we had hosting companies, where you could rent a VM and host your site. Now we have Cloud, which is the same thing but it’s just much smarter and comes with lots of tools. Companies such as Microsoft offer prepackaged services that seamlessly scale your apps horizontally and all you have to do is just upload your binaries via FTP to a folder.
Multitenancy Traditionally each company would get their own “installation”, with multitenancy there is only one installation and everyone is seamlessly using it.

These concepts are very important as they will feed of each other, as I will illustrate this later on.
People use cloud, multitenancy and SaaS as synonyms and this is a mistake (I make this mistake as well).

B2C, Consumer Applications


When it comes to selling directly to customers things are easier, there are a lot less hoops to jump through. Here are some technical requirements that you might need to follow when you build B2C app:

Requirement Description
Availability Consumer facing sites, need to be highly available, if they are not, this can cause reputation damage and loss in revenue.
Authentication Consumers need to be able to login using some form of login, Google, Facebook, etc.
PCI DSS When customers make payments it’s important that card data is used & stored securely.
Recovery Point & Time Objectives Similiar to availability it’s critical that if data was corrupted or upgrade has gone wrong that app can recover quickly with as much data as possible.


B2C architecture before cloud:


B2C architecture after cloud:



This list is not complete, however the point is that typical B2C apps don’t have lots of technical requirements enforced upon them from consumers. This is why B2C service is so much cheaper, how much do you pay for Netflix and Spotify?


B2B, Business Applications


Here are some technical requirements that businesses often enforce on to other businesses:

Requirement Description
Corporate Identity Integration (Single Sign On Authentication) Medium/Large size companies have their own identity apps such as AzureAD, they don’t want to use your built in authentication mechanism if they can enable Single Sign On. This is where Federated identity comes in.
Audit They would like assurance and full trail of what’s happening inside the system, so they will ask for audit.
Data restore & Backup If company makes a mistake in their environment they will want to restore their data back to how it was.
Integration Often they have their own systems and they will want to integrate their existing applications with your application.
Security clearance Depending on the industry your business customers might want your ops staff to be security cleared, if this is not possible then sensitive data needs to be hidden or encrypted.
Escrow Businesses need to ensure businesses continuity, to do this they need to be able keep going even your business will go out of business.
ISO 27001 Key management, Antivirus, Firewalls, Segregation of duties, Pen testing, etc.
Reporting Access to data to be able to report on it.
Customisation & Configuration Data retention, authorisation, workflow, etc.
Availability Business sites don’t necessarily need to be as highly available as in a lot of industries people work 9am-5pm.
PCI DSS When businesses make payments it’s important that card data is used & stored securely.
Recovery Point & Time It’s critical that if data was corrupted or upgrade has gone wrong that app can recover quickly, business finance or patient data can’t be lost, huge amount of productivity can be lost if this not designed correctly.
Support Be able to troubleshoot issues quickly.
On-prem installation Company is able to install your software in their own environment.

This list is not complete, however if you compare B2B to B2C it’s easy to see that there is a lot more to it. So when companies charge other companies a lot more for their services we can appreciate why this happens.


B2B architecture before cloud and multitenancy:




B2B architecture after cloud and multitenancy:


So where is the revolution?


Consumer facing sites were doing SaaS with multitenancy for years. Websites like YouTube and Ebay have been scaling their solutions for a long time. Cloud has just made it easier for them to scale and in turn it has reduced their hosting cost. Business applications were doing SaaS for a long time, however multitenancy and cloud was just not a thing. Most service providers painfully installed software for each customer manually, then, they upgraded each customer one at the time and they monitored each customer one at the time. You get the picture. Customer provisioning took weeks and maybe months. Multitenancy offers something truly awesome, it removes slow per customer installation and upgrade problem and in turn it dramatically speeds up the software delivery and customer onboarding process. Cloud removes the infrastructure maintenance and provisioning burden. Together they simply transform business in a remarkable way. B2B apps can finally compete with B2C apps in terms of feature delivery speed and cost.

Conclusion


Let’s come back to the conversation with my friend. He thought that nothing was new here because he was thinking of B2C evolution. While I was thinking of B2B where multitenancy and cloud is transforming businesses and revolutionising business applications.

Sunday, 12 February 2017

Azure App Services Web App Antivirus - Concept

Introduction


Good web application antivirus solution needs to meet the following acceptance criteria:
  • Real-time inbound and outbound file scanning
  • Low latency, sub second performance
  • Transparent, no need to make changes to your web app
  • Requires no maintenance and is easy to setup

If you are using PaaS service such as Azure Web Apps your antivirus options are:

Option Limitation Installation Effort Maintenance Effort Cost Meets criteria
Firewall with ICAP integration Expense and ongoing maintenance High Medium High Yes
Send files to the antivirus API from your app Extra unnecessary development work Medium/High Medium Low No
Background service that scans your files Not real time, high possibility that viruses will be uploaded and downloaded Medium Low Low No
Reverse proxy binary content forwarding Scans only uploads Medium Low Low No
IIS Filter (this solution) PoC, not production ready Medium Low Low Yes

In depth exploration of different antivirus options is beyond the scope this article.


IIS Filter Solution, inspired by ModSecurity





This article will only cover the interesting parts of the solution, if you have questions please do comment.


AVFilter, File Upload/Download Filter


Filtering uploads and downloads is relatively straight forward thanks to IIS http modules, read more about them here.


Filtering Uploads


Here is an example of how you can detect file upload inside the http module:
 private void Context_PreRequestHandlerExecute(object sender, EventArgs e)
  {
      HttpApplication httpApplication = (HttpApplication)sender;
      HttpRequest httpRequest = httpApplication.Context.Request;
      
      if (httpRequest.RequestType == WebRequestMethods.Http.Post)
      {
          if (httpRequest.Files.Count != 0)
          {
              //Check files
          }
      }
  }
Take a look at the full implementation here.


Filtering Downloads


When it comes to intercepting downloads it’s not as simple. Unfortunately outgoing traffic is written to the System.Web.HttpResponseStreamFilterSink and this stream is write only, so you can’t read it. So the only option that I and Google can think of is to proxy the outgoing stream. To make this happen I have created proxy class that changes write only steam to write/read stream, I have conveniently called it ReadWriteProxyStream, take a look at the implementation here.

Now that we have ReadWriteProxyStream class, we need to hook it up response filter, this is done when web request is first received:
    private void Context_PreRequestHandlerExecute(object sender, EventArgs e)
    {
        HttpApplication httpApplication = (HttpApplication)sender;
        
        httpApplication.Context.Response.Filter = new ReadWriteProxyStream(httpApplication.Context.Response.Filter);
    }
Take a look at the full implementation here.

Before web response is returned back to user we need to check it, we can do this by reading the stream:
    private void Context_PreSendRequestContent(object sender, EventArgs e)
    {
        HttpApplication httpApp = (HttpApplication)sender;
        HttpResponse httpResponse = httpApp.Context.Response;
        
        if(httpResponse.ContentType == "application/octet-stream")
        {
            ReadWriteProxyStream filter = httpResponse.Filter as ReadWriteProxyStream;

            if (filter != null)
            {
                byte[] data = new byte[filter.Length];
                filter.Position = 0;
                filter.Read(data, 0, data.Length);

                //Check files
            }
        }
    }
Take a look at the full implementation here.


AVFileReceiver, Virus Scanning


For this proof of concept I have used Symantec virus scanner which comes with CLI, and to test it I have used EICAR test virus. EICAR test virus is harmless, it can’t actually do anything to your machine.

Symantec CLI will scan the specified file synchronously, so if the file is no longer there you know that it was not safe:
  using (Process process = Process.Start(
        "C:\\Program Files (x86)\\Symantec\\Symantec Endpoint Protection\\DoScan.exe",
        "/ScanFile " + filePath))
    {
        process.WaitForExit();
    }

    if (!File.Exists(filePath))
        throw new Exception("File was not safe!")
Take a look at the full implementation here.



Final Thoughts


If you like this concept then please do share and rate this blog post. If this post will generate enough interest I will productionise this concept. However, if you just can’t wait, then here is what you need to do to make this production ready:
  • Secure communication between AVFilter and AVFileReceiver
  • Secure AVFileReceiver access (authentication, network security group, etc)
  • Reduce VM maintenance through automation, use PowerShell files provided and place Cloud Service inside availability and upgrade group
  • Reduce memory consumption by intercepting binary responses only
  • Make debugging easier by logging exceptions
  • Improve performance by forwarding binary content using MTOM encoding to the AVFileReceiver and remove AVFilter temp storage
  • Improve scalability and responsiveness by using Async in the AVFileReceiver and AVFilter