Unfortunately Dot Net Solutions will not be at MIX this year, at least not in body even if we will be in spirit. Our absence only serves to drive our commitment to contribute to the event and its delegates.

Following on from the highly successful PDC session gadget that we released for PDC last year we are pleased to announce the availability of our MIX 10 Session Gadget!

Install this on your Windows 7 (or Windows Vista) laptop and you can view session information for each day, search for sessions by keyword and stay up-to-date with constantly updating session schedules. The gadget also works offline. Enjoy MIX!

I’ve been speaking about Azure at a number of events recently. One of the biggest challenges that I have had each time is due to local firewall restrictions. Firewalls will block port 1433/1434 outbound and this a major hassle to get changed, especially for a 20 minute demo. These ports are generally restricted due to the SQL slammer and SQL snake viruses (http://www.grc.com/port_1434.htm) from a few years ago.

SQL Azure uses port 1433 for communication, for instance when connecting from management studio to SQL Azure you get a dialog such as this:

With the port blocked, the connection attempt simply times out. One simple solution is to create a VPN connection to another network that has firewall rules configured to allow access. While this works, it does add latency, not only, to the SQL Azure communication but to all internet communication. However, there are other approaches.

Port Bridge

If you haven’t read Clemens Vasters’ post on Port Bridge, then I can highly recommend it: http://vasters.com/clemensv/CommentView.aspx?guid=3e35d8bd-b755-453f-8c63-1a57c570eb4c. Port Bridge is a sample application that provides point to point tunnelling between a client and server. It does this by wrapping the protocol and port used for the communication within a SOAP message. This message can then be relayed using the Windows Azure AppFabric Service Bus and unwrapped at the destination.

The Service Bus provides the ability to connect on-premise applications together or to connect an on-premise application to an application in the cloud. The Service Bus addresses the challenges presented by firewalls, NATs and dynamic IP, so your endpoints do not have to be published directly to the Internet just have Internet access.

The diagram is from Clemens’ original port and describes exactly the situation and solution to my current issue, where SQL Server is actually SQL Azure.

Using Port Bridge

Back to the original challenge: to demo SQL Azure by connecting from SQL Server Management Studio and an application on my local machine.

My office network is set up so that I have access to SQL Azure, so my first approach was to run the Port Bridge Service from my PC back in the office. With the Port Bridge Agent running on my demo laptop listening on port 1433. To set up the Port Bridge Agent to proxy message to SQL Azure, configure the local TCP port and remote TCP port to 1433 and set the target host as the SQL Azure database server name.

  <portBridgeAgent serviceBusNamespace="myservice" 
serviceBusIssuerName="owner" serviceBusIssuerSecret="x1x1x1x1x1x1xx1x1x1=">
    <portMappings>
      <port localTcpPort="1433" targetHost="x2wqed1zpk.database.windows.net" 
remoteTcpPort="1433">
        <firewallRules>
          <rule source="255.255.255.255" /> <!-- My local IP address -->
        </firewallRules>
      </port>
    </portMappings>
  </portBridgeAgent>

In order for the Port Bridge Agent to listen on port 1433, I need to ensure that anything else listening on port 1433 is stopped (such as the SQL Server service). To connect to the SQL Azure database I need to specify “localhost” as the server name when connecting and provide the login/password as usual. This traffic is intercepted by the Port Bridge Agent and routed via the Service Bus to the Port Bridge Service and then to the target host, “x2wqed1zpk.database.windows.net”, the SQL Azure database.

Using Port Bridge in Windows Azure

While I can conveniently run the Port Bridge Service on my PC as a windows service (or a console application), it would be useful to have a little more control over when this service is running and a little more resilience. These are exactly the features that a Windows Azure worker role would provide. I would be able to stop and start the Port Bridge Service remotely and would not need any access to my PC or my office network.

Starting from the Port Bridge sample application, to create a Port Bridge Service worker role requires a few steps.

• Add a new Cloud project and a worker role to the sample solution.
• To the worker role, add references to the “Microsoft.Samples.ServiceBus.Connections” and “PortBridge” projects in the solution.
• Also add a reference to Microsoft.ServiceBus.dll, ensuring that you set the “Copy Local” property to true.
• With a little refactoring of the PortBridge Program Main, this method can be used from the worker role (there is certainly scope to provide a much neater solution).
• Finally, update the app.config with the service settings:

  <portBridge serviceBusNamespace="myservice" serviceBusIssuerName="owner" 
serviceBusIssuerSecret="x1x1x1x1x1x1xx1x1x1=">
    <hostMappings>
      <add targetHost="x2wqed1zpk.database.windows.net" allowedPorts="1433" 
allowedPipes="sql/query" />
    </hostMappings>
  </portBridge>

NB: The Windows Azure instances do not have a copy of the Microsoft.ServiceBus.dll assembly installed in the GAC. Therefore to ensure that the application will run correctly, a copy of this assembly must be deployed as part of the package. This is achieved by setting the “Copy Local” property to true. This is one of the differences between the development fabric and the live Windows Azure instance.

With the Port Bridge Service deployed to a worker role, I can now connect to SQL Azure from my demo laptop without the challenges presented by firewalls. By using a Worker Role I also reduce the network latency as all the components are hosted in the Cloud. By ensuring that I select the same data centre i.e. North Europe for all three components, this adds very little latency to the whole communication compared to direct SQL Azure access.

Logging diagnostic information plays a key part for any application. The .Net framework provides a number of Diagnostic features, which can be used to provide logging functionality in your application.

However, in Windows Azure, the diagnostic API available in early CTPs was very limited. In the November release of Windows Azure, a new feature for  Windows Azure Diagnostics was launched. I will not delve into details of how to use the Azure diagnostic API. You can read about this here and also here.

In this article, I will be focusing only on a recent challenge we came across while implementing the Windows Azure Diagnostics API for ScrumWall (http://scrumwall.cloudapp.net). ScrumWall uses a factory pattern to determine which concrete log writer to use. It had both a trace listener logger and a custom Windows Azure Table storage logger. The trace listeners implementation could be configured in web.config.

The section for trace listeners in web.config for ScrumWall is:

<system.diagnostics>
    <switches>
      <!-- Off = 0, Error 1, Warning = 2, Info = 3,

 Verbose = 4 -->
      <add name="logLevel" value="2" />
    </switches>
    <trace autoflush="true" indentsize="2">
      <listeners>
       <!-- add file logger outside Azure -->
        <add name="TextWriterTraceListener" 
             type="System.Diagnostics.TextWriterTraceListener" 
             initializeData="TextWriterOutput.log"/>
      </listeners>
    </trace>
</system.diagnostics> 

A static Logger class provides a facade which writes to the underlying trace listeners using the static overloaded method of System.Diagnostics.Trace class WriteLine(string message) using our TraceListenerLogWriter:

public class TraceListenerLogWriter : ILogWriter
{
    private static TraceSwitch _logLevelConfigured = 
        new TraceSwitch("logLevel", "Switch log level");

    public void WriteLog(string message)
    {
        Trace.WriteLine(message);
    }

    public bool ShouldLog(TraceLevel traceLevel)
    {
        if ((int)_logLevelConfigured.Level >= (int)traceLevel)
        {
            return true;
        }
        else
        {
            return false;
        }
    }
}

The Windows Azure trace listener is provided by “DiagnosticMonitorTraceListener”, which we added in the existing web.config :

<!-- add file logger for Azure -->
<add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
     name="AzureDiagnostics">          
</add>

and in our WebRole, which  is responsible for Diagnostic Monitor start up, we have the following Diagnostic configuration.

//set Trace logs to Warning and above every 5 minutes 
DiagnosticMonitorConfiguration dmc = 
     DiagnosticMonitor.GetDefaultInitialConfiguration(); 
 
dmc.Logs.ScheduledTransferPeriod = TimeSpan.FromMinutes(5);   
dmc.Logs.ScheduledTransferLogLevelFilter = LogLevel.Warning;  

DiagnosticMonitor.Start("DiagnosticsConnectionString", dmc); 

A very odd behaviour was observed when running the application, it would only store Verbose level messages. When setting the Diagnostic monitor log level filter for Warning no log messages were logged.

It was quite a puzzle!

How are we filtering our trace messages?

In configuration, a TraceSwitch is defined called “logLevel”. The static Logger applies the filter using the ShouldLog method of TraceListenerLogWriter. All of the attached listeners then receive trace messages appropriate to the filter.

However, Azure Diagnostic has another level of filtering, which is used to send the logged messages to Azure storage at a particular interval defined through DiagnosticMonitorConfiguration.

You might still be wondering why this filtering will affect it.

How is the Trace.WriteLine method in DiagnosticMonitorTraceListener class implemented?

As explained earlier that Azure Diagnostic operates with two levels of filtration:

a. Filter for logging to attached trace listeners.

b. Filter for sending the logged messages to Azure storage.

The underlying logger requires the log level in message so that the filter can be applied before sending to Azure storage.

But looking at the DiagnosticMonitorTraceListener class in Reflector, the WriteLine method calls an overridden method Write, which is hijacked to be Verbose only:

public override void Write(string message)   
{   
    this.WriteStringEtw(TraceEventType.Verbose, 0, message);   
}

The use of Verbose does make sense, as a default value for use by the second level filter. But this is hidden inside the implementation of the trace listener. This is why just replacing the trace listener did not work for our application.

What is the solution?

The solution is quite straightforward. Instead of using Trace.WriteLine, use Trace.TraceError, Trace.TraceWarning, Trace.TraceInformation appropriately and Trace.WriteLine for the Verbose event.

The modified version of our TraceListenerLogWriter looks like :

public class TraceListenerLogWriter : ILogWriter   
{  
    public void WriteLog(TraceLevel level, string message)   
    {  
        switch (tracelevel)   
        {  
            case TraceLevel.Error:  
                Trace.TraceError(message);  
                break;  
            
            case TraceLevel.Warning:
                Trace.TraceWarning(message); 
                break;  
            
            case TraceLevel.Info: 
                Trace.TraceInformation(message);
                break;  
            
            case TraceLevel.Verbose:
                Trace.WriteLine(message);
                break;
            
            default:
            //no logging
            break;
        }  
    }
} 

In summary, only use the Trace.WriteLine method to log Verbose messages. Keep in mind that Trace.WriteLine is Trace.TraceVerbose within the Azure trace listener.

Read more about work on an Azure utility, which had to deal with the XML responses provided by the Service Management REST API: http://www.dotnetsolutions.co.uk/blogs/markrendle/archive/2010/02/17/working-with-namespaces-in-linq-to-xml/ on Mark Rendle’s new blog here: http://www.dotnetsolutions.co.uk/blogs/markrendle.

This post should really be title: “how we (could have) migrated ScrumWall to Azure in less than 20 minutes”. But I’ll come on to that later.

I’ve been speaking about Azure at a number of events recently. As part of the talks, I have given a quick overview of the developer portal for Azure and the Visual Studio tools. I’ve wrapped this up in a demo using our ScrumWall product (http://scrumwall.cloudapp.net).

Winding back time conveniently to about a year ago, I explain how we could have migrated ScrumWall to Azure in less than 20 minutes. I then go on to highlight some of the additional requirements that would be required to enable this to be managed, supported and maintained in production.

The Demo

The majority of functionality provided by ScrumWall is contained in a Silverlight application. There are a number of WCF services to provide data to this application and some fairly basic ASP.NET pages. There are also a couple of HTTP handlers to manage the authentication, which is provide by Microsoft Live ID.

Overall ScrumWall contains a reasonable number of different components and is a good “test” candidate to migrate to Azure.

Step 1: Creating the SQL Azure Database

Using the “Generate Script” option to script the whole database (including the triggers used for audit) produces something very close to the SQL needed e.g. for the Story table the following SQL is generated:

CREATE TABLE [dbo].[Story](
    [StoryId] <removed_for_clarity>
 CONSTRAINT [PK_Story] PRIMARY KEY CLUSTERED 
(
    [StoryId] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF,
 ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]
GO

This can be modified as follow to make is SQL Azure compatible:

CREATE TABLE [dbo].[Story](
    [StoryId] <removed_for_clarity>
 CONSTRAINT [PK_Story] PRIMARY KEY CLUSTERED 
(
    [StoryId] ASC
)
)
GO

Any data that needs to be migrated can be scripted (as this is a small application).

Step 2: Update the web.config

The existing connection string is simply replace with a new one, ensuring that encrypt is set to true:

<add name="ScrumwallConnectionString"
    connectionString="Data Source=v2wqed1zpx.database.windows.net;
Initial Catalog=ScrumWall; User ID=scrumwall@v2wqed1zpx;Password=password;
Trusted_Connection=false;Encrypt=true;"
    providerName="System.Data.SqlClient" />

Step 3: Creating a Cloud Project

To migrate the ASP.NET application project to Azure, first I add a new Windows Azure Cloud Service project with no Web or Worker roles. Then by using the option to add a Web Role project from the solution, I can associate the existing ASP.NET project with the Cloud Service project.

Testing this locally in the Development Fabric means using a different port, port 81 for my machine. In order to ensure that Live ID redirects my authenticated users back to the correct place I need to create a new Live Services application ID (https://live.azure.com/). I have used the “domain” of “azure.machine.local” and the return URL “http://azure.machine.local:81/LiveAuthHandler.ashx”. Next I add an entry into to my hosts file, a mapping of host name “azure.machine.local” to 127.0.0.1. Last, the web.config is updated with the new application ID and secret:

    <add key="wll_appid" value="0000000044028C05"/>
    <add key="wll_secret" value="x1x1x1x1x1x1x1x1x1x1x1"/>

I can now run the application on my local Development Fabric.

Step 4: Deploying to Azure

The migration is nearly finished. However, before I can publish this application to Azure, I need to create another Live Services application ID for the final domain “scrumwall.cloudapp.net” and once more update the web.config.

The application can now be published to Windows Azure and deployed. To ensure that the Windows Azure application can connect to SQL Azure, tick the Allow Microsoft Services access to this server.

All of these steps including running the newly migrated project by clicking the http://scrumwall.cloudapp.net can be completed in less than 20 minutes!

The Real World

Now in the real world, there are a significant number of real world questions that you need to ask before migrating an application to Azure, for a fuller list check out this book: http://www.dotnetsolutions.co.uk/blog/archive/2009/11/04/“thinking-of-delivering-solutions-on-the-windows-azure-platform”-book-now-published/.

For ScrumWall there are three areas in particular that we chose to address as part of our real migration of ScrumWall to Azure:

1. Management of configuration values, in particular the Live Services application ID. This is discussed here: http://www.dotnetsolutions.co.uk/blog/archive/2009/04/23/moving-scrumwall-to-the-windows-azure-platform-part-3-windows-live/
2. Use of Windows Azure Table storage in preference to SQL Azure. This is discussed in another previous blog: http://www.dotnetsolutions.co.uk/blog/archive/2009/02/24/moving-scrumwall-to-the-windows-azure-platform-(part-1-the-data)/
3. Access to logging and diagnostic information. This has been significantly improved from the early CTP for v1 of Windows Azure and has been well covered elsewhere, for instance: http://blogs.msdn.com/windowsazure/archive/2009/12/01/introducing-windows-azure-diagnostics.aspx

The Big Demo Challenge

On a side note, one of the biggest challenges for the demo was actually connecting to SQL Azure from SQL Server Management Studio. Often firewalls will block port 1433/1434 outbound and this a major hassle to get changed, especially for a 20 minute demo. For more information on one approach to resolve this issue see my next blog post.