At Dot Net Solutions, we are ardent advocates of the Agile philosophy. In fact we want to get t-shirts made with Eat Agile, Sleep Agile, Drink Coffee. Not that we need the caffeine, we’re connoisseurs don’t you know!

The Agile umbrella is a way of working – a set of principles to advocate and emulate. There are many ways of achieving this philosophy: Lean, XP, and most people who have heard about Agile have probably heard about Scrum. Scrum is our chosen flavour for our projects, and it can be applied to most areas of life. It works from development teams up to enterprise organisations. This doesn’t mean it’s a magic bullet – it’s not going to solve everything in one go and it’s not going to do the work for you.

One thing I always take away from Scrum is the principle of using things appropriately. There are lots of misconceptions about Agile and its associated frameworks – things like “no documentation” and “no planning”. These are false! To be fair someone could probably get along without these thinking they are living the Agile dream, but to be honest they are setting themselves up for failure.

What is more important is choose appropriate options. Include documentation; just make sure it’s documentation for a specific purpose, and not superfluous record keeping or speculation. And definitely plan! Think about the short and medium term, don’t get bogged down with planning the long term in a huge amount of detail that will inevitably change. Also think about how you work. This means choosing the appropriate tools.

We wanted to start tracking the general stuff that we had to do as an office and as an organisation – you know: the re-supplies, upgrades and improvements we are constantly rolling out in order to be more productive as a company. Of course, using Scrum every day our first inclination was to use Scrum for this purpose as well. We set up a wall in the Staff room, populated it with backlog items and then set to work working through sprints lasting 4 weeks.

2 Sprints in we did what every good Scrum team does – we inspect and we adapt. When we looked back on the last sprint it dawned on me that we were not living up to the Scrum framework – we had become a “Scrum But” team. You know, those teams that when you ask them what methodology they use they reply “Oh it’s Scrum, but we don’t do retrospectives” or “We use Scrum, but we don’t do daily standup meetings.”

We had fallen into the same trap! This is why I love retrospectives – they give insight into where things are not working so that you can adapt your approach. Our conclusions in this case? Perhaps Scrum isn’t the right tool to use. So we have looked to another sister approach under the Agile umbrella – Kanban. This is an approach that still uses a task board / wall to view items moving through the pipeline, so that you can identify holdups and blockages visually. We think we chose the appropriate tool for the job; it’s suiting us better, but I am sure we’ll be adapting to improve things again in the future!

It goes to show that by keeping to the principle, and being open to change, you can apply Agile to all walks of life – it’s a way of business that’s constantly growing and evolving. Agile allows us to deliver cutting-edge high-quality software solutions to our customers quickly. Talk to us to find out what we could do for your business.

Author: Dan Woodward

 Follow me!

 Share this post on Twitter

Problem statement

Lately, we have been recruiting developers and architects at all levels. Our recruitment process comprises a review of a prospective employee’s CV, followed by a brief telephone interview with a few simple questions designed to eliminate the disastrous, dishonest or downright delusional applicants. The few people who make it past that stage are invited into the office for a good, in-depth, face-to-face interview where we talk tech and write code on whiteboards.

This process has been unsatisfying. Unfortunately, we get far too many CVs from people saying they have 8 years experience in .NET programming who are then unable to describe garbage collection, or explain what the IDisposable interface is for. Clearly that is acceptable for some companies, but we expect a little more. Not a lot, just some sign that you’ve taken the slightest little bit of interest in the platform you claim to have been developing on for a quarter of your life.

So I was pondering how we could better filter the CVs, and wishing that they could be statically typed and checked by a compiler, when it occurred to me that they can: if we ask people to write their CV in C#.

The solution

I’ve created a project with an assembly that exposes two interfaces – IResume and IEmploymentHistoryItem – and a smattering of supporting classes and enums, such as Skill and Proficiency.

I’ve used Sandcastle to document these interfaces and classes, and we are putting the documentation on our website.

Starting soon, if you would like to apply for a job at Dot Net Solutions – and trust me, it’s a fantastic place to work – then you will only be able to send us your CV in the form of a Visual Studio 2010 solution containing a project which implements the interfaces. We’ll take a quick look through the code to check for malicious intent, then we’ll build it. If it doesn’t build, we won’t call you. Then, the resulting assembly will be loaded in a WPF application which, if you have implemented the interfaces correctly, will display your CV and let us print it. If we can’t view or print your CV, we won’t call you.

If we can build your solution and print your CV, then we will call you, we’ll have a very quick chat and, if we have an opening, we’ll get you in for an interview.

Oh, and don’t get your friend to write the application for you, because we’ll find out.

I’m currently trialling this process with a friendly guinea pig; if that goes smoothly then we’ll roll it out more widely very soon.

This is the third in a series of articles describing the issues we have encountered when using Sockets in the .Net Micro Framework and highlights quite an acute problem.

Symptoms

You are attempting to connect to a remote endpoint over TCP. Your code looks not too dissimilar to this:

public void SendData(IPAddress address, int port)

{

    IPEndPoint endpoint = new IPEndPoint(address, port);

    using (var socket = new Socket(AddressFamily.InterNetwork

                        , SocketType.Stream, ProtocolType.Tcp))

    {

        socket.Connect(endpoint);

        /*communicaton code 

 *removed to keep 

 *example terse*/

        socket.Close();

    }

}

This code has worked absolutely fine for a while now without any defects. All of a sudden your application now appears to hang indefinitely every time it tries to communicate using Sockets. You step through your code line by line in Visual Studio and it turns out that the troublesome line is the call to Socket.Connect(). When you call this method it does not return; even if you wait for 2-3 minutes; but what has changed? It always used to work.

Not sure that this is correct. If the service is down then it should connect using the Socket Connect, but will return an exception and 500 error code. The problem with the Socket Connect is that if the network connectivity is removed. I.e. if there is no network, then the Connect “never” returns.

What’s actually happening here? Why did we not get an exception? Surely it would have timed out after 3 minutes?

It turns out that Socket.Connect does not actually timeout; at least it doesn’t do it for an interval that will be of any use to you or your application. We run a test for more than 2 hours.

Conclusion

Knowing this odd nuance of the .Net Micro Framework highlights the need to think very carefully about the architecture of your application.

· Is it wise to run your Socket code on the same thread as any other functionality?
Would you be potentially blocking time sensitive operations? Probably not a good design to use this model an event model would be much better.

· When network is reconnected, the connection that was in progress will error. So you need to manage this error and retry etc

· If no exceptions are being thrown how do detect these events and are they important to you and your application?

· How can I mitigate against the chances of this happening in the first place?
For example you should consider using DNS to delegate the responsibility of identifying which IP your application should be connecting to. That can optimize for the edge case scenario where your server is down and you need to move to a new (secondary) server in a hurry.

This was a bug identified in the .Net Micro Framework 3.0 and due to rather unfortunate conflicts between the version 4 release plan and when we identified the bug the same is true for Microsoft’s latest offering as well. We can’t comment on MS’s release plan.

This was one of several interesting issues encountered when working with Sockets in the .Net Micro Framework. If this was interesting / relevant for you then you might also want to consider reading the following articles:

• .Net Micro Framework Sockets: A lingering smell
• .Net Micro Framework Sockets: Intermittent performance

Author: Gavin Osborn
@gavinosborn

Tweet

In a previous blog post I described the Linger error that we were running into from time to time and how you can mitigate against this by setting the ‘Linger’ option on the socket. This was the first of several issues we identified when working with Sockets in the .Net Micro Framework 3.0.

The next challenge we faced was to do with intermittent performance problems when communicating over HTTPS.

Example: The Socket Client Demo

We had noticed that the time it took for our application to communicate with HTTP and HTTPS endpoints (even on our internal network) seemed to vary greatly. These requests were consistent in both size and complexity yet some would take <1s and others appeared to take 10s.

To highlight this problem we used the ‘SocketClient’ demo application provided as part of the Micro Framework 3.0 SDK. This demo application makes a HTTP GET request to a given endpoint out on the internet and reads the response. Choosing this sample application for our testing allowed us to rule out the complexities in our own project as the source of the problem.

We made a small change to the code so that it would tell us how long it took to make each request and then performed the same GET operation 15 times and recorded how long it took each time.

We performed this test twice...

1. Once running on the Microsoft supplied device emulator

2. Once running on our Tahoe II hardware.

... and the results were interesting.

Running the test against the emulator (highlighted on the graph by the blue line) gave us consistent sub 200ms response times 100% of the time. Running on the Tahoe II hardware (the red line on the graph) gave us very erratic response times. Half of the requests returned in under 1 second and the other half took as long as 8 seconds, not a good time at all!

We then ran the same test on a GHI Embedded Development Master System in an attempt to rule out the hardware manufacturer as the cause of the problems and sure enough this device provided just as inconsistent results.

The Solution:

Migrate to .Net 4.0... Now this sounds like a fairly dramatic solution but if the success of your project is defined by delivering good, constant response times when communicating over HTTP or indeed any other protocol over TCP then you will need to use the latest Micro Framework offering from Microsoft.

We approached Microsoft about this issue, presenting our evidence and reproducible steps for them to see with their own eyes. They were quick to acknowledge that we had indeed discovered a bug in their TCP stack and took the appropriate steps to fix this in version 4 of the .Net Micro Framework before releasing it.

Whilst no fix has been provided for version 3 of the Micro Framework we were able to mitigate against these problems because we did not require sub 1 second response times (although it would have been nice).

As a side note...

The other issue that this highlighted was the importance of having real hardware to develop against as early as possible in your .Net Micro Framework projects. As our testing shows – had we spent the majority of our project working on software emulated devices we would have never discovered this issue.

Unit testing and a strong emphasis on code quality can only help you so much in your project. There are some things that you will only ever notice with real hardware. I suspect the same is true of all projects that have a hardware element – be it other embedded development platforms or indeed mobile development.

Author: Gavin Osborn
@gavinosborn

Tweet

One of the more significant challenges that we have faced during our last project presented itself to us about halfway through the project timeline. Our .Net Micro Framework application regularly sends messages to remote endpoints out on the internet over HTTPS/TCP. In isolation a single HTTPS request using the .Net Micro Framework was a pain free operation (even if there is no HTTPS stack out of the box). However, the problem was that after an indeterminate period of time our application would stop sending HTTP requests and would die a horrible death.

To confound matters further no exception were being raised and so we were not even able to catch the exception to a) gather more information or b) gracefully handle the error. Instead the board would just lock up and stop working, as the .Net Micro Framework runtime had crashed.

Given that we were not receiving an exception of any kind initial testing focused on the hardware that we were running on. We tested our software on both the GHI Master Development System and the Tahoe II board from Device Solutions Ltd yet both products exhibited the same problems.

Focus shifted back to software, but given that no exceptions were throw and connecting to the device using the MF Deploy tool yielded no additional insight we could not ascertain the cause of the problem. Having scoured the code-base for any clues we decided to approach Microsoft with our rather odd problem.

After discussing the many intricacies of our application with the .Net Micro Framework team they provided us with an early build of the MF Deploy tool that was to be released with version 4 of the Framework. As soon as we started using this new version of MF deploy we started making progress.

The first thing that happened is we started to get an exception and stack trace information in the MF Deploy trace information. Now we had something to work from!

#### Exception System.Net.Sockets.SocketException - E_FAIL (1) ####
#### Microsoft.SPOT.Net.SocketNative::socket [IP: 0000] ####
#### System.Net.Sockets.Socket::.ctor [IP: 001f] ####
#### SocketException ErrorCode = 10024

A brief search using Reflector quickly led us to a brief description of the problem:

OK, so apparently we have opened too many sockets. Unfortunately; as is so often the case; this only succeeded in posing more questions than it answered.

Our HTTPS/TCP code opened and closed a socket on every request. Our Socket object was also wrapped in a using statement which meant that it would be disposed of after every call.

IPAddress address = GetHostAddress(uri);

IPEndPoint endpoint = new IPEndPoint(address, 80);

using (Socket socket = new Socket(

    AddressFamily.InterNetwork,

       SocketType.Stream, ProtocolType.Tcp))

{

    //open the socket

    socket.Connect(endpoint);

    //HTTP writing code removed 

    //to keep sample terse.

    //close the socket

    socket.Close();

//dispose the socket

}

So then why do we have too many sockets open? Another look at Reflector only served to strengthen our resolve that we had done everything correctly. The Dispose method did appear to clean up any of the native resources used by the Socket object:

Confused once more we approached Microsoft for an answer to our problems.

It was then that they informed us that by default when you close a Socket in the .Net Micro Framework it does not actually release the Socket for a further 2 minutes! This value is an option that you can set yourself with the following line of code:

socket.SetSocketOption(SocketOptionLevel.Tcp

                               , SocketOptionName.Linger

                               , new byte[] { 0, 0, 0, 0 });

So what is SocketOptionName.Linger? Assuming that it is the same as its big brother equivalent in the .Net Framework it is an option that is used to tell the socket to wait before closing so that any unsent data can be sent.

It should only be necessary to set the Linger time to 0 if you are expecting to open another Socket to the same address and port. If you are to set the Linger time to 0 then it is also recommended that you also set the NoDelay option to true so that any messages are sent immediately.

socket.SetSocketOption(SocketOptionLevel.Tcp

                      , SocketOptionName.NoDelay

                      , true);

Complete example:

IPAddress address = GetHostAddress(uri);

IPEndPoint endpoint = new IPEndPoint(address, 80);

using (Socket socket = new Socket(

        AddressFamily.InterNetwork

       ,SocketType.Stream

       ,ProtocolType.Tcp))

{

    socket.SetSocketOption(SocketOptionLevel.Tcp

        , SocketOptionName.Linger

        , new byte[] { 0, 0, 0, 0 });

    socket.SetSocketOption(SocketOptionLevel.Tcp

        , SocketOptionName.NoDelay, true);

    //open the socket

    socket.Connect(endpoint);

    //HTTP writing code removed 

    //to keep sample terse.

    //close the socket

    socket.Close();

//dispose the socket

}

While we expecienced this problem using version 3 of the .Net Micro Framework, I am not aware whether the default behaviour has been changed in version 4.

This was the first (and simplest) of several intricacies in the .Net Micro Framework that we encountered during the course of this project. More in another blog post soon.

Author: Gavin Osborn
@gavinosborn

Tweet