Facebook Server side vs Client side (JavaScript) Graph API Libraries

I’ve been integrating applications with Facebook for nearly a year and during that time I’ve found it difficult to find reliable documentation, guides, and just a good set of best practices out there. A friend of mine recently voiced similar misgivings which spurred me to write about some of the obstacles I have had to overcome. I’ve aggregated information (with sources when available) about topics I’ve had to deal with when integrating with Facebook.  I will break up these topics into a series of posts that will be published over the next few weeks.

Server side vs Client side (JavaScript) libraries

The Graph API is a definite improvement over previous API’s Facebook has implemented.  It’s a RESTful API and intuitive to work with.  There are numerous libraries that you can use to wrap calls to Graph and the most frequently used appears to be the JavaScript implementation.  An implication of using the JavaScript library is that all your interaction with Facebook is actually being done on the client side.  Using the JavaScript library for the Graph API is actually a Facebook best practice for one sole reason: performance and  they’re right.  Client <-> Facebook versus Client <-> App Server <-> Facebook would have less latency, but I believe the gain is negligible and does not outweigh several concerns I will discuss in this post.

  • Violation of the Separation of Concerns (SoC) principle
  • Impossible data validation
  • Testing is more difficult than it needs to be

SoC Violation

When using the JavaScript library all your communication with Facebook is done on the client side tier.  That means that any data you exchange with Facebook must be transformed using JS into a format both parties can consume (which will no doubt lead to bloat in your JS libraries).  In a strict MVC sense, the View should not be doing anything but render itself for the user to consume.  It shouldn’t introduce extra boundaries that need to be crossed in order to integrate with a 3rd party system like Facebook.  Instead of translating messages between the application server, the View, and Facebook, the integration point should instead be on the application server itself.  This way there is only a single boundary to cross: your core business logic to an abstraction of the 3rd party system.  All parameters of the integration can be defined in a controlled setting (on your server).  Your application can decide the channel (means & protocol of message transmission), the message, the message’s data format, and the destination.  The correctness of any of these parameters cannot be guaranteed so long as the integration point lies on the client.

For example, if your client is sharing something on Facebook on behalf of your application, then the client will need to download all the information required to construct a Graph API post from your application server.  The client will then pass the constructed post object to the Facebook JS library.  When a call back is executed the application may now want to track whether or not the post was successful, as well as persist other attributes of the post and the user.  Again, this information will need to pass through several boundaries and tiers of your application to get where it needs to be.

More information on proper patterns for integration of systems can be found in the book titled Enterprise Integration Patterns.

Data Validation

Anyone with limited experience in web development can tell you that you never trust data from the client side.  When using the JavaScript library all your integration with Facebook is open to the unscrupulous sets of eyes of malicious users.  You forfeit any capability to validate data being sent from the client to Facebook on behalf of your application.  Again, using the example of sharing on Facebook, you likely have a set of immutable Facebook Post parameters the user should not be able to change such as call to action links, the redirection override URI, reference string, etc.  In most cases, all the user should have the capability of changing is their custom message associated with the post[2].  Treat data you share with Facebook like any other data you accept from the client, validate it on the server.

Testing

When your integration point lies on the client you make your system much more difficult to test.  If you want to test your application without the integration it’s more difficult to mock or stub Facebook out of the picture on the client than it is on the server side.  I realize that there are numerous libraries out that do allow for control over the client side layer during testing, but in my experience the tooling involved in testing an application that integrates with a 3rd party resource is much easier to setup, stub/mock, and test on the server than it is on the client.

Conclusion

Although the JavaScript library is tempting to use to facilitate a Rapid Application Development (RAD), I think all Graph API calls made on behalf of your application should be done on the server.  There are numerous libraries you can use on the server side, depending on the language you use.  Here’s a list provided by Facebook.  I usually work with .NET and would highly recommend the Facebook C# SDK.  At the end of the day, the Graph API is a pretty easy REST API to use, so if you can’t find a suitable drop-in solution then rolling your own adapter shouldn’t involve all that much work.

Notes

[1] Post to user feed with the Graph API Post – http://developers.facebook.com/docs/reference/dialogs/feed/#graphapicall
[2] Immutable parameters in Graph API Post – It’s a typical requirement of applications integrating with Facebook to have a user make an endorsement of something on their user feed.  The application can not prefill or modify the user’s custom message in any way (see Facebook’s Platform Policies, Section IV titled “Application Integration Points”, point #2).  However, the content of the rest of the posts parameters can be prefilled as long as the user executed a workflow in a way that the parameters are expected to be in the post.  For example, when sharing an endorsement of a product of a company, the image parameter post could be a picture of the product itself, the link parameter could be a link to a product on the company’s website, and the action and description text could describe the product in more detail.

Non-functional requirements of a good Team Lead

In engineering disciplines requirements can be split into functional and non-functional requirements. Functional requirements describe what a system will do whereas non-functional specify criteria that can be used to judge the operation of a system. In the Agile world there are lots of processes that can be employed by technical/project/team leaders to deliver working software. For the most part, these processes are tried and true. There is a concise set of steps to follow that can reliably produce measurable output in a number of different forms (knowledge sharing, consistency, quality, etc). I like to think that the knowledge of these processes and the capability to execute them are functional requirements of an experienced Team Lead.

As for non-functional requirements, when I first took the role of Team Lead I identified some key skill sets I needed to improve upon. Being somewhat of a middle of the ground introvert-type it took me some time to recognize some of these shortcomings and then practice them enough to become proficient. When I recognize that there’s something I have to get better at look then I look no further than Martin Fowler for some words of wisdom “… if it hurts do it more often…”.

Confidence in Public Speaking

It took me some time to gain the confidence to speak my mind and to project my opinions in a convincing manner. The key to speaking confidently on a subject is to know it well. So if I’m unable to debate the merits, faults, and alternatives to an idea I’ll hit the books and learn it as best I can. Depending on the impact of an idea I am proposing I’ll spend the appropriate amount of time to study and make notes on the subject. This often times naturally leads to an artifact I can share with the team after a discussion. Running subject matter by trusted colleagues before proceeding is also a great way to reinforce research. In the end it’s not about me knowing every path a discussion can take, but to at least get to a point where I can debate it intelligently and get feedback (and hopefully suggestions) from the rest of the team. As the saying goes, it’s all in the delivery. If you’re emitting non-verbal cues that you lack confidence your team might lose their confidence in you.

Stay Positive

I’ve worked in a number of different teams and unfortunately I’ve been a member of teams with less than ideal circumstances. Toxic environments can be created by a myriad of reasons, but as a Team Lead it’s your job not to help propagate these feelings into your team. If a team member voices concern over a project, team, or the very organization itself, it’s important to stay positive and provide a “best case” response. During conflict resolution you can’t let yourself be drawn into the feeling of unease being shared by those you’re trying facilitate a resolution for. Help them come to a compromise and acknowledge the value in occasionally bumping heads; it’s a learning experience for everyone. When you stay positive your team mates will appreciate the effort because it’s a lot easier to succumb to the situation than it is to be constructive in crappy situation.

Patience

One of the hardest things for me to learn was to be patient when working with a team mate. Criticism comes easy to most developers and I’m no exception! For example, it’s easy to make an off-the-cuff remark when reviewing a sloppy implementation or naive design. In situations like these I’ll hold my tongue and wait for an invitation to provide feedback. I choose my words wisely and resist the urge to be sarcastic or say something along the lines of “I told ya so!” for it would do more harm than good. Not only will it affect your team mates morale, but she may also be less inclined to seek your advice in the future. Don’t let your emotions get the better of you and swallow your pride!

This is certainly not a de-facto list of valuable people management skills, but they are those that ring most true to me. When I became more practiced in all of these areas I noticed a much more positive vibe in the people I worked with on a daily basis.

Edit: For great advice on software team leadership I highly recommend reading Roy Osherove‘s blog titled: Elastic Leadership: Notes to a Software Team Leader.

Wait! Using Selenium Web Driver with jQuery animations

Selenium Web Driver is a terrific tool for automating User Acceptance Testing (UAT) of your front end web layer.  I don’t have a ton of experience with this form of testing, but I’ve quickly learned to appreciate its usefulness.  Our team recently attempted to make a reliable set of UAT tests for a legacy application. We had a number of 3rd party social media dependencies and a less than ideal page architecture. The end goal was to have a simple test suite that was reliable enough to be automated with as few transitive fails as possible.  As with most things, this was easier said than done; especially when done after the fact (kudos to BDD/ATDD, I’ll be employing this technique for greenfield projects).

One aspect in particular that took some thinking was how to use WebDriver to wait for a jQuery animation to complete.  There are several ways to skin this cat, but I chose to make use of a jQuery extension to the CSS specification known as :animated.  Using a WebDriverWait, I was able to detect when an animation was complete by executing some JavaScript.  Now, I know this isn’t good practice with Web Driver, in fact the Web Driver FAQ answers the “How do I execute JavaScript directly?” as follows.

“We believe that most of the time there is a requirement to execute Javascript there is a failing in the tool being used: it hasn’t emitted the correct events, has not interacted with a page correctly, or has failed to react when an XmlHttpRequest returns. We would rather fix Web Driver to work consistently and correctly than rely on testers working out which Javascript method to call.” – Selenium Web Driver FAQ

However, it goes on to recognize that not having the capability to execute arbitrary JavaScript can be a limitation and proceeds to explain the usage of the IJavaScriptExecutor implementation of Web Driver. IMO, I had a case to use this functionality because at the time I had no other way to determine when an animation was fully complete (short of re-factoring the page itself).

Below is a snippet of a new WebDriverWait helper class that I’ve been using in my test suite. It includes a method called UntilAnimationIsDone. I use this after I trigger a page event that causes a series of animations so that I know the page is in the expected “ready” state and that I can continue.

public class Wait
{
    private const bool Debug = true;
    private const int ImplicitWaitInSeconds = 30;

    private readonly IWebDriver _driver;

    public Wait(IWebDriver webDriver)
    {
        _driver = webDriver;
    }

    public void Until(Func<IWebDriver, bool> waitCondition, int timeoutInSeconds = Constants.ImplicitWaitInSeconds)
    {
        var wait = new WebDriverWait(_driver, TimeSpan.FromSeconds(timeoutInSeconds));
        wait.Until(waitCondition);
    }

    public void UntilAnimationIsDone(string elementId, int timeoutInSeconds = ImplicitWaitInSeconds)
    {
        Until(driver =>
        {
            var javaScriptExecutor = (IJavaScriptExecutor)driver;
            var isAnimated = javaScriptExecutor
                .ExecuteScript(string.Format("return $('#{0}').is(':animated')", elementId))
                .ToString().ToLower();

            if (Debug)
                Console.WriteLine(string.Format("Element: '{0}' Is Animated: {1}", elementId, isAnimated));

            // return true when finished animating
            return !bool.Parse(isAnimated);
        }, timeoutInSeconds);
    }
}

Automating ASP.NET Site Deployment with Web Deploy: A survivor’s tale

I recently finished a spike into MS Deploy. I wanted to find an easy way to deploy an ASP.NET website to a test, QA/demo, and Staging/Production environments and in the spirit of Continuous Delivery I wanted the whole process to be automated.  It wasn’t long until I found a wonderful tease of an article titled “Automating Deployment with Microsoft Web Deploy” by Scott Guthrie.  I call it a tease because as much information as it contains, it still glosses over a lot of details I think are essential to getting a Web Deploy implementation production-ready.  Scott was a member of the team who developed the Web Deploy technology for release with Visual Studio 2010 and IIS 7 in Windows 7 & Windows Server 2008.

I quickly discovered that configuring both your ASP.NET project and the destination server is no picnic.  I hope that some of research I did to get my own Web Deploy implementation working will be of help to others who decide to go down a similar path.

Errors while building the web deployment package

Part of my CI build is to build a web deployment package that I can re-use for all my deployments.  When I first attempted to do this from the command line I received the following error.

c:\windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets(483,9): error : The OutputPath property is not set for project 'My Project.csproj'.  Please check to make sure that you have specified a valid combination of Configuration and Platform for this project.  Configuration='Debug'  Platform='Any CPU'.  You may be seeing this message because you are trying to build a project without a solution file, and have specified a non-default Configuration or Platform that doesn't exist for this project. [C:\CCNetWD\My Project\My Project.csproj]

I’m no MSBuild expert, but my project file appeared to have the appropriate PropertyGroup defined for my Debug and AnyCpu build configuration, an OutputPath was defined.  My call to MS Deploy also appeared to use the correct configuration (Debug).

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
 <DebugSymbols>true</DebugSymbols>
 <DebugType>full</DebugType>
 <Optimize>false</Optimize>
 <OutputPath>bin\</OutputPath>
 <DefineConstants>DEBUG;TRACE</DefineConstants>
 <ErrorReport>prompt</ErrorReport>
 <WarningLevel>4</WarningLevel>
 <IncludeSetAclProviderOnDestination>False</IncludeSetAclProviderOnDestination>
</PropertyGroup>

It wasn’t until I found the following post titled ‘MSBuild Package fails with “OutputPath property is not set” error’ by JMarkHowell that led me to a solution.  I removed the Platform specific information from the PropertyGroup conditions for my Debug and Release builds with the following and low and behold, my web package was built.

<PropertyGroup Condition=" '$(Configuration)' == 'Debug' ">

Keep in mind that since this is a build configuration you must apply the same to each dependent project your web project references.

Is this the right way to go about this problem?  Probably not.  I hope someone smarter than I stumbles across this post and corrects me on a better way to setup my build configuration to play nice with MS Deploy on the command line or at least tells me the proper way to choose a build configuration with a Platform type.

Using IIS Manager Users to Sandbox Deployment Permissions

On internal infrastructure it’s not (usually) critical to lock down destination boxes with deployed solutions because they’re only usually accessible by individuals directly responsible for developing, testing, or demonstrating the project.  In these cases you may be able to get away with using Integrated Security (the default MS Deploy security package) on your corporate domain network.  If the identity doing the deploying has administrator priveleges on the remote machine then Web Deploy will work marvelously without error.  However, if you are not deploying to a machine on your corporate network and your destination server is on a remote machine on a network you may not have control of (i.e. a colocated machine accessible over the Internet) then you’ll likely have requirements to lock down access to the box.  In this case you have the option of either using a less priveleged local machine account or an IIS Manager User (a special user created and administered by IIS for the sole purpose of Deployment).

I chose to setup an IIS Manager User.  This is a straightforward task and is documented extensively on the MSDN page “Configurating the Web Deployment Handler“.  However, as I learned to expect during this whole process there are some subtle details to take into account to get Web Deploy working nicely with IIS Manager Users.

  1. Use basic authentication when calling MS Deploy at the Command line.  Windows Authentication security packages (NTLM or Kerberos) won’t cut it because IIS Manager Users are not Windows accounts.  The only way you will be able to use IIS Manager users is to explicitly specify basic authentication at the command line like in the following example. 
    msdeploy.exe -source:package="_My_Project_webdeploy.zip" -dest:auto,computerName="https://MyDestinationServer:8172/MSDeploy.axd?site=My Project",userName="autobuild",password="shhhh",authtype="basic",includeAcls="False" -verb:sync -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension -setParamFile:"_My_Project_webdeploy.SetParameters.xml" -allowUntrusted -verbose -debug

    Keep in mind that when you using basic authentication you are sending your username and password in clear text to the destination server.  This is one reason why the Web Deployment Handler is setup by default to only be accessible via HTTPS.

  2. Create the web site and its file system root directory before calling web deploy.  Before you run your MSDeploy command in a less priveleged mode it’s necessary to do some initial setup tasks.  Create a directory on the destination server where your web site will be deployed to.  Keep in mind that the identity the Web Deployment Handler uses (Local Service by default) must have write access to this folder in order to deploy the package.Create an empty web site in IIS and point it to the root directory you just created.
  3. Specify the IIS site name in your destination server name query string.  This is another subtle detail I can’t fully explain.  I can’t find any documentation on this query string parameter so I’m not certain why it’s required.  When this parameter is omitted I get a 401 error when attempting to deploy with the IIS Manager User.  My guess is that when the site is not specified, the web deploy path will attempt to interrogate IIS at the server level (where your IIS Management User likely will not have access to via IIS Feature Delegation) and fail.  See the bold section in the following MSDeploy system call for an example. 
    msdeploy.exe -source:package="_My_Project_webdeploy.zip" -dest:auto,computerName="https://MyDestinationServer:8172/MSDeploy.axd?site=My Project",userName="autobuild",password="shhhh",authtype="basic",includeAcls="False" -verb:sync -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension -setParamFile:"_My_Project_webdeploy.SetParameters.xml" -allowUntrusted -verbose -debug

    This and a lot of other sage advice can be found in this StackOverflow answer.

  4. Ignore setting ACL’s on the destination machine during deployment.  This should only be applicable if you’re deploying to a destination machine without Integrated Security and with a sandboxed IIS user.  MSDeploy is clever and will attempt to mirror the NTFS ACL’s you have defined on your source machine.  In certain cases the identities in your source machine ACL’s are not available on your destination machine nor will your IIS Manager Users have the capability to set such ACL’s if they were available.  Therefore it was necessary for me to once again hack my web project file and include the following in my PropertyGroup’s. 
    <IncludeSetAclProviderOnDestination>False</IncludeSetAclProviderOnDestination>

    You can also define this as a parameter for MSBuild at the command line if you’re so inclined.

    msbuild.exe "My Project.csproj" /p:IncludeSetAclProviderOnDestination=False

    The following article titled “Prevent Visual Studio 2010 and Web Deploy from Changing ACLs” by Rick Barber tipped me off to this solution.

Iterative Database Deployment with Web Deploy?

One feature I find lacking in MSDeploy is a simple way to perform iterative updates to a database.  Most websites have a backend relational component so I’m perplexed as to why this hasn’t been addressed in MSDeploy.  MSDeploy does have a SQL provider (dbFullSql), but it can only be used to fully create a database from scratch.  At this time, I believe my only option to implement iterative database updates is to build a solution myself and launch it via the MSDeploy runCommand provider or bypass MSDeploy entirely and work directly with SQL Server.

I hope to spike research on a solution for this problem in the near future.  I’ll document my findings.

Conclusion

Microsoft Deploy is a fantastic system for deploying pretty much anything to Windows environments.  It’s unfortunate it’s not documented more thoroughly.  However, I still believe it is the best solution available at this time.

Clearly some of the solutions I discuss in this post have not been researched exhaustively.  To be honest, I was just happy to get this whole process working in one form of another in the time allotted to the task.  I plan on iteratively refining my deployment pipeline as time allows.

For others in the same situation as my own, I hope that some of the advice in this post helped you with your work.  In addition to the links I’ve already mentioned I would like to close with some other online resources I found invaluable while working on my Web Deploy implementation.

Iteration Number and SVN Revision in a CruiseControl.NET build label

I’ve started a new project and I wanted a build label that includes both iteration number and the SVN revision of the code.  This would give me two very important and easy to track reference points when I review a deployment of my project.

There are already two CruiseControl.NET (CCNet) labellers that accomplish these tasks, but unfortunately there wasn’t one that could do both.

Iteration Labeller
http://confluence.public.thoughtworks.org/display/CCNET/Iteration+Labeller

The Iteration Labeller is built into CCNet 1.6.  It is configured by defining a start date and the length of your iteration in weeks.  More info can be found in the online CCNet documentation.

SVN [Subversion] Revision Labeller
http://code.google.com/p/svnrevisionlabeller/

The SVN Revision Labeller is an open source project plugin for CCNet.  It allows you to integrate the last SVN revision number of a folder of your choosing into a build label.  It currently contains a bug that makes CCNet 1.6 crash on certain conditions of the <pattern></pattern> configuration.  However a workaround I discovered can be achieved by prefixing the pattern with a space.  The resulting label gets trimmed at some point by CCNet, so it is not persisted into any of your libraries at build time.

<pattern> {major}.{minor}.{build}.{revision}</pattern

Indra reported a similar problem on his blog, but took a more invasive approach to fixing it!  The issue has been reported on the project’s issues page, but is yet to be fixed

Use revision and iteration numbers in the build label

Bugs aside, I found this plugin to work pretty reliably and I extended it to include an optional configuration that was already present in the builtin Iteration Labeller.  The iterationDuration parameter allows the developer to length of the the iteration in weeks.  I added this functionality to SVN revision labeller and setup my labeller configuration in CCNet as shown below.  Also added was a new {iteration} token that evaluates to the calculated iteration number in the build pattern.

<labeller type="svnRevisionLabeller">
 <major>2</major>
 <minor>0</minor>
 <startDate>10/21/2010</startDate>
 <iterationDuration>1</iterationDuration>
 <pattern> {major}.{minor}.{iteration}.{revision}</pattern>
 <url>svn://svnServer/ProjectPancake/trunk</url>
</labeller>

And the last build label as shown by the CCNet web dashboard.

I’m not sure if the svnrevisionlabeller author will ever incorporate this into the project itself so I have attached an archive that includes my changes.  Click the link below to download it.

SVN Revision Labeller with Iteration Duration

Building sustainable NAnt build scripts

Anyone with experience at a software shop that goes through some kind of release cycle of client software will indubitably have some form of automated build process (at least one can hope this is the case!).  In my experience one of the most monotonous and time consuming tasks is working on a build script that nobody cares enough about to develop or debug properly.  I’ve found that even the most seasoned of developer will revert to the debugging techniques analogous to alert(‘this should never happen!’);.  This technique usually involves a developer making a change, committing to SCM (your build script IS in SCM right?) and forcing a build.

Would you normally develop software in this manner?  Of course not.  Why is a build script so much different from any other system or process you develop?  In modern monolithic IDE’s like Visual Studio, it’s easy to take for granted how much our environment is already honed to increase a developer’s productivity.  As part of my effort creating a Deployment Pipeline I quickly discovered I needed a good way to develop NAnt scripts.  With a little research I’ve found an easy recipe to develop sustainable build scripts.

There are some key points I want to discuss in a series of posts.  For this post I’ll only go over point 1.

  1. Take the time to setup your IDE
  2. Using NAntAddin (TODO)
  3. Isolate environment specific variables so you can develop your script without testing on your build server (TODO)

Setting up your Visual Studio 2010 IDE

Install NAnt

NAnt is one of those wonderful pieces of software that require nearly no deployment what-so-ever (which is nice for a deployment-centric technology!).  Just extract and use.  If you’ve never heard of NAnt then you’ll want to check out the site and start reading up, because it’s the tool I use for all the following build scripts.  Download the latest stable version of NAnt (0.91 alpha2 as of this writing) and extract it to a directory your choosing (i.e. C:\nant-0.91-alpha2\).  Add the \bin path to your PATH system environment build for ease of use at the command line.

Install NAntContrib

NAntContrib is a wonderful toolkit of NAnt extensions that are invaluable for build engineering.  You can find out all about it at their website, but the main task I use from this project is the msbuild task.  If you want to use the latest .NET framework you’ll have to get one of the nightly builds.  There are several different ways to install and use NAntContrib.  I prefer to unpack the libraries into the “<nant>\bin\tasks” directory as mentioned in their readme.  This allows you to use all its functionality without explicity referencing it in each script.  However, for the sake of clarity I’ll explicitly load the library in my sample script.

Enable IntelliSense for NAnt Scripts

One of the greatest things that about developing in a modern IDE is IntelliSense.  Visual Studio easily extends this capability to XML files by allowing you to copy an XML Schema Definition (XSD) file to your Visual Studio XSD folder (usually in the following location: C:\Program Files (x86)\Microsoft Visual Studio 10.0\Xml\Schemas).  For my NAnt deployment this located at the following location, C:\nant-0.91-alpha2\schema\nant.xsd.

Enable IntelliSense for NAntContrib tasks

In order to get NAntContrib “extra goodness for NAnt” (as their project description gloats), you will want to generate a combined XSD for both NAnt and NAntContrib and use this as the namespace of your root build script <project> node.  Thankfully, NAntContrib makes this easy with the following script.

<?xml version="1.0" encoding="utf-8" ?>
<project name="Foo" default="build" xmlns="http://nant.sourceforge.net/release/0.91-alpha2/nant-0.91-alpha2.xsd-and-nantcontrib-0.85.xsd">
 <property name="NAntContrib" value="C:\Users\Sean\Documents\Source\seandev\blog\building sustainable nant scripts\lib\nantcontrib-0.85\bin" />
 <loadtasks assembly="${NAntContrib}\NAnt.Contrib.Tasks.dll" />
 <target name="go">
  <nantschema output="nant-0.91-alpha2-and-nantcontrib-0.85.xsd" target-ns="http://nant.sourceforge.net/release/0.91-alpha2/nant-0.91-alpha2.xsd-and-nantcontrib-0.85.xsd"/>
 </target>
</project>

I gave it a new namespace name in order to avoid ambiguity with scripts that may reference the plain old NAnt XSD, but this is really a matter of personal taste.  Once this target successfully runs it will create an XSD in the same directory as the build script.  Copy this file into your Visual Studio XSD folder as outlined above for the plain old NAnt XSD.  Restart Visual Studio.

From now on whenever you build a build script you’ll want to reference the namespace by the name you assigned above.  Now you should be able to view all the NAnt and NAntContrib targets when you’re working on your build scripts.

A High Contrast (Dark) Visual Studio 2010 Theme

Traditionally, I’ve never been a huge fan of high contrast themes.  Though they always look way cool in the movies, I’ve never really enjoyed staring at them day after day.  I started getting bored with the default colour scheme of Visual Studio.  Sure, I had customized it a bit, but it didn’t deviate much from stock settings.  So about a year ago I did some looking around for a new theme and I stumbled across a post by Oren Ellenbogen that led me to a blog posting on Jeff Atwood‘s site titled Is your IDE hot or not? I don’t know about you, but mine certainly wasn’t. I like this theme because it isn’t stark white on black, but has more muted contrast between foreground and background.  My design criticism’s will stop there because anyone knows me knows any kind of visually appealing graphic design is not my forté!

A custom Visual Studio theme is a personal thing.  It can directly affect your productivity.  It’s a good idea to have a set of themes you can switch between in order to keep your visual perspective fresh and your brain sharp when the code just doesn’t do it for you.

I honestly don’t know exactly how much has been modified from the original version (and didn’t care much to investigate). Some things off the top of my head were the following.

  • Duplicated settings between code and script colours
  • Changed region and URL foregrounds to yellow so they stood out better

Here’s a screen cap.

Here’s the plaintext code snippet that Jeff created to showcase his own theme.

#region codinghorror.com
class Program : Object
{
    static int _I = 1;

    /// <summary>
    /// The quick brown fox jumps over the lazy dog
    /// THE QUICK BROWN FOX JUMPS OVER THE LAZY DOG
    /// </summary>
    static void Main(string[] args)
    {
        Uri Illegal1Uri = new Uri("http://packmyboxwith/jugs.html?q=five-dozen&amp;t=liquor");
        Regex OperatorRegex = new Regex(@"\S#$", RegexOptions.IgnorePatternWhitespace);

        for (int O = 0; O < 123456789; O++)
        {
            _I += (O % 3) * ((O / 1) ^ 2) - 5;
            if (!OperatorRegex.IsMatch(Illegal1Uri.ToString()))
            {
                Console.WriteLine(Illegal1Uri);
            }
        }
    }
}
#endregion

If you use ReSharper there is one caveat, you must disable line highlighting.

Download my version of this theme here: http://randonom.com/m/darktheme/wladek_DarkTheme_20110801.zip

A few day to day tasks of a Development Team Lead

I recently left a company and had some time to reflect.  I was a Development Team Lead, and though the job definitely has its perks I found that the role wasn’t the best for me.  That said, people said I was a good team lead, so at the request of my replacement I decided to draft a list of some day to day activities I performed.  These are just suggestions, but I think they’re all excellent practices that all teams should try to adopt or have alternatives for.  They’re mostly centered around practices found in the Agile manifesto.  There are many other Agile practices I could have adopted, but unfortunately I didn’t receive the kind of buy-in necessary to achieve them.  This post documents what practices I’ve actually employed while on the job.

Daily Scrum

This is your typical daily scrum.  Scrums are part of most modern software development methodologies.  It gives everyone on the team the opportunity to hear about what each is other is working on.  It fosters collaboration & debate among the group.

I’ve scheduled scrums for 9:45 AM each morning because that works best with everyone’s schedule.  Scrums should happen whether I’m around or not (pick an alternate “scrum master”).   They should be time-boxed to 15 minutes and it’s important to keep people from delving into too much technical detail.  The format of the scrum I use is the following.

  • What did you do yesterday?
  • What will you be doing today?
  • Discuss any roadblocks or ask the group any generic questions about the work.

Case Pick up User

Another aspect of Agile development methodologies I’ve tried to adopt is a general case bucket that anyone on the team can poll and pick up a new case to work on.  This gives the dev more freedom to pick what they may not have normally been assigned.  It allows devs to pick up new skills and spread knowledge across the group.

I assign all types of tickets to this bucket, but it’s important to make certain everyone know the proper priority of cases to withdraw from it.  Client support cases should be the highest priority. This is followed by current release milestones and then triage cases.  I generally assign about 50-75% of all cases assigned to me by product management to this user.  There are exceptions where I’ll assign a case directly to a developer (i.e., they had worked on the case previously or have a particular skill set required for the case).  I will always review the case before assigning it to the pick up user.  In some cases I’ll call a dev over for their opinion.

Mandatory Code Reviews

I try to get everyone on the team to perform code reviews before every commit.  Sometimes it’s hard to get devs to buy into it, but I think it’s worth it.  A colleague recommended that a good alternative to mandatory code reviews is peer programming, but I don’t think it’s realistic to assume every dev will be attached at the hip with someone for all their tasks.  Code reviews don’t have to take too much time.

It’s worth mentioning that a couple of recent events have rekindled my passion for the code review.

Here are some rough notes I compiled for my team as Here are some notes on my justification for code reviews for every commit.

I do code reviews for three reasons

  • It will force the implementer to account for a chosen design
  • It will allow for more than 1 member of the team to be aware of changes going into the project
  • Drive remediation work and make the implementation or fix that much better (two brains are better than one)

How & what to review

  • This should not be seen as a time to critique team members, but an opportunity to learn from one another, collaborate better as a team, and produce higher quality results
  • If there’s a disagreement on an implementation then include the rest of the team for discussion
  • Reviewer should challenge what use cases have been tested
  • Depending on the feature you may review with a different member of the team
  • If you don’t know who to review with then review with myself

Collaborations

Don’t review with the same person every time, spread them out.  This will maximize the dispersion of knowledge among the team.

Team case tracking

One of the most important functions of a responsible Team Lead is to know the progress of everyone’s activities on the team.  In an Agile shop, this is easy for everyone to see via the project planning board.  In my last position we didn’t employ these boards, but instead used case tracking software FogBugz (FB).  FogBugz is the best case management software I’ve ever used.  This should be no surprise as the Fog Creek Software CEO is none other than Joel Spolsky of the popular Joel on Software blog.  I’ll put a shameless plug below to entice you to learn more.

“FogBugz is the world’s easiest bug tracking system, built for teams who are serious about shipping great software.

  • cooperate with teammates
  • meet deadlines
  • maintain control of your projects
  • integrate with source control

FogBugz incorporates the lessons Joel Spolsky and the team at Fog Creek have learned over a decade of learning how to write software better.”

I track cases in my team through a few different filters, but in order to see what everyone on my team is actively working on in one list, this is a screen cap of the generic filter I use in FB

You can apply this filter to any of the predefined FB filters available.  It’s a simple convention.  Just add the following to the “containing” section of your existing filter.  Notice that I also include a virtual user that I use as a bucket to put cases anyone on the team can pick up and work on.

assignedto:"Team Pick Up User" or assignedto:"Team Member 1" or assignedto:"Team Member 2" ... or assignedto:"Team Member n"

Case triage

When I get dumped with a lot of cases and need to provide estimates (i.e. for a new milestone).  I book a room with the whole team and go through them one at a time.  I’ll take notes in a spreadsheet about each case.  I’ll apply notes to actual cases after I’ve had some time to digest what was said.

The team can assign the majority of cases estimates, but sometimes it’s not possible until further investigation is performed.  I use my best judgement on how long it will take to complete these cases based on our current calculated throughput of cases per day per developer.  Historically, on my team this has worked out to about 1.25 cases/day per developer.

An Overview of Continuous Delivery for the Newbie

The purpose of this post is to provide an overview of Continuous Delivery to people who may have never heard of it.  I’ve been a fan of Continuous Delivery for about 6 months now.  For the latest news on the topic I would recommend checking out ContinuousDelivery.com[1].  I still have a lot to learn, but I think I’m comfortable enough with the basics to talk about it.

What is Continuous Delivery?

One of the biggest pet peeves a stakeholder or PM will experience is not being able to see the ongoing progress of an application.  Admittedly, development methodologies like SCRUM[3] with its iterative development cycles (or sprints) have made feedback pretty quick and easy to quantify, but it still usually has to wait until the end the cycle.  A short sprint of two weeks is sometimes too long for PM to wait.  So what’s the solution?  Always have something built, deployed, and tested in a client environment.  This gives an Agile team the ability to release software more often and with more confidence.  Enter the world of Continuous Delivery, colloquially known as “the last mile of software development”.

A colleague of mine got me onto the topic of Continuous Delivery when he plopped a book of the same title[2] on my desk.  It has been an area of interest ever since.  Continuous Delivery is an extension of Continuous Integration.  Running a CI build with exhaustive unit, integration, and other automated tests is great, but there is still work to gain confidence that a build is ready for release.  A release candidate (RC) needs to be reviewed by product management (PM) and other development sections like quality assurance (QA).  This is the last mile part, because it is often not automated and prone to human error.

  • In order for an RC to be trustworthy it needs to be deployed to a number of client system/network configurations and supported technology stacks
  • It needs to be thoroughly tested in all these environments

I’m still not convinced.  Why bother with it?

Setting up Continuous Delivery can require a significant amount of upfront investment, so it’s not something you can do under the radar without the knowledge of the rest of the development department.  You need buy-in, so how do you get it?  You tell PM how much smoother and often it will make releasing your software to the client. Releasing an application is often a nightmarish scenario for all people in development.   In a lot of cases it requires a symphony of manual human interaction to be confident a release is ready to go.  Upper management is often even more stressed than you at this time (hey, it’s their butts on the line at the end of the day), proposing the adoption Continuous Delivery should be easy when you can describe how much risk it will remove from these difficult times.

The easiest way to alleviate release concerns is to be able to automate and audit the release process and have it run as often as possible.

Sounds cool!  How do I apply Continuous Delivery?

The application of Continuous Delivery is creating a fully automated Deploy Pipeline, or as my colleague liked to call it, a Deployment Pipedream ;) .  The ultimate goal of which is to make the feedback cycle between developer commit and PM review in a range of different client configurations as short as possible.  In order to achieve such a goal you need an initial investment in the following areas.

  1. Engineer a Silent Install for your app.  A fully automated silent install that can be configured for any supported app configuration and client environment.
  2. Know how to configure all your dependencies through Configuration Management.  An extension of the above point.  Host system configuration and 3rd party dependencies should be inventoried and work should be performed in determining a way to automatically implement these pre-requisites in a client environment.  When choosing a component for the product consideration should be given to whether or not its deployment can be automated.  If a dependency deployment does not lend itself well to automated deployment then other components should be considered (if possible).Configuration management is the process of keeping a record of all supported client configurations for the product and its dependencies.  This includes sample data and other information used in setting up a test environment.  The information should be stored in source control.
  3. Provisioning client environments.  This process would involve automatically deployment client images that we can apply the above configuration to.  It could be accomplished by either working with a VMWare API to automatically restore a host image or by somehow restoring a host image to a clean state.  Client testing environments should be engineered environments, not works of art.  The process to engineer an environment to an acceptable pre-requisite state for our application should be a thoroughly documented process.  Ideally, there should be no surprises or manual intervention required in setting up these environments.
  4. Run testing.  Run automated client acceptance testing on the product and send reports to QA, developers, and PM.

A hypothetical scenario:  Deploying a .NET application server from scratch

If you have an application server component it will likely have dependencies on other components such as your host platform, web server, and RDBMS.  All these dependencies require configuration along with any configuration of the application server itself.  For this example I’ve chosen the following technologies.

  • Host: Windows 2003 x64 SP2
  • Relational: Microsoft SQL Server 2005
  • Web Server: IIS 6

Setting up each of these dependencies requires us to know how they should be setup (or configured).  It’s a relatively straightforward process (if time consuming) to automate the deployment of all these dependencies.  For the above technology stack I will briefly summarize how its configuration can be automated and what I would actually have in my Configuration Management repository. Then I would want to apply configuration and perform the following host environment setup steps.

1) The Host

Deploy an unattended install of Windows 2003 x64.  We need to communicate to the installer what is the right license key, install to the right partition, and to install .NET, IIS, and other system components.

What’s stored in Configuration Management?

Answer files (.cif extension) drive an unattended install.  See the following source on Unattended Windows Install[4].

2) The Relational Server

Deploy an unattended install of SQL Server 2005.  We need to communicate to the install what is the right default data and log directories of SQL Server, what components to install (SQL Server is a mammoth software suite comprised of Analysis Services, Reporting Services, Integration Services, etc), what service account credentials to use, etc.

What’s stored in Configuration Management?

Installation script (.ini extension) to drive an unattended install.  See the MSDN article How to install SQL Server 2005 from the command prompt[5] for more details.

3) The Web Server

Setup a virtual directory or web site on Windows Internet Information Services (IIS) 6.0.  Things get a little tricky when setting up IIS automatically, but it can be done.  Once you setup a web site or virtual directory option once, then IIS allows you to export this configuration to an XML file.  This file can be used as an input to a Windows Server script to setup IIS for you application server appropriately.

What’s stored in Configuration Management?

An exported IIS configuration (.xml extension) that can be run by the iiscnfg.vbs system script.  See the Microsoft Technet article Importing IIS Configuration using IISCNFG.vbs[6] for more details.

4) The Application Server

You’ve made it, you’ve setup your host environment and all its dependencies.  Now hopefully deploying your own application server should be as easy as dumping your application target directory to the root directory of your setup IIS web site or server, and applying a tailored application config for this environment.  If you were smart, you would have already engineered a silent install and preconfiguration script option into your project so this deployment is a piece of cake.

What’s stored in Configuration Management?

Your application’s silent install configuration script and the application config that reflects the defined technology stack of your dependencies.

Final thoughts

The key to this whole process is automation and Configuration Management.  I know it’s a huge scope, and it’s not something you’ll achieve over night, but it’s the definitely the direction all projects should move toward.  I would highly recommend the book titled Continuous Delivery[2].  It’s an excellent source of information and contains a lot of pragmatic recommendations to achieve this goal.  It’s part of the Martin Fowler signature series of technical books by Addison-Wesley.

One thing I would suggest is breaking up Continuous Delivery into smaller phases.  The completion of each will have a working Deployment Pipeline, albeit not a very robust one.  At some point I will post and easy to implement first step for teams that want to start down this path.

Works cited

Hello world

I go by wladek.  This is my first experimental posting in the blog-o-sphere.  I want to use this space as a medium for me to express my thoughts on subjects relating to software development, technology, and occasionally an opinion I feel might be of interest to someone other than myself.  If you feel that you can add to anything I may post then I encourage you to speak up and tell me about it (especially if you have an opinion perpendicular to my own).  I’m a man of few words and have always held the belief that if you don’t have something of value to add to a conversation then there’s no sense in saying it.  With that in mind I’ll try to keep my posts meaningful and concise.  Let the experiment begin.