Friday, March 28, 2014

Mixing C# and VB User Controls in the Same Web Application

Authored with Kusuma Kora

At my employer, DealerOn, we have a VB web application and want to move completely to C#. Unfortunately, we do not have the luxury of doing a mass-conversion. For standard class files it is straight forward to place them in another project and reference the project, but web controls are not so simple. We were able to come up with a seamless way to make this possible and we wanted to share details about the solution here to help others with this situation.

First, a bit about our situation:

  • We have a web application (as opposed to a web site)
  • We pre-compile our web application before deploying it to our various servers
If you don't pre-compile, you'll be able to get by without the last step.

Step 1: Create a C# Web Application & User Control

You'll need a web application to place your C#-based user controls in. Add this to the solution, and then add a project reference to this project in your VB web application.

Now, you'll want to create a C# user control so we have something to work with to test later.

Step 2: Embed the User Controls in the assembly

The way that this works is that we embed the ascx files in the C# assembly, and then we adjust the VB web application to use a custom virtual path provider that is capable of finding the ascx files via the assembly instead of via a physical path to the control.

First, we create a class (in C# of course) called EmbeddedResourcePathProvider:
public class EmbeddedResourcePathProvider : VirtualPathProvider
{
  private bool AssemblyPathExists(string path)
  {
    var relativePath = VirtualPathUtility.ToAppRelative(path);
    return relativePath.StartsWith("~/Assembly/", StringComparison.OrdinalIgnoreCase);
  }

  public override bool FileExists(string virtualPath)
  {
    return AssemblyPathExists(virtualPath) || base.FileExists(virtualPath);
  }

  public override VirtualFile GetFile(string virtualPath)
  {
    if (AssemblyPathExists(virtualPath))
    {
      return new EmbeddedResourceFile(virtualPath);
    }

    return base.GetFile(virtualPath);
  }

  public override System.Web.Caching.CacheDependency GetCacheDependency(string virtualPath, System.Collections.IEnumerable virtualPathDependencies, DateTime utcStart)
  {
    if (AssemblyPathExists(virtualPath))
    {
      return null;
    }

    return base.GetCacheDependency(virtualPath, virtualPathDependencies, utcStart);
  }
}


The inspiration for this code is this MSDN article: http://code.msdn.microsoft.com/CSASPNETAccessResourceInAss-6725d61a

Second, we need to change the Application_Start method to use the custom path provider. Have it call this static function in C# to do this:

public static void ConfigureResourcePathProviderBasedOnRuntimeEnvironment()
{
  if (!IsAppPrecompiled())
  {
    var embeddedResourcePathProvider = new EmbeddedResourcePathProvider();

    // we get the current instance of HostingEnvironment class. We can't create a new one
    // because it is not allowed to do so. An AppDomain can only have one HostingEnvironment instance.
    var hostingEnvironmentInstance = (HostingEnvironment)typeof(HostingEnvironment).InvokeMember("_theHostingEnvironment", BindingFlags.NonPublic | BindingFlags.Static | BindingFlags.GetField, null, null, null);

    if ((hostingEnvironmentInstance != null))
    {
      // we get the MethodInfo for RegisterVirtualPathProviderInternal method which is internal and also static.
      var mi = typeof(HostingEnvironment).GetMethod("RegisterVirtualPathProviderInternal", BindingFlags.NonPublic | BindingFlags.Static);

      if ((mi != null))
      {
        mi.Invoke(hostingEnvironmentInstance, new object[] { embeddedResourcePathProvider });
      }
      else
      {
        throw new ApplicationException("RegisterVirtualPathProviderInternal MethodInfo is NULL");
      }
    }
    else
    {
      throw new ApplicationException("HostingEnvironment is NULL");
    }
  }
}

If you don't plan on pre-compiling your application then you can change this code to remove that check. If you are going to pre-compile your application then create a function that returns false for now and we'll revisit this later in the article.

Step 3: Wire it up and Test

Now that we have the user control and the web application, its time to put it together.

To begin, you need to adjust the user control to be an embedded resource. For your user control, open the properties window and change the build action to embedded resource as illustrated in this screenshot:


Now, back in VB-land you need to load the user control from your C# project. Because the control is now an embedded resource the path changes to include the embedded path. Whereas typically your call to load the control looks like this:

LoadControl(~/x/y/mycontrol.ascx)

This will now change to:

LoadControl(Assembly.MyC#ProjectDllName.dl.MyC#ProjectDefaultNamespace.x.y.mycontrol.ascx)

It looks a little ugly, but wait until you get to the part where we pre-compile. :-) More on that in a minute, but now is a good time to get your web application running locally with a VB aspx page loading a C# control. Embedded resources are always a little tricky to work with as it is very easy to get the path wrong.

If you are not pre-compiling your web application you can stop here. 

Step 4: Precompilation

Pre-compilation adds another layer of complexity, and this causes the embedded resource system that we just implemented to not work. We'll touch on this later, but for now it comes down to this: IIS has a completely different way of locating things when the application is pre-compiled. 

What to do? It ends up being pretty straight forward. We use our custom path provider only when the application is not pre-compiled. We also need to do some structuring on the build side of the house to combine two pre-compiled web applications together.

First, change your implementation to flag if the application is currently pre-compiled our not. We use an application setting: 

return bool.Parse(ConfigurationManager.AppSettings["PRECOMPILED"]);

Second, we need to handle the situation where our path to LoadControl is sometimes going to use the embedded resource format and sometimes it is going to use the standard path lookup. What we decided to is introduce an extension method to the TemplateControl class that takes in the standard path format (e.g. ~/x/y/mycontrol.ascx) and call it instead of LoadControl():

public static Control LoadControlExternalToCurrentProject(this TemplateControl container, string virtualPath)
{
  if (string.IsNullOrWhiteSpace(virtualPath) || !virtualPath.StartsWith("~/"))
  {
    throw new ArgumentException("Argument is not a valid virtual path that starts with ~/", "virtualPath");
  }

  if (IsAppPrecompiled())
  {
    return container.LoadControl(virtualPath);
  }
  else
  {
    return container.LoadControl(~/Assembly/FileNameOfDLL.dll/AssemblyDefaultNamespace. + virtualPath.Replace("~/", "").Replace("/", "."));
  }
}

Note that this requires matching the folder structure to the namespace perfectly to work (probably a good idea anyway). In our VB code, this is what the call to load controls now looks like:

Me.LoadControlExternalToCurrentProject("~/a/b/mycontrol.ascx")

Third, we need to merge the two pre-compiled web applications. Your way of doing it may vary (we use PowerShell) but what you need to do is pre-compile both applications, and copy certain files out of the C# one into the VB one. The file patterns are:
  • ~\bin\App_Web*.dll
  • ~\bin\.*compiled
In PowerShell, this command will do it:

Copy-Item $pathToCSharpPrecompiledApp\bin\App_Web*.dll $pathToVBPrecompiledApp\bin -ErrorAction stop
Copy-Item $pathToCSharpPrecompiledApp\bin\*.compiled $pathToVBPrecompiledApp\bin -ErrorAction stop

Don't forget to flip your web.config setting to indicate that the application is pre-compiled.

Interestingly enough, we like that the pre-compiled application does not use our custom path provider. The application runs "pure" on the actual server; our "hack" really is to run things locally. 

Conclusion

One thing worth mentioning is that we benefited greatly by spending the time to dig into how IIS loads "stuff". We used the wonderful, free dotPeek decompiler tool from JetBrains to understand at at least a high-level what happens when LoadControl() is called. People may pan ASP.NET for its quirks but it is pretty obvious that there is a method to the madness. The hour or so of digging definitely lead us down the path of getting it all working pretty quickly. Don't be afraid to dig around; up-voting that stale Microsoft Connect case to improve the design of LoadControl() will get you nowhere!

Before you know it, your ratio of C# to VB will be greatly increased -- a happier outcome for code and developers alike. 

Hopefully you find this article helpful and are able to employ it in your situation.

Wednesday, February 26, 2014

Frequently Unasked Questions by Job Seekers

As a seasoned professional who has served as both the interviewer and the interviewee, I’ve noticed that there is a tendency for candidates not to ask certain important questions. If you’re a candidate looking for a software development job, and you don’t ask these questions then you could make an uninformed decision about your would-be employer. Be sure to ask these questions as part of a follow-up interview, phone-call, or visit.

What is the Retention Rate?

If the team you are considering joining has a poor retention rate then that is a red flag the size of Texas. This typically indicates that there are systemic constructs in place that are causing poor retention and it is likely to impact you - and your career - considerably and negatively. People do come and go, and if the best developer left for GoogleAmaFaceSoft or today’s hot startup, well OK, but if the average tenure for a developer is less than two years then something is terribly wrong.

All companies know at least anecdotally what their retention rate is, so ask about it. Ask what happened to the last person who joined. Ask for specifics, and ask both the recruiter and hiring manager. Don't accept evasion or inconsistency in these answers. Also talk to members of the team and find out how long they've been there. Since your job position is likely advertised as added headcount to a "rapidly growing team" really confirm this is the case.

Do you require a coding sample?

No company should hire any developer without requiring a coding sample. The mechanics may vary. It may be a whiteboard only or pen-and-paper problem during the interview (ok). It may be a homework assignment (better). It may be that you submit information about open-source contributions or a link to your active GitHub account (best). But you should be evaluated in this way, mostly because you want to ensure that the others at the company you will ultimately work for have been evaluated this way.

The fact that a company doesn't require it indicates that they either don't know much about recruiting people (leading to lots of churn) or they are not interested in recruiting high-caliber developers. It’s best for your career to work with the best talent possible. Long story short: avoid any company that does not ask for a sample of your work.

Can I see where I'll be sitting?

The company will likely take you on a tour of the office and point out things, but they probably won't necessarily show you where exactly you will be sitting. Ask your manager where this will be and ask to see it.

First, you need to know what the environment is like. Is it loud or quiet? Is it free of distractions? Private or not? Cramped or spacious? Is there abundant natural light? Ask all these questions and more to arrive at the answer to this one: Are you able and willing to do your best work there? (Beware of promises regarding "new" or "better" or "different" offices as the people who promise these things are generally not the decision makers regarding how the future space will be built.)

Second, you can use this as an opportunity to learn about the culture by seeing it as opposed to reading it on a website or asking about it. What rooms do you pass on the way to the office? Are there "toys" such as a scooter or someone's pet dog? Do people decorate their cubes for the holidays and put personal effects in them? Is there an empty liquor bottle on someone's desk? Sometimes what you don't see is just as important as what you do see. Take good mental notes and allow this to impact your decision – after all you’ll be there for 8+ hours a day.

What tools are available to me as a developer?

Your company should be willing to provide you with the best tools that money can buy. Even the most expensive tool pales in comparison to the cost of a developer. If the company is not willing to invest in great tools than this can be generally extrapolated to mean that the company is not willing to invest in the development team as a whole.

This can be a bit difficult to probe for. When talking to a development team member, ask about the IDE setup. For Java, if the company shuns IntelliJ in favor of Eclipse due to cost, you have your indicator. For .NET, if the team doesn't use ReSharper due to cost, same thing. Ask the manager what the process is for purchasing a useful piece of shareware or a book (it should be minimal - "just buy it and turn in the receipt" is best). And, find out what the computing rig you will be using is like. It should be generally described as "new, state of the art, with two monitors".

What will you do to help me grow?

It is essential to understand both the tangible and intangible things that a company does to invest in you and the development team as a whole.

Find out what activities the employer does to encourage you to grow in the field. So many potential employers extol the virtues of open source software contributions, attending conferences and being recognized in the field, but many companies don’t do anything to foster these things within their own teams. I don’t think “20% time” (ala Google) is reasonable to expect, but the company should put some skin in the game when it comes down to it. How to tell? Ask what the company policy is regarding time off if you were invited to speak at a conference. Would you have to burn your own vacation time for it? Would the company pay for your travel expenses? Also find out if your company sponsors any meetups or user groups, or otherwise encourages attendance to these types of events.

It is also important to gauge a company’s interest in keeping current. It is not uncommon to deal with legacy systems and languages, but the company should be generally interested in using modern tools and doing new things. Find out how the reaction would be if you were to propose a new programming language or major system change (e.g. new source control, swap out the ORM mapper, etc.). The best companies debate these things – oftentimes fiercely – instead of responding with a knee-jerk “no” or “we don’t have time”. Another evidence-based approach is to see what the developers of the company contribute in terms of blog postings, open source contributions, and so forth.

What were listed as some of the things that are "done well" from the last retrospective?

So you're joining an Agile team. All (four of) my readers believe in Agile and don't want to work on a Waterfall team. No problem, you say, because the position description mentions it. Don't take anyone's word for it - many teams are actually doing Waterfall but say they are Agile, or are "transitioning" to Agile, or are doing some chaotic development process that they simply call Agile. My favorite question to root this out: When was the last retrospective, and what was indicated as done well?

If the team does not do retrospectives then you are not joining an Agile team. This is a good launching off point to ask other questions about the engineering processes in place and learn about how projects are managed. Be sure you are comfortable working in the framework that is in place and be sure that the team is engaged in confronting the issues that make development inefficient.

Conclusion

While there can never be 100% certainty that there will be a good fit based on an interview, I feel that if developers asked just a few more questions during the interview that they could know more about what they are getting into. I hope you find it useful and I am happy to take comments and suggestions as to other excellent questions that I may have missed.

Wednesday, December 18, 2013

Software Engineering at DealerOn

During interviews or in conversations with other tech folks, a lot of people want to know what software development is like here at DealerOn. Unlike my colleague Frank DeCaire’s posting “Working at DealerOn” which talks about our technology and growth this one talks a bit about our practices and processes which permit us to deliver software. 

Kanban

As we have grown our team during all of 2013, it was clear that we needed to formalize our software development practices. We settled on Kanban as our chosen software development methodology. We chose Kanban because it works hand-in-hand with continuous delivery practices (more on that in a moment), it allows us to always work on the most important thing for the business (they change their mind a lot), and it is less estimate / metric driven than other Agile approaches.

The single hardest part about adopting Kanban is moving away from giving firm completion dates. In our pre-Kanban days we used to do this and unsurprisingly we would routinely miss them for the usual reasons of unknown complexities or change in priorities. We now have been doing Kanban long enough to provide forecasts of when something may be done based on our past performance. As priorities change, so does the forecast.

There are certain commitments for customers or things we need to get done by a trade show. This is reflected by prioritizing those items at the top or near-top of the queue so that they will get done before other things.

Bugs and Enhancements vs. Projects

This gets a little bit more into the nuts-and-bolts of our Kanban implementation. We maintain two separate backlogs. One has our projects (and all technical tasks and pre-release bugs chained to them) and another has our post-release bugs and enhancements (which are non-bug small improvements to the software).
Generally everyone on the team works on projects. As a team, we typically review new projects and task them out together. This allows everyone to know what is going on, make suggestions, and contribute to the project. In reality, it works out that one or two people carry a project to completion with limited involvement by other team members, but others can get involved at any time to push it forward to completion. All team members are involved with all areas of our platform (we’re still small enough for this to work) so it is important for everyone to be knowledgeable about every project regardless of whether or not they work on it.
Each day one person (according to daily rotation) does not work on projects. They instead work on bugs and enhancements. Also they serve as the “point of contact” for our customer teams if they have questions. In true Kanban style, the bugs are taken in order of priority and a lack of knowledge on that system is not a pass at fixing it. This has resulted in almost no silos of knowledge about all but perhaps a handful of our legacy systems. One non-Kanban twist is that we flag certain bugs and enhancements as “quick” and action those first which prevents fifteen-minute things (e.g. “Change the label from ‘New vehicles:’ to ‘Vehicles, New:’) from clogging the backlog.

Continuous Delivery 

DealerOn has always been, and continues to be, a continuous delivery company and this goes in hand with Kanban nicely, which has no real notion of release dates. We release the various components of our software once or sometimes multiple times per day. This is enabled through fully scripted packaging and deployment processes which permit us to deploy with a single click of a button to any environment, all orchestrated through Team City. A variety of automated tests at all levels (unit, integration, and end-to-end) as well as server and log monitoring help ensure we don’t introduce any regression bugs. Rollback (or “rolling forward”) is just as easy which serves as an excellent insurance policy in the event of an undetected issue.
The business owners and our customers love this because they can see something that was broken get fixed or something small that they asked for in just a few hours or a day. We never have to say “it is fixed but our next release is on the 1st” which is sure to disappoint.

One issue this can cause is that it is difficult to ensure everyone is aware of all the changes to the software, especially as the teams grow. We need to work on better ways of creating release notes and ensuring that they are digested by everyone.

From a development perspective, this requires a slightly different mindset. Developers commit knowing that their changes will be tested immediately and may go live. There is no “tightening for the release” period often found in classic waterfall and many Scrum teams. Good test coverage, code reviews, and adherence to coding standards -- discipline -- are important to ensure successful releases and maintainability. All of our team members have ownership of the packaging and deployment systems as well as our infrastructure – not just a “release manager” or the IT folks.
We use branches for larger code changes, typically done for projects but is sometimes done for certain bugs. The determination of what gets done on a branch is part art and science, but as a general rule if multiple people need to work on something before something goes live we use a branch. We use Mercurial as our source control system, and it works very well with Team City which conducts our automated builds, testing, and deployment.

Conclusion

I hope that this article provides some insight into what software delivery is like at my current company, and perhaps you'll find it useful if you are looking to learn more about us or you are looking into adopting Kanban practices at your organization.  If you have any experiences to share or have any questions, feel free to get in touch with me.

Monday, April 29, 2013

Kanban in Practice

In this posting I talk about my experience using Kanban, why it worked and what is required to make the use of Kanban successful in software development companies. A few things motivated me to write this long overdue posting (sorry @amazonv), specifically a discussion that I had at a recent Meetup group and also that we are thinking about adopting Kanban at DealerOn.

This experience goes back to my tenure at Apprenda, where we had successfully adopted and utilized classic Agile development practices. We adhered to a two week iteration cycle, and produced fully tested, releasable packages every iteration though we didn't necessarily release every package to the public. While at Apprenda I served as the Test / Release Manager and so some thoughts are from that perspective as well.

Why we tried Kanban

I first heard about Kanban at a conference, and around the same time my colleagues Bryan and Dan begun to advocate for it, and finally we gave it a try at the beginning of a major release cycle. There were a couple of reasons why we liked Kanban even though nothing was really "broken" with the Agile cycle:
  • Too much work tracking metrics for too little benefit
  • The two week cycle was jarring, providing an inconsistent workload throughout the cycle
With Agile we spent a lot of time tracking velocity and attempting to produce accurate and reliable forecasts for the business. We never were particularly good at producing forecasts for the business owner despite all of the time spent trying to do so, but no one noticed anyway because the forecasts were disregarded since we were a startup and change our mind all the time. We ended up manipulating the numbers to make ourselves look productive, and it worked fine for the most part. But one thing was always true: no matter what the velocity, we'd work on the highest priority items and try to release it as soon as possible. The forecasts, quite frankly, were not that useful for any sort of long term planning.

In Kanban, everyone works on the next highest priority item as determined by the business. When the release date comes, the highest priority items that could successfully be completed by the team during the release cycle are done. In otherwords the business got what the team was capable of producing at the time.

Another issue we had with our classic Agile approach was due to our strict two week iteration cycle. This was rather jarring for the team. The burden was on development at the beginning of the iteration, and it transfered to the testing team near the end of the cycle. I can remember basically blocking every other Wednesday and Thursday evening to stay late to "test stuff" and rush for basically what amounts to be a false-deadline since we would not actually release every Friday.

In Kanban, the cycle goes away. The work load is evenly distributed all throughout because there is no start and finish officially defined. There are work limiters for each phase which help prevent the sub-teams from being overburdened.

What about the ceremonies?

Rather than have one of the many "agile ceremonies" for the sake of having it on a schedule, you have them when you need them. If you have a cool new feature to show, schedule a showcase. If you need to plan out a feature implementation, gather the team and conduct a planning meeting. The benefit is the meetings are shorter and more focused. Gone are the 5-minute showcases with nothing but a half-finished feature and bug fixes.

What enables Kanban to work?

A major risk to a Kanban team is producing unstable software. Since there is no end to an iteration where the software is tested and stabilized, this has to be enforced in a different way. As a part of the adoption of Kanban at Apprenda, we implemented a policy of requiring a 100% stable trunk. Features were tested and stablized on independent branches (we used Mecurial as a DVCS) and we made the appropriate investment in our testing infrastructure to be able to automatically deploy and run regression tests on all feature branches. This was no small effort, but was absolutely necessary to keep quality high and make Kanban work for us.

Another key to enable Kanban is discipline. While it is easy to pan the agile ceremonies as artificial and wasteful in a Kanban setting it is necessary to conduct each of the meetings when it is appropriate to do so. Planning meetings are important to ellicit a full understanding of the requirements and get feedback from the business before setting out on a work assignment. The daily standup meeting is important to "walk the board" and make sure all development and testing is progressing. The retrospective is still important to the team's self-improvement. You'll find yourself having less meetings overall but if you are suddenly having no meetings after introducing Kanban that is a problem.

The final item that is important to Kanban's success is adhearing to work limiters. This forces the team to confront bottlenecks. For example, at one point the testing team had a bit too much on its plate and the development team as a result made improvements to the testing framework and infrastructure that typically would be done by the testing team. At the same time, members of the testing team assist development with advanced debugging, bug fixes, and other development that typically is done by the development team when they reached their work limit.

In this posting I talked a little bit about how Kanban contrasts with Agile and some of the items that I feel make Kanban work well for software companies. If you have any feedback or questions please feel free to comment or contact me!