Technical Difficulties

So, app_offline.htm isn’t a fullproof as we’d like. But just to catch up a few people who might be lost already…

What is it? How does it work?

The general idea is that you make a plain-jane html file, name it app_offline.htm, and drop it in the root directory of your ASP.NET website in IIS. IIS automagically knows that if it sees this file, that it should serve it for ALL requests. That way, when you’re doing maintenance, you can have a nice pretty message instead of an application in flux. Great idea right? Yes, it is! Too bad it doesn’t work 100% straightforward that way.

My current understanding (as in, probably wrong) is that IIS only checks for this when serving up ASP.NET content. Which I guess means that IIS isn’t actually doing it at all, but some ASP.NET framework gobbledegook is at some point. This has some pretty serious differences, the two I’ve ran into being that it’s not going to do this for static content requests, and that you have to actually get ASP.NET loaded before this file can be served.

Why I’m running into it

I’ve been working on automated build and deployment scripts for some website applications at work. I have it plugged into our CI server, and we do a lot of QA and UA on the destination environments (note: UA at this stage is done by our internal Product Owner, not an outside client). Since those environments will obviously get messed with whenever we check-in code changes, showing them a nice “under maintenance” to let them know what’s up is a good idea. They know how these servers work, but at least this way they don’t think they uncovered some bug that caused the site to start throwing errors.

We cranked out a quick app_offline.htm, and I worked it into the deploy script. The deployment script works in these steps:

1. Copy over the app_offline.htm
2. Delete everything else in the virtual directory (this is done so old pages/usercontrols/files/css/image/etc don’t stay around)
3. Copy over the new build
4. Delete app_offline.htm

However, when trying to access the site during this, most of the time you’d just get exceptions. WOW THAT’S WORKING GREAT!

How to avoid issues

First off – since the app_offline.htm done by the ASP.NET framework, it still has to be able to complete initialization. What this means is that if you have a global.asax defined in your web application (and you probably do), it still has to be able to load properly. And if you’re deleting your /bin/ folder before global.asax, then your global.asax can’t finish loading. BAM! Exceptions. This also is important when copying items back over. So, I modified my scripts to delete global.asax FIRST, then everything else. When I’m copying the new build over, I copy /bin/ first, then everything else.

But there’s more!

You’re probably also using a default document. Default.aspx is the one that’s probably actually in your project. If your site’s home page doesn’t have a specific actual page (like 99% of websites), users could just be on http://mysite.com/, and get the “Virtual listing not denied”. Why? Because you’ve deleted Default.aspx, so the ASP.NET processing isn’t happening, so your user gets the “virtual listing not allowed” error. You could add app_offline.htm to the default document rule, but that’s kind of crappy. The real solution here is not to delete Default.aspx. Instead, just copy over it.

My final deployment process

So, now it looks like this:

Copy over the app_offline.htm

Delete global.asax

Delete everything but Default.aspx and web.config (I know those files will ALWAYS exist – so I’m still cleaning out old garbage, but not getting nasty errors)

Copy over new /bin/ contents

Copy over the rest of the new build

Delete app_offline.htm

And now I’ve greatly reduced the change for a user to get nasty errors. Why only greatly reduced? Because if they were trying to get to static content, they’ll STILL get 404 (if the old file hasn’t been copied over yet), since the ASP.NET framework isn’t invoked. The solution for that? IIS7 and the integrated pipeline. You’ll want to add this to your web.config:

<?xml version="1.0"?>
  <configuration>
    <system.webServer>
      <modules runAllManagedModulesForAllRequests="true" />
    </system.webServer>
  </configuration>

Which is just yet another reason to only copy over your web.config, and not delete it. This setting tells IIS now that ANY requests in this virtual directory should fire the ASP.NET pipeline. However, our servers are IIS6, so we cannot take use of this feature. It’s low risk for us, so it’s not a large deal, but for those of you using IIS7 this is a nice thing to add, especially if you’re using MVC since none of your URLs directly link to an aspx page anyways.

Hope you find this information useful!

Yeah, this is a pain.

So, you want to use “out of the box” controls like a listview, or gridview, etc. You have a custom interface for getting your data, let’s say something like:

public interface IGetData
{
     IEnumerable<MyData> GetAll(int pageSize, int page, string sort);
}

public partial class MyPage : Page
{
  public IGetData MyData {get;set;}
}

That’s a pretty basic interface. You have a large database, and you need to very specifically control how paging and sorting is done (no returning everything, and sorting in memory, especially). Unfortunately, if you want to display that information with the standard webform controls, you will run into MANY issues. All your sort/paging commands will be jacked up, and attempts at manually setting what page number you’re on will leave you smashing your head into a wall. A wall with spikes (hooray for read-only properties…).

(To be honest here, I tackled this nonsense a few months back, and don’t remember the specifics of the trouble I ran into, so feel free to try it anyways!)

The way ASP.NET wants you to do it, is to have your (let’s say ListView) point to an object datasource. After fighting these things, I can say that object datasources are complete garbage. But, here we are anyways. Here’s what you’ll need to do to get it work correctly.

You’ll create your ListView, and your ObjectDatasource. Have your ListView’s DataSourceID set to the name of your ObjectDatasource. Then you’ll set various properties on the datasource for what functions to call at certain events. The overall way how the ObjectDatasource is going to work? It’s going to create it’s own f’in instance of your control/page, and call those functions. Yes. You heard me correctly. That means any controls/data on your page are USELESS. Since your ObjectDatasource has it’s own instance, those values won’t exist. Awesome, right! So how do we set this up anyways? Like so…

On your ObjectDatasource, you’ll want to set the following properties:

• MaximumRowsParameterName, SortParameterName, StartRowIndexParameterName – These are optional. If set, when the ObjectDatasource uses reflection to search for it’s select function, it’ll look for parameters with those names.
• SelectMethod – A name of a function in your codebehind, that has parameters equal to what you have set (in markup or codebehind) to your SelectParameters as well as the parameters listed above
• SelectCountMethod – Function with the same parameters as SelectMethod, minus paging and sorting parameters.
• TypeName – FQDN of your page or control
• EnablePaging – Set to true if paging is enabled. Obvious at least.
• OnObjectCreated – This is the bastard one. This is an event that you’ll want to wire into.

So, we’ve set up our ListView and set up our ObjectDatasource with a bunch of functions that don’t do anything. The first thing that you’ll notice is a different function for Select and SelectCount. We don’t want to actually have to search our database twice. The good news is that Select is called first, so I made a tiny custom object to hold the records we’re going to show, as well as the total count. I modified my interface to return this object instead, as well.

public class DataControlResults<T>
{
   public IEnumerable<T> Data { get; set; }
   public int TotalRecords { get; set; }
}

Now, we, can make a private field on our page (let’s define it as “private int _totalRecords”), and we call our Select method, we’ll store it.

public List<MyData> dsItems_Select(int pageSize, int startRow, string sortColumn)
{
  var results = MyData.GetAll(pageSize, (int)(startRow/pageSize + 1), sortColumn);

  var listData = new List<MyData>();
  listData.AddRange(results.Data);

  _totalRecords = results.TotalRecords;

  return listData;
}

Again, unfortunately, we have to do a little wonkiness with the paging. I prefer page and pageSize, but the data source uses starting row and page size. Oh well.

Now, our SelectCount just returns _totalRecords, so easy enough there that I can skip formatting code!

However, if you, like most people, are using a DI/IoC framework, your actual concrete implementation of your interface will be null! Most frameworks, when in the ASP.NET world, are instantiating those properties in your global.asax, or perhaps a custom Page base control, etc. Because the ObjectDatasource, though, is creating a copy of your page/control through Activator.CreateInstance, that’s all gone! SWEET! (That’s sarcasm, by the way.) So, that’s where the OnObjectCreated event is used.

protected void dsItems_ObjectCreated(object sender, ObjectDataSourceEventArgs e)
{
  var newObj = e.ObjectInstance as MyPage

  newObj.MyData = MyData;
}

There ya go, now it’ll actually have a facade to use. Sigh.

At the end of the day, it works pretty simply, but I originally just wanted to throw up a ListView control, capture the paging/sorting events from the ListView (triggered from Commands from objects displayed in the ListView), and manually call my facade with proper paging/sorting/search parameters as needed. Then I’d just bind the results to the ListView.DataSource property, and be set. But that really just isn’t how they intend on you using these controls. They really want you to use some DataSource control, which unfortunately adds nothing but more noise to your markup and code-behind. If you want to have Updates and such as well, it’s the same pattern again.

Hopefully this approach can help someone else who wants control over their data access in their webform project! Thank god our next project is MVC!

This was an idea that recently came up during training. One of the large values of using Scrum is the visibility it can provide to the rest of the business organization – clients, product owners, managers, etc. It helps plainly set out what the team can handle, and identify issues that are preventing features from getting done.

As a team gets it’s velocity defined, it may be beneficial to not assign story points to stories in an iteration related to working on legacy systems, production issues, long-standing (or recurring) bugs/defects, or things that in general derive from various forms of “technical debt”. Many managers (or product owners, etc), especially the more teams they have to track, will usually not take the time to really investigate why a team may be having troubles, and making the issues more obvious (such as a low velocity) may be a way to help clearly signal them there are issues that need to be addressed. After given the “green light” to tackle some technical debt, future iterations would see an increased velocity.

On the other hand, you’d still probably need to story point those “non-business value” stories, because you will still want to track when they’ll be resolved, and to be able to include them in releases. Perhaps you’d run multiple iterations in parallel – one for business value, one for everything else, and assign velocity independently. Of course, that has the overhead of managing two parallel iterations, which might be problematic depending on your Scrum tracking methods.

I think it’s an interesting idea that has some merit, but still has some kinks. I like the principals behind it, just not sure how it’s best to proceed with it.

Maybe it’s just me. Maybe my perceptions are skewed for some reason or another. It’d be nice. And no, this is not an “Evil Rob” post.

It seems to me that the .NET community is becoming more and more of a popularity contest. A person’s ability to “play around” with new technology, and blog/twitter/whatever a lot about it seems to be more important than providing actual working solutions. I really hope that’s not the case. Developers should be able to get job based on their ability to provide business value to the client, in whatever form that may be – which is unlikely to be by making small demos or “widgets” of functionality.

It seems that unless you’re working with the latest and greatest technology, that any work you’re doing is useless, and you’re a subpar developer. It seems that cliques and an elitist mentality are emerging, which is unavoidable, but there’s a chance it’s becoming the prominent (or at least prominent enough) mentality in our community. Instead of being a community focused on sharing knowledge, it’s more of a one-ups-manship contest.

The reason all this is coming to a head, is because I’d like to blog about the work I’m doing. I found myself thinking “Man, no one cares about this.”, and “Eh it’s webforms, I don’t want to catch flack for it”. Then it occurred to me that there’s no way I’m the only developer out there still dealing with a legacy webforms app, that will be facing the same challenges I have. Maintenance is a large part of what we do, and something we should always plan for. So it’s always good to know about how to make those old apps easier to work with, or to fix that random bug that pops up in software no one has looked at in a year.

By the same point, we definitely need people blogging and trying out new things with the new tech. I know that for myself, and surely many others, a lot of the enjoyment comes from facing challenges, and finding new, efficient, and interesting ways to overcome them. Without checking out the new stuff, we’d be stuck in a rut.

So it all matters. New stuff matters. Old stuff matters. And it’s important that we keep a good balance with everything, so we can accept and all grow together as a community, and not become a circle-jerk of assholes.

I should also take this time to say that this is NOT an attack on any one (or group) of people. I’m not saying that “popular” people in the community are not (or can not) be competent developers producing fully functional software. This has just been my perception of how things have trended over the past couple of years. I really, really, REALLY hope I’m wrong.

With all this said, I plan on blogging about what I’m actually doing more. I’m also very lazy, so if it doesn’t happen, it’s not because I became shy again.

So please leave me some comments! Am I way off of my rocker here? Am I insecure because my mommy didn’t hold me when I was a child and my vagina is full of sand? Or did I hit on a few nuggets of truth?

I swear, if I hear that damn “designing software is like building a house” analogy again, I’m going to shoot someone in the face (probably myself).

Software is organic. It evolves. It changes. When you build a house, it’s set in stone – literally. All the floors depend on one another and build on one another. You can’t radically change one below it without disrupting those above it. If you’re designing your software to be like a house is built – you are doing it WRONG. Your software shouldn’t be so rigid and brittle that it’s a great pyramid of dependencies, and changing one thing brings the whole thing crumbling down. We have methodologies and technologies to keep us from doing that exact problem – and they’re not new! They’ve been around for a decade or more. Coping with and embracing change is exactly what good software is about. The only constant in software you can rely on is that things will need to be changed. If you’re designing your software so that it can’t meet that requirement, then it fails at it’s most fundamental level.

If you still don’t agree with me, google some of these terms: Dependency Inversion (also known as Inversion of Control) (DI / IoC), Agile software, Single Responsibility Principal (SRP), Open-Closed Principle (OCP), Liskov Substitution Principle (LSP), or just buy (and read) this book – Agile Software by Robert Martin.

That’s not to say that you can write software that will be able to gracefully handle every change ever. You can’t (not without an unlimited budget/time that is, which doesn’t exist in The Real World). But most business domains have areas that are more volatile than others, and you can reasonably predict which areas you need to be more careful and purposeful when adhering to proper software design principles. You want to be able to say that you can handle most changes and refactoring easily, but sometimes things will require change in an unexpected way. So you implement those changes, and adjust your architecture to adhere to similar changes of that nature in the future. That’s why we, as software developers, get paid what we get paid – to learn from our mistakes, to improve our practices, and help provide more business value in the future by providing the ability to handle owner’s requests more efficiently. When a change request / new feature comes down the pipe that we didn’t anticipate, we explain to them the cost (time/money/effort/resources) to handle that request, and we do it just like we do any other work. We don’t bitch and moan that doing our job is hard because we fucked up the initial design, or to whine about how software is exactly like masonry work.

Maybe no one uses that analogy anymore, everyone agrees with me, and I’m overreacting. But I did just see it again the other day. Maybe that’s the freak occurrence though? I hope so.

The production app I’m currently working on involves reading a client-provided extract (on the magnitude of hundreds of thousands of lines), transforming that data, and importing it into our system for processing. This happens once a day. The current process involves importing this information into staging tables on the DB side, then running stored procedures to transform the data.

As part of an effort to both improve efficiency as well as add in scalability, the DB team has decided this process should import into session-level temporary tables. This will allow multiple imports to go on at once if need be, as well as give some performance enhancements. The software side of this requires that the importing software opens a connection, and then creates the temporary tables on that connection. From there, the software reads the client-provided extract in about one thousand lines batches, and uses SqlBulkCopy to transfer it to SQL server.

Most of the architecture in this problem was laid out already, and not having any previous experience dealing with data sets this large, I followed the rest of the team’s leads. The temporary table creation needed to be done on the software side, and the scripts themselves needed to be part of the project. Not wanting to rely on folder paths to read in the data, and really not wanting to have string variables that held the scripts, I opted to leave the scripts intact, but use them as embedded resources. I haven’t worked with embedded resources before, honestly, so I did find out a couple of things.

The main call you’ll need is:

Assembly.GetAssembly(yourAssembly).GetManifestResourceStream(fullResourceName)

The tricky thing can be the file path. It uses periods to denote folder paths to load your project. Not difficult, but not the most obvious thing either. I wanted to “hide” that from my consuming apps, as they really don’t care about how the resource accessor likes to make it’s names. They just want their file. They do know about the project structure of the assembly that has their resource, however.

The next thing that I accidentally did wrong was the assembly – you need to provide the assembly that the resource lives in, for obvious reasons. Given that this helper class lives in a “Utilities” type of project, it’s no shock that as soon as I refactored my class out, my tests broke.

So, with those two thoughts in mind, here’s the API I wound up with, along with the full class:

public static class ResourceHelper
    {
        public static string GetResourceAsString(Type typeOfCallingObject, string namespaceName, string fileName)
        {
            return GetResourceAsString(typeOfCallingObject, string.Format("{0}.{1}", namespaceName, fileName));
        }

        public static string GetResourceAsString(Type typeOfCallingObject, string namespaceName, string folderName, string fileName)
        {
            return GetResourceAsString(typeOfCallingObject, string.Format("{0}.{1}.{2}", namespaceName, folderName, fileName));
        }

        public static string GetResourceAsString(Type typeOfCallingObject, string namespaceName, string[] folderPaths, string fileName)
        {
            var fullFolderPath = new StringBuilder();

            if(folderPaths != null && folderPaths.Count() > 0)
            {
                foreach (var folderPath in folderPaths)
                {
                    fullFolderPath.AppendFormat("{0}.", folderPath);
                }

                // Remove the trailing period
                fullFolderPath.Remove(fullFolderPath.Length -1, 1);
            }

            return GetResourceAsString(typeOfCallingObject, string.Format("{0}.{1}.{2}", namespaceName, fullFolderPath, fileName));
        }

        public static string GetResourceAsString(Type typeOfCallingObject, string fullResourceName)
        {
            var fileStream = Assembly.GetAssembly(typeOfCallingObject).GetManifestResourceStream(fullResourceName);

            if (fileStream == null)
            {
                throw new Exception(string.Format("Resource '{0}' not found!", fullResourceName));
            }

            var sqlStreamReader = new StreamReader(fileStream);

            return sqlStreamReader.ReadToEnd();
        }
    }

Note that most times, the first parameter is liable to just be “GetType()”. I could’ve maybe made it an extension method, or made the type a generic parameter, but it really felt like overkill for such a small class. Here’s some examples of calling it (from my unit tests):

ResourceHelper.GetResourceAsString(GetType(), "UtilitiesTests.RootResource.txt");
ResourceHelper.GetResourceAsString(GetType(), "UtilitiesTests", "RootResource.txt");
ResourceHelper.GetResourceAsString(GetType(), "UtilitiesTests", "ResourcesFolder", "FolderResource.txt");

ResourceHelper.GetResourceAsString(GetType(), "UtilitiesTests", new[] { "ResourcesFolder" }, "FolderResource.txt");
ResourceHelper.GetResourceAsString(GetType(), "UtilitiesTests", new[] { "ResourcesFolder", "NestedFolder" }, "NestedResource.txt");

Yeah, this is all extremely basic, but thought it still might be useful for someone. I had fun doing it, none-the-less.

I know. It’s crazy. I had been dissatisfied with my old position for an extremely long time, and it eventually came to a boiling point, leading me to switch jobs in the middle of THE RECESSION!!! ZOMGS!!!!!

Things worked out amazingly, though. I had my new job before my severance even ran out. Nothing like getting paid to nurse my crack addiction play World of Warcraft.

I’m the start of what is basically a new team. I’ve been here a little over 3 weeks now. The previous team consisted of a single developer, who has been here for about eight years. He’s moving to more of a management position, while me and another developer (a Web/UI focused developer) will be working for him. There’s also 2 DBAs that we work closely with, but aren’t technically on this team. We steal a lot of their time though.

My role is “Senior .NET Software Engineer”. What do they mean by “Engineer” exactly, you ask? Well, that’s a good question that took me a bit to find out myself. I think I can explain it best by thinking of what an engineer by any definition really does – makes things work! My job is to make it work, however I feel that should be accomplished. My boss helps critique my proposed solutions, typically by giving me information about how many of the external systems interact with what I’m working on. That’s where a lot of the complexity is currently. The company deals with the health care supply chain. Not only is it my first foray into the health care arena, which is complex in and of itself, but the supply chain has many unique factors and forces at play at the same time. It’s a lot to take in. I’m finally getting a good base grounding (although not very in-depth yet) of how the entire system works.

My current “day-to-day” focus involves refactoring an existing system made for a single client to make it extendable for the second client it needs to support. I’m finally getting a chance to put to use a lot of the great information I had read recently. The code is pretty straight forward, so while it’s not hard to understand what it’s doing, it’s very hard to break up and make extendable. I’ve been referencing some books pretty frequently to help me make the right decisions, however, and I’m excited about how it’s turning out. My boss is liking my work as well (which is very important to the checkbook).

The work here is also extremely data heavy. Collecting and processing the data is a huge concern. Working with tables that contain millions of records, and importing, exporting, and converting them. So I’m learning a lot of information about how to write efficient SQL and designing secure databases.

On top of that, the company is fully committed to doing Scrum in a proper fashion. That’s great – but of course, each company will have it’s own “spin” on it. This is an extremely large company, and we’re actually a remote office, so it involves a lot of phone conferencing. So that’s been another challenge to learn how to work with properly.

All in all, however, this is going pretty great. The work is challenging. It’s a new business domain, which is a great change of pace, if nothing else. I’m being allowed to propose architecture improvements, and implement them. My boss is committed to making sure we have good hardware and the latest development tools. He’s also always looking to the future – we want to wrap up the current project, then move on to the next. Then the next. Then the next.

So, I hope to blog a little here and there about what I’m doing. I figure most people will find it mind-numbingly boring or introductory. Hopefully not though. A lot of it will be difficult to explain or may not make sense. We’ll see how it goes.

Oh, and a freaking huge 11.5% raise. It was over the top end of the range I was asking for. And a bonus plan that actually pays about 10% of your salary a year (and yes, people actually get their bonus). Take that recession!