I have this habit of underestimating Microsoft, especially when it comes to the Office product line. At first blush, I was unimpressed with Office 2007.  To the average user, on first glance, it looks like Microsoft has traded one set of GUI widgets for another set of more complicated GUI widgets.  Combining this with the IT pundits declaring Office 2007 dead on arrival, I ignored it.  It's hard for me to get very excited about new widgets.

But last night I chanced upon the VSTO site (Visual Studio Tools for Office), and took a peek at this video.  I was amazed, and had to concede that Microsoft had pulled it off once again.  They have raised the bar for us programmers and expectations of users.  Briefly, your WinForms user controls can plug right into the Office environment, into either the ribbon or in the side docking panel.  Having worked on a system in C# that emulates a similar behavior, I was very interested, and a little disappointed that my craftily written code would soon be obsolete.

The thing about Office, and Word and Excel in particular, is that everyone uses it.  People get a warm fuzzy from software they use frequently and have some understanding of.  The number one feature request I get is to be able to download data into an Excel spreadsheet.  And VSTO 2005 SE (Second Edition) makes the job for programmers integrating into Office a whole lot better.  And it gives a compelling reason to upgrade to Office 2007.  This is only a minor upgrade for the end-user, but this is a sea-change upgrade for the Office developer.

VSTO 2005 SE will also allow Office programmers to achieve the type of service-oriented architecture that was difficult to achieve with our previous tools.  Since your user controls can do all those wonderful things like consume web services, chat over networks, query databases, access the file system, etc., the integration possibilities are nearly endless.  And the ability to dock your components right at the user's fingertips within the Office application makes the software convenient to summon for the end-user.

Now to work on my "reasons to upgrade" spiel for my clients...

I don't know how anyone can reconcile the two sayings "the best things in life are free" and "you only get what you pay for".  Still there are some good things you can get for free, including software.  Here are two links to some good software freebies, some of which I already use:

Tech Support Alert Best 46 Free Utilities

Tech Support Alert Page 2

I recently tuned into the podcast on IT Conversations, hosted by Phil Windley, and featuring David Platt, .NET programmer and author of the new book "Why Software Sucks".  In the podcast, Platt adds some new light to problems of user-facing software (as opposed to programmer-facing software).  Much of what he has to say has already been covered in Alan Cooper's unforgettable book, "About Face".

So what makes some software suck?  Software is supposed to make our lives and jobs easier to accomplish.  When software makes it difficult for a user to do things, or forces the user to remember things, or demands that the user do a complex dance with her keyboard and mouse, then that software sucks unequivocally.

But if you visit the computer lab and ask the programmers what makes software suck, they will have a totally different perspective.  Code that's hard to read, lack of unit test coverage, tight coupling of objects, redundancy: these are the problems that make software suck.

My opinion is that you can have the most highly refactored code, in all of its object-oriented goodness, and the software can still suck.  My fear is that there are many devotees of Martin Fowler and Uncle Bob, who in their own minds believe they are writing great software, but realize only after it has been foisted on the end user that it actually sucks.

When I tried to de-suckify a feature in the web application we recently released, I was often met with fierce resistance from programmers, whereas the business analysts were very open to new ideas on how to make the software more user-friendly.  One example was when we were required to build an interface whereby a loan representative could transfer the responsibility for a loan application to another loan rep.  In our lingo, this is called "pushing a loan".  The original specification required that the end user would enter the name of the receiving loan rep or their office then run a search against the database.  I believed that this method would be error-prone and that there ought to be any easier way.  I sat down with the business analyst in charge of this new feature and asked her "What do people use this for 90% of the time?"  She stated that most often, people would push the loan to someone else within their own loan office.  So I asked how she would like it if, when the user first selects the feature, he is presented immediately with a list of the loan reps from his own office, and that this list would precede the search function on the page.  The business analyst thought this was an excellent alternative, and she went to draw up the new plans.  Back in the development lab, things ran afoul because "everyone knows" that the search function "has" to be the first thing on the page.  This was going to "break the convention" of the look and feel of the rest of the web application, and this self-imposed directive should take priority over giving the user what they truly want.  Also, this alternative was going to require extra programming that we had not originally planned.  I stood my ground on the customer's behalf, and ultimately we all did agree on this alternative presentation of the UI.  But this incident illustrates that sometimes even software developers who are in good communication with the end user will fail to have the end-user's best interest as their top priority.

In our development shop, we developers would often lapse into the flawed mentality of "the customer's always right", and I have been just as guilty as others on this.  The customer would dictate the requirements and the developers' task was to merely implement whatever they came up with.  Occasionally, the customer would ask for something so goofy that we developers would have to put our foot down and everyone would go back to the drawing board.  Such are the risks of assigning the responsibility of software design to the customer, in this case the business analysts.  The business indeed knows best what the business needs, but is typically not the best choice in deciding how to get there.  The developer has the knowledge to create whatever a specification calls for, but his knowledge of the business need is often very shallow.  This leaves a disconnect between the customer and the developer.  This disconnect was partially alleviated by the agile process we used, during the breakout sessions.  These sessions are great at defining the business need at a high level that could be understood by both business analysts and developers.  These session are not so good at defining the user interface.  And user acceptance is all about the interface.

In Cooper's book, "The Inmates are Running the Asylum", he introduces the concept of the "interaction designer", a person who's job is to define the user interface and overall orchestration for a software project.  In software development circles, the "architect" is more like the lead programmer who has view of the entire system as a whole, including database structures, development frameworks, and the code base.  Cooper's interaction designer is more like an architect in the real world, and they are not so concerned about the mechanics of making it happen as they are with getting it absolutely right with the customer.  Programmers don't necessarily make good interaction designers, because of their intense focus on the "how" of creating software, they lose sight of the "what".  The interaction designer is not so concerned that the chosen framework doesn't directly support a certain feature, they are focused on the end product, and have the artistic license to devise the product so that it works intuitively and flawlessly for the end user.

In Platt's interview, he does not acknowledge many of his observations that have already been articulated at length by Cooper.  He does offer a few helpful tips for both end users and developers.  His tips for developers include:

1) Make sure the project includes a "virgin" - that is, someone who is knowledgeable in software development but comes to the table with a clean slate, unaware of the software currently under development.  This fresh perspective will cause new ideas to bloom outside of the box of groupthink that evolves within a development team.

2) Be willing to break with convention. Doing things the way they've always been done is no longer good enough.  If you can't find the right GUI widget from your class library, get a better class library or build it yourself.

3) Don't let edge cases complicate the mainstream.  Programmers tend to write software that is mathematically correct, when they should be focused on making the actions and decisions most often taken by the end-user the top priority. Platt says that it should be easy for the user to do things that are "good, smart and safe" and difficult for users to do things that are "bad, stupid and dangerous".  Platt also demonstrates that it's faster and easier to track a UPS delivery by searching the UPS tracking number in Google than it is in UPS' own web site.  In Cooper's "About Face", he makes the absurd analogy of putting the eject button right next to the radio switch in the pilot's cockpit.

4) Instrument the user experience.  Today's software allows you to be able to report on the user's experience, so you should monitor how the user uses the software and then use that information to improve the software for the user's actual needs instead of their stated needs.  If your software is a web application, the simplest thing you can do is monitor the web server logs, but you can even do more than that.

One of the problems I see with judging software usability is that it relies on subjective experience.  There is no one good metric to determine how much a piece of software sucks or doesn't suck.  Aesthetic appeal is the first thing that hits the eye, but the lasting impression is the ease of interaction.  For instance, my T-Mobile service lets me pay my bill over the phone, and I don't even have to push a single button; its voice commands are intuitive, succinct and accurate, and I have even paid the bill while driving.

I'll have more to say about these issues on future blogs.

Last Friday, I got a chance to meet with the new customer.  I got up early, dusted off my hard hat and put on my best business casual clothes and took a 4-hour road trip to "Dinoland" (Texas' Permian Basin, land of dinosaur fossils and black gold).  We had our kickoff meeting at the local BBQ place, and then drove over to the gas processing plant, where I was able to sink my nicest shoes into 2 inches of mud.  I should have worn jeans and work boots like I had originally planned.  Anyway, we had a rather long engineering planning meeting.  At the meeting, the engineers drew all sorts of cryptic diagrams and spoke a language with which I am unfamiliar, though it did use English to connect the jargon.  And I thought to myself, so this is how we programmers must look like to outsiders - cryptic indecipherable language, UML and ORD diagrams that have the appearance of being "right", and inside jokes that they don't get but seem to crack everyone else up.  It was educational to be on the "outside" for a change.

The tool that no .NET programmer should suffer without has gotten better!  New features are enumerated here.

NAnt is great.  I have a script that compiles all my source files, builds the standalone installation packages, fires off Virtual PC for testing, and it can even FTP the installation package to my client's site.  It seems like there's nothing NAnt can't do.  But I needed to peel off the version number from the executable, and I couldn't find any easy way to do this in NAnt.  Fortunately, I discovered that it's really not that hard to create a custom task in NAnt in C#, then use it from the target.

Basically, you have to create a <script> block (as shown below) and create a class that inherits from Task.  The TaskName attribute is what you will use in the main NAnt script to call it.  Any property with a TaskAttribute attribute can be used as an XML attribute in the script, and you can set Required=true to require it.  The Task object has the Project property, which can be used to access anything at all to do with the build process.  I use it here to set a NAnt property and log to StdOut.  NAnt will compile the C# code on the fly (if you did everything correctly) and voila, anything you can think of is now possible.

Here is a sample from the build script I created. 

<project name="SomeNAntProject" default="DefaultTarget">

    <property name="version.number" value="0.0.0.0" />

    <target name="DefaultTarget">
        <getversioninfo filename="C:\Folder\MyExecutable.exe" />
        <echo message="${version.number}" />
    </target>

    <script language="C#">
        <code>
        <![CDATA[
        [TaskName("getversioninfo")]
        public class GetVersionInfoTask: Task {
            
        private string _fileName;

           [TaskAttribute("filename", Required=true)]
            public string FileName {
                get {return _fileName; }
                set {_fileName = value; }
            }
            
            protected override void ExecuteTask() {
            
                System.Diagnostics.FileVersionInfo fvi = 
                    System.Diagnostics.FileVersionInfo.GetVersionInfo(_fileName);
                    
                Project.Properties["version.number"] = fvi.ProductVersion;
                
                Project.Log(Level.Info, 
                    "   [script] Got version info for " + _fileName);
            }
        }
        ]]>
        </code>
    </script>
    
</project>

My absolute favorite feature in MSIE 7.0 is the custom search providers.  If you can search via the URL's query string, you can use it.  One of my most used custom searches is BugMeNot, the compulsory registration bypass web site.  It's easy to set up BugMeNot so you can access it as easily as Google:

Go to the search dropdown and select "Find More Providers"

On the web page that comes up, type http://www.bugmenot.com/view/TEST as the URL and BugMeNot for the name.

Click install.  Now you can search BugMeNot straight from IE7.

I found an interesting alternative data access layer (DAL) to NHibernate called SubSonic.  It was formerly called "ActionPack", but I guess there must be enough confusion with Microsoft's "Action Pack" that they changed the name.  SubSonic is open-source (MPL)

SubSonic has been hailed as "Ruby on Rails" for .NET developers.  There is a screencast of it here, prepare to be amazed.  It works kind of like this:

1) Add a reference to the SubSonic DLL
2) Modify your web.config with connection string info and SubSonic build provider info
3) Build the project
4) ...
Copyright (C) Sidney Harris.

5) Amazing!  Domain objects and data access layer have been automagically created!

I foresee this as a great way to quickly get a data-driven site up with a minimum of development effort.  Realizing this is not the end-all and be-all of data access, you are still free to use "traditional" ADO.NET techniques.  A recent benefit is that CodePlex has come out with an XmlProvider, so that if you would rather persist to XML instead of a database, it can handle that data access layer, too.  This thing keeps getting better and better.

I just recently started using Microsoft SQL Server Management Studio Express (SSMSE).  This free tool is intended for SQL Server 2005 databases, but it works just fine with MSDE and SQL 2000 databases, as far as I can tell.  SSMSE obsoletes the tools I have been using up to this point, namely Enterprise Manager and Query Analyzer.

Enterprise Manager is slow to open on most systems I've used.  I have a server instance with 20 databases and I can take a nap before the thing finishes opening up.  SSMSE uses demand loading of the tree nodes, so performance is a lot better.

The whole integration between Enterprise Manager and Query Analyzer has a more natural feel.  Nice job, MS!

OK, I'm definitely not the first person in the world to figure this out, but it bit me in the behind last week.  So I figured I would share this bit of wisdom with you so it doesn't happen to you.

Most of the data that's stored in this MSDE (SQL Server) database lives in an "engineering variable data" table.  In this parallel universe, the term "variable" doesn't mean quite what we programmers think it should mean, so I'll call it a "datum"  A "datum" can have a value, a specification (that determines how it is used in a calculation), upper and lower limits, residuals, bound type, and other properties of interest to engineers.

I experienced a dreadful performance problem in the production system.  Out of necessity, the program would sometimes have to delete 10,000 records from the VariableData table and insert a new copy of those records.  In the development system with a much smaller database, this worked as intended.  However, when we started getting upwards of a million records in the table, this data replacement step would bring the program to its knees for 30-120 seconds.  Sometimes the queries would time out, gobble up RAM like candy on Halloween, or both.  I tried various hacks and workarounds to improve performance, including using a bulk insert, but to no avail.

We finally took a look at the indexes:

IX_SolutionCaseID: key to an integer in SolutionCase table
IX_Specification: key to a varchar(7), whose value can be one of 7 possibilities
IX_Guid: unique key for a uniqueidentifier
IX_VariableID: key to an integer in Variable table
IX_VariableVariableID: a redundant key

I realized that the way I get data out of the database is by SolutionCaseID and VariableID, which uniquely identifies a datum.  So the queries I was currently running were using one of the existing indexes and then doing a table scan to find the record I actually wanted. Ick.  So I added a new two-field key:

IX_SolutionCaseIDVariableID: added key for both SolutionCaseID and VariableID

Adding this index to a populated table only took a few seconds and it cut the wait time in half!

Next I started hacking away indexes that I never use.

IX_VariableVariableID: deleted - this had no noticeable effect on performance.
IX_Specification : deleted - this improved performance.  This is analogous to have an index on sex "M" or "F", which is always a bad idea.
IX_Guid: deleted - I never look up data by Guid, this is for UI magic only.  This had a huge benefit on performance

So the conclusions can be:

• Avoid indexes on GUID columns, unless they are primary keys and that's how you look things up
• If you look up on two fields a lot, make sure you have an index that contains both fields
• Don't index on columns that have lots of repetitive data.  You might actually win with a table scan on this one!

And if you didn't know this already, you can always force an index to be used in a select query (use at your own discretion):

SELECT Field1, Field2, Field3 FROM TableName WITH (INDEX = MyIndexName) WHERE Field4 = 42

And now my client is much happier, and there was much rejoicing!

Last week, I came across XML Marker, a free XML editor that is great for simple tasks.  It doesn't have the full range of abilities of a commercial product like Altova XMLSpy or StylusStudio.  I have been using the evaluation versions of these commercial products to do simple XML editing, but I can't justify coughing up $500 for a simple XML Editor.  XML Marker bridges the gap: it doesn't suck like Microsoft's XML Notepad but it give you multiple views of your XML document like the big boys.  I edited the config file for this web site, and it works great.  It doesn't have schema validation built-in, and that would be a nice addition.

Recently, I bought my wife and myself his-and-hers Windows Mobile 5.0 gadget phones.  Now I'll admit freely that I'm not a huge cell phone user, and I've always considered maintaining a cell phone as a necessary evil in the consulting world.  Since I have been experimenting with the .NET Compact Framework SDK, I figured this could have potential in software development projects I'm working on.

What is truly amazing is the amount of power that can be packed into one of these little devices.  This device, the T-Mobile MDA (link) has built-in WiFi, bluetooth and infrared beaming, it synchs with Outlook, et al, using ActiveSync, it comes with a mini MSIE (approximately IE 5.5), Pocket Word and Excel, a PDF viewer, some games, and other stuff.  There is a plethora of free and commercial software for the phone, and I downloaded a Command Prompt program and a Registry Editor.  Using Visual Studio 2005, you can easily develop and deploy your own .NET software to this device, including SQL CE 5.0, which comes with a cute little Query Analyzer for ad hoc querying and debugging.  Equipping this device with a 1 GB MiniSD memory card gives you ample room for about 100 MP3s that you can play with the mini version of Media Player.

Oh, and did I mention that you can make and receive phone calls with this device?

It is amazing that this thing runs on about 500 MHz processor and delivers this kind of punch.  When I think back to my first PC, that I bought 13 years ago, this little gadget blows it away.

I love almost everything about this device except for one thing: most web sites look downright awful on the web browser.  Nearly all web sites are designed for 1024x768 screen size, with the assumption that the user will be using it at a desktop or laptop.  Most web sites use a boatload of graphics, with the assumption that bandwidth is not a problem.  When my device is connected via WiFi, this really isn't a problem, but more often I connect to the Internet using GPRS, which is much slower a typical broadband connection and more latent than even a dial-up modem connection.

As a developer, I see the conundrum. Today, most people do not use mobile devices to browse the Internet.  But I bought this device for $249 last year, and I am sure that it will be half that price by the end of the year.  Why would I want to develop a web site for such a niche market?  Until Firefox started gaining popularity, I only designed for MSIE since that was biggest target.  But thinking forward, the mobile web market is poised to boom in the next two years.

So what choices do we have as developers?  We could work by redesigning one site at a time to comply with the new miniature standards.  This is a large development effort and will cost businesses a lot of money to implement.  Do I think this will actually happen?  Only the largest companies or the ones most directly involved in web businesses will do it.  For instance, GoDaddy has a mobile web interface that I used to register a domain using my gadget phone.  But most vendors are not doing this yet.

So what about medium to small-business web sites?  I think there needs to be a handly plug-in the can convert a web site to mobile formatting when a mobile web browser makes a request.  My idea is that I could create an HTML proxy that makes the actual HTTP request from the web server, but then reformats the HTML and imagery that is sent back as a response.  Current mobile web browsers make the full request and drop formatting on the client.  An HTML proxy that can be installed on the web site would provide an alternative where the crash diet occurs on the server, resulting in faster loading web pages on the mobile web browser.

My thought is that there would need to be certain flavors that are available immediately.  Most dynamic pages are written using PHP, ASP and ASP.NET these days.  These would be the immediate target systems.  I have yet to flesh out the details of this server HTML proxy, but I'm going to be investigating this this year.  If you are interested in enabling people to use the Internet more effectively please e-mail me.  I think this is a pretty good idea, but I'm willing to listen to better ones!