Failures in tooling

I said my little piece about tool development, and how coders shouldn’t assume they are good at it. Of course, every coder thinks they are the exception because the tools we write always work fine for us and we shrug off the little details we know they have because we know about those and you’d have to be an idiot to let that be a problem.

shameful.jpg Personal case in point: The Deployment Manager. Even as I write this, I’m not ashamed enough of this piece of crap. This was the tool that Doc had to use for managing the “rare spawn” deployments. It was something I cobbled together and never revisited. Most coders would probably be as comfortable as I was that it did the job sufficiently.

But it was littered with shortcomings and completely ignored the practicalities of how it was used.

Each of those little yellow ‘?’s is a button that must be clicked if you want to change an entry. It brings up a separate window which then lists all the facilities this deployment could be deployed to. Issues #1 and #2.

As a means to letting you configure rare spawns for an upcoming intermission or campaign, the tool lets you pick any town and any facility. It does try to impose some limits on which towns are listed to those which have an air/sea/land facility depending on the weapon type of the deployment. But it doesn’t enforce this restriction on facilities. Issue #3.

Once you have changed one or more fields on a row, the “Update” button on that row becomes clickable. Clicking the button saves the row and reloads the page. Issue #3 and #4.

When you’ve saved your change, and the page refreshed, each row has a status field. This is actually a field the game uses to tell that you have changed something and it needs to reload the information and modify the deployment. So after a change it says “Pending”. When, and if, the game finally loads it, it would read “OK” if you refreshed the page. If it failed, it would read “FAILED” if you loaded the page. If the server wasn’t running, or if it rejected the line outright, it would remain “Pending”.

Not only is it left to the user to understand that this page is not an accurate reflection of the server state but a sort of combined “best guess” and “would like to have”, they’re also expected to understand the asynchronous nature of the tool. Something lots of coders have trouble with. And no way to distinguish between a) a running server refusing to apply a change to a deployment in-play, b) a running server refusing to load a deployment, c) a non-running server. Issues #2 and #3 again.

Lastly the page design was very cobbled together using basic RXML to generate the context; a few extra hours scripting would have merged the complex queries into a sensible structure. As a result, the facilities would often fail to have their CP listed (see Issue #2) and the page was generally horribly slow, Issue #2 and #5.

Issue #1

The user is told, and the tooltip reinforces, that clicking the button will “bring up a window”. But if the window is already open, the user may not see it. And since it is a child window of the browser, it might not show on the task bar. It also had no easily recognizable title so if the user is working and has lots of windows open, they might go right past it.

Issue #2

Practicality.

Rare deployments were typically to rare locations – like naval, rather than riverine, harbors; paratroop airfields, etc. We often make the point to players that “CP is not equal to city” citing examples like “Brussels CP”, which includes Wizernes airfield and Wizernes armybase.

nomercy.jpgWith over 700 CPs in the game, it has to be noted that these locations may be rare but they are not few. With over 50 such instances, you can be hard pressed to remember where most of them are.

The window that comes up does nothing to make it easier to find “Glisy Airfield” for instance, you just have to know that its in Amiens, or else go look it up for yourself.

Here we have the cardinal sin of all coder art tools. User input that requires calculation or reference-lookup – in this case, you often have to use another tool to find the facility you want and then come back. A minor inconvenience, but we’re already only on Issue #2 and those minors are starting to add up.

Issue #3

Special rules or “the user’s clue is not yours to borrow”.

How do you remove a deployment? Simple. Withdraw it. Type the letter ‘w’ into number of units box. Heck, if you mouse over it, it will even tell you that…

assuming you know to mouse over the units box.

I can’t tell you how many times we ran into issues on a map reset because deployments were misconfigured. Typically paratroopers were deployed to an airfield or bombers were deployed to an armybase: cases that the server would reject and shutdown in a bad way that often resulted in no error message thanks to a quirk in our logging system.

My user is not clueless, my user is simply unaware that the tool will accept unacceptable configurations on the presumption of intent. The user’s clues are fully occupied with other aspects of this task, and like myself the user is making assumptions about the co-operative participation in the task of the tool I have provided him with.

What results is a case of two drivers, both looking at the side mirrors, and thus both unaware of the deer in the road infront of them.

Issue #4

Knowing that you could only save one line at a time, it never occurred to me that someone would make multiple changes at a time (Issue #3). But an important derivative of this fact is that users believe what it says on the box. They also don’t speak coder, they speak convention.

Nothing about my page says that you can only save one line at a time, least of all the tool itself. If you change one row, the tool doesn’t even remotely flinch, flash, bleat or beep when you try to change another. A user is going to anticipate that each update button thus somehow magically saves the row.

And so they would update entire pages and then click “Save”, wait 5-15 seconds while the page reloaded and then be presented with one of their changes.

Issue #5

When the page finally reloaded, because the actual execution of the data occurred in the game server, on its own schedule, what you then saw was speculative. If you told the French paratroopers to deploy to a German latrine, what you would see was “Pending”.

While writing the page, I thought it would be perfectly acceptable to enter a line at a time and then review the results when you were done. Because when I tested it, I input 4 deployments. The live game had 8 deployments per country, and if you put in bad data, the game would load it while you were fiddling with other rows. Not a good state to be in.

In building this page I achieved the ability to change the properties of some game data.

What I failed to achieve was a tool that let you manage the rare spawn deployments in a live game environment.

Aside from the clunkiness of the interface, the dependency on the user looking up data to feed into the tool, the lack of validation to allow special functionality rather than special casing it visibly, the slowness of the page in practical operation… The tool overlooked a fundamental aspect of what the user was doing: spatial organization.

When you looked at my screenshots, at what point did you think ahh, this lets someone place pieces on a map? I’ll bet it wasn’t your first thought.

The CPs are listed alphabetically. The fallback is just a field with no indication of distance or relationship. There are no flags to indicate who owns a facility or what type it is. Just text.

Issue #6

Yes, I only mentioned 5. But all of them relate to this issue.

Users generally have no measure by which to distinguish a bug from bad behavior. As a result, every time Doc had to call on me to resolve issues with this page, he left the interaction with the impression I had ironed out some issues and I went away with the impression he had learned better the parameters of this utility.

And so each next time we ran into very similar issues. In the end, Doc did become quite adroit at chewing on the glass-riddled granola bar I’d given him, but it did nothing to ease his fear of coders’ tools.

Mea culpa

To be fair to myself, the rare deployment stuff was equally just something I cobbled up to solve a completely unrelated problem that turned into a feature and grew beyond my expectations.

The page was more of an act of desperation – we needed some way to control the deployments, and when I added something in-game, it still proved awkward because the user then had to track, by hand, which deployments had been updated and which needed updating.

So I put together the page, and I left it open to abuse so that we could solve multiple problems with it. The trouble is that I then handed it over to the producers.

You can bet that if I had been the user, much of this stuff would have gotten sorted out. But see again Issue #6. Doc/Gophur/Hatch/Corn/etc can’t have been expected to know that the tool would configure the game so as to be unable to start up. Why would someone write a tool to do that? That’s just stupid.

Meanwhile I was assuming that if the tool was a problem, someone would task me with actually writing something – otherwise they’d bear with it while we got rid of the issue by implementing TOEs.

Production, meanwhile, assumed the tool was just buggy and that each time we went thru it I would be ironing out more of the kinks. And production were smart enough to know that as non-coders they can’t distinguish symptom from cause in an application and would give me the benefit of the doubt that it was a different problem this time.

Thankfully, we no-longer use this system at all. TOEs finally and completely rid us of the need for it, and the pages we have for managing TOEs are infinitely superior.

My one lingering frustration is that they still lack the visual component of a map. That’s not something I’m a good enough web-programmer to do with web tools, and both my Java and C# are rusty enough that I’d really need to be tasked and set aside some time to set up an off-line tool for managing them.

I hope at some point I might persuade some WWIIOL Web Geeks to enhance their map-tools to let people set up “scenarios” complete with brigade deployments etc, and export the resulting setup in a standard format to us. I’ll probably draw up a DTD for that just to see if anyone decides to take a shot at it.

I’m not going to try and close with any clever anecdotes on how you should build tools, I thought it only fair to put one of my worst mistakes in recent times up there as a punching bag, but also as a fully-laden punching bag of what-not-to-do examples.

But I will say this:

A good tool is supposed to come between a user and a resource of some variable level of complexity. The primary expectation of a user is that a tool is an expression of the rules. If your ATM failed to mention that the $40 you asked for came with a $40 overdrawn fee, would you really appreciate not being inconvenienced by it mentioning your $0 balance?

7 Comments

Lots of good stuff in there, Oli, thanks for posting this. I know I’ve built my own share of “expert-friendly” systems over the years, never anticipating how far they’d spread. The thought processes can be evilly familiar…

“Why should I bother validating this field? I’m always going to be the one using this tool, I would never put anything other than X or Y in here…”

Six months later, I’ve forgotten that I took that shortcut on cobbling something together, and the other folks on my team could use this to help… so why not share it with them?

*cue ominous horns*

Luckily I’m in the position where I only tend to write read-only tools; we’re not allowed to alter our trouble ticketing database externally, just query it. That means my glitch didn’t wipe out a huge swath of data or plug insane numbers in to SAP… this time.

On another note, would you think about submitting that to Worse Than Failure (the renamed Daily WTF)? I think they might get a kick out of it.

The problem is magnified the better a coder you are. What with the expectations.

I, as the user, trust you not to be fucking up this simple shit (vulgarity for effect). So…as you said…I give you the benefit of the doubt. It has to be something that bastard *the code* is doing to screw you up, and probably something one of *those other bastard coders*…less competent than my wonderful, pure, favorite pet coder…did.

So I (as the user who also tasks you with stuff like, making tools) start thinking: “Man, what a snaketrap clusterfuck pile of snags and pits and minefields that whole section of the code is. Obviousy. I mean, even wonderful pure favorite pet coder can’t tame it. It’s screwing him up on something as simple as this, and I’m watching him kick much much much more complex task’s asses and hand them to me on a platter. Best stay away from tasking him with tools there. It’ll eat up gobs and gobs of time I can’t afford.”

Again, I’m speaking above as that user. Not me :)

I came to appreciate all of this a long time ago and correct for it as I went and dealt with the coders. Honestly…in the made-me-a-better-manager-of-coders sort of way….though when I talk about it, you read “idiot savants” as I poke irony in my comments and assume I must have been a right bastard to work for.

See…I worked with a fucking genius wonderful pure favorite pet coder named Dale “Hitech” Addink, once upon a time. And as smart as that bastard was…he had the same annoying issues biting his butt. I learned what to expect.

Problem later on was, again I keep coming back to this…not being able to hire CRS a good solid programming lead to know what was a clusterfuck section of code, and what was a bonehead dumbass simple set of oversight on the part of an otherwise wonderful, pure, favorite pet coder.

Because…hey…there *are* clusterfuck sections of code in this project, no two ways about it. And I can’t look at any section of code and tell you what may or may not be wrong with it…or even if I’m looking at the right code :P

…@/

Kfs..you should think about writing a book on tools. These have been your most interesting posts by far.

My wife has been patiently waiting in bed for me… but still, I look forward to your third entry on the trilogy of tools.

Something I have done a lot of. I’m sure that if were a rat this would be my area of responsibility.

Other than the occasional chimp or bird, humans are nature’s toolbuilders.

I like the question mark button. Nice touch. Would have been better tho if it had a caption of “Eh?”.

And I suppose “idiot savant” is a notch up from “all coders suffer from some degree of autism”. That one brought to us by the wonderful community of Second Life (an amateur game development world, for those who don’t know).

I’ve known a few autistic folks, and not a few with Asperger’s or something approaching it.

Coder’s (as, ya know, the stereotype which has bits of truth innit) certainly share some of the traits…when they’re not one or t’other.

…@/

Leave a Reply

Name and email address are required. Your email address will not be published.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

You may use these HTML tags and attributes:

<a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <pre> <q cite=""> <s> <strike> <strong> 

%d bloggers like this: