chalakanth

User experience in a back-office application

I can’t ever seem to explain what ‘user experience‘ is, leave alone make the case that it is important.  I say ‘user experience‘ and people hear ‘user interface‘.

User experience in an in-house, back-office application

Let’s try this.

Imagine I am speaking to a bean counter type middle manager in some sclerotic insurance company. I say this to him.

You make sure that the user knows the business.

I will build a system that a user can run the business with.

The user will not require training to use the system.

The system is not the business.

The system is a transparent tool that anyone with business knowledge will simply ‘get’.

The system will ensure that the user executes the business correctly, and efficiently.

New Enrollment into a Medicare Prescription Drug Plan

Consider new enrollment into a Medicare Part D plan (Prescription Drug Plan).  The insurance processor must know the business, described below.

  • A person comes to us, asking to enroll in a prescription drug plan.
  • In response we send a query to CMS (Centers for Medicare and Medicaid Services) to confirm that the applicant is eligible for the benefits she is asking for.  In industry jargon, this is the BEQ (Batch Eligibility Query).
  • If CMS finds that the applicant is eligible, we complete the application, and formally notify CMS that the person has enrolled into a Medicare Prescription Drug Plan.   This happens through the MARx Batch Input Transaction File, which we send to CMS every day.
  • Then we wait for CMS to tell us that it has accepted this person’s enrollment.  This is through the Transaction Reply Report (aka TRR), which CMS sends us daily.
  • Now, the member is completely enrolled.

Say we build a software system that allows the processor to execute the above workflow.   In addition, say we promise that the processor can simply be put in front of the system, and she will be able figure out how to perform the business without any further training from us.

More than ‘user interface’

If we can build such a system, we will have mastered user experience.

Think about what it takes to accomplish that.

Clearly we are talking about more than proficiency in HTML and Javascript.  At the very least, we the engineers, have to know the business, and we have to know the business users.

chalakanth

Process is very often, much ado about little

The first principle in the Agile Manifesto is “Individuals and Interactions, over processes and tools”. A recent development task illustrated that point starkly – skill trumps process.

Time’s a wastin ….

I was working through a weekend, exhaustively specifying the UI (mockups, WORD docs, UML State diagrams) for a recent user story, when it occurred to me that we were wasting our time.

Here is more or less the user story I was working on.

As a documentation processor, I must be able to review all of the documents in a mailbag. Upon review, I must be able to accept, or reject each of these documents.

In the time that it was taking to describe the UI to its last gory detail,  I could create a working prototype of the UI, which would be close to 80% of what the user needs.  The business folks could take the prototype for a spin, and give me feedback, which could quickly iron out the remaining 20% of the work.

See, we are building this for processors in an insurance company.   They are not looking for sophisticated, space age UI.  They just need something that is easy to understand, learn, and allows them to do the work efficiently.

Between the business analyst and me, we could have the UI squared away in a couple of weeks – a single sprint.  Add a tester if you must.

I will test every damn line of code I write anyway, before this leaves my hands.   The tester is only a second line of defense.   Even without that tester, I can get this very close to done.

So why am I specifying the UI when I can just build the thing myself?  I am having to do this because, collectively, the primary resources that are assigned to this task cannot or will not, produce an effective UI on their own.  That amazes me.  I am shaking my head as I write this.   If you have several years of experience with this problem domain, with these business analysts, with these end users, and with the same UI technology, you should be able to take the user story, fill in all the missing details, and end up at the UI.

Again, here is the user story, for reference.

As a documentation processor, I must be able to review all of the documents in a mailbag. Upon review, I must be able to accept, or reject each of these documents.

Let’s think through this.

Analysis

  • Users have to review all the documents in the mail bag.
    • This means you have to list the documents to the user
  • So how do you present a list of documents.
    • A table, where each column is some attribute of the document, and each row is a document.
  • How do you make it easy for a user to study a large list of data that is in a table?
    • Allow filtering and sorting by various attributes.
    • Break up the list into several pages, and allow the user to page
      • Or simply allow the user to scroll up and down the large list.
  • How do you help the user make changes to large amounts of data safely, and correctly?
    • You must ask the user to confirm a large change.  Give her a chance to back out.
    • You must provide the user a way to undo a change.  Mistakes will happen.
  • What attributes (columns) of the documents must be included in the list
    • Leave out the attributes of the documents that are implementation-centric, which users have no need to know.
    • Pick the attributes that are business information.  Make a guess on which of these the user might be interested in.   Changing this list after review should not be hard.

Can’t the developer do this?

Analysis

If you have been developing UI for any time at all, you should be able to do the analysis I just did, and arrive at a similar, if not the same place.

Business Info

You have been working in this domain for several years.  That means you must already have all the business information you need.

You must know the business-centric attributes that characterize the mailbag and documents.   These become the columns of your listing, and suggest what the user can sort and filter on.

Language and jargon that end-users will recognize must also be familiar to you by now.  This is where static labels, feedback messages and such will come from.

Implementation

You have been working with your UI framework for several years.  So you should know how to implement the various UI features mentioned above – a table of data, sorting, filtering, paging, scrolling.

But none of this happened

But none of this happened, and a UI designer had to be brought in to spell everything out.   Why?

The Developer

Though they were well-versed with the UI framework, the developers have little knowledge of, or interest in, what makes an effective UI.

They are happy to be told what to build.  They want to be told what to build.  When left to their own initiative earlier, they produced a UI that so defied common sense and business reality, that the end-users’ job turned into a labor-intensive, error-prone, hell.

The Tester

If the testers know the lingua franca of the business (English, in this case), if the testers know the business, and if they know the end-users, the testers can judge if the UI satisfies the business need, simply by relying on the 2 sentence user story.

Unfortunately, testers’ familiarity with the business is uneven.  They are simply kids that compare a spec to delivered software.  If the spec is incomplete, or incorrect, these testers are thrown off course.

Further, more often than not, the testing leadership subscribes to the view that testers must only work off an exhaustively detailed spec.  Applying their own knowledge of the business, their own analytical and communication skills, is considered risky. The focus on learning the business themselves is weak.

Manager

Well, managers have all the power, which makes them responsible for everything, doesn’t it?

Who are they hiring?

The UI developer does not exhibit knowledge of effective UI design.    So how did we end up hiring him?

Say the manager that is evaluating the developer does not know anything about UI design himself.   So the manager cannot determine the developer’s competence in the matter.  That is reasonable and common.   A manager is constantly confronted with problems in which he does not have direct expertise.   Yet, she must hire people that will be responsible for solving that problem.    In that situation, a manager has to be able to recognize competence (and conversely, incompetence) as a characteristic separate from concretely observable skill.   Competence has an aura, a body language, a speech pattern, a look in the eye, as does incompetence. You have to be able to smell the bullshit before you see it.

When managers neither have concrete expertise, nor do they have a nose for fakery, we end up with these half-baked developers that require to be spoon fed.

Bad drivers

Many resources are able to learn and do more if we create the right environment.  This is usually some combination of large grained carrot and stick, coupled with the pushing of buttons that we can only learn by getting to know the resource personally.

Effective UI design for the corporate world, is a skill that can be learned by just about anyone in a few weeks, or over the course of a few UI development tasks.  If a developer has worked on UI development for going on three years, and still does not have a coherent, useful view of the matter, the leadership has been asleep at the wheel.

The same argument applies to the testers.   The management enabled them to be unthinking, ‘garbage in garbage out’ type grunts.

Skill trumps process

What a business analyst and a skilled engineer can take care, is now distributed over a whole platoon – The business analyst, the UI designer, the developer, the tester, tech and test leads to oversee the grunts, managers to oversee all of this.

Where the work could simply be a conversation between two people, now is a multi-step workflow involving many more people than that.  Two people getting a task done is just work.    Eight people, each with their own inputs, and deliverables, working on narrow parts of the whole, and that with their vision constrained by blinders, is a process.

What does the process buy us?   Nothing that I can see.  I see overhead. An 8 to 10 person team requires more management and book-keeping than a 2 person team. I see a machine with more moving parts, which exponentially increases the complexity, the amount of co-ordination required, and the number of ways the process can fail.

One or two committed, skilled engineers can replace the whole process. Why not learn to identify, hire and hold on to such resources? Why the fixation on process? Do you want get work done?  Or do you want to run a process?

chalakanth

Story points of a user story that spans sprints

We start a story in sprint 7, but we finish the story in sprint 8. Should we count those story points against 7, or against 8?

Barking up the wrong tree

We are asking the wrong question.

It does not matter whether we add those story points to sprint 7, or 8. That line of thought tells us nothing useful about how we are performing the primary task at hand – deliver correct, well built software in a timely manner.

Make no mistake, when a user story slips into a second sprint, you have a problem with how the work is going. That problem, whatever it is, demands examination and remedy. However, deciding which sprint to stick the story points in neither helps the examination of what went wrong, nor does it contribute to correcting the failure.

Big story

The user story may have been too big. Perhaps we bit off more than we could chew. We should have broken the user story into smaller ones. Watch out for that next time.

How do you split a story into smaller ones? That is a separate question. There are a lot of references on that, which I’ll leave you to research.

We are not efficient

Say we decide that we cannot break the story into smaller pieces. Perhaps something is not quite right with our productivity.

In my current place of work, we get issues that can be finished (analyzed, fixed, tested) in a couple of days by a developer that knows the domain, and the system. However we also have folks that take a couple of weeks to finish the same story. Their work often span sprints. These folks have to come up to speed. There is your problem. Address it.

Perhaps the developer finishes his part, but the issue languishes with the testers because they are swamped with other things. This exposes problems with how we have organized our teams. Protect team members from being dragged into work that is extraneous to the current sprint.

We should have known early

We should have known that a story would not finish, well before the end of the sprint, and we should have taken remedial action right then.

If we know our team, and our engineering environment, we should have been aware even before we started, that we would not finish a story. If we cannot make such judgements even after a couple of sprints, the sprints are not teaching us anything. If we are not gaining a more accurate estimate of our team’s capability, sprint by sprint, we are not being Agile. What obstacles keep us from understanding our team’s capacity to deliver? Investigate that.

Sometimes we misread a user story. We clearly see the story only after working on it for a couple of days. At this point it should become clear whether we will finish the story or not. This kind of early feedback is the cornerstone of the Agile way. If you become aware of an out of control story only at the end of a sprint, you are not being Agile. Why is this feedback not happening? Or is it happening, but the information is worthless? Perhaps your team is dissembling, and the scrum master cannot tell fact from fiction. Look into this.

Velocity – Weather or Climate?

It is not hard to see why we fixate on which sprint earns a partially completed story’s points. We are trying to figure out how much our team can deliver in a sprint; the so called Velocity.

Do not get too hung up on Velocity. As Mike Cohn of Mountain Goat Software points out, Velocity is characterized by a dichotomy that is very much like the one between weather and climate. On any given day, weather can be good or bad. You can make valid generalizations about the weather of a particular location, only after observing it over a large span of time. We have a word for that. Climate. Think of Velocity as climate, not weather.

The number and size of user stories that you start and finish within a single sprint, will vary from one sprint to another. Don’t worry about it too much. Over several sprints, you will get a good measure of how much work you can do. Over several sprints, Velocity will reveal itself.

Here are three slides from Mike Cohn’s Scrum Master training, which also make the above point.

Velocity Fluctuates from Sprint to Sprint

Velocity Fluctuates from Sprint to Sprint

Velocity - Some are Outliers, Some are Typical

Velocity – Some are Outliers, Some are Typical

Infer Reliable Velocity From Sample

Infer Reliable Velocity From Sample

Focus on learning from the trials and tribulations of each sprint.  Address these issues as you become aware of them.  Velocity will take care of itself.

chalakanth

User Stories and Building Blocks – Another Difference

An earlier post described how the Agile philosophy implicitly divides software development into two types of work – user stories and building blocks. In addition, the post delved into how to distinguish between the two types of work. Here, I point out another seminal difference between user stories and building blocks.

How much is done?

During the course of a project, a critical question comes up again and again. Business stakeholders, and managers, keep asking, how much is done? What exactly is completed? What remains?

What answer do you think is most useful for them? Investigating that question reveals a defining difference between user stories and building blocks.

Status by User Story

This is a list of features that we must build. Clearly these are user stories.

As a customer service rep, I want to be able to generate an auto insurance quote.

As a customer service rep, I want to be able to email, fax, and mail a paper copy, to the customer.

As a customer service rep, I want to order a customer’s credit report.

As a customer service manager, I want to see all enrollments that are in process at this moment.

When a business stakeholder wants to know what is completed and what remains, we can simply point her to this list of business features. Is an item production ready? Is there a metaphorical green check mark against the item?

This list of work represents the value that business users are looking for, in the system we are building. Hence, the list of user stories communicates status that is immediately relevant to business, in an unambiguous, even visceral manner.

Status by Building Block

Now consider this list of work.

Build the data acces layer of the enrollment sub-system.

Implement the business logic of enrollment as production rules.

Build a REST service wrapper around the Enrollment API.

Build a web based UI for managing enrollments. This is meant for internal processors and customer service reps. This UI will sit directly on top of the business layer.

Build a mobile app for use by customers to manage their enrollments. This will talk to the REST web service.

If you tell business users, you have completed the data access layer, and business layer, does that tell them anything about what they really care about, the business features that we are building? It does not. Business stakeholders really have to wait till all of the above tasks are done before they can begin to evaluate what shape, and how much of the business features, and in what shape they have been delivered. Until then all is uncertain.

Out of this uncertainty is born magic status measurements like 35% done, 60% done, etc. These numbers convey little information that is useful for business stakeholders. Confidence in such status’ are low and always feel pathetically forced.

Thus

You can differentiate between a user story and a building block by what information they convey when we complete them. Completion of a user story unambiguously tells business what features are done, and what remain. Completing a building block leaves those questions unanswered.

chalakanth

User Stories and Building Blocks

When I look at software development from the Agile point of view, I perceive two types of work. One is what Agile calls a user story, and the other, for lack of a better term, let me call, building block.

All of us who have had any truck with Agile, have run up against these two types of work.

Further, many of us are stumped by these questions below.

How to distinguish between the two types of work?  “Is this a user story, could that be a user story, shouldn’t this be a user story”, is a familiar song.

If a task is not a user story, and yet it is a significant effort, how do we fit it into a plan of action whose purpose is to only implement user stories?   In other words, how do I fit the second type of work, the ‘building block’, into an iteration, sprint, time box, or whatever your particular flavor of Agile calls it.

Should this be a user story?

One of the first principles of the Agile philosophy provides the answer.

If you have not delivered value to the end user, you have not done the job.  
The driving focus of the Agile way is delivery to the end user, of a feature 
that she requires, asked for, and can recognize as such.

To illustrate, let us consider two different kinds of projects.

Auto Insurance Policy Administration System

You are building a policy administration system that can manage Auto Insurance policies.

Consider these tasks.

User Story

As a customer service rep, I want to query the DMV for a driver’s motor vehicle report.

This is a business function, expressed strictly in terminology that comes from the business user’s world. When you build this completely, from UI to backend, you will have a full vertical slice of business functionality, which you can set aside. Now, you can change focus to another vertical slice of business functionality, perhaps, querying a vehicle owner’s credit score.

Business folks know the auto insurance business. They do not know software engineering. Business folks can only evaluate the value of the system you are building, in terms of the business functions it helps them perform.

Can I query for the Motor Vehicle Report? Check.

Can I query for the Credit Report? Check.

We can write the sweetest code imaginable, but unless we place these business features in the business users’ hands, our job is incomplete. Such features are user stories.

Building Block – One

Build a library of UI widgets, backed by Twitter Bootstrap, which allows entry of entities in our domain model, like postal addresses, a vehicle description, an accident report, and so on. You should just be able to refer to these widgets, as necessary, in various screens of the UI

Creating such a widget set requires considerable skill, and time.  However what can a policy admin representative do with a UI widget kit, however sophisticated it is? Nothing. The policy rep wants to take a driver’s information over the phone, and prepare an insurance quote for the driver. That is a user story, which will be implemented using, among other things, the above UI widget set. Creating the widget set itself, is not a user story. The widget set is a ‘building block’.

Building Block – More

Build a fault tolerant API that can send any query to DMV.  The API must tolerate connection failures up to a certain number of times, which must be configurable.

When we have built said API, does the business user have a feature that they can do business with? No. However, you can build the ability to request a Motor Vehicle Report, on top of this API.  Only then do you have a tool that a business user can clearly identify as an essential cog of their business.

Here are some more tasks that are merely building blocks in the Policy Adminstration project.

You have to build a data access API that hides the details of your application’s datastore, MongoDB, from the rest of the application. In other words, this is one of the layers in your application.

Your application has to schedule various activities. So you create a simplified facade around some proprietary, and complex scheduler that your company just bought.

After several sprints of work, you decide that a critical API method (EnrollmentService.enroll, for instance), has become too unwieldy and must be refactored, as defined by Martin Fowler et al.   However this is a high risk task, which will require significant skill, effort, and time.

To summarize, a project often has tasks which you require to reliably implement user stories. Completing such a building block puts you on the road to producing a business feature that users recognize, but it does not get you all the way there.

Re-engineer the enrollment, and billing sub-systems

Now, let us consider a different kind of project.

Your policy administration system was delivered and has been in use for 
over a year. Periodically, problems have arisen, but you have kept the system 
going through bug fixes, work arounds, cleaning up data directly, and so on. 
After a year or so, the code has become so baroque, that even bug fixes are 
becoming difficult to do.

So you decide that two critical sub-systems, enrollment, and billing must 
be re-engineered. The UI cannot be touched. All features that the user has 
been using for a year now must stay intact. Any changes you make must be 
under the hood.

Can this project be planned, and executed in an Agile manner? How can we break up the work into user stories, when we are not adding any business features at all?

Agile is not so much about any one kind of project, as it is about how to drive a project to completion.

  • Figure out the primary stakeholder that asked for the project
  • Figure out how that stakeholder views the deliverables of the project.
  • Slice up those deliverables into smaller goals (sometimes called user stories), such that the stakeholder is always able to unambiguously see, in terms that matter to him, what has been completed and what remains to be done.
  • Pick a fixed interval (aka sprint) of no more than 2 or 3 weeks, within which you can start and completely finish one or more of the above user stories
  • Prioritize, and schedule the user stories across the sprints
  • Execute a sprint, examine how you did at the end of the sprint, adjust, and execute next sprint

I can see no reason why we cannot apply the above method to a purely technical project, like the one under consideration. Some technical lead will be responsible for defining what the problem is, and what success looks like. This technical lead should be able to break up the larger goal into smaller ones which are easily measurable.

Perhaps, the enrollment and billing subsystems consist of 3 interfaces, and 3 implementation classes. The interfaces must stay untouched. The 3 implementation classes must be refactored. Perhaps each class can be a ‘user story’.

If those ‘user stories’ turn out to be too big to fit into a sprint, we can break them up into individual interface methods. Perhaps each ‘user story’ is simply the re-engineering of a couple of interface methods.

Perhaps, the enrollment and billing subsystems support a dozen business use cases, and we know exactly which service methods support which use case. Perhaps we could break up the refactoring work according to these business use cases. Each user story consists of refactoring the service methods that support one of those dozen business use cases.

The larger point is that all projects have to be delivered. All projects have some stakeholder who knows what shape that delivery must take. Slices of that delivery are user stories. All other work that must happen in order to deliver those user stories in good order, are building blocks. In this refactoring project, say you want to write automated integration tests, which will ensure that user functionality has not been broken. This would be a ‘building block’. A whole class, refactored to our satisfaction, which passes those integration tests, would be a ‘user story’.

How to fit ‘building blocks’ into sprints?

If the ‘building blocks’ do not already exist, before you start the Agile project, let them be subsumed by the user story.

Say that first story, of the first sprint, is the Motor Vehicle Report query that we saw earlier.

As a customer service rep, I want to query the DMV for a driver’s motor vehicle report.

Building Block – One

Before you are able to start implementing the user story, you need a development environment, which might include a source control project, a build script, private DB schemas for each developer, a continuous integration server, and a test environment where the latest build can always be found.

Already available?

It is possible that much, if not all of this, is already available. You have established a development environment over time, and across previous projects. As part of the first user story, in the first sprint, simply do what little is required to setup your development environment for this particular project.

Starting from scratch?

Perhaps you have decided that from this project onwards, you are switching to Git for source control, and your build scripts will be in Gradle instead of the Maven that you used previously.

As part of the first user story, you prepare your Git repositories, and write whatever Gradle script is required to get you through the first user story. Since this is new, you will spend more time on this, than you would have, if you had stuck with the previous technologies. That is fine. It simply means that it will take longer to complete that same first user story.

Is this Greek to you?

You have never written a build script before. You have never used ANT, nor Maven, and hence it will signficantly longer to learn the Gradle necessary for your build scripts.

This just means that it will take you even longer to complete that first user story, since it will take even longer to create your development environment.

Building blocks simply get factored into Velocity

The more time you spend getting your building blocks in place, the more time it takes to complete a user story. The faster you are able to complete your building blocks, the faster you will complete your user story. That is really all there is to it.

Many factors determine how much time you will spend on the building blocks. Do you already have them built, before starting the user story? Are you accomplished at designing and building the particular building block in question, which means you can get through it quickly?

After using Agile in some form or the other for over 10 years, not always rigorously, nor effectively, I finally took a class on Scrum a couple of weeks ago. It was taught by Mike Cohn, of Mountain Goat Software, one of the earliest practitioners of Scrum for software development, and a prominent proponent of the methodology.

As you might imagine, folks brought up the questions we have been considering. Here are a couple of slides from the class, which speak to the matter.

Should 'building blocks' be assigned their own sprint?

Should ‘building blocks’ be assigned their own sprint?

Let 'building blocks' be subsumed by 'user stories'

Let ‘building blocks’ be subsumed by ‘user stories’

Mike Cohn used the term ‘Architecture’ to refer to what I have been calling ‘building blocks’. Mike posited that you do not want to think of ‘Architecture’ as separate from ‘user stories’. Consider ‘Architecture’ an intrinsic part of ‘user stories’. You will typically spend more time on ‘Architecture’ (my ‘building blocks’) earlier in a sprint, and in the earlier sprints. As you move through the project, if you are doing your engineering right, you will simply use already built pieces, or simply apply previously made design decisions, more and more.

As the above image shows, how good you are at getting the building blocks out the way, affects how much ‘user story’ you complete in a sprint, which in a word, is Velocity.

Look to the first principles

All through the four days of Scrum training with Mountain Goat Software, I heard question, after question, which would prompt the same thought in me. Just look to the first principles of Agile. The answers are there.

Of course, the challenge, for them and for me, is knowing those first principles.

chalakanth

Software Engineers, Clerks, Baby-sitters

What kind of a software engineer am I, I often wonder.

Consider a Java developer, Tom Ws, who has worked on SOAP based web services for over a year, yet does not know how XML schema and WSDL are related.

Tom gave the front-end guy, Dick Ui, an updated WSDL. Dick then tells Tom, ok, now I need the XSD. Tom Ws thrashed about for a day, giving Dick Ui this XSD file and that, which Dick kept rejecting. Finally, Tom goes to the purported architect, Harry Bs, who, wonder of wonders, asked Dick Ui the right question. The WSDL contains all the XSDs you need, embedded within it. What on earth are you talking about? After this, the situation resolved itself fairly rapidly.  It turned out that Dick Ui was looking for the schema of JMS payloads, a related but different unit.

Engineers

There are resources, that know your domain, technologies, tools coming in.  These are few and far between. You cannot always count on finding these folks.

However, there is viable alternative.  There are resources that will not know your problem space, nor the technologies and tools that you use, but upon arrival, they are perfectly able to dig into the existing system, discover what they have to learn, and learn it themselves.

They will not have answers right away, but they know the correct questions to ask, and can find the answers with minimal hand-holding.

Clerks

They begin work at pretty much the same gate as the engineer.  There is much they do not know about the business domain, and the technical solutions of your choice.  That is where the similarity ends.

They learn nothing that you do not teach them explicitly.   They wait to be told what to learn.  Then they wait to be taught what they have to learn.    They cannot fill in any of the gaps themselves.

When they run into a roadblock, they will send out an email and are happy to wait, and wait, and wait.  You see, they believe “answers” are someone else’s responsibility.

Baby-sitters

Someone must spoon-feed the clerks.   Enter, the baby-sitter.  You know those people that say to you pityingly, “Do you still code?”.   That’s a baby-sitter.

The baby-sitter thinks that programmers are simply draftsman, translating a perfectly conceived blueprint into Java, C#, or what have you.  That writing computer code is an act of communication, which requires thought and practice to master, a baby-sitter considers a fanciful concern.   The baby-sitter believes that the coder will never have to make a consequential choice when he writes code, and actively encourages the coder to dumb himself down.

If you got rid of the clerks, you would not need baby-sitters.

Imagine

Imagine an engineering outfit that has only engineers. Imagine a development team that religiously avoids both clerks and baby-sitters.

This seems like a reasonable goal to me, especially, if you factor in the trope that a good developer is usually worth 4 or 5 mediocre developers (a case that I must learn to make one of these days). There will be human resource challenges; how to hire and keep high-skilled, and hence high-valued resources, but you have man-management challenges with low-skilled workers too.

As for me

What kind of a developer am I?

I suppose I have been around long enough to be some kind of an Engineer.

On some days, though, I am happy to be a baby-sitter.  That is all the energy I can muster. Further, some poor soul is happy that I am making his job easier, and that is not a small thing.

On other days, I dream about a rigorous development outfit that is entirely, of and by, engineers.  I would very probably bring up the caboose in a place like that, but that may not be such a bad thing.

chalakanth

Early error detection is paramount

Is early error detection, while developing software, really all that important?

This question has probably been asked and answered a thousand times over in the last 50 years.  Let me try to rehash the argument in my own words.

Catch the error while the cement is wet

Construction, of the brick and mortar variety, sometimes provides an apt analogy for software development, so I am going to give that a whirl.

See if you can correlate the story I lay out below, to the various stages in a software development task – DEV, QA, Bug Fixing, and the Aftermath.

Say you decide to build a house. You create some kind of design, and completely construct your house.

After the construction is all finished, and only after this, you call in an inspector to see if the construction is up to code. The inspector finds that your electrical wiring is the wrong gauge.  You must change it.

The wiring is inside the walls.  You have to tear into the walls to get to the wiring.

You have already spent a lot of money and time on the construction.  You want to move in already.  The extra expense is a burden on the pocket book, and the mind.  You are not at your patient, nor enthusiastic best.

The builder had other projects scheduled.  He wants to be done with your house yesterday. He can no longer giving his best attention to your problems.

Some of the folks that worked on this house are needed on the county commissioner’s lake side cottage. The builder brings in some temporary help to make the fixes. These are snot-nosed college kids, who don’t know many of the small, but consequential technical decisions that went into your house’s construction.  They are going to trip over these and make mistakes. Worse, these kids know they will never see you and your house after this summer.

After the wiring is changed, you notice that a couple of windows do not close well. The lights in the stairwell flicker randomly, but noticeably.  The re-painting of the walls in your guest room is not the right shade.  By this time you have no other place to live, so you suck it up, and move in.

After you move in you notice that your water heater does not work well with the new wiring.  You have to replace the water heater.   More aggravation; more time wasted; more expense; send in the plumber.

There is worse.  While monkeying with the walls and the wiring, a construction worker accidentally rammed a 100 pound sander into a load bearing beam.  It now has a crack it in.  No one notices.

Let’s see how this correlates to software development.

DEV

Say you decide to build a house. You create some kind of design, and completely construct your house.

Listen pilgrim, I just write code. I don’t test.

QA

After the construction is all finished, and only after this, you call in an inspector to see if the construction is up to code. The inspector finds that your electrical wiring is the wrong gauge. You must change it.

This happens all the time in enterprise software development. Developers do not test their code effectively. Infrastructure, which enables developers to adequately test the code they write, often does not exist. Everyone, including management, is happy to leave serious testing till after all of the code is turned in.

The bug fixing

The wiring is inside the walls.  You have to tear into the walls to get to the wiring.

The bug is buried somewhere deep in a few thousand lines of code that you blithely turned in. You go digging, make a change some place, with little knowledge of everything else that might now be affected by your change. It is hard to know, there is too much code, code that you don’t even remember exists. The bug fix is a risk.

You have already spent a lot of money and time on the construction.  You want to move in already.  The extra expense is a burden on the pocket book, and the mind.  You are not at your patient nor enthusiastic best.

The builder had other projects scheduled.  He wants to be done with your house yesterday. He is no longer giving his best attention to your problems.

I’ve seen this in just about every large project I have been part of. Developers use all of a sprint to write code and turn it in. Testing of this code happens in the next sprint, when both the stakeholders and the developers are also assigned to the development tasks scheduled for this second sprint. Nobody is able to bring their best selves to the bug fixing.

Some of the folks that worked on this house are needed on the county commissioner’s lake side cottage. The builder brings in some temporary help to make the fixes. These are snot-nosed college kids, who don’t know many of the small, but consequential technical decisions that went into your house’s construction.  This is going to trip them up, and they will make mistakes.   Worse, these kids know they will never see you nor your house after this summer.

You designed and wrote the original code. However bugs are assigned to someone else, who knows little of the business requirements, the design decisions that went into the solution, and the contours of the code base you created.

Sometimes this someone else is a consultant. And we know consultants can be a mixed blessing, don’t we?

The new and not so improved aftermath

After the wiring is changed, you notice that a couple of windows do not close well. The lights in the stairwell flicker randomly, but noticeably.  The re-painting of the walls in your guest room is not the right shade.  By this time you have no other place to live, so you suck it up, and move in.

A bug fix can fundamentally improve the solution you constructed. Or it may just be a jerry-rigged whatchamacallit that Rube Goldberg would look down his nose at. Often it is the latter, which gets you past today’s problem, and sows the seeds for several others.

But you have no choice. The show must go on.

After you move in you notice that your water heater does not work well with the new wiring.  You have to replace the water heater.   More aggravation; more time wasted; more expense; send in the plumber.

There is worse.  While monkeying with the walls and the wiring, a construction worker accidentally rammed a 100 pound sander into a load bearing beam.  It now has a crack it in.  No one notices.

Like I mentioned earlier, you often have no idea what damage you did while making your bug fix.

Lesson Learned

You want to catch the wiring issue in DEV:

  • Before a whole bunch of stuff was built around it
  • Before you build a whole bunch of stuff that depends on the error
  • While the construction crew was focused exclusively on this task
  • When the folks who made the error are available to rectify the error
  • When you have the least risk if you make another mistake
chalakanth

Knowing a tool vs knowing software development

Give an enterprise developer a top notch table saw, and a 500 dollar power drill.

Give that developer all the training he wants on those tools.

Ask the developer to build a chair.

His chair will come out with 4 legs of different lengths, cracks in the seat, and a couple of nails sticking out.

The developer knows how to use his fancy tools, but he does not know how to build a chair.

That is the difference between knowing how to use a tool, and knowing how to do software development.

They are two different bodies of knowledge, two different sets of skills.

By the way, that chair has business value.  If you must sit, and that chair is all you have, you will sit on that chair, carefully, and with a few choice curses.  I am willing to bet that this is how business folks view most enterprise software that they are saddled with.

chalakanth

Bloody-minded software development

I’ve been pre-occupied lately, with the familiar notion that ‘action‘ has its moment.  How much planning did the chicken do, before crossing the road?

I have seen two kinds of software development outfits.

One was all chaotic action, with apparently little method, yet who always managed to deliver something. Some thing went into production.   Some thing of some value was up and running.  The plan, if one can be said to exist, was often brutal but effective, like clearing a minefield by having your platoon walk across it.  Quality was an alien concept, little more than a pretty thought.

The other development shop is all talk, with little to show for it.  Good people, talented, even competent individuals, who collectively can’t seem to program their way out of a paper bag.  They have worked two, three years on something, nothing of which is in production.  The waste is heart-breaking. Some thing, some essential bloody-mindedness, is missing.

If I were a business person, I would have to pick the former team every time.

Delivery is an essential, like the food you put in front of a starving man. Quality provides long term value, with long term benefits, and requires constant application – it is putting healthy food in front of a starving man, and continuing to feed him healthy food, even after he stops starving.

 

 

chalakanth

Who verifies the blueprint?

You have a business problem (sometimes called a ‘business requirement’).   Someone devises a solution.  You create a blueprint (sometimes called a ‘specification‘) of the solution.   Then you construct the solution that the blueprint specifies.  This workflow suggests that there are at least two things to verify.

  • Does my construction adhere to the blueprint?
  • Is the blueprint correct in the first place?

Here is an example.

Business Requirement

To determine the monies that we may have to refund a customer (say an auto-insurance holder), perform the following calculation.

The monies that the customer owes at the moment
minus
the monies that the customer has already paid us

If the customer has paid us more than she owes at the moment, the customer is due a refund.

However, we cannot count all of the payments that we have received from the customer.  There are rules that tell us which payments must be ignored while calculating refunds.  Here is one of them.

The ‘My Dog Died’ rule

Customers that are dis-enrolled because they failed to pay their premiums, can ask for and receive a sort of ‘grace period’, of usually 2 to 3 months, in which they can catch up (or perhaps even pay ahead).  If they hit all the payment targets within this period, the customer is re-instated.  Let’s call this the ‘My Dog Died, So Have Pity On Me‘ rule.

Specification of the solution – The Blueprint

So how will we satisfy the business requirement?  What are we going to build?

The brain trust (the business analyst, the architect, the DBA, the resident loudmouth), go off into their huddle, and produce these instructions for the construction crew (a developer, and a tester).

  • Using already known methods, determine if the customer is dis-enrolled because of failure to pay premiums.
  • Determine if the customer was granted a ‘My Dog Died‘ grace period, and if so how long that period is for.  In particular, look for the following attributes.
    • An attribute called ‘GriefStricken‘.  Its ‘Effective Date’ is the start of the ‘My Dog Died’ period.
    • An attribute called ‘GriefStrickenExpiration‘,  Its ‘Effective Date’ is the end of the ‘My Dog Died’ period.
  • When calculating refunds, ignore all payments received during the ‘My Dog Died’ period.

Verification

Does construction match blueprint

As I mentioned earlier, in my environment the construction crew consists of a developer, and a tester.

The developer writes computer code that implements the blueprint.  The developer and the tester verify that the computer code does indeed do what the blueprint lays out.

They discover bugs – mismatches between the construction and the blueprint.  The developer fixes the bugs – removes the mismatches.  At some reasonable point, the construction crew turns in the solution.

But.

Who verifies the blueprint

Yea, you guessed it – the blueprint was wrong.

It turns out that the ‘My Dog Died’ period is stipulated to start on the day that the customer is dis-enrolled.

What we thought was the start date, the ‘EffectiveDate’ of the ‘GriefStricken’ attribute, is only the day on which the customer was approved for the ‘My Dog Died’ grace process, which is often several days after the dis-enrollment.

There was no way for the developer to know this.  The tester did not know this either.  The construction crew only knows what is in the specification of the solution.

Yet, this is a significant bug.