Monthly Archives: November 2006

Software Quality Assessment

I recently came across some questions about software quality that got me thinking assessing existing software. As a software developer and consultant, I’ve participated in many conversations about software quality. Here are some tools I’ve used in conversations about software quality.

1. Atrtributes of software quality of an existing system

Some of my principal concerns with software quality when I’m evaluating an existing system include:

a) How well does the software meet the user’s needs
b) Can an average user make use of the software reliably enough to get to a desired outcome
c) Ease of maintenance and enhancements
d) Technical soundness of system, primarily severity and number of open defects, frequency of system errors and crashes, and perception of stability by system users

2. Convincing others of your assessment

Historically, I’ve not been successful jamming ideas into peoples heads – they seem to get upset and resist. However, if I can involve them in a conversation about quality – and try to understand their perception – and set an example for them, often we are able to come to terms on a common definition. Even if it isn’t perfect, at least I know what to do to meet their expectations (today).

I recently read a story that I think describes this idea. Two developers walk into a village – and are told by the villagers there is an evil monster on the loose – and point to a watermelon. Both developers want to help the villagers understand that watermelons are not monsters. The first developer attacks the watermelon, easily defeats it and proceeds to eat it – trying to show the watermelon’s helplessness. The villagers find this killing and eating of monster flesh monstrous. The second developer lightly agrees with the villagers – and proceeds to share his knowledge of melons.

My short answer (now that I’ve dragged you through the longer one) is that I try to get them to convince me. Often we end up in a mutually acceptable definition – sometimes we don’t.

3. Quantifing software quality

Anything can be quantified (e.g. this software has blue quality). However, we lack rigorous, well defined industry standards to allow us to do this definitiely for all software systems. The final attributes of software quality are often subjective and context dependent. 

If an engineering team doesn’t have a scheme in place to quantify quality – it should create one. A basic scheme I’ve seen teams use in the past is:

  1. Number of open severe defects
  2. Client satisfaction with product
  3. User satisfaction with product
  4. Ability of engineering team and client to openly and effectively communicate
  5. Difference between desired frequency of use and actual frequency of use of the system
  6. Ability of client to operate and maintain the system
  7. Ability of the engineering team to evolve the system

4. Measuring software quality

A method I have used looks at some basic attributes of a software system for quality measurement:

  1. Outstanding defects and frequency (critical defects count more)
  2. User satisfaction with system
  3. Cost of ongoing maintenance for defect removal
  4. Future plans for system (low quality systems often have a replacement plan earlier than good systems)

5. Relationship between software quality and system scope, cost and schedule

I generally follow the old saying “Good, Fast, Cheap – Pick any 2”. Give me 2 degrees of freedom, and I have a fighting chance to still develop a system of reasonable quality. Mainly because I can made meaningful tradeoffs to reach the quality goal. Nail all my feet to the floor, and I can’t move at all, and the Vorpal Duck comes and eats me.

6. Software Quality as variable constraint

I believe that tradeoffs between software quality and the variables of system scope, cost and schedule happen all the time. My typical goal is to make these tradeoff decisions as explicitly vs. implicitly. This requires reflection on the current situation and goals – as an individual and as a team. I believe that as system implementors, we represent the interests of our non-technical customers when developing the system. If we make tradeoffs by significantly lowering or raising quality, we should engage the customer in understanding the implications of the decision. Significantly lowering quality can cause the customer pain the future. Significantly raising quality can cause the customer possibly uneeded schedule or financial pain in the present. This now (d)evolves into a discussion of good enough software.

Elements of an Enterprise Software System

Enterprise software systems have some common elements. It is helpful to have a checklist of these common elements when designing, evaluating or even using such systems.

This is my checklist of common elements of enterprise software systems:

  • Data Entry
  • Collaboration
  • Workflow
  • Reporting
  • User Management
  • Billing/Invoicing/Collection
  • Instruction
  • Content Management
  • Syndication
  • Logging – Operational and Error
  • Monitoring
  • Notification
  • Audit trails


“I’ve made peace somewhere between my ambition and my limitations.” – (from the movie, Teahouse of the August Moon.)

Getting it right the first time is not nearly as important as getting it right the second time.

Wide experience makes for deep tolerance.

It’s easier to pull down than to build up.

The most important events are often the results of accidents. (Polybius)

Action is the antidote for despair. (Joan Baez)

We cannot direct the wind, but we can adjust the sails. (Proverb)

The shortest way to do many things is to do one thing at once. (Samuel Smiles)

If you cannot feed a hundred people, then feed just one. (Mother Teresa)

Tell me and I’ll forget; show me and I may remember; involve me and I’ll understand. (Chinese proverb)

Sometimes you gotta create what you want to be a part of. (Geri Weitzman)

Ideas won’t keep; something must be done about them. (Alfred North Whitehead)

Do not let what you cannot do interfere with what you can do. (John Wooden)

The world stands aside to let anyone pass who knows where he is going. (Author unknown)

You don’t get to choose how you’re going to die. Or when. You can only decide how you’re going to live. Now. (Joan Baez)

The deepest principle in human nature is the craving to be appreciated and the desire to be important. (Dale Carnegie)

It is useless for us to reason a man out of a thing he has never been reasoned into. (Jonathan Swift)

Start with what you can do; don’t stop because of what you can’t do. (Author unknown)

Whatever you are, be a good one. (Abraham Lincoln)

If you think you can or you think you can’t, you’re right. (Henry Ford)

Be the change you want to see in the world. (Mahatma Gandhi)

Great wealth and contentment seldom live together. (Bob Phillips)

For every failure, there’s an alternative course of action. You just have to find it. When you come to a roadblock, take a detour. (Mary Kay Ash)

Have you got a problem? Do what you can where you are with what you’ve got. (Theodore Roosevelt)

Most people spend more time and energy going around problems than in trying to solve them. (Henry Ford)

When the only tool you own is a hammer, every problem begins to resemble a nail. (Abraham Maslow)

It is easier to do a job right than to explain why you didn’t. (Martin Van Buren)

“Only the ideas that we really live have any value.”
— Hermann Hesse. German novelist and poet, 1946 Nobel Prize for Literature, 1877-1962

Stockdale Principle – You have to be realistic bout your current situation and yet stay optimistic about the future.