Normalisation of Deviance

Piers Bizony's article "Challenging the Establishment" in the 6th September edition of Engineering and Technology introduced me to the phrase "normalisation of deviance"

Essentially, when there is normalisation of deviance people accept the current practice (regard it as normal) despite it deviating from best practice, or they may not realise what best practice is.

If anyone then comes along to change anything, the practitioners of the normalised deviance question the need for change.

"We've been doing this for so long and nothing has broken, so why change?"

Then something catastrophic happens such as the Challenger shuttle exploding because of frozen rubber O-rings causing a leak in the solid fuel booster rockets.

Normalisation of deviance also applies to poor software engineering.

"I've never had to refactor code/write shorter methods/use descriptive names/etc before, so why should I start now?"

"This code was written before and accepted, so why should I change it now?"

It become acceptable that code is hard to maintain because "that's just the way it is".

In software engineering this deviance will manifest itself in high rates of bugs and increased development costs as software maintenance becomes increasingly costly - not as dramatic as a space shuttle exploding.

MIET

I'm now officially a Member of the Institution of Engineering and Technology (IET) after an application process with rather a lot of unexpected to-ing and fro-ing from communications between my sponsor and the IET getting lost. So, I now have another group of letters to put after my name.

The biggest difference between the British Computer Society (BCS) and the IET is in their publications. The IET website and members journal (E&T) are a lot more engaging than the BCS website and its members journal (ITNow) - though ITNow has featured some passionate engagement recently (but I'm still waiting for my copy to arrive). E&T features engaging articles that go into depth, whilst ITNow articles often appear to be lightweight puffs for products and services by their authors. Also, at around 70,000 members, the BCS can only manage to produce a journal with a handfull of pages, whilst the IET with 150,000 members produces a monthly journal with much more than twice as many pages.

Whilst at university I had the choice between the BCS and the Institution of Electrical Engineers (IEE) (who would later merge to form the IET). At that time I considered myself very much to be a computer scientist, not an electrical engineer.

After graduating (a few times) I saw no value in being a BCS member so I resigned.

After a few years I began to see myself as a software engineer, and that what I was doing was engineering. Since the BCS was a member of the Engineering Council I rejoined the BCS and eventually attained Chartered Engineer status. Unfortunately, I see less and less evidence of engineering in the BCS, whilst the IET certainly maintains its engineering face.

Wise Customer went to the house of Proud Businessman

Wise Customer greeted Proud Businessman, "Greetings Proud Businessman, I wish to purchase a service from you, but I must first ask a question."

Proud Businessman replied, "Greetings Wise Customer, my heart is gladdened that you wish to purchase a service. But do tell me what question troubles you"

Wise Customer asked, "Proud Businessman, with your service I hope to increase the quantity of gold flowing into my chest, but the cost of your service will take a great deal of gold from my chest. How do I know that the service you provide will be good?"

Proud Businessman opened the door to a storage room and bade Wise Customer to enter. Proud Businessman pointed to a stack of parchment at the back of the room bearing the legend ISO 9001 and said, "Behold Wise Customer, do not be afraid of the quality of my service, for we have a quality process."

Wise Customer brushed a cobweb from his eyes and said, "Proud Businessman my eyes clearly see that there is much dust gathered on these parchments. How can that be so?"

"The lines on my face and the greyness of my hair show you my age, just as the thickness of the dust shows that I have had a quality process for many years." replied Proud Businessman

Wise Customer thanked Proud Businessman saying that he must visit his apothecary for he had developed an irritation from the dust and must seek an ointment to provide relief.

Proud Businessman was happy for he had just received a wealthy government contract that morning.

Agile Success

I'm always interested in project success, so I was drawn to this article in CIO.com: "5 Principles for Reducing IT Project Failure"

Rather than being IT-specific I would say that these are simply 5 principles for reducing project failure:

1. Communicate: If everyone involved in your IT project doesn’t know what they’re working on, when it’s due, how to get it done and who the audience is, how can you possibly expect your project to succeed?

The aim is to ensure that everyone in the project knows what is going on, what is expected, when it's expected, for whom it's expected, etc. Nobody is then treated like a mushroom kept in the dark and fed fertiliser.

2. Consolidate: We’re talking about information and tasks here.

If you can't find the right information for what you're working on, or it takes you a long time to find it, you are wasting time and effort that can be better spent working constructively towards your project goals. Reduce bureaucracy, cut through the red tape and put information within easy reach so everyone can do the job that they are supposed to be doing.

3. Prioritize: Issues come in, bugs are discovered, and new application features are conceived of every day.

Dropping everything to work on a just-reported bug, or to do a "quick job" for someone is probably not the best use of someone's time. The bug may be of lower priority than what you are currently working on. The "quick job" is a distraction where your time can be better spent working on something delivering higher value.

4. Requirements: In ALM and your SDLC, before you can do anything else, you need to identify, prioritize and agree on application (and project) requirements.

It is fundamental that you need to know what the requirements are before you work on something. Once requirements are elicited, they can be analysed and prioritised. Misunderstandings can be removed to ensure that the project team and stakeholders are of one view as to what the requirements are. Understood requirements mean that something can be designed and built. Understood requirements mean that a plan can be made to test them.

5. Manage Time: As management, it’s up to you to take the first step in setting reasonable deadlines and milestones for your team throughout the project.

Knowing where you are in a project, where you should be and how fast you are delivering are vital pieces of information in determining what will be delivered by when.

Alternatively, these are just, 5 Agile/Scrum principles.


1. Communicate

Scrum revolves around a cross-functional, self-organising Scrum Team that has a daily stand-up meeting where every team member communicates what he did the previous day, what he is committing to do today and if he has any blockages or impediments.

Developers, testers, DBAs, designers, BAs are all kept informed of what all the other members of the Scrum Team are doing.

Scrum also involves the whole Scrum Team being involved in the Sprint Planning.

The Scrum Board and Burndown Chart should be visible to all, so everybody can see what the Scrum Team is working on, and what their progress is - instant reporting!

2. Consolidate

Agility is about individuals and interactions being valued more than processes and tools, and about working code being valued more than comprehensive documentation. "Muda", one of the Toyota Principles, is focused on removing waste, where waste means wasted materials but also means wasted time and effort.

Agile practices refer to doing "just enough" or doing what is "sufficient and necessary". Agile teams commonly use wikis to document information, they record requirements on cards or Post It notes which are kept on the Scrum Board (or Kanban Board).

3. Prioritise

Scrum requires the Product Owner to prioritise work. The Scrum Team only works on what is currently the highest priority collection of requirements. Requirements should be prioritised so that the ones delivering the highest business value are delivered first. Also, those requirements which eliminate the most risk should also be prioritised highest.

Considering simply high and low business value and risk, the best order in which to prioritise requirements according to their business value and risk is:

1. High risk, High business value
2. Low risk, High business value
3. Low risk, Low business value
4. High risk, Low business value

The aim is to eliminate as much risk and deliver as much business value as early as possible, overall delivering a high ROI. Risk can also be categorised as uncertainty - the uncertainty that something is feasible.

Bugs, new requirements, existing requirements all go on the backlog and are prioritised accordingly.


4. Requirements

Scrum can not proceed without requirements.

The list of requirements is maintained in a Product Backlog, in priority order (c.f. Prioritise)


5. Manage Time

Scrum sets regular deadlines for the Scrum Team to commit itself to delivering working software satisfying the requirements.

Progress is visible on the Scrum Board - which presents a snapshot of progress in terms of what's Done, what the team are Doing and what's still To Do. Progress is also visible in the Burndown Chart which shows the rate of progress.

Requirements and tasks are estimated by the people who will do the work - acknowledged to be more accurate than estimates by people who do not do the work - and that knowledge is used to set realistic expectations.

Daily reporting in the Daily Scrum means that people are held accountable to the whole Scrum Team for the commitments they make each day.


So, 5 principles for reducing IT project failure are satisfied by Agile practices. Though I often remark that what are often regarded as Agile practices (coding standards, continuous integration, etc.) are just good engineering practices. So, Alistair Cockburn is right that Agile is the norm - it's just good practice.

Software Development is not a Production Line

I was discussing the software development process with one of the architects in my team this afternoon. The key observation was that software development is not a production line; the only production line in software development is when you're stamping out the media once the software has been developed.

Software development, by the very nature of software, suffers from maleability. Because it is relatively easy to change software, change ensues.

Walter Vincenti in "What Engineers know and how they know it" coined the terms "normal design" and "radical design".

Normal design is when "...the engineer knows at the outset how the device works, customary features, and if you design it properly along such lines, good likelihood of accomplishing the desired task".

Radical design is where you are doing something new and innovative and you don’t know how it should be arranged, you don’t know how it works, you’ve never really seen anything quite like this before and you have no presumption of success.

Microprocessor Development

Microprocessor development is comparable to software development: there is lots of design work and experimentation, then the resulting microprocessor is stamped out on a production line.

The environments in which microprocessors find themselves deployed are a lot more controlled than software, the microprocessor's interface to the circuit board on which it is mounted is well defined, the strength, potential, frequency and shape of the signals the microprocessor receives is fairly well defined.

Software, particularly when used by humans (or other software created by humans), deals with a vastly more complex environment.

Craftsmen

Rather than production line workers the people involved in producing software are like a collection of craftsmen.

Like a craftsman, whenever working on a problem, each one will:

• analyse the problem to some degree
  (the stone sculptor will observe and feel the stone)
• design a solution, or a sufficient first step
  (the stone sculptor will decide where the first major cuts will be)
• implement a solution
  (the stone sculptor will start hammering and chiseling at the stone based on his design)
• test the solution
  (the sculptor will look at the carved stone to decide whether it matches the subject)

The State of Agile in 2009

VersionOne have released their fourth annual State of Agile Development Survey results for 2009

I still find it odd that people regard things like coding standards, refactoring and unit testing, as agile techniques; even continuous integration and automated builds. These techniques are just good software engineering practice and should be used everywhere, irrespective of whether you are trying to be agile or not.

Lack of experience with agile methods was the largest reported cause of failed agile projects. Though it's not clear whether the report means failed projects that were supposed to be agile or projects where the agility failed.