30 April 04 - 14:10The Compensation Game

One reason occasionally advanced for wanting to measure individual performance is that compensation is individual. A manager may therefore want to tie compensation to individual performance or "worth", at the very least so that the ranking of compensation levels reflects the ranking of individual "worth".

In a recent discussion over on the Extreme Programming list, Ron argued (or seemed to argue) that the "better" performers should be paid more, and that this was obviously desirable and proper of any given compensation scheme. My response was that this was far from obvious, on at least two grounds - ethical and utilitarian (i.e. is that policy actually in the best interests of the businesses which practice it).

The utilitarian issue is interesting in its own right, but for some reason the notion that "better people are paid more" is not necessarily fair seemed the more controversial of the two.

The point of forming a team, or for that matter a business, is that several people together will produce more value than the sum of what they could produce separately. An assessment of "better" is to be taken as "contributes proportionately more to the extra value", or variations thereon. A first-order measurement of this might be how fast a person codes - his or her individual velocity.

The owner of the extra value does have the right to distribute it however he/she/it sees fit, by definition of "ownership". (That authority is usually delegated to the manager.) In particular, the owner might well decide to keep all the extra value for him/her/itself. That might have a negative impact on morale - an illustration of the utilitarian issue.

The ethical issue is about redistributing the "extra value". Who is to get a bigger share ? And who is to decide on how big a share everyone gets ?

Our current understanding of the corporation yields definite answers to these two questions, with a small range of variation. Management typically gets a bigger share than performers (regardless of demonstrable contribution to added value), and management gets to decide who gets a bigger share. It is far from obvious that this is a fair arrangement.

I proposed a game which allows exploring both questions.

First, postulate a team: Alex, Bri, Chris and Dee. Three are male, one female. Two are US citizens, two are immigrants. One is wealthy enough (by family) not to really need a salary; two are middle-class; one sends money home to a poor family. One of them, when part of a pair, consistently causes that pair's stories to be done much faster. One of them is "appointed leader" of the team. One has been in the team three years, one joined three months ago. One was brought in during the job market's darkest days at a low salary. And so on; we can make this game more interesting by filling in plausible life details.

In the second round of the game, you (and three other players) will be handed a card: A, B, C, or D. The card stands for one of these life histories. On the table will be a project case study, with financial results. These results may translate into either a global raise budget to be apportioned to the team, or into a decision to reduce personnel cost by a given amount (i.e. firing someone). You will play your hand according to the rules agreed to in the first round.

In the first round, you don't know which of Alex, Bri, Chris or Dee you will play. Nor do each of the three others. The object of the first round is to negotiate a "fair" procedure for conducting the second round.

If "codes faster means paid more" is your move in the first round, it could well come to pass that in the second round you will be awarding the biggest raise to (say) Alexander, a wealthy male with no dependents, who happens to code like a demon but hardly ever cracks a smile, and has consistently refused to chip in for team parties or presents to other members on special occasions, for the three months he's been there. (Someone else is holding the team together, but Alexander definitely isn't her.)

I would argue that this distribution isn't fair. Would you ?

Now, the huge advantage of "worth more means paid more" is that it's a simple procedure (assuming you have a simple, objective, fair way of measuring "worth more"). Simple procedures have (I think) a better shot at being fair than complex procedures, such as one where you would take the weighted average of a gazillion factors (such as suggested by the above life details).

Another simple procedure would be "split the raise budget evenly". There might even be other simple procedures which would also be readily agreed on as fair; if we should fail to find one that simple, maybe we'd have to settle on a less simple one.

A few other pointers can be found at the Wiki page describing a different kind of compensation game: http://c2.com/cgi/wiki?CompensationGame

- Management - four comments / No trackbacks - § ¶

18 April 04 - 22:05Reverse requirements engineering

I'm currently having an extended on-line discussion with a handful of smart and thoughtful folks. We explore the cognitive and technical challenges of learning Java and OO, mainly for the benefit of one of us who's an expert in other areas. This means that there is a lot of discussion going on, but we're also looking at actual Java code being posted and responding with critiques. Sort of a distributed design and implementation review, slanted to self-directed learning.

There's a range of options available for collective discussion, including forums, mailing lists, Usenet, CMS's; blogs, Wikis, and hybrids of the two - blokis, blikis, wiblog, etc.; all with a variety of different preconceptions about how the discussion is to proceed and what tasks are most valuable. Our current choice is perhaps less than optimal for what we're doing.

Wikis are great for producing texts by leveraging the palimpsest effect. They fare less well when it comes to editing code, and are not ideal for keeping track of conversations which bifurcate into multiple subjects. Blogs are great for worked-out ideas and opinions which may yet be refined through further conversation, but they assume that one person is the "primary" author. Web forums and Usenet newsgroups are focused on transient conversations, and keeping track of "threads". The former typically place some importance on authenticating the identity of posters, while the latter are free-for-alls, leading to very specific patterns of verbal behaviour (all too often abusive or inconsiderate). And so on and so forth, with hybrids producing subtle interactions between these various impacts of the medium on the message.

There ought to be a discipline of "reverse requirements engineering" concerned with finding out what goal orientations cause people to favor this or that technology. Then we could trace forward again and possibly match solution to problem in a more effective way. Or maybe that's just a fantasy...

- Software Development - No comments / No trackbacks - § ¶

17 April 04 - 09:24Modeling, systems and software

For quite some time I was content to write programs without ever bothering to draw diagrams (UML or some other kind) before. I rarely resorted to math or formal algorithmic techniques, either.

A lot of the industry talk on modeling strikes me in the same way as the religious talk on sin: intended to make people feel guilty so that they'll toe the line (and line up to buy tool support). So I tended to feel at home in the Agile camp where such forms of modeling are not regarded as a mandatory ritual.

What made me sit up recently was reading Jean-Louis Le Moigne's book on general systems theory, "Théorie du Système Général", the subtitle of which is "Théorie de la modélisation", a theory of modeling. I've been immersed in Jerry Weinberg's books for some time, both the ones explicitly about the systems perspective and the ones which merely take their inspiration from that perspective. And I had previously read Senge's "Fifth discipline", which stresses the importance of mental models. But Le Moigne's exposition is finally making something click as to the connection between models in the most general sense, and what goes on in software development efforts.

I suspect that what we call "development" today consists not so much of programming, but of modeling; not "modeling" in the sense of drawing boxes and arrows in the Uncouth Modeling Language, but more broadly "the act of designing mental structures which aid our dealings with reality, in the context of specific purposes having to do with smaller parts of that reality". Information systems are models of what goes on in a business, software products are models of this or that human activity, etc.

Nowadays, of course, a lot of software is models for dealing with all the software we've surrounded ourselves with. Development tools, languages, lifecycles, etc. embody models of the development activity.

The main problem with our understanding of "modeling" is losing track of the "specific purposes" part of my definition above. Models are not true or accurate in the absolute, only relative to some purpose - generally to some *person's* purposes. The nasty undercurrent in UML advocacy is the suggestion that it's possible, and worthwhile, to shoot for "the one true" model of something. This also leads to the notion that UML itself is "the one true" way to design models - supported by various assertions such as that visual models are better because more abstract, or even: better because visual.

Another area of puzzlement is why all the approaches to "modeling in the context of building software" assume that what is to be modeled is the system as built, finished and seen from a detached and neutral point of view. No one ever seems to discuss building a genetic model linking the system to be built to its ancestor systems, or a developmental model taking into account various "iterations" of growth of the system or stepwise deployment (*), or in general any number of classes of models which it would make sense to use. There are exceptions - Alan Cooper's approach of interaction design consists essentially of modeling users, for instance.

Also, if all "programming" or "development" activity consists of modeling in that sense, then the most generally useful of all models are our models of how we design models - our self-models. These receive scant to nonexistant attention from the industry.

---

(*) I am finding echoes of this concern in Jim McGee's recent entry on "Learning caution": " I still believe that carefully designed and deployed technology is essential for organizations and societies that hope to survive. But that design has to factor in how human systems shape designed systems over time."

- Software Development - No comments / No trackbacks - § ¶

16 April 04 - 01:21Quality software measurement

When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind: it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science.
-- Lord Kelvin

We commonly honor scientists by giving their names to units of measurement, an honor set aside for a select few. This makes Lord Kelvin, among all scientists, an impressive name to have at your side in a conversation.

The above quote is apparently a common fixture of Usenet discussions, and it popped up recently on an Extreme Programming newsgroup. The person invoking the manes of Kelvin was apparently arguing that, in order to make wise choices about certain software development practices, you should be able to cite measurements on the effectiveness of those practices. "You recommend pair programming", goes that cry, "but where is your scientific data ?"

I submit that the cry for data is a stall, a convenient means to sidetrack discussion of the reasons why something is being recommended.

Data matters, but so do, equally, the models or theories which make sense of the data. A classic episode in the history of science is Lord Kelvin's computation of the age of the Earth at about a hundred million years in his most generous allowance, or twenty million years in his later revisions. Geologists of the time suspected this to be a gross underestimate (which it was, by a factor of 50 even for the first figure), but Kelvin's figures were based on the best available scientific data. Given the measured rate of heat dissipation, the Earth couldn't be that old without being frozen solid. His argument had weight and was for a while an embarrassment to his opponents who argued for a much older Earth, to allow for the doctrine of natural selection. It took an entirely new finding to invalidate Lord Kelvin's argument: the discovery of radiation, which implied heat being created in parallel with heat being dissipated. Kelvin's calculations relied solely on dissipation of an original "capital" of heat.

Kelvin had "scientific" data on the age of the Earth, yet the conclusions he reached were wrong. Nor, it seems, was his interest in that particular problem motivated by purely "scientific" motives. And his conclusions were wrong because there were things he didn't know about the system he was studying, and which made the data irrelevant. Now whose knowledge of the whole affair should we say was of "a meager and unsatisfactory kind" ? It's a toss-up. Our modern-day knowledge is certainly more satisfactory, but that is certainly not due to anything we "express in numbers".

Data is good, but it's only data; the hard work is determining whether the data actually supports your hypotheses. Worse, data is usually theory-laden; research motivated by opposite purposes will often turn up data supporting opposite conclusions.

Worst of all, measuring performance data often has side-effects detrimental to what we are supposed to be measuring.

In discussions of performance, what we usually have in mind is a system consisting of various inputs (raw material, such as ore in steel works or ideas in software development) and outputs (steel, software), and processes turning the inputs into outputs. We may legitimately be interested in various dimensions of the system's performance, such as speed, cost, etc. The ideal of performance measurement consists of varying one parameter, "all other things being equal", and measuring the results on one relevant dimension.

This is possible only in a tiny fraction of all real-world situations.

The rest of the time, we are faced with problems where all other things are never equal, and the results have more than one relevant dimension. Some dimensions will be easy to measure (e.g. "faster", since we need only look at the clock on the wall) and others less easy (e.g. quality, accuracy, customer satisfaction, and so on).

There is a problem commonly faced by businesses and other organizations, called "dysfunction" by Robert Austin. Austin has devoted an entire book to exploring the topic, Measuring and Managing Performance in Organizations. It goes as follows.

Measurements are made of the things which are easy to measure, leading to strong pressure for improved performance along these dimensions. At the same time, no measurements are made of the things harder to measure, and no pressure applied there. However, for any given dimension of performance, "ease of measurement" does not necessarily correlate well with "criticality to business results". In fact, the correlation usually goes the other way.

Measurement efforts therefore tend to have adverse effects, because people respond to the differential pressure by slacking off on the aspects not measured, even if they are critical to business results.

An excellent example is software quality. "Productivity", roughly defined as the number of features (or worse, lines of code) delivered per unit time, is all too easy to measure. "Quality" is much harder to measure, because even at its simplest it consists of several distinct dimensions, such as customer satisfaction with the product delivered, programming defects (i.e. "bugs") detected during the development process or after deployment of the product, and various "ilities" such as maintainability.

Austin's model - a purely qualitative one - does a lot to explain the classic paradox of software development: the more pressure you put on a team to have it deliver faster, the later the project will actually ship, as a result of poor quality.

Conversely, Austin's model also explains the "virtuous circle" of agile software development: the more effort you invest in quality, in all its aspects - even the less easily measured ones - the less your software projects will actually cost and the faster they will go.

- Management - five comments / No trackbacks - § ¶

15 April 04 - 12:48On Failure

Failures provide a template in the mind of a project manager. If you've lived through a major failure, then when you hear certain phrases or see things that happened previously, you instantly become more alert and start asking more questions.
-- Ken Orr, "When Dr Kevorkian makes a house call" (Cutter IT Journal 03/2004)

- Management - two comments / No trackbacks - § ¶

12 April 04 - 11:15On Complexity

There are two ways of constructing a software design: One way is to make it so simple there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies.
-- C.A.R. Hoare

These new problems, and the future of the world depends on many of them, requires science to make a third great advance, an advance that must be even greater than the nineteenth-century conquest of problems of simplicity or the twentieth-century victory over problems of disorganized complexity. Science must, over the next 50 years, learn to deal with these problems of organized complexity. [...] Impressive as the progress has been, science has by no means worked itself out of a job. It is soberly true that science has, to date, succeeded in solving a bewildering number of relatively easy problems, whereas the hard problems, and the ones which perhaps promise most for man's future, lie ahead. We must, therefore, stop thinking of science in terms of its spectacular successes in solving problems of simplicity.
-- Warren Weaver

In our time, the technology of machines has drawn its inspiration from mechanics, dealing with complexity by reducing the number of relevant parts. The technology of government, on the other hand, has draw upon statistical mechanics, creating simplicity by dealing only with people in the structureless mass, as interchangeable units and taking averages. [...] For systems between the small and large number extremes, there is an essential failure of the two classical methods. On the one hand, the Square Law of Computation says that we cannot solve medium number systems by analysis, while on the other hand, the Square Root of N Law warns us not to expect too much from averages. By combining these two laws, then, we get a third - the Law of Medium Numbers:

For medium number systems, we can expect that large fluctuations, irregularities, and discrepancy with any theory will occur more or less regularly.

-- Gerald M. Weinberg

- The Universe And Everything - No comments / No trackbacks - § ¶

07 April 04 - 12:55Discourse on Software Methods

...so, in place of the large number of precepts of which logic is composed, I believed that I would have enough with the following four, provided that I were to make a firm and constant resolution not to fail, even a single time, to observe them.

The first was never to accept anything as true that I did not evidently know to be such: that is to say, carefully to avoid precipitation and prejudice; and to include in my judgments nothing more than that which would present itself to my mind so clearly and so distinctly that I were to have no occasion to put it in doubt.

The second, to divide each of the difficulties that I would examine into as many parts as would be possible and as would be required in order to better to resolve them.

The third, to conduct my thoughts in an orderly manner, by beginning with those objects the most simple and the most easy to know, in order to ascend little by little, as by degrees, to the knowledge of the most composite ones; and by supposing an order even among those which do not naturally precede one another.

And the last, everywhere to make enumerations so complete and reviews so general that I were assured of omitting nothing.

-- René Descartes, Discourse on Method

In the midst of a systematic attack on the four precepts, I found the following gem:

Let us note in passing how, in the name of this [last] precept, a few thousand computer science professionals have dramatically rigidified the social dialogue... under the pretext that one could not take new data into account "without rewriting all the programs". It took more than ten years to convince them that it was possible to design programs that... like our own reasoning... could adapt to changing situations. I am not certain that all have accepted, and taken to heart, such a change in their own Discourse on Method.

-- Jean-Louis Le Moigne, La Théorie du Système Général

And Le Moigne proposes four precepts intended to replace Descartes in a new (but no longer intended, as Descartes' was, to be ultimate or universal) Discourse on Method. These are (my translation) :

• The precept of relevance: to allow that any object we consider is to be defined in relation to the implicit or explicit intentions of the [observer]; never to prohibit the revision of such a definition if, as our intentions change, so does our perception of that object.


• The precept of totality: to consider always the object to be known to our intelligence as the visible and active portion of a larger whole; to perceive it in totality, in its functional relationship with its environment without bothering overmuch with obtaining a faithful image of its internal structure, the existence and unicity of which are never taken for granted.


• The teleological precept: to interpret the object not in itself, but by its behaviour, without seeking an explanation a priori of that behaviour by some law relating to its hypothetical structure; but to understand this behaviour, and the resources it depends on, in relation to the purposes that the observer attributes to the object; to hold the identification of such supposed purposes for an act of rational intelligence and allow that to demonstrate their existence shall seldom be possible.


• The precept of aggregation: to allow that any representation is partisan, not through any negligence of the observer, but deliberately so; to seek therefore some techniques whereby one may select relevant aggregates and banish the illusory objectivity of an exhaustive enumeration of the elements to be taken into account.

  - default - one comment / No trackbacks - § ¶

03 April 04 - 18:29Index cards

Why do Extreme Programming practitioners keep requirements on index cards?

Because when you rip a card to shreds, or go through a whole pack when you thought you had a small project, that gives you information about what's going on in the project that's a lot more difficult to ignore than just adding or removing lines in a spreadsheet.

Computers make it easy to keep track of the information, but they make it equally easy to make the information obscure and meaningless, or prevent key members of the project community from gaining access to the information - without them being even aware that they're denied access. The physicality of index cards is a good antidote to that. Index cards are the symbol of a different contract regarding the sharing of project information.

That's only one reason...

- Extreme Programming - one comment / No trackbacks - § ¶