27 January 05 - 21:09Assertions (part I)

Assertions are portions of program text which characterize some boolean expression that is supposed to be true at the point in the program where they appear.

Assertions help find and prevent bugs. Consider the two main ways that programs grow from simple to complex, from small to large: writing new code and modifying existing code. Assertions can be useful in writing new code, but they are crucial when modifying code.

When you modify code, the first thing you want is usually to figure out what the old code does. At the smallest level of detail, this often means looking at a variable and figuring out what values it can hold. Say we're looking at a chunk of code and we know count is an integer. If it appears within a loop, can we figure out whether it goes up or down ? Can we figure out if it will never go over a certain maximum ? If it's a parameter, can we rely on it to be positive ? Could we overflow if we use it in a multiplication ?

Being able to make confident statements in answer to questions like these makes modifying code easier and safer, because typically we can chain such statements together into deductions that end, "and so adding this line will result in correctly computing what is now expected of this function". Or, when looking for bugs, deductions that end, "and that is why the server keels over whenever one user logs in just after another has completed form Z".

Assertions arise in different "methods" for constructing programs - I use the term loosely. One of these is called Design by Contract, and there must be quite a few programmers who associate assertions with DbC. They're also used to construct loops.

Programmers familiar with Test-Driven Development who learn about assertions can have a sense of déjà vu, because the kind of tests that TDD popularizes have a similar role - they help reason about what bits of program do, by providing examples of the sort "when I pass this kind of input to that method, I get back that kind of result".

There are interesting questions which arise when comparing TDD-style unit tests and assertions; technical questions, to start with, but also questions which seem to go a lot deeper, when we ask what kind of things, logically, expressions are, compared to the rest of the program.

More later - stay tuned.

- Software Development - two comments / No trackbacks - § ¶

26 January 05 - 18:47Tales from the Dojo

The "coding dojo", after a few entries here since its inception, now has a blog of its own; the idea is for Dojo attendees to post their thoughts on each session, on the Kata we do, on how their skills are evolving, or whatever is apropos.

- Software Development - No comments / No trackbacks - § ¶

22 January 05 - 14:09And two

Two years ago now, I wrote: "The idea is to put at least one fresh thought down every day. We'll see how that goes."

The actual rate wasn't one a day, but still pretty decent - an average of one posting every 2.8 days, (252 posts over 731 days).

I have enjoyed blogging, in particular as a way to reach out and connect to people I might not have "met" otherwise.

In the past year I have diversified the activities which constitute practice for my writing, adding (private) freewriting to (public) blogging; I have published another few articles, including two in french.

Two years ago I started this blog following the demise of a book project, to discover what it was I needed to write about. That discovery is still ongoing. In the past year I have dallied with one other "long-form" project (not a book, but longer than an article), which didn't pan out; there's a new one I'm considering, about dealing with project risks. The main thing I've learned is to take it easy when one of these doesn't go through as planned.

- Blogversations - No comments / No trackbacks - § ¶

21 January 05 - 23:14Analogies - What You Know That Ain't So

Like Martin Fowler, I'm "suspicious of using metaphors of other professions to reason about software development". But for different reasons.

The problem is that typically, reasoning about software development on the basis of analogy (which is what we really mean by "metaphor") goes in a certain direction: we compare software development with something we don't know really well - say, the construction industry, or the manufacturing industry, or the practice of accounting, etc. And then we use some clichéd insights about these mostly unfamiliar areas to buttress our well-worn, strongly-held convictions about software development, which we think we know well enough, thank you. Hence the perennial arguments: "once a building is erected it's incredibly expensive to make structural changes to it, so get your architecture right first thing".

The maneuver from a position of sketchy knowledge to one of smug complacency has little appeal to me - weakness shoring up weakness. It is a textbook illustration of the old saw that "it ain't what you don't know that kills you, it's what you know that just ain't so".

On the other hand, I know quite a few software practitioners who had (or still have) a completely different career, and real expertise in that field; one example that comes to mind is Dave, who originally set out to be a family therapist. Such people are entitled to draw all the analogies they want, I believe; such a cross-pollination of ideas is in fact key to the vitality of our field, and perhaps of any field.

So that's another combination - strength with strength. I'm a bit twiddler by vocation, so I can't keep myself from thinking of the remaining two. And in fact, we also use analogies to good effect when we discover the "real" truth about a field we didn't actually know that much about, and use that to put our software development knowledge in perspective. Reading about lean manufacturing or the Theory of Constraints woke me up to some problems actually relevant in manufacturing, as opposed to my mental picture inspired mostly by Charlie Chaplin in Modern Times. It also provided me with an abstract view of these problems, which was necessary, absent experience on a shop floor, to understand the narrative. And this abstract view in turn happened to have elements applicable to software work. That would be gaining strength from previous weaknesses of knowledge.

Lastly, I might expect a category where an analogy to something that we are particularly familiar with, an everyday experience maybe, illuminates unprobed questions about software development - unsuspected weaknesses. As an example, I particularly liked George Paci's way of subjecting the term "discipline" to scrutiny, as used in the expression "high discipline methodology", by way of an analogy to the everyday (for some of us) experience of driving a car - this was in a recent thread on the XP mailing list.

- Private Life Of The Brain - two comments / No trackbacks - § ¶

17 January 05 - 22:18Random Staffing

I had the pleasure recently of hearing Johanna Rothman's voice again - not in person, alas, but through her excellent IT Conversations interview (deftly conducted by Roy Osherove). I've been exposed to Johanna's writings for a while, but to hear some of these same thoughts (mixed with reminiscence and advice) in her own, conversational voice got me thinking again about the issue of hiring, and in particular of forming project teams.

If I had to single out one post of Johanna's to summarize what I take away from her writings on hiring, this post summarizing what I call "random staffing" would be the one. The way to hire well is to hire based on a broad range of skills, and to hire based on how well the person is likely to fit her role and fit in with the team. Hiring based on keyword matching over popular technologies and quantitative experience: "3 years of Java, 4 years of C, 2 years of MS-Project..." - that is random staffing: you might be no worse off picking a CV from the pile at random. Hence Johanna's advice and the concrete tools and techniques she recommends: using auditions, asking candidates to reveal their values and principles of action through behaviour-description questions, hiring for diversity along various axes.

What got me thinking is, exactly the same issues apply to picking people to form a team. Suppose the hiring problem is more or less solved - the organization has mostly good people on board now. Are we out of the woods yet ? Well, no - we need to ensure that the same principles are used in staffing teams. Given the effort involved in hiring well, isn't it all too likely that teams will suffer from "random staffing" even if hiring as a whole is on a sound basis?

One thing that bothers me is that in organizations large enough for this to matter (which means a fair number of teams, say from twenty people on up) the hiring process will heavily involve and be largely controlled by a HR department. Where does the knowledge gathered by HR during the hiring process end up once candidates are hired ? How to set things up so that team staffing is consistent with hiring ?

- Management - No comments / No trackbacks - § ¶

14 January 05 - 21:58Good programming

Good programming consists of three components: method, insight, and form. That's a rough conceptual framework intended for attendees of the Development Dojo - an idea I came up with a few months back and which is now becoming a reality.

The dojo is essentially a group of programmers, meeting for the purpose of becoming better programmers through practice, such as Kata. Nominally there are "students" and "masters", though that is problematic at this early stage, because we don't yet have a good way to tell who among us is more "advanced" in the craft of software development relative to the others.

I wanted to explain what the focus of the dojo would be; for instance, why we wouldn't be spending much time on the mechanics of TDD, or why algorithmic knowledge of deep elegance, a beautiful result from graph theory for instance, would also not be relevant. In my view, the point of the dojo is form, and it isn't either method or insight.

"Method, insight, and form" probably applies to a lot more than just programming. For instance, as yet another experiment into capturing the elusive quality known as "beginner's mind", I have taken up the guitar, and I see quite a few parallels.

You see, if you're going to teach something you know backward and forward - or even, actually, something that you're familiar with for a long time - you're going to run into problems because the people to whom you teach live in a different world from yours, with respect to what you teach. You probably can't even imagine what it's like not to know how to read; that's why teaching kids to read is frustrating at times, you have no clue what's going on in their heads as they stumble over a word that looks dead simple. That's why it's hard to teach object orientation, which you grok in an offhand way because you've been coding in C++ and Java for most of your career, to people who have a hard time telling a pointer from a struct.

Carving up the whole thing into three parts might help with understanding how it looks to a beginner, too:

• Method has to do with how you go about your work. What systems or structures you have in place to avoid wasting time, ensure you work on the right thing, avoid mistakes, and so on. In guitar, I've found that practice is a major place for method. To get anywhere you have to practice sitting in the right position, you have to practice with a metronome, and so on. In programming, "method" has a fairly straightforward meaning - TDD is part of it, so is using CVS or some other version control system and having good discipline in its use, and so on.


• Insight has to do with how you relate a solution to a problem, more broadly with how close you are to the core of things. In guitar, I suppose that "insight" is simply a sense of what is musical. I have scoured bookstores and the internet for information on "music theory" and the interesting thing is that no one is able to explain music; there are volumes of "systems" that sort of tie everything up together, notes, vibratory phenomena, scales, rules of harmony and so on, but no one can tell you in simple terms what makes a song sound catchy or a symphony sound inspiring. In programming, "insight" lies in the algorithms, in a sense of the structure of the problem and of the solution. Being able to prove that a given algorithm is O(log n) belongs in insight.


• Form is what makes the execution good as opposed to the idea. I think it's unexpectedly more mysterious than the other two, although it has a number of extremely simple and plain components. In guitar, form consists in part of placing the fingers in the right places, that much I've got already. Don't ask me about the rest. In programming, "form" includes such things as not naming variables with two-letter names, using indents to improve readability, and removing duplication. But there's more to it than that. Insight is what you say with the language; form is the language itself - and language itself is quite mysterious. A lot of poetry, for instance, consists almost solely of form. When we look at a program and say, "Yeah, I can see that this thing works, but really what a lousy design", we're saying that it has bad form.

The point of Kata, and the point of the Dojo, is form.

- Software Development - one comment / No trackbacks - § ¶

12 January 05 - 21:22Working Effectively With Legacy Code

Michael Feathers, author of "Working Effectively With Legacy Code", will be in Paris later this month favoring the XP Practitioner's group in Paris with a visit. The agenda includes beer, talking about legacy code, having your copy of WELC signed, and generally having a fun time. Offhand I can't think of anyone reading this blog who might not already know about this through another channel, but if you're interested and in the neighborhood on the 28th do sign up here.

- Software Development - No comments / No trackbacks - § ¶

12 January 05 - 17:06Employment vs. freedom

I was told that if I discussed anything to do with work then I was representing the company and must conform to their rules. Obviously I dispute this strongly - this is like saying we have a new feudal system where companies are the lords and employees are mere serfs who they own. -- Joe Gordon

I'm tempted to say - "precisely". Elsewhere Esther documents other degradations of the work contract and steps people currently stuck in employee positions can take to restore some sanity to the situation.

When I reflect on what's happening to the work contract in our industry (and many related industries, and I'll come back to how they're related) I try to take a bit of historical perspective. The work contract as we know it arose from the balance of power between the "working masses" and large corporations, and this situation in turn arose as a result of the industrial revolution. So, essentially, the situation covered by the "standard" understanding of the work contract is that of the factory worker.

If you work in a factory, you're essentially loaning out your hands and body to the employer, and during working hours you essentially become part of the "means of production", to use the Marxist term. It seems a logical necessity that your agreeing to the work contract entails giving up some freedom of movement. You can't provide the service the employee is paying for if you wander out of the factory. The employer "owns" your hands etc. during working hours, effectively. This is naturally subject to various regulations, in particular on overtime, safety, etc. to ensure that your body does not become useless to yourself; failing such obligations on the part of the employer, what you would have would indeed be slavery.

This is all kind of obvious, but now consider that this exact same contract with the same kind of safeguards is increasingly being used in what we may call "knowledge industries" - those where what employees bring to the means of production is their brains, not their hands.

This may seem like merely an extension of the concept, but in fact it's much more problematic. Thinking is a lot more holographic than working with one's hands. The insight you apply to today's problem might come as a direct result of musing you did this week-end while out fishing. Does it even make sense to compensate a knowledge worker for the time they spend on the job ? It's not obvious that it does, but that's what we keep right on doing, essentially out of habit.

It's even harder in the case of people like Joe Gordon, whose story I linked to above. What brought matters to a head in Joe's case was ostensibly blogging, but I suspect that the problems are much deeper than this one epiphenomenon of the Internet, and affect the entire range of knowledge industries. Joe works in a bookstore - he's paid, in essence, to have opinions, to be passionate about some books, and so on. His work is who he is, not what he does. The problem is that the freedom to be who you are isn't one that can be "temporarily" turned over to an employer. It's entirely holographic.

All knowledge is basically like that - it's a matter of who you are, not of what you do. It's something you can sell - that's what I do as a consultant. But it's not something you can meter, and that spells trouble for the work contract as we know it. That's the root of both Joe's problem, and the problems of the "EA spouse" with overtime - the employers are not getting out of the contract something measurable enough to be managed well, and they're reacting by trying to extend the claim over employees that the contract gives them.

- The Universe And Everything - No comments / No trackbacks - § ¶

10 January 05 - 13:52Programs are texts

...and, perhaps, more texts are programs than we realize.

One of the reasons software development is hard to do, and harder still to manage, is that there's no accepted wisdom as to exactly what software development is. The nature of the beast is still largely a mystery.

Some have argued that developing software is akin to drafting a prose document - anything from articles to chapters to books and beyond, depending on the discussion. As analogies go, it's a rich and fruitful one in the way good analogies are - it prompts you to consider many alternatives to established wisdom that you may not normally consider.

For instance, authors of successful novels vary wildly in how they design book-length texts: some prefer to work from a detailed outline, that becomes ever more detailed until eventually it has been fleshed out into a book; others prefer to write bits of the story as they occur to them and over time glue the bits together into a book, rewriting large portions as they go along; others still just sit at the typewriter (or PC), start out hammering a beginning and bang away until they come to the ending. This may hold suggestions as to the necessity of always starting a development effort with a detailed schedule, or a detailed design.

There is also a common objection to the analogy, which goes like this: programs are only valuable if they work, and there is no sense in which texts "work" like programs "work" - deterministic, predictable, repeatable. People like a text, or don't like it. They understand it, or they don't - writers try their best, but they can't tell if they'll be understood.

I suspect that the objection betrays a narrow response to the term "text". The objection carries some weight when we are thinking primarily of "literary" texts, where the "work" to be done is emotional; novels, poems, etc. That sense of "works" is very distant from what we mean when we say a program "works".

But in fact there are many more kinds of texts which have to "work" in ways that are much closer to the way programs must "work". Not exactly the way that programs "work" - but the differences are small enough that we may learn something from the comparison.

For instance:

• Poems (before we leave the literary world entirely) have to obey certain formal constraints, such as rhyme, meter or stanza structure. In fact you could conceive of a program to check these properties automatically. A violation of those constraints would be a "bug" in a sense closer to the one we use - a syntax error.


• Mathematical proofs: these provide a good "intermediate" stage between ordinary prose and computer programs, in a conceptual but also historical and sociological sense. A proof may be extremely formal (to the point that it could be checked by a program), or it may be breezily informal ("imagine the triangle flipped and superposed over the non-flipped version"), but the slightest misstep will cause it to "fail" in important ways.


• Scientific theories: it is usually not possible to fully express those in mathematical formalisms, so they represent a distinct category; although some (e.g. particle physics) rely on extensive mathematics, they also contain elements, such as their ontologies ("what objects exist in the theory") which cannot be formalized. In one view, a theory "works" if it predicts observable events in the world, and scientific progress consists of eradicating "bugs" (events not predicted, or events predicted which are not observed).


• Contracts: a loophole is to a contract as a bug is to a program. These are texts which have to "work" in an even closer sense to the way programs must "work". Authors of contracts cannot rely on a sympathetic reader to fill in the blanks for comprehension; rather, they assume that their creations will be scrutinized with, at best, indifference to their aims, and at worst active hostility. Contracts will be interpreted as literally as necessary *and* as liberally as necessary to defend the attackers' conclusions.


• Systems of laws, game rules, etc: these can be considered as generalizations of the idea of "contract" to multiple parties, or specializations of it to various domains. These texts can be considered to "work" or "fail" in more diverse ways than contracts. A game "fails" if it is impossible to play, of course. For a game to "work", its rules must also be simple while generating non-trivial strategies. Games "fail" if it is too obvious how to win, or impossible to win.

Laws in particular are interesting because one could say that they "program" a society, that they are the "software" which runs on the "hardware" of that society's infrastructure. Just as with computers, that software has a wide range of variablity, subject to constraints related to physical law and regularities of human nature. Just as with computers, there are common, best-practice "architectures" such as representative parliamentary democracy.

Given all this, it's unlikely that the programming profession can learn nothing from attentively studying the techniques practiced by the literary or more broadly the "text-oriented" disciplines, when these disciplines succesfully attack problems no less complex than the ones we are presented with.

- Software Development - two comments / No trackbacks - § ¶

03 January 05 - 14:46Years and projects

What do years and projects have in common ?

We like to pretend they have definite endings or beginnings, and make up conventions to mark these endings or beginnings. January 1 is one such convention, but there are also various "fiscal years", and many other cultures have different reckonings of the cycle's starting point, or of what to measure the cycle by.

I'm still unable to think coherently about what happened in South Asia, other than to donate to one of the relief funds. Let alone write much of anything. The disaster's proximity to the new year's festivities was unsettling - death so large and sudden, rubbing shoulders with celebrations of life planned and savored. And the questions that must be faced - if a fraction of the same energy we invest in planning the celebrations had gone into planning for the disaster, how much less would the death toll be ?

Still, there's never a bad time for wishing others well. I'm happy to go with convention today and wish you all a great 2005. What are your hopes and wishes for the coming 363 days ?

- The Universe And Everything - No comments / No trackbacks - § ¶