12 November 07 - 14:13Breaking Acts at Agile 2008

Preparations are starting for Agile 2008. And you can help. The short story is that we're looking for speakers; we'd also like to solicit reviewers. Read on for the details.

In the run-up to Agile 2007 I shared with Elisabeth Hendrickson the post of Tutorials Chair. This means we were responsible for one specific type of session, irrespective of topic: tutorials, which are meant to convey "mainstream agile" content and attract the more polished speakers. We struggled with the scale of the conference - 1200 attendees in 2007 and 1600 expected next year. The number of submissions was quite large - about 600 all told, about 200 in tutorials alone. In reviews we found it hard to give all tutorial submissions, covering as broad a range of topics as they did, the attention each deserved.

Rachel Davies is chairing the 2008 conference. She came up with a metaphor to guide the design of the conference - one that has the advantage of framing this big conference as a cluster of smaller conferences, with the team in charge of each smaller conference free to focus on a topic of interest to them. So, Agile 2008 is organized along the lines of a "music festival", with various Stages; each Stage has a theme and a Producer who tries to put together a great agile conference around that theme. I think it's pretty nifty. Since it's new, there will inevitably be some few hiccups as we get used to it, but I am optimistic it can help reduce the "big conference blues" we experienced in the past two years. There's a stage dedicated to Agile curmudgeons, for instance - I'm thrilled that we're having that. There's one on examples, one on the user experience, one on customers and business value, and so on.

I'm happy to announce that I will be producer for the "Breaking Acts" stage. Quoting from the CFP: "Agile as it stands today is still a work in progress. For Agile software development to remain relevant, it must incorporate new ideas continuously. This stage is for speakers who bring a fresh and surprising look to aspects of Agile we thought familiar, and speakers interested in ideas that are relevant to Agile but not accepted yet as 'mainstream'. First-time speakers are particularly welcome."

If this kind of thing rocks your boat, I would appreciate your helping in one of two ways. Right now, if you'd like to be a reviewer for session proposals, please get in touch with me; I'll give you a quick overview of what to expect and what's expected of you. In a few weeks, if you're interested in speaking at the conference, we'll be opening up our (new) submission system to session submissions. I'll post the details here (if I remember; otherwise, they'll also be on the conference site).

Morendil - default - one comment / fifteen trackbacks - § ¶

06 October 07 - 15:13Principles and patterns of codebase structure ?

Let's say you just had a nifty idea for a software product. Could be a Web site, an IM client, a new Ruby gem, anything. And let's say you've done enough up-front work to suit your purposes and your process preferences. So it's time for you to sit down and prepare a directory tree for the initial import into Subversion or CVS.

What are the rules for doing that ? And what are the rules that govern ongoing elaboration of this directory tree ? I'm not aware that these are written up anywhere (if you know of such a write-up, I'd appreciate a pointer).

Most competent programmers I see sketching a directory tree seem to be doing "the same thing". They'll start at the top level with, say, "src", "test", "doc", "lib" directories. But why ? If their intent is something more complex than a simple app they'll create additional divisions at the top level, or below, which make sense for that intent. These programmers don't seem to be following explicit guidelines or principles or known patterns, and sometimes I find codebases whose authors tried to imitate this "standard" structure in ways that just don't make sense. So, making the principles and patterns explicit could be valuable. I'm thinking that one good way to make those principles explicit would be to look at "ugly" codebases, identify the violations we see that make them "ugly", and reason backwards from that to the "proper" principles.

Based on experience with violations, I'm postulating some "principles of transparency", which suggest things that codebase structure should not reflect:

• A principle of transparency with respect to team organization - the codebase should not reflect the structure of the team. I was quite shocked when one colleague reported to me that he had seen a team using Subversion where the top level of the code base was a set of directories, one for each member of the team. This suggests that the team members "integrate" by manually copying code from their "private" directories to one that is authoritative.


• A principle of transparency with respect to change over time - the codebase should not reflect version evolution. It's always unpleasant (but not quite in the "shocking" category) to come across a directory labeled "old" or "version_before_ajax" or some such. It's the job of version control to track those things.

Primarily I think of the contents of the repository as a "blueprint" for creating a running system. The repository doesn't have to be organized along the same lines as the running system will be organized. (I even debated adding that to the list of "transparency" principles above, but I believe it can actually make sense e.g. to have a top-level division into "client" and "server".) However, the repository does have to contain everything you need to know in order to produce a running system. The repository also has to contain everything you need to know in order to advance to a new version of the system - to fix a bug or add a feature. A memorable formulation could be "For maintenance you need the repository, all the repository and nothing but the repository".

The decomposition of a system into "products" (in the sense of that post by Reg Braithwaite) plays an important role, I suspect, in structuring the code base. Independent "products" should live in separate repositories; however some systems might depend on very closely related "products" (again the example of client and server comes to mind) sharing some common code.

For a single-product repository, the "description of how to produce the end result" theory makes good sense of why many projects have "src", "doc", "lib", "test" and a makefile at the top level. Directly at the top level is what we'll be concerned with - "building" or "deploying" or "installing" a running system from the repository. (Or sometimes a "shippable" rather than a "running" system; a level of indirection is introduced by having the repository be a recipe for building an installation CD and the running system only results from the end user running the installer.) The top-level groupings are according to the function that the various elements play. The "doc" directory is where you go to understand the code, which you then find in "src", after which you can verify your changes by running the tests in "test". In the "lib" directory there are components which are needed to build the system but which you don't need to know, as they are unlikely to require any changes. For multi-product repositories, we can expect that the top level will have a directory for each product, under which the same structure will be found as for a single-product repository.

Do these observations make sense ? Did you come across codebases whose structure you found perfectly clear, or perfectly bewildering, and for what reasons ?

Morendil - default - fourteen comments / eight trackbacks - § ¶

02 July 07 - 15:22Candidate for the Agile Alliance Board

The Agile Alliance will be electing six new board members this year.

I have decided to be a candidate to one of these board positions. The "motivation" part of my position statement reads as follows: "The Agile community is finding ways for software development efforts to deliver more value. Our guiding stars are feedback, transparency, small increments, high collaboration… Yet our efforts to structure the Alliance itself around those values have been limited: for instance our main conference is organized with phases and deadlines, and is still One Big Conference. My intent in joining the Board is to help the Agile Alliance become still more agile itself, while preserving a sense of community that I value very much."

In plainer terms, the Agile Alliance, to serve the Agile community, must be seen to "eat its own dog food". So I have been arguing since the beginning of this year, partly as a result of becoming closely involved in the organization of the Agile 2007 conference. Becoming a candidate is a way for me to do something about it, other than just complain. ;)

There are two concrete objectives that I intend to pursue as a priority. The processes used to develop software that support AA activities should be Agile processes, and the resulting systems bug-free, maintainable, and pleasant to use for all involved. The selection of conference tutorials should be such that tutorial presenters are given several occasions for feedback and improvement, rather than just one known as "The Submission Deadline". Those are the "stories" I want to sign up for, enough to start with. When they are completed, I'll look for more to do, or step down.

I would appreciate your support. If you approve of my intentions, whether you are a member of the Agile Alliance or not, you can help by linking to this entry in your blog, or otherwise saying you support me. If you are a member, you can vote for me at the Agile2007 conference; if you are not attending the conference, you can vote by email. If you are not a member, you can sign up.

Morendil - default - eight comments / eleven trackbacks - § ¶

30 June 07 - 14:57Please influence the Agile community

How ?

By nominating someone for the Gordon Pask award. You can get the official description of the award from Brian's blog; I'll tell you what it means to me.

I was awarded the Gordon Pask last year, at the Agile 2006 conference. It was a weird moment. I was stressed out and tired from presenting too many things. I made a little speech. I started by thanking the people who had welcomed me, so to speak, into the Agile community back in 2002 at the XP conference in Sardinia. Many were in the room. Then I said more or less the following: "It's always hard explaining to my neighbours, to folks I met at parties, and so on, what this Agile thing is that takes up most of my time and has become a full-time job. Lately I ask people if they've used a computer; I ask them if they've been frustrated and angry with it. Usually they say yes. Then I say that we're working to improve the software profession, so that these things eventually become history. That seems to be a pretty good explanation, judging by the encouraging words I get now. For me this is what Agile is primarily about: working to become a profession that people respect, like doctors, firemen or astronauts." Well, that was my speech. In retrospect I might want to add that it takes more than software developers to build software, so there are many roles in our profession for which I would also like more recognition as professionals. Testers, "customers" (product owners, if you prefer), and so on.

The Gordon Pask award is one way for the Agile community to say, "Listen to what these people say Agile is about, and look at what they do about it." It is intended for people who don't already have "star" status, like Kent Beck; for people who are doing things within Agile that seem to matter and who we'd like to have more recognition, so that they will matter a bit more.

You can help define what Agile stands for: look around you at who is doing stuff that you think moves Agile in the right direction, and nominate them for the Gordon Pask award. Just send an email to pask-nominations(at)agilealliance.org .

Morendil - default - No comments / four trackbacks - § ¶

29 June 07 - 14:56One of the best ways to invent the future...

...is to attend a conference. Here is one that just appeared on my radar - and for reasons that will become clear shortly, I'm mightily tempted not just to attend, but to present something.

The best conferences are just that: invitations to shape the future. Back in 2002 when I attended the XP conference in Sardinia, it felt so invigorating, exciting, mind-expanding it felt to be surrounded by people who had a vision of how the world - or some little part of it - could be made a righter place to live. (Another of my favorite quotes: "The greatest obstacle to transforming the world is that we lack the clarity and imagination to conceive that it could be different. -- Roberto Mangabeira Unger")

How could I not be interested in a conference presented as follows: "Computers, networks, and other forms of technology are now pervasive in our information-based society. However, most users still function as passive consumers of technology. To evolve into a true knowledge society, it is critical that we transform computer-based human activities to engage users in the active process of creating, connecting, and collaborating together."

The C5 conference will organized in january 2008 in Poitiers, France by Smalltalker extraordinaire and fellow Dojo enthusiast Serge Stinckwich. When I Googled previous editions of the conference, a number of things became clear.

First and foremost, it appears that it might be an occasion to meet Alan Kay in person. It's not certain yet, but that alone would make the trip worthwhile - for an OO enthusiast, meeting the person who invented OO is too good to pass up.

Next, the conference, which originally had a major focus on Squeak and Croquet, is opening up to the full scope implied by its name (C5 is an acronym for "Conference on Creating, Connecting and Collaborating through Computing"). Serge has confirmed that XP/Agile, with its emphasis on shared responsibility for the design of software systems and customer collaboration, could be a fit topic for the conference. And it seems only logical to want to be more than a "passive consumer" at such a conference...

Morendil - default - 17 comments / eleven trackbacks - § ¶

05 June 07 - 15:34"Just a mistake" ?

Two follow-up observations on my previous post about "four types of process errors".

One is to stress again the difference between (what I think are) the relevant attitudes toward process, as per my previous post, and how much formality accompanies process. The two dimensions are quite orthogonal. If you are very formal, you will define in painstaking detail the Inputs, Transformations, Outputs, Resources and so on, that constitute your process. If you are informal, you'll just say things like "our process involves shipping every two iterations, three if it's a big chunk of functionality". Just how much is to be gained by being more or less formal is still a mystery. To be Agile means, usually, to take a clear position that formalism doesn't gain a whole lot. Still it's interesting to note that people who label themselves Agile tend to vary quite a lot as to how much formalism they like.

One thing you gain by having some formalism is traceability, as in: the ability to resolve disputes by looking at something objective (a written statement or a diagram) when people disagree about what the process is supposed to be. (That distinction is still important. If it's written down, but people do not generally do it, it's not their process.) And as in: looking at the changes that were made to the process over time. And so on.

What is clear is that if you have too much formalism, it can be as much a hindrance to traceability as having too little. If it's written down, but people do not generally read it, look at it, refer to it, it's not their process.

Another thing that came to mind was an objection: "Maybe sometimes a mistake is just a mistake". Say we make a mistake and don't change the process - are we necessarily making a Type III error ? Suppose you've forgotten to pack your razor before going on a trip; it happened just this once, never before. And if you never had it before, you're not necessarily going to decide that from now on, you'll use a "check list" of things to pack, which includes your razor. Perhaps for many people that would be a Type II error: going overboard. On the other hand, before your next trip you're probably going to think twice about your razor before you're done packing. "I remember I forgot it last time". Well, remembering is maybe the simplest type of process there is. You did change your process. So, forgetting the razor would be a Type IV error.

Morendil - default - five comments / six trackbacks - § ¶

04 June 07 - 14:47Four types of process errors

Part of the Agile Manifesto reads: "...we have come to value Individuals and interactions over processes and tools". This has often been paraphrased as "people over process", a phrase which has generated a lot of debate, not all of it very productive.

One reason for that is that everyone knows the meaning of the word "people", but not everyone - far from it - understands the meaning of the word "process". I'm still not sure I do (especially after looking at the various pages of that name on Wikipedia).

In the context of software development, it is used with varying degrees of formality to mean "the generally understood distribution of responsibilites among the various roles involved in getting software out the door". One source of much confusion is in the "generally understood" part: what I usually need to clarify at the start of any discussion of "process" is that people use "process" both to mean any of the following, depending on context:

• "what we're supposed to do, and actually do"


• "what we're supposed to do, but don't actually do"


• "what we actually do, regardless of what we're supposed to"

Of course, what actually matters - what is worth discussing - is what people actually do. A 10-page process document or a flowchart are nice, but generally irrelevant unless they match very closely with what people actually do in the pursuit of shipping software.

Now to the main point of this post - which is still somewhat fuzzy in my mind, but which concerns the way in which mistakes resulting from a process are corrected. Along the same lines as the "three types of errors" known in statistics, I think processes (or maybe process cultures) can be classified in different categories according to how people are seen to handle defects (problems, incidents, mistakes, crises, etc.)

• A Type I error (rigidity) is when you follow the process, produce a bad outcome, but you're not allowed to change the process


• A Type II error (bureaucracy) is when you follow the process, produce a bad outcome, and the process is changed by the addition of an exception that only addresses repetitions of that exact same mistake
• 
  
• A Type III error (hacking) is when you follow the process, produce a bad outcome, and do nothing about the process


• A Type IV error (benign) is when you follow the process, produce a bad outcome, and correct the process so it produces better outcomes overall

You see errors of Type II all the time in attempts to regulate health hazards. For instance, one person dies after doing stupid on an amusement park ride, and that results in new legislation mandating foolproof belt buckles which can only be undone when the ride is stopped. (Until there is a fire trapping dozens of people on a park ride, at which time...)

I'm often concerned that an overly enthusiastic interpretation of "people over process" could lead to Type II errors. "Well, our iterations are supposed to be two weeks long, but Bob is on vacation for a week so it seems to make sense to use three weeks this once." Sensible, but if you keep making exceptions to your process, pretty soon you're going to be in Type III territory: getting bad outcomes but not bothering to fix the "process" any longer, because no one remembers any longer what the "process" is. On the other hand if you treat the process as "sacred" you'll end up in Type I territory.

Errors of types I through III are not just errors, they are meta-errors. They are defects arising from the process that is in place for modifying your (software) process. If you're lucky, your meta process has only Type IV errors: people recognize that they are too rigid, too procedural, or too indifferent, and adapt accordingly.

More useful than calling for "people over process" is, I think, characterizing one's process according to which of the above four types of error it suffers from, and recognizing the possibility of going meta.

Morendil - default - 33 comments / four trackbacks - § ¶

31 May 07 - 13:23Nil nisi bonum

"Nil nisi bonum" is the abbreviated form of a Latin saying, sometimes translated as: "Do not speak badly of those who are dead".

The funny thing is that this translation is itself a violation of a broader "nil nisi bonum" principle, which could be stated as: "focus on the positive". The Latin saying is perhaps better translated as: "Speak kindly of the dead (or not at all)". The parenthetical clause is redundant, we can leave it out altogether.

"Speak kindly" is a lot more efficient than "do not speak badly". There are many ways one could speak of the dead, other than badly: with indifference; in detail; in outline; in prose, in poetry; and so on. This is true of more than just dead people. In general, the things we don't like, don't want, won't tolerate vastly outnumber the things we do like, do want, do find acceptable. To enumerate them all would be a huge waste of time. And to name, for instance, only one thing we don't like - that is just getting started on the enumeration.

So, we communicate more efficiently by saying what we like, what we want, what we prefer, how we'd change things for the better, and so on.

I've been given this advice before by several people, in different forms. For instance, in a conversation with a friend about where I wanted my consulting career to lead, I'd started one explanation with something like "I don't want to be an employee, I really don't like the situation this puts me in", and so on. The reply was "What is it that you *do* want ?" Early on I resisted this advice. To me it smacked of "positive thinking", that is, a mostly motivational thing, like a manager announcing a layoff by saying "the good news is that we'll manage to keep a few people on".

After a while and some practice, it did start to sink in. The point is not to be "positive", as in always all nice and happy, rather to be "constructive", even when a situation is clearly bad. In the previous paragraph the manager's statement is positive, but also besides the point. The positive statements I'd want to hear in such a situation are things like: "Here is how we plan to do right by those of you who are going to be fired."

In discussing a software design, you might come out with a first statement: "I don't like this design." Better, more effective, is to say "This and that is wrong with this design." Better still is to say "I can see where you're coming from with this design, and it does solve the stated problem; we can make it more abstract by renaming this bit, more encapsulated by moving that method, more cohesive by segregating these few classes in a separate cluster." Or more briefly: "Yah, that will do the job, only make it more MVC, y'know ?"

The disturbing thing is that it takes other people to listen to what you're saying and point out negative statements, then coax out of you a formulation that actually adds something to the conversation, that moves it forward. Without an observer I was quite unaware of the way I framed my observations and attempts at solutions. With some help it turns out to make a difference, in terms of being more effective at giving people critical feedback, finding solutions in sticky situations, and so on. And after a while you internalize the habit, and "catch" negative formulations before they're out. (Oh, so you censor yourself, you might ask; how can that be good. Well, no more so than if you have a habit of choosing your words carefully before speaking - of chewing a bit over the first thing that comes to your mind - a good thing, yes ?)

Listening is such a great habit in general. Listening to others, listening to yourself. More than a habit; more accurately, "listening" is an umbrella term for a whole array of techniques and models; a subdivision of an even larger umbrella term, "paying attention". Techniques that turn out to be very, very effective... and very, very hard to acquire. What makes them hard is that resolve is not enough. You can wake up in the morning, determined to listen more closely. To actually do it, though, you need specific techniques and models, and it's usually other people who provide them; if you could come up with them on your own, you already would have.

Perhaps that's more a statement about me than a general truth; there might be people who are better at introspection than I am, and who do "pull themselves by their own bootstraps" in this regard. Me, I get by with a little help from my friends... (You know who you are; thanks !)

Morendil - default - fourteen comments / six trackbacks - § ¶