12 June 09 - 18:16Coke machine projects

"Coke machine" is my tag for a category of projects. I can't believe I haven't blogged this before.

The joke goes like this - this blonde walks up to a coke machine, puts in fifty cents, gets a coke. She does a little dance, grabs the can, puts it on the counter nearby. Opens her purse, puts in another fifty cents, gets a coke. Puts the can away, starts all over again. Meanwhile as it's a busy time of day, a small queue is forming behind her. People want to have a go and start to get equally annoyed and amused. "Lady, when do you think you'll be done here ?" And she answers, of course: "Hell, buster, as long as I keep winning I intend to keep playing !"

(I know it's a sexist joke told like that. The first time I heard it, in French, it was about a Belgian. The geek way to tell it would be "this [insert favorite stupid stereotype] walks up to a coke machine...".)

Anyway, I've come across projects which are exactly like that. The business keeps paying, and as long as the business keeps paying the developers keep developing.

In these situations, you won't need the exact same scheduling practices you would use in a project where "predicting the end date" is an important concern, or even knowing what will be in the product by a certain date. For instance, I'm OK if this kind of project keeps a small "planning horizon" (the number of iterations which the team is capable of planning ahead), such as one iteration. Or uses a continuous scheduling tactic, such as kanban.

As the name implies I feel vaguely uncomfortable about this kind of "project". For one thing, it contradicts the common wisdom that projects are well-defined endeavors with limited resources. I like it better when even long-running efforts are cut up into major milestones which have the feel of a distinct "project". If nothing else, that gives the team defined opportunities to create "generations" of their product, opportunities to cut loose from a design grown too old for instance. That's a topic for another blog...

- default - two comments / No trackbacks - § ¶

05 May 09 - 15:19Asking for help with AgileAlliance.org

As the Internet Archive shows, the Agile Alliance's web site has a long history of embracing change. This is understandable, I suppose, given the growth of the Agile community itself and the need to keep up with these changes.

The most recent version has been around since late 2006. (I joined the Agile Alliance board in mid-2007.)

We've been discussing how to serve members of the Agile Alliance better for over a year now. In order to better serve our members better we figured we needed to know more about them, and generally provide people with a smooth experience as members. We also want to help members connect with each other and support each other (the fashionable term for which is now "social networking").

Our current web site doesn't do enough for those purposes, so we've decided to invest in improvements. We've written up our current ideas of our requirements, in two parts)

• here (managing the membership process) and
• here (providing "social networking" features)

There are two ways you can help.

First, whether or not you are a member, we'd appreciate feedback on the above list of features. Maybe there are some we haven't thought of, and maybe there are items you'd appreciate our adding.

Second, you can get us in touch with vendors or open source projects which you think would be a good fit for the above purposes.

Our reasoning at the moment is that we would prefer SaaS offerings, because the job of the Agile Alliance board is to think up ways to support the agile community, not to host or develop Web sites. And we think theses services - interacting with members and helping them connect with each other - are basic enough that it's likely there's a better implementation out there than we could get by funding a new project or spending time tinkering with servers. (We're open to changing our minds about that.)

We definitely wish to retain the possibility to develop further services on top of such a platform. We are also aware of the possibility of calling on volunteers: after all, we serve a community of top-notch software professionals. The chicken-and-egg issue is that, to effectively organize those of our members who'd like to serve as volunteers, we first need to reach out to our members in a more effective way!

Ready to help ? If so, please contact me, through comments or by email.

- default - No comments / No trackbacks - § ¶

05 April 09 - 11:45Looking for conscious competence

This is sort of a survey. Assuming there was a ranking system in software similar to that in Go, from 30kyu to 9dan (or ELO, from 1 to 2800), what would you guess your level is ? Why ?

I know, difficult question. Here is one that may prove even harder. What is it about software that, for you, represents the hardest "learning barrier" ? Some aspect of the craft, some topic, that you find particularly difficult to wrap your head around ?

- default - one comment / No trackbacks - § ¶

04 April 09 - 12:37Of processes, babies and batwhater

Do you have issues with the word "process" ?

There is a temptation, which is to think of "process" as meaning "a set of instructions which, if people would only follow them to the letter, would result in reliably complete some kind of work in the minimum time and with highest quality". That was Frederick Taylor's insight, and you can't entirely blame some people for wanting to apply it to software. After all, it worked (or seemed to work) for reliably completing high quality cars on time.

Well, it turns out that Taylor's ideas do not, in fact, transpose well to software. Or at least, not given the current state of our technical knowledge on software; I'm pretty sure that not everyone has given up on the idea of making the programmer a kind of factory worker. And I can understand Naresh's association between Taylorism and the word 'process', and coming to hate the word as a result.

Where I balk is when Naresh suggests banning the word itself, and suggests that one would want "zero process". That strikes me as a case of throwing the baby out with the bathwater.

The notion of "process" is quite useful. "What is your software process" is a valuable question to ask, for instance you'll see it asked here on Stackoverflow to good effect. Why would you want to deprive yourself of a word that is such a compact shorthand for the set of questions it implies ?

A question about process is basically a question about regularities. "Suppose one of your users runs into trouble with your software. What do they typically do ?"

This could have different answers. "Well, half of the time they call up one of the developers on the phone, and the developers fix it right away. And half of the time they'll send email instead, to the lead developer,  who forwards them to one of the devs for a fix." Or you could hear "They file a trouble report using our ticketing system, it goes to a support person for investigation and qualification, possibly as a defect, and if it is a defect QA will write a test exposing the bug if they can repro, then after a fix is found by development we'll either issue a patch or it gets into the next release."

These answers have something in common. They describe a process. You can ask some follow-up questions: how is that working out for the users ? Are they happy with how their reports are handled ? How is it working out for the developers ? What (if anything) would you like to improve ? Do you think you might shorten fix times if you agreed to do X instead of Y ? These questions could even be asked by team members themselves, in a retrospective, taking a process perspective on improving their results.

Of course you could hear other kinds of answer: "I don't know. I haven't even thought about users having trouble with our software." Or something like "I trust them to do whatever is appropriate." Or you could hear (at length) about Bob, the user in Accounts Receivable who calls at inappropriate times, yells at the developers and is generally a jerk. All these answers are interesting, and they are not about process.

Even Naresh, while calling for "zero process", is concerned with describing regularities:

Personally I’ve been executing projects quite differently. When I think about the various things we are doing, they don’t quite fit what the books or the standard training courses are talking about. In fact in some cases it contradicts them. Interestingly, I see a few folks executing projects similar to my style. There is certainly some common patterns out there. [...] How do we package these evolved concepts and patterns? Just so that I can differentiate it from the rest, I prefer calling it “Naked Agile”.

...and starts making the exact same mistake that he's so worried about. That is, thinking that a process description obtained in one team, one project, can always be turned into a prescription for other people to follow:

Then the question comes, do new comers really have to go through the same evolution process to understand and appreciate Agile? Or can they skip some of ancient practices & concepts and jump start with what we collectively believe is the most suitable now?

Certainly that's a good question. But, if it is a good question to ask about the "ancient" practices and concepts, why not ask it also about your "current" practices and concepts ? What is it that makes any process description something suitable for anyone to "jump start with" ?

This is an open question, and a difficult one. I'll say more about it later - lately I've been writing about it, and speaking at conferences about it. But the observation for today is that it would be hellishly difficult to examine that question if we deprived ourselves of the useful word "process".

- default - No comments / No trackbacks - § ¶

02 April 09 - 08:53WeVouchFor

At present WeVouchFor is a zombie project. It's definitely not dead, lots of people are signing up. It's also definitely not alive, no one is actively working on it.

People are signing up because there's a definite need. Recently one more "rip off" certification was announced to the world, and once more the people in the agile community who are sincere agents of change responded to that by offering alternatives - and specifically by pointing to WeVouchFor. This blip in the project's popularity is both encouraging to me, because it suggests that the idea of "peer certification" may be workable, and a little embarrassing - because I have more or less abandoned the project. I'm wondering how to revive it, or what to do about it generally.

What suggestions do you have ?

- default - two comments / No trackbacks - § ¶

21 March 09 - 12:19Quality, Safari and Firefox

Lately I've been mulling over James Bach's recent post on "The Quality Creation Myth". It's well worth reading; I find James' posts often thought-provoking, though occasionally disappointing, as when bashing test automation more than seems reasonable. But this one really hits some important points.

One of the thoughts I had upon reading it was how the quality of an end user's experience depends on factors that, at least in my experience, changed dramatically over the years. In the DOS days I used to write (and maintain) software that took over a user's entire screen. Along came the Mac, and I learned to share windows and menus with other applications; I gained appreciation for well-thought-out frameworks that made such interactions not only possible but orderly and harmonious. There was an period of "regression" when I wrote CD-Rom apps; we went back to taking over the whole screen, though this time around we were no longer coding the smallest detail of each application but starting to rely on large and complex libraries of graphical components. Then I started developing for the Web; my applications were not even sharing space on the user's computer, but pieces of a much larger system stored elsewhere that users could summon on demand.

To think that "quality" can be approached exactly the same way in all those situations would be absurd. It would be the same as saying that one person on a desert island, one small village, or a huge nation-state can all be served by the same kind of governance and social structure. I use this image because that's the trend I perceive in my description above of what software has become: from individual efforts to larger and larger "coalitions".

I was reminded of this on the occasion of switching (experimentally) from Safari to Firefox, because it's hard to comment on the impressions of "quality" I got from either browser without speaking about what feels like political decisions.

Apple has made Safari a closed enclave; no extensions allowed in. This allows its designers to pick certain strategies toward quality. Firefox on the other hand is designed as an open community; I'm typing this post in FireScribe, an add-on for writing blog posts inside the browser. On the one hand this makes for an experience that is better in some respects. On the other hand, this creates far too many opportunities for these various "features" to interact with one another and generate moments of breakdown for the user. For instance, keyboard shortcuts for editing text don't work in this window; Command-Left-Arrow (which I normally use to back up the cursor by one word) is taken over for navigating to the previously viewed page.

These observations are quite in line with James' post insofar as they point to "quality" as a complex relationship. What I think needs to be added to his analysis is this view of pieces of software as individual agents which can assemble in small or large coalitions, and that this is an important determinant of "quality".

- default - one comment / No trackbacks - § ¶

12 November 07 - 14:13Breaking Acts at Agile 2008

Preparations are starting for Agile 2008. And you can help. The short story is that we're looking for speakers; we'd also like to solicit reviewers. Read on for the details.

In the run-up to Agile 2007 I shared with Elisabeth Hendrickson the post of Tutorials Chair. This means we were responsible for one specific type of session, irrespective of topic: tutorials, which are meant to convey "mainstream agile" content and attract the more polished speakers. We struggled with the scale of the conference - 1200 attendees in 2007 and 1600 expected next year. The number of submissions was quite large - about 600 all told, about 200 in tutorials alone. In reviews we found it hard to give all tutorial submissions, covering as broad a range of topics as they did, the attention each deserved.

Rachel Davies is chairing the 2008 conference. She came up with a metaphor to guide the design of the conference - one that has the advantage of framing this big conference as a cluster of smaller conferences, with the team in charge of each smaller conference free to focus on a topic of interest to them. So, Agile 2008 is organized along the lines of a "music festival", with various Stages; each Stage has a theme and a Producer who tries to put together a great agile conference around that theme. I think it's pretty nifty. Since it's new, there will inevitably be some few hiccups as we get used to it, but I am optimistic it can help reduce the "big conference blues" we experienced in the past two years. There's a stage dedicated to Agile curmudgeons, for instance - I'm thrilled that we're having that. There's one on examples, one on the user experience, one on customers and business value, and so on.

I'm happy to announce that I will be producer for the "Breaking Acts" stage. Quoting from the CFP: "Agile as it stands today is still a work in progress. For Agile software development to remain relevant, it must incorporate new ideas continuously. This stage is for speakers who bring a fresh and surprising look to aspects of Agile we thought familiar, and speakers interested in ideas that are relevant to Agile but not accepted yet as 'mainstream'. First-time speakers are particularly welcome."

If this kind of thing rocks your boat, I would appreciate your helping in one of two ways. Right now, if you'd like to be a reviewer for session proposals, please get in touch with me; I'll give you a quick overview of what to expect and what's expected of you. In a few weeks, if you're interested in speaking at the conference, we'll be opening up our (new) submission system to session submissions. I'll post the details here (if I remember; otherwise, they'll also be on the conference site).

- default - No comments / No trackbacks - § ¶

06 October 07 - 15:13Principles and patterns of codebase structure ?

Let's say you just had a nifty idea for a software product. Could be a Web site, an IM client, a new Ruby gem, anything. And let's say you've done enough up-front work to suit your purposes and your process preferences. So it's time for you to sit down and prepare a directory tree for the initial import into Subversion or CVS.

What are the rules for doing that ? And what are the rules that govern ongoing elaboration of this directory tree ? I'm not aware that these are written up anywhere (if you know of such a write-up, I'd appreciate a pointer).

Most competent programmers I see sketching a directory tree seem to be doing "the same thing". They'll start at the top level with, say, "src", "test", "doc", "lib" directories. But why ? If their intent is something more complex than a simple app they'll create additional divisions at the top level, or below, which make sense for that intent. These programmers don't seem to be following explicit guidelines or principles or known patterns, and sometimes I find codebases whose authors tried to imitate this "standard" structure in ways that just don't make sense. So, making the principles and patterns explicit could be valuable. I'm thinking that one good way to make those principles explicit would be to look at "ugly" codebases, identify the violations we see that make them "ugly", and reason backwards from that to the "proper" principles.

Based on experience with violations, I'm postulating some "principles of transparency", which suggest things that codebase structure should not reflect:

• A principle of transparency with respect to team organization - the codebase should not reflect the structure of the team. I was quite shocked when one colleague reported to me that he had seen a team using Subversion where the top level of the code base was a set of directories, one for each member of the team. This suggests that the team members "integrate" by manually copying code from their "private" directories to one that is authoritative.


• A principle of transparency with respect to change over time - the codebase should not reflect version evolution. It's always unpleasant (but not quite in the "shocking" category) to come across a directory labeled "old" or "version_before_ajax" or some such. It's the job of version control to track those things.

Primarily I think of the contents of the repository as a "blueprint" for creating a running system. The repository doesn't have to be organized along the same lines as the running system will be organized. (I even debated adding that to the list of "transparency" principles above, but I believe it can actually make sense e.g. to have a top-level division into "client" and "server".) However, the repository does have to contain everything you need to know in order to produce a running system. The repository also has to contain everything you need to know in order to advance to a new version of the system - to fix a bug or add a feature. A memorable formulation could be "For maintenance you need the repository, all the repository and nothing but the repository".

The decomposition of a system into "products" (in the sense of that post by Reg Braithwaite) plays an important role, I suspect, in structuring the code base. Independent "products" should live in separate repositories; however some systems might depend on very closely related "products" (again the example of client and server comes to mind) sharing some common code.

For a single-product repository, the "description of how to produce the end result" theory makes good sense of why many projects have "src", "doc", "lib", "test" and a makefile at the top level. Directly at the top level is what we'll be concerned with - "building" or "deploying" or "installing" a running system from the repository. (Or sometimes a "shippable" rather than a "running" system; a level of indirection is introduced by having the repository be a recipe for building an installation CD and the running system only results from the end user running the installer.) The top-level groupings are according to the function that the various elements play. The "doc" directory is where you go to understand the code, which you then find in "src", after which you can verify your changes by running the tests in "test". In the "lib" directory there are components which are needed to build the system but which you don't need to know, as they are unlikely to require any changes. For multi-product repositories, we can expect that the top level will have a directory for each product, under which the same structure will be found as for a single-product repository.

Do these observations make sense ? Did you come across codebases whose structure you found perfectly clear, or perfectly bewildering, and for what reasons ?

- default - No comments / No trackbacks - § ¶