29 October 03 - 15:40Team-based object languages

I recently attended a mini-conference on the topic of "object languages after 20 years". Have objects failed ? Is a new "post-object" era being ushered in, and can we guess what new ground will be broken in that era ?

The answers seem to cluster around two major and opposing axes. There are those who predict the end of Objects as an inclusive paradigm and herald the coming of age of Models. They are the tenants of MDA, model-driven architecture; of "platform-independent models", among which business process models, service models etc.

Actually I'm not quite sure that these people are at all talking about the same thing as the people who discuss object languages, but the impression is definitely that languages in general (object or otherwise) are not something of interest, and models are where all the action will be. Maybe. To my way of thinking, "model" is to "class" as "class" is to "object" : a drive to generalization, the grander and more sweeping the better.

The ideal of MDA, as stated, is to be able to describe the essence of a business in a set of models, and then automate more or less fully the generation of code (actual applications, plus the platform-specific plumbing) that allows people to perform the work. The unstated implication, if you throw in the notion of "reusable models", is that there is a sort of Platonic abstraction which every business under the sun is a somehow distorted reflection of; modeling captures the Platonic essences, and programming is mere correction, a way of adjusting for the inevitable imperfections and divergences that arise when the essence is realized in the physical world.

Several authors, Jim Coplien most recently, have warned against the error of identifying object languages with the notions of classes and inheritance. This is the second trend I see - a direction away from the classifying and Platonic school. Actually, according to Richard Gabriel this was the original object philosophy but was derailed along the way:

[...] with C++ and Java, the dynamic thinking fostered by object-oriented languages was nearly fatally assaulted by the theology of static thinking inherited from our mathematical heritage and the assumptions built into our views of computing by Charles Babbage whose factory-building worldview was dominated by omniscience and omnipotence.

Coplien goes on to argue that "type" is a less useful notion in an object design than "role". The way I think of object designs based on roles and responsibilities is by analogy with organizational concerns. If you are building a project team, you can proceed in two different ways to select the next person you hire. You can be interested in whether that person "is a" Programmer, or Architect, or Tester, or Manager - and at a finer level, a Senior Programmer or Junior Programmer, an Integration Tester or an Acceptance Tester. In other words, you can take an interest in types and classifications. Or you can be concerned with making sure that your team, as a whole, includes the appropriate mix of programming skills, testing skills, management skills, etc. The next person you hire doesn't have to "be a" Programmer, but might be a good recruit because they can in part fulfill that role thanks to their programming skills, and also complement existing testing skills.

We see this kind of approach in a recent programming language, Scarlet, which does away completely with class inheritance and class hierarchies; rather, the main approach to reusing implementation is aggregation. In Scarlet you find interfaces, which are akin to roles, and implementations - which we can think of as persons with the skills required in these roles. You compose objects which are associations of implementations brought together to perform tasks - in other words, objects which look a lot like teams.

- Object Lessons - four comments / No trackbacks - § ¶

27 October 03 - 21:29Test-driven example for game programmers ?

In teaching, examples are everything. In this morning's practical for the class I'm taking on distributed/concurrent programming, I worked on implementing a simple Web server. This kind of example works: it's obvious why such a piece of software might be useful, but it's non-trivial and raises interesting issues directly related to the content of the class.

Of course I did the example test-first. It should have been easy; it wasn't. Things went smoothly for the first hour, but I started feeling the time pressure shortly into the second hour when I still hadn't gotten around to servicing a trivial GET request. I ran into intermittent test failures with ServerSockets raising "already listening on this port" exceptions that I couldn't figure out, and sort of lost control from then on. The problem with this kind of context is that recovery isn't an option: at work or at home, I could just take a break for coffee, or work on something else for a while and let the back of my brain work on the problems.

The exercise also got me thinking about choosing examples to teach test-driven design. I've taught a TDD workshop with Joseph Pelrine, who uses a domain-oriented example from the wonderful world of insurance products. I've also taught one of my own to developers involved with Web-ish stuff. In that workshop, the example I use is a simple tabular display of a collection of objects, rather like the popular displaytag component.

I'm currently preparing a longer TDD workshop, intended for programmers in the game industry. That's a different context from what I'm used to: much less exposure to the notions of object languages and object design, more familiarity with C and assembler. Much more concern with issues of hardware compatibility, less for standards or "component-oriented" development. My current theory is that the tabular display example wouldn't be a suitable learning situation there.

Any game developers out there ? What sort of programming problem do you think would work for demonstrating TDD ? It has to be small enough that measurable progress can be made, test after test, toward a goal which is obviously valuable; yet it should also bring into play some interesting problems which a games developer is likely to have encounter frequently.

- Extreme Programming - four comments / No trackbacks - § ¶

26 October 03 - 16:38Java exam - I'm a failure...

The class I'm taking includes some Java-related lectures and practicals. It's quite possible I will learn new things about Java, but I'm hoping to learn more about something rather elusive: how it feels to be a student. (The theory is, that will make me a better teacher or trainer.)

I remembered a mail from a friend who's been learning Java, about a site which provides you with sample questions from Sun's Java exam, and figured that would be a fun thing to try.

My conclusion: too much knowledge may actually drive your score down on exams of this type. The question of the day:

Which of the following statements are true?
1. An interface can only contain method and not variables
2. Interfaces cannot have constructors
3. A class may extend only one other class and implement only one interface
4. Interfaces are the Java approach to addressing its lack of multiple inheritance, but require implementing classes to create the functionality of the Interfaces.

I answered 1/2/4. That seems to be "wrong": syntactically, it is perfectly allowable to write the following

public interface Foo {
  public int bar = 5;
}

It looks like an instance var, it smells like an instance var - that is, you don't need to declare it "static" or "final". But the compiler does that for you : behind the scenes it declares the variable "static" (has only one value across all instances of Foo) and "final" (can only be assigned once).

Me, I call this kind of thing a "constant", not a "variable".

- Object Lessons - two comments / No trackbacks - § ¶

24 October 03 - 15:29On becoming a student again

There is a sort of karmic symmetry in what I've been doing over the past couple months or so.

Today, I became a student again after a twelve-year career in software. It is perhaps arbitrary to select today, the day I attended my first lecture, as a turning point in the process; but I have good reasons to do so.

After this summer's gig consulting with the QA people on a fledgling XP team, I was looking for new opportunities in several areas. One of the things I want to do more of is training/teaching in various areas of XP and agile methods. Demand for professional (on-the-job) training earlier this year and over the summer was low to nonexistant. Then it occurred to me to inquire about opportunities teaching engineering students. The problem, I was told, is that my lack of a degree would probably prevent me from applying successfully. (I removed myself from the academic circuit two years after my high school diploma and without a further degree, to start my career in software.)

I decided to take advantage of a law recently passed here in France, under which professionals with significant experience can now apply to be awarded a degree in recognition of that experience. It's been a slow and tortuous process so far, and still underway, and might still not pan out (i.e. not end up with me having a master's in software engineering by the end of the year). But as a part of that process, I'm currently attending one class in Distributed Systems.

Strangely enough, almost simultaneously with that, three distinct opportunities in that area popped up: one XP training gig, one training company which responded to my offer to do professional training in Java and object design on a freelance basis, and one lecture on XP... at the same university I am now a student of.

It somehow feels fitting to be simultaneously put in the position of student and that of teacher.

- The Universe And Everything - one comment / No trackbacks - § ¶

16 October 03 - 12:06Subsidization, Open Source and Baumol's Disease

As Jim Bullock puts it, software projects are different, but not that different. Here is one way in which software, or indeed knowledge work in general, is not different from other stuff: money is often involved. But do we really understand the economics of software projects ? How do we use economics models ?

Consider a simple distinction, "straight" commercial exchanges vs. subsidization. In the former, one actor in an economic situation provides some kind of value to another, in exchange for money. You make shoes, and people give you money to buy shoes from you. Buyers benefit because you specialize in shoes, and the ones you make are better than the buyers could make for themselves. You benefit by earning money, a generalized means of participating in the rest of the economy. That's a simple model - one step up from barter - and it works. Often it works well, sometimes less well.

More complex systems work, too. You specialize in shoes in the expectation that you will earn a living in return. But it doesn't have to be shoe buyers who provide your compensation. In a subsidized scheme, someone else has been collecting money from people, and gives you that money for making shoes, while you distribue shoes for free to people who need them.

That system also works - and it also has its shortcomings. You need to control more of the whole system, to ensure people don't "defect" by taking advantage of the freebies but defaulting on their payment to the collecting agency. (A problem known as the Tragedy of the Commons.) But controlling the whole system means that there are opportunities for more efficiency.

The relatively simple distinction between commerce and subsidization has subtle consequences, as when one situation should be using one system but is using the other, or when they're being confused for one another, or when we have trouble going from one to the other. I've picked up a few examples to illustrate.

In a recent blog entry, Philip Greenspun deplored the lock-in on the "commercial" system for publishing scientific documents, which he argues ought to be subsidized. There would be a clear advantage in putting research of all kinds on the Web: if you had access to everyone's research, you would avoid duplicating work done by others, and you'd devote more time to your own innovation, or the refinement of others' work. But we're stuck with a commercial system.

As a case of not identifying the system correctly, there's this paper on Open Source software, which argues that what people have interpreted as a "cultural revolution" is a straightforward matter of subsidization economics. People produced software for free, not out of idealism, but because money flowed into their pockets by other routes.

Another case I remember is when I was trying to start an Internet business, back in 1995-1996. The investors and potential partners I talked to at the time laughed me away because there was "no way to make money" off the Internet, in which all information was free for everyone, in a market dominated at the time by the French Minitel system, which collected per-minute connection charges paid (minus a commission) to content and service providers.

Then there's the reverse of the problem Greenspun describes - you have a subsidization deal which is hurting rather than helping. "Cross subsidy" is a well known problem in accounting, where a profitable operation invisibly provides money to an operation which is making losses.

Or consider a related model, "Baumol's cost disease": industries where productivity remains constant will have increasing costs, because the industries where rising productivity is keeping costs down can raise wages to attract workers. To keep costs down, flat-productivity industries may have to lower quality.

Perhaps this is even directly applicable to software development. Consider the productivity of programming compared to that of testing/debugging (the old-fashioned kind). Today it is possible to write code in one hour that does a lot more than one hour's worth of coding thirty years ago: there's huge amounts of function in the OS, in the core class libraries, etc. But the productivity of testing remains constant: there's only so much complexity you can poke and prod given one hour of debugging. So the trend would be toward testing becoming ever more costly compared to development, leading to less testing being done in order to keep costs down, leading to more quality problems.

- The Universe And Everything - No comments / No trackbacks - § ¶

14 October 03 - 15:02The Ancient Art of Architecture

I've always accepted the etymology given for "architect", from Greek roots meaning "chief builder". After reading the chapter on architectural projects in Anthropologie du Projet, I've changed my views somewhat.

The second root is uncontroversial. I was pleasantly surprised to learn of its humble origins in a root which meant "weaving" - humility is good for one who would call himself a "builder", let alone "chief". It is present in the word "technical" or "technique".

The first part of the word is from "arkhein", which means "to rule" but also "to begin", from which words like "archaic" or "archaeology" are derived.

Even if that isn't the correct or most probable etymology, we can thus understand architecture as "the first art". Hegel viewed archicture as primitive, logically as well as historically prior to other arts.

The book brought not only this different perspective, but also two observations on building architecture, which to me raise further questions regarding the applicability of the term to software. The observations: architecture is the art of structuring empty space; but this empty space is intended to be inhabited by people. Also, architecture as a spatial art is peculiar in being ruled by the vertical dimension; its overriding concern, the force that defines architecture, is gravity.

The questions: what in software corresponds to the "empty space" which building architecture deals with ? How do we "inhabit" software ? And what is the overriding force that rules the discipline of software development ?

- Software Development - one comment / No trackbacks - § ¶

12 October 03 - 12:54Skills obsolescence

Among my colleagues it's a fairly common experience to recommend a book to a higher-up, only to be told that he or she is "too busy" to read it. Many of the software management books I've read over the past few years, and thought exceptional enough to lend them to my superiors, were returned to me unread - or turned down outright.

Many countries are considering or passing (or already have) laws to make it mandatory for elderly drivers to take refresher courses, or pass the driving exam every few years, or give up driving altogether.

Now driving is a rather straightforward mechanical/reactive skill; more abstract skills and knowledge should age at a much faster rate, and especially so in an industry as volatile as the software industry. How old is your manager's highest degree ? Your manager's manager's ? People in positions of power are fewer than elderly drivers, and the damage they can inflict to others falls well short of life-threatening, but it is also much more far-reaching.

If skills obsolescence is dangerous in drivers, knowledge obsolescence is certainly dangerous in managers.

So read the damn books, please.

- Management - one comment / No trackbacks - § ¶

08 October 03 - 18:31Paradigms lost: debugging code

I've already written a little about another of my experiences with fruitful paradigm-destruction: the switch to test-driven development.

It is rather hard to describe what the switch was from - to characterize the way I used to program before I became aware of TDD. It would probably be fair to call it "code'n'fix", if that wasn't completely misleading as to my attitudes toward the work.

I started out doing software maintenance, altering other people's code - often with just the code to go on. Then I moved on to writing software for technologies (multimedia stuff, then Web stuff) which I barely knew and had to feel my way around. Methods existed to produce code that was demonstrably, if not provably, correct. I was even aware of one such from the very beginning of my career back in 1988 - Meyer's Design by Contract.

These methods seemed to be no good fit for the programs I was working on, and the technology available to me. DbC doesn't look promising when your next few problems involve unclosed HTML tags or cryptic errors from MySQL. I'm not saying that this is a good justification for "informal" programming, but that's the way it seemed at the time.

My behaviour patterns, then, were typical of this "informal programming" - not a bad name, now that I think of it. I was aware of a need for rigor, and did spend lots of time on up-front thinking and documenting of various sorts. But I did not have the necessary tools for this rigor to go all the way to the code.

I tended to do code "sprints" of one to a few days without necessarily ensuring the code was in a stable state at any time during this period. This sometimes led to massive rework as I lost control of my increments. I tended to test my own work in an over-focused and repetitive manner.

I was quite good at debugging, and I liked it very much as an activity. It felt a bit like detective work, a bit like being a scientist.

Rather than describe again what the experience of TDD felt like after years of working that way, I'll just point you to a few other people's experiences of replacing the "informal programming" paradigm with TDD.

- Software Development - No comments / No trackbacks - § ¶

08 October 03 - 00:15Perl XP Training

Found via Advogato, where I might be tempted to visit more than once a year if I could remember my password, or the site had a "remind password" form.

Welcome to the Perl XP Training website. If you're interested in learning how ExtremeProgramming works, how to write tests in Perl, or how to contribute to multi-user programming projects, you're welcome here.

- Extreme Programming - No comments / No trackbacks - § ¶

06 October 03 - 10:42Paradigms lost: the main() function

My first programming languages were a diverse collection - some Logo, some Forth, but mostly BASIC (specifically Applesoft Basic) and Pascal. I learned them as a kid whose favorite toy was the computer.

Because Basic was, so to speak, the native language of my computer (it was burned into the Apple ][e's ROM), that's what I spent most of my time writing. Because I also got exposed to Structured Programming in the form of Pascal, however, all these GOTOs did only limited damage to my young and impressionable mind. Well, as far as I can tell. At any rate, I never felt like there was any effort involved in "moving up" from old-fashioned high-level programming to structured programming; I was too young to experience what, to many older programmers, was a painful change of paradigms.

A few short years later, when I started programming for money, I had graduated to an Apple Macintosh, and my main languages had become Pascal and C. At first I was only aware of one "normal" way to program. First I needed an understanding of how to represent the data manipulated by the program; then I would write a main() function, or the equivalent 'program begin ... end' construct in Pascal. Programming consisted of fleshing out main() by breaking the whole program into smaller operations, so that main() consisted of a sequence of calls to functions implementing each operation; and further breaking down each of these functions, and so on. Program data was for the most part stored in global variables.

I abandoned this "top down" paradigm in stages, mostly as a result of programming for the Macintosh. My first efforts were "system extension" programs, which required writing patches or replacements for selected bits of existing system codes. For instance, one program intercepted calls to the system's "beep" function and caused the screen to flash instead, silently. Later on, I wrote full-blown Mac applications, which entailed writing "event handlers" called in response to user activity rather than main() functions. All this gradually taught me to think of my programs as small pieces interacting within a larger whole, rather than as linear sequences of operations.

I lost the "top-down" paradigm fully when I discovered object languages through the first edition of "Object-Oriented Software Construction". As Bertrand Meyer says, "Real systems have no top". Even with my previous experiences, though, object systems presented me with a significant hurdle. My first forays into the terrain of object languages were all theoretical: I had read and reread Bertrand Meyer's book, but no Eiffel compiler was available for any hardware I had access to.

One thing struck me as particularly strange in object languages, though inconsequential in retrospect, was a sort of "chicken and egg" problem: if all that existed in such languages was objects, how could a program ever start, since you needed at least one object to create other objects, and your program had to start with no obects in memory ? The solution presented for Eiffel felt at once elegant and foreign, like a solution recalled in the morning to a problem experienced only in a dream: the build system for Eiffel programs used a configuration file in which a "root" class was identified, an instance of which would be created at the start of execution.

It seems to me that learnings of this kind, where a previous paradigm is destroyed and a new one takes its place, are the richest and longest lasting source of inspiration later on. And all have this peculiar feel to them, combining elegance and a dream-like foreignness.

In programming, I've encountered two other cases of paradigm-destruction... which I'll come back to in further entries.

- Object Lessons - one comment / No trackbacks - § ¶

02 October 03 - 11:16Seven maladies of the project

I'm currently reading Jean-Pierre Boutinet's Anthropologie du projet, which might explain the binge of project-related entries. The concept of the "project" is becoming ever more pervasive in our industrial culture. Boutinet's fascinating preface lists seven abuses of the notion of "project" which can lead to detrimental effects in individuals or cultures.

a) "Project disillusion or double binds". There is now a tyranny of the project. Having a (life) project is mandatory, even for individuals whose capacity for projects has been systematically destroyed by the economic and cultural system. In France, a person's unemployment benefits are tied to their enrolling in a "Personalized action plan". Boutinet points out the "risks of illusion and thus of disillusion entailed by a future too quickly and too artificially idealized" when people "must elaborate for themselves a project on which most of the time they will not have the means to follow through".

Compare this with "pretend projects", and projects underfunded, underresourced, underscheduled which are chronic in our profession, or the projects imposed on people to keep them busy in between contracts or just to keep them too busy to think about how the company's doing.

b) "Project hypomania or the obsolescence of time." In other contexts no project ever comes to an end, because every one is replaced by a further project, pulling us "along an unceasing flow of initiatives and through a flight to a hypothetical future suddenly graced with all the virtues not found in the here and now".

Examples abound in large administrations or banks where pharaonic projects are forever being started to rewrite using "new technology" - and of course a grand new architecture - a bunch of applications which are themselves the result of a previous "innovative" project.

c) "The Xeroxed project." These are projects "more or less imposed from outside, which do not have the time to make the most of the peculiarities in the situation they are supposed to take root in; forced to develop 'turnkey' projects, the stakeholders [...] inject into their project a number of borrowed elements which are foreign to the situation."

In this regard, we've all seen Microsoft ads which promise to have whole corporate systems rebuilt in less than three months with "agility" thrown in. Is this really enough time to learn the unique problems a business faces ? Or is there a risk that we will just be handed the xerox of a previous project ?

d) "Project narcissism and the negation of social relationships." The project ("my project") as pretext for everyone to define their own particular rules of conduct and accountability, and not have to worry about larger scales such as that of the company, community or society.

This may overlap with the "pretend project" issue, but it seems to me that this narcissism is often the result of overspecialization in our discipline, itself a result of project gigantism. I've seen several ads for "Acceptance testing project manager", projects which are apparently setting up testing and V&V as a totally separate project instead of integrating testing throughout implementation. The issue isn't so much that this doesn't work, but that the "testing project" will be totally unaccountable for the success or failure of the project as a whole.

e) "Project obsession with procedures and technique", manifest in such bodies of work as CMM or ISO standards for software projects. I won't say more about that topic now.

f) "Project totalitarianism and psychological dominance", on which Boutinet brings up a nice quote by Cioran: "Every project is a disguised form of slavery." He mentions the "refusal to tolerate any divergence between the plan and its realization", a well known aspect of death march projects. "I don't want to hear about your problems, we have a contract to ship by the 8th and we'll damn well ship by the 8th."

g) "Project utopia or self-justification", as "the project itself becomes a pure abstraction and degenerates into wishful thinking". This I'm less sure I understood, but is perhaps close to the famous problem of "analysis paralysis" where the project becomes mired in the immensity of what is possible, but never confronts that with an actual implementation.

- Software Development - No comments / No trackbacks - § ¶