When I look back at my early days in computing, I can identify many traits which would now be attributed to an informal agile process. Over the years, I was exposed to alternative development methodologies, and more recently it seems we’ve come full circle.
As a novice programmer, I was very close to what the agile process would call “product owner”. Much of the time, the “product owner” was just me. I was “product owner”, “developer” and “QA” all rolled into one. I had my “to do” list (aka “backlog of user stories”), and I just worked through it in priority.
Back then, when someone else was the “product owner” (i.e. when I was programming for someone else), I spent considerable time with them. Often I would code, mockup UI or discuss feature design as they sat by my side. It helped that the “product owner” was invested not just in the final product, but typically had a secondary objective of learning about programming. I remember developing a pretty sophisticated student management application that way. It tracked high school students, student assignment to classes, class timetables, grades and other data.
If software development priorities or my assumptions changed, I would rework the “to do” list (“backlog”) accordingly. I did not have to seek approval from anyone else.
My approach to software development was very incremental. Because I was equally responsible for QA as well as coding, I would try to ensure I always ended up with a completely working application before moving on to the next feature development (“user story”). A “sprint” for me would naturally be like a week, as I’d normally have a standing weekly meeting with whoever was most interested in my progress, and I’d want to do a demo at every meeting.
In most cases, my work was test-case driven. By that I mean, prior to coding, I would work out a number of test cases, and then write code until the results matched my pre-planned tests.
As soon as I added a feature, I could release it to the “customer”. I had no buffer between me and the users and their computers. I had no third-party channel, route to market or integration partners to worry about.
It is notable that I followed this approach without any formal training in software development. I’m referring to a time in the mid-1980s when I was still at high school.
I say this style was somewhat like an informal agile process, because some of the trappings of a formal commercial implementation were clearly missing. I was typically the only programmer on a project, and I wasn't being paid a salary. So unlike a commercial agile implementation, I had no daily scrum meetings. Also there was little formal negotiation associated with “sprint planning”, so I didn't get into the formality of assigning points, or maintaining burn-down charts, etc. My motivation for getting things done was primarily pride-driven.
As I studied Computer Science more formally, I would learn about the “waterfall” methodology, and I would experience the implications of working with other developers in teams, having to deal with extended routes to market, and having less direct access to end users. But when I develop software for myself (which is now far from my “day job”, but still happens quite regularly), I find I still follow to some degree the old patterns.
I started thinking, if agile is a natural and obvious way to code, then why did the “waterfall” methodology emerge? And furthermore, why is waterfall now falling out of fashion?
I would argue that it is a structural change in the software world which makes agile now both relevant and preferred. “Waterfall” methodology seems to me to be as much of a reaction to the business conditions of the 90s software business as it was a theory of product development.
The Web, and then the rise of “apps” (with corresponding app stores) have shortened the lines of communication between developers and their users. The change of business models from perpetual licensing and upgrade cycles to subscription-based has quite an impact, as it moves the focus from big, infrequent releases to rewarding smaller, incremental updates. The hosted nature of Web applications and SaaS, plus the relatively effortless way “apps” update themselves, removes the burden of IT having to manage heavyweight installation processes, and again, permits the roll out of incremental updates versus bundling lots of changes into one major release.
There are of course many businesses that have not fully completed this transition, and some that may never fully do. That’s where it can get interesting. Not only does the Development team need to figure out how to proceed given its own agenda and constraints, but the expectations and needs of other departments (and not least of all management), plus that of the company’s customers and channel partners, need to be considered also. Existing products or businesses which originated outside of the modern web-hosted or “apps” worlds are typically faced with having to implement a hybrid approach.
That’s a situation I’m very familiar with, having run both product management and engineering teams facing this scenario. Finding the right compromise between agile and waterfall approaches takes experience and good judgment, and every business has its own unique needs that need to be considered.
The real world can be more messy than the ideal in other ways too. Just as there have been many well-documented examples of organizations “failing with waterfall” over the years, it is possible to make mistakes in implementing agile also. The danger here is that, generally speaking, organizations are not as sensitive to the warning signs. I have found it has been very instructive to dig deep on the causes of undesirable results and bad behaviors in agile teams, and then figure out how to improve the situation.
I believe there is much to be said for agile development methodologies. Recent changes in the software world make it not just a viable approach, but preferable to waterfall in many situations. As with any ideology, however, its introduction and execution needs to be adapted to an organization’s and business’ own unique needs. Luckily there exist good management techniques to ensure optimum results.