Now that Agile is being blessed by mainstream business journals, the inevitable question becomes: is Agile just another management fad? That is to say, an idea that enjoys intense and widely-shared but short-lived enthusiasm without any basis in its actual qualities. The phenomenon is endemic in management. There’s even an acronym, “BOHICA, or “bend over, here it comes again,” to describe it, with each initiative promising so much and each delivering so little. Is Agile just the latest addition to the list?
The Brave New Workplace Of Humanistic Management
In the 20th century, one of the most popular management themes was teams. Theorists were attracted to the language of teams like some kind of managerial aphrodisiac. Talk of “team spirit” and “winning” brought into the workplace the idea of collaboration and fun, the challenge and heroism of sport and the notion that “we are all in this together.” The language conjured up dramatic feats of excellence, the desire for perfection, and the noble commitment to a goal, while masking any question as to whether the goal itself actually made any sense. As in sports, the point of it all could be hidden by the thrilling pursuit of “victory.”
- Thus in the 1920s, Mary Parker Follett was seducing Oxford and Harvard with her lectures on the principle of “power with” rather than “power over.”
- In 1930s, Elton Mayo offered the intriguing possibility that simply paying attention to employees as people could relieve the monotony and boredom of their work and get them to work harder.
- In the 1940s, Abraham Maslow was waxing lyrical to organizations about the highest human need of all: self-actualization.
- By the 1950s, human resource departments were embracing the magic language of teams to persuade organization men in gray flannel suits to adopt more positive attitudes toward the organizational goals.
- By the 1960s, Douglas McGregor was regaling organizations with the astonishing news that people might want to do a good job if only they were given the responsibility.
- In the 1970s, managers were sent to sensitivity training to make it happen.
- In the 1980s, Peters and Waterman were painting thrilling pictures of organizations with a strong corporate cultures and drawing on echoes of Hellenic excellence, using the mythology of the corporate cultures in technology firms that talked of the creativity and collegiality that was endemic in the legendary Hewlett-Packard garage.
- In the 1990s, team building became outward bound. Rather than trainees crawling around on the floor in a room searching for their inner self, now they were twisting their ankles playing volleyball or hanging from ropes in the great outdoors.
- By the 2000s, firms like Google were offering free gourmet food, laundry services and workspaces like playgrounds.
Organizations spent vast sums on these programs—around $30 billion in the 1980s, close to $60 billion by 2000.
And yet despite all this effort and expense, the results in terms of employee commitment were meager. Today only in one in five workers is fully engaged in his or her work and ever fewer are truly passionate.
The actual satisfactions from the language of teams, like aphrodisiacs, proved remarkably scanty and short-lived. Why?
Team Initiatives Were Often Fake
There was no shortage of good intentions and expertise in these schemes. Psychologists, sociologists and management consultants had analyzed what people needed and how they could work in groups. Employees who saw work as meaningful and felt fulfilled in a workplace with a friendly atmosphere would surely be more productive. Managers who were perceived as kind and caring, and who could inspire and coach employees would do better than bosses who were mean and surly.
Yet the actual implementation of these schemes was often a drab caricature of the romantic language in which they were couched. Was there any real difference between the grim mini-tyrant in a suit and the superficially friendly boss wearing jeans and sneakers, who asked you to call him by his first name, drank beer with you in the evening and then fired you then next morning with a smile?
Organizations might talk of “teams” “commitment,” “loyalty” and “trust,” but business practices for the last half century stressed efficiency and downsizing. Employers might ask for trust, loyalty and commitment from employees, but it became increasingly obvious to the employees that firms had little interest in reciprocating. Organizations were using the language of teams and collaboration to get efficiency gains through smoke and mirrors. The language of humanistic management in the mouths of these efficiency-driven managers were desolate echoes of the real thing—mere empty clichés.
Not surprisingly, the language of “team”, “teamwork,” “team spirit” and “victory” became degraded and disconnected from the reality. Many teams were teams in name only. They were groups of individuals who were told to use the term in a clumsy attempt to make them feel good. As David Riesman recognized in the 1950s, teams were actually “antagonistic cooperation”: the language of teams concealed the real struggle for survival amid the downsizing.
While obviously in all this effort, there were some genuine teams, and I participated in some of them. But more often it was a rah-rah world of fake conviviality. For those who saw through the fairy tale of the enchanted corporation, it was disturbing to grasp the contradiction of empowerment programs that required people to think at all times about the same things in the same terms.
In portraying themselves as friendly communities, organizations were often masking the raw, naked, worker-unfriendly power structure that was still in place, often driven by the goal of maximizing shareholder value. Management was cloaking its power in superficial egalitarianism in an effort to achieve ever greater control over work through domination of meanings, values, and feelings while making more money for the shareholders and the C-suite.
All too often it was a world of forced socialization, with relentless pressure not only to attend the coffee breaks and beer busts but also to be friendly. Now there were two kinds of work, the job itself and the work of playing a role in the fake social life. You had to pay attention because you were evaluated on both.
As workers struggled with the relentless socialization, the impersonal world of Scientific Management and Schmidt, Frederick Taylor’s pig-iron handler, could even have some appeal. “’Schmidt’ worked under physical and mental constraints but did not have to smile at the boss,” wrote Joanne Ciulla in The Working Life (2000). “There were few or no social requirements to his job. He was not asked to eat doughnuts with his coworkers and smile, or to talk about his innermost feelings in company training seminars.” Schmitt “didn’t have to suffer the added indignities of polite interactions with people he didn’t like.”
Not Going Deep Enough
Most of the humanistic initiatives of the 20th Century didn’t go deep enough. Despite the language of teams, people were often being denied the opportunity to exercise choice, make decisions, and have their voice heard on subjects on which they knew best. Often they couldn’t decide how to do the task at hand or get impediments to doing the work removed. The voice of the customer was often lost in the battles up and down the hierarchy, between the competing organizational silos and between differing managerial agendas. The drumbeat of maximizing shareholder value turned humanistic values to dust.
“We’ve flattened corporate hierarchies, but haven’t eliminated them,” Gary Hamel explained last year in HBR.
We’ve eulogized empowerment, but haven’t distributed executive authority. We’ve encouraged employees to speak up, but haven’t allowed them to set strategy. We’ve been advocates for innovation, but haven’t systematically dismantled the barriers that keep it marginalized. We’ve talked (endlessly) about the need for change, but haven’t taught employees how to be internal activists. We’ve denounced bureaucracy, but haven’t dethroned it; and now we must.
“What we need,” said Hamel, “is a cluster of radically new management principles and processes that will help us take advantage of scale without becoming sclerotic, that will maximize efficiency without suffocating innovation, that will boost discipline without extinguishing freedom.”
Are Agile And Scrum Different?
As a step in this direction, Agile and Scrum took a different tack from the humanistic initiatives of the 20th century. (Disclosure: Since 2012, I have been on the board of directors of Scrum Alliance, a non-profit association of more than 400,000 members.)
Back in the late 1990s and earlyl 2000s, the founders of Agile and Scrum generally abandoned the romantic language of teams: “team spirit” and “winning”, “self-actualization,” “dramatic feats of excellence,” “noble commitment to a goal” “the desire for perfection,” “victory” and so on.
Instead, they chose a language that was deliberately down-to-earth and even ugly: Scrum, Scrum Masters, Product Owners, burndown charts, working software, sprints, standups and getting work “done.” There was little romance here, no high-falutin’ language about self-actualization or fake camaraderie. Instead there was a rigorous focus on problem solving, enabling people get on with the work without interruption, drawing on real expertise, removing impediments and regularly delivering value to customers in the face of mind-boggling complexity.
The Horrifying Logic Of The Blue Screen
In retrospect, it’s easy to see why Agile and Scrum (which had their origins in manufacturing in Toyota several decades earlier) took off in software. It was a field that had to deal with the horrifying logic of the blue screen. Unlike most other fields, the work of software had become so complex that it has ceased to be amenable to direct managerial authority. In other fields, a manager could tell people to “fix the sales campaign” or “find some cost savings” and have a reasonable expectation that those doing the work could come up with something that looked like a plausible response. Computers were less forgiving. A blue screen was still a blue screen, and didn’t respond to a boss who screamed at it. Managers could send employees on death marches or fire them and hire replacements, but it still didn’t work. Complexity responded to competence, not authority.
Another difference: unlike most of the humanistic change efforts of the 20th century, in Agile and Scrum, there was an explicit and central focus on the customer—delivering something usable, like working software—at the end of the each short sequence of work. It sought to put an end to the pervasive time-wasting bureaucratic games where “progress reports” were shuffled up and down the hierarchy, with fake claims of forward progress that concealed what was really going on. Now radical transparency was key. The only measure of success was whether anything of value to a customer had actually gotten done. And when agilists said “done,” they meant “really done,” something that was actually usable in the here and now, not a report about something which might become usable some time in the future, but which concealed vast amounts of further work to be done and technical debt that to be paid before anything useful could actually be produced.
This approach was also a good fit with a 21st century business reality: the shift in power in the marketplace in which the customer had become the real boss. It was only through an alliance of those in charge and those doing the work that the organization could prosper. An antagonistic relationship spelt inevitable disaster.
Now that success stories of Agile and Scrum are available to review, we can see why the earlier humanistic change initiatives didn’t stick. The problem was insoluble as long as they were seen as a power struggle between bosses and workers, a struggle that bosses would always win. Getting the bosses to smile and dressing them in jeans and sneakers instead of a suit didn’t make their behavior any more palatable. It was the same old unproductive vertical power game. By changing the game towards delivering value to the customer, the basic dynamic of the conflict could shift.
Instead of the prevailing vertical ideology of control, Agile and Scrum reflected a horizontal ideology of enablement. The goal was to create a workspace that could draw on the full talents and capabilities of those doing the work, and systematically remove any impediments to success. This ideology was also a good fit with a business environment that required continuous innovation for success.
The final element that made Agile and Scrum inevitable was the pace of change. Change had become so fast that the big old hierarchical bureaucracies couldn’t cope. Sending commands up and down the management ladder was simply too slow. The top was too far removed from the customer and the work being done to know what was going on, let alone know how to respond to it.
Yes, Agile And Scrum Are Different
So Agile and Scrum are distinguishably different from the humanistic team initiatives of the 20th century.
Whereas the language of the 20th century team initiatives were romantic, the language of Agile and Scrum was down-to-earth and pragmatic.
Whereas the 20th century team initiatives were often internally focused, Agile and Scrum were focused externally on the customer.
Whereas the 20th century humanistic initiatives tended to assume linearity with problems that could be solved, Agile and Scrum accepted unfathomable complexity and the non-linearity of a continually morphing environment as the basic nature of the game.
Whereas the 20th century humanistic initiatives were plugged into vertical organizations, with success being defined by whether the hierarchy was happy, Agile and Scrum were horizontal in conception: the only true measure of success was whether value was being delivered to customers.
Whereas the 20th century humanistic initiatives took place in organizations that were focused on getting things right, not matter how long it took, Agile and Scrum were focused on delivering value to customers in the here and now.
These considerations help us see why Agile—and all its analogues: Scrum, Kanban, XP and the like—have the prospect of being more than just another fake power grab by managers—old wine in new bottles, the same old game with even more inscrutable names.
Agile and Scrum deal directly with current business issues in a way that the 20th century initiatives sidestepped. They give a direct voice of the customer through the product owner and give a voice to competence over authority.
It is even possible that in Agile and Scrum at their best, we can discover that rarest thing in management, an attempt at the genuine.
Agile and Scrum Aren’t Easy
Not that Agile and Scrum are problem-free.
In the first place, let’s be clear. There’s a lot of fake Agile and Scrum around.
And Agile is not for the faint at heart.The first few sprints can be difficult.
But when firms make it through the difficult transition period, and take the ideology to heart, and implement it on a consistent basis, not merely adding a veneer of words, then they get the kind of results that we see at Apple, Autodesk, Alibaba, Salesforce and elsewhere.
Agile Is Not Enough
Nor is there any basis for complacency.
There are risks in presenting Scrum, as some advocates do, as an approach that produces “twice the work in half the time.” Business process re-engineering went down that track in the 1990s and was quickly discredited as a disguised device for downsizing.
Equally, some of the current efforts to “scale Agile,” such as SAFe, seem to be trying to shoehorn the horizontal ideology of Agile and Scrum into vertical management structures. In the process, they run the risk, as in the failed team initiatives of the 20th Century, of degrading and destroying everything in Agile and Scrum that is authentic and useful. All that may remain will be the empty phrases and labels of Agile and Scrum. We will be back in the unproductive vertical world of hierarchical bureaucracy.
As Kent Beck has explained, there is also a need for the Agile Manifesto itself to evolve.
Moreover, the hard truth is that Agile and Traditional Management still don’t get along. In repeated polls of people working in many different firms where Agile and Scrum are being implemented, somewhere between 70 to 90% report tension between the way Agile/Scrum teams are run in their organization and the way the rest of the organization is managed. Generally less than 10% reported “no tension.”
Thus the horizontal ideology of Agile and Scrum on the one hand and the vertical ideology of control that prevails in big hierarchical bureaucracies on the other are at basically at odds with each other. There is a still a massive effort needed to educate traditional managers on why their organizations are in decline and what Agile and Scrum can do for them.
And read also:
Why software is eating the world
Scrum is a major management discovery
Inspect and adapt the Agile Manifesto
Ten perennial managerment objections to Agile
Agile: Best kept management secret
Follow Steve Denning on Twitter @stevedenning
