Agile Adoption Challenges in the Enterprise Applications

[This article is a guest post by Rajesh Raheja, who is a senior director of development in the Oracle Fusion Middleware SOA group, responsible for developing SOA governance tools and providing engineering guidance to key customers and partners for Oracle's Application Integration Architecture. He is a Certified ScrumMaster and a Stanford Certified Project Manager. You can visit his blog or connect on Twitter @RahejaRajesh.

Note: The views expressed on this blog are his personal views and do not necessarily reflect the views of his employer.]

Agile faces some key challenges to overcome when dealing with adoption in the enterprise. Projects in the enterprise come in various flavors – from developing minor utilities to large enterprise business applications. The latter kind of projects have characteristics that are a bit different from other projects. For example, integrating multiple heterogeneous applications to satisfy an end-to-end business process, requires a lot of time doing functional analysis as well as testing. A rough estimate pegs approximately 70% of the project time (of around 9-15 months per release) on just analysis and testing.

(More)

Although a waterfall based approach would be simpler, it would not have the improved visibility and the ability to deliver incremental value, both of which are needed to reduce risk in such complex projects. Looking back at projects which started off excited about using Agile, many reverted to some mix of waterfall and Agile (i.e. “iterfall”) models, highlighting issues with “strict Scrum” (our preferred choice of Agile methodology). This reflects the realities that Agile has to face in the enterprise.

Here are the top 3 challenges faced:

1. Complex inter-dependencies between projects

This was by far the biggest issue faced. Enterprise projects are complex with many layers of inter-dependencies, and this is not addressed effectively in Scrum. For example, an application integration project that sends customer orders between a CRM and ERP fulfillment system would have the following tasks done by each team i.e. the CRM, ERP team and middleware/integration  teams.

  • Configuration of new data model attributes
  • Modifications to API and web services
  • User interface enhancement to add new fields to existing screens
  • Web service configuration to pass the new data elements to middleware
  • Data  migration or clean up scripts
  • Regression testing

The integration team needs to wait on the application teams to develop the above. These application teams handle many unrelated projects and do not necessarily use Scrum (there goes the Scrum of Scrums). Visibility into deliverables is typically at a milestone which is usually communicated upfront. Testing activities are challenging even without Scrum, because even if one application is ready, the end-to-end testing cannot happen till the other application is also ready.

The Scrum approach of dealing with this by avoiding dependencies just does not cut it and immediately relegates Scrum to the “only use for small non-critical projects” bucket. To gracefully handle this issue, most teams I worked with ended up either customizing their approach to “Scrum-like” or “iterfall”.

2. Handling of Specialized and Global Project Resources

Scrum emphasis on self-organizing teams works great when teams are small and co-located. But it gets challenging when the reality is that teams today are global and geographically distributed across two or three continents.

On top of that, Scrum did not seem to cater to specialized resources very effectively. For example, in major organizations, there is often a shared community of expert resources such as architects, usability engineers, QA, performance and scalability team; who serve many project teams. These resources have to be committed well in advance and have specific deliverables,  which also becomes an additional dependency to track. It cannot simply be assumed that these specialized resources would be available for every sprint, forget about every stand-up meeting.

3. Sprint Overhead

The concept of sprints – a time boxed period in which user stories are planned, executed and made production-ready – was initially very alluring, but later became one of the reasons why some teams came up with their own approaches to bypass the overhead caused by it.

Sprint overhead is primarily caused by valid user stories that always exceeded sprint lengths. No, these were not epics, and had been broken down to the level acceptable by the product owner as providing real business value. For complex projects, even a three or four week sprint could not fit an end-to-end feature story due to the dependencies.  Some teams initially resorted to creative hacks e.g. artificial break up of stories (“Architecture – Part 1”), which really defeated the core purpose. Those who did not resort to such hacks were penalized with inaccurate velocity calculations and perception of poor performance since many sprints had incomplete stories with zero velocity. Instead of helping, this need to satisfy strict Scrum logistics interfered with the development of such projects. An unfortunate side-effect of trying to finish off stories in unreasonable sprints was lack of good documentation; with business knowledge sometimes being sacrificed in the name of lean documentation.

This reminds me of a popular Mexican fresh grill slogan:

“Food cannot be made at microwave speed.”

Similarly, it became apparent that complex enterprise projects cannot be built with fixed sprint lengths. Moreover, product roadmaps are built years in advance where the supposed flexibility of dropping features is sometimes not even an option. Overall, business process analysis, attribute mapping, functional interoperability design and architecture design needs flexibility and adaptability in the Agile methodology.

As readers may notice a lot of the emphasis above is on “strict Scrum.” Teams that managed to overcome these limitations did so by adapting Scrum (which I will describe in future posts). Upon hindsight, some projects may have been much better suited to Kanban, something I am now trying on a smaller project. Some practitioners consider Kanban as “watered down Scrum” and others think “Scrum, but” and “Kanban-like” to be the cause of failed Agile projects, but in my experience, it is because of these derivative approaches that complex enterprise agile projects succeeded, without which most would have moved back to waterfall due to the execution risk posed by the strict Scrum limitations. As pointed out by another practitioner, we need to leave the “turf wars” behind and use the right tool for the right job. Agile is just emerging and will need to be flexible in order to move beyond the hype and gain true relevance in the enterprise.

Would you like to contribute as an Agile Scout Guest Poster? Let us know.

7 Responses to “Agile Adoption Challenges in the Enterprise Applications”

  1. Alexandre Gauthier
    November 23, 2010 at 10:56 am #

    Rajesh, I see the same phenomenon.

    Working with a lot of business who are just starting out with Agile Processes, I find that intuitively, this sounds about right.

    However we’ve seen that with some Project Managers using our agile project management tool, discipline was the key to resolving this.

    What I mean by that is that the ability of users to organize themselves meant that they’d often add stories within a sprint (that’s ok, its called prioritizing and reacting to change) without replacing the current stories to keep an accurate time estimate.

    That would often lead to reversed Burn-down charts!

    Keywords here are discipline and education.

    Cheers!

    • ScoutMaster
      November 23, 2010 at 11:30 am #

      Thanks for this helpful comment! How ever much we love tools, foundational elements need to come first: (Standups, retrospectives, wall-boards, etc). But the tool is a good after all these have been solidified!

    • Rajesh Raheja
      November 23, 2010 at 3:16 pm #

      I second ScoutMaster’s observation, but would also add that projects – Scrum or not – are not artifacts ruled by methodical precision, but instead are tasks executed by (largely irrational) people. In addition to the foundational elements, softer aspects also need to be considered e.g. organizational strategy, vision, culture etc.

      Agile needs to be agile enough to incorporate these aspects, and once it is, it would oddly enough look like the “Scrum-like” derivatives that everyone loves to hate.

Trackbacks/Pingbacks

  1. Tweets that mention Agile Adoption Challenges in the Enterprise Applications | Agile Scout -- Topsy.com - November 23, 2010

    [...] This post was mentioned on Twitter by Agile Scout, Agile Scout. Agile Scout said: Agile Adoption Challenges in the Enterprise Applications http://goo.gl/fb/YUh14 #agile #pmot [...]

  2. 3 Reasons Agile Faces Challenges in the Enterprise | RAJESH RAHEJA - November 23, 2010

    [...] Posted on November 23, 2010 by Rajesh Raheja AgileScout invited me to write a guest post on the use of Agile in the enterprise. Having worked and being involved in multiple projects of varying complexities, I found adopting [...]

  3. 10 Tips for Adopting Agile in the Enterprise | RAJESH RAHEJA - May 4, 2011

    [...] few months back I wrote about the challenges facing agile adoption in the enterprise. I got a lot of requests to outline some tips to overcome some of these challenges, which I have [...]

  4. Meeting #2 Topic: Integrating specialized resources into Scrum teams « Center of the Agile Universe - July 10, 2012

    [...] Scrum sometimes does not seem to cater to specialized resources very effectively. For example, in major organizations, there is often a shared community of expert resources such as architects, usability engineers, QA, the performance and scalability team, who serve many project teams. These resources have to be committed well in advance and have specific deliverables, which also becomes an additional dependency to track. It cannot simply be assumed that these specialized resources would be available for every sprint, forget about every stand-up meeting. (AgileScout.com) [...]

Leave a Reply