State of the art descriptions of Agile best practices generally focus on process enhancement within a single, co-located team. And certainly there are enough new concepts to stay busy within that simple team layout. But in reality, EIS development in single team, single location team configuration are rare. Most enterprise development today involves multiple teams, distributed across multiple locations and time zones, with dependency on one or more vendors. How do the established concepts of Agile (and other modern SDLC) scale to address the complexities of these real-world team structures?

A large number of factors contribute to the complexity of highly distributed team development. One key factor is that different teams evolve different adaptations of Agile, and may not even be Agile in the first place. Several important authors discuss agile best practices for distributed teams, including Mike Cohn in Succeeding With Agile. Generally the recommendation is simple: Don’t distribute if you can avoid it.

One thing is certain: there is value in highly distributed development when executed well. 24 hour working cycles, risk mitigation, capability specialization, and economics. These trends are here to stay and will increase over time.

What are solutions and best practices for distributed, cross-vendor execution teams? I will share my experience in upcoming posts. I’m certain that Agile and other leading-edge SDLC will continue to expand and mature to include these scenarios, and this evolution is happening right now. SAFE makes some inroads but is not yet the answer. One thing is certain, these new areas of practice will lead to tremendous gains in efficiency since the potential for leveraging distributed teams is high.

Development of Enterprise Information Systems has historically been the domain of two closed groups: internal software teams, and large proprietary software vendors such as SAP and PeopleSoft. But current broad trends in enterprise information technology are making it both increasingly possible, and likely, that these systems will be created by external companies that deliver targeted incremental functionality through lean and agile practices, and are not associated with any specific technology. IT trends contributing to this change include:

• External hosting. Cloud delivers significant cost savings. But cloud also implies that applications that previously were hosted internally, are now increasingly accessible and may even be hosted externally. This makes it far easier and more viable for a third party to build and integrate these applications.
• XaaS. New generation enterprise services like Box.net and SalesForce.com are easily accessible for development outside of the enterprise. Also, these services are highly standardized, have well defined and broadly understood API, and have mainstream paths for integration. These are natural candidates for third party development.
• Open Source technologies are increasingly mature, have achieved compelling functionality, and have high market adoption. Importantly, they are also becoming well accepted and mainstream for internal enterprise IT teams. This movement away from specialization and toward well understood and broadly licensed components favors the viability of external development.
• UI/UX. Companies have increasing awareness that design and user experience are critically important for EIS. Internal teams continue to turn to vendors with expertise and specialization in this area.
• Company Culture. For cultural reasons, in some situations companies are more comfortable outsourcing software development. Development projects may be seen as too risky for those making decisions inside. Use of a vendor can be an important insurance policy to help manage this risk.
• Integration technologies. Integration is a key competency- perhaps the key competency- in building enterprise information systems. The faster we integrate, the faster we can meet key requirements and deliver really useful functionality. Integration technologies are improving rapidly, especially the movement to message based, asynchronous, reactive systems. Internal teams continue to engage externally for expertise in this area.
• Development culture. With the increasing importance and visibility of EIS, organizations continue to be motivated to engage external teams to help evolve their own internal processes past outdated models that are in need of change. And on the other hand, software developers themselves may be increasingly motivated to work outside the organization.
• Offshore and Managed Service Models. Today it’s far easier than ever before to build systems across globally distributed teams. And this in turn makes it more compelling for organizations to contract with vendors to take effective advantage of this important trend.
• Language and development tools. Software languages and tools are changing rapidly and are more and more powerful. Large companies will engage external companies for expertise in these innovations.
• Patterns. Patterns of software language and software development are a key building block for strategic information systems. Like language and development tools, large organizations are engaging external companies as leaders of innovation in this area.
• Ease of contracts. Today it is easier and more viable for companies and vendors to create high quality contracts for development of enterprise software. Modern development methodologies such as Agile require KPI and performance measurement as critical to their execution. These data also allow companies to measure and guarantee the delivery of contractually agreed rates and quality levels.

Individually, these trends have influenced software development for years and are not new. One important consequence is that cutting edge development practices, including Agile, will increasingly be applied to distributed, cross vendor teams.

Together these trends are driving powerful changes in the way companies view strategic development of software for their own use.

A primary benefit, as well as risk, in the development of enterprise software within the organization is the opportunity to empower and organize people, by providing an exceptionally powerful medium to digitize the culture and core value proposition of the business. Apparently this isn’t a new idea. Conway’s Law, published in 1968 by Melvin Conway, states:

Organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.

This is absolutely my experience. The organization shapes the software it produces. But the production of software will also shape the organization.

This is a powerful concept. A major opportunity in EIS is not the production of software in itself, but the opportunity to question fundamentals and make collective decisions about stakeholders and the way they relate, both internal and external to the organization.

The production of enterprise systems is not only a technical challenge, it is a sociotechnical challenge. Like most powerful things, this leads companies to very highly positive or highly negative outcomes, depending on the quality of the process.

Here’s an excellent article by Box.net CEO Aaron Levie regarding large market forces aligned toward increasing development of strategic enterprise information systems. I absolutely agree.

There’s a key aspect missing from current dialog on approaches to scaling Agile and other contemporary SDLC practices: Unified Communication.

UC is an extensive area of product development; with a huge number of VOIP and telecommunications companies competing to provide compelling new functionality. UC is also a rich area of communications philosophy about how to enable increasingly distributed businesses to work together with increasing efficiency and balance. And this is a key point: software development teams are not facing the challenge of globally distributed teams in a vacuum. Businesses worldwide are seeking solutions to address these challenges. It’s certainly worth understanding what the communications industry recommends.

Relevant UC products include at least the following:

• High Definition (HD) voice conferencing
• HD video multi-stream conferencing
• Presence, messaging, and collaboration
• Mobile

What are best practices for the use of these powerful tools? Where do they fit in Agile and other modern SDLC that emphasize the importance of communication? Global services, the Managed Resource Model, and other drivers for distributed teams, are increasingly important. Unified Communication has a key role to play in enabling them.

Organizations developing software internally encounter the natural tendency to create a stovepipe relationship between those defining the need for specific functionality, versus those that actually implement it. In this pattern, a business  team dictates requirements to the development group following extensive discussion and decision making process. In other words, people responsible for the “What” (e.g. business development) communicate via a one-way channel to people responsible for the “How” (e.g. technical teams).

This is a loss for the enterprise.

First of all, technical teams generally include very smart people, who naturally will have different perspectives than business teams. They likely have a lot of background on previous requirements execution. There is a powerful opportunity here to leverage this diversity of thinking up front and capture a complete picture of fundamental business need and response.

Second, including the input of technical teams from the beginning ensures that solutions will be idiomatic to the systems roadmap and organization of the technical team.  In other words, technical teams will guide the solution to fall most naturally with existing technology and exection responsibility. This is an excellent opportunity to build value across the enterprise.

“How” and “What” have a circular relationship. This is one of the significant characteristics of highly strategic enterprise information systems.

Companies seeking to implement enterprise information systems in order to leverage the digitization of competitive business strategies, have a wide range of powerful commercial and open source systems available to build upon. As I’ve written, the use of integrated standard components for MIS, inventory, accounting, billing and accounts payable, is an enormously powerful opportunity for the enterprise. This is the long-tail of enterprise information systems.

To achieve the benefits that EIS can deliver, the organization has to make key decisions about where to develop customization into the system. These decisions can be unclear and contentious, and require a lot of effort.

The answer is part of an emerging set of patterns specifically related to the development of large scale software for internal use: Own the glue. Spend as little time and effort as possible on standard functions that are available in common and economical commercial and open source components. Focus the effort of development on the relationships between the systems, because this is where significant value can be created by closely matching the functionality of the system to the real, detailed needs of the enterprise. The core value proposition of every enterprise is unique, and can be powerfully digitized in the integrations between standard, commonly available products for general business functions.

Agile Scrum, XP, Kanban, Lean, DevOps and other cutting edge SDLC generally share the common goal of faster frame rate for delivery of functionality. “Deliver smaller, but complete, slices of functionality faster,” according to the core guiding principles, “and benefit from stakeholder feedback collected as early as possible.” Among other impacts, transitioning to these newer lifecycles puts tremendous pressure on the traditional practice of architecture.

Functional architecture, solution architecture, application architecture, integration architecture, and in particular, enterprise architecture, are traditionally areas where companies will invest a lot of time in planning. And few people will question why it is important to be cautious in charting a 5-10 year course for major systems. This work is at the heart of the value proposition of enterprise software development.

So, how can long-term planning and deep insight be accommodated in the much shorter delivery cycles recommended by the latest systems of development?

In fact the answer actually predates many of the new approaches: do just enough architecture in just enough detail, to meet the immediate need for development. And while doing so, make progress toward a coarsely defined ultimate vision for the future.

Or, as stated by Bente, Bombosch, and Lagande in Collaborative Enterprise Architecture (2012):

The most efficient EA [Enterprise Architecture]… exercises just so much control that the organization operates at the edge of chaos- structured enough not to let the IT slip into anarchy but not so rigid that it is locked into bureaucratic permafrost.

Agreed. Although I’m not sure the term edge of chaos will help create consensus! But in the end, strong architecture absolutely has a place in agile and other new SDLC. Getting agreement on this point is the first step.

Big software has lots of moving pieces. In enterprise software, this complexity is compounded due to the diversity and number of stakeholders, and the corresponding diversity and quantity of functionality.

This diversity can be a source of strength. After all, solving complex problems requires complete perspective, and collectively the stakeholders can provide it. However, success depends on the strength of the underlying structure of the system. If a system is well organized, diversity is sustainable, and complex solutions can be digitized. If not, disorganization will increase, accelerating end of life.

Patterns are one of the most important tools addressing this critical area. Patterns are recurring, mainstream solutions to common enterprise software problems, that have evolved over the history of the industry to become very powerful recommended approaches for enterprise software development. Patterns form the “frame” of enterprise software, providing a strong base to sustain growth while supporting maintainability and manageability.

Patterns make complex software products understandable. And understandability is the key to creating a strong structure. Understandability creates a common language that stakeholders can use to talk about the product without requiring a low level understanding of structures and other aspects of the coding domain.

This includes business stakeholders. Consider the integration of systems in a small company into the much larger systems of an acquiring enterprise. Technical language about low level details including protocol, object hierarchies, schemas, namespaces, networks, etc., is ineffective. But if we describe the overall approach as a Delegate Pattern, we then provide a clear description and characterization leading to a very powerful picture and roadmap for the integration going forward.

Another essential benefit is that software developers and those involved as direct contributors have the ability to discuss very complex concepts in an organic, highly visual style. Context Object, Intercepting Filter, and Factory provide clear indication of function and purpose. In other words, strong encapsulation, a primary design goal of every project and a key to sustainability.

Because the patterns result in code structures that are well encapsulated and isolated, technical teams don’t need to describe individual code components and relationships during communication. The identification of the pattern provides context to the domain problem space, to the component relationships, and to the dynamic behavior of the system.

Software patterns convey meaning between the development environment and the business domain. Patterns convey meaning to business stakeholders just as they convey meaning to code contributors. For example, the Delegate pattern conveys meaning that classes in one project have an awareness of classes in another project via a specific area of code that shields other areas from direct dependency. From a business perspective, Delegate implies a demarcation of ownership of systems, a distribution of responsibilities, and a way that one group can quickly integrate to the services provided by another. Delegate implies an organizational relationship. The same is true of many other patterns, such as Presentation Tier, Business Object, and Web Service Broker.

Principal references for enterprise software patterns include:

Design Patterns: Elements of Reusable Object-Oriented Software (Gamma, Helm, Johnson, Vlissides, 1995): This book is a standard reference and is the basis of modern practice. The authors are known collectively as the Gang of Four (GoF). The patterns identified in this work are building blocks of many more complex patterns.

Core J2EE Patterns: Best Practices and Design Strategies (Alur, Crupi, Malks, 2003) This book extends the work of the GoF. It is the standard reference for those developing software using Sun and Oracle, and other enterprise java technologies.

Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions (Hohpe, Woolf, 2004) This book outlines new, highly complex patterns that are largely the result of applying software patterns to the new, larger patterns that are possible using enterprise solutions. Despite the message oriented reference, the book applies well to all general business software theory.

Agile Principles, Patterns, and Practices in C# (Martin, Martin, 2007) Demonstrating implementations patterns specific to C#, this book extends important work in describing the generalized importance of patterns to enterprise software systems development.

Professional Java EE Design Patterns (Yener, Theedom, Rahman, 2015) Professional Java EE Design Patterns provides fresh views and insight into GoF and more recent established patterns specific to Java Enterprise technology.  

“I’m not technical; I’m not smart enough. I’ll leave that to others in the room.”

I’ve heard this statement in different forms, from different people, several times. It’s a disappointing message that indicates that the real agenda of the speaker may be different from that of the executing team.

We are in business together. Technology is so closely interrelated and essential to our work that it is difficult to describe this relationship. Technology is the language and medium of business. Can people really claim to be ignorant and justify their continued participation in the forward movement of the organization?

I don’t think so. Technology is essential to what we do, and we each have an important role in it.