The Human Side of SOA – Part 1

Application architectures have evolved greatly over the last 30 years of computing. They’ve shifted back and forth from proprietary character-mode terminals with centralised processing to client-server applications with distributed processing. Then, they shifted to the centralised hosting of applications but this time with open standards-based, web-delivered front-ends.  Each of these trends included a variety of methods to connect disparate systems—many proprietary, some without any re-usable characteristics and all typically requiring the developers, operators and maintainers to learn a new set of skills and the organisation to put new processes in place. These upheavals invariably created friction and frustration for the individuals and the organisation involved.

Enter the Service-Oriented Architecture (SOA). Not owned by any one vendor or organization, it is logically and physically a loosely connected set of principles and technologies that collectively constitutes the next, possibly the last, major generation of application deployment architecture.

The ‘human side’ of SOA ensures that planning, politics and personnel issues do not derail otherwise carefully designed SOA technology initiatives.

SOA changes the way applications are designed, developed, deployed and maintained. Application architectures that were once “built to last” are now explicitly “built to change.”

A SOA offers true technical benefits. Yet a successful transition to SOA requires not just technical change—but organizational change as well. If managed properly, this organizational change can further foster the growth, adoption and success of an SOA initiative. However, if not managed properly, it can cause disruption and organizational conflict.

Greatly generalized, there are two groups involved in any application: the business users and the IT group.

In a traditional application- development environment, interaction between these groups is limited to the times when there is either a new application to be developed or a problem with an existing application.

Interaction occurs primarily when the stakes are high and, therefore, often in an emotionally loaded environment. By breaking down the delivery of application functionality into discrete chunks or services, the two camps can lower the pressure, lower the stakes and collaborate more effectively.

The consequences of changing the level of granularity at which IT and business users interact is both subtle and profound. By concentrating on (relatively) simple business functions, IT can concentrate on automating tasks that remain relatively stable year-to-year.

In a world of globalised businesses, the biggest source of competitive advantage is found in business process innovation. Consequently, the parts of a business that change most rapidly and completely are the highest-level, most strategic business processes. In the traditional model, these business processes are implicitly captured in people’s heads or in hundreds of lines of monolithic application code. In the business world enabled by SOA, these business processes are instead moved into explicit, model driven Business Process Management (BPM) tools under the control of business analysts, where they can be changed and reconfigured at the click of a mouse.

If BPM is the way to enable business agility, SOA is the way to provide the IT agility that goes along with it. So, beyond the changes that occur in the way development teams and business users interact, SOA and BPM have the potential to change the way the entire organization operates. For this reason, the success of any SOA journey must be aligned to the corporate goals and vision and requires commitment and buy-in from the highest levels in the organization. Truly, the impact of SOA is both subtle and profound.

All organizations have both short term and long-term goals. For most businesses, IT is no longer viewed as a strategic differentiator or even as a strategic enabler. It has become “part of the scenery” at best and, at worst, a hindrance. The powerful combination of SOA and BPM has the potential to return IT to its former position as an agent for strategic advantage. The optimal value of SOA then is to support initiatives that are aligned with corporate strategy, especially those that focus on a move to exploit BPM.

What Makes a Good SOA Project?

I was recently approached by Project Managers who asked several questions about SOA.
 
This got me thinking and I decided to respond to their questions by sharing an article written in July 2008 by a colleague of mine (Technical Services Director from Progress Software) which may help.
 
The original article is available here.
Have you ever heard of this question, or something similar: “What is a typical or good SOA project”?

Let us examine the whole idea of and confusion around SOA projects in more detail.

First, let us consider what SOA is.
 
SOA is a conceptual architecture. It is not a product. For that reason, SOA is often misunderstood because it is not something that you can touch and click.

Also, every vendor’s SOA products and implementations are different. So no two vendors can agree on what is the best SOA implementation.

As if this is not complex enough, different vendors will stress certain features and functionalities to their advantage. Some stress the usability advantage. Other talks about a complete product stack while others discuss standards support, performance, or continuous availability as key advantages.

No wonder to the end user, SOA can seem like the latest 3 letter acronym that vendors are using to make money.

However, the reality is not that grim.

There have been plenty of SOA success stories, especially in North America and Europe, from every vendor. The difference between these regions and Asia is that SOA adoption occurred earlier and therefore organizations have gone through the learning phase of utilizing SOA and are now starting to see the SOA initiative paying off.

The other difference is the IT infrastructure build out in these regions have occurred much earlier so organizations can focus much more on SOA than on both infrastructure and SOA.

Also, SOA has clearly replaced EAI as the technology of choice. You can tell by looking at all of the traditional EAI vendors. They all have a SOA story and no one is talking about EAI anymore.

Asian organizations, which are behind in SOA adoption, should look past the confusion and leverage SOA to their advantage.

Organizations that are new to SOA can learn from the mistakes of the early adopters by focusing on the following:

1. Adopt SOA soon if you have not already. 

 
Despite the confusion, SOA offers many advantages like service reusability and service abstraction. In fact, SOA closely resemble what happens in real life. For example, when you use a service like stock trading, whether you call a broker or do it online, you are not aware of the complexity that goes on behind the scenes, nor should you have to. That is what SOA is about! At every step of the way, what the service does to achieve its objective is hidden from the service user. It translate to a much more flexible and agile infrastructure, which means you can add services or change your business process much faster to suit your business needs (i.e. business agility). Agility is a powerful business advantage.

2. Find the vendor that matches your needs. 

 
Every platform has its strong selling points. The question is not which vendor has the best products. Rather, it is which vendor has the right products for you. For example, if your enterprise is highly distributed or your business needs to be available 24×7, then a highly scalable, distributed, and available product is what you need. Having a sexy GUI may not be the most critical thing.

3. Most importantly, focus on the business objectives, not the technology.

 
The mistake I see organizations make is deciding on an ambitious SOA project or to SOA enable everything when the infrastructure is being revamped.

Many times the SOA project is part of a bigger infrastructure project so people are dealing with a lot of moving parts at the same time. When you have that many moving parts, it is even more important to focus on the business objectives instead of the technology: objectives such as business integration, factory automation, CRM, straight through processing, etc. There are many reasons for this.

Doing SOA for the sake of having the latest technology will get you no where fast. If you cannot show results, the best technology will get shelved eventually.

With every technology or platform, there is a learning curve for the organization. While it is the right thing to have a clear picture of the ultimate goal, you need to start taking small steps to get over the initial learning curve.

It takes a journey to reach certain level of SOA maturity. It is difficult to go from no SOA to highly optimized SOA infrastructure in one step. Organizations need to take manageable steps to reach such maturity.

Organizations that have tried SOA but have failed should ask a few questions:

  1. Is the project too ambitious?
  2. Is there a clear business driver? Is there management buy-in?
  3. Can the benefits be shown easily?
  4. Did I pick the right platform for my business?
  5. Answering these questions can help you find out where the problem is and regroup for the next SOA initiative.

So, this goes back to my original question and the answer is:

  • A good SOA project is a project that has a proper focus and clearly defined, achievable goals.
  • Your objective is integration, factory automation, BPM, etc. SOA is just the means to achieve these objectives.
 

Magic Quadrant for Integrated SOA Governance Technology Sets, 2009

Introduction

The recession and the need for cost optimization, combined with the market’s need of service-oriented architecture in 2008 and the “cover your back” mentality, have caused many companies to rethink the importance they give to SOA governance and related technologies.

Service-oriented architecture (SOA) governance technologies, much like any other set of technologies, are striving to address needs beyond the SOA realm. Process governance, governing cloud interactions and interactions with business partners (brokerages) are on the radar for most technology providers. Getting the basics correct is a necessity, but these technologies will likely be used to govern peripheral activities.

Market Overview

The market for SOA governance technologies is still in turmoil, with acquisitions, mergers, new players, and (most importantly) the dramatic maturation of companies procuring and deploying SOA infrastructure and SOA governance technologies. In creating the 2009 Magic Quadrant for Integrated SOA Governance Technology Sets, we immediately noticed a dramatic shift of all the participants toward the median of the quadrant. When assigning vendor weightings for this Magic Quadrant, we detected, via our analysis, client interactions and vendor briefings, maturity of the market. A few key data points became very apparent:

  • Customers are no longer looking for just a registry; instead, they were looking for a suite of projects.
  • A growing percentage of customers are including SOA governance technologies in their initial SOA projects.
  • Customers and technology providers are putting more emphasis on SOA validation and monitoring; the latter is seen as critical to monitoring metrics and measuring success.
  • Customers are deploying SOA centers of excellence, supported by the architecture group and empowered by the CIO.
  • Customers are looking for solutions to easily integrate with their SOA and integration platforms.
  • Customers are looking to govern their interactions with business partners and services provided via the cloud.

However, it’s not just the market shifting. A few new players have entered the market that represent various niches of customer wants and needs, focused on life cycle management, monitoring and policy enforcement. Vendors that offer some or most of this functionality include:

  • Alcatel-Lucent
  • Intel
  • MuleSource
  • Nastel
  • Oracle
  • Sensedia
  • Sonoa Systems

In addition, some vendors from the 2008 Magic Quadrant were acquired, and, as a result, the acquiring company has entered this market. Oracle, which acquired BEA Systems in 2008, is now a player in this space. An impressive number of acquisitions have taken place, with vendors in this market acquiring vendors in markets peripheral to the SOA governance technology market. Examples include:

  • Oracle acquiring ClearApp
  • SOA Software acquiring LogicLibrary
  • Progress Software acquiring Mindreef

These acquisitions highlight the vendors’ acknowledgement that monitoring, validation and life cycle management are core to SOA governance.

The job of integrating SOA governance among disparate domains remains complex, because, although obtaining SOA governance technologies from your installed middleware vendor may be less expensive than from a third-party vendor (in many cases, new technology licenses can be combined with existing ones), ensuring that the technology supports heterogeneous environments is essential. Interoperability can sometimes be difficult, and SOA governance standards and specifications are immature in some areas, such as WS-Federation (WS-FED), and nonexistent in others (governance interoperability). Ensuring that your vendor participates in some governance interoperability specification can help ease deployments, especially in best-of-breed situations, where appropriateness can no longer be ignored.

Market Definition/Description

As the SOA governance market matured in 2008, organizations that had been new to the market began to gain a new level of sophistication in the understanding of organizational requirements and vision for SOA deployments. Still, the market for SOA governance is a varied one, with many different types of products providing support for governing the behavior of an SOA. SOA governance is about ensuring and validating that assets and artifacts within the architecture are operating as expected and maintaining a certain level of quality. Now in its second year, this Magic Quadrant reduces the market to one set of technologies with strong architectural cohesion (integration), promoting ease of use and the interoperability of products. This integration includes the idea that multiple personas will be involved in governing an SOA. Each of these personas will bring a different perspective to the process of performing different kinds of tasks. However, all these tasks must be part of a unified governance effort, instead of a different, but related, effort.

To read and download the full report – click here.

Oracle to Buy AmberPoint, Strengthen APM and SOA Governance

Oracle on 8 February 2010, Oracle announced that it has agreed to acquire AmberPoint , SOA Management, Business Transaction Management and run-time Governance software. The deal is likely to close in 1H10.

Acquisition is expected to extend Oracle’s leadership in Fusion Middleware and provide customers with advanced capabilities in SOA Governance and SOA management.  Amberpoint is among leaders in SOA Management mainly to enforce policies that improve security, and to diagnose transactions not only within composite application but also across different applications.

For further reading on AmberPoint’s Governance and Management System click here  and for AmberPoint Express click here.

Extending Service Boundaries to Infrastructure Resources

The article appeared in the MSDN Architecture Journal, Issue #21, Sep 2009.

To download a copy of this edition of the MSDN Architecture Journal, click here.


Extending Service Boundaries to Infrastructure Resources

by Nayan B. Ruparelia and Salim Sheikh

With data centers considering infrastructure resources as service-on- demand resources and amalgamating them with application services, we redefine an SOA service as comprising the server-side application and the infrastructure that hosts it.

The business benefits of this type of architecture are twofold—a reduced infrastructure footprint and a lower cost of service delivery— and are enunciated as follows:

  • Commoditization of services enables true SOA-service rationalization that takes place at the service level, instead of separately at the software and hardware boundaries. This reduces the data-center (or infrastructure) footprint for the business and, thus, provides a more environmentally friendly SOA service at lowered costs.
  • Encapsulation of an SOA service across the hardware-software stack provides a complete measure of return on investment (ROI)—from application to infrastructure. This gives a more transparent, accurate, and rationalized view of the cost of a service and its time-to-market (TTM) cost. These factors drive down the cost of an SOA service, because it allows the business to measure reusability of the SOA service across all tiers of IT and, thereby, create a framework for more efficient use of services.

Problem Statement

Generally, an SOA service comprises applications that serve client (or consumer) applications. The management of infrastructure resources that the service consumes—specifically, in relation to capacity and performance—is left to the engineer to plan and model manually. This means that service demands might not always be met within the required time constraints, if the capacity management is reactive instead of proactive; thus, there is a danger that the infrastructure will act as a bottleneck.

Conversely, infrastructure resources can remain idle should the service demand be below that which is provisioned by capacity planning. This represents a waste of money on unused infrastructure. Furthermore, capacity planning and modeling, when performed properly, can itself be a very costly and time-consuming exercise and would need to be performed regularly in order to track changing SOA-service workloads over time. Therefore, a more efficient and cost-effective solution is needed for designing and implementing SOA services.

In this context, we refer to infrastructure resources as the hardware platform and its related services that are required for hosting the SOA applications, not the ESB.

Proposed Solution

We propose an SOA service that makes up the entire hardware- software stack. We term such an SOA service a “composite service element” that would be provided in a virtualized environment, in order to better provision the resources of the service and measure its resource usage.

The host platform (comprising hardware computing resources such as CPU, RAM, and local storage) as well as the operating system would be provisioned as a virtual machine (VM). The VM, as much as the software that it hosts, can be considered a service; we denote this VM as an SOA service element. Furthermore, the entire hardware-software stack can be virtualized to include the storage (such as SAN and NAS devices), the LAN, and the interfaces to both of these. This means that all of the layers in the architecture that comprise the logic, data- access, and data-storage layers can be virtualized and provisioned as part of a single SOA service.

To address performance, scalability, and availability requirements, the VM can be configured in a cluster to provide both integrity and workload sharing. The number of VMs that are used in the cluster would be defined by orchestration software that would instantiate VMs as needed. Indeed, the resources of the physical computer that hosts the VM would also be provisioned to suit demand.

In addition, virtualization of the storage and the LAN means that these components can be provisioned individually to meet the workload demand. Demand forecasting can be performed by instruments that measure the resource requirements and predict their usage over certain timeframes by using various load-prediction models.1 This demand forecast can then be used to drive workflow- automation software (such as the HP Blade Matrix System) that would provision the virtualized resources to meet the predicted demand.

Service-Element Pattern

A pattern is a solution to a common problem that is documented in a consistent format and used as part of a larger collection.2 We propose the service element as a pattern that is defined by a VM; thus, the SOA-service design pattern comprises the producer application and its virtual host.

Conclusion

In this article, we redefine an SOA service as being composed of the entire hardware-software stack; we refer to this type of service as a composite service element. Furthermore, virtualization is used so that we might be able to provision our SOA service in an efficient and complete manner and with the smallest cost overhead possible. Also, our approach predicts the workload demand and integrates it with workflow-orchestration technologies, so as to automate the provisioning of resources that are used by the composite service element. Apart from a reduction in cost, a benefit of this approach is that it reduces resource consumption in an environmentally friendly manner.

Resources

For additional information, please refer to the blog of the lead author.

References

  1. Andreolini, Mauro, and Sara Casolari. “Load-Prediction Models in Web-Based Systems.” ACM International Conference Proceeding Series. Vol. 180, October 2006.
  2. Erl, Thomas. SOA Design Patterns. Upper Saddle River, NJ: Prentice Hall, 2009. Page 86.

Nayan B. Ruparelia (nayan@acm.org) is an IT Architect at EDS, part of Hewlett Packard’s TSG group.

Salim Sheikh (salim.sheikh@gmail.com) is an Independent Consultant and Enterprise Architect, certified in a number of frameworks including TOGAF, Zachman, COBIT and ITIL. He is also a Certified Process Professional and LEAN/Six Sigma practitioner.

The Business Case for SOA & The Role of NextGen Architects

ABSTRACT

Five years ago, the business case for Service-Oriented Architecture (SOA) didn’t matter. Organizations pursued SOA initiatives based on a desire to achieve ‘competitive advantage’ and ‘agility’ without specifying clear metrics and success criteria. A great deal of time and effort is spent on technology architecture, governance and vendor assessments, which is good, but the fundamental point is that SOA is about business.

 Today’s budgetary environment of doing less with less, coupled with the current economic climate, makes it important for SOA champions to make their business cases for investment compelling and to create and maintain momentum.

What is required is a new generation of architects capable of taking a business-led and multi-disciplinary approach to SOA enabling better Business/IT alignment. Next generation (NextGen) architects are no longer part of an IT department or individual IT projects. NextGen architects are “architects of the business” – organizational resources delivering business-led SOA solutions to directly support new business initiatives, oversee business change, governance and budgetary controls.

This webinar includes advices and recommendations based on actual experiences and successes across a range of Public and Private industries.

FUTHER INFORMATION

Join me on 26 Feb 2010 at 11:00 am (GMT) by clicking here.

This presentation has been postponed. I will provide further updates once a new date/time has been agreed.

Apologies for any inconvenience.