Delivering exceptional customer experiences and product for your business take speed and flexibility. More than ever before, speed and flexibility are required from every part of your organization, business and IT alike. DevOps provides your business leaders, enterprise architects, developers and I&O leaders a philosophy to achieve, not only the velocity that customers desire but also drive innovation and enforces quality. One example is ING. The company is undergoing a major digital transformation in which DevOps is a primary driver supporting their transformation. ING CIO Ron van Kemenade has initiated DevOps as the vehicle to aggressively support ING’s evolving customer needs. At ING, technology is the beating heart of the bank.[i]
DevOps requires a transition from technical silos to product centered teams
Effective DevOps will require the tearing down of the technology based silos within an organization. Instead, teams need to focus on the products (or service) delivered and be empowered to own the complete lifecycle. Key performance metrics such as such as availability, the number of features added are used to measure the speed and quality of how these product centered teams work. In some organizations, the team may even own support of the designed and delivered services. This integrated product team is a fusion of developers, infrastructure & operations, quality assurance, and release managers into a single team that works on the entire pipeline, from commit to deployment. Existing centers of excellence such as DBA’s or security teams will remain and support the DevOps team; in some cases, they might even be allocated to the team for a particular duration. [ii]
Deconstruct silos of automation and replace with full pipeline automation
In the days of old, not very long ago, release cycles were measured in years —organizations were using “on-time” and “on-budget" as the mantra for project efficacy. Business today is compelled to deliver business technology in cycles of hours, or days. Faster cycles render not only tradition “waterfall” processes and silo based IT obsolete, it also renders traditional metrics ineffective! These arcane metrics no longer deliver the visibility and granularity tech pros need to fine-tune their delivery capability. The mission has transitioned to rapidly deliver high quality, high value solutions. For all, this is a significant shift from the past, when the main points of focus were schedule, cost, and efficiency. Modern software metrics — speed, quality, and value — are based on continuous feedback from business partners and customers.
Much has been written about how artificial intelligence (AI) will put white-collar workers out of a job eventually. Will robots soon be able to do what programmers do best — i.e., write software programs? Actually, if you are or were a developer, you’ve probably already written or used software programs that can generate other software programs. That’s called code generation; in the past, it was done through “next” generation programming languages (such as a second-, third-, fourth-, or even fifth-generation languages), today are called low code IDEs. But also Java, C and C++ geeks have been turning high level graphical models like UML or BPML into code. But that’s not what I am talking about: I am talking about a robot (or bot) or AI software system that, if given a business requirement in natural language, can write the code to implement it — or even come up with its own idea and write a program for it.
In the latter half of last year, I started researching mobile application testing tools. My research focused, so far, on functional testing, primarily around mobile app front-end testing. As I began the research, it became clear that the automation capabilities testers needed to validate app UIs was there, but application development and delivery teams felt that device labs were too expensive to be practical. During the research for the Vendor Landscape: Front-End Mobile Testing Tools report, we expected that device labs would be a differentiator among products only to discover that most of the major mobile testing solutions provide them in one way or another. There are differences between vendors when it comes to the flexibility, configurability, and management of their device lab offerings, but if you’re delivering customer-facing mobile apps you can do much of your testing on physical devices (our recommended method).
In earlier reports, we recommended that, because of the cost of on-device testing, development organizations focused their testing efforts on the most important aspect of their apps, letting users find issues in less popular areas of the app for them. With most of the major mobile testing vendors offering device labs plus Amazon and Google’s entry into the device cloud space, competition will drive down cost and make on-device testing the more common option for mobile app testing. Microsoft’s acquisition of Xamarin now gives Microsoft a robust and capable device lab, stuffed with a variety of Android and iOS devices, which adds to the competition in this space as well.
A few months ago, I blogged about testing quality@speed in the same way that F1 racing teams do to win races and fans. Last week, I published my F(TA)1 Forrester Wave! It examines the capabilities of nine vendors to evaluate how they support Agile development and continuous delivery teams when it comes to continuous testing: Borland, CA Technologies, HP, IBM, Microsoft, Parasoft, SmartBear, TestPlant, and Tricentis. However, only Forrester clients can attend “the race” to see the leaders.
The market overview section of our evaluation complements the analysis in the underlying model by looking at other providers that either augment FTA capabilities, play in a different market segment, or did not meet one of the criteria for inclusion in the Forrester Wave. These include: 1) open source tools like Selenium and Sahi, 2) test case design and automation tools like Grid-Tools Agile Designer, and 3) other tools, such as Original Software, which mostly focuses on graphical user interface (GUI) and packaged apps testing, and Qualitia and Applitools, which focus on GUI and visualization testing.
We deliberately weighted the Forrester Wave criteria more heavily towards “beyond GUI” and API testing approaches. Why? Because: