The Forrester Wave™ Evaluation Of Functional Test Automation (FTA) Is Out And It's All About Going Beyond GUI Testing

Diego Lo Giudice

A few months ago, I blogged about testing quality@speed in the same way that F1 racing teams do to win races and fans. Last week, I published my F(TA)1 Forrester Wave! It examines the capabilities of nine vendors to evaluate how they support Agile development and continuous delivery teams when it comes to continuous testing: Borland, CA Technologies, HP, IBM, Microsoft, Parasoft, SmartBear, TestPlant, and Tricentis. However, only Forrester clients can attend “the race” to see the leaders.

The market overview section of our evaluation complements the analysis in the underlying model by looking at other providers that either augment FTA capabilities, play in a different market segment, or did not meet one of the criteria for inclusion in the Forrester Wave. These include: 1) open source tools like Selenium and Sahi, 2) test case design and automation tools like Grid-Tools Agile Designer, and 3) other tools, such as Original Software, which mostly focuses on graphical user interface (GUI) and packaged apps testing, and Qualitia and Applitools, which focus on GUI and visualization testing.

We deliberately weighted the Forrester Wave criteria more heavily towards “beyond GUI” and API testing approaches. Why? Because:

Read more

A Glimpse Into The Future: Software Goes Invisible

Michael Yamnitsky

Software is getting smarter, thanks to predictive analytics, machine learning, and artificial intelligence (AI). Whereas the current generation of software is about enabling smarter decision-making for humans, we’re starting to see “invisible software" capable of performing tasks without human intervention. 

One such example is x.ai, a software-based personal assistant that schedules meetings for you. With no user interface, you simply cc “Amy” on an email thread and she goes to work engaging with the recipient to find a date and optimal place to meet.

It’s not a perfectly automated system. AI trainers oversee Amy’s interactions and make adjustments on the fly. But over time, she becomes a great personal assistant who is sensitive to your meeting and communication preferences. 

One can imagine Amy extending into new domains — taking on parts of sales/customer service operations or business processes like expense management and DevOps. Indeed, we’ll see a new generation of AI-powered apps, as predicted here.

Read more

What I&O Leaders Should Look For At CES

JP Gownder

This weekend, I’ll be heading off to Las Vegas for the 2015 Consumer Electronics Show (CES). Infrastructure & Operations leaders should – and do – keep tabs on the news coming out of CES. In this era of consumerization, bring-your-own (BYO) technology, and Shadow IT, CES announcements affect the I&O role more than ever before. I have three tips for how to think about CES 2015:

  • Look at consumer technologies through a workforce lens. So many smart, connected products quickly migrate to the workforce. Sometimes these technologies enter via BYO and segue into company-owned, as tablets have done over the past few years. In other cases, vendors that target consumers immediately see the value their products can bring to workforce scenarios. For example, I recently spoke with Jonathan Palley, CEO of Spire, a wearable device that tracks not just activity but also state of mind (tension versus calm, focus versus distraction, and related states). While the product was launched to the consumer market just about a week ago, Jonathan made clear that “workforce is a huge part of our strategy as well.” Imagine helping workers remain in a more productive, less stressed state of mind via wearables.
Read more

Death To CMDB! Long Live The Dream!

Glenn O'Donnell

I’m sitting on my sofa at home (Yes! Home!) on Sunday morning just before Christmas. I’m “shut down” for the holidays now, but of course, I’m watching Twitter and now listening to my brilliant friends Chris Dancy and Troy DuMoulin discussing CMDB (configuration management database) on the Practitioner Radio podcast. It’s a marvelous episode, covering the topic of CMDB in with impressive clarity! I highly recommend you listen to their conversation. It’s full of beautiful gems of wisdom from two people who have a lot of experience here – and it's pretty entertaining too!

I agree with everything these guys discussed. In particular, I love the part where they cover systems thinking and context as the key to linking everything conceptually. I only have one nit about this podcast, and the greater community discussion about CMDB, though. Let’s stop calling this “thing” a CMDB!

I coauthored a book with the great Carlos Casanova (his real name!) called The CMDB Imperative, but we both hate this CMDB term. This isn’t hypocritical. In fact, we make this point clear in the book. Like the vendors, we used CMDB to hit a nerve. We actually struggled with this decision, but we realized we needed to hit those exposed nerves if we were going to sell any books. Our goal is not to fund a new Aston Martin with book proceeds. If so, we failed miserably! We just wanted to get the word out to as many as possible. I hope we've been able to make even a small difference!

Read more

Cisco Services Leverages Software Assets To Transform Its Services Value Proposition

Fred Giron

As you’re all well aware by now, a perfect storm of technology innovations — including cloud, analytics, mobile, and social­ ­— is fundamentally disrupting the way your company engages with its customers (as well as employees and partners). For service providers in particular, the main challenge is understanding how to best leverage these technology innovations to remain relevant and ultimately generate more business value. So it’s exciting to see a service provider like Cisco Services come up with new offerings that respond to this challenge in innovative ways.

I met with Cisco Services Asia Pacific Japan and China (APJC) executives last week in Seoul to discuss their strategy in Asia. I wanted to highlight a few takeaways that I believe will be important for sourcing professionals in Asia and beyond:

  • Cisco Services is a key enabler of Cisco’s overall transformation. Cisco Services used to be a captive consulting organization providing support and technology services for a product company. In a recent analyst call, John Chambers identified Cisco Services as one of the main levers that will help Cisco transition from a transaction-oriented to an annuity-based business model and help the company become the largest IT company globally. The company’s aim is for Cisco Services to represent 24-26% of total revenues in the next 3-5 years. These goals are extremely audacious; achieving them will require huge efforts from Cisco, including some targeted acquisitions in the services space.
Read more

Are You Really Ready To Test Agile?

Diego Lo Giudice

Early this year, on January 15, I published our first research on testing for the Agile and Lean playbook. Connected to that research, my colleague Margo Visitacion and I also published a self-assessment testing toolkit. The toolkit helps app-dev and testing leaders understand how mature their current testing practices and organization are for Agile and Lean development.

The Agile Testing Self-Assessment Toolkit

So what are the necessary elements to assess Agile testing maturity?  Looking to compromise between simplicity and comprehensiveness, we focused on the following:

  • Testing team behavior. Some of the questions we ask here look at collaboration around testing among all roles in the Scrum teams. We also ask about unit testing: Is it a mandatory task for developers? Are all of the repeititive tests that can be run over and over at each regression testing automated?
  • Organization. In our earlier Agile testing research, we noticed a change in the way testing gets organized when Agile is being adopted. So here we look at the role test managers are playing: Are they focusing more on being coaches and change agents to accelerate adoption of the new Agile testing practices? Or are managers still operating in a command-and-control regime? Is the number of manual testers decreasing? Are testing centers of excellence (TCOEs) shifting to become testing practice centers of excellence (TPCOEs)?
Read more

The Essence Of Agile Testing: Make Testing Part Of Your SDLC (And Much More)

Diego Lo Giudice

DevOps is a movement for developers and operations professionals that encourages more collaboration and release automation. Why? To keep up with the faster application delivery pace of Agile. In fact, with Agile, as development teams deliver faster and in shorter cycles, IT operations finds itself unprepared to keep up with the new pace. For operations teams, managing a continuous stream of software delivery with traditional manual-based processes is Mission Impossible. Vendors have responded to DevOps requirements with more automation in their release management, delivery, and deployment tools. However, there is a key process that sits between development and operations that seems to have been given little attention: testing.   

In fact, some key testing activities, like integration testing and end-to-end performance testing, are caught right in the middle of the handover process between development and operations. In the Agile and Lean playbook, I’ve dedicated my latest research precisely to Agile testing, because I’ve seen testing as the black beast in many transformations to Agile because it was initially ignored.

Read more

Optimism For The Gelsinger Era At VMware

Glenn O'Donnell

The most notable news to come out of the VMworld conference last week was the coronation of Pat Gelsinger as the new CEO of VMware. His tenure officially started over the weekend, on September 1, to be exact.

For those who don’t know Pat’s career, he gained fame at Intel as the personification of the x86 processor family. It’s unfair to pick a single person as the father of the modern x86 architecture, but if you had to pick just one person, it’s probably Pat. He then grew to become CTO, and eventually ran the Digital Enterprise Group. This group accounted for 55% of Intel’s US$37.586B in revenue according to its 2008 annual report, the last full year of Pat’s tenure. EMC poached him from Intel in 2009, naming him president of the Information Infrastructure Products group. EMC’s performance since then has been very strong, with a 17.5% YoY revenue increase in its latest annual report. Pat’s group contributed 53.7% of that revenue. While he’s a geek at heart (his early work), he proved without a doubt that he also has the business execution chops (his later work). Both will serve him well at VMware, especially the latter.

Read more

Dell Is On A Quest For Software

Glenn O'Donnell

 

One of the many hilarious scenes in Monty Python and the Holy Grail is the "Bridge of Death" sequence. This week's news that Dell plans to acquire Quest Software makes one think of a slight twist to this scene:

Bridgekeeper:   "What ... is your name?"
Traveler:           "John Swainson of Dell."
Bridgekeeper:   "What ... is your quest?"
Traveler:           "Hey! That's not a bad idea!"

We suspect Dell's process was more methodical than that!

This acquisition was not a surprise, of course. All along, it has been obvious that Dell needed stronger assets in software as it continues on its quest to avoid the Gorge of Eternal Peril that is spanned by the Bridge of Death. When the company announced that John Swainson was joining to lead the newly formed software group, astute industry watchers knew the next steps would include an ambitious acquisition. We predicted such an acquisition would be one of Swainson's first moves, and after only four months on the job, indeed it was.

Read more

Suddenly, Dell Is A Software Company!

Glenn O'Donnell

The Dell brand is one of the most recognizable in technology. It was born a hardware company in 1984 and deservedly rocketed to fame, but it has always been about the hardware. In 2009, its big Perot Systems acquisition marked the first real departure from this hardware heritage. While it made numerous software acquisitions, including some good ones like Scalent, Boomi, and KACE, it remains a marginal player in software. That is about to change.

Read more