» Publishers, Monetize your RSS feeds with FeedShow: More infos (Show/Hide Ads)
As you may have noticed, things are a bit slow around QA Podcast. That's because QA Labs--the folks who run QA Podcast--was recently acquired, and we've been pretty busy with that process.
Once things settle down, we hope to relaunch QA Podcast with new sessions. If you'd like to be notified when we relaunch, enter your email address in the 'Keep Up to Date' box to the right. We promise to only email you to let you know that we've posted new episodes.
Alternately, you can always subscribe to our RSS feed. Thanks for your patience!
As you may have noticed, things are a bit slow around QA Podcast. That's because QA Labs--the folks who run QA Podcast--was recently acquired, and we've been pretty busy with that process.
Once things settle down, we hope to relaunch QA Podcast with new sessions. If you'd like to be notified when we relaunch, enter your email address in the 'Keep Up to Date' box to the right. We promise to only email you to let you know that we've posted new episodes.
Alternately, you can always subscribe to our RSS feed. Thanks for your patience!
Today we talk about that old chestnut (or bugbear, if you like) of the software industry: performance testing. We discussed this subject back in August with Keith Moon, but it's a big topic, so we thought we'd revisit it.
Wolfgang is in the studio talking with Larry Ng. Larry is a senior QA specialist at QA Labs.
Here are a few quotes from today's show:
"When the customer asks us to do performance testing, what they have in mind is usually the user response time and how the system performs under load--as in system resources and database performance. So it's not always easy to define what a 'meaningful result' means--it's not easy to define what measurements we need to gather."
"The bottom line is that when we're doing performance testing, we try to generate the load to the system by whatever means. We don't need to necessarily adhere to business scenarios. We can split it up into multiple scenarios and still generate the same load."
"In terms of metrics, most people want to see a solid result about user response time and system performance--how much CPU got used, is there enough memory to handle the load and so forth."
Technorati tags: qa quality+assurance software+testing qa+labs performance+testing load+testing server+load podcast
audio/mpeg (20 309 ko) 
This week we're joined by Ian Langworth and Chromatic. They're the authors of Perl Testing: A Developer's Notebook. Predictably, the topic du jour is Perl and its role in QA.
This was recorded over the phone, and there's a fair bit of hiss on the line. Apologies for that--we figure that the sound quality degrades as we get multiple parties in on the call.
Here are some quotes from today's show:
"If you have other applications, such as Web applications, that Perl has testing harnesses, Perl would be an ideal tool for that sort of thing...My brother works for a large computer company testing networking equipment and things like that. It's very helpful to be able to send out a million packets a second with Perl and analyze what comes back--look at timings, look at errors, things like that."
"The advantages of testing with perl stem from the advantages of the language. For example, Perl is loosely-typed. So, even with the basic testing libraries, being able to say is the result of this action this, and not having to worry about types or other factors. It makes testing really readable, really logical and easy to interpret."
"Perl is a very appropriate language for doing extreme programming in. From the standpoint of testing or test-driven development, you want to verify the assumptions you're making in your head. Test-driven design enables you to sit down and write a test, and then write just enough code to make that test pass."
Links for this conversation's topic:
- The book on O'Reilly, and Ian and Chromatic's author pages
- Perl.com
- A list of Perl modules in the Test namespace
- An article: Testing Files and Test Modules
- An interview with Ian
- Ian's homepage
- Chromatic's homepage
audio/mpeg (20 408 ko) 
On today's show we're on
the phone with the creator of Extreme
Programming (XP) and one of the founders of the Agile Manifesto, Kent
Beck. Kent and Wolfgang discuss the relationship between XP and QA.
This will be our final podcast of 2005. Stay tuned, as we'll be returning in January with more testing topics. As always, if you've got feedback or a suggestion for a future episode, leave a comment or drop us an email.
Here are some quotes from today's show:
"10 years ago, when I started talking about the precursors to XP, programmers weren't writing tests, ever. But, there are a couple of trends that contribute to changing this. One is shorter and shorter software release cycles. If you have to release software once a month, you don't have time for a two-month QA cycle...The other large-scale trend is toward asking for more visibility and accountability in the software development process."
"In XP-style development, two people typically write code together. They're having a conversation--the keyboard is going back and forth. So you already have that second set of eyes looking at the code. Say you're working on the code, I might be thinking about the next test that we could be writing."
"You can't know what you don't know. Part of the XP philosophy is keeping your options open as long as possible. Rather than having a big architecture developed up front. In XP, you would develop the architecture along with the code along with the tests for the non-functional requirements."
Links for this conversation's topic:
- Getting Started with XP (PDF)
- Testing Resources for Extreme Programming
- Extreme Programming Explained
Technorati tags: quality assurance, qa, software testing, qa labs, podcast, requirements, agile development, extreme programming, kent beck
audio/mpeg (15 436 ko) 
Today we're talking to Fergal McGovern, CTO at Steeltrace. They're an Irish company that makes tools that help developers capture, document and
model business, system and test requirements. Wolfgang and Fergal talk about the relationship
between requirements and testing.
Due to some URL-related fussiness, the iTunes directory hadn't been updating the last couple of podcasts. We diagnosed and resolved this issue with help from a listener--thanks a bunch, Chris! iTunes users may have noticed that episodes #6 and #7 arrived immediately after the issue was resolved, but now you should be all up to date.
Here are some quotes from today's show:
I'm quite intrigued about the way you position the starting point as being goals. A lot of people talk about requirements being the starting point. But you talk about goals or needs that first get translated into requirements and then finally have an impact on how the system is tested.
Unfortunately, simple communication is rarely simple in the real world. I'll give you a case in point. What we found recently with Agile [software development] on one engagement that I Was involved in, a major US-based corporation. They were deploying Agile--specifically extreme programming (XP). I asked a senior project manager what the biggest challenge was with XP. Her answer was "no documentation".
We frequently see business analysts play crucial roles in QA. On the other hand, that can often be a bad thing, because the analyst is too close to the actual domain. Just like a coder doing testing, they're almost contaminated by what's in their head as the primary flows and what they should focus on...whoever's playing the role needs to be quite diligent.
Links for this conversation's topic:
Technorati Tags: quality assurance, qa, software testing, qa labs, qa podcast, requirements, user acceptance, traceability, agile, xp, programming
audio/mpeg (16 953 ko) 
We've
been away, but we're back! This week sees a change of pace here at QA Podcast,
with Wolfgang and Sophos's
Luke Closs talking about Selenium,
a test tool for Web applications that runs in your browser.
Selenium is an open-source tool developed by some smart people at ThoughtWorks. It's only at version 0.6, but it's under active development, so we figure a version 1.0 is coming sooner or later. You can download Selenium if you want to take it for a test drive (with apologies for the lousy pun).
Here are some quotes from today's show:
"What pushed us towards Selenium: First, the fact that it runs in the browser. What you're testing, is actually what the user would see. Whereas with other tools, they're just a script that grabs the page content. Because the page is loading in the browser, you'll be aware of any problems in the page. Secondly, we liked the platform independence--it handled all the browsers we wanted to support."
"The core of Selenium is written exclusively in Javascript and HTML. It works in two modes. The first is a table-driven mode, where you write test cases in simple HTML tables--each table would have three columns, say, a command, some option and another option. Selenium also has a mode called Driven, where you can have a Python, .Net, Java or Ruby script actually driving what Selenium does."
"I don't know why anybody didn't think of this before. The QA developer that I talked to said 'Wow, this is so good. I've spent so many hours doing regression testing of our Web managers.'"
Other comments on Selenium from around the Web:
- Selenium - Initial Impressions
- Browser-Based Testing Survey
- Web app testing with Python part 2: Selenium and Twisted
- Making Selenium Tests More Dynamic
- Selenium (if you can read German)
- Watir and Selenium
Technorati Tags: quality assurance, qa, software testing, qa labs, sophos, thoughtworks, selenium, test tool, web applications, qa podcast
UPDATE: We've just changed the location of this episode's MP3 file, to try to troubleshoot an iTunes-specific issue some users are experiencing. The content of the file has remained the same.
audio/mpeg (14 949 ko) Due to a couple of scheduling mishaps and a vacation (France was lovely, thanks), we're running behind on our Oct. 15 podcast, so it may be delayed for a few days.
Due to a couple of scheduling mishaps and a vacation (France was lovely, thanks), we're running behind on our Oct. 15 podcast, so it may be delayed for a few days.
Due to a couple of scheduling mishaps and a vacation (France was lovely, thanks), we're running behind on our Oct. 15 podcast, so it may be delayed for a few days.
In today's session, Wolfgang talks with Dr. Jeff Joyce about testing safety-critical systems. That's a fancy term for things that can put lives at risks, such as medical hardware, air traffic control systems, military applications and automobiles. Their discussion, interestingly, includes reference to an early talk with James Bach on exploratory testing.
Jeff is a co-founder of a newly formed Vancouver-based company, Critical System Labs, that specializes in the development of safety-critical software-intensive systems. He formerly worked as a Senior Systems Engineer with Raytheon, leading the safety analysis of the Canadian Automated Air Traffic System (CAATS) and a related military system.
Here are a few quotes from today's conversation:
One of the key standards in the area of safety-critical systems and that's particularly important in the aerospace and automotive industries is something called D0178B. That really emphasizes requirements-based testing and scripted testing that differs significantly from exploratory testing.
A common mistake that's made is that people will specify a bunch of safety requirements, and they'll decide that the only testing that needs to be done is to verify those safety requirements. That really misses out on understanding where all the vulnerabilities of the system are.
80% of the defects are typically in 20% of the code. So, if you apply smart testing and you can find the 20% that is most vulnerable, and you focus your testing there, you'll get a much higher return on investment or results than if you blindly or stupidly test like a machine.
Technorati Tags: jeff joyce, safety critical systems, qa, quality assurance, qa labs, testing, software, hardware, medical, automotive, aerospace, military
UPDATE: We've just changed the location of this episode's MP3 file, to try to troubleshoot an iTunes-specific issue some users are experiencing. The content of the file has remained the same.
audio/mpeg (14 684 ko) 
In the studio today we've got Dan Zrymiak. He's an expert in risk management, which (not coincidentally) happens to be today's subject.
Dan has over 12 years of QA experience in software, medical products, and business services. He's a Six Sigma Black Belt with professional certifications in Quality Management, Software Quality, Quality Engineering, and ISO Quality Auditing.
Here are a few quotes from today's conversation:
A lot of medical and financial industry products are defined not only the features of the products, but also the risks they prevent. In the medical industry, if your product fails, there are lives at stake. In banking, as we all know, some of the major banks, if there are any malfunctions, people don't get paid, they pay their mortgage twice...a lot of unfortunate events.
The squeaky wheel gets the grease. In product development, the squeaky wheel tends to be marketing and sales. Controlling risk is perceived as secondary, until something goes wrong.
There has to be traceability from requirements through all the phases of the development life cycle. The design, the architecture, the construction all play a very pivotal role in ensuring the mitigation of that risk is preserved in the product.
Technorati Tags: danzrymiak, qapodcast, podcast, qualityassurance, software, development, risk, management, qalabs
audio/mpeg (17 158 ko) In the studio today we've got Dan Zrymiak. He's an expert in risk management, which (not coincidentally) happens to be today's subject.
Dan has over 12 years of QA experience in software, medical products, and business services. He's a Six Sigma Black Belt with professional certifications in Quality Management, Software Quality, Quality Engineering, and ISO Quality Auditing.
Here are a few quotes from today's conversation:
A lot of medical and financial industry products are defined not only the features of the products, but also the risks they prevent. In the medical industry, if your product fails, there are lives at stake. In banking, as we all know, some of the major banks, if there are any malfunctions, people don't get paid, they pay their mortgage twice...a lot of unfortunate events.
The squeaky wheel gets the grease. In product development, the squeaky wheel tends to be marketing and sales. Controlling risk is perceived as secondary, until something goes wrong.
There has to be traceability from requirements through all the phases of the development life cycle. The design, the architecture, the construction all play a very pivotal role in ensuring the mitigation of that risk is preserved in the product.
Technorati Tags: danzrymiak, qapodcast, podcast, qualityassurance, software, development, risk, management, qalabs
Today we're very fortunate to be joined by exploratory testing guru James Bach. He's an accomplished QA expert, who trains and consults through his company Satisfice. He's also one of the authors of Lessons Learned in Software Testing: A Context-Driven Approach. As you might guess, Wolfgang and James talk about exploratory testing.
Today's session runs a little bit long at 25 minutes, but we're sure you'll find it interesting enough to stick with it. This is also our first phone interview. We were a little concerned about audio quality issues, but it turned out pretty well.
Here are a few quotes from today's conversation:
"This idea that 'why don't we just figure out all the tests, and then we'll write them down and then we'll do all those tests' is a formula for bad or at best mediocre testing."
"Testing textbooks seem to have very little say about how test projects really do work or really can work. I saw a lot of folklore and mythology but when I tried to use them, I found that it didn't work on the commercial software projects that I wasn't part of."
"Exploratory testing is simply what happens when I place something in front of you and say 'test this right now'. When people respond to that, they're doing exploratory testing. It's simultaneous learning, test design and test execution."
Links for this conversation's topic:
Technorati Tags: jamesbach, qa, podcast, software, qualityassurance, programming, exploratorytesting
audio/mpeg (17 870 ko) 
Today we welcome Keith Moon to the show to talk all about performance and load testing. Keith's day job is at a large Vancouver software company that builds business intelligence software.
Here are some quotes from today's show:
"I would think being a performance tester would be a very exciting job. Because, as you say, it requires a understanding of the system, architectural thinking, a sharp detective mind to understand where the bottlenecks come from."
"There's this game called Mastermind which is basically a troubleshooting game. That's what testing is like, except for this is on a grander scale, when you have a complex, interdependent system. The most fun is when the product comes together and you can start taking it to the upper extremities of load and start to monitor how the whole ecosystem works together."
"The weakest link is what determines the performance of the system, and there's loads of opportunities for weakest links to slip through until you do the full system test."
Technorati Tags: qa, quality assurance, software, performance testing, load testing
We're claiming My Odeo Channel (odeo/3d1047c26241d9b6).
audio/mpeg (19 720 ko) 
This week's show features two software professionals from QA Labs. Ciprian Ticea, Director of Projects and Technology and Wolfgang Strigel, President, talk about the state of the art in testing, best practices for your organization and ways to improve your QA department.
Here are some quotes from today's show:
"Overall, there's a rule of thumb that testing should take about 25% of the total effort for the software development lifecycle and very often...we find that a lot less is spent. Even in cases where companies spend more, we find that it isn't spent very efficiently."
"The analogy I use is that, for a city, the standards would be in the form of bylaws and that would help builders to know what they have to build and yet allow enough flexibility to be creative."
"I use this defect incidence curve. Normally this curve goes initially up, and then it tapers off. A company can define a threshold at which point in time it is acceptable for them to stop. That's one way to decide when you're done with testing."
Links for this conversation's topic:
audio/mpeg (14 13 ko) 
Welcome to our first podcast. This show features Geoff Flamank of Clarrus Consulting and Wolfgang Strigel of QA Labs.
This show discusses outsourcing your QA, including a discussion of types of outsourcing organizations and guidelines for best practices.
Here are some quotes from today's show:
"There are a bunch of reasons why people outsource testing. Testing is lumpy in terms of its resource needs. Companies often find it painful to ramp up and down their test teams."
"There are actually more Indian companies that are CMM level 5 certified than in all of the rest of the world. So the Indians are very process oriented, and they do this because they think it's a competitive advantage. It's like the Indian Olympics."
"A lot of people don't know, that as a general rule of thumb, 1 to 1.5% of the total lines of code have bugs."
Welcome to our first podcast. This show features Geoff Flamank of Clarrus Consulting and Wolfgang Strigel of QA Labs.
This show discusses outsourcing your QA, including a discussion of types of outsourcing organizations and guidelines for best practices.
Here are some quotes from today's show:
"There are a bunch of reasons why people outsource testing. Testing is lumpy in terms of its resource needs. Companies often find it painful to ramp up and down their test teams."
"There are actually more Indian companies that are CMM level 5 certified than in all of the rest of the world. So the Indians are very process oriented, and they do this because they think it's a competitive advantage. It's like the Indian Olympics."
"A lot of people don't know, that as a general rule of thumb, 1 to 1.5% of the total lines of code have bugs."
audio/mpeg (16 968 ko) QA doesn't get enough attention. When a software project is late (and which ones aren't?), testing is the first thing that gets cut. The QA department often seems like the domain of new recruits, cutting their teeth before they can move on to more exciting projects.
We think QA is important. Really important. That's why we started QA Podcast, a bi-weekly show in which experts discuss quality assurance issues in software. Each show will be 15 to 20 minutes, and feature an informal conversation between two or more software professionals. They're all about technical people having indepth conversations about QA strategies, issues and technologies.
For more information, visit our About page.
QA doesn't get enough attention. When a software project is late (and which ones aren't?), testing is the first thing that gets cut. The QA department often seems like the domain of new recruits, cutting their teeth before they can move on to more exciting projects.
We think QA is important. Really important. That's why we started QA Podcast, a bi-weekly show in which experts discuss quality assurance issues in software. Each show will be 15 to 20 minutes, and feature an informal conversation between two or more software professionals. They're all about technical people having indepth conversations about QA strategies, issues and technologies.
For more information, visit our About page.