Rubymine Rocks

Of course I missed another post last week. Amazing how soccer, work, life in that order controls your time. Any rate I was not sure what to post this week. I have been extremely busy learning more about JMeter, BrowserMon and performance testing and creating new automated regression tests.

I decided to pass on writing about performance testing and make a couple statements as to why Rubymine ROCKS.

Number one reason Rubymine ROCKS is that when you are working with Ruby and XPath you can pause the script at the point of failure. Then you highlight the XPath code that is failing and hit F11. Now edit the day lights out of that XPath until you get the XPath to work.

Typically I have a copy of the target page up in another Firefox instance and viewing the HTML closely.

A second reason I like Rubymine is the ability to quickly see what methods are available and command-B to see what the methods are doing (is the correct term intellisense?).

Probably not the last reason I find, but when you want to refactor a term you can find it and replace it for the entire project. Most IDE do this, but I like how Rubymine does it.

I also like how Rubymine tells me gems are not attached. I like the fact that you can can place code in test mode and test one method at a time. And I also really like the fact that they fixed the rename functionality so the name actually appears and I can make minor changes versus having to type in an entire new name.

I have definitely become a fan of Rubymine. Go Horns! I mean RubyMine!

Rocking the Ruby code!

Selenium Brittleness

This will be a quick post, but I wanted to share a little tip. When using the Selenium IDE you often get xpath that looks similar to this.

browser.click "//div[@id='location-bar']/div[2]/div[4]/a"

Well when attempting to do cross-browser testing this Xpath may not work for all browsers and relying on an index value that may change is a bit brittle.

One way to adjust this Xpath would be to make it look like this.

browser.click "//div[@id='location-bar']/div[@class='this']/div[@class='that']/a"

Of course everything is dependent on how the object on the page is represented. Inspect the HTML closely.

Here are another helpful hints on clicking links.

browser.click "//a[contains(@href,'stuff')]"
or
browser.click "//a[contains(text(),'otherstuff')]"

I hope you find these pointers helpful.

Happy coding!

Using Selenium GRID

For the past 3 months I have been on a journey with a team of extremely sharp developers and test engineers to build out a test automation framework. We settled on using Selenium and the Selenium GRID.

There are so many framework options, but I have to say at this point Selenium GRID rocks.

Here are the main pieces to the puzzle.

• MacBook Pro
• Use Rubymine for development
• Now using Selenium GRID 1.04
• Code base is Ruby leveraging the use of Rake
• VMware Fusion
• Team City
• Rally

Locally we can now develop a script and execute it against Safari on MacOSX, Firefox 3.0, Firefox 3.5, IE 6.0, IE 7.0 & IE 8.0. We can execute tests in parallel. I love watching 6 - 10 firefox icons bouncing up and down on my MacBook.

Tests are launched by the test teams in production environment using a job configured in Team City. One click of a button kicks of the test suite. Soon teams can choose to run only smoke test, only regression tests, or all. In production the tests are executed in parallel for each browser configured, which us currently IE 7.0, IE 8.0, Safari on Mac Mini, Firefox 3.0 and Firefox 3.5. After the test execution the test leads can see results input directly into Rally test cases or they can view a nice HTML report. They can also view the results in Team City.

If you cannot tell, I love this stuff! It is really fun to develop in this environment because you can see immediate cross browser results.

Yes I still struggle as a developer and create some brittle tests. But the good news is collaborating with a great team we quickly re-factor brittle code.

Our automation targets a complex and ever changing environment, but having a dedicated automation team is the key to success.

Happy Testing!

JMeter to the Rescue

Recently I was involved in a high priority performance testing effort. The tools at hand were Webload and WAPT. I have previous experience with Webload so that is the initial path we took. The testing effort is relatively simple. The goal was to have concurrent users doing a series of gets and posts. Clarification the posts are actually http puts. I quickly could not find a way to get Webload to emulate an http put. Further complicating things was the system under test, SUT, requires secure header keys. Information on the internet on Webload seemed very limited and I was not able to rapidly find the answers to these two concerns.

The time line was limited so there was not bandwidth for a coding effort nor to purchase any new tool.

We quickly turned to JMeter. This is going to be a short post, but there are a couple of key advantages we had with JMeter.

Observed advantages:

• JMeter out of the box could handle puts
• JMeter worked well with RAKE
• Fast installation and short learning curve
• Easily extensible
• Ability to run headless (no GUI) via RAKE

Observed dis-advantantages:

• Executing tests with the GUI dropped threads as load increased due to utilization of machine java processes
• graphic listeners are of extremely poor quality
• A single instance appears to be limited to approximately 300 concurrent threads
  

The bottom line to this post is that JMeter came to the rescue and has sound potential for future performance projects.

There are some key action items for me.

• Evaluate the ability to distribute load across several machines
• Evaluate the ability of JMeter to interact with Web-based applications (ajax stuff etc...)
• Develop Ruby scripts to calculate and report out data metrics as the number of transactions increase.
• Understand the limitations of this tool
• Evaluate other open source plugins developed for JMeter

I am looking forward to working more closely with the development teams to expand the use of jMeter for performance testing.

Happy Testing!

Selenium RC plus RSpec

It has been an extremely long time since I posted to this humble blog. I have currently changed jobs and I am newly energized to learn, implement, and build out a test automation framework.

There are a zillion frameworks to choose from. I have been using the WATIR framework for several years as I had bandwidth. I never had the opportunity to learn and execute a full solution. I recently interviewed several teams at the new company and I found one common theme with respect to test automation. Every team has selected Selenium RC as the foundation.

Initially I attempted WATIR to execute the tasks required by the team. With the complexity of the web pages and solutions, I quickly learned that WATIR may not be the best solution.

One of the teams built a Selenium & RSpec framework. Although there is some learning curve for me, I am liking what I see.

As I go on this adventure I will attempt to do a weekly post on the good, the bad, and the ugly.

Keep on Testing!

Kung Fu Testing

“Colloquially, kung fu (or gung fu) alludes to any individual accomplishment or cultivated skill obtained by long and hard work.”1 What in the world does Kung Fu have to do with Software Testing? Last week we had a software release due to our client and it was one of those weeks, if something would or could go wrong, it did. The database became corrupted, the testing lab network became ultra-slow, and connectivity to our client site was intermittent. With all of these challenges we managed to battle through and get the release out on schedule, although with a couple of qualifications. During this crazy fray the lead developer coined our style of testing as “Kung Fu Testing”.

Not giving the phrase much thought at first, I liked it. After more thought I was pondering why would she draw the analogy of our testing style to that of Kung Fu. Was it because we did a good job of dissecting and targeting the critical risk areas to test or perhaps the long hours and dedication toward meeting the delivery date? Was it because out of self defense we effectively adjusted our testing strategy or was it the precision of the weapons selected to accomplish the task at hand?

I conclude that it was a little bit of everything stated above.

Dissection - We carefully chose to execute test cases and test plans in a precise order of risk.

Dedication – We worked into the wee hours of the morning for several nights in a row resurrecting our test environment.

Self Defense - We completed testing of all vital functions. We then executed regression test suites based on efficiency.

Weapons – Our WATIR framework allowed for some minor adjustments to the methods and we could readily execute the system regression tests.

We executed almost 600 test cases in only three days. Typically we allocate 5 days, but several days were needed to resurrect the testing lab. The precision in which the test plans were executed gave us the confidence to deliver the release on schedule. One test plan could not be executed, but we explained the risk in the release notes. After the delivery of the code we were able to execute this test plan further mitigating any risk.

Having recently completed the AST course Black Box Software Testing Foundations, I hope to expand upon this concept of Kung Fu Testing to be oracle and heuristic focused. There is always room for improvement, but accomplishments happen because of cultivating testing skills and hard work.



1 Wikipedia

To act or Not to Act

Quick question for all test enthusiasts.

1. You are browsing the web.
2. You find a site, perhaps a major casino or an airline
3. The site is not user-friendly
4. Site has a major defect

Do you act an attempt to report the issue?

Or

No act and accept the flaw?

I am finding myself more and more documenting my experience and submitting it to the site. Which brings me to another point in that often it is difficult to find on the site the appropriate link to submit a defect.

Recently I sent a browser compatibility observation to a company and the CEO replied with a Thanks!

Comment and let me know what you would do.

Long live WATIR!