Comparisons between Watir and SilkTest
By Tuyet Cong-Ton-Nu, 21 July 2005
Two scripts are being used to do the comparison. --> What type of an app was being tested? Where are the samples of these 2 scripts?
We used the following criteria to compare the two tools:
- Elapsed time
- Reliability
- Maintainability
- Lines of Code
Each script was run 15 times to get the elapsed time and test out reliability.
| Import script | Watir | SilkTest |
|---|---|---|
| Lines of Codes | 60 lines of code | 200+ |
| Elapsed Time (average of 15 runs) | 25 seconds | 60 seconds |
| Reliability | No failure when ran 15 times in a row | 87% with 15 runs |
| Maintainability | All changes are readily "seen" via View Source or getHTML command. Compare files can be done and changes can easily be made. | Changes in B1 code require updates to the page declarations. This can take between 10 and 30 minutes. |
| Analysis/Positions script | Watir | SilkTest |
|---|---|---|
| Lines of Codes | 105 lines of code | 250+ |
| Elapsed Time (average of 15 runs) | 52 seconds | 120 seconds |
| Reliability | No failure when ran 15 times in a row | 80% with 15 runs |
| Maintainability | All changes are readily "seen" via View Source or getHTML command. Compare files can be done and changes can easily be made. | Changes in B1 code require updates to the page declarations. This can take between 10 and 30 minutes. |
Comments (1)
Feb 11, 2009
Bill Mosteller says:
I found this comparison fascinating, and thank the author very heartily for it. ...I found this comparison fascinating, and thank the author very heartily for it.
I used SilkTest from 2002 through 2008, and somewhere I think in 2008 a new release came out that increased its typing speed dramatically. Studying the time stamps on this article, I think probably the reported trials were run before the speed-up release (whose identification escapes me) was available, so SilkTest might now be faster. On the other hand, I don't usually worry about speed issues, as I figure that mostly I'm at the mercy of the application's response time anyway.
The reliability information was of great interest. Despite my efforts at robustness, I was never able to get my SilkTest scripts to stop having random inexplicable failures. I've always assumed that commotion from the underlying Windows environment (e. g., focus being stolen from the window I'm scripting on) was the underlying cause, and this data is consistent with that theory. Thanks again.