All Posts

  • One of the important deliverable in software testing are Bug reports. Writing a good bug report is an important skill of a software tester. In order to document a well-written bug report, tester requires a combination of testing and communication skill. The bug report is a medium of communication between tester and developer when the tester uncovers a defect. The bug report explains the gap between the expected result and the actual result.

    A bug report should have the following: 

    • The Title

    • Steps To Reproduce

    • Test Data

    • Expected and Actual Results

    • Attachments

    The Title

    A good title helps reduce duplicate issues and accurately summarize the issue. Include the specific name of the components involved in your bug in the title. A good summary will not be more than 50-60 character.

     You can avoid generic problems in the title. For example: 

    • ABC is not working properly

    • Issue with the page

     When we define a title we should specify what makes it “not working”. 

    Bad – “Application crashed”

    Good – “Canceling from Change Password dialog caused application crash”

    Bad : Issues with GUI on navigation bar.

    Good : Navigation bar is wrapping to a second line.

    Steps To Reproduce

    This is body of the report. This section tells how to reproduce the bug. We should keep the section concise and easy to read. The number of steps should be short and to the point. It is better to write prerequisite to reduce the number of steps. It’s a good exercise to reproduce the bug by following the steps you’ve just outlined. This will help ensure you’ve included everything the developer will need to reproduce it as well.

    Test Data

     

    If the bug is specific for the particular scenario it is better to give the test data, so that developer can recreate the scenarios.

    Expected and Actual Results 

    When describing expected results, explain what should happen, not what shouldn’t happen.

    Instead of writing “The app got crashed”, we can write as “The user should take to XYZ screen".

    When describing actual results, describe what did happen, not what didn’t happen.

    Instead of writing “The user wasn’t taken to the page”, we can write as “The user remained in the ABC page". 

    Attachments 

    Attachments add the bug value by offering proof. The attachment can be images, videos or log files. 

    Images 

    Images are an essential part of the bug report. The bug reports should be effective enough to enable the developers to reproduce the problem. Screen shots should be a medium just for verification. 

    • If you attach screen shots to your bug reports, ensure that they are not too heavy in terms of size. Use a format like jpg or gif, but definitely not bmp.

    • Attach the image files directly to the report. Don’t put images in a Word document or a zip file.

    • Highlight the areas of bug in the image. 

    Video

    Video should be provided if the steps are complex. Actions in the video should match the steps listed in the bug report. Videos should be trimmed to only show the bug.

    Log Files

    Make it a point to attach logs from the logs. This will help the developers to analyze and debug the system easily. If the logs are not too large, say about 20-25 lines, you can paste it in bug report. But if it is large enough, add it to your bug report as an attachment. Avoid proprietary file types (like .docx). Use .txt instead. 

     

  • TestLink is a web-based test management system that offers support for test cases, test suites, test plans, test projects and user management, as well as various reports and statistics. It is developed and maintained by Teamtest that facilitates software quality assurance.

    How to work with TestLink

    1. Create a Project
    2. Create Test Cases (Test Suites) for this Project
    3. Create Test Plan
    4. Specify Build of the Project you are going to test
    5. Add Test Cases to the Test Plan
    6. Assign Test Cases to Test Engineers
    7. Execute Test Cases (Test Engineers)
    8. See Reports and Charts

    Additional facilities 

    • Assigning Keywords (we may form a group of Test Cases for Regression tests)
    • Specifying Requirements (we may bind them with Test Cases in the many-to-many relation and see if our Test Cases cover our requirements)
    • Events log (you can see here the history of all the changes)


    STEP 1. CREATE A PROJECT

    To create a project, go to the Test Project Management section:


    STEP 2. CREATE A PROJECT - IMPORTANT FIELDS

    • Name
    • ID (used for forming a unique Test Cases ID) E.g. FT-03 means that the Test Case is created for Fenestra project and it has ID=3
    • Project Description (what is the aim of the Project, what is the target group, what is the business logic, what is the Test Environment)

      Enhanced features:

    • Requirements feature – we may specify requirements and see if they are well-covered by Test Cases
    • Testing priority – we may assign priority to Test Cases (high, medium, low)
    • Test Automation – we may specify whether the test should be performed manually or automatically

      You can now set this project here, like in Mantis, in the top right corner


    STEP 3. CREATE TEST CASES

                             Or here: 


    STEP 4. CREATE TEST CASES - CREATE TEST SUITE


    • Test Case Title
    • Summary
    • Preconditions
    • Execution type (manual or automated)
    • Test importance (High, Medium or Low)


    • We may also import and export Test Suites and Test Cases (in the .XML/XLS format):
    • We import them from one project

    •  And export the file in other



    STEP 5. SPECIFY TEST PLAN

    • TestLink will not allow you to execute Test Suites if you do not create a Test Plan and specify Test Build.
    • How to do that ? Let’s begin from the Plan


    • Current Test Plan will appear in the top right corner

    STEP 6. SPECIFY BUILD

    • After you’ve added a Test Plan menu, the adding Test Build appears.

    • Add a new build there


    STEP 7. ADD TEST CASES TO THE PLAN

    • Unfortunately, only Test Cases, not Test Suites or the whole Test Specification can be added to a Test plan. So, until you don’t select one separate TC, the button “Add to Test Plans” will not appear.

    • Then you can choose what Test Plans you want to add the selected TC to.


    STEP 8. ASSIGN TEST CASE EXECUTION TO TESTERS

    • Before assigning TC to testers you should create a DB of users with appropriate roles here.
                                    
    • Add the users you need filling in the form.

     

    • Then you can assign TC execution here.

     

    • You can assign test cases to testers and send them email notifications.

     

    STEP 9. EXECUTE TESTS

    • To start executing tests, Test Engineer should go to test Execution section.

     

    • Then choose a TC.

     

    • You may also connect TestLink with our bug-tracking system Mantis, then during execution you will see as below.

     

    • After click on “Create New Bug”, to create the bug using mantis user interface and reorganizing the window

     

    • Test engineer writes the issue ID on Testlink

     

    • It looks like this after saving

     

    • Execution history is being saved

     

    STEP 10. SEE REPORTS AND CHARTS

    • After test case execution is finished you may see the results of it using Test Reports section

     

                                     Or here:

     

    • You can see the following page

     

    Test Plan Report - the document has options to define a content and a document structure. You may choose the info you want to get.

     

    • Test Plan report (part of it)

     

     

    • The document 'Test Report' has options to define a content and document structure. It includes Test cases together with test results.

     

    • Test result matrix

     

    •  Charts

     

    Charts – results by tester (there are only unassigned test cases in the
    diagram)

     

     Charts – Results for top level suites:

    1. Log in the application

    2. News module

     


    Blocked, Failed and Not Run Test Case Reports
    These reports show all of the currently blocked, failing, or not run test cases.
    E.g.

     

    General Test Plan Metrics
    This page shows you only the most current status of a Test plan by test suite, owner and keyword.

     

    • Query metrics – work like filters in Mantis

     

    • Requirements based report

    If we have some requirements specified and have connected them with TC we can see the following report:

    ADDITIONAL FACILITIES - ASSIGNING KEYWORDS

    Go to the “Assign Keywords” section

    Select some Test Suite and then you will be able to go to “Keywords Management”


    Add keywords if there are no KW at all, or if there are no KW you need

     

    • Now you can add Keywords both to Test Suites & Test Cases, either all the Keywords (>>) or only one KW (>)

     

    • Then you will be able to see such a useful chart demonstrating the Results by KW

     

    • You can open the section in this way

     

                                  Or in this:

     

    • Requirements Specification adding

     

    • Then we create Requirements

     


    Pay attention that there are different types of the Requirements



    Then assign requirements to Test Cases



    Select Test Suite or Test Case and assign it to 1 or more requirements
    (R. can be assign to TC in the relation many-to-many)

     


    1. We have all the documentation structured and organized.
    2. We solve the problem of version control.
    3. We can control the testing process (Events log + different kinds of Reports)
    4. We can see if all the requirements are covered with Test Cases.
    5. We can select Test Cases for Regression Testing.
    6. We can see the results of testing in a very clear and easy-to-use form.

    Demo Site

    http://demo.testlink.org/latest/login.php

     

  • Sharable Content Object Reference Model (SCORM) is a repository of technical standards and specifications for web-based e-learning. It is an XML-based framework used to define and access information about learning objects, so they can be easily shared among different learning management systems (LMSs).

    SCORM was developed in response to a United States Department of Defense (DoD) initiative to promote standardization in e-learning. DoD was frustrated by the problems they encountered when trying to share distance learning courses among different learning management systems used within the Department, so in 1997 they formed the Advanced Distributed Learning (ADL) specification group to create a way to make learning content portable across various systems. ADL created the first version of SCORM, which originally stood for Shareable Courseware Object Reference Model. It was designed to facilitate moving course content and related information (such as student records) from one platform to another, to make course content into modular objects that can be reused in other courses, and to enable any LMS to search others for usable course content.

    The current official version is 1.2. SCORM specification does not cover all aspects of a learning enterprise; for example, it does not specify how tracking information is stored and what reports are generated, what pedagogical or training models should be used, or how learner information is compiled.

  • Installing Burp's SSL certificate in your browser

    One of the functions of SSL is to authenticate the identity of webservers. To intercept traffic between your browser and webservers, Burp needs to break the SSL connection. This causes a security warning in your browser, because it detects that it is not communicating directly with the authentic web server. Burp generates an SSL certificate for that host which is signed by the CA certificate. Burp's CA certificate can be installed as a trusted root in your browser, so that the per-host certificates are accepted without any alerts. 

    Installing Burps SSL certificate is detailed in the following procedures.

    Browser making an SSL connection.

    Fig_1.jpg

    Burp is to break the SSL connection.

    Fig_2.jpg

    This causes a security warning in your browser because it identifies that its not directly communicating with the authentic web service.

    Fig_3.jpg

    This how the SSL warning looks like in different browsers:

    IE

    Fig_4.jpg

    Mozilla Firefox

    Fig_5.jpg

    Chrome

    Fig_6.jpg

    Safari

    To allow HTTPS websites to load properly they use their own certificate authority.

    Then creates an SSL certificate for each host you visit and signs this using the CA certificates.

    To prevent security warnings you should install Burp CA certificate as a trusted root in your browser. This will cause your browser to trust the SSL connections that it makes to Burp.

    Installing SSL certification is simple but the details depend on your browser.

    IE - should first launch IE as Administrator.

    Then using Burp as your proxy visit any HTTPS URL and click “Continue to this website (not recommended)”.

    Click on 'Certificates Error' and 'View Certificates'.

    Go to 'Certification Path' and select 'PortSwingger CA' and 'View Certificate'.

    This displays the Certificate screen.

    Click on 'Install Certificate' and in the wizard click 'Next'.

    Select “Place all certificates in the following store”, browse and select “Trusted Root Certification Authorities”. Click ‘Next’ and then 'Finish'.

    Confirm the action and restart IE. Now you will be able to visit any HTTPS URL without any warnings.

    Mozilla Firefox - Using Burp as your proxy visit any HTTPS URL.

    Click ‘I Understand the Risks’ and 'Add Exception'.

    View the certificate and from 'Details' tab select 'PortSwingger CA', 'Export' the certificate, save it somewhere and close all pop-ups.

    Go to 'Options'.

    From the pop-up select 'Advanced' –> 'Encryption' –> ‘View Certificate’. 

    Click ‘Import’.

    Select the certificate that you have saved and select the check box ‘Trust this CA to identify websites.’ Click ‘Ok’ on all pop-ups to close. Now you should be able to visit any HTTPS URL without warning messages.

    Chrome - It uses the certificate from the trust store of your host computer. Normally, if you install Burp using the default browser of your computer, chrome will use this.

    Using Burp as your proxy visit any HTTPS URL and click on ‘Proceed anyway’ and click on the broken lock and view the certificate information. This will link you to the relevant settings in your host computer.

    Click on ''PortSwingger CA'' certificate.

    Safari - Visit any HTTPS URL using Burp as your proxy. Click 'show certificate' and select 'Portswingger CA' certificate.

    Click on ‘Trust’ and select option ‘Always Trust’.

    Click ‘Continue’ and enter password, if you need to update the settings.

    Now you will be able to visit any HTTPS URL without warning messages.

     

     

     

     

     

     

     

     

     

     

  • Why Security Testing?

    With the cyber world becoming more-and-more vulnerable to attacks, security is something that cannot be compromised with. In order to develop secure applications, one really needs to use a security development lifecycle. Security must be considered and tested throughout the project lifecycle of any application.

    What are the processes involved in Security Testing?

    The security testing process involves evaluating the quantum of risks within the application under test and to point out the security vulnerabilities using various techniques and tools.  By this it is possible to ensure that there is no data theft, there is no unauthorized access or there is no security compromise that has been made through assistance. Security testing involves Vulnerability scanning, Security scanning, Penetration testing, Security Auditing and Security Review.

    Vulnerability scanning is usually performed using automated software tool which scans for the basic known vulnerability. It is an automated process performed using the vulnerability scanning tool like SARA. Next in line is Security scanning, where an assessment is done manually along with the software scanning. Although tools help in building a robust application, every tool has its own bottlenecks.  That is the reason, in addition to automated scanning one is required to perform manual testing, that is going through system responses, examining the log files, error messages, error codes and the like.

    The other aspect is Pen testing or Penetration testing. A real-time simulation environment is used to perform penetration testing. It is totally a Black Box, a hackers approach, the way in which Hackers use it but is done in a controlled environment. It is performed internally within the organization without breaching any security terms. Security Auditing is for specific control or compliance issue. Usually the compliance team or the risk evaluating team performs this security assessment. So, very frequent audits make the application more error prone and less vulnerable. 

    Finally, Security Review, which is static testing, wherein security review is perform as per the industry standards by reviewing documents, architecture diagrams and performing gap analysis. It is basically done for code reviews considering the architecture diagrams and documents which are very important. All these processes in security testing ensures that the applications developed are prone to any kind of security risks. 

  • To record HTTPS traffic, one needs to configure the browser proxy settings and JMeter proxy server.

    In the browser proxy server the following changes should be made.

    Go to the option tab in the firefox browser and click Advanced >> View Certificates >> Authorities.

    Check for the apache Software Foundation, JMeter Proxy Certificate and select that certificate, then click on the edit button and tick all
    the boxes and click Ok.

    When recording HTTPS with JMeter, do the following steps in JMeter proxy server:

    1.In HTTP Request Defaults: 

    Test Plan >> Thread Group >> HTTP Request Defaults 

    Server Name or IP[IP of the server] 
    Port Number[Port number of the server] 
    Implementation [HttpClient4] 
    Protocol [https] 
    Path [/] 

    2. In Recording Controller: HTTP Request 

    Server Name or IP[IP of the server] 
    Port Number[Port number of the server] 
    Implementation [HttpClient4] 
    Protocol [https] 
    Path [/]

  • What is JMeter?

    JMeter is an open source Java application designed to load test functional behavior and measure performance. JMeter is an Apache project used by a large open source community. Being a part of Apache, JMeter has comprehensive protocol coverage and scripting capabilities.

    What can you do with JMeter?

    JMeter is used to test performance both on static and dynamic resources such as static files, Java Servlets, CGI scripts, Java objects, databases, FTP servers and more. JMeter can be used to stimulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. JMeter can run on any environment/platform such as Windows, Linux, Mac, etc. Its multithreading framework is highly extensible and can be used to perform automated and functional testing.

    When compared to other testing application, 80% of what is required can be accomplished with a simple, intuitive GUI and not much of scripting is required to achieve that. Since JMeter is backed by such a large community, any use case that comes to mind probably has an answer within JMeter. With JMeter one can build test scripts that are realistic and accurate.

    What are JMeter limitations?

    JMeter is not a browser as it does not perform all the actions supported by browsers. To be more precise, it does not execute the JavaScript present in HTML pages nor does it render the html page as a browser does. It has limited support for JavaScript, AJAX and complicated frameworks. Also the total number of threads (virtual users) generated by the test plan should be less than 300 per engine. 

    One of the major limitation is that everything goes through a single console. Under heavy load the GUI consumes a lot of memory and the console server alone cannot sustain such a heavy load which leads to out of memory and disconnection logs.

  • Selenium 2 is the newest addition to the Selenium toolkit. This brand new automation tool provides all sorts of test features, including a more cohesive and object oriented API as well as an answer to the limitations of the old implementation. Selenium2Library is a popular Robot Framework test library. Selenium2Library runs tests in a real browser instance which works with most modern browsers and used with both Python and Jython interpreters.

    Selenium is a set of different software tools each with a different approach to supporting test automation. The entire suite of tools results in a rich set of testing functions specifically geared to the needs of testing of web applications of all types. One of Selenium’s key features is the support for executing one’s tests on multiple browser platforms.

    Selenium is highly flexible as there are many ways one can add functionality to both Selenium test scripts and Selenium’s framework to customize test automation. Since Selenium is Open Source, the source code can always be downloaded and modifiedOperations performed are highly flexible, allowing many options for locating UI elements and comparing expected test results against actual application behavior. This is perhaps Selenium’s greatest strength when compared with other automation tools

    • by User in on 2-Dec-2013

    Need for Cloud Testing – Issues and Challenges 

    Traditional testing has limitations like latency, performance, concurrency, planning issues and is way too expensive. Cloud testing is a big game changer and surpasses the challenges faced with traditional testing. It can be used to provide flexible, scalable and affordable testing environment at all times or on demand. 

    Cloud testing typically involves monitoring and reporting on real-world user traffic conditions as well as load balance and stress testing for a range of simulated usage conditions. The availability of virtual machines eases the process of setting up, using, reusing and running test setups. Complex test setups are available as stacked templates, making it easy to integrate complex automation into various processes to build complex cloud testing systems.  

    Cloud testing is a great fit for an agile environment. It can leverage for the whole life cycle of web or mobile application, right from the beginning of development until the application is into production. Today, if you need to generate thousands of virtual users to test a specific web application then the number of servers required for that test can be deployed within a couple of minutes. Best of all, you only need to pay those servers for the duration of the test thus making it more economical and viable. 

    Cloud testing is flexible enough such that it can be used for continuous performance testing.  Test maker runs tests in multiple cloud testing environments making it possible to manage performance from different geographical locations. Tester gets a real time testing experience of applications on browsers and OS rather than simulated environments. Cloud testing eliminates the cost of building and maintaining a test lab for load and performance testing. If a specific test environment is required, just use it via the cloud. There is no need to provision expensive and difficult to manage quality test labs. 

    Cloud-based testing poses different operational challenges in the real world scenario. One of the major challenges would be creating an on-demand test environment. The current cloud technology does not have any supporting solutions that will help cloud engineers build a cost effective cloud test environment. For scalability and performance testing, the current framework and solutions do not support the features such as dynamic scalability, scalable testing environments, SLA-based requirements and cost-models. Testing security is yet another concern inside clouds as security services become a necessary part in modern cloud technology. Engineers must deal with issues and challenges in security validation and quality assurance for SaaS (Software as a Service) and clouds. Integration testing in cloud may not be performed due to lack of time or additional integration cost which subsequently affects performance of the application.  

    Cloud testing is under constant evolution, continuously bringing in new opportunities and challenges. It reduces the need for hardware and software resources and offers a flexible and efficient alternative to the traditional testing.  Finally, moving testing to the cloud is seen as a safe bet as it does not include sensitive corporate data and has minimal impact on the organizations business activities. Migration of self-testing to the cloud would bring about a notion of test support as-a-service.

  • First off what is Robot Framework? - We know the definition. It is a generic test automation framework for acceptance testing and acceptance test-driven development (ATDD). But what does this mean? Well, put simply it is a platform that empowers QA Engineers and Testers to create test, automate and manage complex workflow scripts very efficiently with its natural language syntax. 

    It uses a keyword driven automation approach and has a BDD ( Behavior Driven Development)  and a Data Driven approach as well. Once the Robot Framework is installed, a set of standard test libraries are installed with it and are available for use. These test libraries can further be extended to include Python or Java libraries.  

    (There are quite a number of external libraries that can be used with this framework and Selenium2Library is one such publically available test library. It is a python implementation of Selenium WebDriver (tool for automated testing of websites and web applications) for use with Robot Framework. It has 126 keywords that can execute Selenium Commands against the browser. The framework also allows users to extend the library and write custom commands.

    So what’s the kicker – its all open source.

    This is a blog post series on the framework and so for more on Robot Framework and Selenium2 stay tuned)

  • by admin
  • on 29 May 2012

Comments

Be the first one to write about this.