Setting up the Rally/Eyes integration
Eyes allows you to annotate any step image in a test result with one or more issue regions. How to do this is described in the article Add issue and remark regions to the step images.
Eyes also allows you to associate an issue region with either a Rally work item or a Jira issue (but not both). This article describes how a user with Eyes team rights can set up the Test Manager so that users can associate issue regions with Rally work items. Once the required setup is performed, you will see the option to create or link to a Rally issue in the Create issue dialog of the Test editor or Step editor. This is described in detail in the article Using the Eyes/Rally integration.
Typically Eyes users will continue to use Rally for their defect reporting workflow. The Eyes/Rally integration augments and extends this flow by providing easily accessed links between the Eyes step issue region and a Rally work item. When a Rally work item is created in Eyes, it is automatically populated with the Eyes information relating to the step:
Test: The test name A test in Eyes terminology is a named sequence of check points that are compared as a unit against a sequence of baseline images associated with the test.
Step: The step tag When a checkpoint is defined using the SDK every checkpoint can have a descriptive name (aka tag) that describes the checkpoint. The names is not used to identify the checkpoint uniquly.
Batch: The batch name Two or more test runs can be grouped together as a "batch" using the SDK set batch method/property. In the Eyes Test Manager Test result page, you can see a list of all the batches that have run. You can select the batch and can then see the tests that ran in that batch. Eyes Test Manager provides has various features that allow you to operate on a subset of the batch tests or steps as a whole.
At: The data/time when the Batch started to run
App: The application name Every Eyes test is associated with an application name and test name. These uniquely identify the test. The baseline for the test is based on these names along with the OS name, the Browser name and the viewport size.
Branch: The Eyes baseline branch name A branch is a container of baselines that typically represent the expected results of one version of an application. A branch can be created from a parent branch, so that initially the baselines are identical to the parent, meaning that the expected restuls are the same. As the application in a version changes, the baseline in the branch is updated to represent the changed expectations. A branch can be merged into a "target branch", the resulting branch consists of baselines from both branches, and therefore represents expected results signified by the baselines from both branches.
Env: The execution environment An execution environment is defined as a triplet of <OS, browser, viewport size>. When a test runs, its' execution environment, is defined by the platform on which the test runs or if run on the Ultrafast Grid, by the configuration of the Ultrafast Grid. A baseline is also associated with an execution environment. By default, the baseline chosen for a test is the baseline that has the same application name, test name, and execution environment as the test.
The match level The match level describes the way that Eyes matches two images, or two regions of an image. For example, a Strict level checks if images are visibly the same, whereas a Layout level allows different text as long as the general geometry of the area is the same. You set the match level at run time via the SDK, or in the baseline using the test manager.
The captured checkpoint image
A link to the step issue region - clicking on this link will open the Test manager and display the linked Eyes issue region
A user with Eyes team admin rights defines which Rally project, work items and fields are set via Eyes and provides default values. Users that create new Rally work items via Eyes can override the default values using Eyes and create the link. Then, with a single click they can open the Rally UI to the linked work item and fill in any other work item specific details required.
Apart from providing the access and authorization information, the Eyes team admin sets up Rally usage by defining:
One or more projects the user can select when creating a new Rally item from an Eyes issues region.
The type of work item created for each project.
Default values for the work item compulsory fields.
Additional optional fields to set and their default values.
An option to update Rally when an issue is closed in Eyes.
To set up Rally integration with Eyes, you must have Eyes team admin rights. You will need to provide the obtain and provide the information in the table below as part of the integration process, so we recommend you obtain it before starting the configuration. Note in the 3rd column if the setting is per organization or per team. If the value is per organization then the sames value need to be set for all teams.
|The URL of your Rally server||Per Organization|
|Your Rally API key||
If not defined yet in your organization then obtain this value using the Rally create a new API keyfeature at https://rally1.rallydev.com/login/accounts/index.html#/keys.
Change the URL to reflect your Rally server name)
Eyes uses the Rally API key to fetch project, workspace metadata, and other non user specific information.
|OAuth Client ID and Client Secret||
|A list of the Rally entities that you want to integrate with Eyes.||
To set up Rally integration with Eyes, take the following steps:
- Load the Admin page using the page selector. Select the Teams panel and click the row containing the team you want to configure. On the team configuration page, click on the Integrations tab and then click on the Rally integration section to expand it.
- Click on the Enabled toggle so that it is in the highlighted (right) state. A team can integrate with either Rally or Jira.
If Jira integration is currently enabled for your team, a warning alert is displayed. You must disable Jira before enabling Rally by entering the Jira section on this page and setting the Enabled button to the disabled state.
- Click on the Rally server Set up server button and in the dialog that opens, enter your Rally server URL and API Key, then click the Done button.
- After the dialog closes, Eyes tries to connect to Rally using the information you provided. If an error occurs, Eyes will display an error dialog. Click Edit server setup to change the values entered previously.
- Click on the Set up OAuth button.
- In the dialog that opens, enter the Rally Client ID and Client secret and then click Done.
- Click on the Test setup configuration button. In the dialog that opens, enter your user name and password and the click Log in. Eyes tries to log in to Rally using this information.
- If the test is successful, then a tick is displayed next to the Test setup configuration button. If the login does not succeed, then an X appears next to the button. Click Edit authentication setup and enter the correct information.
- The rest of the integration defines the options that will be available to Eyes users when creating a Rally issue from Eyes. You can define options for each Rally project, and you can set specific options for particular Eyes applications Every Eyes test is associated with an application name and test name. These uniquely identify the test. The baseline for the test is based on these names along with the OS name, the Browser name and the viewport size..
- Define which Rally projects Eyes users can report defects to. For the first project, click on the Project name menu and select the Rally project you want to configure.
- Now set up that project as follows:
- Select the Work item type to be associated with that project (only one such work item can be associated with each project).
- The compulsory fields for the work item type you selected are displayed. Enter appropriate values for each field.
- You can enable a field that updates Rally when an issue is closed in Eyes.