TFL Reviews

Review Periods give reviewers a window of opportunity to view, comment, and request changes to TFLs. Review Periods are managed by Statisticians or Programming Managers (may vary on your Organizations configuration).

How Review Periods Work

A Review Period has 4 workflow stages:

An authorized user selects which TFLs will be available for review. Only QC'd, Statistician-approved TFLs are allowed to participate in a Team Review.

TFLs are accessible and ready for comments to authorized reviewers and all analysis team members (except Observers).

The Team Review stage is over. Commenting is closed. At this stage the Biostats team determines what action to take based on collected comments.

All comments have been reviewed and adjudicated. All analysis team members and assigned reviewers can view the TFLs, the comments and the adjudication results.

How to Access a TFL Review Period

Method 1: From Review dashboard

1. From the main menu click the Review to open the Review dashboard.
2. In the OPEN section, click the Go Review for the analysis you want to work on.
3. You will be taken to the review period summary page.

Method 2: From the Analysis detail page

1. Navigate to the Analysis you would like work to on.
2. On the Analysis Details page, in the Review Periods section (right side, middle of page), click the 1st at Team Review (or 2nd, 3rd, etc.).
3. You will be taken to the review period summary page.
4. Click the Table of Contents button to see the list of TFLs available for review. Or, click the Review TFLs to go directly to the commenting tool.

Review Tool

There are three levels to the review tool.

1. Summary page provides an overview of the review period, providing metrics and summary of review period activity.
2. Table of Contents lists the TFLs participating in the review period and metrics of review period activity.
3. Review Tool is where reviewers can view and add comments to TFLs.

Summary

• Status – displays the current status of the review period.
• Links – click the Table of Contents button to see the list of TFLs available for review. Or, click the Review TFLs to go directly to the commenting tool.
• My Progress – based on the number of TFLs in the review period you have marked as Reviewed. Communicates your progress (if you are a reviewer) toward completing the reviews assigned to you
• Team Progress – the team's progress toward completing all reviews (based on other reviewers marking TFLs ).
• Schedule – displays the start and end date window for users to complete their review.
• Top Ten TFLs – lists the top 10 (less if there are fewer than 10 TFLs in the review period) TFLs with the most comments. Click one of them will take you to the that TFL page in the Review Tool

Table of Contents

The Review Period Table of Contents page displays the list of TFLs participating in the Review Period and how each TFL in the review is progressing.

• Reviewed? "Done" ⇒ you have completed the review "Team Done" ⇒ all assigned team members have completed the review
• Titles – TFL titles provided by the Biometrics team
• TFL Name – unique name for TFL
• Group – name of the group the TFL belongs to (if provided). For example: 'Top-line'.
• Comment Summary – displays a count of the comments and responses by all team members
• Download/Review – click the download icon to download the TFL or click the eye icon to open the review tool to that TFL.

Review Tool

The Review Tool is where TFLs can be viewed and comments can be added.

• Displays the TFL in a viewer (only PDF file types are displayed)
• Displays comments and replies
• Allow adding comments and reply to comments from others
• Allows navigating TFLs in the review period
• Allows opening the TFL (outside of Candid)
• Allows keyword searching, resizing and multi-page scrolling

The review tool displays the pdf version of the TFL, displays all comments and replies to comments, allows navigating to all TFLs in the review period.

Using the Review Tool

The Candid Review Tool allows Reviewers to add general comments, add markup with Candid's built-in markup tools, and reply to comment, all in the browser!

Navigating TFLs

Review Tool toolbar

• Email – send an email to all of the team members involved in the Review Period.
• TFLs (#/#) – displays the number of completed TFL reviews against the total number of TFLs to be reviewed.
• Assigned Reviewers – displays the number of reviewers who have completed their review of the current TFL against the total number of reviewers.
• Previous TFL – click to review the previous TFL.
• Reviewed – clicking this button communicates that you have reviewed this TFL and have no further comments. Often a reviewer has no comments (the TFL 'looks good'), in those cases clicking Reviewed communicates to everyone that you have reviewed the TFL and have no feedback.
• Next TFL – click to review the next TFL in the list.

Commenting

Add a TFL Comment (without markup tools)

TFL Comments are text-only and apply only to the currently viewed TFL.

1. To the right of the Comments about this TFL heading, click the Add button.
2. Type your comment into the input box.
3. Click the Save button (or Cancel to cancel).

Add a General Review Comment

A general review comment is one that intended to apply to all TFLs in the review period.

1. Click the General tab in the Comments sidebar.
2. To the right of the Comments About All TFLs in This Review Period heading, click the Add button.
3. Enter your general review comment into the input box.
4. Click Save (or Cancel to cancel).

Add Comments Using the Markup Tools

There are 4 markup tools you can use to make detailed comments directly on the TFL image:

1. ⇒ the Marker Tool places a icon on the document and attaches a text box so you can add comments.
2. ⇒ the Square Tool allows you to use your mouse cursor to draw a square around data on the TFL and attaches a text box so you can add comments.
3. ⇒ the Circle Tool allows you to use your mouse cursor to draw a circle around data on the TFL and attaches a text box so you can add comments.
4. ⇒ the Highlight Tool allows you to use your mouse cursor to highlight text in the TFL and attaches a text box so you can add comments.

Using the Marker Tool

1. Hover your cursor over the marker tool option and click once.
2. Single-click on the TFL surface where you wish to make your marker comment.
3. Let go of the mouse button and a form will pop up.
4. Enter your comment in the box.
5. Click the Save button (or Cancel to cancel).

Using the Square Tool or Circle Tool

1. Click the square or circle tool option.
2. Click anywhere on the TFL canvas and, while holding down the mouse button, drag down then horizontally to outline an area with a blue-bordered box or circle.
3. Let go of the mouse button, and a comment form will appear.
4. Type your comment.
5. Click the Save button (or Cancel to cancel).

Using the Highlight Tool

1. Click the highlight markup tool
2. Click on the TFL canvas and, while holding down the mouse button, drag horizontally to highlight the area with a bright green color
3. Let go of the mouse button, and a comment form will appear
4. Type your comment
5. Click the Save button (or Cancel to cancel).

Replying to Comments

1. Users can reply to any comment.
2. Click the Reply button to open the reply form. If there are 1 or more replies the link will be: 2 replies, where 2 is the number of replies, then expand the replies list and click the Reply button.
3. Enter your comment into the text box that opens.
4. Click the Save button (or Cancel to cancel).

Editing your own Comment or Reply

Users can only edit their own comments and only while the team review period is open for comments. Users cannot edit or delete any comment that has 1 or more replies.

1. Click the Edit button to the right of the comment or reply.
2. Make your changes to the comment.
3. Click the Save button (or Cancel to cancel).

Deleting your own Comment

Users can only delete their own comments or replies and only while the team review period is open for comments.

1. Click the Edit button to the right of the comment you want to delete.
2. Click the Delete button and wait 2 seconds for the confirmation buttons to appear.
3. Click the Yes to confirm deletion.

Deleting your own Reply

1. Click the 2 replies (where 2 is the number of replies) to display the replies list.
2. Click the Edit button to the right of the reply you want to delete.
3. Click the Delete button.

Hiding/Showing Comments

You can hide/show the General comments to better see the comments for a given TFL.

1. Just to the right of the General Comments (n) heading, click the to hide the general comments.
2. Click the icon to show the general comments.

You can also hide/show the entire left-hand comments section to make the TFL window larger.

1. Click the Hide Comments button to close the comments sidebar.
2. Click the Show Comments button to open the comments sidebar.

Marking TFLs as Reviewed

Marking TFLs as Reviewed lets the Statistician and the review team that you have reviewed the TFL and you have no further comments.

The most common TFL review scenario is a reviewer having looked at TFL and have not comments at all. This 'sign-off' process let's you communicate that to the team.

Sign off a single TFL

1. Click the Reviewed button located in the upper right hand of the Review Tool.
2. The Reviewed button will be appear white when unchecked (not signed off) and green when checked (signed off) for the TFL currently in the viewer.
3. You can toggle the sign-off to Reviewed or Not Reviewed by clicking this button.

Sign off TFLs in bulk

1. Navigate to the Review Period Table of Contents page.
2. Click the Bulk Sign-off button.
3. Check the boxes (or Select all or None) for the TFLs you want to toggle as Signed Off (Reviewed).
4. Click the Done button.

Review a sub-set of TFLs in Review Tool

To send a sub-set of TFLs to the review tool, you can filter the list of TFLs from the Table of Contents page and send the filtered list to the Review Tool

1. Navigate to the Revew Period's Table of Contents page.
2. Filter the list by entering text into the Type to Filter input box.
3. Candid will filter the list matching the text entered against the data displayed in the table of contents index.
4. Check the boxes on the left or click the Select All button to select the filtered TFLs to send to the Review Tool.
5. Click the Go Review Selected button to open the review tool containing just the checked TFLs.

Setup a Review Period

Review Period Administration Overview Video

Add/Remove TFLs to Review Period

1. Navigate to the Analysis you would like to work on.
2. On the Analysis Details page, in the Review Periods section (right side, middle of page), click the 1st at Setup (or 2nd, 3rd, etc.).
3. Click on the Choose TFLs to move to Team Review button.
4. In the Selected column, click the checkboxes for each TFL you would like to add to or remove from the review period; alternatively, you can click the Select All or Deselect All buttons.
5. Click the Done button.

Change the Name of the Review Period

1. Navigate to the Review Period Adminstration page.
2. Change the name of the Review Period by clicking the Edit button next to Name. Enter the new name and click Save.

Add/Update Reviewer Instructions

1. Navigate to the Review Period Adminstration page.
2. To add or update Reviewer Instructions for the Reviewers, click the Edit button next to Reviewer Instructions. Enter the text, then click Save.

Update the Schedule

1. Navigate to the Review Period Adminstration page.
2. In the Review Period workflow section at the top of the page, click the pencil icon to the right of the date range underneath the Team Review status.
3. The date range is highlighted in blue. Change the end date by click a different day in the calendar. Change the start date by clicking a different day (before the end date). The start date can only be changed if the status of the Review Period is at Setup and the Review Period has never been opened for Team Review.

Add/Remove Reviewers to Access the Review Period

1. Navigate to the Analysis you would like to work on or click the Review Period link from a dashboard.
2. From the Analysis Details page, in the Review Periods section (right side, middle of page), click the 1st at Setup (or 2nd, 3rd, etc.).
3. Or, if you land directly to the Review Period Summary page, then click the Administer Review Period button.
4. On the Analysis Details page, in the Review Periods section (right side, middle of page), click the 1st at Setup (or 2nd, 3rd, etc.).
5. Click inside the Review Period Team input box and start typing the name of the User or User Group to filter the list.
6. Select the User or Group name you want to add from the result of your search.
7. Click the Add button or hit the Enter key
8. Or, click the icon to remove a user from the Review Period Team.

Open Team Review

Once 1 or more TFLs have been added to the review period, it becomes eligible to open to reviewers.

1. Navigate to the Analysis you would like to work on or click the Review Period link from a dashboard.
2. From the Analysis Details page, in the Review Periods section (right side, middle of page), click the 1st at Setup (or 2nd, 3rd, etc.).
3. Or, if you land directly to the Review Period Summary page, then click the Administer Review Period button.
4. Double check to make sure all the TFLs you want reviewed have been added to the review period.
5. Click on the Open team review button.
6. Click the Submit button to confirm.
7. The review will move into the active Team Review status and all reviewer users on the analysis will be able to access the review period, view the TFLs in it and provide their feedback.

Close Team Review

1. Navigate to the Analysis you would like to work on or click the Review Period link from a dashboard.
2. From the Analysis Details page, in the Review Periods section (right side, middle of page), click the 1st at Setup (or 2nd, 3rd, etc.).
3. Click the Administer Review Period button from the Review Period Summary page.
4. Click on the Close team review button to close the team review.
5. Click the Submit button to confirm.
6. The review will move into Adjudication state.

Adjudication

Overview

• Adjudication is a comment review and decision making wizard. It a framework for tagging which comments should lead to FTL changes.
• The process involves reviewing each comment and tagging each one as either Actionable or No Action.
• The number of steps in the wizard is driven by the presence of comments. Each FTL with 1 or more comments will be listed as Step in the wizard.
• When completing the wizard, Actionable comments are automatically turned into Issues.

Instructions

1. Use the Jump buttons to view the comments for a given FTL.
2. Review each comment.
3. (optional) Update the Instructions for programmer input box with additional information. These can be instructions for the programmer to make a change, clarifying statements for the team, or any additional information helpful to record.
4. Click Save - No Action or Save - Actionable to save the decision.
5. Jump to Confirm Adjudications to review your decisions FTLs Next State.
7. Click the Finish Adjudication and Close Review Period button to complete the wizard and update the status of the review period to Complete.

Adjudicating Comments (step by step)

1. Navigate to the Analysis you would like to work on or click the Review Period link from a dashboard.
2. From the Analysis Details page, in the Review Periods section (right side, middle of page), click the 1st at Setup (or 2nd, 3rd, etc.).
3. Click the Administer Review Period button from the Review Period Summary page.
4. Click the Adjudicate button to open the Adjudication wizard, which will guide you through each TFL that contains comments.
5. The left side of the adjudication page contains wizard steps. The number of steps is determined by the presence of comments entered. If one or more General comments are present, then the first step will be adjudicating those. The following steps will be for each TFL that has 1 or more comments on them. The last step is to complete the adjudication stage and close the review period.

6. Within each step of the wizard tag each comment as requiring Action or No Action:

7. Actionable

   1. First, review the Instructions for programmer input box (it is pre-loaded with the comment text) and add, if necessary, any details that will aid the programmer in addressing the change. This text will become the Description for the Issue that will be created when you close the review period.
   2. Any responses to comments will appear below the comment. Users can click the Append button to add the response to the end of any text already in the Instructions for Programmer input box.
   3. Then, click the Save - Actionable button. The comment is adjudicated and changes saved.

8. No Action

   1. First, review the Instructions for programmer input box (it is pre-loaded with the comment text) and add to it (or clear) the text and start from scratch. Often reviewers find it helpful to see the reason No Action was selected for their comments.
   2. Any responses to comments will appear below the comment. Users can click the Append button to add the response to the end of any text already in the Instructions for programmer input box.
   3. Click the Save - No Action button. The comment is adjudicated and changes saved.

9. Click Next or Previous buttons to navigate forward, back through the TFLs with comments. Or, you can click Jump buttons to jump from step to another.

Confirm Adjudications & Close Review Period

The last step in the adjudication wizard is to confirm which status the TFLs will be changed to, when closing adjudication.

TFLs that have 1 or more actionable comments must go back to programming. TFLs that have 0 actionable comments can be advanced to Complete or can be sent back to Programming.

1. Review the TFL status change options. Change the Next State option for any TFLs with 0 actionable comments you want sent back to Programming.

2. If everything looks correct, click the Finish Adjudication & Close Review Period button.

3. The TFL's status will be changed to the value selected in the Next State column.

4. Issues will be created for each actionable comment.
5. The Review Period's status will advance to Complete and a new Review Period can be started.

Export Comments and Adjudication Results to Excel

1. Navigate to the Analysis you would like to work on.
2. On the Analysis Details page, in the Review Periods section (right side, middle of page), click the Excel icon to the right of the Review Period link.
3. An Excel file of all comments will be created and saved to the Downloads folder on your machine.

How to Communicate I am 'Done Reviewing'

1. Navigate to the Review Period Table of Contents page.
2. Click the Done Reviewing button.
3. After clicking Done Reviewing the button will turn green with a check icon and the Go Review button and buttons in the Table of Contents index will become disabled. A green check mark will also appear next to your name on the Review Period Summary page.
4. You can undo this action by clicking the button again.