Annotating Studies

This guide is for reviewers who annotate studies in SyRF projects. If you are a project administrator looking to design annotation questions, see Annotation Questions.

Getting Started with Annotation

Accessing Your Annotation Stage

To begin annotating studies:

1. Navigate to your project
2. Select the annotation stage from the left navigation panel
3. Click “Start Reviewing” to begin

When you enter an annotation stage, you will be presented with studies from your project’s systematic search. The study’s title and abstract are displayed, and if PDFs have been uploaded, you can view them as well.

Understanding the Study Queue

Studies are presented to you based on:

• Studies that need annotation (not yet completed by enough reviewers)
• Studies you have started but not finished (incomplete studies)
• Random selection from available studies

You will not be shown studies that have been sufficiently annotated by other reviewers.

Working with the Annotation Form

Form Layout

The annotation form is organised into tabs (also called categories). Each tab contains questions relevant to a specific aspect of the study:

• Study Level - Questions about the overall study
• Disease Model Induction - Questions about disease models used
• Treatment - Questions about treatments and interventions
• Outcome Assessment - Questions about outcome measures

If data extraction is enabled on your stage, you may also see additional tabs for Cohort and Experiment.

Question Types

Your project administrator will have designed questions using different input types:

Required vs Optional Questions

• Required questions are marked with an asterisk (*)
• You cannot submit your annotation until all required questions are answered
• Optional questions can be left blank if not applicable

Nested and Conditional Questions

Some questions only appear based on your previous answers:

• A question might appear only if you select “Yes” to a parent question
• Nested questions are indented under their parent question
• If you change your answer to the parent question, nested questions may appear or disappear

Managing Your Work

Saving Progress

You don’t have to complete all questions in one session:

• Click “Save” to save your progress and return later
• Your incomplete annotations are saved and can be resumed
• Each saved study counts towards any session limit your administrator has set

Completing an Annotation

When you have answered all required questions:

1. Review your answers across all tabs
2. Click “Submit” to complete your annotation
3. SyRF will record your annotation and move you to the next study

Returning to Incomplete Studies

To continue working on a study you previously started:

1. Look for the “Incomplete Studies” panel on your annotation stage
2. Click on a study to resume where you left off
3. Complete and submit when ready

Session Limits

Your project administrator may have set a limit on incomplete studies:

• This limits how many studies you can have “in progress” at once
• If you reach the limit, you must complete or submit existing studies before starting new ones
• This encourages reviewers to finish studies rather than accumulate many incomplete ones

Understanding Your Progress

Progress Indicators

The progress bar shows:

• Completed - Studies you have fully annotated
• In Progress - Studies you have started but not finished
• Available - Studies waiting to be annotated
• Unavailable - Studies already sufficiently annotated by others

Multi-Reviewer Workflows

Your project may require multiple reviewers to annotate each study:

• By default, SyRF expects two independent reviewers per study
• If reviewers disagree, a third reviewer may reconcile
• You will not see studies already sufficiently annotated by others

Working with Study Information

Reading the Study

While annotating, you have access to:

• Title and Abstract - Displayed prominently in the form
• PDF Viewer - Click the PDF button to view uploaded documents
• Study Metadata - Authors, publication year, and other details

Navigate between the study information and the annotation form as needed to answer questions accurately.

Tips for Efficient Annotation

1. Read the study first - Skim the abstract before starting questions
2. Work through tabs in order - This often follows the logical flow of information
3. Use keyboard navigation - Tab between fields, Enter to select
4. Save frequently - Don’t lose work if you need to step away
5. Check conditional questions - Changing an answer may reveal new questions

When Data Extraction is Enabled

If your administrator has enabled data extraction for your stage, you will see additional question categories and a more complex form structure. This allows you to:

• Create units (disease models, treatments, outcomes)
• Link units together to form cohorts
• Group cohorts into experiments
• Enter quantitative outcome data (means, medians, error values)

For detailed instructions on data extraction, see Data Extraction Guide.