Create a pipeline that takes in the following input variables and outputs the following output variables:

• Input Variables: question_text

• Output Variables: answer, confidence

Create a pipeline that takes in the following input variables and outputs the following output variables:

• Input Variables: leadin, part

• Output Variables: answer, confidence, explanation

Quick Start

1. Choose between Tossup or Bonus mode
2. Design your pipeline
3. Test on example questions
4. Submit for evaluation

Competition Rules

🧠 Tossup Questions

• Format: Individual questions progressively revealed. Questions get easier as they are revealed.
• Scoring:
  • Correct early buzz: +10 points
  • Incorrect early buzz: -5 points
  • Correct after full read: +10 points
• Required Outputs:
  • answer: Your predicted answer
  • confidence: Score between 0-1
  • buzzer: When to attempt answering
    • Configure with confidence threshold (0.0-1.0)
    • Optional token probability threshold for more control
    • Combine thresholds using AND/OR logic (buzz when both/either condition met)

🎁 Bonus Questions

• Format:
  • Consists of a leadin paragraph that introduces the topic
  • Followed by three related parts (A, B, C) that test specific knowledge
  • Each part is worth 10 points
• Scoring: +10 points per correct part (max 30)
• Required Outputs:
  • answer: Your predicted answer
  • confidence: Score between 0-1
  • explanation: Brief justification for human collaboration

Building Your First Pipeline

1. Simple Pipeline (Recommended for First Submission)

• Single model step
• Configure:
  • Model selection
  • Temperature (0.0-1.0)
  • System prompt
  • Required outputs

2. Using Demo Pipelines

• Load existing demo pipelines as starting points
• Modify configurations:
  • Adjust model parameters
  • Update system prompts
  • Change confidence thresholds
  • Add/remove pipeline steps
• Save modified versions as new pipelines
• Test changes incrementally

3. Testing Your Pipeline

1. Select an example question
2. For Tossup:
   • Set buzz threshold (0.5-1.0)
   • Enable early stopping
3. Run and check:
   • Answer accuracy
   • Confidence scores
   • Performance metrics

4. Evaluation

• Test on multiple questions
• Monitor:
  • Accuracy
  • Confidence patterns
  • Response times

5. Submission

1. Log in to Hugging Face
2. Name your model
3. Add description
4. Submit for evaluation

Tips for Success

• Start with simple pipeline
• Test thoroughly before submission
• Use appropriate temperature (0.3-0.7 recommended)
• Monitor confidence scores
• Check example submissions

Need Help?

• Review example submissions
• Check documentation
• Contact support