Evaluator Internals

Runtime Environment

• Isolation: Each submission runs in a completely isolated environment
• Language Support: Native support for Python, Java, and C compilers/interpreters
• Stateless: No persistent state between executions—every run is fresh

I/O Harness

For I/O-based testing, the evaluator follows a simple but robust pipeline:

Resource Limits

To prevent abuse and ensure fair resource allocation, the following limits are enforced:

* Limits are subject to change as we optimize performance. Check back for updates.

The Challenge

Unlike simple I/O testing, assertion-based testing requires:

• Merging student code with teacher test cases
• Ensuring single-file compilation (no package conflicts)
• Extracting and substituting method bodies safely
• Running JUnit/assertion tests in a sandboxed environment

Our Approach: Hybrid Parsing + Templating

VeraCodeChecker uses a sophisticated multi-step pipeline combining regex patterns, AI-assisted parsing, and code templating:

public class Calculator {
    // Method to be tested
    public int add(int a, int b) {
        // PLACEHOLDER - will be replaced with student code
    }
    
    // Other methods remain intact...
}

Parsing Techniques

Constraints & Rules