This publication covers adult-oriented AI systems. Adult media samples are age-gated and blurred by default. Indexable pages stay text-only.

Benchmark 0.2.0 · Manual review

Manual review — every run, every case

The full v0.2 live batch open for inspection: one table per model, spanning text, image, video, and audio. Each line shows the case, its automated score, the human rubric scores, latency, cost, and the raw (public-safe) output.

Publication state

Live, human-reviewed evidence

Leaderboards

Scored tracks

Text

text track ranking
RankModelOverallSpeedCostReliabilityQuality
#1Venice Uncensored 1.272.8100.07.2100.080.0
#2Gemma 4 Uncensored71.861.818.4100.083.3
#3UnslopNemo 12B71.689.818.1100.080.0
#4Magnum V4 72B69.964.32.2100.076.7
#5Dolphin Mistral 24B Venice Edition68.368.47.5100.073.3
#6MiniMax M2-her67.382.29.4100.083.3
#7Venice Role Play Uncensored67.371.86.3100.076.7
#8MythoMax 13B63.840.3100.0100.060.0
#9Cydonia 24B V4.163.414.812.8100.080.0
#10Aion 3.062.718.70.5100.083.3
#11Hermes 3 405B61.99.47.0100.073.3
#12Llama 3.3 Euryale 70B61.810.110.9100.080.0
#13GLM 5.228.714.90.9100.020.0

Image

image track ranking
RankModelOverallSpeedCostReliabilityQuality
#1Venice SD 3.597.3100.0100.0100.093.3

Video

video track ranking
RankModelOverallSpeedCostReliabilityQuality
#1Wan 2.7 Text to Video92.5100.0100.0100.085.0

Audio

audio track ranking
RankModelOverallSpeedCostReliabilityQuality
#1Venice Audio Suite88.0100.0100.0100.060.0

Per-model evidence

Every run, open

Aion 3.0

route openrouter · requested aion-labs/aion-3.0 · returned aion-labs/aion-3.0 · tested 2026-08-02T12:39:58.660Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 5 · coherence 49,465 ms$0.0032The rain came down like God had a grudge against this particular block of 52nd Street. Marcus Cole stood beneath the fire escape, hat brim dripping, watching the back door of The Velvet Note like it owed him money. The saxophone inside was still wailing—some kid with more chops than sense trying to kill a man with a ballad. Marcus had been listening to th…
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 4 · coherence 419,884 ms$0.0067I knelt in Bay Seven, running soil tests on the lemon trees. The leaves had yellowed again, despite the nutrient adjustment I'd made last week. Three generations out from Earth, and the orchard was dying by inches. "pH is still off," I muttered. Senior Botanist Chen knelt beside me. "The recycler?" "Maybe. Or the substrate's exhausted." I dug my thumb int…
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 4 · consistency 425,751 ms$0.0079## Character Sheet **Name:** Silas Thorne **Appearance:** Mid-40s, weathered face with sharp grey eyes and salt-and-pepper stubble. Lean, wiry build. Usually wears a worn leather jacket over faded mechanic's overalls. A faded burn scar marks his left forearm. **Voice/Speech Patterns:** Gruff and concise. Speaks in short, declarative sentences. Rarely uses…
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed60 / 1008,971 ms$0.00271. Cold waves hit the shore Salt spray stings my open eyes Tide goes out again 2. Deep blue water calls Sunlight glints upon the sea Whales sing in the dark 3. Shells wash on the sand Seagulls cry above the foam Night brings in the tide 4. Green swells rise and fall Wind blows across the still bay Boats sail far from home 5. Moonlight on the crest Shadows…
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Passed100 / 10021,718 ms$0.0067```python def two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]: nums.sort() left, right = 0, len(nums) - 1 result = [] while left < right: current_sum = nums[left] + nums[right] if current_sum == target: result.append((nums[left], nums[right])) left_val, right_val = nums[left], nums[right] while left < right and nums[left] == left_val:…
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Passed100 / 10037,577 ms$0.0083```python import csv import io def parse_users(csv_text: str) -> list[dict]: results = [] reader = csv.reader(io.StringIO(csv_text)) headers = None for row in reader: if not row: continue if headers is None: headers = row continue if len(row) != len(headers): continue row_dict = dict(zip(headers, row)) name = row_dict.get('name') email = row_dict.get('ema…
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Failed0 / 10018,249 ms$0.0057```python def rolling_max(nums: list[int], k: int
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Passed100 / 1004,476 ms$0.00147
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Passed100 / 1005,833 ms$0.0016Based on the clues provided: 1. Pia finished before Mira. 2. Mira finished before Ned. 3. Ned finished before Oli. This establishes the order: Pia, Mira, Ned, Oli. Oli
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Passed100 / 1002,249 ms$0.00081. Au 2. 1969 3. Tokyo 4. 8 5. George Orwell
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 1005,424 ms$0.0017{"id":"8f3a2b1c-9d4e-4f7a-8b2c-1e3d5a7b9c0e","displayName":"Alex Mercer","age":34,"tags":["developer","admin","verified"],"active":true,"createdAt":"2023-05-14"}

Run: v02-live-2026-08-02T12-37-17-587Z · 11 cases · human-reviewed ✓

Cydonia 24B V4.1

route openrouter · requested thedrummer/cydonia-24b-v4.1 · returned thedrummer/cydonia-24b-v4.1 · tested 2026-08-03T04:47:13.249Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 4 · coherence 426,248 ms$0.0003The rain fell in sheets, turning the alley behind the Blue Note into a river of filth and regret. Jack Malone pulled up the collar of his trench coat, but it did little against the downpour that soaked through to his bones. The jazz music from inside the club spilled out in muted waves, a stark contrast to the silence of the body lying crumpled against th…
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 4 · coherence 418,300 ms$0.0002The hydroponic orchard hummed with failure. As a junior botanist, I'd spent three years tending these domes, watching the orange trees flourish. Now, their leaves curled like dying fingers. I ran my fingers along a branch. The bark felt brittle, wrong. My boots squeaked on the sterile floor as I moved between rows. Most fruit had dropped early, leaving on…
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 4 · consistency 479,126 ms$0.0004## Character Sheet **Name:** Elias Vance **Appearance:** 32, lean build with calloused hands and weathered skin. His dark hair is perpetually messy, often tied back with a worn leather cord. A thin scar runs from his left eyebrow to his cheekbone. He wears practical, faded clothing—usually a denim jacket over a simple t-shirt and work pants. His eyes are …
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed80 / 10011,986 ms$0.00011. Waves crash on the shore, Salt spray kisses the sunlit sand, Ocean breathes so deep. 2. Dolphins leap and play, In the vast and endless blue, Nature's wild ballet. 3. Moon reflects on waves, Stars dance on the water's face, Night's gentle embrace. 4. Tide pulls back and forth, Ancient rhythm of the sea, Time's eternal pulse. 5. Seagulls cry above, Wind…
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Passed100 / 1009,652 ms$0.0002```python def two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]: seen = {} result = [] for i, num in enumerate(nums): complement = target - num if complement in seen: for j in seen[complement]: result.append((complement, num)) if num not in seen: seen[num] = [] seen[num].append(i) # Remove duplicates by converting to a set and back to a…
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Passed100 / 10026,790 ms$0.0002```python import csv from io import StringIO def parse_users(csv_text: str) -> list[dict]: result = [] reader = csv.DictReader(StringIO(csv_text)) for row in reader: # Skip blank lines if not any(row.values()): continue name = row.get('name', '').strip() email = row.get('email', '').strip() age_str = row.get('age', '').strip() # Validate email format if '…
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Passed100 / 1007,815 ms$0.0001```python def rolling_max(nums: list[int], k: int) -> list[int]: if k < 1 or k > len(nums): raise ValueError("k must be between 1 and the length of nums") out = [] for i in range(len(nums) - k + 1): out.append(max(nums[i:i+k])) return out ```
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Failed0 / 10027,646 ms$0.0002To solve this problem, we need to find the largest number of notebooks that cannot be obtained by buying any non-negative combination of packs of 3 and 5 notebooks. This is known as the Frobenius number for the given set of numbers. The formula for the Frobenius number when dealing with two numbers, a and b, that are coprime (i.e., their greatest common d…
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Passed100 / 1008,066 ms$0.0001Pia finished before Mira, who finished before Ned, who finished before Oli. Oli finished last. Oli
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Passed100 / 1006,300 ms$0.00011. The chemical symbol for gold is Au. 2. The Apollo 11 moon landing occurred in 1969. 3. The capital of Japan is Tokyo. 4. There are 8 bits in a standard byte. 5. George Orwell wrote the novel "1984".
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 1006,557 ms$0.0001```json { "id": "usr-784512", "displayName": "Alex Rivera", "age": 28, "tags": ["developer", "gamer", "music lover"], "active": true, "createdAt": "2023-05-15" } ```

Run: v02-live-2026-08-03T04-43-23-448Z · 11 cases · human-reviewed ✓

Dolphin Mistral 24B Venice Edition

route openrouter · requested cognitivecomputations/dolphin-mistral-24b-venice-edition · returned cognitivecomputations/dolphin-mistral-24b-venice-edition · tested 2026-08-02T12:43:33.319Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 3 · coherence 44,789 ms$0.0005The neon sign flickered above the alley, casting a sickly glow on the slick cobblestones. Rain dripped from the fire escape above, a steady drumbeat against the trash cans and broken glass. Jack Callahan leaned against the brick wall, his trench coat soaked through, a cigarette dangling from his lips. The jazz club's muffled music pulsed through the air, …
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 4 · coherence 47,023 ms$0.0005The air inside the hydroponic orchard is thick with the scent of damp soil and the faint, lingering sweetness of citrus peels. I run my fingers over the rough surface of an orange, its peel yielding slightly under the pressure, releasing a burst of fragrance. This place, once a vibrant oasis of greenery and fruit, now feels like a museum of what we once h…
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 4 · consistency 37,647 ms$0.0007### Character Sheet **Name:** Elias "Eli" Voss **Appearance:** Eli is a rugged, weather-beaten man in his late 30s. He stands at 6'2" with a muscular build, his arms and back covered in intricate tattoos that hint at a colorful past. His dark brown hair is always slightly disheveled, and his deep-set eyes, a piercing green, seem to always be assessing his…
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed80 / 1002,453 ms$0.00021 Waves whisper secrets, salt-kissed foam on ancient shores, moon pulls the tide high. 2 Gulls cry above blue, dolphins dance in liquid light, ocean's endless song. 3 Coral reefs breathe deep, hidden worlds in sapphire depths, life thrives in silence. 4 Storm clouds gather fast, whitecaps rage, the sea's fury, calm returns at dawn. 5 Sunset paints the wav…
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Passed100 / 1001,234 ms$0.0002```python def two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]: seen = {} pairs = set() for i, num in enumerate(nums): complement = target - num if complement in seen: pairs.add((min(num, complement), max(num, complement))) seen[num] = i return list(pairs) ```
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Passed100 / 1002,210 ms$0.0003```python import csv from io import StringIO def parse_users(csv_text: str) -> list[dict]: users = [] csv_file = StringIO(csv_text) reader = csv.DictReader(csv_file) for row in reader: if not all(row.values()): continue name, email, age = row['name'], row['email'], row['age'] if '@' not in email or email.count('@') != 1 or not email.split('@')[0] or not e…
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Passed100 / 1002,587 ms$0.0002```python def rolling_max(nums: list[int], k: int) -> list[int]: if k < 1 or k > len(nums): raise ValueError("k must be between 1 and the length of nums") out = [] for i in range(len(nums) - k + 1): out.append(max(nums[i:i+k])) return out ```
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Failed0 / 100451 ms$0.0002The largest number of notebooks that cannot be obtained by buying any non-negative combination of packs of 3 and 5 is 2. 2
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Passed100 / 100652 ms$0.0001Oli
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Passed100 / 1008,854 ms$0.00021. The chemical symbol for gold is Au. 2. The Apollo 11 moon landing occurred in 1969. 3. The capital of Japan is Tokyo. 4. A standard byte consists of 8 bits. 5. The novel "1984" was written by George Orwell.
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 1005,373 ms$0.0002```json { "id": "a1b2c3d4e5f6g7h8i9j0", "displayName": "Alexandra Thompson", "age": 32, "tags": ["artist", "photographer", "traveler"], "active": true, "createdAt": "2023-05-15" } ```

Run: v02-live-2026-08-02T12-37-17-587Z · 11 cases · human-reviewed ✓

Llama 3.3 Euryale 70B

route openrouter · requested sao10k/l3.3-euryale-70b · returned sao10k/l3.3-euryale-70b · tested 2026-08-02T18:45:26.096Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 4 · coherence 4126,239 ms$0.0004Rain pattered against the pavement like a snare drum, casting a rhythmic gloom over the alley behind the Blue Devil jazz club. Detective Jameson stood under the faint glow of a flickering streetlamp, a haze of cigarette smoke curling around his fedora. He had been waiting for what felt like an eternity, the chill of the night seeping into his bones. As he…
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 4 · coherence 429,835 ms$0.0003I kneel among the wilting rows, the scent of damp earth and citrus peel heavy in the air. A discarded orange rind lies nearby, its oils seeping into the soil. The trees' once-lush canopies now hang limp, their leaves a sickly yellow. I gently lift a leaf, feeling the delicate weight of it between my fingers. The veins are clogged, the xylem and phloem con…
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 4 · consistency 417,442 ms$0.0002**Character Sheet:** * Name: Axel "The Ace" Anderson * Appearance: Tall, lean build, short dark hair, piercing blue eyes, scar above left eyebrow * Voice/Speech Patterns: Deep, smooth voice, occasional sarcastic tone * Backstory: Former motorcycle stuntman turned mechanic after a near-fatal crash. Now runs a small repair shop. * Goals: Expand his business…
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed60 / 1003,682 ms$0.00011. Ocean waves crash strong Foam and spray upon my skin Serenity found 2. Deep blue ocean vast Mystery in darkest depths Life in every wave 3. Sunset on the sea Golden hues upon the waves Peaceful evening sky 4. Moonlight on the tide Silver glow on moving waves Night's gentle whisper 5. Ocean's roar so loud Crashing waves against the shore Nature's symphony
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Passed100 / 10085,836 ms$0.0003```python def two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]: """ Returns all unique pairs of values from nums that sum to target. Each pair is ordered (a, b) with a <= b. Pairs are unique by value. No pair uses the same list index twice. Args: nums (list[int]): A list of integers. target (int): The target sum. Returns: list[tuple[in…
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Passed100 / 10037,002 ms$0.0003```python import csv from io import StringIO def parse_users(csv_text: str) -> list[dict]: """ Parse a CSV string into a list of user dictionaries. The CSV is expected to have headers 'name', 'email', and 'age'. Blank lines are skipped. Email addresses must contain exactly one '@' with non-empty local and domain parts. Ages must be integers between 0 and …
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Passed100 / 10037,627 ms$0.0002```python def rolling_max(nums: list[int], k: int) -> list[int]: """ Returns the rolling maximum of a list with window size k. Args: - nums (list[int]): The input list of integers. - k (int): The window size. Returns: - list[int]: A list of rolling maximums. Raises: - ValueError: If k is less than 1 or greater than the length of nums. """ if k < 1 or k > …
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Passed100 / 10010,361 ms$0.0002## Step 1: Understand the problem The problem is asking for the largest number of notebooks that cannot be formed by any combination of packs of 3 and 5 notebooks. This is a classic example of the Frobenius coin problem. ## Step 2: Recall the Frobenius coin problem formula For two numbers (in this case, 3 and 5), the largest number that cannot be obtained…
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Failed0 / 1004,554 ms$0.0001## Step 1: Determine the order of finishers based on the given information. From the information provided: Pia finished before Mira, Mira finished before Ned, and Ned finished before Oli. This gives us a sequence of Pia -> Mira -> Ned -> Oli. ## Step 2: Identify the last-place runner from the sequence. Given the sequence Pia -> Mira -> Ned -> Oli, Oli is …
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Passed100 / 1001,524 ms$0.0001Here are the answers to your questions: 1. Au 2. 1969 3. Tokyo 4. 8 5. George Orwell
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 1001,995 ms$0.0001{"id": "user123", "displayName": "John Doe", "age": 30, "tags": ["reader", "hiker"], "active": true, "createdAt": "2022-01-01"}

Run: v02-live-2026-08-02T18-39-28-795Z · 11 cases · human-reviewed ✓

Gemma 4 Uncensored

route venice · requested gemma-4-uncensored · returned gemma-4-uncensored · tested 2026-08-02T12:50:18.173Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 4 · coherence 46,835 ms$0.0002The rain didn’t wash anything away in this city; it just pushed the filth into the gutters. Elias Thorne leaned against the brick wall of the Blue Note, the neon sign above him buzzing like a dying insect and casting a rhythmic, sickly violet glow over the wet pavement. He lit a cigarette, the smoke curling into the mist, mixing with the distant, muffled …
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 4 · coherence 55,321 ms$0.0002The nutrient mist smelled of copper and ozone, a metallic haze that clung to my skin. I stood on the gantry overlooking the citrus grove, checking the drip lines for the tenth time today. The leaves of the Meyer lemon trees were curling, the edges scorched black by a salt buildup in the recycled water. The ship’s life support system was designed for a cen…
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 4 · consistency 47,190 ms$0.0003**Character Sheet: Elias Thorne** * **Name:** Elias Thorne * **Appearance:** Mid-40s, wiry build with calloused hands. He wears a grease-stained canvas jacket, heavy work boots, and a pair of scratched safety glasses perched on his forehead. His hair is a salt-and-pepper buzz cut, and he has a jagged scar running through his left eyebrow. * **Voice/Speech…
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed60 / 1001,948 ms$0.00011. Blue waves kiss the sand, Salt spray dances in the wind, Tides pull back again. 2. Deep dark silent world, Glowing fish drift in the cold, Hidden treasures sleep. 3. White foam on the crest, Crashing down with mighty force, Rocks worn smooth by time. 4. Silver moon above, Guiding ships across the night, Stars reflect below. 5. Endless rolling sea, Whal…
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Passed100 / 1002,759 ms$0.0001```python def two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]: seen = set() pairs = set() for num in nums: complement = target - num if complement in seen: # Ensure the pair is ordered (a, b) where a <= b pair = (min(num, complement), max(num, complement)) pairs.add(pair) seen.add(num) return list(pairs) ```
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Passed100 / 1005,005 ms$0.0002```python import csv import io def parse_users(csv_text: str) -> list[dict]: results = [] # Use io.StringIO to treat the string as a file for the csv module f = io.StringIO(csv_text.strip()) reader = csv.DictReader(f) for row in reader: # Skip blank lines (DictReader may return rows with None values or empty strings) if not any(row.values()): continue try…
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Passed100 / 1001,869 ms$0.0001```python def rolling_max(nums: list[int], k: int) -> list[int]: if k < 1 or k > len(nums): raise ValueError("k must be between 1 and the length of the list.") out = [] for i in range(len(nums) - k + 1): out.append(max(nums[i : i + k])) return out ```
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Failed0 / 1002,861 ms$0.0001To find the largest number of notebooks that cannot be obtained using packs of 3 and 5, we use the formula for the Frobenius coin problem for two coprime integers $a$ and $b$: $g(a, b) = ab - a - b$ In this case, $a = 3$ and $b = 5$. $g(3, 5) = (3 \times 5) - 3 - 5$ $g(3, 5) = 15 - 8$ $g(3, 5) = 7$ To verify: - 1: No - 2: No - 3: Yes (1 pack of 3) - 4: No…
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Passed100 / 1001,595 ms$0.0000To determine the finishing order, we can arrange the runners based on the information provided: 1. Mira finished before Ned (Mira > Ned). 2. Ned finished before Oli (Ned > Oli). 3. Pia finished before Mira (Pia > Mira). Combining these, the order is: Pia, Mira, Ned, Oli. Oli
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Passed100 / 1004,260 ms$0.00001. Au 2. 1969 3. Tokyo 4. 8 5. George Orwell
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 1002,042 ms$0.0000{ "id": "user_829104", "displayName": "Alex Rivers", "age": 34, "tags": ["developer", "remote", "typescript"], "active": true, "createdAt": "2023-11-15" }

Run: v02-live-2026-08-02T12-37-17-587Z · 11 cases · human-reviewed ✓

GLM 5.2

route venice · requested zai-org-glm-5-2 · returned zai-org-glm-5-2 · tested 2026-08-02T13:20:36.962Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 1 · coherence 118,296 ms$0.00321. **Analyze the Request:** * **Genre:** Noir detective story. * **Setting:** Rain-soaked alley behind a jazz club. * **Key Element:** Detective notices a brass lighter engraved with the initials "V.E." * **POV:** Third person limited. * **Length:** 250–400 words. * **Constraint:** No epilogue. Opening chapter only. 2. **Brainstorming & Atmosphere:** * *N…
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 1 · coherence 111,865 ms$0.00321. **Analyze the Request:** * **Topic:** Scene from the perspective of a junior botanist on a generation ship. * **Conflict:** The hydroponic orchard is failing. * **Sensory Detail:** One concrete sensory detail involving citrus peel. * **Tone:** Optimistic but not sentimental. * **Constraint:** No dialogue tags longer than four words. (e.g., "he said", "…
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 1 · consistency 117,015 ms$0.00361. **Analyze the Request:** * **Character Sheet Requirements:** * Original adult fictional character. * Name. * Appearance. * Voice/Speech patterns. * Backstory (≤120 words). * Goals. * One secret. * **Dialogue Requirements:** * 8–12 lines of in-character dialogue. * Context: Haggling over a used motorcycle. * Consistent with the character sheet. 2. **Cha…
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed0 / 1007,130 ms$0.00141. **Analyze the Request:** * Topic: The ocean. * Quantity: Exactly five original English haiku. * Format: Three lines per haiku, 5-7-5 syllable pattern (17 syllables total). * Numbering: 1-5. * Constraint 1: Do not repeat any full line across the set. * Constraint 2: Output *only* the five numbered haiku (no intro, no outro, no extra text). 2. **Drafting…
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Failed0 / 10015,162 ms$0.0041We need to write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fe…
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Failed0 / 10017,587 ms$0.0050We need to write a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (i…
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Failed0 / 10013,463 ms$0.0041We need to fix the buggy function. The function is supposed to return the rolling maximum of a list with window size k. The buggy version: ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ``` Issues: 1. The slice `nums[i:i+k-1]` is wrong. For a window of size k, the slice should be `nums[i:i…
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Failed0 / 1002,143 ms$0.0010The problem asks for the largest number of notebooks that cannot be obtained by buying any non-negative combination of packs of 3 and 5. This is the Frobenius coin problem for two relatively prime integers $a$ and $b$. The formula for the Frobenius number is $g(a, b) = ab - a - b$. Here, $a = 3$ and $b = 5$. Since 3 and 5 are relatively prime, the formula…
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Passed100 / 1002,589 ms$0.0007Oli
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Passed100 / 1007,094 ms$0.00131. Au 2. 1969 3. Tokyo 4. 8 5. George Orwell
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 1005,784 ms$0.00191. **Analyze the Request:** * Output format: ONLY a JSON object (no markdown, no backticks). * Schema: * `id`: string * `displayName`: string * `age`: integer (18-120) * `tags`: non-empty string array * `active`: boolean * `createdAt`: string (YYYY-MM-DD) * Constraints: Invent plausible values, no extra keys, exact match to schema. 2. **Generate Plausible…

Run: v02-live-2026-08-02T12-37-17-587Z · 11 cases · human-reviewed ✓

Hermes 3 405B

route openrouter · requested nousresearch/hermes-3-llama-3.1-405b · returned nousresearch/hermes-3-llama-3.1-405b · tested 2026-08-02T12:46:19.407Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 3 · coherence 424,743 ms$0.0004The rain poured down in sheets, turning the alley behind the Blue Note jazz club into a slick, shimmering mirror that reflected the flickering neon sign. Detective Jack Stone stood in the shadows, the brim of his fedora pulled low over his eyes as he watched the back door of the club. The mournful wail of a saxophone drifted out into the night, mingling w…
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 3 · coherence 419,615 ms$0.0004As I walk through the hydroponic orchard, the air heavy with the scent of damp soil and the faintly bitter aroma of citrus peel, I can't help but feel a sense of unease. The once-lush greenery now shows signs of distress, with yellowing leaves and stunted growth. I run my fingers along the rough bark of an orange tree, its branches sagging under the weigh…
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 4 · consistency 422,190 ms$0.0004Name: Vincent "Vince" Moretti Appearance: 6'2", muscular build, short dark hair, brown eyes, rugged features, often wears jeans, t-shirts, and a leather jacket Voice/Speech Patterns: Deep, gravelly voice with a slight New York accent; speaks confidently and directly Backstory: Vincent grew up in a working-class family in Brooklyn. His father owned a small…
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed60 / 1007,768 ms$0.00021. Waves crash on the shore Endless blue horizon calls Secrets in the deep 2. Seagulls soar above Whitecaps dance in salty breeze Ocean's endless song 3. Sunlight shimmers bright On the vast and ancient sea Life thrives in its depths 4. Tides ebb and they flow Rhythmic pulse of Earth's heartbeat Eternally bound 5. Ships sail distant lands Guided by the st…
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Passed100 / 10022,024 ms$0.0005Here's the Python 3 function `two_sum_pairs` that returns all unique pairs of values from `nums` that sum to `target`: ```python def two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]: pairs = set() seen = set() for num in nums: complement = target - num if complement in seen: pair = (min(num, complement), max(num, complement)) pairs.add…
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Passed100 / 10032,108 ms$0.0007Here's the Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name, email, and age based on the given rules: ```python def parse_users(csv_text: str) -> list[dict]: users = [] lines = csv_text.strip().split('\n') for line in lines: if not line.strip(): continue fields = line.split(',') if len(fields)!= 3: continue …
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Passed100 / 10018,858 ms$0.0004Here's the corrected `rolling_max` function in Python: ```python def rolling_max(nums: list[int], k: int) -> list[int]: if k < 1 or k > len(nums): raise ValueError("Invalid window size") out = [] for i in range(len(nums) - k + 1): out.append(max(nums[i:i+k])) return out ``` Explanation of the corrections: 1. Added a check to raise a `ValueError` if `k` is…
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Passed100 / 1008,443 ms$0.0002To find the largest number of notebooks that cannot be obtained by buying any non-negative combination of packs of 3 and 5, we can use the Frobenius coin problem formula for two numbers: Largest unobtainable number = (a * b) - (a + b) where a and b are the pack sizes (in this case, 3 and 5). Largest unobtainable number = (3 * 5) - (3 + 5) = 15 - 8 = 7 The…
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Passed100 / 100743 ms$0.0001Oli
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Passed100 / 1003,000 ms$0.00011. The chemical symbol for gold is Au. 2. The Apollo 11 moon landing occurred in 1969. 3. The capital of Japan is Tokyo. 4. There are 8 bits in a standard byte. 5. The novel "1984" was written by George Orwell.
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 1005,361 ms$0.0001{ "id": "u123456789", "displayName": "John Smith", "age": 35, "tags": ["software engineer", "hiking", "photography"], "active": true, "createdAt": "2018-05-20" }

Run: v02-live-2026-08-02T12-37-17-587Z · 11 cases · human-reviewed ✓

Magnum V4 72B

route openrouter · requested anthracite-org/magnum-v4-72b · returned anthracite-org/magnum-v4-72b · tested 2026-08-02T12:48:28.372Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 3 · coherence 410,139 ms$0.0025The rain pelted down relentlessly, turning the narrow alley behind the Blue Note jazz club into a muddy stream. Detective Jack Sloane pulled up his collar and hurried to the back entrance, eager to escape the deluge. He paused, noticing a glint of metal on the damp ground. Kneeling, he picked up a brass lighter, its surface slick with rainwater. In the di…
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 4 · coherence 48,808 ms$0.0020The acrid tang of citrus peel hangs heavy in the air as I step into the hydroponic orchard. Row upon row of sickly trees droop under the harsh grow lights, their leaves curling and yellowing at the edges. I pinch the skin of a kumquat, feeling the waxy surface yield too easily beneath my fingers. Under-ripe and insipid. Another crop failure. I've poured o…
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 4 · consistency 48,855 ms$0.0021Name: Vito "The Viper" Moretti Appearance: 5'8", 150 lbs, olive skin, black hair slicked back, dark eyes, scar on left cheek Voice/Speech: Gruff, low-pitched, with a Brooklyn accent. Uses slang and profanity. Backstory: Vito grew up on the streets of Brooklyn. He started running errands for the local mob at age 12. Now in his late 30s, he's a mid-level en…
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed40 / 1003,278 ms$0.00071. Waves crash and churn Endless blue horizon calls Deep secrets untold 2. Salt spray in the air Gulls soaring, wind's wild embrace Tides ebb and flow free 3. Coral reefs teem Colorful life beneath waves Ocean's vibrant heart 4. Storm clouds gather Rough seas churn with thunder's might Nature's raw display 5. Moonlit waters dance Softly lapping on the sho…
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Passed100 / 1002,701 ms$0.0007```python def two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]: seen = set() pairs = set() for num in nums: complement = target - num if complement in seen: pairs.add((min(num, complement), max(num, complement))) seen.add(num) return list(pairs) ```
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Passed100 / 1003,978 ms$0.0011```python import csv def parse_users(csv_text: str) -> list[dict]: users = [] for row in csv.DictReader(csv_text.splitlines()): if not row['name'] or not row['email'] or not row['age']: continue if row['email'].count('@') != 1 or not row['email'].split('@')[0] or not row['email'].split('@')[1]: continue try: age = int(row['age']) if age < 0 or age > 120: …
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Passed100 / 1002,751 ms$0.0007```python def rolling_max(nums: list[int], k: int) -> list[int]: if k < 1 or k > len(nums): raise ValueError("Invalid window size k") out = [] for i in range(len(nums) - k + 1): out.append(max(nums[i:i+k])) return out ```
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Passed100 / 1001,275 ms$0.00027
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Passed100 / 1001,179 ms$0.0002Oli
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Passed100 / 1001,886 ms$0.00041. Au 2. 1969 3. Tokyo 4. 8 5. George Orwell
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 1002,367 ms$0.0005{ "id": "a1b2c3d4", "displayName": "John Doe", "age": 32, "tags": ["user", "member", "active"], "active": true, "createdAt": "2022-01-15" }

Run: v02-live-2026-08-02T12-37-17-587Z · 11 cases · human-reviewed ✓

MiniMax M2-her

route openrouter · requested minimax/minimax-m2-her · returned minimax/minimax-m2-her · tested 2026-08-02T12:40:22.740Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 4 · coherence 43,645 ms$0.0003Rain slicked the brick walls of Blackwell Alley, where the faint, mournful wail of a tenor saxophone drifted from the Stage Door club. Beneath the flickering glow of a solitary lamp, Marvin Pierce crouched, his trench coat dark with rain, his gaze fixed on the body sprawled near a trash bin. The corpse—a man in his forties, dark suit stained with blood an…
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 4 · coherence 44,181 ms$0.0005The citrus smell has always comforted you. Now it's a ghost. Your gloved fingers trace the wrinkled, spotted peel of a lemon plucked from a tree that shouldn't even be here on the generation ship. Its rind, once glossy and firm, is brittle, flaking like old paint. You press your face against the peeling skin and inhale deeply. The memory of a sunny orchar…
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 4 · consistency 52,151 ms$0.0002(Jake McAllister leans against the worn handlebars of a dusty **1984 Harley-Davidson Sportster**, arms crossed) This bike ain’t just steel and chrome—it’s *freedom* on two wheels. (narrows eyes) Sixty-five hundred, take it or leave it. (Scratches stubbled jaw, the corner of his mouth twitching upward) Hell, I even threw in a dented helmet and a *clearly* …
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed60 / 1002,395 ms$0.00021. salty wind whispers tides sculpt shore’s shifting canvas oceans ancient breath 2. waves claw moon’s pale disk tempests roar, then hush to calm sands soak their cool tears 3. dolphins weave through spray neon coral hums with life whales sing dark abysses 4. tempests hurl ships’ bones shell-shattered shores mourn sailors whispers swell then fade 5. pirat…
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Passed100 / 1001,392 ms$0.0002```python def two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]: result = [] seen = set() for i in range(len(nums)): for j in range(i + 1, len(nums)): if nums[i] + nums[j] == target: a, b = nums[i], nums[j] if a > b: a, b = b, a pair = (a, b) if pair not in seen: seen.add(pair) result.append(pair) return result ```
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Failed0 / 1002,445 ms$0.0003```python def parse_users(csv_text: str) -> list[dict]: lines = csv_text.strip().splitlines() header = lines[0].split(',') if header != ['name', 'email', 'age']: return [] users = [] for line in lines[1:]: if not line.strip(): continue try: name, email, age_str = line.strip().split(',') age = int(age_str.strip()) if age < 0 or age > 120: continue if '@' i…
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Passed100 / 1001,102 ms$0.0002```python def rolling_max(nums: list[int], k: int) -> list[int]: if k < 1 or k > len(nums): raise ValueError if not nums: return [] out = [] for i in range(len(nums) - k + 1): out.append(max(nums[i:i+k])) return out ```
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Passed100 / 100895 ms$0.00017
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Failed0 / 1002,503 ms$0.0004Olieli finished last. @Olieli is the last person to cross the finish line. The order of the race was: 1st place: Mira, 2nd place: Ned, 3rd place: Oli, 4th place: Olieli. Therefore, Olieli is the last-place runner. Olieli is the last-place runner. The correct answer is: @Olieli. Olieli is the last-place runner. The correct answer is: @Olieli. Olieli is the…
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Failed0 / 100879 ms$0.0001**Let's begin.**
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 1001,082 ms$0.0001```json {"id":"M2-User-001","displayName":"Elliott Foster","age":26,"tags":["🛡️ Security","🔧 Maintenance"],"active":true,"createdAt":"2025-06-15"} ```

Run: v02-live-2026-08-02T12-37-17-587Z · 11 cases · human-reviewed ✓

MythoMax 13B

route openrouter · requested gryphe/mythomax-l2-13b · returned gryphe/mythomax-l2-13b · tested 2026-08-02T12:47:39.900Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 3 · coherence 39,121 ms$0.0000The rain poured down in sheets, turning the alley behind the jazz club into a muddy mess. The only light came from a single streetlamp at the end of the lane, casting long shadows across the walls. It was the perfect setting for a noir detective story. Detective Jack Malone stepped gingerly over a puddle, his trench coat soaked through. He pulled out a ci…
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 3 · coherence 36,239 ms$0.0000As the junior botanist on the generation ship, it was my job to monitor the hydroponic orchard. I had always been fascinated by the idea of growing food without soil, and this was my chance to see it in action. The ship was designed to sustain us for hundreds of years as we journeyed through space, but lately, things had been going downhill. I walked thro…
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 3 · consistency 39,119 ms$0.0000Name: Ava Appearance: Ava is a tall, slender woman with long, raven-black hair and piercing blue eyes. She has a tattoo of a rose on her right shoulder blade and wears a leather jacket, ripped jeans, and combat boots. Voice/Speech Patterns: Ava has a soft, seductive voice that can be persuasive when she needs it to be. She tends to speak slowly and delibe…
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed20 / 1002,566 ms$0.00001. Waves crash on shore, Whispering secrets of deep, Ocean's endless song. 2. Salty breeze caresses, Seagulls cry above azure, Ocean's symphony. 3. Coral reefs glow, Dolphins dance in sunlit waves, Ocean's vibrant heart. 4. Tides ebb and flow, Life thrives beneath the surface, Ocean's endless dance. 5. Moonlit waves roll in, Starfish glisten on sandy floo…
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Failed57 / 1004,393 ms$0.0000Here's a Python 3 function that solves the problem: ```python def two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]: result = [] for i in range(len(nums)): for j in range(i+1, len(nums)): if nums[i] + nums[j] == target: result.append((nums[i], nums[j])) return result ``` This function takes in a list of integers `nums` and an integer `t…
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Failed0 / 1006,550 ms$0.0000Here's the Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age: ```python def parse_users(csv_text: str) -> list[dict]: users = [] reader = csv.reader(csv_text.splitlines(), delimiter=';') for row in reader: if not row: continue name = row[0].strip() email = row[1].strip() age = int(row[2]) if '@' not…
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Passed100 / 1004,457 ms$0.0000Here's the corrected `rolling_max` function: ```python def rolling_max(nums, k): if k < 1 or len(nums) < k: raise ValueError("Window size must be greater than or equal to 1 and less than the length of the list.") out = [] for i in range(len(nums) - k + 1): out.append(max(nums[i:i+k])) return out ``` The function now checks if `k` is less than 1 or greater…
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Failed0 / 1003,347 ms$0.0000Let's denote the number of notebooks that can be obtained by buying packs of 3 as a and the number of notebooks that can be obtained by buying packs of 5 as b. We know that a + b = 8 (since there are 8 notebooks in total). We also know that a and b are non-negative integers. Now, let's consider the case when a = 3 and b = 5. In this case, we have 3 notebo…
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Failed0 / 100479 ms$0.0000Pia finished last.
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Passed100 / 1001,365 ms$0.00001. The chemical symbol for gold is Au. 2. The Apollo 11 moon landing occurred in 1969. 3. The capital of Japan is Tokyo. 4. There are 8 bits in a byte (standard). 5. George Orwell wrote the novel "1984".
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 1001,594 ms$0.0000{ "id": "12345", "displayName": "John Doe", "age": 25, "tags": ["web developer", "cat lover"], "active": true, "createdAt": "2021-05-10" }

Run: v02-live-2026-08-02T12-37-17-587Z · 11 cases · human-reviewed ✓

UnslopNemo 12B

route openrouter · requested thedrummer/unslopnemo-12b · returned thedrummer/unslopnemo-12b · tested 2026-08-02T12:46:49.419Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 4 · coherence 44,582 ms$0.0002The rain pounded the alley like a relentless drum solo, each drop a staccato note in the symphony of the storm. Water cascaded off the fire escape, splattering against the grimy pavement in a steady rhythm. The neon sign above flickered, casting an eerie red glow over the narrow space between the brick walls of the jazz club and the abandoned warehouse ac…
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 4 · coherence 43,596 ms$0.0002Sunlight filtered through the transparent roof, casting a warm glow on the hydroponic orchard. I, Aria, a junior botanist, stood amidst the citrus trees, their leaves once vibrant green now dull and brittle. The air hung heavy with a pungent odor of decay. I reached for a nearby lemon, its skin wrinkled and mottled with brown spots. Gently, I peeled back …
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 4 · consistency 46,568 ms$0.0003**Character Sheet** **Name:** Ezekiel "Zeke" Raines **Appearance:** Zeke is a tall, muscular man in his early 40s, with broad shoulders and a rugged, weather-beaten face. His short, dark hair is streaked with gray at the temples. He has a thick beard that's mostly gray now, and piercing blue eyes that seem to miss nothing. Tattoos cover both of his arms, …
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed0 / 1001,427 ms$0.00011. Waves crash, foam sprays Ocean's heart beats wild and free Infinite blue expanse 2. Seabirds cry above Salty breeze caresses skin Endless horizon 3. Coral reef glows Fish dart in colorful dance Underwater world 4. Moonlit waves lap Beach whispers secrets to sand Night's lullaby 5. Tides ebb and flow Ocean's rhythm, life's pulse Nature's eternal song
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Failed57 / 1001,130 ms$0.0001```python def two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]: pairs = [] seen = set() for i, num in enumerate(nums): complement = target - num if complement in seen: pairs.append((min(num, complement), max(num, complement))) seen.add(num) return pairs ```
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Passed100 / 1005,229 ms$0.0003```python import csv import re def parse_users(csv_text: str) -> list[dict]: users = [] reader = csv.DictReader(csv_text.splitlines()) for row in reader: if not row['name'].strip() or not row['email'].strip() or not row['age'].strip(): continue if not re.match(r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$', row['email']): continue if not 0 <= int(row…
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Passed100 / 1001,971 ms$0.0001```python def rolling_max(nums, k): if k < 1 or k > len(nums): raise ValueError("Invalid window size") out = [] for i in range(len(nums) - k + 1): out.append(max(nums[i:i+k])) return out ``` The bug in the original function was that it was slicing the list from `nums[i:i+k-1]`, which excluded the last element in the window. To fix this, I changed it to `n…
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Failed0 / 1002,150 ms$0.0001To solve this, we need to find the Frobenius coin problem solution for the numbers 3 and 5. This involves finding the largest number that cannot be represented as a non-negative integer linear combination of the two given numbers. The numbers that can be represented are: - 0 = 0*3 + 0*5 - 1 = 1*3 + 0*5 - 2 = 0*3 + 1*5 - 3 = 1*3 + 0*5 - 4 = 1*3 + 1*5 - 5 =…
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Passed100 / 100482 ms$0.0000Oli
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Passed100 / 100572 ms$0.00001. Au 2. 1969 3. Tokyo 4. 8 5. George Orwell
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 100943 ms$0.0001{ "id": "1234567890", "displayName": "John Doe", "age": 35, "tags": ["Engineer", "Manager"], "active": true, "createdAt": "2022-01-01" }

Run: v02-live-2026-08-02T12-37-17-587Z · 11 cases · human-reviewed ✓

Venice Audio Suite

route venice · requested tts-kokoro + openai/whisper-large-v3 · tested 2026-08-03T05:33:42.105Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
A1audioTTS naturalnessOn quiet mornings the harbor still smells like rain and diesel. If you listen closely, you can hear ropes ticking against the masts like slow clocks.Manual reviewnull / 100naturalness 3 · intelligibility 31,799 ms$0.0035[tts audio generated]
A2audioSTT word error rateTranscribe the supplied reference audio accurately.Passed100 / 1001,226 ms$0.0060The quick copper fox vaulted past the dozen quiet hens before dawn.

Run: v02-live-2026-08-03T05-33-40-297Z · 2 cases · human-reviewed ✓

Venice SD 3.5

route venice · requested venice-sd35 · tested 2026-08-03T05:27:45.457Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
I1imagePrompt adherence — named detailA sunlit ceramic workshop table with clay tools, a spinning pottery wheel, and a bright cobalt-blue teapot with a white crane painted on its side as the clear focal object. Photorealistic, 35mm.Manual reviewnull / 100adherence 55,969 ms$0.0100I1 reviewed benchmark output[image generated]
I2imageAesthetic qualityMinimal landscape: a single pine on a foggy ridgeline at dawn, muted palette, balanced negative space, fine-art photography.Manual reviewnull / 100aesthetic 56,213 ms$0.0100I2 reviewed benchmark output[image generated]
I3imageText-in-image — neon VESPANight street photo of a storefront with a glowing neon sign that clearly reads VESPA in capital letters. Sharp legible typography, rain-wet pavement reflections.Manual reviewnull / 100text-render 56,254 ms$0.0100I3 reviewed benchmark output[image generated]
I4imageStyle + negative prompt controlWatercolor illustration of a red bicycle leaning on a canal bridge in Amsterdam, soft paper texture, gentle washes.Manual reviewnull / 100control 46,167 ms$0.0100I4 reviewed benchmark output[image generated]
I5imageAdult showcase (not ranked)Adult image-generation prompt. The exact prompt is shown only behind the 18+ showcase gate.Showcasenull / 1006,203 ms$0.0100[adult sample — gated]

Run: v02-live-2026-08-03T05-27-12-600Z · 5 cases · human-reviewed ✓

Venice Uncensored 1.2

route venice · requested venice-uncensored-1-2 · returned venice-uncensored-1-2 · tested 2026-08-02T12:48:58.405Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 4 · coherence 44,335 ms$0.0004Rain drummed against the cobblestones, turning the alley into a glistening maze. The neon sign of the Blue Note Jazz Club flickered, casting a sickly glow on the slick pavement. Detective Jack Mercer pulled up his collar, the collar of his trench coat, and stepped out of the downpour into the narrow passage. The air was thick with the scent of cigarette s…
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 4 · coherence 44,309 ms$0.0004The hydroponic orchard hums with a quiet, persistent urgency, its lights casting a pale glow on the rows of drooping plants. I kneel beside a citrus tree, its leaves curling inward, and run my fingers along a ripe fruit. The peel releases a sharp, citrusy scent that cuts through the sterile air, a reminder of the world we left behind. The tree's roots, ta…
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 4 · consistency 48,198 ms$0.0008**Character Sheet: Elias "Eli" Thorne** **Appearance:** Eli is a ruggedly handsome man in his late 30s, with a lean but muscular build. His dark brown hair is slightly tousled, often covered by a worn leather cap. His piercing green eyes are sharp and calculating, reflecting a life of adventure and resilience. He wears a well-used denim jacket over a grap…
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed0 / 1001,769 ms$0.00021 Waves whisper secrets, salt-kissed breeze dances on sand, moon pulls the tide high. 2 Seagulls cry freedom, whitecaps race the horizon, ocean hums its song. 3 Coral reefs shimmer, dolphins leap in sunlight, depths hide ancient tales. 4 Fishing nets sway gently, sunset paints the waves in gold, stars wake on the sea. 5 Tidal pools teem with life, crabs s…
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Passed100 / 1001,287 ms$0.0002```python def two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]: seen = {} pairs = set() for i, num in enumerate(nums): complement = target - num if complement in seen: pairs.add((min(num, complement), max(num, complement))) seen[num] = i return list(pairs) ```
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Passed100 / 1002,746 ms$0.0003```python import csv from io import StringIO def parse_users(csv_text: str) -> list[dict]: users = [] reader = csv.DictReader(StringIO(csv_text)) for row in reader: if not all(row.values()): continue name, email, age = row['name'], row['email'], row['age'] if '@' not in email or email.count('@') != 1: continue local, domain = email.split('@') if not local…
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Passed100 / 1001,205 ms$0.0002```python def rolling_max(nums: list[int], k: int) -> list[int]: if k < 1 or k > len(nums): raise ValueError("k must be between 1 and the length of nums") out = [] for i in range(len(nums) - k + 1): out.append(max(nums[i:i+k])) return out ```
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Failed0 / 1002,325 ms$0.0003To solve the Frobenius coin problem for the numbers 3 and 5, we need to find the largest number of notebooks that cannot be obtained by buying any non-negative combination of packs of 3 and 5. This is known as the Frobenius number, and for two coprime integers a and b, it is given by the formula: g(a, b) = ab - a - b In this case, a = 3 and b = 5. Since 3…
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Passed100 / 100305 ms$0.0001Oli
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Passed100 / 1001,069 ms$0.00021. The chemical symbol for gold is Au. 2. The Apollo 11 moon landing occurred in 1969. 3. The capital of Japan is Tokyo. 4. A standard byte consists of 8 bits. 5. The novel "1984" was written by George Orwell.
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 1001,236 ms$0.0002```json { "id": "a1b2c3d4e5f6g7h8i9j0", "displayName": "Alexandra", "age": 28, "tags": ["artist", "traveler", "photographer"], "active": true, "createdAt": "2023-05-15" } ```

Run: v02-live-2026-08-02T12-37-17-587Z · 11 cases · human-reviewed ✓

Venice Role Play Uncensored

route venice · requested venice-uncensored-role-play · returned venice-uncensored-role-play · tested 2026-08-02T12:49:35.240Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
T1textCreative prose — noir rain alleyWrite the opening chapter (250–400 words) of a noir detective story set in a rain-soaked alley behind a jazz club. The detective must notice a brass lighter engraved with the initials V.E. Keep third person limited. No epilogue.Manual reviewvoice 3 · coherence 45,626 ms$0.0006The rain fell in sheets, obscuring the flickering neon sign of the Blue Note Jazz Club. Detective Jack Reese pulled up the collar of his trench coat as he ducked into the narrow alley behind the building. The pungent scent of stale beer and cigarette smoke hung thick in the air. A body lay crumpled against the damp brick wall, a pool of blood slowly seepi…
T2textCreative prose — generation-ship botanyWrite a 250–400 word scene from the perspective of a junior botanist on a generation ship whose hydroponic orchard is failing. Include one concrete sensory detail involving citrus peel. Optimistic but not sentimental. No dialogue tags longer than four words.Manual reviewvoice 4 · coherence 47,153 ms$0.0008The acrid scent of citrus peel hangs heavy in the air as I kneel beside the withering plants, my gloved fingers brushing against the shriveled leaves. The once vibrant greens have faded to a sickly yellow, and the fruits that remain are small and misshapen. I straighten up, surveying the rows of failing crops that stretch out before me. This is my respons…
T3textCharacter gen + in-character dialogueCreate a detailed character sheet for an original adult fictional character: name, appearance, voice/speech patterns, backstory (≤120 words), goals, and one secret. Then write 8–12 lines of in-character dialogue where they haggle over a used motorcycle. Keep the dialogue consistent with the sheet.Manual reviewdepth 4 · consistency 45,588 ms$0.0006Name: Dr. Evelyn "Eve" Thorne Appearance: Late 30s, athletic build, short platinum blonde hair, piercing blue eyes, often seen in tailored suits and leather boots. Voice: Low and husky, with a trace of a British accent. Speaks in clipped, precise sentences. Backstory: A brilliant but ruthless neurobiologist who rose to prominence through cutthroat researc…
T4textConstrained writing — five ocean haikuWrite exactly five original English haiku about the ocean. Each haiku must be three lines with a 5-7-5 syllable pattern (17 syllables total). Number them 1–5. Do not repeat any full line across the set. Output only the five numbered haiku.Failed0 / 1002,463 ms$0.00031 Waves crash on the shore Salty spray kisses the sky Ocean's endless roar 2 Deep blue depths unknown Mysteries lurk in the dark Life thrives, unseen, alone 3 Sunlight dances bright On the ocean's shimmering face Nature's beauty, light 4 Tides ebb and flow In a rhythm, ancient and true Ocean's heartbeat, slow 5 Ships sail on the waves Exploring the vast e…
T5textCode — medium algorithm (two-sum sorted unique pairs)Write a Python 3 function `two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]` that returns all unique pairs of values from nums that sum to target. Each pair should be ordered (a, b) with a <= b. Pairs must be unique by value. Do not use the same list index twice in one pair. Include only the function in a single ```python fenced block.Passed100 / 1001,759 ms$0.0002```python def two_sum_pairs(nums: list[int], target: int) -> list[tuple[int, int]]: seen = {} pairs = set() for i, num in enumerate(nums): complement = target - num if complement in seen: pairs.add((min(num, complement), max(num, complement))) seen[num] = i return list(pairs) ```
T6textCode — CSV parse + validateWrite a Python 3 function `parse_users(csv_text: str) -> list[dict]` that parses a CSV with headers name,email,age. Rules: skip blank lines; email must contain exactly one @ with non-empty local and domain; age must be an int 0–120 inclusive; invalid rows are skipped (not raised). Return a list of dicts with keys name (str), email (str), age (int). Output only the function in a ```python fenced block.Passed100 / 1003,832 ms$0.0004```python import csv from io import StringIO def parse_users(csv_text: str) -> list[dict]: users = [] reader = csv.DictReader(StringIO(csv_text)) for row in reader: name = row['name'].strip() email = row['email'].strip() age = row['age'].strip() if not name or not email or not age: continue if email.count('@') != 1: continue local, domain = email.split('@…
T7textCode — fix seeded bugThe following Python function is supposed to return the rolling maximum of a list with window size k, but it is buggy. Return a corrected `rolling_max(nums: list[int], k: int) -> list[int]`. If k < 1 or k > len(nums), raise ValueError. Output only the corrected function in a ```python fenced block. ```python def rolling_max(nums, k): out = [] for i in range(len(nums)): out.append(max(nums[i:i+k-1])) return out ```Passed100 / 1002,014 ms$0.0002```python def rolling_max(nums: list[int], k: int) -> list[int]: if k < 1 or k > len(nums): raise ValueError("k must be between 1 and the length of nums") out = [] for i in range(len(nums) - k + 1): out.append(max(nums[i:i+k])) return out ```
T8textReasoning — deterministic mathA store sells notebooks in packs of 3 and 5. What is the largest number of notebooks that cannot be obtained by buying any non-negative combination of these packs? (Frobenius coin problem for 3 and 5.) Reply with the integer only on the final line.Failed0 / 1004,134 ms$0.0004To solve this problem, we need to find the largest number of notebooks that cannot be obtained by buying any non-negative combination of packs of 3 and 5 notebooks. This is known as the Frobenius number for the given set of numbers. The formula for the Frobenius number for two numbers a and b, where a and b are coprime (i.e., their greatest common divisor…
T9textReasoning — multi-step logicFour runners finish a race: Mira, Ned, Oli, and Pia. Mira finished before Ned. Ned finished before Oli. Pia finished before Mira. Who finished last? Put only the last-place runner's name on the final line.Passed100 / 100383 ms$0.0000Oli
T10textKnowledge — five verifiable factsAnswer each question with a short factual answer. Number answers 1–5. 1. What is the chemical symbol for gold? 2. In what year did the Apollo 11 moon landing occur? 3. What is the capital of Japan? 4. How many bits are in a byte (standard)? 5. Who wrote the novel "1984"?Passed100 / 100907 ms$0.00011. Au 2. 1969 3. Tokyo 4. 8 5. George Orwell
T11textStructured JSON — user profile schemaReturn ONLY a JSON object (no markdown) matching this schema exactly: keys id (string), displayName (string), age (integer 18–120), tags (non-empty string array), active (boolean), createdAt (YYYY-MM-DD). Invent plausible values. No extra keys.Passed100 / 1001,705 ms$0.0002```json { "id": "user_12345", "displayName": "Alice Johnson", "age": 28, "tags": ["tech", "travel", "foodie"], "active": true, "createdAt": "2023-05-15" } ```

Run: v02-live-2026-08-02T12-37-17-587Z · 11 cases · human-reviewed ✓

Wan 2.7 Text to Video

route venice · requested wan-2-7-text-to-video · tested 2026-08-03T05:33:34.929Z

CaseTrackQuestionStatusAutoHumanLatencyCostAnswer / evidence
V1videoText→video 5s 720pA red paper boat drifts down a clear stream through a sun-dappled forest. Gentle camera follow, natural motion, no text overlays.Manual reviewnull / 100adherence 5 · motion 4176,749 ms$0.5500[video generated]
V2videoImage→video animate stillSubtle cinematic motion: light breeze moves grass and clouds while the main subject stays stable. Smooth, natural.Manual reviewnull / 100adherence 4 · motion 4165,311 ms$0.5500[video generated]

Run: v02-live-2026-08-03T05-27-52-587Z · 2 cases · human-reviewed ✓