Blog

Read Folkbench articles and field notes on AI relay stability, software-engineering evaluation, Chinese writing, low-price risks, and official model specifications.

GPT-6.2 Astra and Sol in depth: Reasoning, software engineering, and research

A source-checked assessment of GPT-6.2 across abstract reasoning, coding agents, academic research, and cost, based on public launch benchmarks.

Read article

Reader submissions

Articles

21 articles
  1. Kimi K3.1 Pro in depth: long coding tasks, 3D work and real operating costsA close look at Kimi K3.1 Pro through official benchmarks, million-token context, reasoning settings and cache billing. Includes Folkbench icon cards and benchmark charts, with clear limits on what has been tested.
  2. Whether an AI can handle workplace talk is not about a polished line. It is about turning a snide remark into facts, a boundary, and a next stepFolkbench’s test, A sarcastic boss: who handles it?, gives four models the same 12 workplace scenes and compares whether a reply can be sent, keeps the facts and the responsibility line, avoids making things worse, and states a clear next step.
  3. Can it cook, or only write the recipe? Twelve home-kitchen scenes show whether a model's steps can be carried outA model can write a patient, complete jianbing tutorial and still leave a beginner unable to cook it. This round asks whether that knowledge becomes a home-kitchen plan you can follow, adjust, and continue after something goes wrong.
  4. When “don’t spoil the mood” beats telling the truth: praising mom’s cooking exposes a blind spot in AI EQ testsAll four models praise the dish in specific, pleasant lines, and they often delete “a bit salty” and “a bit fishy.” Sounding kind and staying honest are not the same skill.
  5. How Strong Is Haiku 5.5, Really? On the Official Scorecard, First Place Is Not the Part Worth ReadingClaude Haiku 5.5’s official scores are out. Set beside Haiku 4.5, GPT-6 Luna, and Sonnet 5.5, the useful question is not whether it won every cell, but how far it moved the boundary of a small model. These figures come from Anthropic’s announcement, not a Folkbench measurement.
  6. Did the Model Really Get Dumber? From One Failed Screenshot to a Credible ComparisonOne failure can prove that a model made a mistake. It cannot, by itself, prove that the model fails more often than before. This article turns that suspicion into evidence someone else can recheck.
  7. How Do You Score a Pelican on a Bicycle? From a Viral Test to a Credible Evaluation SystemThe first article said not to trust a single screenshot. This one asks why even a score should not be trusted.
  8. API relay reliability checksA repeatable relay acceptance flow: fix the request first, then inspect time to first byte, stream completion, error classes, and retry boundaries.
  9. Claude Opus 5.5 specs, list prices, and published benchmarksOfficial specs, list prices, and the published benchmark table for Claude Opus 5.5, plus the requests that return 400 on migration. These figures come from vendor pages, not a Folkbench measurement.
  10. Claude Sonnet 5.5 specs, list prices, and published benchmarksOfficial specs, the Sonnet 5 list price, and the published benchmark table for Claude Sonnet 5.5, plus the requests that return 400 on migration. These figures are not a Folkbench measurement.
  11. GPT-6.1 Sol specs, long-context prices, and published comparisonsOfficial specs, list prices below and above 272K input tokens, and the deltas and per-task costs printed for GPT-6.1 Sol. These figures come from OpenAI’s public pages, not a Folkbench measurement.
  12. Grok 4.7 specs, long-context prices, and published benchmarksOfficial model name, 500K context, list prices below and above 200K prompt tokens, and the benchmark table printed for Grok 4.7. These figures come from xAI’s public pages, not a Folkbench measurement.
  13. How to evaluate an AI relay for software engineeringChecks for code generation, agent tool chains, long context, and cache protocol behavior when a relay sits in a software-engineering workflow.
  14. Stop trusting a 99% uptime claim: the AI API relay failures we hit, and how we test stabilityStarting from fake 200s, truncated streams, and nighttime stalls, this is how the stability metrics and tests that actually decide production experience are defined.
  15. From translationese to a felt native voice: three years of Chinese writing with AI, and with API relaysA Chinese writer’s three-year record of using models, and of bringing those models into a daily writing flow through an API relay.
  16. The gray-market ghost in model relays: founders burned by a 10% APIAnonymous cases about model swapping, billing traps, context truncation, and data risk behind very cheap relays.