Why Gherkin Is the Right Testing Tool for RPG Migration — And It Has Nothing to Do With BDD

Translating the code is nice, but what you really need is a knowledge transfer protocol: the medium that gets thirty years of undocumented business logic out of a retiring developer’s head and into a form a junior Java developer can execute and the business people can understand (and approve).

We will see how to use Gherkin for that. The BDD crowd will tell you Gherkin is about collaboration and living documentation for greenfield features. They’re not wrong, they’re just describing the least interesting thing it does.

Unit tests are useful, but they solve technical questions. Gherkin instead answers an organizational one. In an RPG migration the organizational question is the hard one, and pretending otherwise is how projects die.

Stop Optimizing for Coverage. Optimize for Auditability.

Everyone benchmarks migration test suites on coverage. Wrong metric. When a regulated business migrates a core RPG system, the deliverable is not coverage — it is a demonstration of behavioral equivalence that a non-programmer will stake their signature on. You may put forward all the witty arguments you want, but in the end you need to convince the business people you did not break anything or you will not be allowed to ship.

Here is the number that should end the debate: 72% of mainframe applications contain undocumented business rules. That statistic is not a data-quality footnote — it is the entire problem. Those rules were encoded directly in RPG, maintained by people who have been there since the Reagan administration, and formalized precisely nowhere else. The migration is not a code translation project. It is a project to extract a specification that has never existed, and to prove you extracted it correctly.

In a lot of projects they are just content with Approval Tests that will happily capture program output and flag regressions — great, you’ve automated the coverage. But the artifact it produces is readable only by developers. The business owner who knows that the 2.5% early-payment discount doesn’t apply if the customer’s payment-hold flag was set at any point in the prior 90 days — that person cannot open a Java assertion and confirm you got it right. So they don’t. They trust the developer who translated the rule. And that trust — a developer who has never seen RPG, vouching for logic nobody wrote down — is the single weakest joint in the entire project. It is exactly where migrations snap. As a solution you can do a very pedantic migration, translating bit by bit the RPG code into Java or Python or whatever. This means that given you do not understand what you are translating it, you translate word by word.

Gherkin Is Readable by the Only People Who Actually Know the Rules

Gherkin’s plain-language syntax is not a nicety. In an RPG migration it is load-bearing structural steel. Look at a scenario for a discount calculation:

Given the accounts receivable database contains customer C12345 with balance 1250.00
  And the pricing table contains item I99 at unit price 47.50 with discount tier 3

 When program CALCDISC runs for customer C12345 ordering 10 units of item I99

 Then the calculated discount should be 7.50
  And the net invoice amount should be 467.50
  And no payment hold flag should be set
  And the audit log should record one pricing calculation event

A retiring RPG developer reads this and knows in five seconds whether 7.50 is right. A business analyst who has lived with these programs for twenty years knows whether the payment-hold logic is captured or missing. They can tell you what edge case this scenario doesn’t cover. They can do none of that from a Unit test, and no amount of developer discipline changes that — it’s not a skill gap, it’s a language gap. The people who own the truth do not read Java, Python, or C#. Any testing strategy that ignores this is optimizing for the convenience of the people who know the least about whether the system is correct.

This is not theory. Société Générale Corporate Investment Banking runs it at scale: 400 Cucumber scenarios, 20–30 minutes to execute the suite, six business analysts who write and own the Gherkin, sixty to seventy developers who write the step definitions, one QA person holding it together. Read that ratio again. Six BAs carry the specification, dozens of developers do the mechanical automation. That inversion is the point. The scarce, irreplaceable resource — domain knowledge — is where the specification authority lives. The developers’ job is deliberately dumb: run the same inputs against both systems, assert they match. In an RPG migration that is precisely the org chart you want, because your domain knowledge is walking out the door and your developers were hired for knowing Java, not RPG, COBOL, CL or DDS.

The Legacy System Is the Oracle. This Makes Migration the Easiest Place to Use Gherkin.

Here’s the objection BDD skeptics love: Gherkin scenarios are written up-front, before the system exists, and writing scenarios from scratch against undocumented legacy behavior sounds masochistic. Reasonable objection, and yet it is just plain wrong.

The legacy RPG system is the oracle. You are not guessing what correct behavior looks like — you observe it and record it. Migration is the one context where the oracle problem is already solved for you, which makes it strictly easier to write good Gherkin here than in the greenfield work BDD was invented for.

The workflow is record-then-specify, and the order is non-negotiable:

1. Run the legacy RPG program against controlled, representative inputs. Capture the DB changes, output files, flags set, every side effect.
2. Use an LLM to draft candidate Gherkin in bulk from the program name, the data dictionary, and the observed behavior. The LLM translates captured behavior into readable scenarios. It does not invent business logic — the moment you let it, you’ve reintroduced the exact problem you’re trying to kill.
3. Run a three-amigos review — domain expert, QA, developer. The domain expert corrects the Then clauses, challenges the edge-case coverage, and adds the cases bulk generation missed.
4. Developers write step definitions that run the same inputs against both legacy and migrated systems and compare.
5. Pass/fail on the migrated system is your behavioral-equivalence proof.

And be ruthless about this: the legacy system is always right. If the new system produces a different output for the same input, the new system is wrong — full stop, even when the RPG was doing something no sane person would have designed on purpose. A forty-year-old batch job is a sediment of accidental behavior, and that sediment is the specification. Your job is not to fix it. Your job is to reproduce it and get someone who understands it to sign that you did.

LLMs Make This Fast — Right Up Until You Skip the Review Step

The BMW/AToMIC industrial study is the number I’d put on a slide: across 13 real JIRA issues, 93.3% of LLM-generated Gherkin was syntactically valid with no edits, 95% was rated helpful by practitioners, ~95% time saved versus writing it by hand. The acceleration is real and it is enormous.

Now the number that keeps you honest. A separate study pitting GPT-4, Claude 3, and Gemini against 500 user stories found that which model you pick barely matters — input quality dominates. In a sense this is bad news, as it is easier to swap models than doing your homework. I am sorry, but we are not in a fairy tale and some reasoning (gasp!) is still needed.

Feeding the model thin user stories instead of detailed requirements dropped output quality 19–28%. Translate that into RPG terms: hand the LLM a program name and ask it to imagine the scenarios, and it will confidently hallucinate a specification for logic nobody documented. That is not acceleration. That is automating your way into the $41M failure faster. Or higher, depending on inflation, I guess.

So the rule is blunt. Do not ask the LLM to invent. Feed it observed behavior — actual captured outputs — plus the data dictionary and program context, and treat every generated Then clause as a draft that a domain expert has not yet approved. Teams that skip the review are not doing the fast version of this method. I do not know what they are doing, and I do not want to find out.

Worth noting where the format itself is headed. Aslak Hellesøy — who built Cucumber and Gherkin — has been developing Markdown with Gherkin (MDG), which lets scenarios live inside ordinary Markdown files (.feature.md), a strict superset of GitHub-Flavored Markdown. It’s a small signal pointing in exactly the direction this argument runs: the creator of Gherkin is pushing it toward prose, not toward code — toward the document a domain expert already reads, where the executable scenario and the narrative explaining why the rule exists can sit in the same file. For migration work that’s a nice bonus, not the reason to adopt Gherkin. The reason is everything above.

The JUnit Counterargument, Taken Seriously

I’ll give the other side its best shot, because it has one. Golden-master testing with ApprovalTests gets you coverage with almost no ceremony — no step definitions, no BA/developer coordination, no framework to babysit. For a technical team with deep domain knowledge, that is genuinely the better tool. Less overhead, same regression safety. I’d use it.

But RPG migration is definitionally not that team, and this is where the counterargument evaporates. The Java and Python developers writing your replacement have never seen RPG — many were hired specifically because they don’t carry legacy baggage. The domain knowledge sits with people who have used these systems for decades and with RPG developers who are, right now, leaving. There is no “technical team with deep domain knowledge” in an RPG migration. If there were, you wouldn’t be migrating in a panic. And if you are not in a panic you would be still kicking the can down the road, as you did for the last 15 years. The knowledge gap is not a side condition of the project — it is the project. Choosing a testing strategy that only your least-informed participants can read, in a project whose entire difficulty is a knowledge gap, is not a defensible trade-off. It’s a category error.

Gherkin bridges the gap. Not because a BDD manifesto blessed it, but because it is the only format in which a retiring RPG developer, a junior Java developer, and a business analyst can sit at one table and agree on what “correct” means before the person who knows is gone for good.

Three Ways Teams Get This Wrong

They treat equivalence as a developer problem. It is not. Developers writing the Java replacement cannot independently verify that their Then clauses encode the right rules — they don’t know the edge cases, that’s the whole reason the migration is hard. A suite only developers can read produces confidence, not correctness: the tests pass against the behavior the developers understood, which is not the behavior the business requires. False confidence is more dangerous than no tests, because it stops people from looking. And people are always looking for ways to be lazy: so do not give them one more!

They think BDD is for greenfield only. Backwards. Migration is the easiest place to use Gherkin, because the oracle already exists. You aren’t speculating about what the system should do. You are recording what it does and making the record reviewable by the people who can tell you whether “what it does” is even legal.

They write scenarios from documentation. There is no documentation. RPG systems are their own documentation, which means no documentation. Start from observation — run the programs, capture behavior, derive Then clauses from evidence — never from a hopeful reconstruction of what someone meant to build in 1987.

The Strumenta RPG-to-Python case study shows this playing out against a real ERP system.

The methodology behind this workflow — the step-definition patterns and the three-amigos session restructured for legacy migration — is covered in depth in Migrating RPG Code to Modern Languages.

The retiring RPG developer across the table from you is not a technical resource. They are the specification, and they have a leaving date. Gherkin is the format that gets what they know into something that survives their departure. Pick it for that reason, and ignore anyone who wants to argue with you about BDD.