AI migrated legacy COBOL programs to Java, bugs included

arxiv.org

97 points by felineflock 4 days ago


matsemann - 4 days ago

Before, only the senior cobol programmers at the company understood and knew the codebase.

Now, no one does.

I can see the allure of moving away from a legacy cobol system. But an AI rewrite doesn't actually solve any of the issues with having a legacy cobol codebase. You just have a new system no one knows or understands.

toplinesoftsys - 4 days ago

The biggest problem is not that bugs are migrated with COBOL, but that lots of new bugs are going to be introduced. AI is not deterministic, it will be making tons of mistakes. The only realistic low-error approach is incremental step-by-step migration using Cursor or similar tools. However, it requires much more time as each step must be prompted, tested and committed manually. Any hope that one-shot migraton of a large code base will not introduce enormous number of bugs is very naive. LLM is very bad on handling long context - it is their nature unfortunately. There is no answer to this problem yet.

pacaro - 4 days ago

The largest test case was 4kloc.

There are hundreds of billions of lines of cobol in production.

The IRS alone has approx 160 cobol programs, averaging 230kloc each.

Taikonerd - 4 days ago

This sounds like what the company Mechanical Orchard offers: https://www.mechanical-orchard.com/

Their pitch is like, "we take your COBOL code, and all the real-world data you can give us. Then we model your COBOL program as a graph, where each node has inputs and outputs. Then we use AI to port each node, making sure that it has the same (input => output) mapping for all the test data you gave us."

madduci - 4 days ago

I remember a similar story shared this year at JAX2026 from the Sparkasse Group, they said they were using AI to migrate from COBOL, but they still were in the middle of the migration. Maybe they faced the same issues / problems? It seemed pretty zealous to me, that everything was working smoothly, but this article highlights the limitations

pjmlp - 4 days ago

Stuff like this are already a commercial product, it isn't only Zig to Rust rewrites going around.

https://www.pega.com/insights/resources/break-free-legacy-ma...

https://www.ibm.com/products/watsonx-code-assistant-z

https://global.fujitsu/en-global/pr/news/2026/03/30-01

https://www.rocketsoftware.com/en-us/insights/ai-powered-cob...

mtct88 - 4 days ago

Why should I convert COBOL to Java? LLM can write COBOL just as fine.

kukkeliskuu - 4 days ago

While not everything can be easily converted (IMS, CICS, reports, batch processing etc.), there are many situations where automated tooling can be helpful in migrations.

Related to this, I created a tool for situations where you want to compare COBOL code with Java code. It includes a preprocessing step where IMS etc. calls are converted to mocks that return JSON (from file), and also use JSON for input/output, and GnuCOBOL to run the program. More a proof-of-concept than production, but here is a link if somebody finds it helpful.

https://github.com/mikko-ahonen/coboltwin

Surac - 4 days ago

How does ai implement all the not Cobol parts a Cobol program rely on? Job Contol, CICS, sort processors? Cobol and mainframe technologies are non existant in java on any modern machine

aldente0630 - 4 days ago

As I understand it, the translation isn't done by an LLM but by a deterministic AST-based migrator. Also, carrying over the bugs is the stated goal.

moojacob - 2 days ago

I have been translating Motorola 6809 assembly into C. It is not that bad. I don’t find AI saves me much time writing the code because understanding it is still the bottleneck, but man it is good at explaining stuff.

prirun - 3 days ago

I just asked Google AI how LLMs translate COBOL code containing GOTOs into Java, a language that doesn't have GOTO. It gave several examples, one of them being this:

COBOL code:

  PROCESS-DATA.
      ADD 1 TO COUNTER.
      DISPLAY COUNTER.
      IF COUNTER < 10 GO TO PROCESS-DATA.
Java translation:

  while (counter < 10) {
    counter++;
    System.out.println(counter);
  }
If "counter" is 10 on entry, the COBOL code prints 11 while the Java code prints nothing. So not only keeping old bugs, but apparently introducing new ones too!

I wrote COBOL code for a few years at a job when I was a teenager. What makes legacy COBOL code difficult IMO is it can sometimes be very hard to maintain a mental execution state when examining the code, for several reasons:

1. all variables are global, aka, WORKING-STORAGE. You list all the variables used in the program and they are accessible to the entire program.

2. programs are divided into paragraphs. Control normally flows sequentially top to bottom through paragraphs, one executing after another. Except that the PERFORM statement can drastically alter this normal flow control, and you can't tell by looking at a paragraph how it will be executed. To do that, you have to look at all PERFORM statements that mention this paragraph or any paragraph physically before it, because in COBOL you can say PERFORM PARA1 THROUGH PARA27. If PARA13 is physically between PARA1 and PARA27, it's potentially going to get executed.

3. In true legacy COBOL, before structured COBOL was a thing (circa 1985), the main control flow statement in addition to PERFORM was GOTO. Lots of flag setting, and lots of GOTOs. So in the previous example, you can't tell if PARA13 is going to get executed because any prior statement might be a GOTO PARA14, skipping execution of PARA13. But even worse, you are still under the influence of the PERFORM THRU, so after PARA27 is executed, control returns to the statement following the PERFORM THRU, wherever that was. But if you GOTO PARA27, without being under a PERFORM THRU, then PARA27 is executed followed by the next sequential paragraph. Trying to figure this out statically by looking at the program can be very difficult, especially considering PERFORMs that are nested at runtime but may not be anywhere near each other in a code listing.

mpfh - 4 days ago

When they migrate to Java how are they migrating things like the messaging (send, receive) functionality?

anthk - 4 days ago

COBOL is not about the language, it's about the whole environment in the mainframe. LLM fanboys won't understand this. You need something like a mainframe with a 99.99% uptime no matter what happens in hardware, with live CPU swapping and such.

BigJono - 4 days ago

Yeah nah maybe fix the bugs before swapping the average COBOL dev for the average Java dev.

bigbuppo - 4 days ago

COBOL will never die. Whatever this is will only result in more COBOL being written.

hexasquid - 4 days ago

To IBM

As specified, please find 99997 correct parts and the 3 defects (do not use)

- 4 days ago
[deleted]
fock - 4 days ago

300 to 4000 lines of "production like" (whatever that is) cobol code which is easily ported to a non-mainframe env. Our's sometimes uses assembler in its innards, so good luck with real legacy code spanning a dozen files and 50k loc...

I recently threw in (want to check those intelligence metrics!) some real production code into a non-agentic system (just to get a feel how things perform without a custom harness) and results where ... interesting. The particular program uses some preprocessor no LLM we have access to (newest was GPT 5.5) has any clue about - so they confabulate what it could do (Gemini 2.5 didn't even notice there was a preprocessor...). This is expected of course but it somehow seems the problem of this technology that unless you feed it masses of data or mechanically break up the tasks in rote subunits, it just doesn't do anything sensible still...

zoom6628 - 4 days ago

All this ranting about COBOL. Makes me fairly sure AI hasn't tackled RPG3 yet :-D

dzonga - 4 days ago

there was some London company migrating Cobol to Rust - with 'a.i' assistance.

I don't remember their name, hopefully they stayed in business. since probably convincing financial firms that using Rust is better than Java is a mammoth task.

h_mirin - 4 days ago

[dead]

crewlesslab - 4 days ago

[flagged]

TimByte - 4 days ago

[flagged]

luciana1u - 4 days ago

[dead]