Skip to content
Episode from the podcast Foojay.io | Friends of OpenJDK and Java Programming
Testing the Untestable: LLM Security for Java Developers with Tiberius (#99) artwork

Episode

Testing the Untestable: LLM Security for Java Developers with Tiberius (#99)

Released Saturday, June 20th 2026 41 min

Show notes

About this episode

Your Java AI application is live in production. But have you tested whether it can be jailbroken, manipulated into revealing its system prompt, or tricked into printing content it should never output? In this episode, Iryna Dohndorf, Software Engineer at Karakun Group and creator of Tiberius, explains how to bring security testing to LLM-powered Java applications. We cover why traditional unit tests break down with non-deterministic systems, how the Scan-Fixture-Validate workflow works, what buff mutation testing is, and why even well-trained models can be cracked with something as simple as the grandmother attack. Topics include: Why LLM non-determinism breaks the classic input/output test modelThe Scan-Fixture-Validate principle and sharing test artifacts across teamsPrompt injection, jailbreaks, and emotional manipulation attacksBuff mutation: testing linguistic surface coverageProbabilistic security contracts and multi-trial scansFingerprinting and why your model choice should not be detectableLLM as a judge: using a second model as a guardrailGetting started with Tiberius in Spring Boot and LangChain4jGuestIryna Dohndorf - Software Engineer at Karakun GroupLinkedIn LinksArticle on FoojayTiberius on GitHubSecurity Testing Guide Timestamps00:00 Introduction of topic and guest01:05 The problem Tiberius wants to solve06:39 How "traditional" unit tests don't work for LLM integrations10:23 Scan-Fixture-Validate principle and sharing artifacts15:15 Using different skills, for example, the grandmother skill17:33 Testing for required versus forbidden bias19:35 The probes across nine attack categories used by Tiberius20:44 Buff mutation testing26:55 Using Tiberius in your pipelines and when to fail29:35 Using multi-trial scans31:14 Fingerprinting: which model you use, should not be detectable32:55 Combining multiple models, model as a judge34:41 Sharing JSON models to improve tests36:05 How to get started with Tiberius in Spring and with LangChain4j36:41 Quarkus not supp...

More from Foojay.io | Friends of OpenJDK and Java Programming

All episodes

Security Is Your Job: OWASP, AI and Safer Java Dependencies with Jim Manico (Foojay Podcast #105)

48 min

How safe is your Java application, and how safe are the dependencies it ships with? If you build software that handles sensitive data and you have no security process, OWASP's Jim Manico has a blunt word for it: negligent. In this episode, Jim Manico, founder of Manicode Security, Java Champion and OWASP project leader, explains what OWASP is and which of its projects matter to Java developers. He brought the OWASP Java Encoder, Java HTML Sanitizer and JSON Sanitizer back to life with AI-assisted workflows, and tells us what he learned along the way. Topics include: When to use the OWASP Java Encoder versus the Java HTML SanitizerWhy untrusted input must be checked on both the client and the serverHow a few kilobytes of nested CSS can cause a denial-of-serviceWhy chained small vulnerabilities become a major one in the age of AIWhy Jim wants far fewer third-party libraries, and how to keep the rest up to dateWhat to do first when your team has no security process Guest: Jim ManicoFounder of Manicode Security, Java Champion and OWASP project leaderJim on LinkedIn Links: Full show notes with all linksOWASP Java Security Libraries: Please UpdateOWASP Java EncoderOWASP Java HTML Sanitizertrim-reaper on GitHub Timestamps:00:00 Introduction of topic and guest02:11 What is OWASP?04:10 How AI influences the speed of OWASP changes. Who contributes.06:30 Which OWASP projects you should know as a Java developer07:30 How Jim uses AI to evolve his libraries08:28 When to use OWASP Java Encoder and OWASP Java HTML Sanitizer13:58 User input needs to be checked in all steps (UI and backend)16:07 Man-in-the-middle attacks (eMuhdra)17:17 How to keep your systems up-to-date + minimize your dependencies22:34 How CSS can cause a denial-of-service problem24:52 How multiple small security issues can be combined into a major issue26:36 How to handle the growing number of discovered vulnerabilities29:13 The era of coding is coming to an end but you can still be a master developer32:45 How t...

53 CVEs and Two Emergency Releases: Why Java Security Holds Up (#104)

44 min

A record 53 CVEs in one Java update, followed by two emergency releases within days. Yes, a lot of issues are being found, but Java is still strong: a whole community and the commercial distributors are behind every fix. In this episode, Simon Ritter, Deputy CTO at Azul, Java Champion and member of the OpenJDK Vulnerability Group, explains what happened over the past weeks and what it means for your applications. We cover how a vulnerability travels from a report to a patch, why AI is changing the number of fixes, and why you should be ready to update at any moment. If you want to understand how CVEs are handled behind the scenes and what the different types of security updates mean, this episode explains it. Topics include: What a CVE and a CVSS score really tell you, and why low scores can be chained into serious attacksHow vulnerabilities are kept secret and encrypted in the OpenJDK Vulnerability Group until a patch existsWhy the average of 7.6 CVEs per quarter jumped to 53, and the move toward monthly security updatesThe eMudhra certificate problem and the two out-of-cycle releases that followedThe backporting workload behind every supported Java version, from 27 down to 6CPU, PSU and CSPU explained, and what free versus paid OpenJDK builds mean for your patching Guest: Simon RitterDeputy CTO at Azul and Java ChampionSimon on LinkedIn Links: Full show notes with all linksFree Java, Real Risk: Why Unsupported OpenJDK Is Now a Security GambleAzul Zulu October 2026 Quarterly Update and Out-of-Cycle Release Timestamps:00:00 Introduction of topic and guest01:59 What is a CVE and how are they reported and handled?08:45 How the Java security release cycle is changing and each have more CVEs11:48 How AI impacts the discovery of CVEs14:09 What was the eMudhra problem causing two out-of-cycle security updates?21:49 The impact of creating an out-of-cycle update (and backports)26:05 How safe will OpenJDK builds be in the future? And dependencies?29:06 Free versus paid vers...

Killing Reflection at Build Time: Avro, Codename One and Java Codegen (#103)

44 min

Runtime reflection is a quiet tax on the JVM. It is flexible, it is everywhere, and it is easy to stop noticing it until you go looking for where your CPU is actually going. Two teams did exactly that, in completely different corners of the ecosystem, and arrived at the same answer: move the work to build time. In this episode, Dmitry Moskalyuk explains the model layer behind his Foojay article on reflection-free serialization, where JSON, XML, YAML and TOML all end up as Avro. Shai Almog explains the build-time codegen pipeline of Codename One, which aims for no reflection, no service loader and no Class.forName. We compare the two and ask whether generated code is really something to avoid. Topics include: Where Jackson's reflection cost really lives, and what Blackbird does about itJackson versus kotlinx.serialization, and why a missing feature sent the team back to JacksonWhy a UI framework bans reflection, and how that differs from the backendIs generated code bad? Lombok, Manifold and jOOQWhat Records and Project Leyden change for reflectionBuild time cost versus runtime cost, and a single source of truth for your data modelShould you still use reflection in a new project? Guests:Dmitry Moskalyuk, Senior Backend EngineerDmitry on LinkedInShai Almog, Author, DevRel and open source developer working on Codename OneShai on LinkedIn Links: Dmitry's article on FoojayShai's article on FoojayJackson BlackbirdApache AvroProject LeydenFull show notes with every link Timestamps:00:00 Introduction of topic and guests01:50 About the new Foojay, contributing with GitHub pull requests03:17 What is Apache Avro04:26 What is Codename One06:31 What is Blackbird in Jackson and compared to kotlinx mechanisms for different content parsers10:01 Why Codename One focuses on "no reflection, no service loader, no Class.forName"14:23 Is automatically generated code bad (Lombok, Manifold, JOOQ)?17:47 Java evolutions that bring improvements into the system (Records, Project Leyden)21:31 I...

We Left WordPress: CMS vs Static Sites, Hugo vs Jekyll vs Roq (#102)

38 min

Foojay.io quietly moved off WordPress and onto Hugo. No more CMS login, no database, no admin panel — just Markdown and AsciiDoc files, built and deployed straight from GitHub. So is a traditional CMS still the right way to run a content site in 2026, or has static tooling caught up? In this episode, Andy Damevin and Holly Cummins from the Red Hat Quarkus team join to talk through it. Andy created code.quarkus.io and Roq, Quarkus's own Java-based static site framework; Holly led the migration of quarkus.io itself from Jekyll to Roq. We go through the real security numbers behind WordPress's plugin ecosystem, and put very different static site generators head to head: Hugo and Jekyll, the veterans, versus Roq, the newcomer. Topics include: Why WordPress core is usually fine, and the plugins are where the trouble is — over 10,000 plugin vulnerabilities found since 2025What "recovery" means for a static site versus a CMS after a hack or defacementJekyll, Hugo, Roq and JBake compared, and why staying in Java matters (or doesn't)How to extend Roq as a Java developer, from a small PR to writing your own pluginWhat an actual WordPress-to-Roq migration looks like in practiceWhere a CMS still earns its keep despite all of this Guests: Andy Damevin on LinkedIn - Principal Software Engineer at Red Hat, Quarkus core team, creator of code.quarkus.io and Roq Holly Cummins on LinkedIn - Senior Principal Software Engineer, Red Hat Quarkus team, led the quarkus.io migration from Jekyll to Roq Links: Migrate from WordPress to RoqRoqHugoFull show notes Timestamps:00:00 Introduction of topic and guests03:29 Why WordPress is not the best choice for most sites04:36 Difference between CMS-driven websites and static websites07:02 Jekyll versus Roq and other systems11:04 How to extend Roq12:59 Moving from WordPress to Roq14:43 Pulling data with JBang or Roq from external sources into your publication process16:09 Why should I use Roq instead of Jekyll or Hugo?24:33 Static websites can be...

Java 27 in Practice: Smaller Heaps, Smarter Defaults, and Valhalla on the Horizon (#101)

57 min

Episode 101 of the Foojay Podcast. JDK 27 arrives on 15 September 2026 without finalising a single new piece of language syntax. It is still worth the upgrade: object headers shrink from 64 bits to 32, which the JEP measures at 22% less heap and 8% less CPU on SPECjbb2015, and G1 becomes the default garbage collector on every machine. Both changes live inside the JVM, so an application you compiled years ago picks them up with no code change at all. Simon Ritter, Deputy CTO at Azul and Java Champion, takes us through all nine JEPs in the release, explains why a non-LTS version is still worth testing against, and makes the point that "long-term support" describes the binary you download rather than anything in OpenJDK itself. We also cover the move from quarterly to monthly security updates that started in August 2026, and then look ahead to March 2027, when Project Valhalla reaches its first preview after roughly twelve years of work. Topics include: Compact object headers by default, and what 22% less heap actually buys you in productionPost-quantum key exchange in TLS 1.3, and the harvest-now-decrypt-later attack it defends againstJFR redaction: keeping access tokens and passwords out of flight recordings you shareStructured concurrency in its seventh preview, and why an evolving API stays in previewThe Vector API's twelfth incubation, and what it is waiting forValue Objects in Java 28: a new value keyword, 179,000 lines, 1,800 files, and why it will not be final in JDK 29Whether a Simple JSON API in the JDK will survive its previews, or go the way of string templates Guest: Simon RitterDeputy CTO at Azul, Java ChampionSimon on LinkedInSimon on Foojay Links:Full show notes, with every JEP linkedJDK 27What CSPUs Mean for Your Release PipelineWhat to Know About Garbage Collection as a Java Developer!Project Valhalla Timestamps:00:00 Introduction of the topic and guest01:27 How long Simon has been doing Java01:54 Why release 27 is important, even when not being a Lon...

This podcast, its content and its artwork are not owned by, affiliated with, or endorsed by Castquire.

Up next

  1. Nothing queued. Use + Queue on any episode to line it up.

0:00 0:00