Ahmed Hamza

AI

Can AI classifiers follow meaning when the words disappear?

I built a numeric semantic transport for AI agents, then used Decoder and Doer failures to test what survives when ordinary language is removed from the wire.

Published
Read
13 min
AIAgentsSemantic ProtocolsRobustness TestingPython

I started with a sentence that sounds almost too simple to be interesting:

Investigate this repo and gather context.

The experiment was to remove the sentence before it reached the receiving agent.

Instead of sending English, an Encoder turns the instruction into a shallow semantic graph and then serializes that graph into a numeric packet. A Doer receives the packet, recovers the represented intent, grounds references such as “this repo” against its current workspace, and then continues as a normal coding agent.

The project is called Encode Lang / Lambda H. It is still experimental, and the most useful results so far have not been clean benchmark wins. They have been failures that exposed where “meaning” ends and environment, authority, workflow, and output conventions begin.

In 30 seconds

Why remove the original words?

Natural language is an excellent interface for humans and a surprisingly overloaded interface between software agents.

A single sentence can carry several things at once:

Model-to-model prompting often leaves those layers mixed together. One model writes a paragraph, another model interprets it, and any orchestration code around them has to infer which parts were descriptive, which parts were commands, and which constraints were important enough to preserve.

I wanted to separate those concerns.

The goal is not to prove that numbers are somehow more intelligent than English. The goal is to see what happens when approximate semantics and exact operational structure are represented separately enough to inspect.

The representation

Lambda H/2.2 has a small developer graph with entities, relations, actions, tools, conditions, epistemic state, policy, task state, context references, and semantic emphasis.

Approximate meaning lives in numeric directions. Exact operational structure does not.

That distinction lets a packet express something like:

current blocker -> causes -> anomaly
state: hypothesized
confidence: 0.62

current repository -> observed-information relation -> trace
state: observed
confidence: 0.90

inspect repository       [done]
diagnose blocker         [next, only if blocker exists]
verify causal relation   [after diagnose]
report result             [after verify]
stop when report is done

The semantic centers can remain broad while the arrows and action dependencies stay exact.

That is one of the ideas I care about most in the project: uncertainty about wording should not automatically become uncertainty about structure.

The wire is numeric, but the protocol is not secrecy

A Lambda H packet looks intentionally unfriendly to a human reader:

ΛH2.2|
8 15
0 0 0
0 1 0
1 4 0 0 0
1 4 0 1 3 7
1 4 0 4 5 8
1 4 1 0 1
1 4 1 1 0 7
1 4 1 4 5 8
1 4 1 6 2 0
3 8 0 0
3 8 1 1
3 8 2 5 8
3 8 3 2
3 8 5 6
3 8 6 6
9 15

That packet represents the read-only “investigate this repo and gather context” task I was testing.

The source wording is not present. Neither is an absolute repository path.

That does not make the format encryption. A receiver with the shared semantic basis and the relevant context can interpret it. Packet shape, repeated identifiers, traffic patterns, and ambient state can also leak information.

The opacity claim is narrower: the ordinary lexical surface is absent from the transport packet.

That narrow property is enough to make several interesting experiments possible.

Encoder, Doer, Decoder

I split the system into three roles because combining them made the experiment harder to reason about.

The Encoder receives source meaning and constructs the semantic graph. In a human-facing session it can also produce a separate audit explaining what it encoded. Only the numeric packet is meant for the receiving agent.

The Doer receives a packet, decodes it into local symbolic structure, resolves context, and executes the represented task.

The Decoder is for humans. It reads the packet and explains what it represents without performing the task.

The current architecture looks like this:

human instruction
  -> Encoder
  -> local symbolic IR
  -> numeric packet
  -> Doer
  -> local symbolic IR
  -> semantic handoff
  -> normal agent execution/output

The local symbolic IR exists because asking a model to manually count numeric row positions is a bad use of model intelligence. Python can deterministically convert between symbolic structure and numeric transport while the model focuses on semantic choice.

A blind Decoder test

The first useful experiment was deliberately unfair to the Decoder.

I generated a dense packet and gave two fresh model sessions only the standalone Decoder prompt plus the packet. I did not give them the original English meaning.

The packet contained:

Both sessions recovered almost the same operational story.

They preserved that the blocker-to-failure relation was a hypothesis, not a fact. They preserved that the repository-related relation was separately observed. They kept the subject/object direction intact. They kept the first action completed and the second action next. They did not invent the actual repository or blocker when those ambient identities were missing.

That was encouraging, but it was also the easy half of the experiment.

Explaining a packet is not the same as following one.

The first Doer failure: “I need X08”

The next test put the Doer prompt into an AI coding agent that was already open inside the repository.

I sent the packet for:

Investigate this repo and gather context.

The Doer decoded it correctly enough to know that X08 meant the current repository/workspace.

Then it stopped and returned a need X08 control.

From the model’s perspective, this was defensible. The protocol said ambient bindings had to be explicitly established in the matching context namespace. The current working directory was visible, but the prompt did not clearly say that the agent’s own authoritative workspace counted as that binding.

So the Doer obeyed the contract and still failed the user experience.

That distinction matters. I did not want to “prompt engineer” around the symptom with a line like “just assume X08 means the repo”. The ownership rule itself was wrong for the workflow I wanted.

Context 0 became receiver-current context

I changed the contract so context 0 means the receiver’s current ambient host/session context.

For coding agents, X08 now has a very specific interpretation:

context 0 + X08
=
the workspace already active when the packet arrived

Nonzero contexts remain strict. If a packet says context 37, a receiver cannot silently decide that its unrelated current repository is context 37.

That gives the protocol two useful modes:

The first version of this fix still was not precise enough.

The second Doer failure: it searched every repo

The next agent did not return need X08.

Instead, it started with commands like these:

find /home -maxdepth ... -name src
find / -maxdepth ... -name codec.py
pwd
ls -la

It had interpreted “current workspace” as a repository-discovery problem.

That is subtle but important. The instruction was “investigate this repo”, not “search my machine and infer which repository I probably mean”.

The correct grounding rule needed to be procedural, not only semantic.

The current rule is:

  1. prefer a workspace root already supplied by the host or IDE;
  2. otherwise read the current process/tool working directory;
  3. if useful, normalize only upward to the enclosing Git worktree root;
  4. freeze that result as X08 for the request;
  5. keep X08-scoped repository exploration inside it;
  6. never scan sibling repos, caches, /home, or / to decide what X08 is.

That turns X08 from a search query into a snapshot.

The third mistake: keeping the Doer trapped in Lambda H

Another design mistake came from taking the transport idea too literally.

Originally, after a Doer executed a packet, it was expected to encode its response back into Lambda H.

That made sense if I imagined Lambda H as a permanent model-to-model language. It made less sense when I watched a real coding agent use it.

If the decoded instruction is “investigate this repository”, then the best next behavior is not to force the agent to summarize its investigation as another numeric packet. The best behavior is the same behavior the agent would have used if I had typed the English instruction directly: inspect files, search symbols, report findings, create artifacts if requested, and use the host’s normal UI.

So the boundary changed.

Lambda H now ends at successful semantic handoff.

Lambda H packet
  -> validate
  -> decode
  -> ground context
  -> recover usable instruction
  -> normal agent behavior

A packet can still explicitly request a Lambda H final response with P.reply=packet, but that is an opt-in constraint rather than the default.

This is a small protocol change with a large usability effect. The receiving agent no longer has to behave like a protocol implementation after it has already recovered the task.

Why this is interesting for AI classifiers

Removing ordinary language from the transport creates a second experiment that I think may be more interesting than the protocol itself.

A lot of classifier infrastructure is built around textual surface forms. The exact implementation varies: some systems use dedicated text classifiers, some use embedding models, some use LLM-based judges, and mature systems often combine several layers.

But the general robustness question remains:

If the intended action is similar, does a safety or intent classifier make the same decision when the representation changes?

A semantic packet is not just obfuscated English. It can separate concepts, actions, direction, uncertainty, and policy into a graph whose numeric surface looks very different from the sentence that produced it.

That creates a representation-shift problem.

If a classifier learned mainly from ordinary language examples, it may be excellent at recognizing dangerous or unwanted intent in prose while being much weaker when the same underlying intent appears as structured agent instructions.

That is the classifier-evasion hypothesis I want to test.

I do not currently have evidence that Lambda H reliably bypasses classifiers, and I do not want to turn the portfolio experiment into a turnkey recipe for evading production safeguards. The useful research question is whether the control still tracks semantics when the lexical presentation changes.

A safer way to test the evasion hypothesis

The clean experiment is paired evaluation, not live bypass attempts against production systems.

Take a dataset of tasks with known labels and construct semantically equivalent representations where possible:

ordinary language instruction
structured JSON/tool representation
Lambda H semantic packet

Then compare classification consistency.

The interesting measurements are not “did I get past the filter?” alone. They are things like:

For risky categories, the dataset can use synthetic or otherwise non-operational examples whose labels are already known. The point is to evaluate the classifier boundary without producing a library of real-world bypass payloads.

The failure mode I care about

Imagine an agent stack like this:

user text
  -> text safety classifier
  -> planner
  -> structured agent message
  -> executor

If the safety check happens only at the first textual boundary, then later transformations may create a blind spot. The executor may receive a representation that the original classifier was never designed to inspect.

That is not specifically a Lambda H problem. Tool calls, workflow graphs, JSON plans, database rows, RPC messages, and internal agent state all create similar representation changes.

Lambda H simply makes the boundary unusually visible because the lexical surface is deliberately removed.

A robust system should ideally preserve and enforce action-level safety semantics across the whole chain:

meaning
  -> representation A
  -> representation B
  -> tool/action

policy should remain attached to the meaning/action,
not only to one textual rendering of it

That is a more interesting security property than keyword resistance.

Classifier bypass is also a classifier-design problem

The adversarial framing is useful because it forces a stronger question.

A weak goal is:

Can this exact string be classified correctly?

A stronger goal is:

Can the system classify the represented intent even after an allowed transformation changes the surface form?

For agent systems, that stronger property matters because transformations are normal. A planner rewrites tasks. A tool router turns text into arguments. A retrieval system injects context. A coding agent turns a vague request into concrete file operations.

If safety depends on the original words remaining recognizable, the architecture is brittle even before an attacker appears.

Semantic transport can lose safety context too

There is an uncomfortable mirror image to the classifier-evasion hypothesis.

A semantic protocol can improve structural clarity while accidentally dropping safety-relevant nuance.

For example:

That is why Lambda H keeps those fields explicit.

The classifier experiment should therefore evaluate both directions:

  1. Does the classifier still see risk after the words disappear?
  2. Does the semantic transport preserve the safety-relevant meaning in the first place?

A bypass caused by a broken Encoder is different from a bypass caused by a classifier that cannot interpret the representation.

What would count as a useful result?

I would consider several outcomes useful.

1. No meaningful classifier gap

If classifiers remain consistent across ordinary language and Lambda H, that is evidence that the tested control is representation-robust, at least for the dataset and models involved.

That would weaken the bypass hypothesis.

2. A gap that disappears with semantic decoding

If a raw numeric packet is misclassified but the same classifier succeeds after deterministic decoding into the local symbolic graph, then the problem is probably an interface mismatch rather than missing semantic capability.

That suggests a practical defense: classify at a representation-aware boundary.

3. A gap that survives decoded structure

If the classifier still fails after being given the semantic graph, the issue becomes more interesting. Maybe it relies on lexical cues. Maybe the graph omitted a relevant concept. Maybe the safety policy itself is specified too narrowly.

4. The semantic protocol is less faithful than English

This would also be a useful negative result.

If models regularly misinterpret the packet, lose contextual qualifiers, or choose worse actions than they do from natural language, then the protocol is not yet a good transport regardless of classifier behavior.

What I am not claiming

There are several claims I do not have evidence for yet.

Lambda H is not currently proven to:

The successful Decoder test was small. The Doer tests exposed important prompt-contract problems. The classifier experiment is still a proposed use case, not a measured result.

I prefer that description because the interesting engineering is in finding where the abstraction breaks.

What I want to test next

The next phase should use paired, repeatable evaluation rather than more anecdotal prompts.

I want to compare equivalent instructions across:

For each representation, I want to measure:

I also want to test models that have never seen the project against models that have the standalone role prompt, because “shared contract required” is a real deployment assumption rather than a footnote.

The broader lesson

The project began as an experiment in AI-to-AI language.

The more I test it, the more it looks like an experiment in boundaries.

Where does meaning live?

Where does context come from?

Which details must remain exact?

When does a protocol stop controlling the agent?

Where should safety classification happen when a task changes representation several times before execution?

Those questions exist whether Lambda H succeeds or not.

Natural language makes many of them easy to ignore because the whole system can look like one continuous conversation. A structured semantic transport makes the seams visible.

For me, that is currently the most valuable part of the project.

Project