Published: 2024-03-27T11:38:23+01:00
Partly a copy of AI Game dev journal entry
I have succeeded in removing heap allocation and dynamic dispatch from the GBNF grammar-limiting feature of the AI game. This is a mouthful way of saying that it's now easier and more performant to limit the output values of the large language model used to drive the game's content.
The code is much easier to read and write with these changes, and it takes advantage of Rust's powerful generics to avoid unnecessary heap allocation and dynamic dispatch, which is theoretically faster to execute. Not that dynamic dispatch would be particularly slow here.
As the codebase evolves further, I hope to be able to further obfuscate away the layers of abstraction and remove cloning, making limit generation even more performant.
After writing the previous dev journal yesterday about my attempts to remove the dynamic trait objects from GBNF limit creation, I have finally succeeded in figuring out the right set of trait bounds and associated types required to make it work properly. Initial implementation of removing the dynamic trait object was very easy, but I quickly ran into an issue with how "primitives" (i.e. single values like a number or string) vs "complex" (nested types with multiple fields) are handled.
This required creating two new types:
These two wrapper structs have a bunch of fancy trait bounds and associated types on them that allow instances to be created that hold the proper limit rule for the given field which is limited. That might be hard to understand, so here is a simplified example, directly from the game code.
// (Simplified) Struct definitions
pub struct RawCommandExecution {
// other fields removed.
#[gbnf_limit_complex]
pub event: Option,
}
pub struct RawCommandEvent {
pub event_name: String,
#[gbnf_limit_primitive]
pub applies_to: String,
#[gbnf_limit_primitive]
pub parameter: String,
}
// Limit creation
let applies_to = vec!["self", "uuid1", "uuid2", "uuid3"];
let all_uuids = vec!["uuid1", "uuid2", "uuid3"];
let event_limit = RawCommandEventGbnfLimit {
applies_to: GbnfLimitedPrimitive::new(applies_to),
parameter: GbnfLimitedPrimitive::new(all_uuids),
};
let limit = RawCommandExecutionGbnfLimit {
event: GbnfLimitedComplex::new(event_limit),
};
In this code, the event itself has two limited fields:
These are single String values. GbnfLimitedPrimitive takes a Vec of allowed values, without any heap allocation or dynamic dispatch. In contrast, the main struct has an Option field that can contain a single event. The generated GBNF limit struct mirrors the creation of the regular struct, and takes only one instance of GbnfLimitedComplex, again with no heap allocation or dynamic dispatch.
This also makes the code much easier to read, as it no longer requires a bunch of janky Box::new or into() invocations.
I am satisfied with the current state of GBNF grammar limiting. The next major steps for the AI game will focus on implementing a basic interactive command beyond exploring the world. This will test the limits of the events system as-is, and likely force my planned modifications to it (removal of the applies_to field and reworking of how parameters to events are handled).
This will be a stepping stone to implementing a "one-of" feature in the GBNF grammar rules generator, which will then allow hyper-specific events with their own types and specific fields.
Filed under: opensource, ai-game
License: CC-BY-SA-4.0.
‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗
⤴️ [/posts] 🏠 Home