Unreal Agent
Posted by trollied 1 day ago
Comments
Comment by dvt 1 day ago
Fractal tool discovery: tool taxonomy where an agent can "drill deeper" to find what specific tool it's looking for. Helps if/when polluting context with a zillion (mostly unnecessary) tools.
Leveraging splay trees: this is my favorite data structure and I think relatively unused in the context of agents/harnesses. A lot of times, recently-used workflows/tool-chains will be used again, so having those at the top of the search hierarchy is an awesome optimization.
Virtual containerized notebooks: models working in sandboxed (WASI) Python notebooks is incredible. Even local models (if given enough time) will usually converge on a good solution. Being able to mount tools/resources/fs is again, imo quite untapped. Some problems here are running native things (thing numpy/pandas) in containers is a nightmare (or impossible).
Anyway, happy to see other folks seriously doing stuff in this space. If anyone wants to collaborate on anything don't hesitate to reach out :) I'm also actively looking for a job or some contract gigs.
Fun times ahead.
Comment by ravenstine 1 day ago
Ugh, I can't help but respond to this one point. The fact that this is even an issue in the current year just tells us how screwed the software field is in a lot of ways. I don't mean that in existential terms, but of how divided we've become in terms of what's happened to human reasoning. On the one hand, you have people who apply deep thinking to develop the sort of approaches you described, and there's the exponentially growing segment of not-even-programmers who seem to never ask themselves whether any of their ideas have any sort of consequences.
Take MCPs for instance. Sure, I guess it can sometimes make sense to have a stateful API that is optimized for agents. Yet, more often than not, these MCPs frontload a ton of context where it's not needed, and solve problems where none existed. Merely sticking an API (MCP) in front of an API (CLI, REST, GraphQL) without a benefit that can be explained in a single sentence is lunacy and demonstrates a real lack of complex thinking.
Comment by rockostrich 14 hours ago
However, MCPs are a godsend to anyone trying to manage access to systems (especially SaaS systems) in a large organization that includes a ton of non-technical employees. We use at least a couple of SaaS products where RBAC is severely limited in what it can govern or where API keys do not have any kind of governance so handing one out is basically handing out read/write on the entire system. Being able to configure an MCP to only have read endpoints enabled it a very useful piece of duct tape. Obviously it would be great if the underlying API was just designed with proper governance but that's out of the user's control.
Comment by sublinear 1 day ago
But, you just very clearly described why human software engineers are in higher demand than ever before.
It's always been "screwed" in the sense that everyone else sucks at wielding the power.
What we're actually witnessing is a watershed moment where a lot of technologically illiterate people are getting left behind. Those of us on the literate side are left to fight amongst ourselves and the powers that be for control over the future. We have actually been doing an alright job all things considered (else this conversation wouldn't be happening). Politicians are aging out and we're the adults in the room now.
Comment by rozal 1 day ago
Comment by kaurimu 1 day ago
mcp with 2 tools: execute(code) and search(query)
all other tools, mcps, apis, whatever are encapsulated by the one interface. new tools don't bloat the agent's context, and it can write its own code to perform more advanced and batch operations against the available tools (executed within a sandbox).
executor is a great implementation of this - https://executor.sh
opencode v2 also provides its own native implementation
Comment by never_giveup 1 day ago
Comment by spdustin 1 day ago
Comment by tracerbulletx 1 day ago
Comment by dvt 1 day ago
Comment by mejutoco 1 day ago
Comment by joshribakoff 1 day ago
Comment by aeon_ai 1 day ago
In this way, I tend to think of the context environment on an agent is the "agent nav" -- it presents context, allows progressive disclosure, and if poorly designed, makes the agent flounder as a poorly designed UI/UX does.
Comment by mrlowlevel 1 day ago
Comment by wyre 1 day ago
Comment by securetron 1 day ago
Comment by tekacs 1 day ago
For some reason they're comparing their harness running on Astra xhigh to Codex with Astra max?
---
Also worth noting that OpenAI just added support for async tool calling to their harness, which isn't 1:1 with this approach, but is slowly ramping up in being able to provide something similar.
A big part of why Codex uses so many tokens is that it basically hot loops on polling tasks it starts for... absolutely no good reason: https://www.reddit.com/r/codex/comments/1wdlp7q/weve_discove...
I fixed it on my fork of Codex too, also back in Jan/Feb – I keep this patch rebased, for anyone who wants it: https://github.com/tekacs/codex/commit/9ffcf8db9078eae43d411...
It results in token savings similar in scale to those displayed here by Unreal.
---
My harness has used a slightly fancier version of the approach that Unreal is using since ~Feb, and... it definitely works excellently, but it's also assuredly smoother with Astra and other recent models that are more aware of async tool calling.
Comment by NostraDavid 1 day ago
[features.multi_agent_v2]
enabled = true
wait_agent_enabled = true
min_wait_timeout_ms = 10000 # 10s
default_wait_timeout_ms = 300000 # 5m
max_wait_timeout_ms = 3600000 # 1h
Seems to do the job and reduce usage; I just ran Astra for ~5 hours (using a goal) and it used the last 30% of my usage. And now they released GPT-6 Sol and Luna (which is basically 5.6 Sol and Luna, but a bit better and also 50% cheaper) ;_;Comment by AmazingTurtle 1 day ago
Comment by discobot2 1 day ago
each individual benchmark that is combined into agentic coding index was compared on xhigh between ua / codex / pi, and headline improvement was calculated on xhigh, but then agentic coding index pareto chart by default includex codex max, hense the confusion.
Comment by tapoxi 1 day ago
Comment by bryant 1 day ago
This plainly seems like a trademark issue in progress considering it's in the same exact domain and considering how many others were confused the way I probably was.
Comment by esher 20 hours ago
Comment by nosyke 1 day ago
Comment by tantalor 1 day ago
People, I'm begging you, please talk to a lawyer before launching.
Comment by MisterMunchkin 10 hours ago
Comment by wannabe44 15 hours ago
Comment by orliesaurus 1 day ago
Comment by neuronexmachina 1 day ago
>Computer software, namely, game engine software for video game development and operation; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of video games; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of content for virtual worlds and 3D platforms; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of motion pictures, television shows, videos, 3D animations, 3D simulations, 3D visualizations, virtual reality motion pictures, virtual reality television shows; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of virtual reality video games; Virtual reality game software; Virtual reality software for creating multimedia content; Augmented reality game software; Augmented reality software for use in mobile devices for integrating electronic data with real world environments for the purposes of entertainment
Comment by willy_k 1 day ago
Comment by llm_nerd 1 day ago
Comment by seanalltogether 1 day ago
Comment by NolF 1 day ago
Comment by Applejinx 1 day ago
Comment by max-privatevoid 9 hours ago
Comment by mey 1 day ago
Comment by swiftcoder 22 hours ago
Comment by ramesh31 1 day ago
Sure you can, trademarks are contextual. If they were a landscaping business it wouldn't matter. But within the same industry absolutely.
Comment by orliesaurus 1 day ago
Comment by heimidal 1 day ago
Comment by pixl97 1 day ago
Comment by cantalopes 16 hours ago
Comment by j_w 14 hours ago
Comment by pyrolistical 1 day ago
https://github.com/Pyrolistical/pi-notify
Now my pi agent setups its own trigger to notify itself when a background process is done
Comment by solarkraft 1 day ago
Comment by respectattentio 18 hours ago
I'm unlocking savings (in both speed and costs) but using a different way of thinking by integrating Decision models:
In short, LLMs spend so many turns on discovery, triage, and safety checks for everyday tasks. A decision model cuts those turns by preparing the relevant context before the prompt and handling the simple judgments itself, so the LLM only does the real work.
Here is the full article I published yesterday with in-house benchmark: https://tawfeks.github.io/decisionkit/
What do you think, guys? I need feedback on this.
Comment by urbandw311er 6 hours ago
Comment by tontinton 1 day ago
Would be interesting to compare to a harness optimizing for cost reduction too.
Comment by solarkraft 1 day ago
Comment by westoque 1 day ago
Comment by solarkraft 20 hours ago
> but clearly more documentation is better than lack of
I tend to disagree. This makes it very high signal. So far my agent has been successful at navigating the repo.
Comment by hi_hi 1 day ago
Comment by ricardobeat 20 hours ago
They work well, except that the agent will often simply call wait() immediately after, and also do it with a longer timeout, causing crashes/hangs to really slow down the process; whereas immediate tool calls have a built-in 30s timeout in most harnesses.
I imagine this one will suffer from similar problems.
Comment by lostmsu 15 hours ago
Comment by ozozozd 1 day ago
“On the surface, Unreal Agent achieves the same outcomes with fewer model turns and fewer input tokens.”
(Emphasis mine)
Comment by ChaseRensberger 1 day ago
i think the language primitives map well to the natural desire to use these tools across networks and in concurrent workloads.
another Go harness i keep up to date with is: https://github.com/boldsoftware/shelley (exe.dev team)
Comment by nylonstrung 1 day ago
It says that it removes tokens wasted while a model is waiting on synchronous tool calls. What tokens exactly is Pi using when waiting?
Comment by dumberquestions 1 day ago
Comment by zergrush 1 day ago
gpt 6 sol already made a lot of progress with caches
i have a feeling unreal agent might have decided to release now rather than getting sherlocked
i just think its very risky right now to spend too much time building harnesses or anything on top of codex or claude simply because frontier labs will just absorb whatever works
Comment by swiftcoder 22 hours ago
Comment by 332451b 1 day ago
Comment by nojs 1 day ago
What I'd love to see is a harness that deeply optimises for the best results obtainable out of non-frontier models. Many of these have 1M context windows, and most of it remains unused and under utilised in these harnesses, in my opinion.
Comment by jakkos 21 hours ago
Comment by solarkraft 20 hours ago
Comment by discobot2 1 day ago
Comment by jedisct1 19 hours ago
Comment by nojs 14 hours ago
Comment by joshstrange 1 day ago
It's both expensive to test alternative harnesses and it's expensive to develop them (if using API pricing).
Comment by faangguyindia 1 day ago
This is available in DSH via PTC mode.
Basically, instead of the completion API returning a sequence of tool calls, it returns a program that invokes the tools.
I think most are returning Typescript program
With ordinary tool calling, you may have to wait for a tool to return its result to the completion API before the model can continue and issue the next tool call.
With PTC, the generated program can invoke independent tools asynchronously, collect their results, and dispatch the relevant results back to the completion API in a single turn.
Claude and Codex added this as well, but there are some papers on this that show it's not good across all tasks, and models need to be trained for this specific technique, which is what AI labs are doing now.
Comment by discobot2 1 day ago
Comment by faangguyindia 1 day ago
Comment by bryant 1 day ago
How likely is it that they get sued into the ground in a year? They might have a strong suite of offerings even as soon as six months from now, but if the essence of a company's brand seems at jeopardy from the start, can I take the risk as a potential customer that they'd survive that kind of action?
Comment by cbg0 20 hours ago
Comment by solarkraft 1 day ago
What a good time for harness design. Just while OpenCode is growing up a bit and focusing on their harness. I’m delighted to see people focusing on good general solid harness design principles.
Someone make an OpenCode API (the best general agent end UI API I know) compatible server for it!
Sucks that I haven’t upgraded my client to V2 yet and I don’t really want to build on an outdated API ...
Edit: It would be cool to get streaming responses :) But I understand (and actually applaud) that the authors seem to have been very focused on the core mechanics.
Also note that this works with the responses API!
Comment by staticvar 1 day ago
Comment by ironqcold 1 day ago
Comment by vblanco 1 day ago
We found this massively outperforms any kind of agent like this and the official unreal MCP systems. Its similar to the Blender MCP which also just exposes scripting + very minimal api to claude code/others.
Comment by gillesjacobs 1 day ago
Comment by vblanco 1 day ago
Comment by rafaelmn 1 day ago
Comment by dirtbag__dad 1 day ago
Comment by rafaelmn 1 day ago
Actors can have identities and roles for RBAC, etc. you - cross agent communication is the same as sending any other message to a actors.
Not to mention that actors can be on your device, another device, etc. whatever the router can resolve - it's transparent to the agents.
Comment by solarkraft 1 day ago
Let a model get notifications when something it has accessed/ subscribed to changes, a tool produces output or it gets some other kind of message (for example by another AI or flesh agent). Wait until the next turn or wake it up. It can still decide to do nothing and wait.
Comment by ibash 1 day ago
Comment by hombre_fatal 1 day ago
Your technical ideas are just implementation details behind the interface. What's your idea for a better UX?
Comment by rafaelmn 22 hours ago
If you move away from the idea that agents are chat streams and treat them as processes/actors then you can start letting agents represent themselves/build their own interfaces.
And if you build enough introspection into the protocol because everything is message based you can have other agents build interfaces for them.
So like either standard GUI, or a voice assistant talking to you and delegating to agents, etc.
It's not an idea it's a way of thinking about agent systems - basically an agent OS.
Comment by dpc_01234 23 hours ago
Comment by orliesaurus 1 day ago
Comment by kushbhuwalka 1 day ago
Comment by solarkraft 20 hours ago
Comment by vb-8448 1 day ago
Comment by solarkraft 1 day ago
Comment by vb-8448 12 hours ago
Comment by Streamski 22 hours ago
Comment by gregjw 17 hours ago
Comment by monster_truck 17 hours ago
Comment by mrtesthah 1 day ago
Comment by oathvz 1 day ago
Comment by underlipton 1 day ago
Comment by redog 1 day ago
Comment by aslkalska 1 day ago
Comment by fidotron 1 day ago
Comment by lelandbatey 1 day ago
1. Agents usually depend on output of the commands they're running in order to make decisions about what to do next, so how do they behave while they're "waiting around" for the output they need?
2. Agents can _already_ run software async, via multiple mechanisms: raw CLI tools like "nohup", literally running tools in parallel (I see Sol do this often in the Opencode TUI harness), and using parallel sub-agents to e.g. research in parallel.
Thus I wonder, how much does this really improve speed vs only improving the "appearance" of getting more done faster?
Comment by mrtesthah 1 day ago
Comment by basisword 1 day ago
Comment by himata4113 1 day ago
Comment by Animats 1 day ago
Comment by Applejinx 1 day ago
Comment by syngrog66 1 day ago
Comment by FanaHOVA 1 day ago
Comment by jedisct1 1 day ago
Comment by calmingsolitude 1 day ago
Comment by neuronexmachina 1 day ago
Comment by fremi 17 hours ago
Comment by kelvo_ran 17 hours ago
Comment by pushpendraw 19 hours ago
Comment by jcmontx 1 day ago
Comment by SparkyMcUnicorn 1 day ago
Comment by jcmontx 17 hours ago
Comment by faangguyindia 1 day ago
Comment by im_down_w_otp 1 day ago