Joint Optimization of Tool Creation and Use for Large Language Model Agents

Tool use has made language models much more capable, but an agent is still limited to the tools someone already wrote for it: a calculator, a search API, a Python sandbox. When the right tool doesn’t exist, the agent is stuck. We study how to train a single model that writes the tools it needs and then learns to use them well.

So when a tool is missing, why not let the model write one? Recent works like LATM, CRAFT, TroVE and KTCE already do this at inference time, but almost always with a strong frozen model writing the tool and a separate, weaker model using it. The writer never finds out whether its tool was actually easy to call. A tool can fail because its code is wrong, its schema is wrong, the two don’t match, or because it is technically correct but hard to use, and each of these needs a different fix.

So we built SMITH (Schema-grounded Multi-task Iterative Tool Honing), a reinforcement learning framework that trains tool creation and tool use inside the same policy. Each rollout is either a build task or a use task. In a build task, the model sees 4 example questions with answers, then writes a Python function and a JSON schema for it. The tool is tested on 16 harder held-out questions, so a tool that only pattern-matches the easy examples scores near zero. Tools that pass go into a shared tool pool. In a use task, the model gets a new question and three tools from the pool, one matching and two distractors, and only sees their schemas, not the code. Since the same model writes and later calls each tool, a confusing schema directly hurts its own reward.

We also keep the rewards separate instead of collapsing them into one score. A build task gets two reward signals: one for whether the tool is well-formed and actually answers the held-out questions when called, and one from an LLM judge that scores the code and schema quality. A use task is rewarded for a correct answer, with a penalty for using too many turns. This way each failure mode contributes its own gradient instead of being averaged away. The model that calls tools during build-task evaluation is also periodically re-synced to the latest policy weights, so it improves alongside the policy without scoring against weights that are still changing.

On 10 Reasoning-Gym task categories held out from training, a 4B Qwen3 trained with SMITH reaches 79.9 macro-average accuracy, the best of all methods we compared. That’s ahead of more elaborate inference-time pipelines like CRAFT (76.5), TroVE and KTCE, and ahead of an untrained Qwen3-30B-A3B writing its own tools (74.1). It also uses only about 100 output tokens per question, roughly 32x fewer than standard chain-of-thought (3,206). Interestingly, distilling from a bigger model is not enough: ReTool, distilled from Qwen-32B traces, gets the best score on seen tasks (92.2) but drops nearly 30 points on unseen ones.

SMITH also works outside its training data. Without any tabular or visual training data, it reaches 40.4 on TabMWP-Hard, the best of all methods, and 42.6 on GQA, 7.6 points above the best inference-time baseline on the same 4B backbone. The only method ahead on GQA is LATM distilled from GPT-4.1 (56.0).

The tools themselves also transfer to other models. When a frozen LFM2.5-350M calls tools written by our 4B model, its held-out accuracy goes from 11.6 to 42.9, slightly better than with tools written by an untrained Qwen3-30B-A3B (41.5). It works in the other direction too: giving SMITH’s tools to Qwen3-30B-A3B beats the same 30B model writing tools for itself, raising its overall score from 70.2 to 76.6, with the biggest jump on TabMWP-Hard (0.7 to 38.8).

The same recipe works on other backbones without any changes. Qwen3-8B goes from 72.2 to 81.7 on held-out tasks, and Granite-3.3-8B, which starts from a much weaker base, goes from 22.0 to 28.5. On BFCL v4, a function-calling benchmark SMITH never trained on, accuracy improves from 45.1 to 48.6 on Qwen3-4B and from 43.3 to 55.8 on Qwen3-8B.

So is joint training actually what makes the difference? In our ablation, pairing a separately trained 30B tool builder with a separately trained 4B tool user scores 58.9 on held-out tasks, lower than a single 4B model trained only on tool creation (68.8). Removing the evaluator re-sync (66.9) or the LLM judge (67.8) also hurts. Only the full setup, with both skills in one policy and separate reward axes, reaches the best overall score.

There is still plenty left to solve. SMITH is only trained on procedural reasoning tasks with exact verifiers, so we don’t know yet how well it works for open-ended retrieval, web navigation or embodied control, where there is no comparable reward signal. Our tools are stateless Python functions used for at most 5 turns, and multi-tool composition or longer workflows are outside what we tested. We also only trained models up to 8B with a 30B judge, so whether the gains hold at 70B+ is still an open question. We expect the same findings to carry over to richer tool protocols like MCP and Agent Skills, but testing that is an important next step.

Download paper here | Code on GitHub | Project page

@inproceedings{tam2026smith,
  title={Joint Optimization of Tool Creation and Use for Large Language Model Agents},
  author={Tam, Zhi Rui and Lin, Chieh-Yen and Chen, Yun-Nung and Sun, Shao-Hua and Lee, Hung-yi},
  booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
  year={2026},
  eprint={2608.24571},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2608.24571}
}

Recommended citation: Tam, Z.R., Lin, C.Y., Chen, Y.N., Sun, S.H., & Lee, H.Y. (2026). "Joint Optimization of Tool Creation and Use for Large Language Model Agents." NeurIPS.
Download Paper