Ask HN: 人们在使用 Agent 编排工具时如何进行测试？
1 分•作者: spmartin823•3 个月前
我正在使用 Conductor，总的来说，它极大地提高了我的工作效率。唯一的问题是他们的“Spotlight”功能，该功能应该将工作树与我的根目录同步，从而使本地测试成为可能，但它并不可靠。即使它能正常工作，也无法完全满足我的需求，因为我希望每个工作流都能独立测试。
到目前为止，我尝试了三件事，但没有一个效果好：
1. 我使用了一个 Conductor 设置脚本，该脚本在每个工作树中运行我的本地开发设置。这不起作用，因为 Docker 容器之间发生了端口冲突。
2. 我正在使用 Terraform，因此为每个 PR 启动一个我的预发布环境的副本（资源较少）非常简单。这让 Conductor 中的每个 claude 会话都可以使用 Playright 来测试其代码。有两个问题：首先，这相当昂贵（每个 PR 每天 2-5 美元）。我每天推送 20-30 个 PR，所以即使有自动清理，这也要花费我每月 XXX 美元。其次，我的部署大约需要 10-15 分钟，这不算太长，但 claude 经常需要重新提示才能检查已部署的更改。
3. 对于新功能，我只是让 Claude 将代码直接部署到预发布或生产环境，并使用功能标志。这会导致回归，并且需要 Claude 访问特权数据进行测试，所以这并不是一个好的解决方案。
我正在考虑使用类似于与每个工作树相关联的本地 VM 的方案，但我想在深入研究之前，先确认一下我是否忽略了现有的解决方案。
查看原文
I'm using Conductor and overall it's been a game changer for my productivity. The one hiccup is that their "Spotlight" feature, which is supposed to sync the worktree with my root and thus make testing locally possible, doesn't work reliably. Even if it did, it wouldn't be exactly what I need because I want each workstream to be able to test independently.<p>Three things I've tried so far, none of which are working well:<p>1. I used a Conductor setup script that runs my local dev setup in each worktree. This didn't work because of port collisions between docker containers.<p>2. I'm using terraform, so it was trivial to spin up a copy of my staging infra (with fewer resources) for every PR. This let each claude session in Conductor use Playright to test it's code. Two problems: first, this is pretty expensive ($2-5/per day/per pr). I'm pushing 20-30 prs a day, so this was costing me $XXX/month even with automated cleanups. Second, my deploy takes about 10-15 minutes, which isn't that long, but claude would often need to be re-prompted to check on the deployed changes.<p>3. For new features, I just had Claude yolo code to staging or prod behind feature flags. This caused regressions and requires that Claude have access to privileged data for testing, so not a great solution.<p>I'm thinking that something like local VMs tied to each worktree could make sense, but wanted to check if I'm just oblivious to an existing solution before diving into that.