Show HN: 我创建了一个由 AI 驱动的 Python 测试套件，它可以自己编写测试用例
1 分•作者: MarcoDewey•10 个月前
我一直在开发一个项目，很兴奋能与 Hacker News 社区分享。这是一个由 AI 驱动的 Python 测试套件，它使用混合 AI 方法自动生成全面的单元测试，执行模糊测试，甚至进行变异测试以评估现有测试套件的质量。
*问题*
作为一名开发者，我一直认为编写和维护一个强大的测试套件是软件开发中最耗时和最具挑战性的方面之一。 往往很难想到所有可能的边缘情况，并确保你的测试实际上能够有效地捕捉到错误。
*解决方案*
为了解决这个问题，我创建了一个 MCP 服务器，它利用了 Google 的 Gemini AI 和 BAML（边界机器学习）来提供一套智能测试工具。该服务器基于 FastMCP 框架构建，可以轻松集成到您现有的工作流程中。
*技术深入探讨*
以下是关键特性及其工作原理的分解：
* *混合 AI 方法：* 该项目使用混合 AI 方法，结合了 BAML 和 Gemini 的优势。BAML 用于结构化测试生成，确保输出始终以一致且可解析的格式呈现。Gemini 则利用其强大的语言理解能力，生成富有创意和挑战性的测试用例。
* *智能单元测试生成：* 单元测试生成器使用 AI 为您的 Python 代码创建一套全面的测试。它会自动识别边缘情况、错误条件和其他潜在的错误来源。生成的测试使用 `unittest` 框架编写，包括适当的断言和错误处理。
* *AI 驱动的模糊测试：* 模糊测试器使用 AI 生成各种各样的输入，以测试您的函数的鲁棒性。它可以生成从简单的边缘情况到格式错误的数据和大型输入的所有内容，帮助您识别潜在的崩溃和其他意外行为。
* *高级覆盖率测试：* 覆盖率测试器结合了 AST 分析和 AI 驱动的测试生成，以实现最大的代码覆盖率。它识别代码中所有可能的分支、循环和异常路径，然后生成测试来覆盖它们。
* *智能变异测试：* 变异测试器使用自定义的基于 AST 的变异引擎来评估现有测试套件的质量。它会生成一系列对代码的微小、语法上的更改（变异），然后检查您的测试是否能够检测到它们。这有助于您识别测试覆盖范围中的差距，并提高测试的整体有效性。
*行动号召*
我仍在积极开发该项目，并希望得到您的反馈。您可以在 GitHub 上找到源代码：<a href="https://github.com/jazzberry-ai/python-testing-mcp" rel="nofollow">https://github.com/jazzberry-ai/python-testing-mcp</a>
我特别感兴趣的是听取您对以下方面的看法：
* 您希望在套件中添加哪些其他测试工具？
* 您使用该工具发现了任何有趣的错误或边缘情况吗？
* 您对改进提示或 AI 模型有什么建议吗？
感谢您的阅读，期待您的反馈！
查看原文
I've been working on a project that I'm excited to share with the Hacker News community. It's an AI-powered Python testing suite that uses a hybrid AI approach to automatically generate comprehensive unit tests, perform fuzz testing, and even conduct mutation testing to assess the quality of your existing test suites.<p>*The Problem*<p>As a developer, I've always found writing and maintaining a robust test suite to be one of the most time-consuming and challenging aspects of software development. It's often difficult to think of all the possible edge cases and to ensure that your tests are actually effective at catching bugs.<p>*The Solution*<p>To address this, I've created an MCP server that leverages both Google's Gemini AI and BAML (Boundary ML) to provide a suite of intelligent testing tools. The server is built on the FastMCP framework and can be easily integrated into your existing workflow.<p>*Technical Deep Dive*<p>Here's a breakdown of the key features and how they work:<p>* *Hybrid AI Approach:* The project uses a hybrid AI approach that combines the strengths of both BAML and Gemini. BAML is used for structured test generation, ensuring that the output is always in a consistent and parseable format. Gemini is used for its powerful language understanding capabilities, which allows it to generate creative and challenging test cases.<p>* *Intelligent Unit Test Generation:* The unit test generator uses AI to create a comprehensive suite of tests for your Python code. It automatically identifies edge cases, error conditions, and other potential sources of bugs. The generated tests are written using the `unittest` framework and include proper assertions and error handling.<p>* *AI-Powered Fuzz Testing:* The fuzz tester uses AI to generate a diverse range of inputs to test the robustness of your functions. It can generate everything from simple edge cases to malformed data and large inputs, helping you to identify potential crashes and other unexpected behavior.<p>* *Advanced Coverage Testing:* The coverage tester uses a combination of AST analysis and AI-powered test generation to achieve maximum code coverage. It identifies all possible branches, loops, and exception paths in your code and then generates tests to cover each of them.<p>* *Intelligent Mutation Testing:* The mutation tester uses a custom AST-based mutation engine to assess the quality of your existing test suite. It generates a series of small, syntactic changes to your code (mutations) and then checks to see if your tests are able to detect them. This helps you to identify gaps in your test coverage and to improve the overall effectiveness of your tests.<p>*Call to Action*<p>I'm still actively developing the project, and I would love to get your feedback. You can find the source code on GitHub: <a href="https://github.com/jazzberry-ai/python-testing-mcp" rel="nofollow">https://github.com/jazzberry-ai/python-testing-mcp</a><p>I'm particularly interested in hearing your thoughts on the following:<p>* Are there any other testing tools that you would like to see added to the suite?
* Have you found any interesting bugs or edge cases using the tool?
* Do you have any suggestions for improving the prompts or the AI models?<p>Thanks for reading, and I look forward to hearing from you!