应用层能否提升本地模型的输出质量？
1 分•作者: acro-v•7 个月前
您好，
我正在构建一个终端原生代码生成工具，最近的更新之一是为那些不想将代码上传到第三方服务器的用户打包一个本地模型（Qwen 2.5 Coder 7B，首次下载成功）。
用户对这个新增功能的初步反馈是积极的——但我对此表示怀疑：这个模型相当基础，质量无法与在线产品相提并论。
因此，我计划改进 RAG（检索增强生成）能力，构建包含相关源文件片段的消息，添加规划调用，添加验证循环，也许还会进行多样本重排等：所有这些都是常见的技术，如果正确实施，可以提高输出质量。
所以，问题是：我相信（希望？）通过实现所有这些功能，7B 模型可以提升到大约 20B 模型的质量，您是否同意这种可能性，或者您认为这会是徒劳的，并且不会发生这种程度的改进？
源代码在这里——如果您喜欢，请给它点个星：https://github.com/acrotron/aye-chat
查看原文
Hi -<p>I am building a terminal-native tool for code generation, and one of the recent updates was to package a local model (Qwen 2.5 Coder 7B, downloads on the first try) for those users who do not want their code uploaded to third-party servers.<p>Initial response from users to this addition was favorable - but I have my doubts: the model is fairly basic and does not compare in quality to online offerings.<p>So - I am planning to improve RAG capabilities for building a message with relevant source file chunks, add a planning call, add validation loop, maybe have a multi-sample with re-ranking, etc.: all those techniques that are common and when implemented properly - could improve quality of output.<p>So - the question: I believe (hope?) that with all those things implemented - 7B can be bumped approximately to quality of a 20B, do you agree that's possible or do you think it would be a wasted effort and that kind of improvement would not happen?<p>The source is here - give it a star if you like what you see: https://github.com/acrotron/aye-chat