diff --git a/CHANGELOG.md b/CHANGELOG.md index 42d5bc9..2aa710c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ Status of the main branch. Changes prior to the next official version change wil ### This version is a major change and improvement of Serena * **Overhaul and major improvement of editing tools!** - This represents a very important change in Serena. Symbols can now be addressed by their name_path (including nested ones) + This represents a very important change in Serena. Symbols can now be addressed by their `name_path` (including nested ones) and we introduced a regex-based replaced tools. We tuned the prompts and tested the new editing mechanism. It is much more reliable, flexible, and at the same time uses fewer tokens. The line-replacement tools are disabled by default and deprecated, we will likely remove them soon. diff --git a/lessons_learned.md b/lessons_learned.md index d69de69..795c76d 100644 --- a/lessons_learned.md +++ b/lessons_learned.md @@ -1,16 +1,16 @@ # Lessons Learned -In this document we briefly collect what we have learned while developing and using Serena, +In this document we briefly collect what we have learned while developing and using Serena, what works well and what doesn't. ## What Worked ### Separate Tool Logic From MCP Implementation -MCP is just another protocol, one should let the details of it creep into the application logic. -The official docs suggest using function annotations to define tools and prompts. While that may be -useful for small projects to get going fast, it is not wise for more serious projects. In Serena, -all tools are defined independently and then converted to instances of `MCPTool` using our `make_tool` +MCP is just another protocol, one should let the details of it creep into the application logic. +The official docs suggest using function annotations to define tools and prompts. While that may be +useful for small projects to get going fast, it is not wise for more serious projects. In Serena, +all tools are defined independently and then converted to instances of `MCPTool` using our `make_tool` function. ### Autogenerated PromptFactory @@ -23,25 +23,25 @@ We separated out the generation logic into the [interprompt](/src/interprompt/RE ### Tempfiles and Snapshots for Testing of Editing Tools -We test most aspects of Serena by having a small "project" for each supported language in `tests/resources`. -For the editing tools, which would change the code in these projects, we use tempfiles to copy over the code. -The pretty awesome [syrupy](https://github.com/syrupy-project/syrupy) pytest plugin helped in developing +We test most aspects of Serena by having a small "project" for each supported language in `tests/resources`. +For the editing tools, which would change the code in these projects, we use tempfiles to copy over the code. +The pretty awesome [syrupy](https://github.com/syrupy-project/syrupy) pytest plugin helped in developing snapshot tests. ### Dashboard and GUI for Logging -It is very useful to know what the MCP Server is doing. We collect and display logs in a GUI or a web dashboard, +It is very useful to know what the MCP Server is doing. We collect and display logs in a GUI or a web dashboard, which helps a lot in seeing what's going on and in identifying any issues. ### Unrestricted Bash Tool -We know it's not particularly safe to permit unlimited shell commands outside a sandbox, but we did quite some -evaluations and so far... nothing bad has happened. Seems like the current versions of the AI overlords rarely want to execute `sudo rm - rf /`. +We know it's not particularly safe to permit unlimited shell commands outside a sandbox, but we did quite some +evaluations and so far... nothing bad has happened. Seems like the current versions of the AI overlords rarely want to execute `sudo rm - rf /`. Still, we are working on a safer approach as well as better integration with sandboxing. ### Multilspy -The [multilspy](https://github.com/microsoft/multilspy/) project helped us a lot in getting started and stands at the core of Serena. +The [multilspy](https://github.com/microsoft/multilspy/) project helped us a lot in getting started and stands at the core of Serena. Many more well known python implementations of language servers were subpar in code quality and design (for example, missing types). ### Developing Serena with Serena @@ -65,18 +65,18 @@ to the initial instructions and to the tool description that Claude finally star ### Lifespan Handling by MCP Clients -The MCP technology is clearly very green. Even though there is a lifespan context in the MCP SDK, -many clients, including Claude Desktop, fail to properly clean up, leaving zombie processes behind. -We mitigate this through the GUI window and the dashboard, so the user sees whether Serena is running +The MCP technology is clearly very green. Even though there is a lifespan context in the MCP SDK, +many clients, including Claude Desktop, fail to properly clean up, leaving zombie processes behind. +We mitigate this through the GUI window and the dashboard, so the user sees whether Serena is running and can terminate it there. ### Cross-OS Tkinter GUI -Different OS have different limitations when it comes to starting a window or dealing with Tkinter +Different OS have different limitations when it comes to starting a window or dealing with Tkinter installations. This was so messy to get right that we pivoted to a web-dashboard instead ### Editing Based on Line Numbers -Not only are LLMs notoriously bad in counting, but also the line numbers change after edit operations, -and LLMs are also often too dumb to understand that they should update the line numbers information they had +Not only are LLMs notoriously bad in counting, but also the line numbers change after edit operations, +and LLMs are also often too dumb to understand that they should update the line numbers information they had received before. We pivoted to string-matching and symbol-name based editing. \ No newline at end of file