diff --git a/CHANGELOG.md b/CHANGELOG.md index de3a01f..f61ce0d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,9 +8,13 @@ Changes prior to the next official version change will appear here. * one-click setup for Cline enabled * search for pattern tool can now (optionally) search in the entire project * new tool for restarting the language server, in case of other sources of editing apart from Serena + * Fix `CheckOnboardingPerformedTool`: + * Tool description was incompatible with project change + * Returned result was not as useful as it could be (now added list of memories) * Language Servers: * Add further file extensions considered by the language servers for Python (.pyi), JavaScript (.jsx) and TypeScript (.tsx, .jsx) + * Updated multilspy, adding support for Kotlin, Dart and C/C++ and several improvements. # 2025-04-07 diff --git a/README.md b/README.md index b8e906f..ae503d8 100644 --- a/README.md +++ b/README.md @@ -3,15 +3,9 @@

-* :rocket: Serena is a powerful, fully-featured **coding agent that works directly on your codebase**. -* :wrench: Serena **integrates with existing LLMs**, providing them with essential **semantic code retrieval and editing tools!** -* :free: Serena is **free to use**. No API keys or subscriptions required! - -Q: Can I have a state-of-the-art coding agent without paying (enormous) API costs -or constantly purchasing tokens? -A: Yes, you can! -By integrating Serena with your favourite (even free) LLM and thereby enabling it -to perform coding tasks directly on your codebase. +* :rocket: Serena is a powerful **coding agent toolkit** capable of turning an LLM into a fully-featured agent that works **directly on your codebase**. +* :wrench: Serena provides essential **semantic code retrieval and editing tools** that are akin to an IDE's capabilities, extracting code entities at the symbol level and exploiting relational structure. +* :free: Serena is **free & open-source**, enhancing the capabilities of LLMs you already have access to free of charge. ### Demonstration @@ -27,14 +21,17 @@ orchestrating tool use. Serena can be integrated with an LLM in several ways: * by using the **model context protocol (MCP)**. - Serena provides an MCP server which integrates with Claude (and [soon also ChatGPT](https://x.com/OpenAIDevs/status/1904957755829481737)). + Serena provides an MCP server which integrates with + * Claude Desktop, + * IDEs like VSCode, Cursor or IntelliJ, + * and [soon also ChatGPT](https://x.com/OpenAIDevs/status/1904957755829481737) * by using **Agno – the model-agnostic agent framework**. - Serena's Agno-based agent allows you to turn virtually any LLM into a coding agent, whether it's provided by Google, OpenAI or DeepSeek (with a paid API key) + Serena's Agno-based agent allows you to turn virtually any LLM into a coding agent, whether it's provided by Google, OpenAI or Anthropic (with a paid API key) or a free model provided by Ollama, Together or Anyscale. * by incorporating Serena's tools into an agent framework of your choice. Serena's tool implementation is decoupled from the framework-specific code and can thus easily be adapted to any agent framework. -### Programming Language Support & Semantic Analysis +### Programming Language Support & Semantic Analysis Capabilities Serena's semantic code analysis capabilities build on **language servers** using the widely implemented language server protocol (LSP). The LSP provides a set of versatile code querying @@ -56,6 +53,9 @@ With Serena, we provide * Go (untested) * C# (untested) * Rust (untested) + * Kotlin (untested) + * Dart (untested) + * C/C++ (untested) These languages are supported by the language server library [multilspy](https://github.com/microsoft/multilspy), which Serena uses under the hood. But we did not explicitly test whether the support for these languages actually works. @@ -63,15 +63,6 @@ With Serena, we provide Further languages can, in principle, easily be supported by providing a shallow adapter for a new language server implementation. -Coming soon: Kotlin and Dart. - - -> ⚠️ **Note:** Serena is under active development, we are continuously adding features, improving stability and UX. -> As a result, configuration might change in a breaking way. If you have an invalid configuration, -> the MCP server (or the Serena Agent) may not start properly (investigate the MCP logs in the former case). -> Check the [changelog](CHANGELOG.md) -> and the configuration templates when updating Serena and update your configs accordingly. - ## Table of Contents @@ -80,8 +71,8 @@ Coming soon: Kotlin and Dart. -- [Is It Really Free to Use?](#is-it-really-free-to-use) - [What Can I Use Serena For?](#what-can-i-use-serena-for) +- [Free Coding Agents with Serena](#free-coding-agents-with-serena) - [Quick Start](#quick-start) * [Setup and Configuration](#setup-and-configuration) * [MCP Server (Claude Desktop)](#mcp-server-claude-desktop) @@ -113,9 +104,24 @@ Coming soon: Kotlin and Dart. -## Is It Really Free to Use? +## What Can I Use Serena For? -Yes! Even the free tier of Anthropic's Claude has support for MCP Servers, so you can use Serena with Claude for free. +You can use Serena for any coding tasks – whether it is focussed on analysis, planning, +designing new components or refactoring existing ones. +Since Serena's tools allow an LLM to close the cognitive perception-action loop, +agents based on Serena can autonomously carry out coding tasks from start to finish – +from the initial analysis to the implementation, testing and, finally, the version +control system commit. + +Serena can read, write and execute code, read logs and the terminal output. +While we do not necessarily encourage it, "vibe coding" is certainly possible, and if you +want to almost feel like "the code no longer exists", +you may find Serena even more adequate for vibing than an agent inside an IDE +(since you will have a separate GUI that really lets you forget). + +## Free Coding Agents with Serena + +Even the free tier of Anthropic's Claude has support for MCP Servers, so you can use Serena with Claude for free. Presumably, the same will soon be possible with ChatGPT Desktop once support for MCP servers is added. Through Agno, you furthermore have the option to use Serena with a free/open-weights model. @@ -127,14 +133,6 @@ IDE-based subscriptions (such as Windsurf or Cursor) that forced us to keep purc The substantial API costs incurred by tools like Claude Code, Cline, Aider and other API-based tools are similarly unattractive. We thus built Serena with the prospect of being able to cancel most other subscriptions. -## What Can I Use Serena For? - -You can use Serena for any coding tasks – analyzing, planning, editing and so on. -Serena can read, write and execute code, read logs and the terminal output. -"Vibe coding" is possible, and if you want to almost feel like "the code no longer exists", - you may find Serena even more adequate for vibing than an agent inside an IDE -(since you will have a separate GUI that really lets you forget). - ## Quick Start ### Setup and Configuration @@ -147,7 +145,13 @@ Serena can read, write and execute code, read logs and the terminal output. 5. If you want Serena to dynamically switch between projects, add the list of all project files created in the previous step to the `projects` list in `serena_config.yml`. -After this initial setup, continue with one of the sections below, depending on how you +> ⚠️ **Note:** Serena is under active development. We are continuously adding features, improving stability and the UX. +> As a result, configuration may change in a breaking manner. If you have an invalid configuration, +> the MCP server or Serena-based Agent may fail to start (investigate the MCP logs in the former case). +> Check the [changelog](CHANGELOG.md) +> and the configuration templates when updating Serena, adapting your configurations accordingly. + +After the initial setup, continue with one of the sections below, depending on how you want to use Serena. ### MCP Server (Claude Desktop) @@ -195,9 +199,6 @@ necessarily has to be started by the client in order for communication to take p In other words, you do not need to start the server yourself. The client application (e.g. Claude Desktop) takes care of this and therefore needs to be configured with a launch command. -ℹ️ Furthermore note that Serena is always configured *for a single project*. To use it for another, you will have to -write a new configuration file, adjust the configuration to point to it and then restart the client. - For more information on MCP servers with Claude Desktop, see [the official quick start guide](https://modelcontextprotocol.io/quickstart/user). ### Other MCP Clients - Cline, Roo-Code, Cursor, Windsurf etc. diff --git a/pyproject.toml b/pyproject.toml index edced72..68c4058 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -31,6 +31,7 @@ dependencies = [ "jinja2>=3.1.6", "dotenv>=0.9.9", "pathspec>=0.12.1", + "psutil>=7.0.0", ] [project.scripts] diff --git a/src/multilspy/.syncCommitId.remote b/src/multilspy/.syncCommitId.remote index 80a2eae..44ef72b 100644 --- a/src/multilspy/.syncCommitId.remote +++ b/src/multilspy/.syncCommitId.remote @@ -1 +1 @@ -07ffa48166beb0c7f20ac76dbf7724d0bca4bbb4 \ No newline at end of file +43209c8521e93e8c41bdd2060800474fd0b9f8b1 \ No newline at end of file diff --git a/src/multilspy/.syncCommitId.this b/src/multilspy/.syncCommitId.this index 84a09c8..1626aec 100644 --- a/src/multilspy/.syncCommitId.this +++ b/src/multilspy/.syncCommitId.this @@ -1 +1 @@ -e766d55fd568f5313fb46796b8463097bdb9a293 \ No newline at end of file +90c42923c8299348634a3ebf7a23d9411d55ad65 \ No newline at end of file diff --git a/src/multilspy/language_server.py b/src/multilspy/language_server.py index 4d948f3..261edaf 100644 --- a/src/multilspy/language_server.py +++ b/src/multilspy/language_server.py @@ -37,6 +37,9 @@ from .lsp_protocol_handler.server import ( ) from .multilspy_config import Language, MultilspyConfig from .multilspy_exceptions import MultilspyException +from .multilspy_utils import PathUtils, FileUtils, TextUtils +from pathlib import PurePath +from typing import AsyncIterator, Iterator, List, Dict, Optional, Union, Tuple from .multilspy_logger import MultilspyLogger from .multilspy_utils import FileUtils, PathUtils, TextUtils from .type_helpers import ensure_all_methods_implemented @@ -72,11 +75,11 @@ class LSPFileBuffer: # reference count of the file ref_count: int - + # --------------------------------- MODIFICATIONS BY MISCHA --------------------------------- - + content_hash: str = "" - + def __post_init__(self): self.content_hash = hashlib.md5(self.contents.encode('utf-8')).hexdigest() @@ -148,6 +151,12 @@ class LanguageServer: ) return EclipseJDTLS(config, logger, repository_root_path) + elif config.code_language == Language.KOTLIN: + from multilspy.language_servers.kotlin_language_server.kotlin_language_server import ( + KotlinLanguageServer, + ) + + return KotlinLanguageServer(config, logger, repository_root_path) elif config.code_language == Language.RUST: from multilspy.language_servers.rust_analyzer.rust_analyzer import ( RustAnalyzer, @@ -171,6 +180,14 @@ class LanguageServer: from multilspy.language_servers.solargraph.solargraph import Solargraph return Solargraph(config, logger, repository_root_path) + elif config.code_language == Language.DART: + from multilspy.language_servers.dart_language_server.dart_language_server import DartLanguageServer + + return DartLanguageServer(config, logger, repository_root_path) + elif config.code_language == Language.CPP: + from multilspy.language_servers.clangd_language_server.clangd_language_server import ClangdLanguageServer + + return ClangdLanguageServer(config, logger, repository_root_path) else: logger.log(f"Language {config.code_language} is not supported", logging.ERROR) raise MultilspyException(f"Language {config.code_language} is not supported") @@ -205,7 +222,7 @@ class LanguageServer: self.server_started = False self.repository_root_path: str = repository_root_path self.completions_available = asyncio.Event() - + if config.trace_lsp_communication: def logging_fn(source, target, msg): @@ -218,18 +235,22 @@ class LanguageServer: # cmd is obtained from the child classes, which provide the language specific command to start the language server # LanguageServerHandler provides the functionality to start the language server and communicate with it - self.server: LanguageServerHandler = LanguageServerHandler(process_launch_info, logger=logging_fn) + self.server: LanguageServerHandler = LanguageServerHandler( + process_launch_info, + logger=logging_fn, + start_independent_lsp_process=config.start_independent_lsp_process, + ) self.language_id = language_id self.open_file_buffers: Dict[str, LSPFileBuffer] = {} - + # --------------------------------- MODIFICATIONS BY ORAIOS --------------------------------- self._document_symbols_cache: dict[str, Tuple[str, Tuple[List[multilspy_types.UnifiedSymbolInformation], List[multilspy_types.UnifiedSymbolInformation]]]] = {} """Maps file paths to a tuple of (file_content_hash, result_of_request_document_symbols)""" self.load_cache() self._cache_has_changed = bool self.language = Language(language_id) - + # Set up the pathspec matcher for the ignored paths # for all absolute paths in ignored_paths, convert them to relative paths processed_patterns = [] @@ -248,14 +269,14 @@ class LanguageServer: pathspec.patterns.GitWildMatchPattern, processed_patterns ) - + def get_ignore_spec(self) -> pathspec.PathSpec: """Returns the pathspec matcher for the paths that were configured to be ignored through the multilspy config file and the .gitignore file. - + This is is a subset of the full language-specific ignore spec that determines which files are relevant for the language server. - + This matcher is useful for operations outside of the language server, such as when searching for relevant non-language files in the project. """ @@ -265,9 +286,9 @@ class LanguageServer: """ Determine if a path should be ignored based on file type and ignore patterns. - + :param relative_path: Relative path to check - + :return: True if the path should be ignored, False otherwise """ # Check file extension if it's a file @@ -275,13 +296,13 @@ class LanguageServer: abs_path = os.path.join(self.repository_root_path, relative_path) if not os.path.exists(abs_path): raise FileNotFoundError(f"File {abs_path} not found, the ignore check cannot be performed") - + if os.path.isfile(abs_path) and not fn_matcher.is_relevant_filename(abs_path): return True - + # Create normalized path for consistent handling rel_path = Path(relative_path) - + # Check each part of the path against always fulfilled ignore conditions for part in rel_path.parts: if not part: # Skip empty parts (e.g., from leading '/') @@ -289,20 +310,20 @@ class LanguageServer: # Check standard ignores if self.should_always_ignore(part): return True - + # Use pathspec for gitignore-style pattern matching # Normalize path separators for pathspec (it expects forward slashes) normalized_path = str(rel_path).replace(os.path.sep, '/') - + # pathspec can't handle the matching of directories if they don't end with a slash! # see https://github.com/cpburnz/python-pathspec/issues/89 if os.path.isdir(os.path.join(self.repository_root_path, normalized_path)) and not normalized_path.endswith('/'): normalized_path = normalized_path + '/' - + # Use the pathspec matcher to check if the path matches any ignore pattern if self._ignore_spec.match_file(normalized_path): return True - + return False @@ -516,12 +537,7 @@ class LanguageServer: new_item: multilspy_types.Location = {} new_item.update(item) new_item["absolutePath"] = PathUtils.uri_to_path(new_item["uri"]) - try: - new_item["relativePath"] = str( - PurePath(os.path.relpath(new_item["absolutePath"], self.repository_root_path)) - ) - except: - new_item["relativePath"] = str(new_item["absolutePath"]) + new_item["relativePath"] = PathUtils.get_relative_path(new_item["absolutePath"], self.repository_root_path) ret.append(multilspy_types.Location(new_item)) elif ( LSPConstants.ORIGIN_SELECTION_RANGE in item @@ -532,12 +548,7 @@ class LanguageServer: new_item: multilspy_types.Location = {} new_item["uri"] = item[LSPConstants.TARGET_URI] new_item["absolutePath"] = PathUtils.uri_to_path(new_item["uri"]) - try: - new_item["relativePath"] = str( - PurePath(os.path.relpath(new_item["absolutePath"], self.repository_root_path)) - ) - except: - new_item["relativePath"] = str(new_item["absolutePath"]) + new_item["relativePath"] = PathUtils.get_relative_path(new_item["absolutePath"], self.repository_root_path) new_item["range"] = item[LSPConstants.TARGET_SELECTION_RANGE] ret.append(multilspy_types.Location(**new_item)) else: @@ -550,9 +561,7 @@ class LanguageServer: new_item: multilspy_types.Location = {} new_item.update(response) new_item["absolutePath"] = PathUtils.uri_to_path(new_item["uri"]) - new_item["relativePath"] = str( - PurePath(os.path.relpath(new_item["absolutePath"], self.repository_root_path)) - ) + new_item["relativePath"] = PathUtils.get_relative_path(new_item["absolutePath"], self.repository_root_path) ret.append(multilspy_types.Location(**new_item)) elif response is None: # Some language servers return None when they cannot find a definition @@ -602,12 +611,12 @@ class LanguageServer: ) ret: List[multilspy_types.Location] = [] - assert isinstance(response, list) + assert isinstance(response, list), f"Unexpected response from Language Server: {response}" for item in response: assert isinstance(item, dict) assert LSPConstants.URI in item assert LSPConstants.RANGE in item - + abs_path = PathUtils.uri_to_path(item[LSPConstants.URI]) rel_path = Path(abs_path).relative_to(self.repository_root_path) if self.should_ignore_path(str(rel_path)): @@ -621,7 +630,7 @@ class LanguageServer: ret.append(multilspy_types.Location(**new_item)) return ret - + async def request_references_with_content( self, relative_file_path: str, line: int, column: int, context_lines_before: int = 0, context_lines_after: int = 0 ) -> List[MatchedConsecutiveLines]: @@ -638,18 +647,18 @@ class LanguageServer: """ references = await self.request_references(relative_file_path, line, column) return [self.retrieve_content_around_line(ref["relativePath"], ref["range"]["start"]["line"], context_lines_before, context_lines_after) for ref in references] - + def retrieve_full_file_content(self, relative_file_path: str) -> str: """ Retrieve the full content of the given file. """ with self.open_file(relative_file_path) as file_data: return file_data.contents - + def retrieve_content_around_line(self, relative_file_path: str, line: int, context_lines_before: int = 0, context_lines_after: int = 0) -> MatchedConsecutiveLines: """ Retrieve the content of the given file around the given line. - + :param relative_file_path: The relative path of the file to retrieve the content from :param line: The line number to retrieve the content around :param context_lines_before: The number of lines to retrieve before the given line @@ -659,7 +668,7 @@ class LanguageServer: """ with self.open_file(relative_file_path) as file_data: file_contents = file_data.contents - + line_contents = file_contents.split("\n") start_lineno = max(0, line - context_lines_before) end_lineno = min(len(line_contents) - 1, line + context_lines_after) @@ -673,10 +682,10 @@ class LanguageServer: # after the line for lineno in range(line + 1, end_lineno + 1): text_lines.append(TextLine(line_number=lineno, line_content=line_contents[lineno], match_type=LineType.AFTER_MATCH)) - + return MatchedConsecutiveLines(lines=text_lines, source_file_path=relative_file_path) - - + + async def request_completions( self, relative_file_path: str, line: int, column: int, allow_incomplete: bool = False ) -> List[multilspy_types.CompletionItem]: @@ -795,8 +804,8 @@ class LanguageServer: return result else: self.logger.log(f"Content for {relative_file_path} has changed. Overwriting cache", logging.INFO) - - + + response = await self.server.send.document_symbol( { "textDocument": { @@ -804,7 +813,7 @@ class LanguageServer: } } ) - + def turn_item_into_symbol_with_children(item: GenericDocumentSymbol): item = cast(multilspy_types.UnifiedSymbolInformation, item) if "location" not in item: @@ -821,9 +830,9 @@ class LanguageServer: if include_body: item['body'] = self.retrieve_symbol_body(item) item[LSPConstants.CHILDREN] = item.get(LSPConstants.CHILDREN, []) - + flat_all_symbol_list: List[multilspy_types.UnifiedSymbolInformation] = [] - assert isinstance(response, list) + assert isinstance(response, list), f"Unexpected response from Language Server: {response}" root_nodes: List[multilspy_types.UnifiedSymbolInformation] = [] for item in response: if "range" not in item and "location" not in item: @@ -878,7 +887,7 @@ class LanguageServer: :return: A list of root symbols representing the top-level packages/modules in the project. """ - + if within_relative_path is not None: within_abs_path = os.path.join(self.repository_root_path, within_relative_path) if not os.path.exists(within_abs_path): @@ -890,7 +899,7 @@ class LanguageServer: else: _, root_nodes = await self.request_document_symbols(within_relative_path, include_body=include_body) return root_nodes - + # Helper function to recursively process directories async def process_directory(dir_path: str) -> List[multilspy_types.UnifiedSymbolInformation]: abs_dir_path = self.repository_root_path if dir_path == "." else os.path.join(self.repository_root_path, dir_path) @@ -935,6 +944,7 @@ class LanguageServer: elif os.path.isfile(abs_item_path): _, root_nodes = await self.request_document_symbols(item_path, include_body=include_body) + # TODO: Not sure if this is actually still needed given recent changes to relative path handling def fix_relative_path(nodes: List[multilspy_types.UnifiedSymbolInformation]): for node in nodes: path = Path(node["location"]["relativePath"]) @@ -965,7 +975,7 @@ class LanguageServer: ), children=root_nodes ) - + package_symbol["children"].append(file_symbol) return result @@ -989,15 +999,15 @@ class LanguageServer: async def request_dir_overview(self, relative_dir_path: str) -> dict[str, list[tuple[str, multilspy_types.SymbolKind, int, int]]]: """ - An overview of the given directory. - - Maps relative paths of all contained files to info about top-level symbols in the file + An overview of the given directory. + + Maps relative paths of all contained files to info about top-level symbols in the file (name, kind, line, column). """ symbol_tree = await self.request_full_symbol_tree(relative_dir_path) # Initialize result dictionary result: dict[str, list[tuple[str, multilspy_types.SymbolKind, int, int]]] = defaultdict(list) - + # Helper function to process a symbol and its children def process_symbol(symbol: multilspy_types.UnifiedSymbolInformation): if symbol["kind"] == multilspy_types.SymbolKind.File: @@ -1015,15 +1025,15 @@ class LanguageServer: # For package/directory symbols, process their children for child in symbol["children"]: process_symbol(child) - + # Process each root symbol for root in symbol_tree: process_symbol(root) return result - + async def request_document_overview(self, relative_file_path: str) -> list[tuple[str, multilspy_types.SymbolKind, int, int]]: """ - An overview of the given file. + An overview of the given file. Returns the list of tuples (name, kind, line, column) of all top-level symbols in the file. """ _, document_roots = await self.request_document_symbols(relative_file_path) @@ -1036,24 +1046,24 @@ class LanguageServer: ) for root in document_roots ] - + async def request_overview(self, within_relative_path: str) -> dict[str, list[tuple[str, multilspy_types.SymbolKind, int, int]]]: """ An overview of all symbols in the given file or directory. - + :param within_relative_path: the relative path to the file or directory to get the overview of. :return: A mapping of all relative paths analyzed to lists of tuples (name, kind, line, column) of all top-level symbols in the corresponding file. """ abs_path = (Path(self.repository_root_path) / within_relative_path).resolve() if not abs_path.exists(): raise FileNotFoundError(f"File or directory not found: {abs_path}") - + if abs_path.is_file(): symbols_overview = await self.request_document_overview(within_relative_path) return {within_relative_path: symbols_overview} else: return await self.request_dir_overview(within_relative_path) - + async def request_hover(self, relative_file_path: str, line: int, column: int) -> Union[multilspy_types.Hover, None]: """ Raise a [textDocument/hover](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_hover) request to the Language Server @@ -1084,9 +1094,9 @@ class LanguageServer: assert isinstance(response, dict) return multilspy_types.Hover(**response) - + # ----------------------------- FROM HERE ON MODIFICATIONS BY MISCHA -------------------- - + def retrieve_symbol_body(self, symbol: multilspy_types.UnifiedSymbolInformation | LSPTypes.DocumentSymbol | LSPTypes.SymbolInformation) -> str: """ Load the body of the given symbol. If the body is already contained in the symbol, just return it. @@ -1094,7 +1104,7 @@ class LanguageServer: existing_body = symbol.get("body", None) if existing_body: return existing_body - + assert "location" in symbol symbol_start_line = symbol["location"]["range"]["start"]["line"] symbol_end_line = symbol["location"]["range"]["end"]["line"] @@ -1102,19 +1112,19 @@ class LanguageServer: symbol_file = self.retrieve_full_file_content(symbol["location"]["relativePath"]) symbol_lines = symbol_file.split("\n") symbol_body = "\n".join(symbol_lines[symbol_start_line:symbol_end_line]) - + # remove leading indentation symbol_start_column = symbol["location"]["range"]["start"]["character"] symbol_body = symbol_body[symbol_start_column:] return symbol_body - - + + async def request_parsed_files(self) -> list[str]: """ Retrieves relative paths of all files analyzed by the Language Server. - - This is slow, as it finds all files by finding all symbols. - + + This is slow, as it finds all files by finding all symbols. + This seems to be the only way, the LSP does not provide any endpoints for listing project files.""" if not self.server_started: self.logger.log( @@ -1126,27 +1136,27 @@ class LanguageServer: # I don't know why # params = LSPTypes.WorkspaceSymbolParams(query="") # Empty query returns all symbols # symbols = await self.server.send.workspace_symbol(params) or [] - + # Thus, instead of calling all symbols, we hack this and use the symbol tree instead, which # seems to work in all these language servers # walk through all children recursively, find all symbols of type Module and collect their relative paths roots = await self.request_full_symbol_tree() - paths = [] + paths = [] def collect_module_files(symbol): if symbol["kind"] == multilspy_types.SymbolKind.File: assert "location" in symbol paths.append(symbol["location"]["relativePath"]) - + elif symbol["kind"] == multilspy_types.SymbolKind.Package: for child in symbol["children"]: collect_module_files(child) - + for root in roots: collect_module_files(root) - + return paths - - + + async def search_files_for_pattern( self, pattern: re.Pattern | str, @@ -1157,7 +1167,7 @@ class LanguageServer: ) -> list[MatchedConsecutiveLines]: """ Search for a pattern across all files analyzed by the Language Server. - + :param pattern: Regular expression pattern to search for, either as a compiled Pattern or string :param context_lines_before: Number of lines of context to include before each match :param context_lines_after: Number of lines of context to include after each match @@ -1167,7 +1177,7 @@ class LanguageServer: """ if isinstance(pattern, str): pattern = re.compile(pattern) - + relative_file_paths = await self.request_parsed_files() return search_files( relative_file_paths, @@ -1178,7 +1188,7 @@ class LanguageServer: paths_include_glob=paths_include_glob, paths_exclude_glob=paths_exclude_glob ) - + async def request_referencing_symbols( self, relative_file_path: str, @@ -1200,7 +1210,7 @@ class LanguageServer: :param include_imports: whether to also include imports as references. Unfortunately, the LSP does not have an import type, so the references corresponding to imports will not be easily distinguishable from definitions. - :param include_self: whether to include the references that is the "input symbol" itself. + :param include_self: whether to include the references that is the "input symbol" itself. Only has an effect if the relative_file_path, line and column point to a symbol, for example a definition. :param include_body: whether to include the body of the symbols in the result. :param include_file_symbols: whether to include references that are file symbols. This @@ -1236,7 +1246,7 @@ class LanguageServer: # TODO: HORRIBLE HACK! I don't know how to do it better for now... # THIS IS BOUND TO BREAK IN MANY CASES! IT IS ALSO SPECIFIC TO PYTHON! # Background: - # When a variable is used to change something, like + # When a variable is used to change something, like # # instance = MyClass() # instance.status = "new status" @@ -1247,7 +1257,7 @@ class LanguageServer: # by using the text of the reference to find the variable name (In a very heuristic way) # and then look for a symbol with that name and kind Variable ref_text = file_data.contents.split("\n")[ref_line] - if "." in ref_text: + if "." in ref_text: containing_symbol_name = ref_text.split(".")[0] all_symbols, _ = await self.request_document_symbols(ref_path) for symbol in all_symbols: @@ -1288,7 +1298,7 @@ class LanguageServer: ) if containing_symbol is None or not include_file_symbols and containing_symbol["kind"] == multilspy_types.SymbolKind.File: continue - + assert "location" in containing_symbol assert "selectionRange" in containing_symbol @@ -1305,7 +1315,7 @@ class LanguageServer: else: self.logger.log(f"Found self-reference for {incoming_symbol['name']}, skipping it since {include_self=}", logging.DEBUG) continue - + # checking whether reference is an import # This is neither really safe nor elegant, but if we don't do it, # there is no way to distinguish between definitions and imports as import is not a symbol-type @@ -1321,11 +1331,11 @@ class LanguageServer: logging.DEBUG ) continue - + result.append(containing_symbol) return result - + async def request_containing_symbol( self, relative_file_path: str, @@ -1374,7 +1384,7 @@ class LanguageServer: return None symbols, _ = await self.request_document_symbols(relative_file_path) - + # make jedi and pyright api compatible # the former has no location, the later has no range # we will just always add location of the desired format to all symbols @@ -1397,8 +1407,8 @@ class LanguageServer: # Allowed container kinds, currently only for Python container_symbol_kinds = { - multilspy_types.SymbolKind.Method, - multilspy_types.SymbolKind.Function, + multilspy_types.SymbolKind.Method, + multilspy_types.SymbolKind.Function, multilspy_types.SymbolKind.Class } @@ -1425,7 +1435,7 @@ class LanguageServer: s for s in symbols if s["kind"] == multilspy_types.SymbolKind.Variable ] candidate_containers.extend(var_containers) - + if not candidate_containers: return None @@ -1445,11 +1455,11 @@ class LanguageServer: return containing_symbol else: return None - + async def request_container_of_symbol(self, symbol: multilspy_types.UnifiedSymbolInformation, include_body: bool = False) -> multilspy_types.UnifiedSymbolInformation | None: """ Finds the container of the given symbol if there is one. - + :param symbol: The symbol to find the container of. :param include_body: whether to include the body of the symbol in the result. """ @@ -1461,7 +1471,7 @@ class LanguageServer: strict=True, include_body=include_body, ) - + async def request_defining_symbol( self, relative_file_path: str, @@ -1471,10 +1481,10 @@ class LanguageServer: ) -> Optional[multilspy_types.UnifiedSymbolInformation]: """ Finds the symbol that defines the symbol at the given location. - + This method first finds the definition of the symbol at the given position, then retrieves the full symbol information for that definition. - + :param relative_file_path: The relative path to the file. :param line: The 0-indexed line number. :param column: The 0-indexed column number. @@ -1487,29 +1497,29 @@ class LanguageServer: logging.ERROR, ) raise MultilspyException("Language Server not started") - + # Get the definition location(s) definitions = await self.request_definition(relative_file_path, line, column) if not definitions: return None - + # Use the first definition location definition = definitions[0] def_path = definition["relativePath"] def_line = definition["range"]["start"]["line"] def_col = definition["range"]["start"]["character"] - + # Find the symbol at or containing this location defining_symbol = await self.request_containing_symbol( def_path, def_line, def_col, strict=False, include_body=include_body ) - + return defining_symbol - + @property def _cache_path(self) -> Path: return Path(self.repository_root_path) / ".serena" / "cache" / "document_symbols_cache.pkl" - + def save_cache(self): if self._cache_has_changed: self.logger.log(f"Saving updated document symbols cache to {self._cache_path}", logging.INFO) @@ -1522,7 +1532,7 @@ class LanguageServer: f"Failed to save document symbols cache to {self._cache_path}: {e}. " "Note: this may have resulted in a corrupted cache file.", logging.ERROR ) - + def load_cache(self): if not self._cache_path.exists(): return @@ -1534,11 +1544,38 @@ class LanguageServer: # cache often becomes corrupt, so just skip loading it self.logger.log( f"Failed to load document symbols cache from {self._cache_path}: {e}. Possible cause: the cache file is corrupted. " - "Check for any errors related to saving the cache in the logs.", + "Check for any errors related to saving the cache in the logs.", logging.ERROR ) + async def request_workspace_symbol(self, query: str) -> Union[List[multilspy_types.UnifiedSymbolInformation], None]: + """ + Raise a [workspace/symbol](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_symbol) request to the Language Server + to find symbols across the whole workspace. Wait for the response and return the result. + + :param query: The query string to filter symbols by + + :return Union[List[multilspy_types.UnifiedSymbolInformation], None]: A list of matching symbols + """ + response = await self.server.send.workspace_symbol({"query": query}) + if response is None: + return None + + assert isinstance(response, list) + + ret: List[multilspy_types.UnifiedSymbolInformation] = [] + for item in response: + assert isinstance(item, dict) + + assert LSPConstants.NAME in item + assert LSPConstants.KIND in item + assert LSPConstants.LOCATION in item + + ret.append(multilspy_types.UnifiedSymbolInformation(**item)) + + return ret + @ensure_all_methods_implemented(LanguageServer) class SyncLanguageServer: """ @@ -1546,31 +1583,38 @@ class SyncLanguageServer: It is used to communicate with Language Servers of different programming languages. """ - def __init__(self, language_server: LanguageServer) -> None: + def __init__(self, language_server: LanguageServer, timeout: Optional[int] = None): + """ + :param language_server: the async language server being wrapped + :param timeout: the timeout, in seconds, to use for requests to the language server. + """ self.language_server = language_server self.loop = None self.loop_thread = None - + self.timeout = timeout + self._server_context = None @classmethod def create( cls, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str, add_gitignore_content_to_config=True, + timeout: Optional[int] = None ) -> "SyncLanguageServer": """ Creates a language specific LanguageServer instance based on the given configuration, and appropriate settings for the programming language. If language is Java, then ensure that jdk-17.0.6 or higher is installed, `java` is in PATH, and JAVA_HOME is set to the installation directory. - :param repository_root_path: The root path of the repository. + :param repository_root_path: The root path of the repository (must be absolute). :param config: The Multilspy configuration. :param logger: The logger to use. :param add_gitignore_content_to_config: whether to add the content of the .gitignore file (if any found) to the config, so that the paths ignored there are also ignored by the language server + :param timeout: the timeout, in seconds, to use for requests; if None, use no timeout :return SyncLanguageServer: A language specific LanguageServer instance. """ - return SyncLanguageServer(LanguageServer.create(config, logger, repository_root_path, add_gitignore_content_to_config=add_gitignore_content_to_config)) + return SyncLanguageServer(LanguageServer.create(config, logger, repository_root_path, add_gitignore_content_to_config=add_gitignore_content_to_config), timeout=timeout) @contextmanager def open_file(self, relative_file_path: str) -> Iterator[LSPFileBuffer]: @@ -1637,7 +1681,7 @@ class SyncLanguageServer: """ result = asyncio.run_coroutine_threadsafe( self.language_server.request_definition(file_path, line, column), self.loop - ).result() + ).result(timeout=self.timeout) return result def request_references(self, file_path: str, line: int, column: int) -> List[multilspy_types.Location]: @@ -1653,15 +1697,15 @@ class SyncLanguageServer: """ result = asyncio.run_coroutine_threadsafe( self.language_server.request_references(file_path, line, column), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_references_with_content( self, relative_file_path: str, line: int, column: int, context_lines_before: int = 0, context_lines_after: int = 0 ) -> List[MatchedConsecutiveLines]: """ Like request_references, but returns the content of the lines containing the references, not just the locations. - + :param relative_file_path: The relative path of the file that has the symbol for which references should be looked up :param line: The line number of the symbol :param column: The column number of the symbol @@ -1674,7 +1718,7 @@ class SyncLanguageServer: self.language_server.request_references_with_content(relative_file_path, line, column, context_lines_before, context_lines_after), self.loop ).result() return result - + def request_completions( self, relative_file_path: str, line: int, column: int, allow_incomplete: bool = False ) -> List[multilspy_types.CompletionItem]: @@ -1691,7 +1735,7 @@ class SyncLanguageServer: result = asyncio.run_coroutine_threadsafe( self.language_server.request_completions(relative_file_path, line, column, allow_incomplete), self.loop, - ).result() + ).result(timeout=self.timeout) return result def request_document_symbols(self, relative_file_path: str, include_body: bool = False) -> Tuple[List[multilspy_types.UnifiedSymbolInformation], List[multilspy_types.UnifiedSymbolInformation]]: @@ -1707,17 +1751,17 @@ class SyncLanguageServer: self.language_server.request_document_symbols(relative_file_path, include_body), self.loop ).result() return result - + def request_full_symbol_tree(self, within_relative_path: str | None = None, include_body: bool = False) -> List[multilspy_types.UnifiedSymbolInformation]: """ Will go through all files in the project and build a tree of symbols. Note: this may be slow the first time it is called. - + For each file, a symbol of kind Module (3) will be created. For directories, a symbol of kind Package (4) will be created. All symbols will have a children attribute, thereby representing the tree structure of all symbols in the project that are within the repository. Will ignore directories starting with '.', language-specific defaults and user-configured directories (e.g. from .gitignore). - + :param within_relative_path: pass a relative path to only consider symbols within this path. If a file is passed, only the symbols within this file will be considered. If a directory is passed, all files within this directory will be considered. @@ -1728,38 +1772,38 @@ class SyncLanguageServer: """ result = asyncio.run_coroutine_threadsafe( self.language_server.request_full_symbol_tree(within_relative_path, include_body), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_dir_overview(self, relative_dir_path: str) -> dict[str, list[tuple[str, multilspy_types.SymbolKind, int, int]]]: """ - An overview of the given directory. - - Maps relative paths of all contained files to info about top-level symbols in the file + An overview of the given directory. + + Maps relative paths of all contained files to info about top-level symbols in the file (name, kind, line, column). """ assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_dir_overview(relative_dir_path), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_document_overview(self, relative_file_path: str) -> list[tuple[str, multilspy_types.SymbolKind, int, int]]: """ - An overview of the given file. - + An overview of the given file. + Returns the list of tuples (name, kind, line, column) of all top-level symbols in the file. """ assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_document_overview(relative_file_path), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_overview(self, within_relative_path: str) -> dict[str, list[tuple[str, multilspy_types.SymbolKind, int, int]]]: """ An overview of all symbols in the given file or directory. - + :param within_relative_path: the relative path to the file or directory to get the overview of. :return: A mapping of all relative paths analyzed to lists of tuples (name, kind, line, column) of all top-level symbols in the corresponding file. """ @@ -1768,7 +1812,7 @@ class SyncLanguageServer: self.language_server.request_overview(within_relative_path), self.loop ).result() return result - + def request_hover(self, relative_file_path: str, line: int, column: int) -> Union[multilspy_types.Hover, None]: """ Raise a [textDocument/hover](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_hover) request to the Language Server @@ -1782,30 +1826,44 @@ class SyncLanguageServer: """ result = asyncio.run_coroutine_threadsafe( self.language_server.request_hover(relative_file_path, line, column), self.loop - ).result() + ).result(timeout=self.timeout) return result - + + def request_workspace_symbol(self, query: str) -> Union[List[multilspy_types.UnifiedSymbolInformation], None]: + """ + Raise a [workspace/symbol](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_symbol) request to the Language Server + to find symbols across the whole workspace. Wait for the response and return the result. + + :param query: The query string to filter symbols by + + :return Union[List[multilspy_types.UnifiedSymbolInformation], None]: A list of matching symbols + """ + result = asyncio.run_coroutine_threadsafe( + self.language_server.request_workspace_symbol(query), self.loop + ).result(timeout=self.timeout) + return result + # ----------------------------- FROM HERE ON MODIFICATIONS BY MISCHA -------------------- - + def retrieve_symbol_body(self, symbol: multilspy_types.UnifiedSymbolInformation) -> str: """ Load the body of the given symbol. If the body is already contained in the symbol, just return it. - + :param symbol: The symbol to retrieve the body of. :return: The body of the symbol. """ return self.language_server.retrieve_symbol_body(symbol) - + def request_parsed_files(self) -> list[str]: - """This is slow, as it finds all files by finding all symbols. - + """This is slow, as it finds all files by finding all symbols. + This seems to be the only way, the LSP does not provide any endpoints for listing project files.""" assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_parsed_files(), self.loop ).result() return result - + def request_referencing_symbols( self, relative_file_path: str, line: int, column: int, include_imports: bool = True, include_self: bool = False, @@ -1823,7 +1881,7 @@ class SyncLanguageServer: :param include_imports: whether to also include imports as references. Unfortunately, the LSP does not have an import type, so the references corresponding to imports will not be easily distinguishable from definitions. - :param include_self: whether to include the references that is the "input symbol" itself. + :param include_self: whether to include the references that is the "input symbol" itself. Only has an effect if the relative_file_path, line and column point to a symbol, for example a definition. :param include_body: whether to include the body of the symbols in the result. :param include_file_symbols: whether to include references that are file symbols. This @@ -1833,20 +1891,20 @@ class SyncLanguageServer: assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_referencing_symbols( - relative_file_path, - line, - column, - include_imports=include_imports, + relative_file_path, + line, + column, + include_imports=include_imports, include_self=include_self, include_body=include_body, include_file_symbols=include_file_symbols, - ), + ), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_containing_symbol( - self, relative_file_path: str, line: int, + self, relative_file_path: str, line: int, column: Optional[int] = None, strict: bool = False, include_body: bool = False, ) -> multilspy_types.UnifiedSymbolInformation | None: @@ -1854,24 +1912,24 @@ class SyncLanguageServer: Finds the first symbol containing the position for the given file. For Python, container symbols are considered to be those with kinds corresponding to functions, methods, or classes (typically: Function (12), Method (6), Class (5)). - + The method operates as follows: - Request the document symbols for the file. - Filter symbols to those that start at or before the given line. - From these, first look for symbols whose range contains the (line, column). - If one or more symbols contain the position, return the one with the greatest starting position (i.e. the innermost container). - - If none (strictly) contain the position, return the symbol with the greatest starting position + - If none (strictly) contain the position, return the symbol with the greatest starting position among those above the given line. - If no container candidates are found, return None. - + :param relative_file_path: The relative path to the Python file. :param line: The 0-indexed line number. :param column: The 0-indexed column (also called character). If not passed, the lookup will be based only on the line. :param strict: If True, the position must be strictly within the range of the symbol. Setting to true is useful for example for finding the parent of a symbol, as with strict=False, - and the line pointing to a symbol itself, the containing symbol will be the symbol itself + and the line pointing to a symbol itself, the containing symbol will be the symbol itself (and not the parent). :param include_body: whether to include the body of the symbol in the result. :return: The container symbol (if found) or None. @@ -1879,32 +1937,32 @@ class SyncLanguageServer: assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_containing_symbol(relative_file_path, line, column=column, strict=strict, include_body=include_body), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_container_of_symbol(self, symbol: multilspy_types.UnifiedSymbolInformation, include_body: bool = False) -> multilspy_types.UnifiedSymbolInformation | None: """ Finds the container of the given symbol if there is one. - + :param symbol: The symbol to find the container of. :param include_body: whether to include the body of the symbol in the result. """ assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_container_of_symbol(symbol, include_body=include_body), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_defining_symbol( - self, relative_file_path: str, line: int, column: int, + self, relative_file_path: str, line: int, column: int, include_body: bool = False, ) -> Optional[multilspy_types.UnifiedSymbolInformation]: """ Finds the symbol that defines the symbol at the given location. - + This method first finds the definition of the symbol at the given position, then retrieves the full symbol information for that definition. - + :param relative_file_path: The relative path to the file. :param line: The 0-indexed line number. :param column: The 0-indexed column number. @@ -1914,19 +1972,19 @@ class SyncLanguageServer: assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_defining_symbol(relative_file_path, line, column, include_body=include_body), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def retrieve_full_file_content(self, relative_file_path: str) -> str: """ Retrieve the full content of the given file. """ return self.language_server.retrieve_full_file_content(relative_file_path) - + def retrieve_content_around_line(self, relative_file_path: str, line: int, context_lines_before: int = 0, context_lines_after: int = 0) -> MatchedConsecutiveLines: """ Retrieve the content of the given file around the given line. - + :param relative_file_path: The relative path of the file to retrieve the content from :param line: The line number to retrieve the content around :param context_lines_before: The number of lines to retrieve before the given line @@ -1934,7 +1992,7 @@ class SyncLanguageServer: :return MatchedConsecutiveLines: A container with the desired lines. """ return self.language_server.retrieve_content_around_line(relative_file_path, line, context_lines_before, context_lines_after) - + def search_files_for_pattern( self, pattern: re.Pattern | str, @@ -1945,7 +2003,7 @@ class SyncLanguageServer: ) -> list[MatchedConsecutiveLines]: """ Search for a pattern across all files analyzed by the Language Server. - + :param pattern: Regular expression pattern to search for, either as a compiled Pattern or string :param context_lines_before: Number of lines of context to include before each match :param context_lines_after: Number of lines of context to include after each match @@ -1956,9 +2014,9 @@ class SyncLanguageServer: assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.search_files_for_pattern(pattern, context_lines_before, context_lines_after, paths_include_glob, paths_exclude_glob), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def start(self) -> "SyncLanguageServer": """ Starts the language server process and connects to it. Call shutdown when ready. @@ -1971,23 +2029,23 @@ class SyncLanguageServer: self._server_context = self.language_server.start_server() asyncio.run_coroutine_threadsafe(self._server_context.__aenter__(), loop=self.loop).result() return self - + def is_running(self) -> bool: """ Check if the language server is running. """ return self.loop is not None and self.loop_thread is not None and self.loop_thread.is_alive() - + def stop(self) -> None: """ Shuts down the language server process and cleans up resources. - + If the language server is not running, this method will log a warning and do nothing. """ if not self.is_running(): self.language_server.logger.log("Language server not running, skipping shutdown.", logging.INFO) return - + assert self.loop asyncio.run_coroutine_threadsafe(self._server_context.__aexit__(None, None, None), loop=self.loop).result() self.loop.call_soon_threadsafe(self.loop.stop) @@ -1995,13 +2053,13 @@ class SyncLanguageServer: self.loop = None self.loop_thread = None self.save_cache() - + def save_cache(self): """ Save the cache to a file. """ self.language_server.save_cache() - + def load_cache(self): """ Load the cache from a file. @@ -2014,20 +2072,20 @@ class SyncLanguageServer: in Python and node_modules in JS/TS should be ignored always. """ return self.language_server.should_always_ignore(dirname) - + def should_ignore_path(self, relative_path: str) -> bool: """ Whether the given path should be ignored. """ return self.language_server.should_ignore_path(relative_path) - + def get_ignore_spec(self) -> pathspec.PathSpec: """Returns the pathspec matcher for the paths that were configured to be ignored through the multilspy config file and the .gitignore file. - + This is is a subset of the full language-specific ignore spec that determines which files are relevant for the language server. - + This matcher is useful for operations outside of the language server, such as when searching for relevant non-language files in the project. """ diff --git a/src/multilspy/language_servers/clangd_language_server/clangd_language_server.py b/src/multilspy/language_servers/clangd_language_server/clangd_language_server.py new file mode 100644 index 0000000..17253c4 --- /dev/null +++ b/src/multilspy/language_servers/clangd_language_server/clangd_language_server.py @@ -0,0 +1,180 @@ +""" +Provides C/C++ specific instantiation of the LanguageServer class. Contains various configurations and settings specific to C/C++. +""" + +import asyncio +import json +import logging +import os +import stat +import pathlib +from contextlib import asynccontextmanager +from typing import AsyncIterator + +from multilspy.multilspy_logger import MultilspyLogger +from multilspy.language_server import LanguageServer +from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo +from multilspy.lsp_protocol_handler.lsp_types import InitializeParams +from multilspy.multilspy_config import MultilspyConfig +from multilspy.multilspy_utils import FileUtils +from multilspy.multilspy_utils import PlatformUtils + + +class ClangdLanguageServer(LanguageServer): + """ + Provides C/C++ specific instantiation of the LanguageServer class. Contains various configurations and settings specific to C/C++. + As the project gets bigger in size, building index will take time. Try running clangd multiple times to ensure index is built properly. + Also make sure compile_commands.json is created at root of the source directory. Check clangd test case for example. + """ + + def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str): + """ + Creates a ClangdLanguageServer instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead. + """ + clangd_executable_path = self.setup_runtime_dependencies(logger, config) + super().__init__( + config, + logger, + repository_root_path, + ProcessLaunchInfo(cmd=clangd_executable_path, cwd=repository_root_path), + "cpp", + ) + self.server_ready = asyncio.Event() + + def setup_runtime_dependencies(self, logger: MultilspyLogger, config: MultilspyConfig) -> str: + """ + Setup runtime dependencies for ClangdLanguageServer. + """ + platform_id = PlatformUtils.get_platform_id() + + with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f: + d = json.load(f) + del d["_description"] + + assert platform_id.value in [ + "linux-x64" + ], "Only linux-x64 is supported for in multilspy at the moment" + + runtime_dependencies = d["runtimeDependencies"] + runtime_dependencies = [ + dependency for dependency in runtime_dependencies if dependency["platformId"] == platform_id.value + ] + assert len(runtime_dependencies) == 1 + dependency = runtime_dependencies[0] + + clangd_ls_dir = os.path.join(os.path.dirname(__file__), "static/clangd") + clangd_executable_path = os.path.join(clangd_ls_dir, "clangd_19.1.2", "bin", dependency["binaryName"]) + if not os.path.exists(clangd_ls_dir): + os.makedirs(clangd_ls_dir) + if dependency["archiveType"] == "zip": + FileUtils.download_and_extract_archive( + logger, dependency["url"], clangd_ls_dir, dependency["archiveType"] + ) + assert os.path.exists(clangd_executable_path) + os.chmod(clangd_executable_path, stat.S_IEXEC) + + return clangd_executable_path + + def _get_initialize_params(self, repository_absolute_path: str) -> InitializeParams: + """ + Returns the initialize params for the clangd Language Server. + """ + with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r") as f: + d = json.load(f) + + del d["_description"] + + d["processId"] = os.getpid() + assert d["rootPath"] == "$rootPath" + d["rootPath"] = repository_absolute_path + + assert d["rootUri"] == "$rootUri" + d["rootUri"] = pathlib.Path(repository_absolute_path).as_uri() + + assert d["workspaceFolders"][0]["uri"] == "$uri" + d["workspaceFolders"][0]["uri"] = pathlib.Path(repository_absolute_path).as_uri() + + assert d["workspaceFolders"][0]["name"] == "$name" + d["workspaceFolders"][0]["name"] = os.path.basename(repository_absolute_path) + + return d + + @asynccontextmanager + async def start_server(self) -> AsyncIterator["ClangdLanguageServer"]: + """ + Starts the Clangd Language Server, waits for the server to be ready and yields the LanguageServer instance. + + Usage: + ``` + async with lsp.start_server(): + # LanguageServer has been initialized and ready to serve requests + await lsp.request_definition(...) + await lsp.request_references(...) + # Shutdown the LanguageServer on exit from scope + # LanguageServer has been shutdown + """ + async def register_capability_handler(params): + assert "registrations" in params + for registration in params["registrations"]: + if registration["method"] == "workspace/executeCommand": + self.initialize_searcher_command_available.set() + self.resolve_main_method_available.set() + return + + async def lang_status_handler(params): + # TODO: Should we wait for + # server -> client: {'jsonrpc': '2.0', 'method': 'language/status', 'params': {'type': 'ProjectStatus', 'message': 'OK'}} + # Before proceeding? + if params["type"] == "ServiceReady" and params["message"] == "ServiceReady": + self.service_ready_event.set() + + async def execute_client_command_handler(params): + return [] + + async def do_nothing(params): + return + + async def check_experimental_status(params): + if params["quiescent"] == True: + self.server_ready.set() + + async def window_log_message(msg): + self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO) + + self.server.on_request("client/registerCapability", register_capability_handler) + self.server.on_notification("language/status", lang_status_handler) + self.server.on_notification("window/logMessage", window_log_message) + self.server.on_request("workspace/executeClientCommand", execute_client_command_handler) + self.server.on_notification("$/progress", do_nothing) + self.server.on_notification("textDocument/publishDiagnostics", do_nothing) + self.server.on_notification("language/actionableNotification", do_nothing) + self.server.on_notification("experimental/serverStatus", check_experimental_status) + + async with super().start_server(): + self.logger.log("Starting Clangd server process", logging.INFO) + await self.server.start() + initialize_params = self._get_initialize_params(self.repository_root_path) + + self.logger.log( + "Sending initialize request from LSP client to LSP server and awaiting response", + logging.INFO, + ) + init_response = await self.server.send.initialize(initialize_params) + assert init_response["capabilities"]["textDocumentSync"]["change"] == 2 + assert "completionProvider" in init_response["capabilities"] + assert init_response["capabilities"]["completionProvider"] == { + "triggerCharacters": ['.', '<', '>', ':', '"', '/', '*'], + "resolveProvider": False, + } + + self.server.notify.initialized({}) + + self.completions_available.set() + # set ready flag + self.server_ready.set() + await self.server_ready.wait() + + yield self + + await self.server.shutdown() + await self.server.stop() diff --git a/src/multilspy/language_servers/clangd_language_server/initialize_params.json b/src/multilspy/language_servers/clangd_language_server/initialize_params.json new file mode 100644 index 0000000..4330560 --- /dev/null +++ b/src/multilspy/language_servers/clangd_language_server/initialize_params.json @@ -0,0 +1,36 @@ +{ + "_description": "The parameters sent by the client when initializing the language server with the \"initialize\" request. More details at https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#initialize", + "processId": "os.getpid()", + "locale": "en", + "rootPath": "$rootPath", + "rootUri": "$rootUri", + "capabilities": { + "textDocument": { + "synchronization": { + "didSave": true, + "dynamicRegistration": true + }, + "completion": { + "dynamicRegistration": true, + "completionItem": { + "snippetSupport": true + } + }, + "definition": { + "dynamicRegistration": true + } + }, + "workspace": { + "workspaceFolders": true, + "didChangeConfiguration": { + "dynamicRegistration": true + } + } + }, + "workspaceFolders": [ + { + "uri": "$uri", + "name": "$name" + } + ] +} \ No newline at end of file diff --git a/src/multilspy/language_servers/clangd_language_server/runtime_dependencies.json b/src/multilspy/language_servers/clangd_language_server/runtime_dependencies.json new file mode 100644 index 0000000..ab30fff --- /dev/null +++ b/src/multilspy/language_servers/clangd_language_server/runtime_dependencies.json @@ -0,0 +1,13 @@ +{ + "_description": "Used to download the runtime dependencies for running Clangd.", + "runtimeDependencies": [ + { + "id": "Clangd", + "description": "Clangd for Linux (x64)", + "url": "https://github.com/clangd/clangd/releases/download/19.1.2/clangd-linux-19.1.2.zip", + "platformId": "linux-x64", + "archiveType": "zip", + "binaryName": "clangd" + } + ] +} diff --git a/src/multilspy/language_servers/dart_language_server/dart_language_server.py b/src/multilspy/language_servers/dart_language_server/dart_language_server.py new file mode 100644 index 0000000..b814306 --- /dev/null +++ b/src/multilspy/language_servers/dart_language_server/dart_language_server.py @@ -0,0 +1,146 @@ +from contextlib import asynccontextmanager +import logging +import os +import pathlib +import shutil +import stat +from typing import AsyncIterator +from multilspy.language_server import LanguageServer +from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo +import json +from multilspy.multilspy_utils import FileUtils, PlatformUtils + + +class DartLanguageServer(LanguageServer): + """ + Provides Dart specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Dart. + """ + + def __init__(self, config, logger, repository_root_path): + """ + Creates a DartServer instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead. + """ + + executable_path = self.setup_runtime_dependencies(logger) + super().__init__( + config, + logger, + repository_root_path, + ProcessLaunchInfo(cmd=executable_path, cwd=repository_root_path), + "dart", + ) + + def setup_runtime_dependencies(self, logger: "MultilspyLogger") -> str: + platform_id = PlatformUtils.get_platform_id() + + with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f: + d = json.load(f) + del d["_description"] + + runtime_dependencies = d["runtimeDependencies"] + runtime_dependencies = [ + dependency for dependency in runtime_dependencies if dependency["platformId"] == platform_id.value + ] + + assert len(runtime_dependencies) == 1 + dependency = runtime_dependencies[0] + + dart_ls_dir = os.path.join(os.path.dirname(__file__), "static", "dart-language-server") + dart_executable_path = os.path.join(dart_ls_dir, dependency["binaryName"]) + + if not os.path.exists(dart_ls_dir): + os.makedirs(dart_ls_dir) + FileUtils.download_and_extract_archive( + logger, dependency["url"], dart_ls_dir, dependency["archiveType"] + ) + + + assert os.path.exists(dart_executable_path) + os.chmod(dart_executable_path, stat.S_IEXEC) + + return f"{dart_executable_path} language-server --client-id multilspy.dart --client-version 1.2" + + + def _get_initialize_params(self, repository_absolute_path: str): + """ + Returns the initialize params for the Dart Language Server. + """ + with open( + os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r" + ) as f: + d = json.load(f) + + del d["_description"] + + d["processId"] = os.getpid() + assert d["rootPath"] == "$rootPath" + d["rootPath"] = repository_absolute_path + + assert d["rootUri"] == "$rootUri" + d["rootUri"] = pathlib.Path(repository_absolute_path).as_uri() + + assert d["workspaceFolders"][0]["uri"] == "$uri" + d["workspaceFolders"][0]["uri"] = pathlib.Path( + repository_absolute_path + ).as_uri() + + assert d["workspaceFolders"][0]["name"] == "$name" + d["workspaceFolders"][0]["name"] = os.path.basename(repository_absolute_path) + + return d + + @asynccontextmanager + async def start_server(self) -> AsyncIterator["DartLanguageServer"]: + """ + Start the language server and yield when the server is ready. + """ + + async def execute_client_command_handler(params): + return [] + + async def do_nothing(params): + return + + async def check_experimental_status(params): + pass + + async def window_log_message(msg): + self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO) + + self.server.on_request("client/registerCapability", do_nothing) + self.server.on_notification("language/status", do_nothing) + self.server.on_notification("window/logMessage", window_log_message) + self.server.on_request( + "workspace/executeClientCommand", execute_client_command_handler + ) + self.server.on_notification("$/progress", do_nothing) + self.server.on_notification("textDocument/publishDiagnostics", do_nothing) + self.server.on_notification("language/actionableNotification", do_nothing) + self.server.on_notification( + "experimental/serverStatus", check_experimental_status + ) + + async with super().start_server(): + self.logger.log( + "Starting dart-language-server server process", logging.INFO + ) + await self.server.start() + initialize_params = self._get_initialize_params(self.repository_root_path) + self.logger.log( + "Sending initialize request to dart-language-server", + logging.DEBUG, + ) + init_response = await self.server.send_request( + "initialize", initialize_params + ) + self.logger.log( + f"Received initialize response from dart-language-server: {init_response}", + logging.INFO, + ) + + self.server.notify.initialized({}) + + yield self + + await self.server.shutdown() + await self.server.stop() diff --git a/src/multilspy/language_servers/dart_language_server/initialize_params.json b/src/multilspy/language_servers/dart_language_server/initialize_params.json new file mode 100644 index 0000000..e90e408 --- /dev/null +++ b/src/multilspy/language_servers/dart_language_server/initialize_params.json @@ -0,0 +1,23 @@ +{ + "_description": "This file contains the initialization parameters for the Dart Language Server.", + "processId": "$processId", + "rootPath": "$rootPath", + "rootUri": "$rootUri", + "capabilities": {}, + "initializationOptions": { + "onlyAnalyzeProjectsWithOpenFiles": false, + "suggestFromUnimportedLibraries": true, + "closingLabels": false, + "outline": false, + "flutterOutline": false, + "allowOpenUri": false + }, + "trace": "verbose", + "workspaceFolders": [ + { + "uri": "$uri", + "name": "$name" + } + ] + +} \ No newline at end of file diff --git a/src/multilspy/language_servers/dart_language_server/runtime_dependencies.json b/src/multilspy/language_servers/dart_language_server/runtime_dependencies.json new file mode 100644 index 0000000..8010b94 --- /dev/null +++ b/src/multilspy/language_servers/dart_language_server/runtime_dependencies.json @@ -0,0 +1,13 @@ +{ + "_description": "Used to download the runtime dependencies for running Dart Language Server, downloaded from https://dart.dev/get-dart/archive", + "runtimeDependencies": [ + { + "id": "DartLanguageServer", + "description": "Dart Language Server for Linux (x64)", + "url": "https://storage.googleapis.com/dart-archive/channels/stable/release/3.7.1/sdk/dartsdk-linux-x64-release.zip", + "platformId": "linux-x64", + "archiveType": "zip", + "binaryName": "dart-sdk/bin/dart" + } + ] +} \ No newline at end of file diff --git a/src/multilspy/language_servers/eclipse_jdtls/runtime_dependencies.json b/src/multilspy/language_servers/eclipse_jdtls/runtime_dependencies.json index bc2cb74..c224c69 100644 --- a/src/multilspy/language_servers/eclipse_jdtls/runtime_dependencies.json +++ b/src/multilspy/language_servers/eclipse_jdtls/runtime_dependencies.json @@ -23,6 +23,16 @@ "jdtls_launcher_jar_path": "extension/server/plugins/org.eclipse.equinox.launcher_1.6.500.v20230717-2134.jar", "jdtls_readonly_config_path": "extension/server/config_mac_arm" }, + "osx-x64": { + "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@darwin-x64-1.23.0.vsix", + "archiveType": "zip", + "relative_extraction_path": "vscode-java", + "jre_home_path": "extension/jre/17.0.8.1-macosx-x86_64", + "jre_path": "extension/jre/17.0.8.1-macosx-x86_64/bin/java", + "lombok_jar_path": "extension/lombok/lombok-1.18.30.jar", + "jdtls_launcher_jar_path": "extension/server/plugins/org.eclipse.equinox.launcher_1.6.500.v20230717-2134.jar", + "jdtls_readonly_config_path": "extension/server/config_mac" + }, "linux-arm64": { "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@linux-arm64-1.23.0.vsix", "archiveType": "zip", @@ -59,4 +69,4 @@ "intellisense_members_path": "extension/dist/bundledModels/java_intellisense-members" } } -} \ No newline at end of file +} diff --git a/src/multilspy/language_servers/kotlin_language_server/initialize_params.json b/src/multilspy/language_servers/kotlin_language_server/initialize_params.json new file mode 100644 index 0000000..a7b2ec0 --- /dev/null +++ b/src/multilspy/language_servers/kotlin_language_server/initialize_params.json @@ -0,0 +1,521 @@ +{ + "_description": "The parameters sent by the client when initializing the language server with the \"initialize\" request. More details at https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#initialize", + "processId": "os.getpid()", + "clientInfo": { + "name": "Multilspy Kotlin Client", + "version": "1.0.0" + }, + "locale": "en", + "rootPath": "repository_absolute_path", + "rootUri": "pathlib.Path(repository_absolute_path).as_uri()", + "capabilities": { + "workspace": { + "applyEdit": true, + "workspaceEdit": { + "documentChanges": true, + "resourceOperations": [ + "create", + "rename", + "delete" + ], + "failureHandling": "textOnlyTransactional", + "normalizesLineEndings": true, + "changeAnnotationSupport": { + "groupsOnLabel": true + } + }, + "didChangeConfiguration": { + "dynamicRegistration": true + }, + "didChangeWatchedFiles": { + "dynamicRegistration": true, + "relativePatternSupport": true + }, + "symbol": { + "dynamicRegistration": true, + "symbolKind": { + "valueSet": [ + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8, + 9, + 10, + 11, + 12, + 13, + 14, + 15, + 16, + 17, + 18, + 19, + 20, + 21, + 22, + 23, + 24, + 25, + 26 + ] + }, + "tagSupport": { + "valueSet": [ + 1 + ] + }, + "resolveSupport": { + "properties": [ + "location.range" + ] + } + }, + "codeLens": { + "refreshSupport": true + }, + "executeCommand": { + "dynamicRegistration": true + }, + "configuration": true, + "workspaceFolders": true, + "semanticTokens": { + "refreshSupport": true + }, + "fileOperations": { + "dynamicRegistration": true, + "didCreate": true, + "didRename": true, + "didDelete": true, + "willCreate": true, + "willRename": true, + "willDelete": true + }, + "inlineValue": { + "refreshSupport": true + }, + "inlayHint": { + "refreshSupport": true + }, + "diagnostics": { + "refreshSupport": true + } + }, + "textDocument": { + "publishDiagnostics": { + "relatedInformation": true, + "versionSupport": false, + "tagSupport": { + "valueSet": [ + 1, + 2 + ] + }, + "codeDescriptionSupport": true, + "dataSupport": true + }, + "synchronization": { + "dynamicRegistration": true, + "willSave": true, + "willSaveWaitUntil": true, + "didSave": true + }, + "completion": { + "dynamicRegistration": true, + "contextSupport": true, + "completionItem": { + "snippetSupport": false, + "commitCharactersSupport": true, + "documentationFormat": [ + "markdown", + "plaintext" + ], + "deprecatedSupport": true, + "preselectSupport": true, + "tagSupport": { + "valueSet": [ + 1 + ] + }, + "insertReplaceSupport": false, + "resolveSupport": { + "properties": [ + "documentation", + "detail", + "additionalTextEdits" + ] + }, + "insertTextModeSupport": { + "valueSet": [ + 1, + 2 + ] + }, + "labelDetailsSupport": true + }, + "insertTextMode": 2, + "completionItemKind": { + "valueSet": [ + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8, + 9, + 10, + 11, + 12, + 13, + 14, + 15, + 16, + 17, + 18, + 19, + 20, + 21, + 22, + 23, + 24, + 25 + ] + }, + "completionList": { + "itemDefaults": [ + "commitCharacters", + "editRange", + "insertTextFormat", + "insertTextMode" + ] + } + }, + "hover": { + "dynamicRegistration": true, + "contentFormat": [ + "markdown", + "plaintext" + ] + }, + "signatureHelp": { + "dynamicRegistration": true, + "signatureInformation": { + "documentationFormat": [ + "markdown", + "plaintext" + ], + "parameterInformation": { + "labelOffsetSupport": true + }, + "activeParameterSupport": true + }, + "contextSupport": true + }, + "definition": { + "dynamicRegistration": true, + "linkSupport": true + }, + "references": { + "dynamicRegistration": true + }, + "documentHighlight": { + "dynamicRegistration": true + }, + "documentSymbol": { + "dynamicRegistration": true, + "symbolKind": { + "valueSet": [ + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8, + 9, + 10, + 11, + 12, + 13, + 14, + 15, + 16, + 17, + 18, + 19, + 20, + 21, + 22, + 23, + 24, + 25, + 26 + ] + }, + "hierarchicalDocumentSymbolSupport": true, + "tagSupport": { + "valueSet": [ + 1 + ] + }, + "labelSupport": true + }, + "codeAction": { + "dynamicRegistration": true, + "isPreferredSupport": true, + "disabledSupport": true, + "dataSupport": true, + "resolveSupport": { + "properties": [ + "edit" + ] + }, + "codeActionLiteralSupport": { + "codeActionKind": { + "valueSet": [ + "", + "quickfix", + "refactor", + "refactor.extract", + "refactor.inline", + "refactor.rewrite", + "source", + "source.organizeImports" + ] + } + }, + "honorsChangeAnnotations": false + }, + "codeLens": { + "dynamicRegistration": true + }, + "formatting": { + "dynamicRegistration": true + }, + "rangeFormatting": { + "dynamicRegistration": true + }, + "onTypeFormatting": { + "dynamicRegistration": true + }, + "rename": { + "dynamicRegistration": true, + "prepareSupport": true, + "prepareSupportDefaultBehavior": 1, + "honorsChangeAnnotations": true + }, + "documentLink": { + "dynamicRegistration": true, + "tooltipSupport": true + }, + "typeDefinition": { + "dynamicRegistration": true, + "linkSupport": true + }, + "implementation": { + "dynamicRegistration": true, + "linkSupport": true + }, + "colorProvider": { + "dynamicRegistration": true + }, + "foldingRange": { + "dynamicRegistration": true, + "rangeLimit": 5000, + "lineFoldingOnly": true, + "foldingRangeKind": { + "valueSet": [ + "comment", + "imports", + "region" + ] + }, + "foldingRange": { + "collapsedText": false + } + }, + "declaration": { + "dynamicRegistration": true, + "linkSupport": true + }, + "selectionRange": { + "dynamicRegistration": true + }, + "callHierarchy": { + "dynamicRegistration": true + }, + "semanticTokens": { + "dynamicRegistration": true, + "tokenTypes": [ + "namespace", + "type", + "class", + "enum", + "interface", + "struct", + "typeParameter", + "parameter", + "variable", + "property", + "enumMember", + "event", + "function", + "method", + "macro", + "keyword", + "modifier", + "comment", + "string", + "number", + "regexp", + "operator", + "decorator" + ], + "tokenModifiers": [ + "declaration", + "definition", + "readonly", + "static", + "deprecated", + "abstract", + "async", + "modification", + "documentation", + "defaultLibrary" + ], + "formats": [ + "relative" + ], + "requests": { + "range": true, + "full": { + "delta": true + } + }, + "multilineTokenSupport": false, + "overlappingTokenSupport": false, + "serverCancelSupport": true, + "augmentsSyntaxTokens": true + }, + "linkedEditingRange": { + "dynamicRegistration": true + }, + "typeHierarchy": { + "dynamicRegistration": true + }, + "inlineValue": { + "dynamicRegistration": true + }, + "inlayHint": { + "dynamicRegistration": true, + "resolveSupport": { + "properties": [ + "tooltip", + "textEdits", + "label.tooltip", + "label.location", + "label.command" + ] + } + }, + "diagnostic": { + "dynamicRegistration": true, + "relatedDocumentSupport": false + } + }, + "window": { + "showMessage": { + "messageActionItem": { + "additionalPropertiesSupport": true + } + }, + "showDocument": { + "support": true + }, + "workDoneProgress": true + }, + "general": { + "staleRequestSupport": { + "cancel": true, + "retryOnContentModified": [ + "textDocument/semanticTokens/full", + "textDocument/semanticTokens/range", + "textDocument/semanticTokens/full/delta" + ] + }, + "regularExpressions": { + "engine": "ECMAScript", + "version": "ES2020" + }, + "markdown": { + "parser": "marked", + "version": "1.1.0" + }, + "positionEncodings": [ + "utf-16" + ] + }, + "notebookDocument": { + "synchronization": { + "dynamicRegistration": true, + "executionSummarySupport": true + } + } + }, + "initializationOptions": { + "workspaceFolders": "[pathlib.Path(repository_absolute_path).as_uri()]", + "storagePath": null, + "codegen": { + "enabled": false + }, + "compiler": { + "jvm": { + "target": "default" + } + }, + "completion": { + "snippets": { + "enabled": true + } + }, + "diagnostics": { + "enabled": true, + "level": 4, + "debounceTime": 250 + }, + "scripts": { + "enabled": true, + "buildScriptsEnabled": true + }, + "indexing": { + "enabled": true + }, + "externalSources": { + "useKlsScheme": false, + "autoConvertToKotlin": false + }, + "inlayHints": { + "typeHints": false, + "parameterHints": false, + "chainedHints": false + }, + "formatting": { + "formatter": "ktfmt", + "ktfmt": { + "style": "google", + "indent": 4, + "maxWidth": 100, + "continuationIndent": 8, + "removeUnusedImports": true + } + } + }, + "trace": "verbose", + "workspaceFolders": "[\n {\n \"uri\": pathlib.Path(repository_absolute_path).as_uri(),\n \"name\": os.path.basename(repository_absolute_path),\n }\n ]" +} diff --git a/src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py b/src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py new file mode 100644 index 0000000..76bc1e8 --- /dev/null +++ b/src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py @@ -0,0 +1,234 @@ +""" +Provides Kotlin specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Kotlin. +""" + +import asyncio +import dataclasses +import json +import logging +import os +import stat +import pathlib +from contextlib import asynccontextmanager +from typing import AsyncIterator + +from multilspy.multilspy_logger import MultilspyLogger +from multilspy.language_server import LanguageServer +from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo +from multilspy.lsp_protocol_handler.lsp_types import InitializeParams +from multilspy.multilspy_config import MultilspyConfig +from multilspy.multilspy_utils import FileUtils +from multilspy.multilspy_utils import PlatformUtils + + +@dataclasses.dataclass +class KotlinRuntimeDependencyPaths: + """ + Stores the paths to the runtime dependencies of Kotlin Language Server + """ + java_path: str + java_home_path: str + kotlin_executable_path: str + + +class KotlinLanguageServer(LanguageServer): + """ + Provides Kotlin specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Kotlin. + """ + + def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str): + """ + Creates a Kotlin Language Server instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead. + """ + runtime_dependency_paths = self.setup_runtime_dependencies(logger, config) + self.runtime_dependency_paths = runtime_dependency_paths + + # Create command to execute the Kotlin Language Server script + cmd = f'"{self.runtime_dependency_paths.kotlin_executable_path}"' + + # Set environment variables including JAVA_HOME + proc_env = {"JAVA_HOME": self.runtime_dependency_paths.java_home_path} + + super().__init__( + config, + logger, + repository_root_path, + ProcessLaunchInfo(cmd=cmd, env=proc_env, cwd=repository_root_path), + "kotlin", + ) + + def setup_runtime_dependencies(self, logger: MultilspyLogger, config: MultilspyConfig) -> KotlinRuntimeDependencyPaths: + """ + Setup runtime dependencies for Kotlin Language Server. + """ + platform_id = PlatformUtils.get_platform_id() + + # Verify platform support + assert platform_id.value.startswith("win-") or platform_id.value.startswith("linux-") or platform_id.value.startswith("osx-"), "Only Windows, Linux and macOS platforms are supported for Kotlin in multilspy at the moment" + + # Load dependency information + with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f: + d = json.load(f) + del d["_description"] + + kotlin_dependency = d["runtimeDependency"] + java_dependency = d["java"][platform_id.value] + + # Setup paths for dependencies + static_dir = os.path.join(os.path.dirname(__file__), "static") + os.makedirs(static_dir, exist_ok=True) + + # Setup Java paths + java_dir = os.path.join(static_dir, "java") + os.makedirs(java_dir, exist_ok=True) + + java_home_path = os.path.join(java_dir, java_dependency["java_home_path"]) + java_path = os.path.join(java_dir, java_dependency["java_path"]) + + # Download and extract Java if not exists + if not os.path.exists(java_path): + logger.log(f"Downloading Java for {platform_id.value}...", logging.INFO) + FileUtils.download_and_extract_archive( + logger, java_dependency["url"], java_dir, java_dependency["archiveType"] + ) + # Make Java executable + if not platform_id.value.startswith("win-"): + os.chmod(java_path, 0o755) + + assert os.path.exists(java_path), f"Java executable not found at {java_path}" + + # Setup Kotlin Language Server paths + kotlin_ls_dir = os.path.join(static_dir, "server") + + # Get platform-specific executable script path + if platform_id.value.startswith("win-"): + kotlin_script = os.path.join(kotlin_ls_dir, "bin", "kotlin-language-server.bat") + else: + kotlin_script = os.path.join(kotlin_ls_dir, "bin", "kotlin-language-server") + + # Download and extract Kotlin Language Server if script doesn't exist + if not os.path.exists(kotlin_script): + logger.log("Downloading Kotlin Language Server...", logging.INFO) + FileUtils.download_and_extract_archive( + logger, kotlin_dependency["url"], static_dir, kotlin_dependency["archiveType"] + ) + + # Make script executable on Unix platforms + if os.path.exists(kotlin_script) and not platform_id.value.startswith("win-"): + os.chmod(kotlin_script, stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR | stat.S_IRGRP | stat.S_IXGRP | stat.S_IROTH | stat.S_IXOTH) + + # Use script file + if os.path.exists(kotlin_script): + kotlin_executable_path = kotlin_script + logger.log(f"Using Kotlin Language Server script at {kotlin_script}", logging.INFO) + else: + raise FileNotFoundError(f"Kotlin Language Server script not found at {kotlin_script}") + + return KotlinRuntimeDependencyPaths( + java_path=java_path, + java_home_path=java_home_path, + kotlin_executable_path=kotlin_executable_path + ) + + def _get_initialize_params(self, repository_absolute_path: str) -> InitializeParams: + """ + Returns the initialize params for the Kotlin Language Server. + """ + with open(str(pathlib.PurePath(os.path.dirname(__file__), "initialize_params.json")), "r") as f: + d: InitializeParams = json.load(f) + + del d["_description"] + + if not os.path.isabs(repository_absolute_path): + repository_absolute_path = os.path.abspath(repository_absolute_path) + + assert d["processId"] == "os.getpid()" + d["processId"] = os.getpid() + + assert d["rootPath"] == "repository_absolute_path" + d["rootPath"] = repository_absolute_path + + assert d["rootUri"] == "pathlib.Path(repository_absolute_path).as_uri()" + d["rootUri"] = pathlib.Path(repository_absolute_path).as_uri() + + assert d["initializationOptions"]["workspaceFolders"] == "[pathlib.Path(repository_absolute_path).as_uri()]" + d["initializationOptions"]["workspaceFolders"] = [pathlib.Path(repository_absolute_path).as_uri()] + + assert ( + d["workspaceFolders"] + == '[\n {\n "uri": pathlib.Path(repository_absolute_path).as_uri(),\n "name": os.path.basename(repository_absolute_path),\n }\n ]' + ) + d["workspaceFolders"] = [ + { + "uri": pathlib.Path(repository_absolute_path).as_uri(), + "name": os.path.basename(repository_absolute_path), + } + ] + + return d + + @asynccontextmanager + async def start_server(self) -> AsyncIterator["KotlinLanguageServer"]: + """ + Starts the Kotlin Language Server, waits for the server to be ready and yields the LanguageServer instance. + + Usage: + ``` + async with lsp.start_server(): + # LanguageServer has been initialized and ready to serve requests + await lsp.request_definition(...) + await lsp.request_references(...) + # Shutdown the LanguageServer on exit from scope + # LanguageServer has been shutdown + ``` + """ + async def execute_client_command_handler(params): + return [] + + async def do_nothing(params): + return + + async def window_log_message(msg): + self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO) + + self.server.on_request("client/registerCapability", do_nothing) + self.server.on_notification("language/status", do_nothing) + self.server.on_notification("window/logMessage", window_log_message) + self.server.on_request("workspace/executeClientCommand", execute_client_command_handler) + self.server.on_notification("$/progress", do_nothing) + self.server.on_notification("textDocument/publishDiagnostics", do_nothing) + self.server.on_notification("language/actionableNotification", do_nothing) + + async with super().start_server(): + self.logger.log("Starting Kotlin server process", logging.INFO) + await self.server.start() + initialize_params = self._get_initialize_params(self.repository_root_path) + + self.logger.log( + "Sending initialize request from LSP client to LSP server and awaiting response", + logging.INFO, + ) + init_response = await self.server.send.initialize(initialize_params) + + capabilities = init_response["capabilities"] + assert "textDocumentSync" in capabilities, "Server must support textDocumentSync" + assert "hoverProvider" in capabilities, "Server must support hover" + assert "completionProvider" in capabilities, "Server must support code completion" + assert "signatureHelpProvider" in capabilities, "Server must support signature help" + assert "definitionProvider" in capabilities, "Server must support go to definition" + assert "referencesProvider" in capabilities, "Server must support find references" + assert "documentSymbolProvider" in capabilities, "Server must support document symbols" + assert "workspaceSymbolProvider" in capabilities, "Server must support workspace symbols" + assert "semanticTokensProvider" in capabilities, "Server must support semantic tokens" + + self.server.notify.initialized({}) + self.completions_available.set() + + yield self + + try: + await self.server.shutdown() + except Exception as e: + self.logger.log(f"Error during Kotlin server shutdown: {str(e)}", logging.WARNING) + finally: + await self.server.stop() diff --git a/src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json b/src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json new file mode 100644 index 0000000..a7b64b9 --- /dev/null +++ b/src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json @@ -0,0 +1,41 @@ +{ + "_description": "Used to download the runtime dependencies for Kotlin Language Server from https://github.com/fwcd/kotlin-language-server", + "runtimeDependency": { + "id": "KotlinLsp", + "description": "Kotlin Language Server", + "url": "https://github.com/fwcd/kotlin-language-server/releases/download/1.3.13/server.zip", + "archiveType": "zip" + }, + "java": { + "win-x64": { + "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@win32-x64-1.23.0.vsix", + "archiveType": "zip", + "java_home_path": "extension/jre/17.0.8.1-win32-x86_64", + "java_path": "extension/jre/17.0.8.1-win32-x86_64/bin/java.exe" + }, + "linux-x64": { + "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@linux-x64-1.23.0.vsix", + "archiveType": "zip", + "java_home_path": "extension/jre/17.0.8.1-linux-x86_64", + "java_path": "extension/jre/17.0.8.1-linux-x86_64/bin/java" + }, + "linux-arm64": { + "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@linux-arm64-1.23.0.vsix", + "archiveType": "zip", + "java_home_path": "extension/jre/17.0.8.1-linux-aarch64", + "java_path": "extension/jre/17.0.8.1-linux-aarch64/bin/java" + }, + "osx-x64": { + "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@darwin-x64-1.23.0.vsix", + "archiveType": "zip", + "java_home_path": "extension/jre/17.0.8.1-macosx-x86_64", + "java_path": "extension/jre/17.0.8.1-macosx-x86_64/bin/java" + }, + "osx-arm64": { + "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@darwin-arm64-1.23.0.vsix", + "archiveType": "zip", + "java_home_path": "extension/jre/17.0.8.1-macosx-aarch64", + "java_path": "extension/jre/17.0.8.1-macosx-aarch64/bin/java" + } + } +} diff --git a/src/multilspy/language_servers/typescript_language_server/typescript_language_server.py b/src/multilspy/language_servers/typescript_language_server/typescript_language_server.py index 85b1a18..2bd0666 100644 --- a/src/multilspy/language_servers/typescript_language_server/typescript_language_server.py +++ b/src/multilspy/language_servers/typescript_language_server/typescript_language_server.py @@ -32,6 +32,11 @@ else: return type('obj', (), {'pw_name': os.environ.get('USERNAME', 'unknown')})() +# Conditionally import pwd module (Unix-only) +if not PlatformUtils.get_platform_id().value.startswith("win"): + import pwd + + class TypeScriptLanguageServer(LanguageServer): """ Provides TypeScript specific instantiation of the LanguageServer class. Contains various configurations and settings specific to TypeScript. @@ -50,7 +55,7 @@ class TypeScriptLanguageServer(LanguageServer): "typescript", ) self.server_ready = asyncio.Event() - + @override def should_always_ignore(self, dirname: str) -> bool: return super().should_always_ignore(dirname) or dirname in [ @@ -95,22 +100,29 @@ class TypeScriptLanguageServer(LanguageServer): if not os.path.exists(tsserver_ls_dir): os.makedirs(tsserver_ls_dir, exist_ok=True) for dependency in runtime_dependencies: - # Handle platform-specific user settings - subprocess_kwargs = { - 'shell': True, - 'check': True, - 'cwd': tsserver_ls_dir, - 'stdout': subprocess.DEVNULL, - 'stderr': subprocess.DEVNULL - } - - # Only add user parameter on Unix-like systems - if os.name != 'nt': # Not Windows + # Windows doesn't support the 'user' parameter and doesn't have pwd module + if PlatformUtils.get_platform_id().value.startswith("win"): + subprocess.run( + dependency["command"], + shell=True, + check=True, + cwd=tsserver_ls_dir, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL + ) + else: + # On Unix-like systems, run as non-root user user = pwd.getpwuid(os.getuid()).pw_name - subprocess_kwargs['user'] = user - - subprocess.run(dependency["command"], **subprocess_kwargs) - + subprocess.run( + dependency["command"], + shell=True, + check=True, + user=user, + cwd=tsserver_ls_dir, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL + ) + tsserver_executable_path = os.path.join(tsserver_ls_dir, "node_modules", ".bin", "typescript-language-server") assert os.path.exists(tsserver_executable_path), "typescript-language-server executable not found. Please install typescript-language-server and try again." @@ -209,4 +221,4 @@ class TypeScriptLanguageServer(LanguageServer): yield self await self.server.shutdown() - await self.server.stop() \ No newline at end of file + await self.server.stop() diff --git a/src/multilspy/lsp_protocol_handler/lsp_constants.py b/src/multilspy/lsp_protocol_handler/lsp_constants.py index 6026802..149af38 100644 --- a/src/multilspy/lsp_protocol_handler/lsp_constants.py +++ b/src/multilspy/lsp_protocol_handler/lsp_constants.py @@ -57,3 +57,6 @@ class LSPConstants: # key used to represent children in document symbols CHILDREN = "children" + + # key used to represent the location in symbols + LOCATION = "location" diff --git a/src/multilspy/lsp_protocol_handler/server.py b/src/multilspy/lsp_protocol_handler/server.py index cc87c33..b9dd0de 100644 --- a/src/multilspy/lsp_protocol_handler/server.py +++ b/src/multilspy/lsp_protocol_handler/server.py @@ -33,6 +33,7 @@ import dataclasses import json import logging import os +import psutil from typing import Any, Dict, List, Optional, Union from .lsp_requests import LspNotification, LspRequest @@ -173,9 +174,18 @@ class LanguageServerHandler: the asynchronous tasks created by the handler. task_counter: An integer that represents the next available task id for the handler. loop: An asyncio.AbstractEventLoop object that represents the event loop used by the handler. + start_independent_lsp_process: An optional boolean flag that indicates whether to start the + language server process in an independent process group. Default is `True`. Setting it to + `False` means that the language server process will be in the same process group as the + the current process, and any SIGINT and SIGTERM signals will be sent to both processes. """ - def __init__(self, process_launch_info: ProcessLaunchInfo, logger=None) -> None: + def __init__( + self, + process_launch_info: ProcessLaunchInfo, + logger=None, + start_independent_lsp_process=True, + ) -> None: """ Params: cmd: A string that represents the command to launch the language server process. @@ -197,6 +207,7 @@ class LanguageServerHandler: self.tasks = {} self.task_counter = 0 self.loop = None + self.start_independent_lsp_process = start_independent_lsp_process async def start(self) -> None: """ @@ -214,6 +225,7 @@ class LanguageServerHandler: stderr=asyncio.subprocess.PIPE, env=child_proc_env, cwd=self.process_launch_info.cwd, + start_new_session=self.start_independent_lsp_process, ) # Check if process terminated immediately @@ -234,24 +246,111 @@ class LanguageServerHandler: """ Sends the terminate signal to the language server process and waits for it to exit, with a timeout, killing it if necessary """ - for task in self.tasks.values(): - task.cancel() - - self.tasks = {} - + # First cancel all tasks + await self._cancel_pending_tasks() + process = self.process self.process = None + + if not process: + return + + # Clean up the process + await self._cleanup_process(process) - if process: - # TODO: Ideally, we should terminate the process here, - # However, there's an issue with asyncio terminating processes documented at - # https://bugs.python.org/issue35539 and https://bugs.python.org/issue41320 - # process.terminate() - wait_for_end = process.wait() + async def _cancel_pending_tasks(self): + """Cancel all pending tasks and wait for them to complete or timeout.""" + pending_tasks = [] + for task in self.tasks.values(): + if not task.done(): + task.cancel() + pending_tasks.append(task) + + if pending_tasks: try: - await asyncio.wait_for(wait_for_end, timeout=60) - except asyncio.TimeoutError: - process.kill() + await asyncio.wait_for(asyncio.gather(*pending_tasks, return_exceptions=True), timeout=5.0) + except (asyncio.TimeoutError, Exception): + pass + + self.tasks = {} + + async def _cleanup_process(self, process): + """Clean up a process: close stdin, terminate/kill process, close stdout/stderr.""" + # Close stdin first to prevent deadlocks + # See: https://bugs.python.org/issue35539 + self._safely_close_pipe(process.stdin) + + # Terminate/kill the process if it's still running + if process.returncode is None: + await self._terminate_or_kill_process(process) + + # Close stdout and stderr pipes after process has exited + # This is essential to prevent "I/O operation on closed pipe" errors and + # "Event loop is closed" errors during garbage collection + # See: https://bugs.python.org/issue41320 and https://github.com/python/cpython/issues/88050 + self._safely_close_pipe(process.stdout) + self._safely_close_pipe(process.stderr) + + # Small delay to ensure OS has released file handles + await asyncio.sleep(0.5) + + def _safely_close_pipe(self, pipe): + """Safely close a pipe, ignoring any exceptions.""" + if pipe: + try: + pipe.close() + except Exception: + pass + + async def _terminate_or_kill_process(self, process): + """Try to terminate the process gracefully, then forcefully if necessary.""" + # First try to terminate the process tree gracefully + self._signal_process_tree(process, terminate=True) + + # Wait for the process to exit (with timeout) + try: + await asyncio.wait_for(process.wait(), timeout=10) + except (asyncio.TimeoutError, Exception): + # If termination failed, forcefully kill the process tree + self._signal_process_tree(process, terminate=False) + try: + # Give it one more chance to exit + await asyncio.wait_for(process.wait(), timeout=2) + except Exception: + pass + + def _signal_process_tree(self, process, terminate=True): + """Send signal (terminate or kill) to the process and all its children.""" + signal_method = "terminate" if terminate else "kill" + + # Try to get the parent process + parent = None + try: + parent = psutil.Process(process.pid) + except (psutil.NoSuchProcess, psutil.AccessDenied, Exception): + pass + + # If we have the parent process and it's running, signal the entire tree + if parent and parent.is_running(): + # Signal children first + for child in parent.children(recursive=True): + try: + getattr(child, signal_method)() + except (psutil.NoSuchProcess, psutil.AccessDenied, Exception): + pass + + # Then signal the parent + try: + getattr(parent, signal_method)() + except (psutil.NoSuchProcess, psutil.AccessDenied, Exception): + pass + else: + # Fall back to direct process signaling + try: + getattr(process, signal_method)() + except Exception: + pass + async def shutdown(self) -> None: """ @@ -310,7 +409,7 @@ class LanguageServerHandler: line = await self.process.stderr.readline() if not line: continue - self._log("LSP stderr: " + line.decode(ENCODING)) + self._log("LSP stderr: " + line.decode(ENCODING, errors='replace')) except (BrokenPipeError, ConnectionResetError, StopLoopException): pass diff --git a/src/multilspy/multilspy_config.py b/src/multilspy/multilspy_config.py index a93f122..bf53750 100644 --- a/src/multilspy/multilspy_config.py +++ b/src/multilspy/multilspy_config.py @@ -30,10 +30,13 @@ class Language(str, Enum): PYTHON = "python" RUST = "rust" JAVA = "java" + KOTLIN = "kotlin" TYPESCRIPT = "typescript" JAVASCRIPT = "javascript" GO = "go" RUBY = "ruby" + DART = "dart" + CPP = "cpp" def __str__(self) -> str: return self.value @@ -56,8 +59,14 @@ class Language(str, Enum): return FilenameMatcher("*.go") case self.RUBY: return FilenameMatcher("*.rb") + case self.CPP: + return FilenameMatcher("*.cpp", "*.h", "*.hpp", "*.c", "*.hxx", "*.cc", "*.cxx") + case self.KOTLIN: + return FilenameMatcher("*.kt", "*.kts") + case self.DART: + return FilenameMatcher("*.dart") case _: - raise ValueError + raise ValueError(f"Unhandled language: {self}") @dataclass @@ -67,6 +76,7 @@ class MultilspyConfig: """ code_language: Language trace_lsp_communication: bool = False + start_independent_lsp_process: bool = True ignored_paths: list[str] = field(default_factory=list) """Paths, dirs or glob-like patterns. The matching will follow the same logic as for .gitignore entries""" gitignore_file_content: str | None = None diff --git a/src/multilspy/multilspy_types.py b/src/multilspy/multilspy_types.py index 982959a..4a0f97e 100644 --- a/src/multilspy/multilspy_types.py +++ b/src/multilspy/multilspy_types.py @@ -82,7 +82,7 @@ class Location(TypedDict): uri: DocumentUri range: Range absolutePath: str - relativePath: str + relativePath: Union[str, None] class CompletionItemKind(IntEnum): """The kind of a completion entry.""" diff --git a/src/multilspy/multilspy_utils.py b/src/multilspy/multilspy_utils.py index 6e42aa5..bdda697 100644 --- a/src/multilspy/multilspy_utils.py +++ b/src/multilspy/multilspy_utils.py @@ -5,7 +5,7 @@ This file contains various utility functions like I/O operations, handling paths import gzip import logging import os -from typing import Tuple +from typing import Tuple, Union import requests import shutil import uuid @@ -94,6 +94,16 @@ class PathUtils: """Check if a pattern contains glob-specific characters.""" return any(c in pattern for c in '*?[]!') + @staticmethod + def get_relative_path(path: str, base_path: str) -> Union[str, None]: + """ + Gets relative path if it's possible (paths should be on the same drive), + returns `None` otherwise. + """ + if PurePath(path).drive == PurePath(base_path).drive: + return str(PurePath(os.path.relpath(path, base_path))) + return None + class FileUtils: """ Utility functions for file operations. diff --git a/src/serena/agent.py b/src/serena/agent.py index 7fce051..c3a317c 100644 --- a/src/serena/agent.py +++ b/src/serena/agent.py @@ -973,15 +973,13 @@ class InsertAtLineTool(Tool): class CheckOnboardingPerformedTool(Tool): """ - Checks whether the onboarding was already performed. + Checks whether project onboarding was already performed. """ def apply(self) -> str: """ - Check if onboarding was performed yet. - You should always call this tool in the beginning of the conversation, - before any question about code or the project is asked. - You will call this tool only once per conversation. + Checks whether project onboarding was already performed. + You should always call this tool before beginning to actually work on a project/after activating a project. """ list_memories_tool = self.agent.get_tool(ListMemoriesTool) memories = json.loads(list_memories_tool.apply()) @@ -991,7 +989,7 @@ class CheckOnboardingPerformedTool(Tool): + "You should perform onboarding by calling the `onboarding` tool before proceeding with the task." ) else: - return "Onboarding already performed, no need to perform it again." + return json.dumps({"result": "Onboarding already performed.", "available_memories": memories}) class OnboardingTool(Tool): diff --git a/src/serena/symbol.py b/src/serena/symbol.py index 1819b1a..8228024 100644 --- a/src/serena/symbol.py +++ b/src/serena/symbol.py @@ -23,9 +23,9 @@ class SymbolLocation: Represents the (start) location of a symbol identifier """ - relative_path: str + relative_path: str | None """ - the relative path of the file containing the symbol + the relative path of the file containing the symbol; if None, the symbol is defined outside of the project's scope """ line: int | None """ @@ -39,13 +39,14 @@ class SymbolLocation: """ def __post_init__(self) -> None: - self.relative_path = self.relative_path.replace("/", os.path.sep) + if self.relative_path is not None: + self.relative_path = self.relative_path.replace("/", os.path.sep) def to_dict(self) -> dict[str, Any]: return asdict(self) def has_position_in_file(self) -> bool: - return self.line is not None and self.column is not None + return self.relative_path is not None and self.line is not None and self.column is not None class Symbol(ToStringMixin): @@ -71,7 +72,7 @@ class Symbol(ToStringMixin): return self.s["kind"] @property - def relative_path(self) -> str: + def relative_path(self) -> str | None: return self.s["location"]["relativePath"] @property @@ -239,6 +240,8 @@ class SymbolManager: return symbols def find_by_location(self, location: SymbolLocation) -> Symbol | None: + if location.relative_path is None: + return None symbol_dicts, roots = self.lang_server.request_document_symbols(location.relative_path, include_body=False) for symbol_dict in symbol_dicts: symbol = Symbol(symbol_dict) @@ -268,6 +271,7 @@ class SymbolManager: """ if not symbol_location.has_position_in_file(): raise ValueError("Symbol location does not contain a valid position in a file") + assert symbol_location.relative_path is not None assert symbol_location.line is not None assert symbol_location.column is not None symbol_dicts = self.lang_server.request_referencing_symbols( @@ -301,7 +305,8 @@ class SymbolManager: def _edited_symbol_location(self, location: SymbolLocation) -> Iterator[Symbol]: symbol = self.find_by_location(location) if symbol is None: - raise ValueError("Symbol not found") + raise ValueError("Symbol not found/has no defined location within a file") + assert location.relative_path is not None with self._edited_file(location.relative_path): yield symbol @@ -313,6 +318,7 @@ class SymbolManager: :param body: the new body """ with self._edited_symbol_location(location) as symbol: + assert location.relative_path is not None self.lang_server.delete_text_between_positions(location.relative_path, symbol.body_start_position, symbol.body_end_position) self.lang_server.insert_text_at_position( location.relative_path, symbol.body_start_position["line"], symbol.body_start_position["character"], body @@ -327,6 +333,7 @@ class SymbolManager: """ with self._edited_symbol_location(location) as symbol: pos = symbol.body_end_position + assert location.relative_path is not None self.lang_server.insert_text_at_position(location.relative_path, pos["line"], pos["character"], body) def insert_before(self, location: SymbolLocation, body: str) -> None: @@ -338,6 +345,7 @@ class SymbolManager: """ with self._edited_symbol_location(location) as symbol: pos = copy(symbol.body_start_position) + assert location.relative_path is not None self.lang_server.insert_text_at_position(location.relative_path, pos["line"], pos["character"], body) def insert_at_line(self, relative_path: str, line: int, content: str) -> None: diff --git a/uv.lock b/uv.lock index 10c6be6..8c410be 100644 --- a/uv.lock +++ b/uv.lock @@ -687,6 +687,21 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e4/ea/d836f008d33151c7a1f62caf3d8dd782e4d15f6a43897f64480c2b8de2ad/prompt_toolkit-3.0.50-py3-none-any.whl", hash = "sha256:9b6427eb19e479d98acff65196a307c555eb567989e6d88ebbb1b509d9779198", size = 387816 }, ] +[[package]] +name = "psutil" +version = "7.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/2a/80/336820c1ad9286a4ded7e845b2eccfcb27851ab8ac6abece774a6ff4d3de/psutil-7.0.0.tar.gz", hash = "sha256:7be9c3eba38beccb6495ea33afd982a44074b78f28c434a1f51cc07fd315c456", size = 497003 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ed/e6/2d26234410f8b8abdbf891c9da62bee396583f713fb9f3325a4760875d22/psutil-7.0.0-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:101d71dc322e3cffd7cea0650b09b3d08b8e7c4109dd6809fe452dfd00e58b25", size = 238051 }, + { url = "https://files.pythonhosted.org/packages/04/8b/30f930733afe425e3cbfc0e1468a30a18942350c1a8816acfade80c005c4/psutil-7.0.0-cp36-abi3-macosx_11_0_arm64.whl", hash = "sha256:39db632f6bb862eeccf56660871433e111b6ea58f2caea825571951d4b6aa3da", size = 239535 }, + { url = "https://files.pythonhosted.org/packages/2a/ed/d362e84620dd22876b55389248e522338ed1bf134a5edd3b8231d7207f6d/psutil-7.0.0-cp36-abi3-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:1fcee592b4c6f146991ca55919ea3d1f8926497a713ed7faaf8225e174581e91", size = 275004 }, + { url = "https://files.pythonhosted.org/packages/bf/b9/b0eb3f3cbcb734d930fdf839431606844a825b23eaf9a6ab371edac8162c/psutil-7.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4b1388a4f6875d7e2aff5c4ca1cc16c545ed41dd8bb596cefea80111db353a34", size = 277986 }, + { url = "https://files.pythonhosted.org/packages/eb/a2/709e0fe2f093556c17fbafda93ac032257242cabcc7ff3369e2cb76a97aa/psutil-7.0.0-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a5f098451abc2828f7dc6b58d44b532b22f2088f4999a937557b603ce72b1993", size = 279544 }, + { url = "https://files.pythonhosted.org/packages/50/e6/eecf58810b9d12e6427369784efe814a1eec0f492084ce8eb8f4d89d6d61/psutil-7.0.0-cp37-abi3-win32.whl", hash = "sha256:ba3fcef7523064a6c9da440fc4d6bd07da93ac726b5733c29027d7dc95b39d99", size = 241053 }, + { url = "https://files.pythonhosted.org/packages/50/1b/6921afe68c74868b4c9fa424dad3be35b095e16687989ebbb50ce4fceb7c/psutil-7.0.0-cp37-abi3-win_amd64.whl", hash = "sha256:4cf3d4eb1aa9b348dec30105c55cd9b7d4629285735a102beb4441e38db90553", size = 244885 }, +] + [[package]] name = "ptyprocess" version = "0.7.0" @@ -951,6 +966,7 @@ dependencies = [ { name = "mcp" }, { name = "overrides" }, { name = "pathspec" }, + { name = "psutil" }, { name = "pydantic" }, { name = "pyright" }, { name = "python-dotenv" }, @@ -1000,6 +1016,7 @@ requires-dist = [ { name = "overrides", specifier = ">=7.7.0,<8" }, { name = "pathspec", specifier = ">=0.12.1" }, { name = "poethepoet", marker = "extra == 'dev'", specifier = ">=0.20.0" }, + { name = "psutil", specifier = ">=7.0.0" }, { name = "pydantic", specifier = ">=2.10.6" }, { name = "pyright", specifier = ">=1.1.396,<2" }, { name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0.2" },