From 90c42923c8299348634a3ebf7a23d9411d55ad65 Mon Sep 17 00:00:00 2001 From: Dominik Jain Date: Tue, 8 Apr 2025 20:10:34 +0200 Subject: [PATCH 01/14] multilspy 43209c8521e93e8c41bdd2060800474fd0b9f8b1 commit 6c2565baababc9c978b7c1ab40a56fadef16c816 Author: Developer Date: Fri Apr 4 16:08:10 2025 +0530 address review comments for clangd support src/multilspy/language_servers/clangd_language_server/clangd_language_server.py commit 78f1c2b43ee16bb29b222a1c1af77956d6fd906d Author: IDontHaveBrain Date: Thu Apr 3 09:00:26 2025 +0900 chore: Remove win-arm64 support from kotlin runtime dependencies src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json commit b907cc227a01f4a500d3445bfa17d7237f91c5ec Author: nj.jo Date: Tue Apr 1 22:39:18 2025 +0900 chore: use vscode-java runtime dependencies src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json commit 3d19a282fe3d68d1a0b589c879775cac7ec23b8e Author: moz-mdauer Date: Tue Apr 1 13:56:45 2025 +0200 feat: add support for workspace/symbol request src/multilspy/language_server.py src/multilspy/lsp_protocol_handler/lsp_constants.py commit e60081cbcac5b938ee8936f715478321c2be9b5c Author: nj.jo Date: Sat Mar 29 14:55:16 2025 +0900 refactor: improve process termination handling src/multilspy/lsp_protocol_handler/server.py commit 88345e14e2172390b5f0d48d7877b2ee8620c187 Author: nj.jo Date: Sat Mar 29 14:12:28 2025 +0900 feat: add JDK runtime dependency src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json commit ef9ad63083ebbbf929fd1cc4d0cc084e5cb69ad4 Author: Andrej730 Date: Sun Mar 23 19:26:08 2025 +0500 Location.relativePath to use `None` value if relative path is not available #85 src/multilspy/language_server.py src/multilspy/multilspy_types.py src/multilspy/multilspy_utils.py commit 85b1fe84140ec7bb84db843d618ea4bd8f4a096c Author: Huang YunKun Date: Fri Mar 28 11:40:34 2025 +0800 Update runtime_dependencies.json src/multilspy/language_servers/eclipse_jdtls/runtime_dependencies.json commit f0a42717cd95f9d7a845603cee60ac2fd22f16ac Author: Andrej730 Date: Sun Mar 23 19:15:21 2025 +0500 SyncLanguageServer.create - small documentation note To clarify that relative paths for repositories are not supported. Currently if it's using relative path, `create_server` fails with traceback similar to the one below: ```python Traceback (most recent call last): File "third_test.py", line 11, in with lsp.start_server(): File "\Python311\Lib\contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "\Python311\Lib\site-packages\multilspy\language_server.py", line 744, in start_server asyncio.run_coroutine_threadsafe(ctx.__aenter__(), loop=self.loop).result() File "\Python311\Lib\concurrent\futures\_base.py", line 456, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "\Python311\Lib\concurrent\futures\_base.py", line 401, in __get_result raise self._exception File "\Python311\Lib\contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "\Python311\Lib\site-packages\multilspy\language_servers\jedi_language_server\jedi_server.py", line 101, in start_server initialize_params = self._get_initialize_params(self.repository_root_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "\Python311\Lib\site-packages\multilspy\language_servers\jedi_language_server\jedi_server.py", line 50, in _get_initialize_params d["rootUri"] = pathlib.Path(repository_absolute_path).as_uri() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "\Python311\Lib\pathlib.py", line 566, in as_uri raise ValueError("relative path can't be expressed as a file URI") ValueError: relative path can't be expressed as a file URI ``` src/multilspy/language_server.py commit 9f69f75b1c9a460d63dbcc9d1b2b50596ee8a49b Author: Developer Date: Tue Mar 18 18:54:18 2025 +0530 clangd language server commit src/multilspy/language_server.py src/multilspy/language_servers/clangd_language_server/clangd_language_server.py src/multilspy/language_servers/clangd_language_server/initialize_params.json src/multilspy/language_servers/clangd_language_server/runtime_dependencies.json src/multilspy/multilspy_config.py commit edeb1771b8cd6064e6eca28a1126988780fc4608 Author: nj.jo Date: Sun Mar 9 01:26:59 2025 +0900 test: update Kotlin executable permissions and adjust Java version in workflow src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py commit 79a8f6902e91c0a1301c63b9f8737efbcbcccf24 Author: nj.jo Date: Sun Mar 9 00:47:38 2025 +0900 fix: improve process termination handling src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py src/multilspy/lsp_protocol_handler/server.py commit 90622f2788df412021565008dd83c27ac4d0b467 Author: nj.jo Date: Sun Mar 9 00:10:28 2025 +0900 fix: improve process termination handling src/multilspy/lsp_protocol_handler/server.py commit 907e6f340dec6b6e50c5712059bf3563fd7f064c Author: nj.jo Date: Sat Mar 8 23:52:07 2025 +0900 fix: improve process termination handling src/multilspy/lsp_protocol_handler/server.py commit 694b30108ff3025b7f6f2e7923c6077f6b0dee23 Author: nj.jo Date: Sat Mar 8 23:45:48 2025 +0900 chore: cleanup src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py commit 635a6a83617e02a24c141a85b8dd8cabbf3a4601 Author: nj.jo Date: Sat Mar 8 21:14:00 2025 +0900 fix: platform check for Kotlin LSP from architecture-specific to OS-only src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py commit 2b053ee8b1f8b8d2ff391110baf27bb21163b715 Author: nj.jo Date: Sat Mar 8 20:25:05 2025 +0900 feat: Update Kotlin Language Server configuration and runtime dependencies src/multilspy/language_servers/kotlin_language_server/initialize_params.json src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json commit 59f84082b9cd64e64bfc9a7644a644fdb8cc881e Author: nj.jo Date: Sat Mar 8 17:51:18 2025 +0900 feat: Add Kotlin language server support src/multilspy/language_server.py src/multilspy/language_servers/kotlin_language_server/initialize_params.json src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json src/multilspy/multilspy_config.py commit c85e2825aaa85d64611e7bfb392daa6eb7aa54ac Author: Jet Zhou Date: Thu Mar 6 16:10:28 2025 -0800 Add `start_independent_lsp_process` configuration option to `MultilspyConfig` Default value is `True`, which means language server process will be in its own process group. SIGINT and SIGTERM will only be received by the `multilspy` python process itself, not propagated. We rely on the existing `shutdown` and `stop` code to clean up the process. If set to `False`, the language server process will be under the same process group as the `multilspy` python process. This means SIGINT and SIGTERM will be received by both processes. This may cause issues if the python process has clean up code that runs after receiving SIGINT/SIGTERM because the `start_server` context manager will begin running its cleanup code, part of which invokes the `shutdown` and `stop` in `language_protocol_handler.server`. This will hang forever while awaiting an response from the language server process, since it would have already terminated itself after receiving SIGINT/SIGTERM. src/multilspy/language_server.py src/multilspy/lsp_protocol_handler/server.py src/multilspy/multilspy_config.py commit ff17f9365f4862627f00c43f8f88cd7a76f10fc6 Author: Jet Zhou Date: Tue Mar 4 22:27:55 2025 -0800 Let language server subprocess be its own process group In situations where `multilspy` is launched from another process, a SIGINT or SIGTERM signal will be received by both the launching process as well as the process running the language server because `asyncio.create_subprocess_shell` defaults to `start_new_session=False`. In such a situation, `lsp_protocol_handler.server.shutdown` will be called but will hang since the underlying process is already terminated, `lsp_protocol_handler.server` won't ever receive an answer back for `shutdown`. This change makes it so that SIGINT and SIGTERM won't propagate to the language server process. We rely on the clean up code already in place to shut down the process as desired. See [python documentation](https://docs.python.org/3.13/library/subprocess.html#subprocess.Popen) for more details on `start_new_session` src/multilspy/lsp_protocol_handler/server.py commit ee104b062d1e0fda4a063937f5ca6488ac42683e Author: Jet Zhou Date: Mon Mar 3 18:08:27 2025 -0800 Add assertion string to `references` and `document_symbol` requests to language server These endpoints could return `None` in case the symbol is one that's not found. Adding the assertion string can help caller catch these exceptions and handle the errors as appropriate. src/multilspy/language_server.py commit 2edff10b31585fbcd5829afc07d1258b3b83b7c8 Author: nj.jo Date: Sun Mar 2 12:49:34 2025 +0900 refactor: Change to use `PlatformUtils` instead of `sys.platform` src/multilspy/language_servers/typescript_language_server/typescript_language_server.py commit e2cc4dd5f54adf89a2e4e8140387fc71fd5ebc56 Author: v4rgas <66626747+v4rgas@users.noreply.github.com> Date: Sat Mar 1 20:25:38 2025 -0300 feat: add initialization options for Dart Language Server src/multilspy/language_servers/dart_language_server/initialize_params.json commit 807ec41b1098a2aecccbbe074f80d8c2dbd15d5d Author: v4rgas <66626747+v4rgas@users.noreply.github.com> Date: Sat Mar 1 20:15:38 2025 -0300 fix: update description for Dart Language Server runtime dependencies src/multilspy/language_servers/dart_language_server/runtime_dependencies.json commit ca5e3e91e7093838f3cc33ef5fb5561fdf9cf7ee Author: v4rgas <66626747+v4rgas@users.noreply.github.com> Date: Sat Mar 1 20:12:53 2025 -0300 feat: implement Dart Language Server runtime dependency setup src/multilspy/language_servers/dart_language_server/dart_language_server.py src/multilspy/language_servers/dart_language_server/runtime_dependencies.json commit 6295dd19db279033e2507dd665bd5c7df79e2623 Author: nj.jo Date: Sat Mar 1 15:18:41 2025 +0900 fix: resolve 'No module named pwd' error on Windows src/multilspy/language_servers/typescript_language_server/typescript_language_server.py commit 2390a21479553e2b42e06eb10049d5c09ab9ef0a Author: v4rgas <66626747+v4rgas@users.noreply.github.com> Date: Fri Feb 28 09:54:58 2025 -0300 fix race codnition in process termination src/multilspy/lsp_protocol_handler/server.py commit e193501fb605ba86d52513f9fe044c2576bfc4cd Author: v4rgas <66626747+v4rgas@users.noreply.github.com> Date: Thu Feb 27 22:05:03 2025 -0300 feat: add Dart language support and corresponding tests src/multilspy/language_server.py src/multilspy/language_servers/dart_language_server/dart_language_server.py src/multilspy/language_servers/dart_language_server/initialize_params.json src/multilspy/multilspy_config.py commit 11c0da5450d0ce032b5ea832561e87ead6a0ca5b Author: v4rgas <66626747+v4rgas@users.noreply.github.com> Date: Thu Feb 27 21:43:32 2025 -0300 Add psutil dependency and enhance process termination handling src/multilspy/lsp_protocol_handler/server.py commit 7b85f39636e1971b0e9e0b9e69f4f8d7be51f931 Author: v4rgas <66626747+v4rgas@users.noreply.github.com> Date: Thu Feb 27 09:25:02 2025 -0300 chore: undo styling changes src/multilspy/language_server.py commit 0d567618dda795c0789b374667c0325c3df4c87e Author: Jet Zhou Date: Tue Feb 11 17:55:35 2025 -0800 Force `npm i` to use current directory, in case some parent directory already contains a node_modules folder src/multilspy/language_servers/typescript_language_server/runtime_dependencies.json --- src/multilspy/.syncCommitId.remote | 2 +- src/multilspy/.syncCommitId.this | 1 - src/multilspy/language_server.py | 389 +++++++------ .../clangd_language_server.py | 180 ++++++ .../initialize_params.json | 36 ++ .../runtime_dependencies.json | 13 + .../dart_language_server.py | 146 +++++ .../initialize_params.json | 23 + .../runtime_dependencies.json | 13 + .../eclipse_jdtls/runtime_dependencies.json | 12 +- .../initialize_params.json | 521 ++++++++++++++++++ .../kotlin_language_server.py | 234 ++++++++ .../runtime_dependencies.json | 41 ++ .../typescript_language_server.py | 46 +- .../lsp_protocol_handler/lsp_constants.py | 3 + src/multilspy/lsp_protocol_handler/server.py | 131 ++++- src/multilspy/multilspy_config.py | 4 + src/multilspy/multilspy_types.py | 2 +- src/multilspy/multilspy_utils.py | 12 +- 19 files changed, 1608 insertions(+), 201 deletions(-) delete mode 100644 src/multilspy/.syncCommitId.this create mode 100644 src/multilspy/language_servers/clangd_language_server/clangd_language_server.py create mode 100644 src/multilspy/language_servers/clangd_language_server/initialize_params.json create mode 100644 src/multilspy/language_servers/clangd_language_server/runtime_dependencies.json create mode 100644 src/multilspy/language_servers/dart_language_server/dart_language_server.py create mode 100644 src/multilspy/language_servers/dart_language_server/initialize_params.json create mode 100644 src/multilspy/language_servers/dart_language_server/runtime_dependencies.json create mode 100644 src/multilspy/language_servers/kotlin_language_server/initialize_params.json create mode 100644 src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py create mode 100644 src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json diff --git a/src/multilspy/.syncCommitId.remote b/src/multilspy/.syncCommitId.remote index 80a2eae..44ef72b 100644 --- a/src/multilspy/.syncCommitId.remote +++ b/src/multilspy/.syncCommitId.remote @@ -1 +1 @@ -07ffa48166beb0c7f20ac76dbf7724d0bca4bbb4 \ No newline at end of file +43209c8521e93e8c41bdd2060800474fd0b9f8b1 \ No newline at end of file diff --git a/src/multilspy/.syncCommitId.this b/src/multilspy/.syncCommitId.this deleted file mode 100644 index 84a09c8..0000000 --- a/src/multilspy/.syncCommitId.this +++ /dev/null @@ -1 +0,0 @@ -e766d55fd568f5313fb46796b8463097bdb9a293 \ No newline at end of file diff --git a/src/multilspy/language_server.py b/src/multilspy/language_server.py index f387446..d60f679 100644 --- a/src/multilspy/language_server.py +++ b/src/multilspy/language_server.py @@ -37,6 +37,9 @@ from .lsp_protocol_handler.server import ( ) from .multilspy_config import Language, MultilspyConfig from .multilspy_exceptions import MultilspyException +from .multilspy_utils import PathUtils, FileUtils, TextUtils +from pathlib import PurePath +from typing import AsyncIterator, Iterator, List, Dict, Optional, Union, Tuple from .multilspy_logger import MultilspyLogger from .multilspy_utils import FileUtils, PathUtils, TextUtils from .type_helpers import ensure_all_methods_implemented @@ -72,11 +75,11 @@ class LSPFileBuffer: # reference count of the file ref_count: int - + # --------------------------------- MODIFICATIONS BY MISCHA --------------------------------- - + content_hash: str = "" - + def __post_init__(self): self.content_hash = hashlib.md5(self.contents.encode('utf-8')).hexdigest() @@ -148,6 +151,12 @@ class LanguageServer: ) return EclipseJDTLS(config, logger, repository_root_path) + elif config.code_language == Language.KOTLIN: + from multilspy.language_servers.kotlin_language_server.kotlin_language_server import ( + KotlinLanguageServer, + ) + + return KotlinLanguageServer(config, logger, repository_root_path) elif config.code_language == Language.RUST: from multilspy.language_servers.rust_analyzer.rust_analyzer import ( RustAnalyzer, @@ -171,6 +180,14 @@ class LanguageServer: from multilspy.language_servers.solargraph.solargraph import Solargraph return Solargraph(config, logger, repository_root_path) + elif config.code_language == Language.DART: + from multilspy.language_servers.dart_language_server.dart_language_server import DartLanguageServer + + return DartLanguageServer(config, logger, repository_root_path) + elif config.code_language == Language.CPP: + from multilspy.language_servers.clangd_language_server.clangd_language_server import ClangdLanguageServer + + return ClangdLanguageServer(config, logger, repository_root_path) else: logger.log(f"Language {config.code_language} is not supported", logging.ERROR) raise MultilspyException(f"Language {config.code_language} is not supported") @@ -205,7 +222,7 @@ class LanguageServer: self.server_started = False self.repository_root_path: str = repository_root_path self.completions_available = asyncio.Event() - + if config.trace_lsp_communication: def logging_fn(source, target, msg): @@ -218,18 +235,22 @@ class LanguageServer: # cmd is obtained from the child classes, which provide the language specific command to start the language server # LanguageServerHandler provides the functionality to start the language server and communicate with it - self.server: LanguageServerHandler = LanguageServerHandler(process_launch_info, logger=logging_fn) + self.server: LanguageServerHandler = LanguageServerHandler( + process_launch_info, + logger=logging_fn, + start_independent_lsp_process=config.start_independent_lsp_process, + ) self.language_id = language_id self.open_file_buffers: Dict[str, LSPFileBuffer] = {} - + # --------------------------------- MODIFICATIONS BY ORAIOS --------------------------------- self._document_symbols_cache: dict[str, Tuple[str, Tuple[List[multilspy_types.UnifiedSymbolInformation], List[multilspy_types.UnifiedSymbolInformation]]]] = {} """Maps file paths to a tuple of (file_content_hash, result_of_request_document_symbols)""" self.load_cache() self._cache_has_changed = bool self.language = Language(language_id) - + # Set up the pathspec matcher for the ignored paths # for all absolute paths in ignored_paths, convert them to relative paths processed_patterns = [] @@ -253,9 +274,9 @@ class LanguageServer: """ Determine if a path should be ignored based on file type and ignore patterns. - + :param relative_path: Relative path to check - + :return: True if the path should be ignored, False otherwise """ # Check file extension if it's a file @@ -263,13 +284,13 @@ class LanguageServer: abs_path = os.path.join(self.repository_root_path, relative_path) if not os.path.exists(abs_path): raise FileNotFoundError(f"File {abs_path} not found, the ignore check cannot be performed") - + if os.path.isfile(abs_path) and not fn_matcher.is_relevant_filename(abs_path): return True - + # Create normalized path for consistent handling rel_path = Path(relative_path) - + # Check each part of the path against always fulfilled ignore conditions for part in rel_path.parts: if not part: # Skip empty parts (e.g., from leading '/') @@ -277,20 +298,20 @@ class LanguageServer: # Check standard ignores if self.should_always_ignore(part): return True - + # Use pathspec for gitignore-style pattern matching # Normalize path separators for pathspec (it expects forward slashes) normalized_path = str(rel_path).replace(os.path.sep, '/') - + # pathspec can't handle the matching of directories if they don't end with a slash! # see https://github.com/cpburnz/python-pathspec/issues/89 if os.path.isdir(os.path.join(self.repository_root_path, normalized_path)) and not normalized_path.endswith('/'): normalized_path = normalized_path + '/' - + # Use the pathspec matcher to check if the path matches any ignore pattern if self.ignore_spec.match_file(normalized_path): return True - + return False @@ -505,6 +526,7 @@ class LanguageServer: new_item.update(item) new_item["absolutePath"] = PathUtils.uri_to_path(new_item["uri"]) try: + # TODO can we use PathUtils.get_relative_path? (same elsewhere!) new_item["relativePath"] = str( PurePath(os.path.relpath(new_item["absolutePath"], self.repository_root_path)) ) @@ -538,9 +560,7 @@ class LanguageServer: new_item: multilspy_types.Location = {} new_item.update(response) new_item["absolutePath"] = PathUtils.uri_to_path(new_item["uri"]) - new_item["relativePath"] = str( - PurePath(os.path.relpath(new_item["absolutePath"], self.repository_root_path)) - ) + new_item["relativePath"] = PathUtils.get_relative_path(new_item["absolutePath"], self.repository_root_path) ret.append(multilspy_types.Location(**new_item)) elif response is None: # Some language servers return None when they cannot find a definition @@ -590,12 +610,12 @@ class LanguageServer: ) ret: List[multilspy_types.Location] = [] - assert isinstance(response, list) + assert isinstance(response, list), f"Unexpected response from Language Server: {response}" for item in response: assert isinstance(item, dict) assert LSPConstants.URI in item assert LSPConstants.RANGE in item - + abs_path = PathUtils.uri_to_path(item[LSPConstants.URI]) rel_path = Path(abs_path).relative_to(self.repository_root_path) if self.should_ignore_path(str(rel_path)): @@ -609,7 +629,7 @@ class LanguageServer: ret.append(multilspy_types.Location(**new_item)) return ret - + async def request_references_with_content( self, relative_file_path: str, line: int, column: int, context_lines_before: int = 0, context_lines_after: int = 0 ) -> List[MatchedConsecutiveLines]: @@ -626,18 +646,18 @@ class LanguageServer: """ references = await self.request_references(relative_file_path, line, column) return [self.retrieve_content_around_line(ref["relativePath"], ref["range"]["start"]["line"], context_lines_before, context_lines_after) for ref in references] - + def retrieve_full_file_content(self, relative_file_path: str) -> str: """ Retrieve the full content of the given file. """ with self.open_file(relative_file_path) as file_data: return file_data.contents - + def retrieve_content_around_line(self, relative_file_path: str, line: int, context_lines_before: int = 0, context_lines_after: int = 0) -> MatchedConsecutiveLines: """ Retrieve the content of the given file around the given line. - + :param relative_file_path: The relative path of the file to retrieve the content from :param line: The line number to retrieve the content around :param context_lines_before: The number of lines to retrieve before the given line @@ -647,7 +667,7 @@ class LanguageServer: """ with self.open_file(relative_file_path) as file_data: file_contents = file_data.contents - + line_contents = file_contents.split("\n") start_lineno = max(0, line - context_lines_before) end_lineno = min(len(line_contents) - 1, line + context_lines_after) @@ -661,10 +681,10 @@ class LanguageServer: # after the line for lineno in range(line + 1, end_lineno + 1): text_lines.append(TextLine(line_number=lineno, line_content=line_contents[lineno], match_type=LineType.AFTER_MATCH)) - + return MatchedConsecutiveLines(lines=text_lines, source_file_path=relative_file_path) - - + + async def request_completions( self, relative_file_path: str, line: int, column: int, allow_incomplete: bool = False ) -> List[multilspy_types.CompletionItem]: @@ -783,8 +803,8 @@ class LanguageServer: return result else: self.logger.log(f"Content for {relative_file_path} has changed. Overwriting cache", logging.INFO) - - + + response = await self.server.send.document_symbol( { "textDocument": { @@ -792,7 +812,7 @@ class LanguageServer: } } ) - + def turn_item_into_symbol_with_children(item: GenericDocumentSymbol): item = cast(multilspy_types.UnifiedSymbolInformation, item) if "location" not in item: @@ -809,9 +829,9 @@ class LanguageServer: if include_body: item['body'] = self.retrieve_symbol_body(item) item[LSPConstants.CHILDREN] = item.get(LSPConstants.CHILDREN, []) - + flat_all_symbol_list: List[multilspy_types.UnifiedSymbolInformation] = [] - assert isinstance(response, list) + assert isinstance(response, list), f"Unexpected response from Language Server: {response}" root_nodes: List[multilspy_types.UnifiedSymbolInformation] = [] for item in response: if "range" not in item and "location" not in item: @@ -870,10 +890,10 @@ class LanguageServer: if self.should_ignore_path(within_relative_path): self.logger.log(f"You passed a file explicitly, but it is ignored. This is probably an error. File: {within_relative_path}", logging.ERROR) return [] - + _, root_nodes = await self.request_document_symbols(within_relative_path, include_body=include_body) return root_nodes - + # Helper function to recursively process directories async def process_directory(dir_path: str) -> List[multilspy_types.UnifiedSymbolInformation]: abs_dir_path = self.repository_root_path if dir_path == "." else os.path.join(self.repository_root_path, dir_path) @@ -948,7 +968,7 @@ class LanguageServer: ), children=root_nodes ) - + package_symbol["children"].append(file_symbol) return result @@ -972,15 +992,15 @@ class LanguageServer: async def request_dir_overview(self, relative_dir_path: str) -> dict[str, list[tuple[str, multilspy_types.SymbolKind, int, int]]]: """ - An overview of the given directory. - - Maps relative paths of all contained files to info about top-level symbols in the file + An overview of the given directory. + + Maps relative paths of all contained files to info about top-level symbols in the file (name, kind, line, column). """ symbol_tree = await self.request_full_symbol_tree(relative_dir_path) # Initialize result dictionary result: dict[str, list[tuple[str, multilspy_types.SymbolKind, int, int]]] = defaultdict(list) - + # Helper function to process a symbol and its children def process_symbol(symbol: multilspy_types.UnifiedSymbolInformation): if symbol["kind"] == multilspy_types.SymbolKind.File: @@ -998,15 +1018,15 @@ class LanguageServer: # For package/directory symbols, process their children for child in symbol["children"]: process_symbol(child) - + # Process each root symbol for root in symbol_tree: process_symbol(root) return result - + async def request_document_overview(self, relative_file_path: str) -> list[tuple[str, multilspy_types.SymbolKind, int, int]]: """ - An overview of the given file. + An overview of the given file. Returns the list of tuples (name, kind, line, column) of all top-level symbols in the file. """ _, document_roots = await self.request_document_symbols(relative_file_path) @@ -1019,7 +1039,7 @@ class LanguageServer: ) for root in document_roots ] - + async def request_hover(self, relative_file_path: str, line: int, column: int) -> Union[multilspy_types.Hover, None]: """ Raise a [textDocument/hover](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_hover) request to the Language Server @@ -1050,9 +1070,9 @@ class LanguageServer: assert isinstance(response, dict) return multilspy_types.Hover(**response) - + # ----------------------------- FROM HERE ON MODIFICATIONS BY MISCHA -------------------- - + def retrieve_symbol_body(self, symbol: multilspy_types.UnifiedSymbolInformation | LSPTypes.DocumentSymbol | LSPTypes.SymbolInformation) -> str: """ Load the body of the given symbol. If the body is already contained in the symbol, just return it. @@ -1060,7 +1080,7 @@ class LanguageServer: existing_body = symbol.get("body", None) if existing_body: return existing_body - + assert "location" in symbol symbol_start_line = symbol["location"]["range"]["start"]["line"] symbol_end_line = symbol["location"]["range"]["end"]["line"] @@ -1068,19 +1088,19 @@ class LanguageServer: symbol_file = self.retrieve_full_file_content(symbol["location"]["relativePath"]) symbol_lines = symbol_file.split("\n") symbol_body = "\n".join(symbol_lines[symbol_start_line:symbol_end_line]) - + # remove leading indentation symbol_start_column = symbol["location"]["range"]["start"]["character"] symbol_body = symbol_body[symbol_start_column:] return symbol_body - - + + async def request_parsed_files(self) -> list[str]: """ Retrieves relative paths of all files analyzed by the Language Server. - - This is slow, as it finds all files by finding all symbols. - + + This is slow, as it finds all files by finding all symbols. + This seems to be the only way, the LSP does not provide any endpoints for listing project files.""" if not self.server_started: self.logger.log( @@ -1092,27 +1112,27 @@ class LanguageServer: # I don't know why # params = LSPTypes.WorkspaceSymbolParams(query="") # Empty query returns all symbols # symbols = await self.server.send.workspace_symbol(params) or [] - + # Thus, instead of calling all symbols, we hack this and use the symbol tree instead, which # seems to work in all these language servers # walk through all children recursively, find all symbols of type Module and collect their relative paths roots = await self.request_full_symbol_tree() - paths = [] + paths = [] def collect_module_files(symbol): if symbol["kind"] == multilspy_types.SymbolKind.File: assert "location" in symbol paths.append(symbol["location"]["relativePath"]) - + elif symbol["kind"] == multilspy_types.SymbolKind.Package: for child in symbol["children"]: collect_module_files(child) - + for root in roots: collect_module_files(root) - + return paths - - + + async def search_files_for_pattern( self, pattern: re.Pattern | str, @@ -1123,7 +1143,7 @@ class LanguageServer: ) -> list[MatchedConsecutiveLines]: """ Search for a pattern across all files analyzed by the Language Server. - + :param pattern: Regular expression pattern to search for, either as a compiled Pattern or string :param context_lines_before: Number of lines of context to include before each match :param context_lines_after: Number of lines of context to include after each match @@ -1133,7 +1153,7 @@ class LanguageServer: """ if isinstance(pattern, str): pattern = re.compile(pattern) - + matches = [] all_files = await self.request_parsed_files() for path in all_files: @@ -1142,26 +1162,26 @@ class LanguageServer: if paths_include_glob and not fnmatch(path, paths_include_glob): self.logger.log(f"Skipping {path}: does not match include pattern {paths_include_glob}", logging.DEBUG) continue - + if paths_exclude_glob and fnmatch(path, paths_exclude_glob): self.logger.log(f"Skipping {path}: matches exclude pattern {paths_exclude_glob}", logging.DEBUG) continue - + file_content = self.retrieve_full_file_content(path) search_results = search_text( - pattern, - file_content, - source_file_path=path, - allow_multiline_match=True, - context_lines_before=context_lines_before, + pattern, + file_content, + source_file_path=path, + allow_multiline_match=True, + context_lines_before=context_lines_before, context_lines_after=context_lines_after ) if len(search_results) > 0: self.logger.log(f"Found {len(search_results)} matches in {path}", logging.DEBUG) matches.extend(search_results) - + return matches - + async def request_referencing_symbols( self, relative_file_path: str, @@ -1183,7 +1203,7 @@ class LanguageServer: :param include_imports: whether to also include imports as references. Unfortunately, the LSP does not have an import type, so the references corresponding to imports will not be easily distinguishable from definitions. - :param include_self: whether to include the references that is the "input symbol" itself. + :param include_self: whether to include the references that is the "input symbol" itself. Only has an effect if the relative_file_path, line and column point to a symbol, for example a definition. :param include_body: whether to include the body of the symbols in the result. :param include_file_symbols: whether to include references that are file symbols. This @@ -1219,7 +1239,7 @@ class LanguageServer: # TODO: HORRIBLE HACK! I don't know how to do it better for now... # THIS IS BOUND TO BREAK IN MANY CASES! IT IS ALSO SPECIFIC TO PYTHON! # Background: - # When a variable is used to change something, like + # When a variable is used to change something, like # # instance = MyClass() # instance.status = "new status" @@ -1230,7 +1250,7 @@ class LanguageServer: # by using the text of the reference to find the variable name (In a very heuristic way) # and then look for a symbol with that name and kind Variable ref_text = file_data.contents.split("\n")[ref_line] - if "." in ref_text: + if "." in ref_text: containing_symbol_name = ref_text.split(".")[0] all_symbols, _ = await self.request_document_symbols(ref_path) for symbol in all_symbols: @@ -1271,7 +1291,7 @@ class LanguageServer: ) if containing_symbol is None or not include_file_symbols and containing_symbol["kind"] == multilspy_types.SymbolKind.File: continue - + assert "location" in containing_symbol assert "selectionRange" in containing_symbol @@ -1288,7 +1308,7 @@ class LanguageServer: else: self.logger.log(f"Found self-reference for {incoming_symbol['name']}, skipping it since {include_self=}", logging.DEBUG) continue - + # checking whether reference is an import # This is neither really safe nor elegant, but if we don't do it, # there is no way to distinguish between definitions and imports as import is not a symbol-type @@ -1304,11 +1324,11 @@ class LanguageServer: logging.DEBUG ) continue - + result.append(containing_symbol) return result - + async def request_containing_symbol( self, relative_file_path: str, @@ -1357,7 +1377,7 @@ class LanguageServer: return None symbols, _ = await self.request_document_symbols(relative_file_path) - + # make jedi and pyright api compatible # the former has no location, the later has no range # we will just always add location of the desired format to all symbols @@ -1380,8 +1400,8 @@ class LanguageServer: # Allowed container kinds, currently only for Python container_symbol_kinds = { - multilspy_types.SymbolKind.Method, - multilspy_types.SymbolKind.Function, + multilspy_types.SymbolKind.Method, + multilspy_types.SymbolKind.Function, multilspy_types.SymbolKind.Class } @@ -1408,7 +1428,7 @@ class LanguageServer: s for s in symbols if s["kind"] == multilspy_types.SymbolKind.Variable ] candidate_containers.extend(var_containers) - + if not candidate_containers: return None @@ -1428,11 +1448,11 @@ class LanguageServer: return containing_symbol else: return None - + async def request_container_of_symbol(self, symbol: multilspy_types.UnifiedSymbolInformation, include_body: bool = False) -> multilspy_types.UnifiedSymbolInformation | None: """ Finds the container of the given symbol if there is one. - + :param symbol: The symbol to find the container of. :param include_body: whether to include the body of the symbol in the result. """ @@ -1444,7 +1464,7 @@ class LanguageServer: strict=True, include_body=include_body, ) - + async def request_defining_symbol( self, relative_file_path: str, @@ -1454,10 +1474,10 @@ class LanguageServer: ) -> Optional[multilspy_types.UnifiedSymbolInformation]: """ Finds the symbol that defines the symbol at the given location. - + This method first finds the definition of the symbol at the given position, then retrieves the full symbol information for that definition. - + :param relative_file_path: The relative path to the file. :param line: The 0-indexed line number. :param column: The 0-indexed column number. @@ -1470,29 +1490,29 @@ class LanguageServer: logging.ERROR, ) raise MultilspyException("Language Server not started") - + # Get the definition location(s) definitions = await self.request_definition(relative_file_path, line, column) if not definitions: return None - + # Use the first definition location definition = definitions[0] def_path = definition["relativePath"] def_line = definition["range"]["start"]["line"] def_col = definition["range"]["start"]["character"] - + # Find the symbol at or containing this location defining_symbol = await self.request_containing_symbol( def_path, def_line, def_col, strict=False, include_body=include_body ) - + return defining_symbol - + @property def _cache_path(self) -> Path: return Path(self.repository_root_path) / ".serena" / "cache" / "document_symbols_cache.pkl" - + def save_cache(self): if self._cache_has_changed: self.logger.log(f"Saving updated document symbols cache to {self._cache_path}", logging.INFO) @@ -1505,7 +1525,7 @@ class LanguageServer: f"Failed to save document symbols cache to {self._cache_path}: {e}. " "Note: this may have resulted in a corrupted cache file.", logging.ERROR ) - + def load_cache(self): if not self._cache_path.exists(): return @@ -1517,11 +1537,38 @@ class LanguageServer: # cache often becomes corrupt, so just skip loading it self.logger.log( f"Failed to load document symbols cache from {self._cache_path}: {e}. Possible cause: the cache file is corrupted. " - "Check for any errors related to saving the cache in the logs.", + "Check for any errors related to saving the cache in the logs.", logging.ERROR ) + async def request_workspace_symbol(self, query: str) -> Union[List[multilspy_types.UnifiedSymbolInformation], None]: + """ + Raise a [workspace/symbol](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_symbol) request to the Language Server + to find symbols across the whole workspace. Wait for the response and return the result. + + :param query: The query string to filter symbols by + + :return Union[List[multilspy_types.UnifiedSymbolInformation], None]: A list of matching symbols + """ + response = await self.server.send.workspace_symbol({"query": query}) + if response is None: + return None + + assert isinstance(response, list) + + ret: List[multilspy_types.UnifiedSymbolInformation] = [] + for item in response: + assert isinstance(item, dict) + + assert LSPConstants.NAME in item + assert LSPConstants.KIND in item + assert LSPConstants.LOCATION in item + + ret.append(multilspy_types.UnifiedSymbolInformation(**item)) + + return ret + @ensure_all_methods_implemented(LanguageServer) class SyncLanguageServer: """ @@ -1529,23 +1576,25 @@ class SyncLanguageServer: It is used to communicate with Language Servers of different programming languages. """ - def __init__(self, language_server: LanguageServer) -> None: + def __init__(self, language_server: LanguageServer, timeout: Optional[int] = None): self.language_server = language_server self.loop = None self.loop_thread = None - + self.timeout = timeout + self._server_context = None @classmethod def create( cls, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str, add_gitignore_content_to_config=True, + timeout: Optional[int] = None ) -> "SyncLanguageServer": """ Creates a language specific LanguageServer instance based on the given configuration, and appropriate settings for the programming language. If language is Java, then ensure that jdk-17.0.6 or higher is installed, `java` is in PATH, and JAVA_HOME is set to the installation directory. - :param repository_root_path: The root path of the repository. + :param repository_root_path: The root path of the repository (must be absolute). :param config: The Multilspy configuration. :param logger: The logger to use. :param add_gitignore_content_to_config: whether to add the content of the .gitignore file (if any found) to the config, so that @@ -1553,7 +1602,7 @@ class SyncLanguageServer: :return SyncLanguageServer: A language specific LanguageServer instance. """ - return SyncLanguageServer(LanguageServer.create(config, logger, repository_root_path, add_gitignore_content_to_config=add_gitignore_content_to_config)) + return SyncLanguageServer(LanguageServer.create(config, logger, repository_root_path, add_gitignore_content_to_config=add_gitignore_content_to_config), timeout=timeout) @contextmanager def open_file(self, relative_file_path: str) -> Iterator[LSPFileBuffer]: @@ -1620,7 +1669,7 @@ class SyncLanguageServer: """ result = asyncio.run_coroutine_threadsafe( self.language_server.request_definition(file_path, line, column), self.loop - ).result() + ).result(timeout=self.timeout) return result def request_references(self, file_path: str, line: int, column: int) -> List[multilspy_types.Location]: @@ -1636,15 +1685,15 @@ class SyncLanguageServer: """ result = asyncio.run_coroutine_threadsafe( self.language_server.request_references(file_path, line, column), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_references_with_content( self, relative_file_path: str, line: int, column: int, context_lines_before: int = 0, context_lines_after: int = 0 ) -> List[MatchedConsecutiveLines]: """ Like request_references, but returns the content of the lines containing the references, not just the locations. - + :param relative_file_path: The relative path of the file that has the symbol for which references should be looked up :param line: The line number of the symbol :param column: The column number of the symbol @@ -1657,7 +1706,7 @@ class SyncLanguageServer: self.language_server.request_references_with_content(relative_file_path, line, column, context_lines_before, context_lines_after), self.loop ).result() return result - + def request_completions( self, relative_file_path: str, line: int, column: int, allow_incomplete: bool = False ) -> List[multilspy_types.CompletionItem]: @@ -1674,7 +1723,7 @@ class SyncLanguageServer: result = asyncio.run_coroutine_threadsafe( self.language_server.request_completions(relative_file_path, line, column, allow_incomplete), self.loop, - ).result() + ).result(timeout=self.timeout) return result def request_document_symbols(self, relative_file_path: str, include_body: bool = False) -> Tuple[List[multilspy_types.UnifiedSymbolInformation], List[multilspy_types.UnifiedSymbolInformation]]: @@ -1690,17 +1739,17 @@ class SyncLanguageServer: self.language_server.request_document_symbols(relative_file_path, include_body), self.loop ).result() return result - + def request_full_symbol_tree(self, within_relative_path: str | None = None, include_body: bool = False) -> List[multilspy_types.UnifiedSymbolInformation]: """ Will go through all files in the project and build a tree of symbols. Note: this may be slow the first time it is called. - + For each file, a symbol of kind Module (3) will be created. For directories, a symbol of kind Package (4) will be created. All symbols will have a children attribute, thereby representing the tree structure of all symbols in the project that are within the repository. Will ignore directories starting with '.', language-specific defaults and user-configured directories (e.g. from .gitignore). - + :param within_relative_path: pass a relative path to only consider symbols within this path. If a file is passed, only the symbols within this file will be considered. If a directory is passed, all files within this directory will be considered. @@ -1711,32 +1760,32 @@ class SyncLanguageServer: """ result = asyncio.run_coroutine_threadsafe( self.language_server.request_full_symbol_tree(within_relative_path, include_body), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_dir_overview(self, relative_dir_path: str) -> dict[str, list[tuple[str, multilspy_types.SymbolKind, int, int]]]: """ - An overview of the given directory. - - Maps relative paths of all contained files to info about top-level symbols in the file + An overview of the given directory. + + Maps relative paths of all contained files to info about top-level symbols in the file (name, kind, line, column). """ assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_dir_overview(relative_dir_path), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_document_overview(self, relative_file_path: str) -> list[tuple[str, multilspy_types.SymbolKind, int, int]]: """ - An overview of the given file. - + An overview of the given file. + Returns the list of tuples (name, kind, line, column) of all top-level symbols in the file. """ assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_document_overview(relative_file_path), self.loop - ).result() + ).result(timeout=self.timeout) return result def request_hover(self, relative_file_path: str, line: int, column: int) -> Union[multilspy_types.Hover, None]: @@ -1752,30 +1801,44 @@ class SyncLanguageServer: """ result = asyncio.run_coroutine_threadsafe( self.language_server.request_hover(relative_file_path, line, column), self.loop - ).result() + ).result(timeout=self.timeout) return result - + + def request_workspace_symbol(self, query: str) -> Union[List[multilspy_types.UnifiedSymbolInformation], None]: + """ + Raise a [workspace/symbol](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_symbol) request to the Language Server + to find symbols across the whole workspace. Wait for the response and return the result. + + :param query: The query string to filter symbols by + + :return Union[List[multilspy_types.UnifiedSymbolInformation], None]: A list of matching symbols + """ + result = asyncio.run_coroutine_threadsafe( + self.language_server.request_workspace_symbol(query), self.loop + ).result(timeout=self.timeout) + return result + # ----------------------------- FROM HERE ON MODIFICATIONS BY MISCHA -------------------- - + def retrieve_symbol_body(self, symbol: multilspy_types.UnifiedSymbolInformation) -> str: """ Load the body of the given symbol. If the body is already contained in the symbol, just return it. - + :param symbol: The symbol to retrieve the body of. :return: The body of the symbol. """ return self.language_server.retrieve_symbol_body(symbol) - + def request_parsed_files(self) -> list[str]: - """This is slow, as it finds all files by finding all symbols. - + """This is slow, as it finds all files by finding all symbols. + This seems to be the only way, the LSP does not provide any endpoints for listing project files.""" assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_parsed_files(), self.loop ).result() return result - + def request_referencing_symbols( self, relative_file_path: str, line: int, column: int, include_imports: bool = True, include_self: bool = False, @@ -1793,7 +1856,7 @@ class SyncLanguageServer: :param include_imports: whether to also include imports as references. Unfortunately, the LSP does not have an import type, so the references corresponding to imports will not be easily distinguishable from definitions. - :param include_self: whether to include the references that is the "input symbol" itself. + :param include_self: whether to include the references that is the "input symbol" itself. Only has an effect if the relative_file_path, line and column point to a symbol, for example a definition. :param include_body: whether to include the body of the symbols in the result. :param include_file_symbols: whether to include references that are file symbols. This @@ -1803,20 +1866,20 @@ class SyncLanguageServer: assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_referencing_symbols( - relative_file_path, - line, - column, - include_imports=include_imports, + relative_file_path, + line, + column, + include_imports=include_imports, include_self=include_self, include_body=include_body, include_file_symbols=include_file_symbols, - ), + ), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_containing_symbol( - self, relative_file_path: str, line: int, + self, relative_file_path: str, line: int, column: Optional[int] = None, strict: bool = False, include_body: bool = False, ) -> multilspy_types.UnifiedSymbolInformation | None: @@ -1824,24 +1887,24 @@ class SyncLanguageServer: Finds the first symbol containing the position for the given file. For Python, container symbols are considered to be those with kinds corresponding to functions, methods, or classes (typically: Function (12), Method (6), Class (5)). - + The method operates as follows: - Request the document symbols for the file. - Filter symbols to those that start at or before the given line. - From these, first look for symbols whose range contains the (line, column). - If one or more symbols contain the position, return the one with the greatest starting position (i.e. the innermost container). - - If none (strictly) contain the position, return the symbol with the greatest starting position + - If none (strictly) contain the position, return the symbol with the greatest starting position among those above the given line. - If no container candidates are found, return None. - + :param relative_file_path: The relative path to the Python file. :param line: The 0-indexed line number. :param column: The 0-indexed column (also called character). If not passed, the lookup will be based only on the line. :param strict: If True, the position must be strictly within the range of the symbol. Setting to true is useful for example for finding the parent of a symbol, as with strict=False, - and the line pointing to a symbol itself, the containing symbol will be the symbol itself + and the line pointing to a symbol itself, the containing symbol will be the symbol itself (and not the parent). :param include_body: whether to include the body of the symbol in the result. :return: The container symbol (if found) or None. @@ -1849,32 +1912,32 @@ class SyncLanguageServer: assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_containing_symbol(relative_file_path, line, column=column, strict=strict, include_body=include_body), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_container_of_symbol(self, symbol: multilspy_types.UnifiedSymbolInformation, include_body: bool = False) -> multilspy_types.UnifiedSymbolInformation | None: """ Finds the container of the given symbol if there is one. - + :param symbol: The symbol to find the container of. :param include_body: whether to include the body of the symbol in the result. """ assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_container_of_symbol(symbol, include_body=include_body), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def request_defining_symbol( - self, relative_file_path: str, line: int, column: int, + self, relative_file_path: str, line: int, column: int, include_body: bool = False, ) -> Optional[multilspy_types.UnifiedSymbolInformation]: """ Finds the symbol that defines the symbol at the given location. - + This method first finds the definition of the symbol at the given position, then retrieves the full symbol information for that definition. - + :param relative_file_path: The relative path to the file. :param line: The 0-indexed line number. :param column: The 0-indexed column number. @@ -1884,19 +1947,19 @@ class SyncLanguageServer: assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.request_defining_symbol(relative_file_path, line, column, include_body=include_body), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def retrieve_full_file_content(self, relative_file_path: str) -> str: """ Retrieve the full content of the given file. """ return self.language_server.retrieve_full_file_content(relative_file_path) - + def retrieve_content_around_line(self, relative_file_path: str, line: int, context_lines_before: int = 0, context_lines_after: int = 0) -> MatchedConsecutiveLines: """ Retrieve the content of the given file around the given line. - + :param relative_file_path: The relative path of the file to retrieve the content from :param line: The line number to retrieve the content around :param context_lines_before: The number of lines to retrieve before the given line @@ -1904,7 +1967,7 @@ class SyncLanguageServer: :return MatchedConsecutiveLines: A container with the desired lines. """ return self.language_server.retrieve_content_around_line(relative_file_path, line, context_lines_before, context_lines_after) - + def search_files_for_pattern( self, pattern: re.Pattern | str, @@ -1915,7 +1978,7 @@ class SyncLanguageServer: ) -> list[MatchedConsecutiveLines]: """ Search for a pattern across all files analyzed by the Language Server. - + :param pattern: Regular expression pattern to search for, either as a compiled Pattern or string :param context_lines_before: Number of lines of context to include before each match :param context_lines_after: Number of lines of context to include after each match @@ -1926,9 +1989,9 @@ class SyncLanguageServer: assert self.loop result = asyncio.run_coroutine_threadsafe( self.language_server.search_files_for_pattern(pattern, context_lines_before, context_lines_after, paths_include_glob, paths_exclude_glob), self.loop - ).result() + ).result(timeout=self.timeout) return result - + def start(self) -> "SyncLanguageServer": """ Starts the language server process and connects to it. Call shutdown when ready. @@ -1941,23 +2004,23 @@ class SyncLanguageServer: self._server_context = self.language_server.start_server() asyncio.run_coroutine_threadsafe(self._server_context.__aenter__(), loop=self.loop).result() return self - + def is_running(self) -> bool: """ Check if the language server is running. """ return self.loop is not None and self.loop_thread is not None and self.loop_thread.is_alive() - + def stop(self) -> None: """ Shuts down the language server process and cleans up resources. - + If the language server is not running, this method will log a warning and do nothing. """ if not self.is_running(): self.language_server.logger.log("Language server not running, skipping shutdown.", logging.INFO) return - + assert self.loop asyncio.run_coroutine_threadsafe(self._server_context.__aexit__(None, None, None), loop=self.loop).result() self.loop.call_soon_threadsafe(self.loop.stop) @@ -1965,13 +2028,13 @@ class SyncLanguageServer: self.loop = None self.loop_thread = None self.save_cache() - + def save_cache(self): """ Save the cache to a file. """ self.language_server.save_cache() - + def load_cache(self): """ Load the cache from a file. @@ -1984,7 +2047,7 @@ class SyncLanguageServer: in Python and node_modules in JS/TS should be ignored always. """ return self.language_server.should_always_ignore(dirname) - + def should_ignore_path(self, relative_path: str) -> bool: """ Whether the given path should be ignored. diff --git a/src/multilspy/language_servers/clangd_language_server/clangd_language_server.py b/src/multilspy/language_servers/clangd_language_server/clangd_language_server.py new file mode 100644 index 0000000..17253c4 --- /dev/null +++ b/src/multilspy/language_servers/clangd_language_server/clangd_language_server.py @@ -0,0 +1,180 @@ +""" +Provides C/C++ specific instantiation of the LanguageServer class. Contains various configurations and settings specific to C/C++. +""" + +import asyncio +import json +import logging +import os +import stat +import pathlib +from contextlib import asynccontextmanager +from typing import AsyncIterator + +from multilspy.multilspy_logger import MultilspyLogger +from multilspy.language_server import LanguageServer +from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo +from multilspy.lsp_protocol_handler.lsp_types import InitializeParams +from multilspy.multilspy_config import MultilspyConfig +from multilspy.multilspy_utils import FileUtils +from multilspy.multilspy_utils import PlatformUtils + + +class ClangdLanguageServer(LanguageServer): + """ + Provides C/C++ specific instantiation of the LanguageServer class. Contains various configurations and settings specific to C/C++. + As the project gets bigger in size, building index will take time. Try running clangd multiple times to ensure index is built properly. + Also make sure compile_commands.json is created at root of the source directory. Check clangd test case for example. + """ + + def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str): + """ + Creates a ClangdLanguageServer instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead. + """ + clangd_executable_path = self.setup_runtime_dependencies(logger, config) + super().__init__( + config, + logger, + repository_root_path, + ProcessLaunchInfo(cmd=clangd_executable_path, cwd=repository_root_path), + "cpp", + ) + self.server_ready = asyncio.Event() + + def setup_runtime_dependencies(self, logger: MultilspyLogger, config: MultilspyConfig) -> str: + """ + Setup runtime dependencies for ClangdLanguageServer. + """ + platform_id = PlatformUtils.get_platform_id() + + with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f: + d = json.load(f) + del d["_description"] + + assert platform_id.value in [ + "linux-x64" + ], "Only linux-x64 is supported for in multilspy at the moment" + + runtime_dependencies = d["runtimeDependencies"] + runtime_dependencies = [ + dependency for dependency in runtime_dependencies if dependency["platformId"] == platform_id.value + ] + assert len(runtime_dependencies) == 1 + dependency = runtime_dependencies[0] + + clangd_ls_dir = os.path.join(os.path.dirname(__file__), "static/clangd") + clangd_executable_path = os.path.join(clangd_ls_dir, "clangd_19.1.2", "bin", dependency["binaryName"]) + if not os.path.exists(clangd_ls_dir): + os.makedirs(clangd_ls_dir) + if dependency["archiveType"] == "zip": + FileUtils.download_and_extract_archive( + logger, dependency["url"], clangd_ls_dir, dependency["archiveType"] + ) + assert os.path.exists(clangd_executable_path) + os.chmod(clangd_executable_path, stat.S_IEXEC) + + return clangd_executable_path + + def _get_initialize_params(self, repository_absolute_path: str) -> InitializeParams: + """ + Returns the initialize params for the clangd Language Server. + """ + with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r") as f: + d = json.load(f) + + del d["_description"] + + d["processId"] = os.getpid() + assert d["rootPath"] == "$rootPath" + d["rootPath"] = repository_absolute_path + + assert d["rootUri"] == "$rootUri" + d["rootUri"] = pathlib.Path(repository_absolute_path).as_uri() + + assert d["workspaceFolders"][0]["uri"] == "$uri" + d["workspaceFolders"][0]["uri"] = pathlib.Path(repository_absolute_path).as_uri() + + assert d["workspaceFolders"][0]["name"] == "$name" + d["workspaceFolders"][0]["name"] = os.path.basename(repository_absolute_path) + + return d + + @asynccontextmanager + async def start_server(self) -> AsyncIterator["ClangdLanguageServer"]: + """ + Starts the Clangd Language Server, waits for the server to be ready and yields the LanguageServer instance. + + Usage: + ``` + async with lsp.start_server(): + # LanguageServer has been initialized and ready to serve requests + await lsp.request_definition(...) + await lsp.request_references(...) + # Shutdown the LanguageServer on exit from scope + # LanguageServer has been shutdown + """ + async def register_capability_handler(params): + assert "registrations" in params + for registration in params["registrations"]: + if registration["method"] == "workspace/executeCommand": + self.initialize_searcher_command_available.set() + self.resolve_main_method_available.set() + return + + async def lang_status_handler(params): + # TODO: Should we wait for + # server -> client: {'jsonrpc': '2.0', 'method': 'language/status', 'params': {'type': 'ProjectStatus', 'message': 'OK'}} + # Before proceeding? + if params["type"] == "ServiceReady" and params["message"] == "ServiceReady": + self.service_ready_event.set() + + async def execute_client_command_handler(params): + return [] + + async def do_nothing(params): + return + + async def check_experimental_status(params): + if params["quiescent"] == True: + self.server_ready.set() + + async def window_log_message(msg): + self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO) + + self.server.on_request("client/registerCapability", register_capability_handler) + self.server.on_notification("language/status", lang_status_handler) + self.server.on_notification("window/logMessage", window_log_message) + self.server.on_request("workspace/executeClientCommand", execute_client_command_handler) + self.server.on_notification("$/progress", do_nothing) + self.server.on_notification("textDocument/publishDiagnostics", do_nothing) + self.server.on_notification("language/actionableNotification", do_nothing) + self.server.on_notification("experimental/serverStatus", check_experimental_status) + + async with super().start_server(): + self.logger.log("Starting Clangd server process", logging.INFO) + await self.server.start() + initialize_params = self._get_initialize_params(self.repository_root_path) + + self.logger.log( + "Sending initialize request from LSP client to LSP server and awaiting response", + logging.INFO, + ) + init_response = await self.server.send.initialize(initialize_params) + assert init_response["capabilities"]["textDocumentSync"]["change"] == 2 + assert "completionProvider" in init_response["capabilities"] + assert init_response["capabilities"]["completionProvider"] == { + "triggerCharacters": ['.', '<', '>', ':', '"', '/', '*'], + "resolveProvider": False, + } + + self.server.notify.initialized({}) + + self.completions_available.set() + # set ready flag + self.server_ready.set() + await self.server_ready.wait() + + yield self + + await self.server.shutdown() + await self.server.stop() diff --git a/src/multilspy/language_servers/clangd_language_server/initialize_params.json b/src/multilspy/language_servers/clangd_language_server/initialize_params.json new file mode 100644 index 0000000..4330560 --- /dev/null +++ b/src/multilspy/language_servers/clangd_language_server/initialize_params.json @@ -0,0 +1,36 @@ +{ + "_description": "The parameters sent by the client when initializing the language server with the \"initialize\" request. More details at https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#initialize", + "processId": "os.getpid()", + "locale": "en", + "rootPath": "$rootPath", + "rootUri": "$rootUri", + "capabilities": { + "textDocument": { + "synchronization": { + "didSave": true, + "dynamicRegistration": true + }, + "completion": { + "dynamicRegistration": true, + "completionItem": { + "snippetSupport": true + } + }, + "definition": { + "dynamicRegistration": true + } + }, + "workspace": { + "workspaceFolders": true, + "didChangeConfiguration": { + "dynamicRegistration": true + } + } + }, + "workspaceFolders": [ + { + "uri": "$uri", + "name": "$name" + } + ] +} \ No newline at end of file diff --git a/src/multilspy/language_servers/clangd_language_server/runtime_dependencies.json b/src/multilspy/language_servers/clangd_language_server/runtime_dependencies.json new file mode 100644 index 0000000..ab30fff --- /dev/null +++ b/src/multilspy/language_servers/clangd_language_server/runtime_dependencies.json @@ -0,0 +1,13 @@ +{ + "_description": "Used to download the runtime dependencies for running Clangd.", + "runtimeDependencies": [ + { + "id": "Clangd", + "description": "Clangd for Linux (x64)", + "url": "https://github.com/clangd/clangd/releases/download/19.1.2/clangd-linux-19.1.2.zip", + "platformId": "linux-x64", + "archiveType": "zip", + "binaryName": "clangd" + } + ] +} diff --git a/src/multilspy/language_servers/dart_language_server/dart_language_server.py b/src/multilspy/language_servers/dart_language_server/dart_language_server.py new file mode 100644 index 0000000..b814306 --- /dev/null +++ b/src/multilspy/language_servers/dart_language_server/dart_language_server.py @@ -0,0 +1,146 @@ +from contextlib import asynccontextmanager +import logging +import os +import pathlib +import shutil +import stat +from typing import AsyncIterator +from multilspy.language_server import LanguageServer +from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo +import json +from multilspy.multilspy_utils import FileUtils, PlatformUtils + + +class DartLanguageServer(LanguageServer): + """ + Provides Dart specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Dart. + """ + + def __init__(self, config, logger, repository_root_path): + """ + Creates a DartServer instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead. + """ + + executable_path = self.setup_runtime_dependencies(logger) + super().__init__( + config, + logger, + repository_root_path, + ProcessLaunchInfo(cmd=executable_path, cwd=repository_root_path), + "dart", + ) + + def setup_runtime_dependencies(self, logger: "MultilspyLogger") -> str: + platform_id = PlatformUtils.get_platform_id() + + with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f: + d = json.load(f) + del d["_description"] + + runtime_dependencies = d["runtimeDependencies"] + runtime_dependencies = [ + dependency for dependency in runtime_dependencies if dependency["platformId"] == platform_id.value + ] + + assert len(runtime_dependencies) == 1 + dependency = runtime_dependencies[0] + + dart_ls_dir = os.path.join(os.path.dirname(__file__), "static", "dart-language-server") + dart_executable_path = os.path.join(dart_ls_dir, dependency["binaryName"]) + + if not os.path.exists(dart_ls_dir): + os.makedirs(dart_ls_dir) + FileUtils.download_and_extract_archive( + logger, dependency["url"], dart_ls_dir, dependency["archiveType"] + ) + + + assert os.path.exists(dart_executable_path) + os.chmod(dart_executable_path, stat.S_IEXEC) + + return f"{dart_executable_path} language-server --client-id multilspy.dart --client-version 1.2" + + + def _get_initialize_params(self, repository_absolute_path: str): + """ + Returns the initialize params for the Dart Language Server. + """ + with open( + os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r" + ) as f: + d = json.load(f) + + del d["_description"] + + d["processId"] = os.getpid() + assert d["rootPath"] == "$rootPath" + d["rootPath"] = repository_absolute_path + + assert d["rootUri"] == "$rootUri" + d["rootUri"] = pathlib.Path(repository_absolute_path).as_uri() + + assert d["workspaceFolders"][0]["uri"] == "$uri" + d["workspaceFolders"][0]["uri"] = pathlib.Path( + repository_absolute_path + ).as_uri() + + assert d["workspaceFolders"][0]["name"] == "$name" + d["workspaceFolders"][0]["name"] = os.path.basename(repository_absolute_path) + + return d + + @asynccontextmanager + async def start_server(self) -> AsyncIterator["DartLanguageServer"]: + """ + Start the language server and yield when the server is ready. + """ + + async def execute_client_command_handler(params): + return [] + + async def do_nothing(params): + return + + async def check_experimental_status(params): + pass + + async def window_log_message(msg): + self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO) + + self.server.on_request("client/registerCapability", do_nothing) + self.server.on_notification("language/status", do_nothing) + self.server.on_notification("window/logMessage", window_log_message) + self.server.on_request( + "workspace/executeClientCommand", execute_client_command_handler + ) + self.server.on_notification("$/progress", do_nothing) + self.server.on_notification("textDocument/publishDiagnostics", do_nothing) + self.server.on_notification("language/actionableNotification", do_nothing) + self.server.on_notification( + "experimental/serverStatus", check_experimental_status + ) + + async with super().start_server(): + self.logger.log( + "Starting dart-language-server server process", logging.INFO + ) + await self.server.start() + initialize_params = self._get_initialize_params(self.repository_root_path) + self.logger.log( + "Sending initialize request to dart-language-server", + logging.DEBUG, + ) + init_response = await self.server.send_request( + "initialize", initialize_params + ) + self.logger.log( + f"Received initialize response from dart-language-server: {init_response}", + logging.INFO, + ) + + self.server.notify.initialized({}) + + yield self + + await self.server.shutdown() + await self.server.stop() diff --git a/src/multilspy/language_servers/dart_language_server/initialize_params.json b/src/multilspy/language_servers/dart_language_server/initialize_params.json new file mode 100644 index 0000000..e90e408 --- /dev/null +++ b/src/multilspy/language_servers/dart_language_server/initialize_params.json @@ -0,0 +1,23 @@ +{ + "_description": "This file contains the initialization parameters for the Dart Language Server.", + "processId": "$processId", + "rootPath": "$rootPath", + "rootUri": "$rootUri", + "capabilities": {}, + "initializationOptions": { + "onlyAnalyzeProjectsWithOpenFiles": false, + "suggestFromUnimportedLibraries": true, + "closingLabels": false, + "outline": false, + "flutterOutline": false, + "allowOpenUri": false + }, + "trace": "verbose", + "workspaceFolders": [ + { + "uri": "$uri", + "name": "$name" + } + ] + +} \ No newline at end of file diff --git a/src/multilspy/language_servers/dart_language_server/runtime_dependencies.json b/src/multilspy/language_servers/dart_language_server/runtime_dependencies.json new file mode 100644 index 0000000..8010b94 --- /dev/null +++ b/src/multilspy/language_servers/dart_language_server/runtime_dependencies.json @@ -0,0 +1,13 @@ +{ + "_description": "Used to download the runtime dependencies for running Dart Language Server, downloaded from https://dart.dev/get-dart/archive", + "runtimeDependencies": [ + { + "id": "DartLanguageServer", + "description": "Dart Language Server for Linux (x64)", + "url": "https://storage.googleapis.com/dart-archive/channels/stable/release/3.7.1/sdk/dartsdk-linux-x64-release.zip", + "platformId": "linux-x64", + "archiveType": "zip", + "binaryName": "dart-sdk/bin/dart" + } + ] +} \ No newline at end of file diff --git a/src/multilspy/language_servers/eclipse_jdtls/runtime_dependencies.json b/src/multilspy/language_servers/eclipse_jdtls/runtime_dependencies.json index bc2cb74..c224c69 100644 --- a/src/multilspy/language_servers/eclipse_jdtls/runtime_dependencies.json +++ b/src/multilspy/language_servers/eclipse_jdtls/runtime_dependencies.json @@ -23,6 +23,16 @@ "jdtls_launcher_jar_path": "extension/server/plugins/org.eclipse.equinox.launcher_1.6.500.v20230717-2134.jar", "jdtls_readonly_config_path": "extension/server/config_mac_arm" }, + "osx-x64": { + "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@darwin-x64-1.23.0.vsix", + "archiveType": "zip", + "relative_extraction_path": "vscode-java", + "jre_home_path": "extension/jre/17.0.8.1-macosx-x86_64", + "jre_path": "extension/jre/17.0.8.1-macosx-x86_64/bin/java", + "lombok_jar_path": "extension/lombok/lombok-1.18.30.jar", + "jdtls_launcher_jar_path": "extension/server/plugins/org.eclipse.equinox.launcher_1.6.500.v20230717-2134.jar", + "jdtls_readonly_config_path": "extension/server/config_mac" + }, "linux-arm64": { "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@linux-arm64-1.23.0.vsix", "archiveType": "zip", @@ -59,4 +69,4 @@ "intellisense_members_path": "extension/dist/bundledModels/java_intellisense-members" } } -} \ No newline at end of file +} diff --git a/src/multilspy/language_servers/kotlin_language_server/initialize_params.json b/src/multilspy/language_servers/kotlin_language_server/initialize_params.json new file mode 100644 index 0000000..a7b2ec0 --- /dev/null +++ b/src/multilspy/language_servers/kotlin_language_server/initialize_params.json @@ -0,0 +1,521 @@ +{ + "_description": "The parameters sent by the client when initializing the language server with the \"initialize\" request. More details at https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#initialize", + "processId": "os.getpid()", + "clientInfo": { + "name": "Multilspy Kotlin Client", + "version": "1.0.0" + }, + "locale": "en", + "rootPath": "repository_absolute_path", + "rootUri": "pathlib.Path(repository_absolute_path).as_uri()", + "capabilities": { + "workspace": { + "applyEdit": true, + "workspaceEdit": { + "documentChanges": true, + "resourceOperations": [ + "create", + "rename", + "delete" + ], + "failureHandling": "textOnlyTransactional", + "normalizesLineEndings": true, + "changeAnnotationSupport": { + "groupsOnLabel": true + } + }, + "didChangeConfiguration": { + "dynamicRegistration": true + }, + "didChangeWatchedFiles": { + "dynamicRegistration": true, + "relativePatternSupport": true + }, + "symbol": { + "dynamicRegistration": true, + "symbolKind": { + "valueSet": [ + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8, + 9, + 10, + 11, + 12, + 13, + 14, + 15, + 16, + 17, + 18, + 19, + 20, + 21, + 22, + 23, + 24, + 25, + 26 + ] + }, + "tagSupport": { + "valueSet": [ + 1 + ] + }, + "resolveSupport": { + "properties": [ + "location.range" + ] + } + }, + "codeLens": { + "refreshSupport": true + }, + "executeCommand": { + "dynamicRegistration": true + }, + "configuration": true, + "workspaceFolders": true, + "semanticTokens": { + "refreshSupport": true + }, + "fileOperations": { + "dynamicRegistration": true, + "didCreate": true, + "didRename": true, + "didDelete": true, + "willCreate": true, + "willRename": true, + "willDelete": true + }, + "inlineValue": { + "refreshSupport": true + }, + "inlayHint": { + "refreshSupport": true + }, + "diagnostics": { + "refreshSupport": true + } + }, + "textDocument": { + "publishDiagnostics": { + "relatedInformation": true, + "versionSupport": false, + "tagSupport": { + "valueSet": [ + 1, + 2 + ] + }, + "codeDescriptionSupport": true, + "dataSupport": true + }, + "synchronization": { + "dynamicRegistration": true, + "willSave": true, + "willSaveWaitUntil": true, + "didSave": true + }, + "completion": { + "dynamicRegistration": true, + "contextSupport": true, + "completionItem": { + "snippetSupport": false, + "commitCharactersSupport": true, + "documentationFormat": [ + "markdown", + "plaintext" + ], + "deprecatedSupport": true, + "preselectSupport": true, + "tagSupport": { + "valueSet": [ + 1 + ] + }, + "insertReplaceSupport": false, + "resolveSupport": { + "properties": [ + "documentation", + "detail", + "additionalTextEdits" + ] + }, + "insertTextModeSupport": { + "valueSet": [ + 1, + 2 + ] + }, + "labelDetailsSupport": true + }, + "insertTextMode": 2, + "completionItemKind": { + "valueSet": [ + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8, + 9, + 10, + 11, + 12, + 13, + 14, + 15, + 16, + 17, + 18, + 19, + 20, + 21, + 22, + 23, + 24, + 25 + ] + }, + "completionList": { + "itemDefaults": [ + "commitCharacters", + "editRange", + "insertTextFormat", + "insertTextMode" + ] + } + }, + "hover": { + "dynamicRegistration": true, + "contentFormat": [ + "markdown", + "plaintext" + ] + }, + "signatureHelp": { + "dynamicRegistration": true, + "signatureInformation": { + "documentationFormat": [ + "markdown", + "plaintext" + ], + "parameterInformation": { + "labelOffsetSupport": true + }, + "activeParameterSupport": true + }, + "contextSupport": true + }, + "definition": { + "dynamicRegistration": true, + "linkSupport": true + }, + "references": { + "dynamicRegistration": true + }, + "documentHighlight": { + "dynamicRegistration": true + }, + "documentSymbol": { + "dynamicRegistration": true, + "symbolKind": { + "valueSet": [ + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8, + 9, + 10, + 11, + 12, + 13, + 14, + 15, + 16, + 17, + 18, + 19, + 20, + 21, + 22, + 23, + 24, + 25, + 26 + ] + }, + "hierarchicalDocumentSymbolSupport": true, + "tagSupport": { + "valueSet": [ + 1 + ] + }, + "labelSupport": true + }, + "codeAction": { + "dynamicRegistration": true, + "isPreferredSupport": true, + "disabledSupport": true, + "dataSupport": true, + "resolveSupport": { + "properties": [ + "edit" + ] + }, + "codeActionLiteralSupport": { + "codeActionKind": { + "valueSet": [ + "", + "quickfix", + "refactor", + "refactor.extract", + "refactor.inline", + "refactor.rewrite", + "source", + "source.organizeImports" + ] + } + }, + "honorsChangeAnnotations": false + }, + "codeLens": { + "dynamicRegistration": true + }, + "formatting": { + "dynamicRegistration": true + }, + "rangeFormatting": { + "dynamicRegistration": true + }, + "onTypeFormatting": { + "dynamicRegistration": true + }, + "rename": { + "dynamicRegistration": true, + "prepareSupport": true, + "prepareSupportDefaultBehavior": 1, + "honorsChangeAnnotations": true + }, + "documentLink": { + "dynamicRegistration": true, + "tooltipSupport": true + }, + "typeDefinition": { + "dynamicRegistration": true, + "linkSupport": true + }, + "implementation": { + "dynamicRegistration": true, + "linkSupport": true + }, + "colorProvider": { + "dynamicRegistration": true + }, + "foldingRange": { + "dynamicRegistration": true, + "rangeLimit": 5000, + "lineFoldingOnly": true, + "foldingRangeKind": { + "valueSet": [ + "comment", + "imports", + "region" + ] + }, + "foldingRange": { + "collapsedText": false + } + }, + "declaration": { + "dynamicRegistration": true, + "linkSupport": true + }, + "selectionRange": { + "dynamicRegistration": true + }, + "callHierarchy": { + "dynamicRegistration": true + }, + "semanticTokens": { + "dynamicRegistration": true, + "tokenTypes": [ + "namespace", + "type", + "class", + "enum", + "interface", + "struct", + "typeParameter", + "parameter", + "variable", + "property", + "enumMember", + "event", + "function", + "method", + "macro", + "keyword", + "modifier", + "comment", + "string", + "number", + "regexp", + "operator", + "decorator" + ], + "tokenModifiers": [ + "declaration", + "definition", + "readonly", + "static", + "deprecated", + "abstract", + "async", + "modification", + "documentation", + "defaultLibrary" + ], + "formats": [ + "relative" + ], + "requests": { + "range": true, + "full": { + "delta": true + } + }, + "multilineTokenSupport": false, + "overlappingTokenSupport": false, + "serverCancelSupport": true, + "augmentsSyntaxTokens": true + }, + "linkedEditingRange": { + "dynamicRegistration": true + }, + "typeHierarchy": { + "dynamicRegistration": true + }, + "inlineValue": { + "dynamicRegistration": true + }, + "inlayHint": { + "dynamicRegistration": true, + "resolveSupport": { + "properties": [ + "tooltip", + "textEdits", + "label.tooltip", + "label.location", + "label.command" + ] + } + }, + "diagnostic": { + "dynamicRegistration": true, + "relatedDocumentSupport": false + } + }, + "window": { + "showMessage": { + "messageActionItem": { + "additionalPropertiesSupport": true + } + }, + "showDocument": { + "support": true + }, + "workDoneProgress": true + }, + "general": { + "staleRequestSupport": { + "cancel": true, + "retryOnContentModified": [ + "textDocument/semanticTokens/full", + "textDocument/semanticTokens/range", + "textDocument/semanticTokens/full/delta" + ] + }, + "regularExpressions": { + "engine": "ECMAScript", + "version": "ES2020" + }, + "markdown": { + "parser": "marked", + "version": "1.1.0" + }, + "positionEncodings": [ + "utf-16" + ] + }, + "notebookDocument": { + "synchronization": { + "dynamicRegistration": true, + "executionSummarySupport": true + } + } + }, + "initializationOptions": { + "workspaceFolders": "[pathlib.Path(repository_absolute_path).as_uri()]", + "storagePath": null, + "codegen": { + "enabled": false + }, + "compiler": { + "jvm": { + "target": "default" + } + }, + "completion": { + "snippets": { + "enabled": true + } + }, + "diagnostics": { + "enabled": true, + "level": 4, + "debounceTime": 250 + }, + "scripts": { + "enabled": true, + "buildScriptsEnabled": true + }, + "indexing": { + "enabled": true + }, + "externalSources": { + "useKlsScheme": false, + "autoConvertToKotlin": false + }, + "inlayHints": { + "typeHints": false, + "parameterHints": false, + "chainedHints": false + }, + "formatting": { + "formatter": "ktfmt", + "ktfmt": { + "style": "google", + "indent": 4, + "maxWidth": 100, + "continuationIndent": 8, + "removeUnusedImports": true + } + } + }, + "trace": "verbose", + "workspaceFolders": "[\n {\n \"uri\": pathlib.Path(repository_absolute_path).as_uri(),\n \"name\": os.path.basename(repository_absolute_path),\n }\n ]" +} diff --git a/src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py b/src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py new file mode 100644 index 0000000..76bc1e8 --- /dev/null +++ b/src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py @@ -0,0 +1,234 @@ +""" +Provides Kotlin specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Kotlin. +""" + +import asyncio +import dataclasses +import json +import logging +import os +import stat +import pathlib +from contextlib import asynccontextmanager +from typing import AsyncIterator + +from multilspy.multilspy_logger import MultilspyLogger +from multilspy.language_server import LanguageServer +from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo +from multilspy.lsp_protocol_handler.lsp_types import InitializeParams +from multilspy.multilspy_config import MultilspyConfig +from multilspy.multilspy_utils import FileUtils +from multilspy.multilspy_utils import PlatformUtils + + +@dataclasses.dataclass +class KotlinRuntimeDependencyPaths: + """ + Stores the paths to the runtime dependencies of Kotlin Language Server + """ + java_path: str + java_home_path: str + kotlin_executable_path: str + + +class KotlinLanguageServer(LanguageServer): + """ + Provides Kotlin specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Kotlin. + """ + + def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str): + """ + Creates a Kotlin Language Server instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead. + """ + runtime_dependency_paths = self.setup_runtime_dependencies(logger, config) + self.runtime_dependency_paths = runtime_dependency_paths + + # Create command to execute the Kotlin Language Server script + cmd = f'"{self.runtime_dependency_paths.kotlin_executable_path}"' + + # Set environment variables including JAVA_HOME + proc_env = {"JAVA_HOME": self.runtime_dependency_paths.java_home_path} + + super().__init__( + config, + logger, + repository_root_path, + ProcessLaunchInfo(cmd=cmd, env=proc_env, cwd=repository_root_path), + "kotlin", + ) + + def setup_runtime_dependencies(self, logger: MultilspyLogger, config: MultilspyConfig) -> KotlinRuntimeDependencyPaths: + """ + Setup runtime dependencies for Kotlin Language Server. + """ + platform_id = PlatformUtils.get_platform_id() + + # Verify platform support + assert platform_id.value.startswith("win-") or platform_id.value.startswith("linux-") or platform_id.value.startswith("osx-"), "Only Windows, Linux and macOS platforms are supported for Kotlin in multilspy at the moment" + + # Load dependency information + with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f: + d = json.load(f) + del d["_description"] + + kotlin_dependency = d["runtimeDependency"] + java_dependency = d["java"][platform_id.value] + + # Setup paths for dependencies + static_dir = os.path.join(os.path.dirname(__file__), "static") + os.makedirs(static_dir, exist_ok=True) + + # Setup Java paths + java_dir = os.path.join(static_dir, "java") + os.makedirs(java_dir, exist_ok=True) + + java_home_path = os.path.join(java_dir, java_dependency["java_home_path"]) + java_path = os.path.join(java_dir, java_dependency["java_path"]) + + # Download and extract Java if not exists + if not os.path.exists(java_path): + logger.log(f"Downloading Java for {platform_id.value}...", logging.INFO) + FileUtils.download_and_extract_archive( + logger, java_dependency["url"], java_dir, java_dependency["archiveType"] + ) + # Make Java executable + if not platform_id.value.startswith("win-"): + os.chmod(java_path, 0o755) + + assert os.path.exists(java_path), f"Java executable not found at {java_path}" + + # Setup Kotlin Language Server paths + kotlin_ls_dir = os.path.join(static_dir, "server") + + # Get platform-specific executable script path + if platform_id.value.startswith("win-"): + kotlin_script = os.path.join(kotlin_ls_dir, "bin", "kotlin-language-server.bat") + else: + kotlin_script = os.path.join(kotlin_ls_dir, "bin", "kotlin-language-server") + + # Download and extract Kotlin Language Server if script doesn't exist + if not os.path.exists(kotlin_script): + logger.log("Downloading Kotlin Language Server...", logging.INFO) + FileUtils.download_and_extract_archive( + logger, kotlin_dependency["url"], static_dir, kotlin_dependency["archiveType"] + ) + + # Make script executable on Unix platforms + if os.path.exists(kotlin_script) and not platform_id.value.startswith("win-"): + os.chmod(kotlin_script, stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR | stat.S_IRGRP | stat.S_IXGRP | stat.S_IROTH | stat.S_IXOTH) + + # Use script file + if os.path.exists(kotlin_script): + kotlin_executable_path = kotlin_script + logger.log(f"Using Kotlin Language Server script at {kotlin_script}", logging.INFO) + else: + raise FileNotFoundError(f"Kotlin Language Server script not found at {kotlin_script}") + + return KotlinRuntimeDependencyPaths( + java_path=java_path, + java_home_path=java_home_path, + kotlin_executable_path=kotlin_executable_path + ) + + def _get_initialize_params(self, repository_absolute_path: str) -> InitializeParams: + """ + Returns the initialize params for the Kotlin Language Server. + """ + with open(str(pathlib.PurePath(os.path.dirname(__file__), "initialize_params.json")), "r") as f: + d: InitializeParams = json.load(f) + + del d["_description"] + + if not os.path.isabs(repository_absolute_path): + repository_absolute_path = os.path.abspath(repository_absolute_path) + + assert d["processId"] == "os.getpid()" + d["processId"] = os.getpid() + + assert d["rootPath"] == "repository_absolute_path" + d["rootPath"] = repository_absolute_path + + assert d["rootUri"] == "pathlib.Path(repository_absolute_path).as_uri()" + d["rootUri"] = pathlib.Path(repository_absolute_path).as_uri() + + assert d["initializationOptions"]["workspaceFolders"] == "[pathlib.Path(repository_absolute_path).as_uri()]" + d["initializationOptions"]["workspaceFolders"] = [pathlib.Path(repository_absolute_path).as_uri()] + + assert ( + d["workspaceFolders"] + == '[\n {\n "uri": pathlib.Path(repository_absolute_path).as_uri(),\n "name": os.path.basename(repository_absolute_path),\n }\n ]' + ) + d["workspaceFolders"] = [ + { + "uri": pathlib.Path(repository_absolute_path).as_uri(), + "name": os.path.basename(repository_absolute_path), + } + ] + + return d + + @asynccontextmanager + async def start_server(self) -> AsyncIterator["KotlinLanguageServer"]: + """ + Starts the Kotlin Language Server, waits for the server to be ready and yields the LanguageServer instance. + + Usage: + ``` + async with lsp.start_server(): + # LanguageServer has been initialized and ready to serve requests + await lsp.request_definition(...) + await lsp.request_references(...) + # Shutdown the LanguageServer on exit from scope + # LanguageServer has been shutdown + ``` + """ + async def execute_client_command_handler(params): + return [] + + async def do_nothing(params): + return + + async def window_log_message(msg): + self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO) + + self.server.on_request("client/registerCapability", do_nothing) + self.server.on_notification("language/status", do_nothing) + self.server.on_notification("window/logMessage", window_log_message) + self.server.on_request("workspace/executeClientCommand", execute_client_command_handler) + self.server.on_notification("$/progress", do_nothing) + self.server.on_notification("textDocument/publishDiagnostics", do_nothing) + self.server.on_notification("language/actionableNotification", do_nothing) + + async with super().start_server(): + self.logger.log("Starting Kotlin server process", logging.INFO) + await self.server.start() + initialize_params = self._get_initialize_params(self.repository_root_path) + + self.logger.log( + "Sending initialize request from LSP client to LSP server and awaiting response", + logging.INFO, + ) + init_response = await self.server.send.initialize(initialize_params) + + capabilities = init_response["capabilities"] + assert "textDocumentSync" in capabilities, "Server must support textDocumentSync" + assert "hoverProvider" in capabilities, "Server must support hover" + assert "completionProvider" in capabilities, "Server must support code completion" + assert "signatureHelpProvider" in capabilities, "Server must support signature help" + assert "definitionProvider" in capabilities, "Server must support go to definition" + assert "referencesProvider" in capabilities, "Server must support find references" + assert "documentSymbolProvider" in capabilities, "Server must support document symbols" + assert "workspaceSymbolProvider" in capabilities, "Server must support workspace symbols" + assert "semanticTokensProvider" in capabilities, "Server must support semantic tokens" + + self.server.notify.initialized({}) + self.completions_available.set() + + yield self + + try: + await self.server.shutdown() + except Exception as e: + self.logger.log(f"Error during Kotlin server shutdown: {str(e)}", logging.WARNING) + finally: + await self.server.stop() diff --git a/src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json b/src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json new file mode 100644 index 0000000..a7b64b9 --- /dev/null +++ b/src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json @@ -0,0 +1,41 @@ +{ + "_description": "Used to download the runtime dependencies for Kotlin Language Server from https://github.com/fwcd/kotlin-language-server", + "runtimeDependency": { + "id": "KotlinLsp", + "description": "Kotlin Language Server", + "url": "https://github.com/fwcd/kotlin-language-server/releases/download/1.3.13/server.zip", + "archiveType": "zip" + }, + "java": { + "win-x64": { + "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@win32-x64-1.23.0.vsix", + "archiveType": "zip", + "java_home_path": "extension/jre/17.0.8.1-win32-x86_64", + "java_path": "extension/jre/17.0.8.1-win32-x86_64/bin/java.exe" + }, + "linux-x64": { + "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@linux-x64-1.23.0.vsix", + "archiveType": "zip", + "java_home_path": "extension/jre/17.0.8.1-linux-x86_64", + "java_path": "extension/jre/17.0.8.1-linux-x86_64/bin/java" + }, + "linux-arm64": { + "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@linux-arm64-1.23.0.vsix", + "archiveType": "zip", + "java_home_path": "extension/jre/17.0.8.1-linux-aarch64", + "java_path": "extension/jre/17.0.8.1-linux-aarch64/bin/java" + }, + "osx-x64": { + "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@darwin-x64-1.23.0.vsix", + "archiveType": "zip", + "java_home_path": "extension/jre/17.0.8.1-macosx-x86_64", + "java_path": "extension/jre/17.0.8.1-macosx-x86_64/bin/java" + }, + "osx-arm64": { + "url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@darwin-arm64-1.23.0.vsix", + "archiveType": "zip", + "java_home_path": "extension/jre/17.0.8.1-macosx-aarch64", + "java_path": "extension/jre/17.0.8.1-macosx-aarch64/bin/java" + } + } +} diff --git a/src/multilspy/language_servers/typescript_language_server/typescript_language_server.py b/src/multilspy/language_servers/typescript_language_server/typescript_language_server.py index 85b1a18..2bd0666 100644 --- a/src/multilspy/language_servers/typescript_language_server/typescript_language_server.py +++ b/src/multilspy/language_servers/typescript_language_server/typescript_language_server.py @@ -32,6 +32,11 @@ else: return type('obj', (), {'pw_name': os.environ.get('USERNAME', 'unknown')})() +# Conditionally import pwd module (Unix-only) +if not PlatformUtils.get_platform_id().value.startswith("win"): + import pwd + + class TypeScriptLanguageServer(LanguageServer): """ Provides TypeScript specific instantiation of the LanguageServer class. Contains various configurations and settings specific to TypeScript. @@ -50,7 +55,7 @@ class TypeScriptLanguageServer(LanguageServer): "typescript", ) self.server_ready = asyncio.Event() - + @override def should_always_ignore(self, dirname: str) -> bool: return super().should_always_ignore(dirname) or dirname in [ @@ -95,22 +100,29 @@ class TypeScriptLanguageServer(LanguageServer): if not os.path.exists(tsserver_ls_dir): os.makedirs(tsserver_ls_dir, exist_ok=True) for dependency in runtime_dependencies: - # Handle platform-specific user settings - subprocess_kwargs = { - 'shell': True, - 'check': True, - 'cwd': tsserver_ls_dir, - 'stdout': subprocess.DEVNULL, - 'stderr': subprocess.DEVNULL - } - - # Only add user parameter on Unix-like systems - if os.name != 'nt': # Not Windows + # Windows doesn't support the 'user' parameter and doesn't have pwd module + if PlatformUtils.get_platform_id().value.startswith("win"): + subprocess.run( + dependency["command"], + shell=True, + check=True, + cwd=tsserver_ls_dir, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL + ) + else: + # On Unix-like systems, run as non-root user user = pwd.getpwuid(os.getuid()).pw_name - subprocess_kwargs['user'] = user - - subprocess.run(dependency["command"], **subprocess_kwargs) - + subprocess.run( + dependency["command"], + shell=True, + check=True, + user=user, + cwd=tsserver_ls_dir, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL + ) + tsserver_executable_path = os.path.join(tsserver_ls_dir, "node_modules", ".bin", "typescript-language-server") assert os.path.exists(tsserver_executable_path), "typescript-language-server executable not found. Please install typescript-language-server and try again." @@ -209,4 +221,4 @@ class TypeScriptLanguageServer(LanguageServer): yield self await self.server.shutdown() - await self.server.stop() \ No newline at end of file + await self.server.stop() diff --git a/src/multilspy/lsp_protocol_handler/lsp_constants.py b/src/multilspy/lsp_protocol_handler/lsp_constants.py index 6026802..149af38 100644 --- a/src/multilspy/lsp_protocol_handler/lsp_constants.py +++ b/src/multilspy/lsp_protocol_handler/lsp_constants.py @@ -57,3 +57,6 @@ class LSPConstants: # key used to represent children in document symbols CHILDREN = "children" + + # key used to represent the location in symbols + LOCATION = "location" diff --git a/src/multilspy/lsp_protocol_handler/server.py b/src/multilspy/lsp_protocol_handler/server.py index cc87c33..b9dd0de 100644 --- a/src/multilspy/lsp_protocol_handler/server.py +++ b/src/multilspy/lsp_protocol_handler/server.py @@ -33,6 +33,7 @@ import dataclasses import json import logging import os +import psutil from typing import Any, Dict, List, Optional, Union from .lsp_requests import LspNotification, LspRequest @@ -173,9 +174,18 @@ class LanguageServerHandler: the asynchronous tasks created by the handler. task_counter: An integer that represents the next available task id for the handler. loop: An asyncio.AbstractEventLoop object that represents the event loop used by the handler. + start_independent_lsp_process: An optional boolean flag that indicates whether to start the + language server process in an independent process group. Default is `True`. Setting it to + `False` means that the language server process will be in the same process group as the + the current process, and any SIGINT and SIGTERM signals will be sent to both processes. """ - def __init__(self, process_launch_info: ProcessLaunchInfo, logger=None) -> None: + def __init__( + self, + process_launch_info: ProcessLaunchInfo, + logger=None, + start_independent_lsp_process=True, + ) -> None: """ Params: cmd: A string that represents the command to launch the language server process. @@ -197,6 +207,7 @@ class LanguageServerHandler: self.tasks = {} self.task_counter = 0 self.loop = None + self.start_independent_lsp_process = start_independent_lsp_process async def start(self) -> None: """ @@ -214,6 +225,7 @@ class LanguageServerHandler: stderr=asyncio.subprocess.PIPE, env=child_proc_env, cwd=self.process_launch_info.cwd, + start_new_session=self.start_independent_lsp_process, ) # Check if process terminated immediately @@ -234,24 +246,111 @@ class LanguageServerHandler: """ Sends the terminate signal to the language server process and waits for it to exit, with a timeout, killing it if necessary """ - for task in self.tasks.values(): - task.cancel() - - self.tasks = {} - + # First cancel all tasks + await self._cancel_pending_tasks() + process = self.process self.process = None + + if not process: + return + + # Clean up the process + await self._cleanup_process(process) - if process: - # TODO: Ideally, we should terminate the process here, - # However, there's an issue with asyncio terminating processes documented at - # https://bugs.python.org/issue35539 and https://bugs.python.org/issue41320 - # process.terminate() - wait_for_end = process.wait() + async def _cancel_pending_tasks(self): + """Cancel all pending tasks and wait for them to complete or timeout.""" + pending_tasks = [] + for task in self.tasks.values(): + if not task.done(): + task.cancel() + pending_tasks.append(task) + + if pending_tasks: try: - await asyncio.wait_for(wait_for_end, timeout=60) - except asyncio.TimeoutError: - process.kill() + await asyncio.wait_for(asyncio.gather(*pending_tasks, return_exceptions=True), timeout=5.0) + except (asyncio.TimeoutError, Exception): + pass + + self.tasks = {} + + async def _cleanup_process(self, process): + """Clean up a process: close stdin, terminate/kill process, close stdout/stderr.""" + # Close stdin first to prevent deadlocks + # See: https://bugs.python.org/issue35539 + self._safely_close_pipe(process.stdin) + + # Terminate/kill the process if it's still running + if process.returncode is None: + await self._terminate_or_kill_process(process) + + # Close stdout and stderr pipes after process has exited + # This is essential to prevent "I/O operation on closed pipe" errors and + # "Event loop is closed" errors during garbage collection + # See: https://bugs.python.org/issue41320 and https://github.com/python/cpython/issues/88050 + self._safely_close_pipe(process.stdout) + self._safely_close_pipe(process.stderr) + + # Small delay to ensure OS has released file handles + await asyncio.sleep(0.5) + + def _safely_close_pipe(self, pipe): + """Safely close a pipe, ignoring any exceptions.""" + if pipe: + try: + pipe.close() + except Exception: + pass + + async def _terminate_or_kill_process(self, process): + """Try to terminate the process gracefully, then forcefully if necessary.""" + # First try to terminate the process tree gracefully + self._signal_process_tree(process, terminate=True) + + # Wait for the process to exit (with timeout) + try: + await asyncio.wait_for(process.wait(), timeout=10) + except (asyncio.TimeoutError, Exception): + # If termination failed, forcefully kill the process tree + self._signal_process_tree(process, terminate=False) + try: + # Give it one more chance to exit + await asyncio.wait_for(process.wait(), timeout=2) + except Exception: + pass + + def _signal_process_tree(self, process, terminate=True): + """Send signal (terminate or kill) to the process and all its children.""" + signal_method = "terminate" if terminate else "kill" + + # Try to get the parent process + parent = None + try: + parent = psutil.Process(process.pid) + except (psutil.NoSuchProcess, psutil.AccessDenied, Exception): + pass + + # If we have the parent process and it's running, signal the entire tree + if parent and parent.is_running(): + # Signal children first + for child in parent.children(recursive=True): + try: + getattr(child, signal_method)() + except (psutil.NoSuchProcess, psutil.AccessDenied, Exception): + pass + + # Then signal the parent + try: + getattr(parent, signal_method)() + except (psutil.NoSuchProcess, psutil.AccessDenied, Exception): + pass + else: + # Fall back to direct process signaling + try: + getattr(process, signal_method)() + except Exception: + pass + async def shutdown(self) -> None: """ @@ -310,7 +409,7 @@ class LanguageServerHandler: line = await self.process.stderr.readline() if not line: continue - self._log("LSP stderr: " + line.decode(ENCODING)) + self._log("LSP stderr: " + line.decode(ENCODING, errors='replace')) except (BrokenPipeError, ConnectionResetError, StopLoopException): pass diff --git a/src/multilspy/multilspy_config.py b/src/multilspy/multilspy_config.py index a93f122..bbc8d3a 100644 --- a/src/multilspy/multilspy_config.py +++ b/src/multilspy/multilspy_config.py @@ -30,10 +30,13 @@ class Language(str, Enum): PYTHON = "python" RUST = "rust" JAVA = "java" + KOTLIN = "kotlin" TYPESCRIPT = "typescript" JAVASCRIPT = "javascript" GO = "go" RUBY = "ruby" + DART = "dart" + CPP = "cpp" def __str__(self) -> str: return self.value @@ -67,6 +70,7 @@ class MultilspyConfig: """ code_language: Language trace_lsp_communication: bool = False + start_independent_lsp_process: bool = True ignored_paths: list[str] = field(default_factory=list) """Paths, dirs or glob-like patterns. The matching will follow the same logic as for .gitignore entries""" gitignore_file_content: str | None = None diff --git a/src/multilspy/multilspy_types.py b/src/multilspy/multilspy_types.py index 982959a..4a0f97e 100644 --- a/src/multilspy/multilspy_types.py +++ b/src/multilspy/multilspy_types.py @@ -82,7 +82,7 @@ class Location(TypedDict): uri: DocumentUri range: Range absolutePath: str - relativePath: str + relativePath: Union[str, None] class CompletionItemKind(IntEnum): """The kind of a completion entry.""" diff --git a/src/multilspy/multilspy_utils.py b/src/multilspy/multilspy_utils.py index 6e42aa5..bdda697 100644 --- a/src/multilspy/multilspy_utils.py +++ b/src/multilspy/multilspy_utils.py @@ -5,7 +5,7 @@ This file contains various utility functions like I/O operations, handling paths import gzip import logging import os -from typing import Tuple +from typing import Tuple, Union import requests import shutil import uuid @@ -94,6 +94,16 @@ class PathUtils: """Check if a pattern contains glob-specific characters.""" return any(c in pattern for c in '*?[]!') + @staticmethod + def get_relative_path(path: str, base_path: str) -> Union[str, None]: + """ + Gets relative path if it's possible (paths should be on the same drive), + returns `None` otherwise. + """ + if PurePath(path).drive == PurePath(base_path).drive: + return str(PurePath(os.path.relpath(path, base_path))) + return None + class FileUtils: """ Utility functions for file operations. From 956d5b0c6f53911b219e9fd71994bf4f52b5c651 Mon Sep 17 00:00:00 2001 From: Dominik Jain Date: Tue, 8 Apr 2025 20:10:34 +0200 Subject: [PATCH 02/14] Updated multilspy sync commit identifiers (push) --- src/multilspy/.syncCommitId.this | 1 + 1 file changed, 1 insertion(+) create mode 100644 src/multilspy/.syncCommitId.this diff --git a/src/multilspy/.syncCommitId.this b/src/multilspy/.syncCommitId.this new file mode 100644 index 0000000..1626aec --- /dev/null +++ b/src/multilspy/.syncCommitId.this @@ -0,0 +1 @@ +90c42923c8299348634a3ebf7a23d9411d55ad65 \ No newline at end of file From 84c9b6959435894b83158b50f6274c9f60cc33a5 Mon Sep 17 00:00:00 2001 From: Dominik Jain Date: Tue, 8 Apr 2025 20:21:37 +0200 Subject: [PATCH 03/14] List new languages supported via multilspy update (Kotlin, Dart) --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 0172472..595189d 100644 --- a/README.md +++ b/README.md @@ -56,6 +56,8 @@ With Serena, we provide * Go (untested) * C# (untested) * Rust (untested) + * Kotlin (untested) + * Dart (untested) These languages are supported by the language server library [multilspy](https://github.com/microsoft/multilspy), which Serena uses under the hood. But we did not explicitly test whether the support for these languages actually works. From 2fa6a95aac1f051ae6c187e44c3b3b7aa159b884 Mon Sep 17 00:00:00 2001 From: Dominik Jain Date: Tue, 8 Apr 2025 20:29:01 +0200 Subject: [PATCH 04/14] Add psutil dependency (now required by multilspy) --- pyproject.toml | 1 + uv.lock | 17 +++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/pyproject.toml b/pyproject.toml index edced72..68c4058 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -31,6 +31,7 @@ dependencies = [ "jinja2>=3.1.6", "dotenv>=0.9.9", "pathspec>=0.12.1", + "psutil>=7.0.0", ] [project.scripts] diff --git a/uv.lock b/uv.lock index 10c6be6..8c410be 100644 --- a/uv.lock +++ b/uv.lock @@ -687,6 +687,21 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e4/ea/d836f008d33151c7a1f62caf3d8dd782e4d15f6a43897f64480c2b8de2ad/prompt_toolkit-3.0.50-py3-none-any.whl", hash = "sha256:9b6427eb19e479d98acff65196a307c555eb567989e6d88ebbb1b509d9779198", size = 387816 }, ] +[[package]] +name = "psutil" +version = "7.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/2a/80/336820c1ad9286a4ded7e845b2eccfcb27851ab8ac6abece774a6ff4d3de/psutil-7.0.0.tar.gz", hash = "sha256:7be9c3eba38beccb6495ea33afd982a44074b78f28c434a1f51cc07fd315c456", size = 497003 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ed/e6/2d26234410f8b8abdbf891c9da62bee396583f713fb9f3325a4760875d22/psutil-7.0.0-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:101d71dc322e3cffd7cea0650b09b3d08b8e7c4109dd6809fe452dfd00e58b25", size = 238051 }, + { url = "https://files.pythonhosted.org/packages/04/8b/30f930733afe425e3cbfc0e1468a30a18942350c1a8816acfade80c005c4/psutil-7.0.0-cp36-abi3-macosx_11_0_arm64.whl", hash = "sha256:39db632f6bb862eeccf56660871433e111b6ea58f2caea825571951d4b6aa3da", size = 239535 }, + { url = "https://files.pythonhosted.org/packages/2a/ed/d362e84620dd22876b55389248e522338ed1bf134a5edd3b8231d7207f6d/psutil-7.0.0-cp36-abi3-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:1fcee592b4c6f146991ca55919ea3d1f8926497a713ed7faaf8225e174581e91", size = 275004 }, + { url = "https://files.pythonhosted.org/packages/bf/b9/b0eb3f3cbcb734d930fdf839431606844a825b23eaf9a6ab371edac8162c/psutil-7.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4b1388a4f6875d7e2aff5c4ca1cc16c545ed41dd8bb596cefea80111db353a34", size = 277986 }, + { url = "https://files.pythonhosted.org/packages/eb/a2/709e0fe2f093556c17fbafda93ac032257242cabcc7ff3369e2cb76a97aa/psutil-7.0.0-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a5f098451abc2828f7dc6b58d44b532b22f2088f4999a937557b603ce72b1993", size = 279544 }, + { url = "https://files.pythonhosted.org/packages/50/e6/eecf58810b9d12e6427369784efe814a1eec0f492084ce8eb8f4d89d6d61/psutil-7.0.0-cp37-abi3-win32.whl", hash = "sha256:ba3fcef7523064a6c9da440fc4d6bd07da93ac726b5733c29027d7dc95b39d99", size = 241053 }, + { url = "https://files.pythonhosted.org/packages/50/1b/6921afe68c74868b4c9fa424dad3be35b095e16687989ebbb50ce4fceb7c/psutil-7.0.0-cp37-abi3-win_amd64.whl", hash = "sha256:4cf3d4eb1aa9b348dec30105c55cd9b7d4629285735a102beb4441e38db90553", size = 244885 }, +] + [[package]] name = "ptyprocess" version = "0.7.0" @@ -951,6 +966,7 @@ dependencies = [ { name = "mcp" }, { name = "overrides" }, { name = "pathspec" }, + { name = "psutil" }, { name = "pydantic" }, { name = "pyright" }, { name = "python-dotenv" }, @@ -1000,6 +1016,7 @@ requires-dist = [ { name = "overrides", specifier = ">=7.7.0,<8" }, { name = "pathspec", specifier = ">=0.12.1" }, { name = "poethepoet", marker = "extra == 'dev'", specifier = ">=0.20.0" }, + { name = "psutil", specifier = ">=7.0.0" }, { name = "pydantic", specifier = ">=2.10.6" }, { name = "pyright", specifier = ">=1.1.396,<2" }, { name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0.2" }, From f8ab7cd53cb238a8d94e32fd4ebe96fee949021c Mon Sep 17 00:00:00 2001 From: Dominik Jain Date: Wed, 9 Apr 2025 17:21:46 +0200 Subject: [PATCH 05/14] Handle filename matchers for new languages --- src/multilspy/multilspy_config.py | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/multilspy/multilspy_config.py b/src/multilspy/multilspy_config.py index bbc8d3a..bf53750 100644 --- a/src/multilspy/multilspy_config.py +++ b/src/multilspy/multilspy_config.py @@ -59,8 +59,14 @@ class Language(str, Enum): return FilenameMatcher("*.go") case self.RUBY: return FilenameMatcher("*.rb") + case self.CPP: + return FilenameMatcher("*.cpp", "*.h", "*.hpp", "*.c", "*.hxx", "*.cc", "*.cxx") + case self.KOTLIN: + return FilenameMatcher("*.kt", "*.kts") + case self.DART: + return FilenameMatcher("*.dart") case _: - raise ValueError + raise ValueError(f"Unhandled language: {self}") @dataclass From 0954a09171d9b8bbb792a1b4e4ca5be2563f5d17 Mon Sep 17 00:00:00 2001 From: Dominik Jain Date: Wed, 9 Apr 2025 17:22:14 +0200 Subject: [PATCH 06/14] Add 'C/C++' to list of untested languages --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index cff69a7..3aaed8b 100644 --- a/README.md +++ b/README.md @@ -58,6 +58,7 @@ With Serena, we provide * Rust (untested) * Kotlin (untested) * Dart (untested) + * C/C++ (untested) These languages are supported by the language server library [multilspy](https://github.com/microsoft/multilspy), which Serena uses under the hood. But we did not explicitly test whether the support for these languages actually works. From 1aaab5a7c2db387042f6b34de7ee75c20c9ec274 Mon Sep 17 00:00:00 2001 From: Dominik Jain Date: Wed, 9 Apr 2025 19:28:37 +0200 Subject: [PATCH 07/14] Change relative path handling: * Assign relativePath using PathUtils.get_relative_path, which returns None if no relative path exists (e.g. different drive); this aligns with how it was originally handled in multilspy * Adapt all places where we can now enounter None instead of a string --- src/multilspy/language_server.py | 16 +++------------- src/serena/symbol.py | 20 ++++++++++++++------ 2 files changed, 17 insertions(+), 19 deletions(-) diff --git a/src/multilspy/language_server.py b/src/multilspy/language_server.py index 091e7e6..83ec969 100644 --- a/src/multilspy/language_server.py +++ b/src/multilspy/language_server.py @@ -525,13 +525,7 @@ class LanguageServer: new_item: multilspy_types.Location = {} new_item.update(item) new_item["absolutePath"] = PathUtils.uri_to_path(new_item["uri"]) - try: - # TODO can we use PathUtils.get_relative_path? (same elsewhere!) - new_item["relativePath"] = str( - PurePath(os.path.relpath(new_item["absolutePath"], self.repository_root_path)) - ) - except: - new_item["relativePath"] = str(new_item["absolutePath"]) + new_item["relativePath"] = PathUtils.get_relative_path(new_item["absolutePath"], self.repository_root_path) ret.append(multilspy_types.Location(new_item)) elif ( LSPConstants.ORIGIN_SELECTION_RANGE in item @@ -542,12 +536,7 @@ class LanguageServer: new_item: multilspy_types.Location = {} new_item["uri"] = item[LSPConstants.TARGET_URI] new_item["absolutePath"] = PathUtils.uri_to_path(new_item["uri"]) - try: - new_item["relativePath"] = str( - PurePath(os.path.relpath(new_item["absolutePath"], self.repository_root_path)) - ) - except: - new_item["relativePath"] = str(new_item["absolutePath"]) + new_item["relativePath"] = PathUtils.get_relative_path(new_item["absolutePath"], self.repository_root_path) new_item["range"] = item[LSPConstants.TARGET_SELECTION_RANGE] ret.append(multilspy_types.Location(**new_item)) else: @@ -943,6 +932,7 @@ class LanguageServer: elif os.path.isfile(abs_item_path): _, root_nodes = await self.request_document_symbols(item_path, include_body=include_body) + # TODO: Not sure if this is actually still needed given recent changes to relative path handling def fix_relative_path(nodes: List[multilspy_types.UnifiedSymbolInformation]): for node in nodes: path = Path(node["location"]["relativePath"]) diff --git a/src/serena/symbol.py b/src/serena/symbol.py index 1819b1a..8228024 100644 --- a/src/serena/symbol.py +++ b/src/serena/symbol.py @@ -23,9 +23,9 @@ class SymbolLocation: Represents the (start) location of a symbol identifier """ - relative_path: str + relative_path: str | None """ - the relative path of the file containing the symbol + the relative path of the file containing the symbol; if None, the symbol is defined outside of the project's scope """ line: int | None """ @@ -39,13 +39,14 @@ class SymbolLocation: """ def __post_init__(self) -> None: - self.relative_path = self.relative_path.replace("/", os.path.sep) + if self.relative_path is not None: + self.relative_path = self.relative_path.replace("/", os.path.sep) def to_dict(self) -> dict[str, Any]: return asdict(self) def has_position_in_file(self) -> bool: - return self.line is not None and self.column is not None + return self.relative_path is not None and self.line is not None and self.column is not None class Symbol(ToStringMixin): @@ -71,7 +72,7 @@ class Symbol(ToStringMixin): return self.s["kind"] @property - def relative_path(self) -> str: + def relative_path(self) -> str | None: return self.s["location"]["relativePath"] @property @@ -239,6 +240,8 @@ class SymbolManager: return symbols def find_by_location(self, location: SymbolLocation) -> Symbol | None: + if location.relative_path is None: + return None symbol_dicts, roots = self.lang_server.request_document_symbols(location.relative_path, include_body=False) for symbol_dict in symbol_dicts: symbol = Symbol(symbol_dict) @@ -268,6 +271,7 @@ class SymbolManager: """ if not symbol_location.has_position_in_file(): raise ValueError("Symbol location does not contain a valid position in a file") + assert symbol_location.relative_path is not None assert symbol_location.line is not None assert symbol_location.column is not None symbol_dicts = self.lang_server.request_referencing_symbols( @@ -301,7 +305,8 @@ class SymbolManager: def _edited_symbol_location(self, location: SymbolLocation) -> Iterator[Symbol]: symbol = self.find_by_location(location) if symbol is None: - raise ValueError("Symbol not found") + raise ValueError("Symbol not found/has no defined location within a file") + assert location.relative_path is not None with self._edited_file(location.relative_path): yield symbol @@ -313,6 +318,7 @@ class SymbolManager: :param body: the new body """ with self._edited_symbol_location(location) as symbol: + assert location.relative_path is not None self.lang_server.delete_text_between_positions(location.relative_path, symbol.body_start_position, symbol.body_end_position) self.lang_server.insert_text_at_position( location.relative_path, symbol.body_start_position["line"], symbol.body_start_position["character"], body @@ -327,6 +333,7 @@ class SymbolManager: """ with self._edited_symbol_location(location) as symbol: pos = symbol.body_end_position + assert location.relative_path is not None self.lang_server.insert_text_at_position(location.relative_path, pos["line"], pos["character"], body) def insert_before(self, location: SymbolLocation, body: str) -> None: @@ -338,6 +345,7 @@ class SymbolManager: """ with self._edited_symbol_location(location) as symbol: pos = copy(symbol.body_start_position) + assert location.relative_path is not None self.lang_server.insert_text_at_position(location.relative_path, pos["line"], pos["character"], body) def insert_at_line(self, relative_path: str, line: int, content: str) -> None: From e12d8797c7ef8e707ce38e1503e02fd548c54335 Mon Sep 17 00:00:00 2001 From: Dominik Jain Date: Wed, 9 Apr 2025 19:32:03 +0200 Subject: [PATCH 08/14] Add docstrings on timeout --- src/multilspy/language_server.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/multilspy/language_server.py b/src/multilspy/language_server.py index 83ec969..aec6489 100644 --- a/src/multilspy/language_server.py +++ b/src/multilspy/language_server.py @@ -1589,6 +1589,10 @@ class SyncLanguageServer: """ def __init__(self, language_server: LanguageServer, timeout: Optional[int] = None): + """ + :param language_server: the async language server being wrapped + :param timeout: the timeout, in seconds, to use for requests to the language server. + """ self.language_server = language_server self.loop = None self.loop_thread = None @@ -1611,6 +1615,7 @@ class SyncLanguageServer: :param logger: The logger to use. :param add_gitignore_content_to_config: whether to add the content of the .gitignore file (if any found) to the config, so that the paths ignored there are also ignored by the language server + :param timeout: the timeout, in seconds, to use for requests; if None, use no timeout :return SyncLanguageServer: A language specific LanguageServer instance. """ From 3bc401a2b3c7200921bcfc417453a1a3eeade3ea Mon Sep 17 00:00:00 2001 From: Dominik Jain Date: Wed, 9 Apr 2025 19:39:05 +0200 Subject: [PATCH 09/14] Update change log --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 19605d7..aa4f552 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ Changes prior to the next official version change will appear here. * Language Servers: * Add further file extensions considered by the language servers for Python (.pyi), JavaScript (.jsx) and TypeScript (.tsx, .jsx) + * Updated multilspy, adding support for Kotlin, Dart and C/C++ and several improvements. # 2025-04-07 From 893fb10ebaa2912e2dfd6d370825daa93ca93dde Mon Sep 17 00:00:00 2001 From: Dominik Jain Date: Wed, 9 Apr 2025 19:42:53 +0200 Subject: [PATCH 10/14] Move config warning to the proper place --- README.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 3aaed8b..0f86614 100644 --- a/README.md +++ b/README.md @@ -69,13 +69,6 @@ implementation. Coming soon: Kotlin and Dart. -> ⚠️ **Note:** Serena is under active development, we are continuously adding features, improving stability and UX. -> As a result, configuration might change in a breaking way. If you have an invalid configuration, -> the MCP server (or the Serena Agent) may not start properly (investigate the MCP logs in the former case). -> Check the [changelog](CHANGELOG.md) -> and the configuration templates when updating Serena and update your configs accordingly. - - ## Table of Contents @@ -150,7 +143,13 @@ Serena can read, write and execute code, read logs and the terminal output. 5. If you want Serena to dynamically switch between projects, add the list of all project files created in the previous step to the `projects` list in `serena_config.yml`. -After this initial setup, continue with one of the sections below, depending on how you +> ⚠️ **Note:** Serena is under active development. We are continuously adding features, improving stability and the UX. +> As a result, configuration may change in a breaking manner. If you have an invalid configuration, +> the MCP server or Serena-based Agent may fail to start (investigate the MCP logs in the former case). +> Check the [changelog](CHANGELOG.md) +> and the configuration templates when updating Serena, adapting your configurations accordingly. + +After the initial setup, continue with one of the sections below, depending on how you want to use Serena. ### MCP Server (Claude Desktop) From e371c8bec2a962cd72b83774dcac0379ae33ac6d Mon Sep 17 00:00:00 2001 From: Dominik Jain Date: Wed, 9 Apr 2025 23:17:36 +0200 Subject: [PATCH 11/14] Improve clarity in the README, focussing on the important aspects --- README.md | 51 ++++++++++++++++++++++++++++----------------------- 1 file changed, 28 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index 6056b90..33b00d9 100644 --- a/README.md +++ b/README.md @@ -3,15 +3,9 @@

-* :rocket: Serena is a powerful, fully-featured **coding agent that works directly on your codebase**. -* :wrench: Serena **integrates with existing LLMs**, providing them with essential **semantic code retrieval and editing tools!** -* :free: Serena is **free to use**. No API keys or subscriptions required! - -Q: Can I have a state-of-the-art coding agent without paying (enormous) API costs -or constantly purchasing tokens? -A: Yes, you can! -By integrating Serena with your favourite (even free) LLM and thereby enabling it -to perform coding tasks directly on your codebase. +* :rocket: Serena is a powerful **coding agent toolkit** capable of turning an LLM into a fully-featured agent that works **directly on your codebase**. +* :wrench: Serena provides essential **semantic code retrieval and editing tools** that are akin to an IDE's capabilities, extracting code entities at the symbol level and exploiting relational structure. +* :free: Serena is **free & open-source**, enhancing the capabilities of LLMs you already have access to free of charge. ### Demonstration @@ -27,14 +21,17 @@ orchestrating tool use. Serena can be integrated with an LLM in several ways: * by using the **model context protocol (MCP)**. - Serena provides an MCP server which integrates with Claude (and [soon also ChatGPT](https://x.com/OpenAIDevs/status/1904957755829481737)). + Serena provides an MCP server which integrates with + * Claude Desktop, + * IDEs like VSCode, Cursor or IntelliJ, + * and [soon also ChatGPT](https://x.com/OpenAIDevs/status/1904957755829481737) * by using **Agno – the model-agnostic agent framework**. - Serena's Agno-based agent allows you to turn virtually any LLM into a coding agent, whether it's provided by Google, OpenAI or DeepSeek (with a paid API key) + Serena's Agno-based agent allows you to turn virtually any LLM into a coding agent, whether it's provided by Google, OpenAI or Anthropic (with a paid API key) or a free model provided by Ollama, Together or Anyscale. * by incorporating Serena's tools into an agent framework of your choice. Serena's tool implementation is decoupled from the framework-specific code and can thus easily be adapted to any agent framework. -### Programming Language Support & Semantic Analysis +### Programming Language Support & Semantic Analysis Capabilities Serena's semantic code analysis capabilities build on **language servers** using the widely implemented language server protocol (LSP). The LSP provides a set of versatile code querying @@ -80,8 +77,8 @@ Coming soon: Kotlin and Dart. -- [Is It Really Free to Use?](#is-it-really-free-to-use) - [What Can I Use Serena For?](#what-can-i-use-serena-for) +- [Free Coding Agents with Serena](#free-coding-agents-with-serena) - [Quick Start](#quick-start) * [Setup and Configuration](#setup-and-configuration) * [MCP Server (Claude Desktop)](#mcp-server-claude-desktop) @@ -113,9 +110,25 @@ Coming soon: Kotlin and Dart. -## Is It Really Free to Use? +## What Can I Use Serena For? -Yes! Even the free tier of Anthropic's Claude has support for MCP Servers, so you can use Serena with Claude for free. +You can use Serena for any coding tasks – whether it is focussed on analysis, planning, +designing new components or refactoring existing ones. +Since Serena's tools allow an LLM to close the cognitive perception-action loop, +agents based on Serena can autonomously carry out coding tasks from start to finish – +from the initial analysis to the implementation, testing and, finally, the version +control system commit. + +Serena can read, write and execute code, read logs and the terminal output. +While we do not necessarily encourage it, "vibe coding" is certainly possible, and if you +want to almost feel like "the code no longer exists", +you may find Serena even more adequate for vibing than an agent inside an IDE +(since you will have a separate GUI that really lets you forget). + +## Free Coding Agents with Serena + +Serena can +Even the free tier of Anthropic's Claude has support for MCP Servers, so you can use Serena with Claude for free. Presumably, the same will soon be possible with ChatGPT Desktop once support for MCP servers is added. Through Agno, you furthermore have the option to use Serena with a free/open-weights model. @@ -127,14 +140,6 @@ IDE-based subscriptions (such as Windsurf or Cursor) that forced us to keep purc The substantial API costs incurred by tools like Claude Code, Cline, Aider and other API-based tools are similarly unattractive. We thus built Serena with the prospect of being able to cancel most other subscriptions. -## What Can I Use Serena For? - -You can use Serena for any coding tasks – analyzing, planning, editing and so on. -Serena can read, write and execute code, read logs and the terminal output. -"Vibe coding" is possible, and if you want to almost feel like "the code no longer exists", - you may find Serena even more adequate for vibing than an agent inside an IDE -(since you will have a separate GUI that really lets you forget). - ## Quick Start ### Setup and Configuration From ca4215c8a44af4e518a08ea4b1352da337a1528c Mon Sep 17 00:00:00 2001 From: Dominik Jain Date: Wed, 9 Apr 2025 23:57:27 +0200 Subject: [PATCH 12/14] Fix CheckOnboardingPerformedTool: * Tool description was incompatible with project change * Returned result was not as useful as it could be (now added list of memories) --- CHANGELOG.md | 3 +++ src/serena/agent.py | 10 ++++------ 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aa4f552..e839066 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,9 @@ Changes prior to the next official version change will appear here. * bugfix in find_symbol tool (a bug fixed in LS) * merged the two overview tools (for dir and file) int a single one * one-click setup for Cline enabled + * Fix `CheckOnboardingPerformedTool`: + * Tool description was incompatible with project change + * Returned result was not as useful as it could be (now added list of memories) * Language Servers: * Add further file extensions considered by the language servers for Python (.pyi), JavaScript (.jsx) and TypeScript (.tsx, .jsx) diff --git a/src/serena/agent.py b/src/serena/agent.py index e13b1af..8082f6a 100644 --- a/src/serena/agent.py +++ b/src/serena/agent.py @@ -958,15 +958,13 @@ class InsertAtLineTool(Tool): class CheckOnboardingPerformedTool(Tool): """ - Checks whether the onboarding was already performed. + Checks whether project onboarding was already performed. """ def apply(self) -> str: """ - Check if onboarding was performed yet. - You should always call this tool in the beginning of the conversation, - before any question about code or the project is asked. - You will call this tool only once per conversation. + Checks whether project onboarding was already performed. + You should always call this tool before beginning to actually work on a project/after activating a project. """ list_memories_tool = self.agent.get_tool(ListMemoriesTool) memories = json.loads(list_memories_tool.apply()) @@ -976,7 +974,7 @@ class CheckOnboardingPerformedTool(Tool): + "You should perform onboarding by calling the `onboarding` tool before proceeding with the task." ) else: - return "Onboarding already performed, no need to perform it again." + return json.dumps({"result": "Onboarding already performed.", "available_memories": memories}) class OnboardingTool(Tool): From 5f7bac05db8a99104ae34c2a3b83be9d75e855f8 Mon Sep 17 00:00:00 2001 From: Michael Panchenko <35432522+MischaPanch@users.noreply.github.com> Date: Thu, 10 Apr 2025 00:51:20 +0200 Subject: [PATCH 13/14] Readme: fixed copy-paste errors --- README.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/README.md b/README.md index fc283dc..17fa637 100644 --- a/README.md +++ b/README.md @@ -63,8 +63,6 @@ With Serena, we provide Further languages can, in principle, easily be supported by providing a shallow adapter for a new language server implementation. -Coming soon: Kotlin and Dart. - ## Table of Contents @@ -123,7 +121,6 @@ you may find Serena even more adequate for vibing than an agent inside an IDE ## Free Coding Agents with Serena -Serena can Even the free tier of Anthropic's Claude has support for MCP Servers, so you can use Serena with Claude for free. Presumably, the same will soon be possible with ChatGPT Desktop once support for MCP servers is added. Through Agno, you furthermore have the option to use Serena with a free/open-weights model. From f1bd48dae95f566e103125ee7d739245af4534e1 Mon Sep 17 00:00:00 2001 From: Michael Panchenko <35432522+MischaPanch@users.noreply.github.com> Date: Thu, 10 Apr 2025 00:53:59 +0200 Subject: [PATCH 14/14] Readme: removed single project callout --- README.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/README.md b/README.md index 17fa637..55ee4b5 100644 --- a/README.md +++ b/README.md @@ -199,9 +199,6 @@ necessarily has to be started by the client in order for communication to take p In other words, you do not need to start the server yourself. The client application (e.g. Claude Desktop) takes care of this and therefore needs to be configured with a launch command. -ℹ️ Furthermore note that Serena is always configured *for a single project*. To use it for another, you will have to -write a new configuration file, adjust the configuration to point to it and then restart the client. - For more information on MCP servers with Claude Desktop, see [the official quick start guide](https://modelcontextprotocol.io/quickstart/user). ### Other MCP Clients - Cline, Roo-Code, Cursor, Windsurf etc.