tiennm99/serena
1
0
Fork 0
mirror of https://github.com/tiennm99/serena.git synced 2026-09-08 02:20:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into feature/clojure-lsp

Browse Source
This commit is contained in:
Miguel de Benito Delgado
2025-06-28 11:27:17 +00:00
parent 0b280eca98 65a64d75f3
commit 3fe2825538
113 changed files with 8088 additions and 4840 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/codespell.yml
+25
View File
@@ -0,0 +1,25 @@
# Codespell configuration is within pyproject.toml
---
name: Codespell
on:
push:
branches: [main]
pull_request:
branches: [main]
permissions:
contents: read
jobs:
codespell:
name: Check for spelling errors
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Annotate locations with typos
uses: codespell-project/codespell-problem-matcher@v1
- name: Codespell
uses: codespell-project/actions-codespell@v2
.github/workflows/docker.yml
+68
View File
@@ -0,0 +1,68 @@
name: Build and Push Docker Images
on:
push:
branches: [ main ]
tags: [ 'v*' ]
pull_request:
branches: [ main ]
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to Container Registry
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,value=latest,enable={{is_default_branch}}
- name: Build and push production image
uses: docker/build-push-action@v5
with:
context: .
target: production
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Build and push development image
uses: docker/build-push-action@v5
with:
context: .
target: development
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:dev
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
.github/workflows/pytest.yml
+1 -8
View File
@@ -55,11 +55,4 @@ jobs:
run: uv pip install -e ".[dev]"
- name: Test with pytest
shell: bash
# Currently, java and rust seem to cause problems in CI, and snapshot tests seem to cause problems on Windows
# probably due to the way the language servers are started and stopped
run: |
if [[ "$RUNNER_OS" == "Windows" ]]; then
uv run pytest -m "not java and not rust and not snapshot"
else
uv run pytest -m "not java and not rust"
fi
run: uv run poe test
.gitignore
+1
View File
@@ -203,6 +203,7 @@ pylint.html
# dynamic LS installations
/src/multilspy/language_servers/*/static
/src/solidlsp/language_servers/*/static
# clojure-lsp temporary files
.calva/
.serena/memories/current_solid_lsp_architecture.md
+123
View File
@@ -0,0 +1,123 @@
# Current Architecture: Solid-LSP Implementation
## Overview
Solid-LSP (`src/solidlsp/`) is Serena's current language server integration layer, designed as a **simplified, deadlock-resistant** replacement for the problematic multilspy architecture. It enables **single-process operation** with MCP servers while maintaining all semantic code analysis capabilities.
## Current System Status
- **Default Implementation**: Solid-LSP is the only language server implementation (multilspy completely removed)
- **Process Isolation**: **Disabled by default** (`USE_PROCESS_ISOLATION = False`)
- **MCP Integration**: Runs safely in the same process as MCP server
- **Performance**: Lower latency and resource usage than process-isolated multilspy
## Core Architecture
### Primary Components
#### 1. **`SolidLanguageServer`** (`src/solidlsp/ls.py`)
- **1,634 lines**: Main language server interface
- **Enhanced Methods**: Same semantic capabilities as multilspy but with improved async handling
- **Lifecycle Control**: `start()`, `stop()`, `is_running()`, `language_server()` property
- **Clean Async Patterns**: No coroutine leakage between async contexts
#### 2. **`SolidLanguageServerHandler`** (`src/solidlsp/ls_handler.py`)
- **512 lines**: Simplified LSP protocol handler
- **Direct Process Management**: More straightforward than multilspy's complex orchestration
- **Resource Management**: Better cleanup and process termination
#### 3. **Protocol Layer** (`src/solidlsp/lsp_protocol_handler/`)
- **Simplified Protocol**: Direct LSP communication without excessive abstraction
- **Types and Constants**: LSP type definitions and protocol constants
- **Request Handling**: Streamlined request/response pattern
### Language Server Support
Solid-LSP maintains **identical language support** to multilspy:
#### Primary Languages (Directly Supported)
- **Python**: Pyright Language Server (`src/solidlsp/language_servers/pyright_language_server/`)
- **Java**: Eclipse JDTLS (`src/solidlsp/language_servers/eclipse_jdtls/`)
- **TypeScript/JavaScript**: TypeScript Language Server (`src/solidlsp/language_servers/typescript_language_server/`)
#### Additional Languages (Full Support)
- **C#**: OmniSharp (`src/solidlsp/language_servers/omnisharp/`)
- **Rust**: Rust-Analyzer (`src/solidlsp/language_servers/rust_analyzer/`)
- **Go**: Gopls (`src/solidlsp/language_servers/gopls/`)
- **Ruby**: Solargraph (`src/solidlsp/language_servers/solargraph/`)
- **C++**: Clangd (`src/solidlsp/language_servers/clangd_language_server/`)
- **Dart**: Dart Language Server (`src/solidlsp/language_servers/dart_language_server/`)
- **PHP**: Intelephense (`src/solidlsp/language_servers/intelephense/`)
- **Kotlin**: Kotlin Language Server (`src/solidlsp/language_servers/kotlin_language_server/`)
## Key Architectural Improvements
### 1. **Simplified Async Patterns**
- **Clean Boundaries**: Proper async context management prevents MCP server contamination
- **No Coroutine Leakage**: Eliminates the unawaited coroutine warnings from multilspy
- **Direct Operations**: Fewer abstraction layers where async context could be corrupted
### 2. **Single Process Safety**
- **Safe MCP Integration**: Designed to work within MCP server process without deadlocks
- **Eliminated IPC Overhead**: Direct method calls instead of inter-process communication
- **Resource Efficiency**: Lower memory usage and faster startup times
### 3. **Enhanced Process Control**
- **Direct Process Management**: `_start_server_process()`, `_start_server()` methods
- **Better State Management**: `_server_context` attribute for enhanced control
- **Improved Cleanup**: More reliable resource management and process termination
## Integration Points
### MCP Server Integration
Current implementation in `src/serena/mcp.py:590`:
```python
if not USE_PROCESS_ISOLATION:
mcp_factory = SerenaMCPFactorySingleProcess(context=context, project=project_file)
else:
mcp_factory = SerenaMCPFactoryWithProcessIsolation(context=context, project=project_file)
```
### Agent Integration
Language server creation in `src/serena/agent.py:642-689`:
```python
def create_ls_for_project(...) -> SolidLanguageServer:
# Creates LanguageServerConfig with project settings
# Returns SolidLanguageServer.create() instance
```
## Configuration and Settings
### Core Configuration
- **`USE_PROCESS_ISOLATION = False`**: Process isolation disabled by default
- **Language Detection**: Automatic based on project composition
- **Timeout Settings**: Configurable language server timeouts
- **LSP Communication Tracing**: Optional debugging support
### Project-Level Settings
- **Ignored Paths**: Respects project configuration and gitignore
- **Language Selection**: Automatic or manual language server selection
- **Timeout Configuration**: Per-project timeout settings
## Performance Benefits
### vs. Multilspy + Process Isolation
1. **Lower Latency**: Direct method calls instead of IPC
2. **Reduced Memory**: Single process instead of multiple processes
3. **Faster Startup**: No process creation overhead
4. **Simpler Debugging**: All components in same process with unified stack traces
### Stability Improvements
1. **No Asyncio Deadlocks**: Clean async boundaries prevent contamination
2. **Reliable Operations**: `find_symbol` and other tools work consistently
3. **Resource Management**: Better cleanup prevents resource leaks
4. **Error Handling**: Simplified error propagation and handling
## Current Operational State
- **Production Ready**: Default implementation for all Serena deployments
- **Fully Tested**: All semantic tools (`find_symbol`, `replace_symbol_body`, etc.) working reliably
- **MCP Compatible**: Stable operation with Claude Desktop and other MCP clients
- **Cross-Platform**: Works on all supported operating systems
This architecture represents the successful resolution of the multilspy asyncio contamination issues while maintaining full semantic code analysis capabilities and improving overall performance.
.serena/memories/language_servers.md
-16
View File
@@ -1,16 +0,0 @@
# Available Language Servers
Serena supports multiple programming languages through language servers. The language servers currently available in the project are:
## Directly Supported (Out of the Box)
- **Python**: Using Pyright Language Server (previously used Jedi)
- **Java**: Using Eclipse JDTLS (Note: startup is slow, especially initial startup)
- **TypeScript/JavaScript**: Using TypeScript Language Server
## Indirectly Supported (May Require Manual Setup)
- **Ruby**: Using Solargraph
- **Go**: Using Gopls
- **C#**: Using OmniSharp
- **Rust**: Using Rust-Analyzer
These language servers utilize the Language Server Protocol (LSP) to provide semantic code analysis capabilities. The system can be extended to support additional languages by providing adapters for new language server implementations.
.serena/memories/multilspy_historical_context.md
+73
View File
@@ -0,0 +1,73 @@
# Historical Context: Multilspy Era and Migration to Solid-LSP
## What multilspy Was
Multilspy was Serena's original language server integration layer that provided semantic code analysis through Language Server Protocol (LSP). It was located in `src/multilspy/` and included:
### Core Components (No Longer Present)
- **`multilspy.LanguageServer`**: Main language server interface with async methods like `request_full_symbol_tree()`
- **`LanguageServerHandler`**: Complex async task management and protocol handling
- **`lsp_protocol_handler`**: Separate protocol abstraction layer
- **Process Management**: Complex threading and async orchestration
### Supported Languages
Multilspy supported the same languages now supported by solid-lsp:
- Python (Pyright), Java (Eclipse JDTLS), TypeScript/JavaScript, C#, Rust, Go, Ruby, C++, Dart, PHP, Kotlin
## Why We Moved Away from Multilspy
### 1. **Asyncio Contamination Issues**
- **Root Problem**: MCP server runs its own asyncio event loop, while multilspy created additional asyncio loops
- **Coroutine Leakage**: `multilspy.LanguageServer.request_full_symbol_tree()` coroutines leaked into MCP server context but were never awaited
- **Evidence**: MCP server logs showed: `RuntimeWarning: coroutine 'LanguageServer.request_full_symbol_tree' was never awaited`
- **Result**: Accumulated unawaited coroutines caused resource exhaustion and apparent "hangs"
### 2. **Dual Event Loop Interference**
- **MCP Server**: Asyncio loop A for handling requests
- **SerenaAgent**: Asyncio loop B for language servers + loop C for dashboard
- **Conflict**: Multiple asyncio contexts caused contamination and deadlocks
### 3. **Complex Architecture**
- **Over-abstraction**: Multiple layers of protocol handling increased complexity
- **Threading Issues**: Complex async task management made debugging difficult
- **Resource Overhead**: Separate protocol handler processes added overhead
### 4. **MCP-Specific Problems**
- **Only in MCP environments**: Issues didn't occur in direct script execution
- **Mandatory Process Isolation**: Required separate processes to prevent asyncio contamination
- **Performance Penalty**: IPC overhead was necessary for stability
## The Process Isolation Workaround
Before solid-lsp, the **only way** to use multilspy with MCP was through process isolation:
```python
# Old architecture (when multilspy was still present)
if USE_SOLID_LSP:
mcp_factory = SerenaMCPFactorySingleProcess(context=context, project=project_file)
else:
# multilspy required process isolation to prevent asyncio contamination
mcp_factory = SerenaMCPFactoryWithProcessIsolation(context=context, project=project_file)
```
### Why Process Isolation Was Mandatory for Multilspy
- **Complete Separation**: MCP server, SerenaAgent, and language servers in different processes
- **IPC Overhead**: All communication through inter-process communication
- **Resource Cost**: Higher memory usage and startup time
- **Debugging Complexity**: Distributed architecture made troubleshooting harder
## Timeline of Changes
1. **Pre-solid-lsp**: Multilspy was the only option, required process isolation for MCP
2. **Solid-lsp Introduction**: New implementation designed to eliminate asyncio issues
3. **Current State**: Multilspy completely removed, solid-lsp is the only implementation
4. **Process Isolation**: Now disabled by default (`USE_PROCESS_ISOLATION = False`) since it's no longer needed
## Key Lessons Learned
- **Asyncio Complexity**: Multiple event loops in the same application are extremely difficult to manage correctly
- **MCP Integration Challenges**: MCP server's async nature requires careful consideration of async boundaries
- **Simplicity Benefits**: Removing abstraction layers often improves stability and performance
- **Process Isolation Trade-offs**: While effective for isolation, the performance cost is significant when not necessary
This migration represents a successful architectural evolution from a complex, problematic system to a simpler, more reliable one.
.serena/memories/process_isolation_current_status.md
+126
View File
@@ -0,0 +1,126 @@
# Process Isolation: Current Status and Architecture
## Current Status: Disabled by Default
Process isolation is **disabled by default** in the current Serena implementation:
- **Configuration**: `USE_PROCESS_ISOLATION = False` in `src/serena/constants.py:22`
- **Reason**: No longer needed with solid-lsp architecture
- **Default Operation**: Single process mode with MCP server, agent, and language servers in same process
## Why Process Isolation Is No Longer Needed
### Historical Context
Process isolation was **mandatory** when using multilspy because:
1. **Asyncio Contamination**: Multilspy leaked coroutines into MCP server's event loop
2. **Event Loop Conflicts**: Multiple asyncio contexts caused deadlocks
3. **Only Solution**: Complete process separation was the only way to prevent issues
### Current Architecture Benefits
With solid-lsp, process isolation became **unnecessary** because:
1. **Clean Async Boundaries**: No coroutine leakage between MCP server and language server contexts
2. **Single Process Safety**: Solid-lsp designed to work safely within MCP server process
3. **Performance Gains**: Direct method calls instead of expensive IPC
## Architecture Components (Still Present)
The process isolation infrastructure remains **available but unused** by default:
### Core Components (`src/serena/process_isolated_agent.py`)
#### 1. **ProcessIsolatedSerenaAgent** (lines 392-550)
- **Purpose**: Wrapper that manages isolated agent process
- **Communication**: Uses `multiprocessing.Pipe()` for bidirectional communication
- **Process Management**: Creates and manages `SerenaAgentWorker` subprocess
- **Status**: **Available but not used by default**
#### 2. **SerenaAgentWorker** (lines 173-389)
- **Purpose**: Worker process hosting actual SerenaAgent
- **Event Loop**: Polling loop checking for requests every 500ms
- **Request Handling**: INITIALIZE, TOOL_CALL, SHUTDOWN, etc.
- **Status**: **Available but not used by default**
#### 3. **ProcessIsolatedDashboard** (lines 133-170)
- **Purpose**: Runs web dashboard in separate process
- **Integration**: Async web server with port management
- **Status**: **Available but not used by default**
### Global Synchronization (lines 25-28)
```python
_global_log_queue: multiprocessing.Queue = multiprocessing.Queue()
_dashboard_ready_event = multiprocessing.Event()
_dashboard_port_value = multiprocessing.Value("i", 0)
global_shutdown_event = multiprocessing.Event()
```
**Status**: **Available but not actively used**
## Current MCP Factory Selection
In `src/serena/mcp.py:590`:
```python
if not USE_PROCESS_ISOLATION:
mcp_factory = SerenaMCPFactorySingleProcess(context=context, project=project_file)
else:
mcp_factory = SerenaMCPFactoryWithProcessIsolation(context=context, project=project_file)
```
### Default Flow
1. **`USE_PROCESS_ISOLATION = False`** → **`SerenaMCPFactorySingleProcess`**
2. **Single Process**: MCP server, SerenaAgent, and solid-lsp in same process
3. **Direct Communication**: No IPC overhead, direct method calls
### Fallback Option
If `USE_PROCESS_ISOLATION = True`:
1. **`SerenaMCPFactoryWithProcessIsolation`** would be used
2. **Separate Processes**: MCP server and SerenaAgent in different processes
3. **IPC Communication**: Higher latency but complete isolation
## Benefits of Current Single Process Architecture
### 1. **Performance Improvements**
- **Lower Latency**: Direct method calls vs IPC communication
- **Reduced Memory**: No duplicate process memory footprints
- **Faster Startup**: No process creation and initialization overhead
### 2. **Operational Simplicity**
- **Unified Logging**: All components log to same destination
- **Simpler Debugging**: Single process, unified stack traces
- **Resource Management**: Simpler cleanup and shutdown procedures
### 3. **Stability Benefits**
- **No IPC Failures**: Eliminated inter-process communication failure modes
- **Consistent State**: No synchronization issues between processes
- **Reliable Shutdown**: No orphaned processes or cleanup complexity
## When Process Isolation Might Still Be Useful
While not needed by default, process isolation could still be beneficial for:
### 1. **Fault Isolation**
- **Agent Crashes**: Prevent agent failures from affecting MCP server
- **Memory Protection**: Isolate memory leaks to specific processes
- **Recovery**: Restart failed components without full system restart
### 2. **Resource Management**
- **Memory Limits**: Constrain memory usage of specific components
- **CPU Isolation**: Prevent CPU-intensive operations from blocking MCP server
- **Security**: Additional process boundaries for security-sensitive environments
### 3. **Debugging and Development**
- **Component Isolation**: Debug specific components in isolation
- **Performance Analysis**: Measure resource usage per component
- **Development Safety**: Prevent development errors from affecting stable components
## Configuration Management
### Enabling Process Isolation
To re-enable process isolation:
1. **Set**: `USE_PROCESS_ISOLATION = True` in `src/serena/constants.py`
2. **Result**: Automatic fallback to `SerenaMCPFactoryWithProcessIsolation`
3. **Trade-off**: Higher resource usage but complete component isolation
### Current Recommendation
- **Default**: Keep `USE_PROCESS_ISOLATION = False` for optimal performance
- **Special Cases**: Enable only when specific isolation requirements exist
- **Testing**: Both modes should be tested to ensure compatibility
The current architecture successfully eliminated the need for process isolation while maintaining the capability as a fallback option for specialized use cases.
.serena/memories/project_structure.md
-16
View File
@@ -1,16 +0,0 @@
# Project Structure
The Serena codebase is organized as follows:
- **src/serena/**: Main package with core functionality
- **llm/**: LLM integration
- **util/**: Utilities
- Key files: mcp.py, agno.py, agent.py
- **src/multilspy/**: Language server integration
- **language_servers/**: Implementations for different languages
- Python (pyright/jedi)
- Java (Eclipse JDTLS)
- TypeScript/JavaScript
- C#, Rust, Go, Ruby, C++
- **test/**: Tests
- **scripts/**: Utility scripts
.serena/memories/serena_project_structure.md
+173
View File
@@ -0,0 +1,173 @@
# Serena Project Structure Overview
## Top-Level Organization
Serena is organized into several key directories, each serving distinct architectural purposes:
```
serena/
├── src/ # Main source code
│ ├── interprompt/ # Template and prompt management system
│ ├── serena/ # Core Serena functionality
│ └── solidlsp/ # Language server integration layer
├── tests/ # Test suites
├── docs/ # Documentation
├── prompts/ # Prompt templates and configurations
├── contexts/ # Context definitions (desktop-app, agent, ide-assistant)
├── modes/ # Mode definitions (planning, editing, interactive, one-shot)
└── pyproject.toml # Project configuration and dependencies
```
## Core Source Structure (`src/`)
### 1. **Serena Core (`src/serena/`)**
The heart of the Serena system containing:
#### **Agent and Core Logic**
- **`agent.py`**: Main `SerenaAgent` class, tool implementations, project management
- **`mcp.py`**: Model Context Protocol server implementation and MCP factories
- **`config.py`**: Configuration system (contexts, modes, registered configurations)
- **`constants.py`**: System constants including `USE_PROCESS_ISOLATION = False`
#### **Specialized Components**
- **`dashboard.py`**: Web dashboard for monitoring and logging (`MemoryLogHandler`, `SerenaDashboardAPI`)
- **`symbol.py`**: Symbol management (`SymbolLocation`, `Symbol`, `SymbolManager`)
- **`text_utils.py`**: Text processing utilities for search and file operations
- **`gui_log_viewer.py`**: GUI log window implementation
#### **Integration Layers**
- **`agno.py`**: Agno framework integration (`SerenaAgnoToolkit`, `SerenaAgnoAgentProvider`)
- **`process_isolated_agent.py`**: Process isolation infrastructure (available but unused by default)
- **`prompt_factory.py`**: Prompt generation and management
#### **Utility Modules (`src/serena/util/`)**
- **`file_system.py`**: File scanning, gitignore parsing (`GitignoreParser`, `scan_directory`)
- **`git.py`**: Git operations and status checking
- **`shell.py`**: Shell command execution (`execute_shell_command`)
- **`thread.py`**: Threading utilities with timeout support
- **`inspection.py`**: Code inspection and language detection utilities
### 2. **Solid-LSP (`src/solidlsp/`)**
Current language server integration layer:
#### **Core Language Server Components**
- **`ls.py`**: Main `SolidLanguageServer` class (1,634 lines)
- **`ls_handler.py`**: `SolidLanguageServerHandler` for protocol management (512 lines)
- **`ls_request.py`**: Request handling (`LanguageServerRequest`)
- **`ls_types.py`**: LSP type definitions
- **`ls_utils.py`**: Utility classes (`TextUtils`, `PathUtils`, `FileUtils`, `SymbolUtils`)
#### **Protocol Handling (`src/solidlsp/lsp_protocol_handler/`)**
- **`server.py`**: LSP protocol server implementation
- **`lsp_requests.py`**: LSP request and notification classes
- **`lsp_types.py`**: Complete LSP type system definitions
- **`lsp_constants.py`**: LSP protocol constants
#### **Language Server Implementations (`src/solidlsp/language_servers/`)**
Each language has its own subdirectory with specific implementations:
- **`pyright_language_server/`**: Python support via Pyright
- **`eclipse_jdtls/`**: Java support via Eclipse JDTLS
- **`typescript_language_server/`**: TypeScript/JavaScript support
- **`omnisharp/`**: C# support via OmniSharp
- **`rust_analyzer/`**: Rust support via Rust-Analyzer
- **`gopls/`**: Go support via Gopls
- **`solargraph/`**: Ruby support via Solargraph
- **`clangd_language_server/`**: C++ support via Clangd
- **`dart_language_server/`**: Dart support
- **`intelephense/`**: PHP support via Intelephense
- **`kotlin_language_server/`**: Kotlin support
### 3. **Interprompt (`src/interprompt/`)**
Template and prompt management system:
- **`multilang_prompt.py`**: Multi-language prompt templates (`MultiLangPromptTemplate`)
- **`jinja_template.py`**: Jinja2 template integration (`JinjaTemplate`)
- **`prompt_factory.py`**: Prompt factory base classes
- **`util/class_decorators.py`**: Utility decorators like `@singleton`
## Configuration and Templates
### **Prompt Templates (`prompts/`)**
Organized by context and functionality:
- **Context-specific prompts**: Different prompt sets for different execution contexts
- **Tool-specific prompts**: Specialized prompts for specific tools and operations
- **Multi-language support**: Prompts available in multiple languages when applicable
### **Contexts (`contexts/`)**
Execution environment definitions:
- **`desktop-app.yml`**: Desktop application context
- **`agent.yml`**: Agent-specific context
- **`ide-assistant.yml`**: IDE assistant context
- **Custom contexts**: Support for user-defined contexts
### **Modes (`modes/`)**
Behavior mode definitions:
- **`planning.yml`**: Planning mode behavior
- **`editing.yml`**: Editing mode behavior
- **`interactive.yml`**: Interactive mode behavior
- **`one-shot.yml`**: One-shot execution mode
## Key Architectural Patterns
### 1. **Four-Layer Configuration Hierarchy**
1. **Global**: `serena_config.yml`
2. **CLI Arguments**: Runtime overrides
3. **Project**: `.serena/project.yml`
4. **Active Modes**: Runtime behavior modification
### 2. **Tool System Architecture**
- **Base Classes**: `ToolInterface`, `Tool` in `agent.py`
- **Marker Interfaces**: `ToolMarkerCanEdit`, `ToolMarkerDoesNotRequireActiveProject`
- **Tool Registry**: `ToolRegistry` with automatic tool discovery via `_iter_tool_classes()`
- **Tool Categories**: Semantic tools, file operations, project management, meta-tools
### 3. **Integration Patterns**
- **MCP Server**: Primary integration via `SerenaMCPFactory` classes
- **Agno Agent**: Model-agnostic integration via `SerenaAgnoToolkit`
- **Process Isolation**: Optional isolation via `ProcessIsolatedSerenaAgent`
### 4. **Memory Management**
- **Project Memories**: `.serena/memories/` directory for project-specific information
- **Memory Tools**: `WriteMemoryTool`, `ReadMemoryTool`, `ListMemoriesTool`
- **Memory Managers**: `MemoriesManager`, `MemoriesManagerMDFilesInProject`
## Dependencies and Build System
### **Development Tools**
- **`uv`**: Dependency management and virtual environment
- **`poe`**: Task orchestration (defined in `pyproject.toml`)
- **Essential Commands**: `uv run poe lint`, `uv run poe format`, `uv run poe type-check`, `uv run poe test`
### **Key Dependencies**
- **Language Server Protocol**: LSP implementations for each supported language
- **MCP (Model Context Protocol)**: For integration with Claude Desktop and other MCP clients
- **Agno Framework**: For model-agnostic agent implementations
- **AsyncIO**: For concurrent operations (carefully managed to avoid contamination)
## Project State Management
### **Project Configuration**
- **Global Config**: `serena_config.yml` in user home directory
- **Project Config**: `.serena/project.yml` in project root
- **Auto-generation**: Automatic creation of default configurations when missing
### **Language Detection**
- **Automatic**: Based on file composition analysis
- **Manual Override**: Via project configuration
- **Multi-language**: Support for projects with multiple languages
## Runtime Architecture
### **Default Operation Mode**
- **Single Process**: MCP server, agent, and language servers in same process
- **Direct Communication**: No IPC overhead
- **Solid-LSP**: Only language server implementation
- **Process Isolation**: Disabled by default (`USE_PROCESS_ISOLATION = False`)
### **Semantic Tool Integration**
- **Symbol-based Operations**: `find_symbol`, `replace_symbol_body`, `insert_after_symbol`
- **Regex-based Operations**: `replace_regex` for fine-grained edits
- **File Operations**: `read_file`, `create_text_file`, `list_dir`
- **Project Operations**: `search_for_pattern`, `get_symbols_overview`
This structure reflects Serena's evolution from a complex, multi-process system to a streamlined, single-process architecture that maintains all semantic capabilities while improving performance and reliability.
.serena/memories/suggested_commands.md
+16 -2
View File
@@ -4,10 +4,24 @@
The following tasks should generally be executed using `uv run poe <task_name>`.
- `lint`: This is the **only** allowed command for linting. Run as `uv run poe lint`.
- `format`: This is the **only** allowed command for formatting. Run as `uv run poe format`.
- `type-check`: This is the **only** allowed command for type checking. Run as `uv run poe type-check`.
- `test`: This is the preferred command for running tests (`uv run poe test [args]`). However, running tests directly with `uv run pytest [args]` is also permitted.
- `test`: This is the preferred command for running tests (`uv run poe test [args]`). You can select subsets of tests with markers,
the current markers are
```toml
markers = [
"python: language server running for Python",
"go: language server running for Go",
"java: language server running for Java",
"rust: language server running for Rust",
"typescript: language server running for TypeScript",
"php: language server running for PHP",
"snapshot: snapshot tests for symbolic editing operations",
"isolated_process: test runs with process isolated agent",
]
```
By default, `uv run poe test` uses the markers set in the env var `PYTEST_MARKERS`, or, if it unset, uses `-m "not java and not rust and not isolated process"`.
You can override this behavior by simply passing the `-m` option to `uv run poe test`, e.g. `uv run poe test -m "python or go"`.
For finishing a task, make sure format, type-check and test pass! Run them at the end of the task
and if needed fix any issues that come up and run them again until they pass.
.serena/memories/supported_language_servers.md
+97
View File
@@ -0,0 +1,97 @@
# Supported Language Servers in Solid-LSP
Serena currently supports multiple programming languages through language servers via the solid-lsp architecture located in `src/solidlsp/language_servers/`.
## Primary Languages (Directly Supported)
### Python
- **Language Server**: Pyright Language Server
- **Location**: `src/solidlsp/language_servers/pyright_language_server/`
- **Features**: Full semantic analysis, type checking, symbol navigation
- **Previously**: Also supported Jedi (now deprecated in favor of Pyright)
### Java
- **Language Server**: Eclipse JDTLS (Java Development Tools Language Server)
- **Location**: `src/solidlsp/language_servers/eclipse_jdtls/`
- **Features**: Full Java language support, Maven/Gradle integration
- **Note**: Startup can be slow, especially on initial launch
### TypeScript/JavaScript
- **Language Server**: TypeScript Language Server
- **Location**: `src/solidlsp/language_servers/typescript_language_server/`
- **Features**: TypeScript and JavaScript support, type checking, IntelliSense
## Additional Supported Languages
### C#
- **Language Server**: OmniSharp
- **Location**: `src/solidlsp/language_servers/omnisharp/`
- **Features**: Full C# language support, .NET integration
### Rust
- **Language Server**: Rust-Analyzer
- **Location**: `src/solidlsp/language_servers/rust_analyzer/`
- **Features**: Rust language support, cargo integration
### Go
- **Language Server**: Gopls
- **Location**: `src/solidlsp/language_servers/gopls/`
- **Features**: Go language support, module system integration
### Ruby
- **Language Server**: Solargraph
- **Location**: `src/solidlsp/language_servers/solargraph/`
- **Features**: Ruby language support, gem integration
### C++
- **Language Server**: Clangd
- **Location**: `src/solidlsp/language_servers/clangd_language_server/`
- **Features**: C++ language support, CMake integration
### Dart
- **Language Server**: Dart Language Server
- **Location**: `src/solidlsp/language_servers/dart_language_server/`
- **Features**: Dart and Flutter support
### PHP
- **Language Server**: Intelephense
- **Location**: `src/solidlsp/language_servers/intelephense/`
- **Features**: PHP language support, Composer integration
### Kotlin
- **Language Server**: Kotlin Language Server
- **Location**: `src/solidlsp/language_servers/kotlin_language_server/`
- **Features**: Kotlin language support, Gradle integration
## Language Server Protocol (LSP) Integration
All language servers utilize the Language Server Protocol (LSP) to provide:
- **Semantic Analysis**: Symbol definitions, references, and relationships
- **Code Navigation**: Go-to-definition, find-references, symbol search
- **Error Detection**: Syntax and semantic error reporting
- **Code Completion**: IntelliSense-style code completion
- **Documentation**: Hover information and signature help
## Architecture Integration
### Solid-LSP Framework
- **Unified Interface**: All language servers use the same `SolidLanguageServer` interface
- **Configuration**: `LanguageServerConfig` handles language-specific settings
- **Lifecycle Management**: Start, stop, and restart capabilities for all servers
- **Process Management**: Direct process control for each language server
### Project Integration
- **Automatic Detection**: Language selection based on project file composition
- **Configuration**: Project-specific language server settings via `.serena/project.yml`
- **Ignored Paths**: Respects gitignore and custom ignore patterns
- **Timeout Settings**: Configurable per-language server timeouts
## Adding New Language Servers
The architecture supports adding new language servers by:
1. **Creating Language Server Class**: Implement language-specific server in `src/solidlsp/language_servers/`
2. **Configuration**: Add language detection and configuration rules
3. **Integration**: Register with the solid-lsp framework
4. **Testing**: Ensure semantic tools work correctly with the new language server
The system is designed to be extensible while maintaining consistent behavior across all supported languages.
CHANGELOG.md
+37 -13
View File
@@ -1,9 +1,34 @@
# Latest
Status of the main branch. Changes prior to the next official version change will appear here.
## Highlights
Status of the `main` branch. Changes prior to the next official version change will appear here.
### This version is a major change and improvement of Serena
* **Reduce the use of asyncio to a minimum**, improving stability and reducing the need for workarounds
* Switch to newly developed fully synchronous LSP library `solidlsp` (derived from `multilspy`),
removing our fork of `multilspy` (src/multilspy)
* Switch from fastapi (which uses asyncio) to Flask in the Serena dashboard
* The MCP server is the only asyncio-based component now, which resolves cross-component loop contamination,
such that process isolation is no longer required.
Neither are non-graceful shutdowns on Windows.
* Better default and description for restricting the search in `search_for_pattern`
* **Improved editing tools**: The editing logic was simplified and improved, making it more robust.
* The "minimal indentation" logic was removed, because LLMs did not understand it.
* The logic for the insertion of empty lines was improved (mostly controlled by the LLM now)
* Add a task queue for the agent, which is executed in a separate and thread and
* allows the language server to be initialized in the background, making the MCP server respond to requests
immediately upon startup,
* ensures that all tool executions are fully synchronized (executed linearly).
Fixes:
* Fix `ExecuteShellCommandTool` and `GetCurrentConfigTool` hanging on Windows
* Fix project activation by name via `--project` not working (was broken in previous release)
* Improve handling of indentation and newlines in symbolic editing tools
* Fix `InsertAfterSymbolTool` failing for insertions at the end of a file that did not end with a newline
* Fix `InsertBeforeSymbolTool` inserting in the wrong place in the absence of empty lines above the reference symbol
* Fix `ReplaceSymbolBodyTool` changing whitespace before/after the symbol
* Fix repository indexing not following links and catch exceptions during indexing, allowing indexing
to continue even if unexpected errors occur for individual files.
# 2025-06-20
* **Overhaul and major improvement of editing tools!**
This represents a very important change in Serena. Symbols can now be addressed by their `name_path` (including nested ones)
@@ -14,24 +39,23 @@ Status of the main branch. Changes prior to the next official version change wil
create `project.yaml` for each project. Project activation is now always available.
Any project can now be activated by just asking the LLM to do so and passing the path to a repo.
* Dashboard as web app and possibility to shut down Serena from it (or the old log GUI).
* Possibility to index your project beforehand, accelerating Serena's tools.
* Initial prompt for project supported (has to be added manually for the moment)
* Massive performance improvement of pattern search tool
* Use **process isolation** to fix stability issues and deadlocks (see #170).
This uses separate process for the MCP server, the Serena agent and the dashboard in order to fix asyncio-related issues.
# 2025-05-24
## Highlights
Important new feature: configurability of mode and context, allowing better integration in a variety of clients.
See corresponding section in readme - Serena can now be integrated in IDE assistants in a more productive way.
You can now also do things like switching to one-shot planning mode, ask to plan something (which will create a memory),
then switch to interactive editing mode in the next conversation and work through the plan read from the memory.
Also some improvements to prompts.
* Important new feature: **configurability of mode and context**, allowing better integration in a variety of clients.
See corresponding section in readme - Serena can now be integrated in IDE assistants in a more productive way.
You can now also do things like switching to one-shot planning mode, ask to plan something (which will create a memory),
then switch to interactive editing mode in the next conversation and work through the plan read from the memory.
* Some improvements to prompts.
# 2025-05-21
**Signficant improvement in symbol finding!**
**Significant improvement in symbol finding!**
* Serena core:
* `FindSymbolTool` now can look for symbols by specifying paths to them, not just the symbol name
CLAUDE.md
+66
View File
@@ -0,0 +1,66 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Essential Commands
This project uses `uv` for dependency management and `poe` for task orchestration:
- **Linting**: `uv run poe lint` (only allowed command for linting)
- **Formatting**: `uv run poe format` (only allowed command for formatting)
- **Type checking**: `uv run poe type-check` (only allowed command for type checking)
- **Testing**: `uv run poe test [args]` (preferred) or `uv run pytest [args]`
**Important**: Always run `format`, `type-check` and `test` at the end of tasks to ensure code quality. Fix any issues and re-run until they pass.
## Architecture Overview
Serena is a powerful coding agent toolkit that turns LLMs into fully-featured agents working directly on codebases through semantic code retrieval and editing tools.
### Core Components
- **src/serena/**: Main package containing core functionality
- `mcp.py`: Model Context Protocol server implementation
- `agno.py`: Agno framework integration for model-agnostic agents
- `agent.py`: Core agent implementation with semantic tools
- **llm/**: LLM integration modules
- **util/**: Utility functions and helpers
- **src/multilspy/**: Language server integration layer
- **language_servers/**: Language-specific implementations
- Python (pyright/jedi), Java (Eclipse JDTLS), TypeScript/JavaScript
- C#, Rust, Go, Ruby, C++, PHP support
- Provides semantic code analysis through Language Server Protocol (LSP)
- **src/interprompt/**: Template and prompt management system
### Key Design Principles
- **Semantic Code Understanding**: Uses language servers (LSP) for symbol-level code analysis rather than text-based approaches
- **Multiple Integration Methods**: Can be used as MCP server, Agno agent, or integrated into custom frameworks
- **Language Agnostic**: Supports multiple programming languages through language server adapters
- **Context and Mode System**: Configurable behavior for different environments (desktop-app, ide-assistant, agent)
### Integration Patterns
- **MCP Server**: Primary integration method for Claude Desktop and other MCP clients
- **Agno Agent**: Model-agnostic agent framework for any LLM with GUI support
- **Framework Adapter**: Tools can be adapted to any agent framework (example: SerenaAgnoToolkit)
### Configuration System
Four-layer configuration hierarchy:
1. `serena_config.yml` - Global settings
2. CLI arguments - Client-specific overrides
3. `.serena/project.yml` - Project-specific settings
4. Active modes - Runtime behavior modification
The codebase implements sophisticated semantic code operations through language servers, enabling precise symbol-level editing and code understanding that goes beyond simple text manipulation.
## Working on Serena's Code
**Important**: When working on this codebase, remember that you are using Serena's own tools to improve Serena itself. This means:
- You can use Serena's semantic tools (`find_symbol`, `replace_symbol_body`, `search_for_pattern`, etc.) to analyze and edit Serena's own source code
- When asked to "perform a task on Serena", you're being asked to modify/improve the current Serena codebase
- You have access to the full power of Serena's semantic code understanding to work on Serena's code
CONTRIBUTING.md
+6 -9
View File
@@ -55,21 +55,18 @@ An example script for running tools is provided in [scripts/demo_run_tools.py](s
## Adding a New Supported Language
Serena interacts with code through language servers which are included in
the `multilspy` package. It is rather easy to include a new supported language
the `solidlsp` package. It is rather easy to include a new supported language
if an LSP implementation for it exists. You just need to:
1. create a new subclass of `LanguageServer`
1. create a new subclass of `SolidLanguageServer`
2. add a new value to the `Language` enum
3. make a new `elif` case in the `LanguageServer.create` method
4. write minor tests
3. make a new `elif` case in the `SolidLanguageServer.create` method
4. add a test repo of the new language to `test/resources/repos/<new_language>/test_repo`
and new tests in `test/solidlsp/<new_language>`. Similar to existing tests for other languages
5. also add a new case to the parameterized tests in `test/serena/test_serena_agent`
The subclasses are typically easy to write, have a look at the
[PyrightLanguageServer](src/multilspy/language_servers/pyright_language_server/pyright_server.py)
for an example, or at any other implementation to see how non-python
dependencies for language servers are handled there.
There are also some tips from the multilspy admin [here](https://github.com/microsoft/multilspy/issues/5).
⚠️ Important: The LSP allows for lot of optional fields and symbols, so the language servers may differ
in some details, even if they follow the LSP. Therefore you should include some code of the new
language in `test/resources` and add tests for symbolic read operations on that code. Have a look
at `test/multilspy/python/test_symbol_retrieval.py` for an example of such tests for the python LS.
DOCKER.md
+159
View File
@@ -0,0 +1,159 @@
# Docker Setup for Serena (Experimental)
⚠️ **EXPERIMENTAL FEATURE**: The Docker setup for Serena is currently experimental and has several limitations. Please read this entire document before using Docker with Serena.
## Overview
Docker support allows you to run Serena in an isolated container environment, which provides better security isolation for the shell tool and consistent dependencies across different systems.
## Benefits
- **Safer shell tool execution**: Commands run in an isolated container environment
- **Consistent dependencies**: No need to manage language servers and dependencies on your host system
- **Cross-platform support**: Works consistently across Windows, macOS, and Linux
## Important Limitations and Caveats
### 1. Configuration File Conflicts
⚠️ **Critical**: Docker uses a separate configuration file (`serena_config.docker.yml`) to avoid path conflicts. When running in Docker:
- Container paths will be stored in the configuration (e.g., `/workspaces/serena/...`)
- These paths are incompatible with non-Docker usage
- After using Docker, you cannot directly switch back to non-Docker usage without manual configuration adjustment
### 2. Project Activation Limitations
- **Only mounted directories work**: Projects must be mounted as volumes to be accessible
- Projects outside the mounted directories cannot be activated or accessed
- Default setup only mounts the current directory
### 3. GUI Window Disabled
- The GUI log window option is automatically disabled in Docker environments
- Use the web dashboard instead (see below)
### 4. Dashboard Port Configuration
The web dashboard runs on port 24282 (0x5EDA) by default. You can configure this using environment variables:
```bash
# Use default ports
docker-compose up serena
# Use custom ports
SERENA_DASHBOARD_PORT=8080 docker-compose up serena
```
⚠️ **Note**: If the local port is occupied, you'll need to specify a different port using the environment variable.
### 5. Line Ending Issues on Windows
⚠️ **Windows Users**: Be aware of potential line ending inconsistencies:
- Files edited within the Docker container may use Unix line endings (LF)
- Your Windows system may expect Windows line endings (CRLF)
- This can cause issues with version control and text editors
- Configure your Git settings appropriately: `git config core.autocrlf true`
## Quick Start
### Using Docker Compose (Recommended)
1. **Production mode** (for using Serena as MCP server):
```bash
docker-compose up serena
```
2. **Development mode** (with source code mounted):
```bash
docker-compose up serena-dev
```
### Using Docker directly
```bash
# Build the image
docker build -t serena .
# Run with current directory mounted
docker run -it --rm \
-v "$(pwd)":/workspace \
-p 9121:9121 \
-p 24282:24282 \
-e SERENA_DOCKER=1 \
serena
```
## Accessing the Dashboard
Once running, access the web dashboard at:
- Default: http://localhost:24282/dashboard
- Custom port: http://localhost:${SERENA_DASHBOARD_PORT}/dashboard
## Volume Mounting
To work with projects, you must mount them as volumes:
```yaml
# In compose.yaml
volumes:
- ./my-project:/workspace/my-project
- /path/to/another/project:/workspace/another-project
```
## Environment Variables
- `SERENA_DOCKER=1`: Set automatically to indicate Docker environment
- `SERENA_PORT`: MCP server port (default: 9121)
- `SERENA_DASHBOARD_PORT`: Web dashboard port (default: 24282)
## Troubleshooting
### Port Already in Use
If you see "port already in use" errors:
```bash
# Check what's using the port
lsof -i :24282 # macOS/Linux
netstat -ano | findstr :24282 # Windows
# Use a different port
SERENA_DASHBOARD_PORT=8080 docker-compose up serena
```
### Configuration Issues
If you need to reset Docker configuration:
```bash
# Remove Docker-specific config
rm serena_config.docker.yml
# Serena will auto-generate a new one on next run
```
### Project Access Issues
Ensure projects are properly mounted:
- Check volume mounts in `docker-compose.yaml`
- Use absolute paths for external projects
- Verify permissions on mounted directories
## Migration Path
To switch between Docker and non-Docker usage:
1. **Docker to Non-Docker**:
- Manually edit project paths in `serena_config.yml`
- Change container paths to host paths
- Or use separate config files for each environment
2. **Non-Docker to Docker**:
- Projects will be re-registered with container paths
- Original config remains unchanged
## Future Improvements
We're working on:
- Automatic config migration between environments
- Better project path handling
- Dynamic port allocation
- Windows line-ending handling.
Dockerfile
+23 -7
View File
@@ -1,5 +1,5 @@
# Use the official Python image for the base image.
FROM python:3.11-slim
# Base stage with common dependencies
FROM python:3.11-slim AS base
SHELL ["/bin/bash", "-c"]
# Set environment variables to make Python print directly to the terminal and avoid .pyc files.
@@ -21,18 +21,18 @@ RUN python3 -m pip install --no-cache-dir pipx \
# Add local bin to the path
ENV PATH="${PATH}:/root/.local/bin"
# Install the latest version of uv
RUN curl -LsSf https://astral.sh/uv/install.sh | sh
# Set the working directory
WORKDIR /workspaces/serena
# Copy required files into the image
COPY pyproject.toml /workspaces/serena/
COPY README.md /workspaces/serena/
# Development target
FROM base AS development
# Copy all files for development
COPY . /workspaces/serena/
# Create virtual environment and install dependencies
# Create virtual environment and install dependencies with dev extras
RUN uv venv
RUN . .venv/bin/activate
RUN uv pip install --all-extras -r pyproject.toml -e .
@@ -41,3 +41,19 @@ ENV PATH="/workspaces/serena/.venv/bin:${PATH}"
# Entrypoint to ensure environment is activated
ENTRYPOINT ["/bin/bash", "-c", "source .venv/bin/activate && $0 $@"]
# Production target
FROM base AS production
# Copy only necessary files for production
COPY pyproject.toml /workspaces/serena/
COPY README.md /workspaces/serena/
COPY src/ /workspaces/serena/src/
# Create virtual environment and install dependencies (production only)
RUN uv venv
RUN . .venv/bin/activate
RUN uv pip install -r pyproject.toml -e .
ENV PATH="/workspaces/serena/.venv/bin:${PATH}"
# Entrypoint to ensure environment is activated
ENTRYPOINT ["/bin/bash", "-c", "source .venv/bin/activate && $0 $@"]
README.md
+190 -134
View File
@@ -37,13 +37,14 @@ https://github.com/user-attachments/assets/6eaa9aa1-610d-4723-a2d6-bf1e487ba753
Serena provides the necessary [tools](#full-list-of-tools) for coding workflows, but an LLM is required to do the actual work,
orchestrating tool use.
For example, **supercharge the performance of Claude Code** with a [one-line shell command](#claude-code).
Serena can be integrated with an LLM in several ways:
* by using the **model context protocol (MCP)**.
Serena provides an MCP server which integrates with
* Claude Desktop,
* Claude Code and Claude Desktop,
* IDEs like VSCode, Cursor or IntelliJ,
* Extensions like Cline or Roo Code
* Goose (for a nice CLI experience)
* and many others, including [the ChatGPT app soon](https://x.com/OpenAIDevs/status/1904957755829481737)
* by using **Agno – the model-agnostic agent framework**.
Serena's Agno-based agent allows you to turn virtually any LLM into a coding agent, whether it's provided by Google, OpenAI or Anthropic (with a paid API key)
@@ -67,7 +68,7 @@ With Serena, we provide
* direct, out-of-the-box support for:
* Python
* TypeScript/Javascript
* PhP
* PHP
* Go (need to install go and gopls first)
* Rust
* C/C++
@@ -78,8 +79,8 @@ With Serena, we provide
* Kotlin (untested)
* Dart (untested)
These languages are supported by the language server library [multilspy](https://github.com/microsoft/multilspy), which Serena uses under the hood.
But we did not explicitly test whether the support for these languages actually works.
These languages are supported by the language server library, but
we did not explicitly test whether the support for these languages actually works flawlessly.
Further languages can, in principle, easily be supported by providing a shallow adapter for a new language server
implementation.
@@ -87,7 +88,7 @@ implementation.
## Table of Contents
<!-- Created with markdown-toc -i README.md -->
<!-- Created with markdown-toc -i README.md -->
<!-- Install it with npm install -g markdown-toc -->
<!-- toc -->
@@ -95,14 +96,18 @@ implementation.
- [What Can I Use Serena For?](#what-can-i-use-serena-for)
- [Free Coding Agents with Serena](#free-coding-agents-with-serena)
- [Quick Start](#quick-start)
* [Setup](#setup)
* [Running the Serena MCP Server](#running-the-serena-mcp-server)
+ [Usage](#usage)
* [Local Installation](#local-installation)
- [Using uvx](#using-uvx)
- [Using Docker (Experimental)](#using-docker-experimental)
+ [SSE Mode](#sse-mode)
+ [Command-Line Arguments](#command-line-arguments)
* [Configuration](#configuration)
* [Project Activation](#project-activation)
* [MCP Server (Claude Desktop)](#mcp-server-claude-desktop)
+ [Troubleshooting](#troubleshooting)
* [Project Activation & Indexing](#project-activation--indexing)
* [Claude Code](#claude-code)
* [Claude Desktop](#claude-desktop)
* [Other MCP Clients (Cline, Roo-Code, Cursor, Windsurf, etc.)](#other-mcp-clients-cline-roo-code-cursor-windsurf-etc)
* [Goose](#goose)
* [Agno Agent](#agno-agent)
* [Other Agent Frameworks](#other-agent-frameworks)
- [Detailed Usage and Recommendations](#detailed-usage-and-recommendations)
@@ -122,7 +127,7 @@ implementation.
* [Running Out of Context](#running-out-of-context)
* [Combining Serena with Other MCP Servers](#combining-serena-with-other-mcp-servers)
* [Serena's Logs: The Dashboard and GUI Tool](#serenas-logs-the-dashboard-and-gui-tool)
* [Troubleshooting](#troubleshooting-1)
* [Troubleshooting](#troubleshooting)
- [Comparison with Other Coding Agents](#comparison-with-other-coding-agents)
* [Subscription-Based Coding Agents](#subscription-based-coding-agents)
* [API-Based Coding Agents](#api-based-coding-agents)
@@ -166,42 +171,112 @@ We thus built Serena with the prospect of being able to cancel most other subscr
Serena can be used in various ways, below you will find instructions for selected integrations.
- If you just want to turn Claude into a free-to-use coding agent, we recommend using Serena through Claude Desktop.
- If you want to use Gemini or any other model and you want a GUI experience, you should use [Agno](#agno-agent). On macOS you can also use the GUI of [goose](#goose).
- If you prefer using Serena through a CLI, you can use [goose](#goose). There again almost any model is possible.
- If you just want to turn Claude into a free-to-use coding agent, we recommend using Serena through [Claude Code](#claude-code) or [Claude Desktop](#claude-desktop).
- If you want to use Gemini or any other model, and you want a GUI experience, you can use [Agno](#agno-agent) or one of the many other GUIs that support MCP servers.
- If you want to use Serena integrated in your IDE, see the section on [other MCP clients](#other-mcp-clients---cline-roo-code-cursor-windsurf-etc).
### Setup
Serena is managed by `uv`, so you will need to [install it](https://docs.astral.sh/uv/getting-started/installation/)).
Then you can either
### Running the Serena MCP Server
1. Clone the repository and cd into it.
You have several options for running the MCP server, which are explained in the subsections below.
#### Usage
The typical usage involves the client (Claude Code, Claude Desktop, etc.) running
the MCP server as a subprocess (using stdio communication),
so the client needs to be provided with the command to run the MCP server.
(Alternatively, you can run the MCP server in SSE mode and tell your client
how to connect to it.)
Note that no matter how you run the MCP server, Serena will, by default, start a small web-based dashboard on localhost that will display logs and allow shutting down the
MCP server (since many clients fail to clean up processes correctly).
This and other settings can be adjusted in the [configuration](#configuration) and/or by providing [command-line arguments](#command-line-arguments).
###### Local Installation
1. Clone the repository and change into it.
```shell
git clone https://github.com/oraios/serena
cd serena
```
2. Optionally create a config file from the template and adjust it according to your preferences.
```shell
cp src/serena/resources/serena_config.template.yml serena_config.yml
```
If you just want the default config, you can skip this part, and a config file will be created when you first run Serena.
3. Run the server with `uv`:
```shell
uv run serena-mcp-server
```
When running from outside the serena installation directory, be sure to pass it, i.e. use
```shell
uv run --directory /abs/path/to/serena serena-mcp-server
```
or use `uvx` to run Serena directly by relying on
##### Using uvx
`uvx` can be used to run the latest version of Serena directly from the repository, without an explicit local installation.
* Windows:
```shell
uvx --from git+https://github.com/oraios/serena serena-mcp-server.exe
```
* Other operating systems:
```shell
uvx --from git+https://github.com/oraios/serena serena-mcp-server
```
##### Using Docker (Experimental)
⚠️ Docker support is currently experimental with several limitations. Please read the [Docker documentation](DOCKER.md) for important caveats before using it.
You can run the Serena MCP server directly via docker as follows,
assuming that the projects you want to work on are all located in `/path/to/your/projects`:
```shell
uvx --from git+https://github.com/oraios/serena serena-mcp-server.exe
docker run --rm -i --network host -v /path/to/your/projects:/workspaces/projects ghcr.io/oraios/serena:latest serena-mcp-server --transport stdio
```
in your MCP config (delete the `.exe` on macOS or Linux).
Replace `/path/to/your/projects` with the absolute path to your projects directory. The Docker approach provides:
- Better security isolation for shell command execution
- No need to install language servers and dependencies locally
- Consistent environment across different systems
You can now add Serena to your MCP client as described below for various clients and
[activate your first project](#project-activation).
See the [Docker documentation](DOCKER.md) for detailed setup instructions, configuration options, and known limitations.
#### SSE Mode
ℹ️ Note that MCP servers which use stdio as a protocol are somewhat unusual as far as client/server architectures go, as the server
necessarily has to be started by the client in order for communication to take place via the server's standard input/output stream.
In other words, you do not need to start the server yourself. The client application (e.g. Claude Desktop) takes care of this and
therefore needs to be configured with a launch command.
When using instead the SSE mode, which uses HTTP-based communication, you control the server lifecycle yourself,
i.e. you start the server and provide the client with the URL to connect to it.
Simply provide `serena-mcp-server` with the `--transport sse` option and optionally provide the port.
For example, to run the Serena MCP server in SSE mode on port 9121 using a local installation,
you would run this command from the Serena directory,
```shell
uv run serena-mcp-server --transport sse --port 9121
```
and then configure your client to connect to `http://localhost/sse:9121`.
#### Command-Line Arguments
The Serena MCP server supports a wide range of additional command-line options, including the option to run in SSE mode
and to adapt Serena to various [contexts and modes of operation](#modes-and-contexts).
Run with parameter `--help` to get a list of available options.
> In the default configuration, Serena will start a small dashboard on localhost that will display logs and allow shutting down the
> MCP server (since many clients fail to cleanup processes, leaving zombies behind). If you don't want that, simply set `web_dashboard` to `False`
> in your `serena_config.yml` or pass
### Configuration
Serena's behavior (like available projects, active tools and prompts) is configured in four places:
Serena's behavior (active tools and prompts as well as logging configuration, etc.) is configured in four places:
1. The `serena_config.yml` for general settings that apply to all clients and projects
2. In the arguments passed to the `serena-mcp-server` in your client's config (see below),
@@ -224,10 +299,13 @@ want to use Serena.
You can just ask the LLM to show you the config of your session, Serena has a tool for it.
### Project Activation
### Project Activation & Indexing
The recommended way is to just ask the LLM to activate a project by providing it an absolute path to, or,
in case the project was activated in the past, by it's name. The default project name is the directory name.
in case the project was activated in the past, by its name. The default project name is the directory name.
* "Activate the project /path/to/my_project"
* "Activate the project my_project"
All projects that have been activated will be automatically added to your `serena_config.yml`, and for each
project, the file `.serena/project.yml` will be generated. You can adjust the latter, e.g., by changing the name
@@ -237,55 +315,85 @@ same name.
If you are mostly working with the same project, you can also configure to always activate a project at startup
by passing `--project <path_or_name>` to the `serena-mcp-server` command in your client's MCP config.
### MCP Server (Claude Desktop)
ℹ️ For larger projects, we recommend that you index your project to accelerate Serena's tools; otherwise the first
tool application may be very slow.
To do so, run one of these commands the project directory or pass the path to the project as an argument:
Configure the MCP server in your client.
For [Claude Desktop](https://claude.ai/download) (available for Windows and macOS), go to File / Settings / Developer / MCP Servers / Edit Config,
which will let you open the JSON file `claude_desktop_config.json`. Add the following (with adjusted paths) to enable Serena:
* When using a local installation:
```shell
uv run --directory /abs/path/to/serena index-project
```
* When using uvx:
```shell
uvx --from git+https://github.com/oraios/serena index-project
```
```json
{
"mcpServers": {
"serena": {
"command": "/abs/path/to/uv",
"args": ["run", "--directory", "/abs/path/to/serena", "serena-mcp-server"]
}
}
}
### Claude Code
Serena is a great way to make Claude Code both cheaper and more powerful!
From your project directory, add serena with a command like this,
```shell
claude mcp add serena -- <serena-mcp-server> --context ide-assistant --project $(pwd)
```
where `<serena-mcp-server>` is your way of [running the Serena MCP server](#running-the-serena-mcp-server).
For example, when using `uvx`, you would run
```shell
claude mcp add serena -- uvx --from git+https://github.com/oraios/serena serena-mcp-server --context ide-assistant --project $(pwd)
```
ℹ️ Once in Claude Code, you should ask Claude to "Read the initial instructions" as your first prompt, such that it will receive information
on how to use Serena's tools.
### Claude Desktop
For [Claude Desktop](https://claude.ai/download) (available for Windows and macOS), go to File / Settings / Developer / MCP Servers / Edit Config,
which will let you open the JSON file `claude_desktop_config.json`.
Add the `serena` MCP server configuration, using a [run command](#running-the-serena-mcp-server) depending on your setup.
* local installation:
```json
{
"mcpServers": {
"serena": {
"command": "/abs/path/to/uv",
"args": ["run", "--directory", "/abs/path/to/serena", "serena-mcp-server"]
}
}
}
```
* uvx:
```json
{
"mcpServers": {
"serena": {
"command": "/abs/path/to/uvx",
"args": ["--from", "git+https://github.com/oraios/serena", "serena-mcp-server"]
}
}
}
```
* docker:
```json
{
"mcpServers": {
"serena": {
"command": "docker",
"args": ["run", "--rm", "-i", "--network", "host", "-v", "/path/to/your/projects:/workspaces/projects", "ghcr.io/oraios/serena:latest", "serena-mcp-server", "--transport", "stdio"]
}
}
}
```
If you are using paths containing backslashes for paths on Windows
(note that you can also just use forward slashes), be sure to escape them correctly (`\\`).
That's it! Save the config and then restart Claude Desktop. You are ready for activating your first project
That's it! Save the config and then restart Claude Desktop. You are ready for activating your first project.
ℹ️ You can further customize the run command, see
```shell
uv run serena-mcp-server --help
```
ℹ️ You can use Serena without cloning or configuring it explicitly by
{
"mcpServers": {
"serena": {
"command": "/abs/path/to/uv",
"args": ["run", "--directory", "/abs/path/to/serena", "serena-mcp-server"]
}
}
}
#### Troubleshooting
Some client/OS/setup configurations were reported to cause issues when using Serena with the standard `stdio` protocol, where the MCP server is started by the client application.
If you experience such problems, you can start Serena in `sse` mode by running, e.g.,
```shell
uv run serena-mcp-server --transport sse --port 9121
```
Then configure your client to connect to `http://localhost:9121`.
ℹ️ You can further customize the run command using additional arguments (see [above](#command-line-arguments)).
Note: on Windows and macOS there are official Claude Desktop applications by Anthropic, for Linux there is an [open-source
community version](https://github.com/aaddrick/claude-desktop-debian).
@@ -298,35 +406,14 @@ community version](https://github.com/aaddrick/claude-desktop-debian).
After restarting, you should see Serena's tools in your chat interface (notice the small hammer icon).
⚠️ Tool Names: Claude Desktop (and most MCP Clients) don't resolve the name of the server. So you shouldn't
say something like "use Serena's tools". Instead, you can instruct the LLM to use symbolic tools or to
use a particular tool by referring to its name. Moreover, in some clients, if you use multiple MCP Servers, you might get
**tool name collisions** which lead to undefined behavior.
ℹ️ Note that MCP servers which use stdio as a protocol are somewhat unusual as far as client/server architectures go, as the server
necessarily has to be started by the client in order for communication to take place via the server's standard input/output stream.
In other words, you do not need to start the server yourself. The client application (e.g. Claude Desktop) takes care of this and
therefore needs to be configured with a launch command. In SSE transport you control the lifetime of the server yourself.
For more information on MCP servers with Claude Desktop, see [the official quick start guide](https://modelcontextprotocol.io/quickstart/user).
### Claude Code
Serena is a great way to make Claude Code both cheaper and more powerful! We are collecting
several examples for that and have heard very positive feedback so far. Claude Code users can
add serena with
```shell
claude mcp add serena -- /path/to/uv "run" --directory /path/to/serena serena-mcp-server --context ide-assistant
```
### Other MCP Clients (Cline, Roo-Code, Cursor, Windsurf, etc.)
Being an MCP Server, Serena can be included in any MCP Client. The same configuration as above,
perhaps with small client-specific modifications, should work. Most of the popular
existing coding assistants (IDE extensions or VSCode-like IDEs) support connections
to MCP Servers. It is ** recommended to use the `ide-assistant` context** for these integrations by adding `"--context", "ide-assistant"` to the `args` in your MCP client's configuration. Including Serena generally boosts their performance
to MCP Servers. It is **recommended to use the `ide-assistant` context** for these integrations by adding `"--context", "ide-assistant"` to the `args` in your MCP client's configuration. Including Serena generally boosts their performance
by providing them tools for symbolic operations.
In this case, the billing for the usage continues to be controlled by the client of your choice
@@ -334,42 +421,14 @@ In this case, the billing for the usage continues to be controlled by the client
e.g., for one of the following reasons:
1. You are already using a coding assistant (say Cline or Cursor) and just want to make it more powerful.
2. You are on Linux and don't want to use the [community-created Claude Desktop](https://github.com/aaddrick/claude-desktop-debian)
3. You want tighter integration of Serena into your IDE and don't mind paying for that
### Goose
[goose](https://github.com/block/goose) is a standalone coding agent which has an integration for MCP servers and offers a CLI (as well as a GUI on macOS).
Using goose is currently the simplest way of running Serena through a CLI-based UI with an LLM of your choice.
Follow the instructions [here](https://block.github.io/goose/docs/getting-started/installation/) to install it.
After that, use `goose configure` to add an extension. For adding Serena, choose the option `Command-line Extension`, name it `Serena` and add the following as command:
```
/abs/path/to/uv run --directory /abs/path/to/serena serena-mcp-server --project /optional/abs/path/to/project
```
Since Serena can do all necessary editing and command operations, you should disable the `developer` extension that goose enables by default.
For that execute
```shell
goose configure
```
again, choose the option `Toggle Extensions`, and make sure Serena is enabled selected while `developer` is not.
That's it. Read through the configuration options of goose to see what you can do with it (which is a lot, like setting different levels of permissions for tool execution).
> Goose does not seem to always properly terminate python processes for MCP servers when a session ends.
> You may want to disable the Serena GUI and/or to manually cleanup any running python processes after finishing your work
> with goose.
2. You are on Linux and don't want to use the [community-created Claude Desktop](https://github.com/aaddrick/claude-desktop-debian).
3. You want tighter integration of Serena into your IDE and don't mind paying for that.
### Agno Agent
Agno is a model-agnostic agent framework that allows you to turn Serena into an agent
(independent of the MCP technology) with a large number of underlying LLMs. Agno is currently
the simplest way of running Serena in a chat GUI with an LLM of your choice
(unless you are using a Mac, then you might prefer goose, which requires almost no setup).
the simplest way of running Serena in a chat GUI with an LLM of your choice.
While Agno is not yet entirely stable, we chose it, because it comes with its own open-source UI,
making it easy to directly use the agent using a chat interface. With Agno, Serena is turned into an agent
@@ -399,7 +458,7 @@ Here's how it works (see also [Agno's documentation](https://docs.agno.com/intro
3. Copy `.env.example` to `.env` and fill in the API keys for the provider(s) you
intend to use.
5. Start the agno agent app with
4. Start the agno agent app with
```shell
uv run python scripts/agno_agent.py
```
@@ -653,9 +712,7 @@ platform and from client to client. We recommend always using absolute paths, as
errors. The language server is running in a separate sub-process and is called with asyncio – sometimes
a client may make it crash. If you have Serena's log window enabled, and it disappears, you'll know what happened.
Some clients (like goose) may not properly terminate MCP servers,
look out for hanging python processes and terminate them
manually, if needed.
Some clients may not properly terminate MCP servers, look out for hanging python processes and terminate them manually, if needed.
## Comparison with Other Coding Agents
@@ -725,17 +782,16 @@ larger codebases.
We built Serena on top of multiple existing open-source technologies, the most important ones being:
1. [multilspy](https://github.com/microsoft/multilspy).
A beautifully designed wrapper around language servers following the LSP. It
was not easily extendable with the symbolic
logic that Serena required, so instead of incorporating it as dependency, we
copied the source code
and adapted it to our needs.
A library which wraps language server implementations and adapts them for interaction via Python
and which provided the basis for our library Solid-LSP (src/solidlsp).
Solid-LSP provides pure synchronous LSP calls and extends the original library with the symbolic logic
that Serena required.
2. [Python MCP SDK](https://github.com/modelcontextprotocol/python-sdk)
3. [Agno](https://github.com/agno-agi/agno) and
the associated [agent-ui](https://github.com/agno-agi/agent-ui),
which we use to allow Serena to work with any model, beyond the ones
supporting the MCP.
4. All the language servers that we use through multilspy.
4. All the language servers that we use through Solid-LSP.
Without these projects, Serena would not have been possible (or would have been significantly more difficult to build).
compose.yaml
+32
View File
@@ -0,0 +1,32 @@
services:
serena:
image: serena:latest
build:
context: ./
dockerfile: Dockerfile
target: production
ports:
- "${SERENA_PORT:-9121}:9121" # MCP server port
- "${SERENA_DASHBOARD_PORT:-24282}:24282" # Dashboard port (default 0x5EDA = 24282)
environment:
- SERENA_DOCKER=1
command:
- "uv run --directory . serena-mcp-server --transport sse --port 9121 --host 0.0.0.0"
serena-dev:
image: serena:dev
build:
context: ./
dockerfile: Dockerfile
target: development
tty: true
stdin_open: true
environment:
- SERENA_DOCKER=1
volumes:
- .:/workspaces/serena
ports:
- "${SERENA_PORT:-9121}:9121" # MCP server port
- "${SERENA_DASHBOARD_PORT:-24282}:24282" # Dashboard port
command:
- "uv run --directory . serena-mcp-server"
lessons_learned.md
+9 -1
View File
@@ -56,7 +56,7 @@ When developing the `ReplaceRegexTool` we were initially not able to make Claude
examples nor explicit instructions helped. It was only after adding
```
IMPORTANT: REMEMBER TO USE WILDCARDS WEHEN APPROPRIATE! I WILL BE VERY UNHAPPY IF YOU WRITE LONG REGEXES WITHOUT USING WILDCARDS INSTEAD!
IMPORTANT: REMEMBER TO USE WILDCARDS WHEN APPROPRIATE! I WILL BE VERY UNHAPPY IF YOU WRITE LONG REGEXES WITHOUT USING WILDCARDS INSTEAD!
```
to the initial instructions and to the tool description that Claude finally started following the instructions.
@@ -70,6 +70,14 @@ many clients, including Claude Desktop, fail to properly clean up, leaving zombi
We mitigate this through the GUI window and the dashboard, so the user sees whether Serena is running
and can terminate it there.
### Trusting Asyncio
Running multiple asyncio apps led to non-deterministic
event loop contamination and deadlocks, which were very hard to debug
and understand. We solved this with a large hammer, by putting all asyncio apps into a separate
process. It made the code much more complex and slightly enhanced RAM requirements, but it seems
like that was the only way to reliably overcome asyncio deadlock issues.
### Cross-OS Tkinter GUI
Different OS have different limitations when it comes to starting a window or dealing with Tkinter
pyproject.toml
+54 -63
View File
@@ -1,21 +1,17 @@
[build-system]
build-backend = "hatchling.build"
requires = [
"hatchling"
]
requires = ["hatchling"]
[project]
name = "serena"
version = "0.1.0"
description = ""
authors = [
{name = "Oraios AI", email = "info@oraios-ai.de"}
]
authors = [{ name = "Oraios AI", email = "info@oraios-ai.de" }]
readme = "README.md"
requires-python = ">=3.11, <3.12"
classifiers = [
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3.11"
"Programming Language :: Python :: 3.11",
]
dependencies = [
"requests>=2.32.3,<3",
@@ -23,8 +19,7 @@ dependencies = [
"overrides>=7.7.0,<8",
"python-dotenv>=1.0.0, <2",
"mcp>=1.5.0",
"fastapi>=0.115.12",
"fastmcp>=0.4.1",
"flask>=3.0.0",
"sensai-utils>=1.4.0",
"pydantic>=2.10.6",
"types-pyyaml>=6.0.12.20241230",
@@ -36,10 +31,12 @@ dependencies = [
"psutil>=7.0.0",
"docstring_parser>=0.16",
"joblib>=1.5.1",
"tqdm>=4.67.1",
]
[project.scripts]
serena-mcp-server = "serena.mcp:start_mcp_server"
index-project = "serena.agent:index_project"
[project.license]
text = "MIT"
@@ -57,28 +54,24 @@ dev = [
"types-pyyaml>=6.0.12.20241230",
"syrupy>=4.9.1",
]
agno = [
"agno>=1.2.6",
"sqlalchemy>=2.0.40",
]
anthropic = [
"anthropic>=0.49.0",
]
google = [
"google-genai>=1.8.0",
]
agno = ["agno>=1.2.6", "sqlalchemy>=2.0.40"]
anthropic = ["anthropic>=0.49.0"]
google = ["google-genai>=1.8.0"]
[project.urls]
Homepage = "https://github.com/oraios/serena"
[tool.hatch.build.targets.wheel]
packages = ["src/serena", "src/multilspy", "src/interprompt"]
packages = ["src/serena", "src/interprompt", "src/solidlsp"]
[tool.black]
line-length = 140
target-version = [
"py311"
]
target-version = ["py311"]
exclude = '''
/(
src/solidlsp/language_servers/.*/static|src/multilspy
)/
'''
[tool.doc8]
max-line-length = 1000
@@ -100,34 +93,30 @@ warn_no_return = true
warn_redundant_casts = true
warn_unreachable = true
warn_unused_configs = true
warn_unused_ignores = true
warn_unused_ignores = false
exclude = "^build/|^docs/"
[tool.poe.env]
PYDEVD_DISABLE_FILE_VALIDATION = "1"
[tool.poe.tasks]
test = "pytest test --color=yes"
_black_check = "black --check --exclude src/multilspy/ src scripts test"
_ruff_check = "ruff check --exclude .venv/ --exclude src/multilspy/ src scripts test"
_black_format = "black --exclude .venv/|src/multilspy/ src scripts test"
_ruff_format = "ruff check --exclude src/multilspy/ --fix src scripts test"
lint = [
"_black_check",
"_ruff_check",
]
format = [
"_ruff_format",
"_black_format"
]
# Uses PYTEST_MARKERS env var for default markers
# For custom markers, one can either adjust the env var or just use -m option in the command line,
# as the second -m option will override the first one.
test = "pytest test -vv -m \"${PYTEST_MARKERS:-not java and not rust and not isolated_process}\""
_black_check = "black --check src scripts test"
_ruff_check = "ruff check src scripts test"
_black_format = "black src scripts test"
_ruff_format = "ruff check --fix src scripts test"
lint = ["_black_check", "_ruff_check"]
format = ["_ruff_format", "_black_format"]
_mypy = "mypy src/serena"
type-check = [
"_mypy",
]
type-check = ["_mypy"]
[tool.ruff]
target-version = "py311"
line-length = 140
exclude = ["src/solidlsp/language_servers/**/static", "src/multilspy"]
[tool.ruff.format]
quote-style = "double"
@@ -162,7 +151,7 @@ select = [
"TID",
"UP",
"W",
"YTT"
"YTT",
]
ignore = [
"RUF002",
@@ -222,33 +211,27 @@ ignore = [
"W291",
"W293",
"B009",
"SIM103", # forbids multiple returns
"SIM110", # requires use of any(...) instead of for-loop
"G001", # forbids str.format in log statements
"E722", # forbids unspecific except clause
]
unfixable = [
"F841",
"F601",
"F602",
"B018"
]
extend-fixable = [
"F401",
"B905",
"W291"
"SIM103", # forbids multiple returns
"SIM110", # requires use of any(...) instead of for-loop
"G001", # forbids str.format in log statements
"E722", # forbids unspecific except clause
"SIM105", # forbids empty/general except clause
"SIM113", # wants to enforce use of enumerate
"E712", # forbids equality comparison with True/False
"UP007", # forbids some uses of Union
"TID252", # forbids relative imports
"B904", # forces use of raise from other_exception
"RUF012", # forbids mutable attributes as ClassVar
]
unfixable = ["F841", "F601", "F602", "B018"]
extend-fixable = ["F401", "B905", "W291"]
[tool.ruff.lint.mccabe]
max-complexity = 20
[tool.ruff.lint.per-file-ignores]
"tests/**" = [
"D103"
]
"scripts/**" = [
"D103"
]
"tests/**" = ["D103"]
"scripts/**" = ["D103"]
[tool.pytest.ini_options]
markers = [
@@ -259,5 +242,13 @@ markers = [
"rust: language server running for Rust",
"typescript: language server running for TypeScript",
"php: language server running for PHP",
"snapshot: snapshot tests",
"snapshot: snapshot tests for symbolic editing operations",
"isolated_process: test runs with process isolated agent",
]
[tool.codespell]
# Ref: https://github.com/codespell-project/codespell#using-a-config-file
skip = '.git*,*.svg,*.lock,*.min.*'
check-hidden = true
# ignore-regex = ''
# ignore-words-list = ''
pytest.ini
+2
View File
@@ -0,0 +1,2 @@
[pytest]
addopts = --snapshot-patch-pycharm-diff
scripts/demo_run_tools.py
+2 -2
View File
@@ -20,10 +20,10 @@ class InMemorySerenaConfig(SerenaConfigBase):
if __name__ == "__main__":
# project_path = str(Path("test") / "resources" / "repos" / "python" / "test_repo")
agent = SerenaAgent(project=REPO_ROOT)
# apply a tool
find_refs_tool = agent.get_tool(FindReferencingSymbolsTool)
print("Finding the symbol 'SyncLanguageServer'\n")
pprint(json.loads(find_refs_tool.apply(name_path="SyncLanguageServer", relative_path="src/multilspy/language_server.py")))
result = agent.execute_task(lambda: find_refs_tool.apply(name_path="SolidLanguageServer", relative_path="src/solidlsp/ls.py"))
pprint(json.loads(result))
src/README.md
+1 -1
View File
@@ -1,4 +1,4 @@
Serena uses (modified) versions of other libraries/packages:
* [multilspy](https://github.com/oraios/multilspy) (for language server protocol support); original repo: microsoft/multilspy
* solidlsp (our fork of [microsoft/multilspy](https://github.com/microsoft/multilspy) for fully synchronous language server communication)
* [interprompt](https://github.com/oraios/interprompt) (our prompt templating library)
src/multilspy/.syncCommitId.remote
-1
View File
@@ -1 +0,0 @@
606542ed8766bfbcc6490b3115eb0aea78c693e5
src/multilspy/.syncCommitId.this
-1
View File
@@ -1 +0,0 @@
2f5651bd7846ebb6abde121223233bf975ce98ee
src/multilspy/__init__.py
-8
View File
@@ -1,8 +0,0 @@
"""
This module contains the multilspy API
"""
from . import multilspy_types as Types
from .language_server import LanguageServer, SyncLanguageServer
__all__ = ["LanguageServer", "Types", "SyncLanguageServer"]
src/multilspy/type_helpers.py
-27
View File
@@ -1,27 +0,0 @@
"""
This module provides type-helpers used across multilspy implementation
"""
import inspect
from typing import Callable, TypeVar, Type
R = TypeVar("R", bound=object)
def ensure_all_methods_implemented(
source_cls: Type[object],
) -> Callable[[Type[R]], Type[R]]:
"""
A decorator to ensure that all methods of source_cls class are implemented in the decorated class.
"""
def check_all_methods_implemented(target_cls: R) -> R:
for name, _ in inspect.getmembers(source_cls, inspect.isfunction):
if name.startswith("_"):
continue
if name not in target_cls.__dict__ or not callable(target_cls.__dict__[name]):
raise NotImplementedError(f"{name} is not implemented in {target_cls}")
return target_cls
return check_all_methods_implemented
src/serena/__init__.py
+12 -9
View File
@@ -1,20 +1,23 @@
__version__ = "2025-05-21"
__version__ = "2025-06-21"
import logging
log = logging.getLogger(__name__)
def serena_version() -> str:
"""
:return: the version of the package, including git status if available.
"""
from serena.util.git import get_git_status
version = __version__
try:
from sensai.util.git import git_status
from sensai.util.logging import LoggingDisabledContext
with LoggingDisabledContext():
git_status = git_status()
version += f"-{git_status.commit[:8]}"
if not git_status.is_clean:
version += "-dirty"
git_status = get_git_status()
if git_status is not None:
version += f"-{git_status.commit[:8]}"
if not git_status.is_clean:
version += "-dirty"
except:
pass
return version
src/serena/agent.py
+590 -208
View File
File diff suppressed because it is too large Load Diff
src/serena/agno.py
+2 -2
View File
@@ -25,7 +25,7 @@ class SerenaAgnoToolkit(Toolkit):
def __init__(self, serena_agent: SerenaAgent):
super().__init__("Serena")
for tool in serena_agent.get_exposed_tool_instances():
self.functions[tool.get_name()] = self._create_agno_function(tool)
self.functions[tool.get_name_from_cls()] = self._create_agno_function(tool)
log.info("Agno agent functions: %s", list(self.functions.keys()))
@staticmethod
@@ -39,7 +39,7 @@ class SerenaAgnoToolkit(Toolkit):
return tool.apply_ex(log_call=True, catch_exceptions=True, **kwargs)
function = Function.from_callable(tool.get_apply_fn())
function.name = tool.get_name()
function.name = tool.get_name_from_cls()
function.entrypoint = entrypoint
function.skip_entrypoint_processing = True
return function
src/serena/config.py
+57 -1
View File
@@ -3,7 +3,9 @@ Context and Mode configuration loader
"""
import os
from dataclasses import dataclass, field
from copy import copy
from dataclasses import asdict, dataclass, field
from enum import Enum
from pathlib import Path
from typing import TYPE_CHECKING, Self
@@ -30,6 +32,17 @@ class SerenaAgentMode:
description: str = ""
excluded_tools: set[str] = field(default_factory=set)
def to_json_dict(self) -> dict[str, str | list[str]]:
result = asdict(self)
result["excluded_tools"] = list(result["excluded_tools"])
return result
@classmethod
def from_json_dict(cls, data: dict) -> Self:
data = copy(data)
data["excluded_tools"] = set(data["excluded_tools"])
return cls(**data)
def print_overview(self) -> None:
"""Print an overview of the mode."""
print(f"{self.name}:\n {self.description}")
@@ -91,6 +104,17 @@ class SerenaAgentContext:
description: str = ""
excluded_tools: set[str] = field(default_factory=set)
def to_json_dict(self) -> dict[str, str | list[str]]:
result = asdict(self)
result["excluded_tools"] = list(result["excluded_tools"])
return result
@classmethod
def from_json_dict(cls, data: dict) -> Self:
data = copy(data)
data["excluded_tools"] = set(data["excluded_tools"])
return cls(**data)
def get_excluded_tool_classes(self) -> list[type["Tool"]]:
"""Get the list of tool classes that are excluded from the context."""
from serena.agent import ToolRegistry
@@ -144,3 +168,35 @@ class SerenaAgentContext:
print(f"{self.name}:\n {self.description}")
if self.excluded_tools:
print(" excluded tools:\n " + ", ".join(sorted(self.excluded_tools)))
class RegisteredContext(Enum):
"""A registered context."""
IDE_ASSISTANT = "ide-assistant"
"""For Serena running within an assistant that already has basic tools, like Claude Code, Cline, Cursor, etc."""
DESKTOP_APP = "desktop-app"
"""For Serena running within Claude Desktop or a similar app which does not have built-in tools for code editing."""
AGENT = "agent"
"""For Serena running as a standalone agent, e.g. through agno."""
def load(self) -> SerenaAgentContext:
"""Load the context."""
return SerenaAgentContext.from_name(self.value)
class RegisteredMode(Enum):
"""A registered mode."""
INTERACTIVE = "interactive"
"""Interactive mode, for multi-turn interactions."""
EDITING = "editing"
"""Editing tools are activated."""
PLANNING = "planning"
"""Editing tools are deactivated."""
ONE_SHOT = "one-shot"
"""Non-interactive mode, where the goal is to finish a task autonomously."""
def load(self) -> SerenaAgentMode:
"""Load the mode."""
return SerenaAgentMode.from_name(self.value)
src/serena/constants.py
+5
View File
@@ -12,8 +12,13 @@ SERENA_ICON_DIR = str(_serena_pkg_path / "resources" / "icons")
SERENA_MANAGED_DIR_NAME = ".serena"
DEFAULT_ENCODING = "utf-8"
DEFAULT_CONTEXT = "desktop-app"
DEFAULT_MODES = ("interactive", "editing")
PROJECT_TEMPLATE_FILE = str(_serena_pkg_path / "resources" / "project.template.yml")
SELENA_CONFIG_TEMPLATE_FILE = str(_serena_pkg_path / "resources" / "serena_config.template.yml")
USE_PROCESS_ISOLATION = False
SERENA_LOG_FORMAT = "%(levelname)-5s %(asctime)-15s [%(threadName)s] %(name)s:%(funcName)s:%(lineno)d - %(message)s"
src/serena/dashboard.py
+66 -22
View File
@@ -1,24 +1,26 @@
import os
import queue
import socket
import sys
import threading
from collections.abc import Callable
from typing import Any
import uvicorn
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from flask import Flask, Response, request, send_from_directory
from pydantic import BaseModel
from sensai.util import logging
from serena.constants import SERENA_DASHBOARD_DIR
from serena.constants import SERENA_DASHBOARD_DIR, SERENA_LOG_FORMAT
log = logging.getLogger(__name__)
# disable Werkzeug's logging to avoid cluttering the output
logging.getLogger("werkzeug").setLevel(logging.WARNING)
class MemoryLogHandler(logging.Handler):
def __init__(self, level: int = logging.NOTSET) -> None:
super().__init__(level=level)
self.setFormatter(logging.Formatter(logging.LOG_DEFAULT_FORMAT))
self.setFormatter(logging.Formatter(SERENA_LOG_FORMAT))
self._log_buffer = LogBuffer()
self._log_queue: queue.Queue[str] = queue.Queue()
self._stop_event = threading.Event()
@@ -68,33 +70,70 @@ class ResponseToolNames(BaseModel):
class SerenaDashboardAPI:
log = logging.getLogger(__qualname__)
def __init__(self, memory_log_handler: MemoryLogHandler, tool_names: list[str]) -> None:
def __init__(
self, memory_log_handler: MemoryLogHandler, tool_names: list[str], shutdown_callback: Callable[[], None] | None = None
) -> None:
self._memory_log_handler = memory_log_handler
self._tool_names = tool_names
self._app = FastAPI(title="Serena Dashboard")
self._shutdown_callback = shutdown_callback
self._app = Flask(__name__)
self._setup_routes()
@property
def memory_log_handler(self) -> MemoryLogHandler:
return self._memory_log_handler
def _setup_routes(self) -> None:
self._app.mount("/dashboard", StaticFiles(directory=SERENA_DASHBOARD_DIR), name="dashboard")
# Static files
@self._app.route("/dashboard/<path:filename>")
def serve_dashboard(filename: str) -> Response:
return send_from_directory(SERENA_DASHBOARD_DIR, filename)
self._app.add_api_route("/get_log_messages", self._get_log_messages, methods=["POST"], response_model=ResponseLog)
self._app.add_api_route("/get_tool_names", self._get_tool_names, methods=["GET"], response_model=ResponseToolNames)
self._app.add_api_route("/shutdown", self._shutdown, methods=["PUT"])
@self._app.route("/dashboard/")
def serve_dashboard_index() -> Response:
return send_from_directory(SERENA_DASHBOARD_DIR, "index.html")
async def _get_log_messages(self, request: RequestLog) -> ResponseLog:
# API routes
@self._app.route("/get_log_messages", methods=["POST"])
def get_log_messages() -> dict[str, Any]:
request_data = request.get_json()
if not request_data:
request_log = RequestLog()
else:
request_log = RequestLog.model_validate(request_data)
result = self._get_log_messages(request_log)
return result.model_dump()
@self._app.route("/get_tool_names", methods=["GET"])
def get_tool_names() -> dict[str, Any]:
result = self._get_tool_names()
return result.model_dump()
@self._app.route("/shutdown", methods=["PUT"])
def shutdown() -> dict[str, str]:
self._shutdown()
return {"status": "shutting down"}
def _get_log_messages(self, request_log: RequestLog) -> ResponseLog:
all_messages = self._memory_log_handler.get_log_messages()
requested_messages = all_messages[request.start_idx :] if request.start_idx <= len(all_messages) else []
requested_messages = all_messages[request_log.start_idx :] if request_log.start_idx <= len(all_messages) else []
return ResponseLog(messages=requested_messages, max_idx=len(all_messages) - 1)
async def _get_tool_names(self) -> ResponseToolNames:
def _get_tool_names(self) -> ResponseToolNames:
return ResponseToolNames(tool_names=self._tool_names)
async def _shutdown(self) -> None:
print("Shutdown initiated by dashbaord ...", file=sys.stderr)
def _shutdown(self) -> None:
log.info("Shutting down Serena")
# noinspection PyUnresolvedReferences
# noinspection PyProtectedMember
os._exit(0)
if self._shutdown_callback:
self._shutdown_callback()
else:
# Try to use the global shutdown function from process_isolated_agent
from serena.process_isolated_agent import request_global_shutdown
request_global_shutdown()
# noinspection PyProtectedMember
os._exit(0)
@staticmethod
def _find_first_free_port(start_port: int) -> int:
@@ -109,11 +148,16 @@ class SerenaDashboardAPI:
raise RuntimeError(f"No free ports found starting from {start_port}")
def run(self, host: str = "127.0.0.1", port: int = 0x5EDA) -> int:
def run(self, host: str = "0.0.0.0", port: int = 0x5EDA) -> int:
"""
Runs the dashboard on the given host and port and returns the port number.
"""
uvicorn.run(self._app, host=host, port=port, workers=1, log_config=None, log_level="critical")
# patch flask.cli.show_server to avoid printing the server info
from flask import cli
cli.show_server_banner = lambda *args, **kwargs: None
self._app.run(host=host, port=port, debug=False, use_reloader=False, threaded=True)
return port
def run_in_thread(self) -> tuple[threading.Thread, int]:
src/serena/mcp.py
+417 -129
View File
@@ -2,26 +2,48 @@
The Serena Model Context Protocol (MCP) Server
"""
import asyncio
import contextlib
import os
import signal
import sys
from collections.abc import AsyncIterator, Sequence
import threading
import time
from abc import abstractmethod
from collections.abc import AsyncIterator, Iterator, Sequence
from contextlib import asynccontextmanager
from dataclasses import dataclass
from logging import Formatter, Logger, StreamHandler
from pathlib import Path
from typing import Any, Literal
import click # Add click import
import click
import docstring_parser
from mcp.server.fastmcp import server
from mcp.server.fastmcp.server import FastMCP, Settings
from mcp.server.fastmcp.tools.base import Tool as MCPTool
from mcp.server.fastmcp.utilities.func_metadata import func_metadata
from pydantic_settings import SettingsConfigDict
from sensai.util import logging
from sensai.util.helper import mark_used
from serena.agent import SerenaAgent, Tool, show_fatal_exception_safe
from serena.config import SerenaAgentContext, SerenaAgentMode
from serena.constants import DEFAULT_CONTEXT, DEFAULT_MODES
from serena.agent import (
ActivateProjectTool,
Project,
SerenaAgent,
SerenaConfig,
ToolInterface,
ToolRegistry,
create_serena_config,
show_fatal_exception_safe,
)
from serena.config import RegisteredContext, SerenaAgentContext, SerenaAgentMode
from serena.constants import DEFAULT_CONTEXT, DEFAULT_MODES, USE_PROCESS_ISOLATION
from serena.process_isolated_agent import (
ProcessIsolatedDashboard,
ProcessIsolatedSerenaAgent,
ProcessIsolatedTool,
global_shutdown_event,
request_global_shutdown,
)
log = logging.getLogger(__name__)
LOG_FORMAT = "%(levelname)-5s %(asctime)-15s %(name)s:%(funcName)s:%(lineno)d - %(message)s"
@@ -45,134 +67,390 @@ class SerenaMCPRequestContext:
agent: SerenaAgent
def make_tool(
tool: Tool,
) -> MCPTool:
func_name = tool.get_name()
class SerenaMCPFactory:
def __init__(self, context: str = DEFAULT_CONTEXT, project: str | None = None):
"""
:param context: The context name or path to context file
:param project: Either an absolute path to the project directory or a name of an already registered project.
If the project passed here hasn't been registered yet, it will be registered automatically and can be activated by its name
afterward.
"""
self.context = SerenaAgentContext.load(context)
self.project = project
apply_fn = getattr(tool, "apply")
if apply_fn is None:
raise ValueError(f"Tool does not have an apply method: {tool}")
@staticmethod
def make_mcp_tool(tool: ToolInterface) -> MCPTool:
func_name = tool.get_name()
func_doc = tool.get_apply_docstring() or ""
func_arg_metadata = tool.get_apply_fn_metadata()
is_async = False
parameters = func_arg_metadata.arg_model.model_json_schema()
func_doc = apply_fn.__doc__ or ""
is_async = False
docstring = docstring_parser.parse(func_doc)
func_arg_metadata = func_metadata(apply_fn)
parameters = func_arg_metadata.arg_model.model_json_schema()
# Mount the tool description as a combination of the docstring description and
# the return value description, if it exists.
if docstring.description:
func_doc = f"{docstring.description.strip().strip('.')}."
else:
func_doc = ""
if docstring.returns and (docstring_returns_descr := docstring.returns.description):
# Only add a space before "Returns" if func_doc is not empty
prefix = " " if func_doc else ""
func_doc = f"{func_doc}{prefix}Returns {docstring_returns_descr.strip().strip('.')}."
docstring = docstring_parser.parse(func_doc)
# Parse the parameter descriptions from the docstring and add pass its description
# to the parameter schema.
docstring_params = {param.arg_name: param for param in docstring.params}
parameters_properties: dict[str, dict[str, Any]] = parameters["properties"]
for parameter, properties in parameters_properties.items():
if (param_doc := docstring_params.get(parameter)) and param_doc.description:
param_desc = f"{param_doc.description.strip().strip('.') + '.'}"
properties["description"] = param_desc[0].upper() + param_desc[1:]
# Mount the tool description as a combination of the docstring description and
# the return value description, if it exists.
if docstring.description:
func_doc = f"{docstring.description.strip().strip('.')}."
else:
func_doc = ""
if (docstring.returns) and (docstring_returns := docstring.returns.description):
# Only add a space before "Returns" if func_doc is not empty
prefix = " " if func_doc else ""
func_doc = f"{func_doc}{prefix}Returns {docstring_returns.strip().strip('.')}."
def execute_fn(**kwargs) -> str: # type: ignore
return tool.apply_ex(log_call=True, catch_exceptions=True, **kwargs)
# Parse the parameter descriptions from the docstring and add pass its description
# to the parameters schema.
docstring_params = {param.arg_name: param for param in docstring.params}
parameters_properties: dict[str, dict[str, Any]] = parameters["properties"]
for parameter, properties in parameters_properties.items():
if (param_doc := docstring_params.get(parameter)) and (param_doc.description):
param_desc = f"{param_doc.description.strip().strip('.') + '.'}"
properties["description"] = param_desc[0].upper() + param_desc[1:]
def execute_fn(**kwargs) -> str: # type: ignore
return tool.apply_ex(log_call=True, catch_exceptions=True, **kwargs)
return MCPTool(
fn=execute_fn,
name=func_name,
description=func_doc,
parameters=parameters,
fn_metadata=func_arg_metadata,
is_async=is_async,
context_kwarg=None,
)
def create_mcp_server_and_agent(
project: str | None,
host: str = "0.0.0.0",
port: int = 8000,
context: str = DEFAULT_CONTEXT,
modes: Sequence[str] = DEFAULT_MODES,
enable_web_dashboard: bool | None = None,
enable_gui_log_window: bool | None = None,
log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None,
trace_lsp_communication: bool | None = None,
) -> tuple[FastMCP, SerenaAgent]:
"""
Create an MCP server.
:param project: "Either an absolute path to the project directory or a name of an already registered project. "
"If the project passed here hasn't been registered yet, it will be registered automatically and can be activated by its name "
"afterwards.
:param host: The host to bind to
:param port: The port to bind to
:param context: The context name or path to context file
:param modes: List of mode names or paths to mode files
:param enable_web_dashboard: Whether to enable the web dashboard. If not specified, will take the value from the serena configuration.
:param enable_gui_log_window: Whether to enable the GUI log window. It currently does not work on macOS, and setting this to True will be ignored then.
If not specified, will take the value from the serena configuration.
:param log_level: Log level. If not specified, will take the value from the serena configuration.
:param trace_lsp_communication: Whether to trace the communication between Serena and the language servers.
This is useful for debugging language server issues.
"""
mcp: FastMCP | None = None
context_instance = SerenaAgentContext.load(context)
modes_instances = [SerenaAgentMode.load(mode) for mode in modes]
try:
agent = SerenaAgent(
project=project,
# Callback disabled for the time being (see above)
# project_activation_callback=update_tools
context=context_instance,
modes=modes_instances,
enable_web_dashboard=enable_web_dashboard,
enable_gui_log_window=enable_gui_log_window,
log_level=log_level,
trace_lsp_communication=trace_lsp_communication,
return MCPTool(
fn=execute_fn,
name=func_name,
description=func_doc,
parameters=parameters,
fn_metadata=func_arg_metadata,
is_async=is_async,
context_kwarg=None,
annotations=None,
)
except Exception as e:
show_fatal_exception_safe(e)
raise
def update_tools() -> None:
"""Update the tools in the MCP server."""
# Tools may change as a result of project activation.
# NOTE: While we could pass updated tool information on to the MCP server via the callback, Claude Desktop does not,
# unfortunately, query for changed tools. It only queries for changed resources and prompts regularly,
# so we need to register all tools at startup, unfortunately.
nonlocal mcp, agent
tools = agent.get_exposed_tool_instances()
@abstractmethod
def _iter_tools(self) -> Iterator[ToolInterface]:
pass
# noinspection PyProtectedMember
def _set_mcp_tools(self, mcp: FastMCP) -> None:
"""Update the tools in the MCP server"""
if mcp is not None:
mcp._tool_manager._tools = {}
for tool in tools:
# noinspection PyProtectedMember
mcp._tool_manager._tools[tool.get_name()] = make_tool(tool)
for tool in self._iter_tools():
mcp_tool = self.make_mcp_tool(tool)
mcp._tool_manager._tools[tool.get_name()] = mcp_tool
@abstractmethod
def _instantiate_agent(self, serena_config: SerenaConfig, modes: list[SerenaAgentMode]) -> None:
pass
def create_mcp_server(
self,
host: str = "0.0.0.0",
port: int = 8000,
modes: Sequence[str] = DEFAULT_MODES,
enable_web_dashboard: bool | None = None,
enable_gui_log_window: bool | None = None,
log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None,
trace_lsp_communication: bool | None = None,
tool_timeout: float | None = None,
) -> FastMCP:
"""
Create an MCP server with process-isolated SerenaAgent to prevent asyncio contamination.
:param host: The host to bind to
:param port: The port to bind to
:param modes: List of mode names or paths to mode files
:param enable_web_dashboard: Whether to enable the web dashboard. If not specified, will take the value from the serena configuration.
:param enable_gui_log_window: Whether to enable the GUI log window. It currently does not work on macOS, and setting this to True will be ignored then.
If not specified, will take the value from the serena configuration.
:param log_level: Log level. If not specified, will take the value from the serena configuration.
:param trace_lsp_communication: Whether to trace the communication between Serena and the language servers.
This is useful for debugging language server issues.
:param tool_timeout: Timeout in seconds for tool execution. If not specified, will take the value from the serena configuration.
"""
try:
serena_config = create_serena_config(
enable_web_dashboard=enable_web_dashboard,
enable_gui_log_window=enable_gui_log_window,
log_level=log_level,
trace_lsp_communication=trace_lsp_communication,
tool_timeout=tool_timeout,
)
modes_instances = [SerenaAgentMode.load(mode) for mode in modes]
self._instantiate_agent(serena_config, modes_instances)
except Exception as e:
show_fatal_exception_safe(e)
raise
# Override model_config to disable the use of `.env` files for reading settings, because user projects are likely to contain
# `.env` files (e.g. containing LOG_LEVEL) that are not supposed to override the MCP settings;
# retain only FASTMCP_ prefix for already set environment variables.
Settings.model_config = SettingsConfigDict(env_prefix="FASTMCP_")
mcp_settings: Settings = Settings(lifespan=self.server_lifespan, host=host, port=port)
mcp = FastMCP(**mcp_settings.model_dump())
return mcp
@asynccontextmanager
async def server_lifespan(mcp_server: FastMCP) -> AsyncIterator[None]:
@abstractmethod
async def server_lifespan(self, mcp_server: FastMCP) -> AsyncIterator[None]:
"""Manage server startup and shutdown lifecycle."""
nonlocal agent
mark_used(mcp_server)
yield None # ensures MyPy understands we yield None
class SerenaMCPFactorySingleProcess(SerenaMCPFactory):
"""
MCP server factory where the SerenaAgent and its language server run in the same process as the MCP server
"""
def __init__(self, context: str = DEFAULT_CONTEXT, project: str | None = None):
"""
:param context: The context name or path to context file
:param project: Either an absolute path to the project directory or a name of an already registered project.
If the project passed here hasn't been registered yet, it will be registered automatically and can be activated by its name
afterward.
"""
super().__init__(context=context, project=project)
self.agent: SerenaAgent | None = None
def _instantiate_agent(self, serena_config: SerenaConfig, modes: list[SerenaAgentMode]) -> None:
self.agent = SerenaAgent(project=self.project, serena_config=serena_config, context=self.context, modes=modes)
def _iter_tools(self) -> Iterator[ToolInterface]:
assert self.agent is not None
yield from self.agent.get_exposed_tool_instances()
@asynccontextmanager
async def server_lifespan(self, mcp_server: FastMCP) -> AsyncIterator[None]:
self._set_mcp_tools(mcp_server)
log.info("MCP server lifetime setup complete")
yield
if agent.language_server is not None:
agent.language_server.stop()
mcp_settings = Settings(lifespan=server_lifespan, host=host, port=port)
mcp = FastMCP(**mcp_settings.model_dump())
update_tools()
class SerenaMCPFactoryWithProcessIsolation(SerenaMCPFactory):
"""
MCP server factory with process isolation for the SerenaAgent and its language server; they run in a separate process
from the MCP server.
"""
return mcp, agent
def __init__(self, context: str = DEFAULT_CONTEXT, project: str | None = None):
"""
:param context: The context name or path to context file
:param project: Either an absolute path to the project directory or a name of an already registered project.
If the project passed here hasn't been registered yet, it will be registered automatically and can be activated by its name
afterward.
"""
super().__init__(context=context, project=project)
self.active_tool_names: set[str] | None = None
self.serena_agent_process: ProcessIsolatedSerenaAgent | None = None
self.serena_dashboard_process: ProcessIsolatedDashboard | None = None
@staticmethod
def _determine_active_tool_names(context: SerenaAgentContext, project: Project | None) -> set[str]:
"""
Determine the names of tools that should be included in this session based on the context.
"""
tools_excluded_in_this_session = context.get_excluded_tool_classes()
# if a project has been loaded, it will be activated at startup and in ide-assistant context,
# we assume that no other project will be activated in this session.
# Therefore, we exclude the activate project tool
is_ide_assistant = context.name == RegisteredContext.IDE_ASSISTANT.value
if is_ide_assistant and project is not None:
tools_excluded_in_this_session.extend(project.project_config.get_excluded_tool_classes())
tools_excluded_in_this_session.append(ActivateProjectTool)
tool_names_excluded_in_this_session = {tool.get_name_from_cls() for tool in tools_excluded_in_this_session}
all_tool_names = set(ToolRegistry.get_tool_names())
tool_names_included_in_this_session = all_tool_names - tool_names_excluded_in_this_session
return tool_names_included_in_this_session
@staticmethod
def make_mcp_tool(tool: ToolInterface) -> MCPTool:
func_name = tool.get_name()
func_doc = tool.get_apply_docstring() or ""
func_arg_metadata = tool.get_apply_fn_metadata()
is_async = False
parameters = func_arg_metadata.arg_model.model_json_schema()
docstring = docstring_parser.parse(func_doc)
# Mount the tool description as a combination of the docstring description and
# the return value description, if it exists.
if docstring.description:
func_doc = f"{docstring.description.strip().strip('.')}."
else:
func_doc = ""
if docstring.returns and (docstring_returns_descr := docstring.returns.description):
# Only add a space before "Returns" if func_doc is not empty
prefix = " " if func_doc else ""
func_doc = f"{func_doc}{prefix}Returns {docstring_returns_descr.strip().strip('.')}."
# Parse the parameter descriptions from the docstring and add pass its description
# to the parameter schema.
docstring_params = {param.arg_name: param for param in docstring.params}
parameters_properties: dict[str, dict[str, Any]] = parameters["properties"]
for parameter, properties in parameters_properties.items():
if (param_doc := docstring_params.get(parameter)) and param_doc.description:
param_desc = f"{param_doc.description.strip().strip('.') + '.'}"
properties["description"] = param_desc[0].upper() + param_desc[1:]
def execute_fn(**kwargs) -> str: # type: ignore
return tool.apply_ex(log_call=True, catch_exceptions=True, **kwargs)
return MCPTool(
fn=execute_fn,
name=func_name,
description=func_doc,
parameters=parameters,
fn_metadata=func_arg_metadata,
is_async=is_async,
context_kwarg=None,
annotations=None,
)
def _iter_tools(self) -> Iterator[ToolInterface]:
assert self.active_tool_names is not None
assert self.serena_agent_process is not None
for tool_name in self.active_tool_names:
yield ProcessIsolatedTool(process_agent=self.serena_agent_process, tool_name=tool_name)
# noinspection PyProtectedMember
def _set_mcp_tools(self, mcp: FastMCP) -> None:
"""Update the tools in the MCP server"""
if mcp is not None:
mcp._tool_manager._tools = {}
for tool in self._iter_tools():
mcp_tool = self.make_mcp_tool(tool)
mcp._tool_manager._tools[tool.get_name()] = mcp_tool
def _instantiate_agent(self, serena_config: SerenaConfig, modes: list[SerenaAgentMode]) -> None:
if self.project is not None:
self.project_instance = serena_config.get_project(self.project)
self.serena_agent_process = ProcessIsolatedSerenaAgent(
project=self.project, serena_config=serena_config, modes=modes, context=self.context
)
self.active_tool_names = self._determine_active_tool_names(self.context, self.project_instance)
if serena_config.web_dashboard:
assert self.active_tool_names is not None
self.serena_dashboard_process = ProcessIsolatedDashboard(tool_names=sorted(self.active_tool_names))
def create_mcp_server(
self,
host: str = "0.0.0.0",
port: int = 8000,
modes: Sequence[str] = DEFAULT_MODES,
enable_web_dashboard: bool | None = None,
enable_gui_log_window: bool | None = None,
log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None,
trace_lsp_communication: bool | None = None,
tool_timeout: float | None = None,
) -> FastMCP:
"""
Create an MCP server with process-isolated SerenaAgent to prevent asyncio contamination.
:param host: The host to bind to
:param port: The port to bind to
:param modes: List of mode names or paths to mode files
:param enable_web_dashboard: Whether to enable the web dashboard. If not specified, will take the value from the serena configuration.
:param enable_gui_log_window: Whether to enable the GUI log window. It currently does not work on macOS, and setting this to True will be ignored then.
If not specified, will take the value from the serena configuration.
:param log_level: Log level. If not specified, will take the value from the serena configuration.
:param trace_lsp_communication: Whether to trace the communication between Serena and the language servers.
This is useful for debugging language server issues.
:param tool_timeout: Timeout in seconds for tool execution. If not specified, will take the value from the serena configuration.
"""
try:
serena_config = create_serena_config(
enable_web_dashboard=enable_web_dashboard,
enable_gui_log_window=enable_gui_log_window,
log_level=log_level,
trace_lsp_communication=trace_lsp_communication,
tool_timeout=tool_timeout,
)
modes_instances = [SerenaAgentMode.load(mode) for mode in modes]
self._instantiate_agent(serena_config, modes_instances)
except Exception as e:
show_fatal_exception_safe(e)
raise
# Override model_config to disable the use of `.env` files for reading settings, because user projects are likely to contain
# `.env` files (e.g. containing LOG_LEVEL) that are not supposed to override the MCP settings;
# retain only FASTMCP_ prefix for already set environment variables.
Settings.model_config = SettingsConfigDict(env_prefix="FASTMCP_")
mcp_settings = Settings(lifespan=self.server_lifespan, host=host, port=port)
mcp = FastMCP(**mcp_settings.model_dump())
return mcp
@asynccontextmanager
async def server_lifespan(self, mcp_server: FastMCP) -> AsyncIterator[None]:
"""Manage server startup and shutdown lifecycle."""
def signal_handler(signum: int, frame: Any) -> None:
log.info(f"Received signal {signum} in main process")
request_global_shutdown()
def force_exit() -> None:
time.sleep(2.0) # Wait 2 seconds for graceful shutdown
log.warning("Forcing exit after timeout")
# noinspection PyProtectedMember
# noinspection PyUnresolvedReferences
os._exit(1)
threading.Thread(target=force_exit, daemon=True).start()
# Install signal handlers
sigint_singal = signal.signal(signal.SIGINT, signal_handler)
sigterm_signal = signal.signal(signal.SIGTERM, signal_handler)
if self.serena_dashboard_process is not None:
log.info("Starting dashboard process")
assert self.serena_dashboard_process is not None
self.serena_dashboard_process.start()
log.info("Starting serena agent process")
assert self.serena_agent_process is not None
self.serena_agent_process.start()
self._set_mcp_tools(mcp_server)
async def monitor_global_shutdown() -> None:
"""Monitor the global shutdown event and trigger local shutdown."""
while not global_shutdown_event.is_set():
# Poll the multiprocessing Event in async context
await asyncio.sleep(0.1)
continue
log.info("Global shutdown event detected, initiating server shutdown")
request_global_shutdown()
# Send SIGTERM to self to trigger graceful shutdown
os.kill(os.getpid(), signal.SIGTERM)
# Start monitoring task
monitor_task = asyncio.create_task(monitor_global_shutdown())
log.info("MCP server lifetime setup complete")
try:
yield
except (KeyboardInterrupt, SystemExit):
log.info("Received shutdown signal")
request_global_shutdown()
except Exception as e:
log.error(f"Error in server lifespan: {e}")
request_global_shutdown()
finally:
# Cancel monitor task
monitor_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await monitor_task
self.serena_agent_process.stop()
if self.serena_dashboard_process is not None:
self.serena_dashboard_process.stop()
request_global_shutdown()
log.info("Shutting down all processes")
signal.signal(signal.SIGINT, sigint_singal)
signal.signal(signal.SIGTERM, sigterm_signal)
class ProjectType(click.ParamType):
@@ -191,7 +469,7 @@ PROJECT_TYPE = ProjectType()
@click.command()
@click.option(
"--project",
"project_file_opt",
"project",
type=PROJECT_TYPE,
default=None,
help="Either an absolute path to the project directory or a name of an already registered project. "
@@ -200,7 +478,7 @@ PROJECT_TYPE = ProjectType()
# Keep --project-file for backwards compatibility
@click.option(
"--project-file",
"project_file_opt", # Use same destination variable to avoid conflicts
"project", # Use same destination variable to avoid conflicts
type=PROJECT_TYPE,
default=None,
help="[DEPRECATED] Use --project instead.",
@@ -280,8 +558,14 @@ PROJECT_TYPE = ProjectType()
default=None,
help="Whether to trace the communication between Serena and the language servers. This is useful for debugging language server issues.",
)
@click.option(
"--tool-timeout",
type=float,
default=None,
help="Timeout in seconds for tool execution. If not specified, will take the value from the serena configuration.",
)
def start_mcp_server(
project_file_opt: str | None,
project: str | None,
project_file_arg: str | None,
context: str = DEFAULT_CONTEXT,
modes: tuple[str, ...] = DEFAULT_MODES,
@@ -292,6 +576,7 @@ def start_mcp_server(
enable_gui_log_window: bool | None = None,
log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None,
trace_lsp_communication: bool | None = None,
tool_timeout: float | None = None,
) -> None:
"""Starts the Serena MCP server. By default, will not activate any project at startup.
If you want to start with an already active project, use --project to pass the project name or path.
@@ -301,18 +586,24 @@ def start_mcp_server(
"""
# Prioritize the positional argument if provided
# This is for backward compatibility with the old CLI, should be removed in the future!
project_file = project_file_arg if project_file_arg is not None else project_file_opt
project_file = project_file_arg if project_file_arg is not None else project
mcp_server, agent = create_mcp_server_and_agent(
project=project_file,
mcp_factory: SerenaMCPFactory
if not USE_PROCESS_ISOLATION:
mcp_factory = SerenaMCPFactorySingleProcess(context=context, project=project_file)
else:
mcp_factory = SerenaMCPFactoryWithProcessIsolation(context=context, project=project_file)
# Use process isolation by default to prevent asyncio event loop contamination
mcp_server = mcp_factory.create_mcp_server(
host=host,
port=port,
context=context,
modes=modes,
enable_web_dashboard=enable_web_dashboard,
enable_gui_log_window=enable_gui_log_window,
log_level=log_level,
trace_lsp_communication=trace_lsp_communication,
tool_timeout=tool_timeout,
)
# log after server creation such that the log appears in the GUI
@@ -323,9 +614,6 @@ def start_mcp_server(
f"Used path: {project_file}"
)
log.info(
f"Starting serena agent in MCP server with config:\n{agent.get_current_config_overview()}."
f"\n Log level: {agent.serena_config.log_level}"
)
log.info("Starting MCP server ...")
mcp_server.run(transport=transport)
src/serena/process_isolated_agent.py
+571
View File
@@ -0,0 +1,571 @@
import asyncio
import contextlib
import logging
import multiprocessing
import os
import threading
import traceback
import webbrowser
from enum import StrEnum
from logging.handlers import QueueHandler
from multiprocessing.connection import Connection
from multiprocessing.sharedctypes import Synchronized
from multiprocessing.synchronize import Event as EventClass
from typing import Any, Literal, Self
from mcp.server.fastmcp.utilities.func_metadata import FuncMetadata
from serena.agent import SerenaAgent, SerenaConfig, SerenaConfigBase, Tool, ToolInterface, ToolRegistry
from serena.config import SerenaAgentContext, SerenaAgentMode
from serena.dashboard import MemoryLogHandler, SerenaDashboardAPI
log = logging.getLogger(__name__)
# Global synchronization primitives
_global_log_queue: multiprocessing.Queue = multiprocessing.Queue()
_dashboard_ready_event = multiprocessing.Event()
_dashboard_port_value = multiprocessing.Value("i", 0)
global_shutdown_event = multiprocessing.Event()
def request_global_shutdown() -> None:
"""Signal the global shutdown event."""
global_shutdown_event.set()
log.info("Global shutdown event set")
def _dashboard_worker(
tool_names: list[str],
log_q: "multiprocessing.Queue[Any]",
dashboard_ready_event: EventClass,
port_value: "Synchronized[int]",
shutdown_evt: EventClass,
) -> None:
"""Entry point for the dashboard process."""
# Route all logging to the shared queue
root = logging.getLogger()
root.handlers.clear()
root.setLevel(logging.DEBUG)
root.addHandler(QueueHandler(log_q))
async def _process_logs(api: SerenaDashboardAPI) -> None:
while not shutdown_evt.is_set():
while not log_q.empty():
record = log_q.get_nowait()
if record is None:
break
api.memory_log_handler.emit(record)
# Small delay to avoid busy waiting
await asyncio.sleep(0.1)
async def _monitor_shutdown() -> None:
# Poll the multiprocessing Event in async context
# Check every 100ms until shutdown is requested
loop = asyncio.get_event_loop()
while True:
# Check in executor to avoid blocking
result = await loop.run_in_executor(None, shutdown_evt.is_set)
if result:
break
await asyncio.sleep(0.1)
async def _async_main() -> None:
api = SerenaDashboardAPI(
memory_log_handler=MemoryLogHandler(),
tool_names=tool_names,
shutdown_callback=shutdown_evt.set,
)
# Pick a free port and signal readiness
port = api._find_first_free_port(0x5EDA)
port_value.value = port
# Start Flask server in a thread
def run_flask_server() -> None:
api._app.run(host="0.0.0.0", port=port, debug=False, use_reloader=False, threaded=True)
server_thread = threading.Thread(target=run_flask_server, daemon=True)
server_thread.start()
shutdown_task = asyncio.create_task(_monitor_shutdown())
logging_loop_task = asyncio.create_task(_process_logs(api))
dashboard_ready_event.set()
done, pending = await asyncio.wait(
[shutdown_task, logging_loop_task],
return_when=asyncio.FIRST_COMPLETED,
)
# Cancel remaining tasks
for task in pending:
task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await task
try:
asyncio.run(_async_main())
except BaseException:
logging.exception("Dashboard worker crashed")
finally:
logging.info("Dashboard worker exiting")
def _shutdown_process(proc: multiprocessing.Process, timeout: float = 1.0) -> None:
"""Helper to shutdown a process gracefully."""
if proc.is_alive():
proc.terminate()
proc.join(timeout=timeout)
if proc.is_alive():
log.error("Process did not terminate gracefully, forcing kill")
proc.kill()
proc.join(timeout=timeout)
if proc.is_alive():
log.error("Process did not respond to kill")
class ProcessIsolatedDashboard:
"""Dashboard running in a separate process to avoid asyncio contamination."""
def __init__(self, tool_names: list[str]):
self.tool_names = tool_names
self.process: multiprocessing.Process | None = None
def start(self, timeout: float = 10.0) -> None:
"""Start the dashboard process."""
if self.process is not None:
raise RuntimeError("Dashboard already started")
self.process = multiprocessing.Process(
target=_dashboard_worker,
args=(self.tool_names, _global_log_queue, _dashboard_ready_event, _dashboard_port_value, global_shutdown_event),
daemon=True,
)
self.process.start()
if not _dashboard_ready_event.wait(timeout):
self.process.terminate()
self.process.join(timeout=1.0)
self.process = None
raise RuntimeError("Dashboard failed to start within timeout")
port = _dashboard_port_value.value
log.info(f"Dashboard started on port {port}")
webbrowser.open(f"http://localhost:{port}/dashboard/index.html")
def stop(self, timeout: float = 1.0) -> None:
"""Signal shutdown and wait for the dashboard process to exit."""
if self.process is None:
return
log.info("Stopping dashboard process")
request_global_shutdown()
_shutdown_process(self.process, timeout=timeout)
self.process = None
class SerenaAgentWorker:
"""Worker process that hosts the actual SerenaAgent."""
class RequestMethod(StrEnum):
INITIALIZE = "initialize"
TOOL_CALL = "tool_call"
GET_ACTIVE_TOOL_NAMES = "get_active_tool_names"
IS_LANGUAGE_SERVER_RUNNING = "is_language_server_running"
RESET_LANGUAGE_SERVER = "reset_language_server"
GET_EXPOSED_TOOL_NAMES = "get_exposed_tool_names"
SHUTDOWN = "shutdown"
def __init__(self, conn: Connection):
self.conn = conn
self.agent: SerenaAgent | None = None
def run(self, log_queue: "multiprocessing.Queue[str]") -> None:
"""Main worker loop - runs in separate process."""
qh = QueueHandler(log_queue)
root = logging.getLogger()
root.setLevel(logging.DEBUG)
root.addHandler(qh)
log.info("SerenaAgent worker process started")
try:
while not global_shutdown_event.is_set():
try:
# Use polling to avoid blocking indefinitely
if self.conn.poll(timeout=0.5): # Poll every 500ms
try:
request = self.conn.recv()
if request is None: # Explicit shutdown signal
break
response = self._handle_request(request)
self.conn.send(response)
except EOFError:
# Connection closed - parent process terminated
log.info("Connection closed, worker shutting down")
break
except (BrokenPipeError, ConnectionResetError):
# Connection broken, can't communicate
log.info("Connection broken, worker shutting down")
break
except Exception as e:
log.error(f"Error processing request: {e}")
try:
response = {
"error": f"Worker process error: {e!s}",
"traceback": traceback.format_exc(),
}
self.conn.send(response)
except (EOFError, BrokenPipeError, ConnectionResetError):
# Connection is broken, can't send error response
log.info("Connection broken during error response, shutting down")
break
# Continue polling if no data available
except Exception as e:
log.error(f"Error in worker main loop: {e}")
break
except Exception as e:
log.error(f"Fatal error in worker process: {e}")
finally:
self._cleanup()
log.info("SerenaAgent worker process stopped")
os._exit(0) # Exit without raising any further exceptions
def _handle_request(self, request: dict[str, Any]) -> dict[str, Any]:
"""Handle a single request."""
try:
method = request["method"]
params = request.get("params", {})
match method:
case self.RequestMethod.INITIALIZE:
return self._initialize(params)
case self.RequestMethod.TOOL_CALL:
return self._tool_call(params)
case self.RequestMethod.GET_ACTIVE_TOOL_NAMES:
return self._get_active_tool_names()
case self.RequestMethod.IS_LANGUAGE_SERVER_RUNNING:
return self._is_language_server_running()
case self.RequestMethod.RESET_LANGUAGE_SERVER:
return self._reset_language_server()
case self.RequestMethod.GET_EXPOSED_TOOL_NAMES:
return self._get_exposed_tool_names()
case self.RequestMethod.SHUTDOWN:
return self.shutdown()
case _:
return {"error": f"Unknown method: {method}"}
except Exception as e:
return {"error": str(e), "traceback": traceback.format_exc()}
def _initialize(self, params: dict[str, Any]) -> dict[str, Any]:
"""Initialize the SerenaAgent."""
if self.agent is not None:
return {"result": "SerenaAgent already initialized"}
try:
# Extract all possible initialization parameters
context_param = params.get("context")
project = params.get("project")
serena_config = SerenaConfig.from_json_dict(params["serena_config"])
context = SerenaAgentContext.from_json_dict(context_param) if context_param is not None else None
modes = [SerenaAgentMode.from_json_dict(m) for m in params["modes"]]
log_level = params.get("log_level")
trace_lsp_communication = params.get("trace_lsp_communication")
tool_timeout = params.get("tool_timeout")
self.agent = SerenaAgent(
project=project,
serena_config=serena_config,
context=context,
modes=modes,
enable_web_dashboard=False,
enable_gui_log_window=False,
log_level=log_level,
trace_lsp_communication=trace_lsp_communication,
tool_timeout=tool_timeout,
)
return {"result": "SerenaAgent initialized successfully"}
except Exception as e:
return {"error": f"Failed to initialize SerenaAgent: {e!s}", "traceback": traceback.format_exc()}
def _tool_call(self, params: dict[str, Any]) -> dict[str, Any]:
"""Execute a tool call."""
if self.agent is None:
return {"error": "SerenaAgent not initialized"}
try:
tool_name = params["tool_name"]
tool_params = params["tool_params"]
# Get the tool by name
tool = None
for tool_instance in self.agent._active_tools.values():
if tool_instance.get_name_from_cls() == tool_name:
tool = tool_instance
break
if tool is None:
return {"error": f"Tool '{tool_name}' not found or not active"}
# Execute the tool
result = tool.apply_ex(**tool_params)
return {"result": result}
except Exception as e:
return {"error": str(e), "traceback": traceback.format_exc()}
def _get_active_tool_names(self) -> dict[str, Any]:
"""Get list of active tool names."""
if self.agent is None:
return {"error": "SerenaAgent not initialized"}
try:
tool_names = self.agent.get_active_tool_names()
return {"result": tool_names}
except Exception as e:
return {"error": str(e), "traceback": traceback.format_exc()}
def _is_language_server_running(self) -> dict[str, Any]:
"""Check if language server is running."""
if self.agent is None:
return {"error": "SerenaAgent not initialized"}
try:
is_running = self.agent.is_language_server_running()
return {"result": is_running}
except Exception as e:
return {"error": str(e), "traceback": traceback.format_exc()}
def _reset_language_server(self) -> dict[str, Any]:
"""Reset the language server."""
if self.agent is None:
return {"error": "SerenaAgent not initialized"}
try:
self.agent.reset_language_server()
return {"result": "Language server reset successfully"}
except Exception as e:
return {"error": str(e), "traceback": traceback.format_exc()}
def _get_exposed_tool_names(self) -> dict[str, Any]:
"""Get exposed tool names for MCP tool creation."""
if self.agent is None:
return {"error": "SerenaAgent not initialized"}
try:
tool_instances = self.agent.get_exposed_tool_instances()
# Return only tool names - metadata will be reconstructed from ToolRegistry
tool_names = [tool.get_name_from_cls() for tool in tool_instances]
return {"result": tool_names}
except Exception as e:
return {"error": str(e), "traceback": traceback.format_exc()}
def shutdown(self) -> dict[str, Any]:
try:
log.info("Shutting down SerenaAgent worker process on request")
self._cleanup()
request_global_shutdown()
# Return successful response before exiting
return {"result": "Shutdown initiated"}
except Exception as e:
log.error(f"Error during shutdown: {e}")
return {"error": str(e), "traceback": traceback.format_exc()}
def _cleanup(self) -> None:
"""Clean up resources."""
if self.agent is not None:
try:
if self.agent.is_language_server_running() and self.agent.language_server is not None:
self.agent.language_server.stop()
except Exception as e:
log.error(f"Error stopping language server: {e}")
self.agent = None
class ProcessIsolatedSerenaAgent:
"""Process-isolated wrapper for SerenaAgent that prevents asyncio contamination."""
def __init__(
self,
project: str | None = None,
serena_config: SerenaConfigBase | None = None,
context: SerenaAgentContext | None = None,
modes: list[SerenaAgentMode] | None = None,
log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] | None = None,
trace_lsp_communication: bool | None = None,
tool_timeout: float | None = None,
):
self.project = project
self.serena_config = serena_config or SerenaConfig.from_config_file()
self.context = context
self.modes = modes or []
self.log_level = log_level
self.trace_lsp_communication = trace_lsp_communication
self.tool_timeout = tool_timeout
self.process: multiprocessing.Process | None = None
self.conn: Connection | None = None
def start(self) -> None:
"""Start the worker process."""
if self.process is not None:
raise RuntimeError("ProcessIsolatedSerenaAgent already started")
log.info("Starting process-isolated SerenaAgent")
# Create communication pipe
parent_conn, child_conn = multiprocessing.Pipe()
self.conn = parent_conn # type: ignore
# Create and start worker process, passing along the dashboard's queue if available
worker = SerenaAgentWorker(child_conn) # type: ignore
self.process = multiprocessing.Process(target=worker.run, args=[_global_log_queue])
self.process.start()
# Prepare initialization parameters, converting complex objects to dict if present
init_params = {
"project": self.project,
"serena_config": self.serena_config.to_json_dict(),
"context": self.context.to_json_dict() if self.context is not None else None,
"modes": [m.to_json_dict() for m in self.modes],
"log_level": self.log_level,
"trace_lsp_communication": self.trace_lsp_communication,
"tool_timeout": self.tool_timeout,
}
# Initialize the agent in the worker process
try:
self._make_request_with_result(SerenaAgentWorker.RequestMethod.INITIALIZE, init_params)
except Exception as e:
self.stop()
raise RuntimeError(f"Failed to initialize SerenaAgent: {e}") from e
log.info("Process-isolated SerenaAgent started successfully")
def stop(self) -> None:
"""Stop the worker process."""
if self.process is None:
return
log.info("Stopping SerenaAgent process")
try:
# Close connection to signal worker to shutdown
if self.conn is not None:
self.conn.close()
_shutdown_process(self.process, timeout=2.0)
self.process = None
except KeyboardInterrupt:
log.warning("Keyboard interrupt during shutdown - forcing termination")
if self.process and self.process.is_alive():
self.process.kill()
self.process.join(timeout=1.0)
except Exception as e:
log.error(f"Error stopping worker process: {e}")
finally:
self.process = None
self.conn = None
log.info("SerenaAgent stopped")
def _make_request(self, method: SerenaAgentWorker.RequestMethod, params: dict[str, Any] | None = None) -> dict[str, Any]:
"""Make a request to the worker process."""
if self.process is None or not self.process.is_alive():
raise RuntimeError("Worker process is not running")
if self.conn is None:
raise RuntimeError("Connection is not initialized")
request = {"method": method, "params": params or {}}
# Send request
try:
self.conn.send(request)
except (EOFError, BrokenPipeError) as e:
raise RuntimeError("Failed to send request: worker process may have crashed") from e
# Wait for response with timeout
timeout = self.serena_config.tool_timeout
if self.conn.poll(timeout):
try:
return self.conn.recv()
except (EOFError, BrokenPipeError) as e:
raise RuntimeError("Failed to receive response: worker process may have crashed") from e
else:
raise TimeoutError(f"Request {method} timed out after {timeout} seconds")
def _make_request_with_result(self, method: SerenaAgentWorker.RequestMethod, params: dict[str, Any] | None = None) -> Any:
"""Make a request and return the result, raising an exception if there's an error."""
response = self._make_request(method, params)
if "error" in response:
raise RuntimeError(f"Request {method} failed: {response['error']}")
return response["result"]
def tool_call(self, tool_name: str, **tool_params: Any) -> str:
"""Call a tool in the worker process."""
return self._make_request_with_result(
SerenaAgentWorker.RequestMethod.TOOL_CALL, {"tool_name": tool_name, "tool_params": tool_params}
)
def get_tool(self, tool_cls: type[Tool]) -> "ProcessIsolatedTool":
"""Get a process-isolated tool that delegates to this agent."""
tool_name = tool_cls.get_name_from_cls()
return ProcessIsolatedTool(self, tool_name)
def get_active_tool_names(self) -> list[str]:
"""Get list of active tool names."""
return self._make_request_with_result(SerenaAgentWorker.RequestMethod.GET_ACTIVE_TOOL_NAMES)
def is_language_server_running(self) -> bool:
"""Check if language server is running."""
return self._make_request_with_result(SerenaAgentWorker.RequestMethod.IS_LANGUAGE_SERVER_RUNNING)
def reset_language_server(self) -> None:
"""Reset the language server."""
self._make_request_with_result(SerenaAgentWorker.RequestMethod.RESET_LANGUAGE_SERVER)
def shutdown_from_dashboard(self) -> None:
"""Request shutdown from dashboard."""
self._make_request_with_result(SerenaAgentWorker.RequestMethod.SHUTDOWN)
def get_exposed_tool_names(self) -> list[str]:
"""Get tool names for MCP tool creation."""
return self._make_request_with_result(SerenaAgentWorker.RequestMethod.GET_EXPOSED_TOOL_NAMES)
def __enter__(self) -> Self:
self.start()
return self
def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
self.stop()
class ProcessIsolatedTool(ToolInterface):
"""A clean tool wrapper that delegates to ProcessIsolatedSerenaAgent."""
def __init__(self, process_agent: ProcessIsolatedSerenaAgent, tool_name: str):
self.process_agent = process_agent
self._tool_name = tool_name
@property
def _tool_class(self) -> type[Tool]:
return ToolRegistry.get_tool_class_by_name(self._tool_name)
def get_name(self) -> str:
"""Get the tool name for this process-isolated tool."""
return self._tool_name
def get_apply_docstring(self) -> str:
"""Get the docstring for the apply method."""
# in the actual tool, this is a classmethod
return self._tool_class.get_apply_docstring_from_cls()
def get_apply_fn_metadata(self) -> FuncMetadata:
"""Get the metadata for the apply method."""
# in the actual tool, this is a classmethod
return self._tool_class.get_apply_fn_metadata_from_cls()
def apply_ex(self, log_call: bool = True, catch_exceptions: bool = True, **kwargs: Any) -> str:
"""Apply the tool with logging and exception handling."""
try:
return self.process_agent.tool_call(self._tool_name, **kwargs)
except Exception as e:
if catch_exceptions:
return f"Error executing tool {self._tool_name}: {e!s}"
raise
src/serena/resources/config/contexts/ide-assistant.yml
+8 -3
View File
@@ -1,8 +1,13 @@
description: Non-symbolic editing tools and general shell tool are excluded
prompt: |
You are running in IDE assistant context where file operations and shell commands are handled by the IDE.
You should exclusively use symbolic tools for exploring and modifying the code, as the IDE handles
file-level operations.
You are running in IDE assistant context where file operations, basic (line-based) edits and reads,
and shell commands are handled by your own, internal tools.
The initial instructions and the current config inform you on which tools are available to you,
and how to use them.
Don't attempt to use any excluded tools, instead rely on your own internal tools
for achieving the basic file or shell operations.
However, if serena's tools can be used for achieving your task (see initial instructions),
you should prioritize them.
excluded_tools:
- create_text_file
- read_file
src/serena/resources/config/modes/editing.yml
+4 -5
View File
@@ -23,11 +23,10 @@ prompt: |
use `find_symbol` with the name path `Foo/__init__` and `include_body=True`. If you don't know yet which methods in `Foo` you need to read or edit,
you can use `find_symbol` with the name path `Foo`, `include_body=False` and `depth=1` to get all (top-level) methods of `Foo` before proceeding
to read the desired methods with `include_body=True`.
Note that you never need to add additional indentation, as all symbol editing tools will automatically add the indentation of the symbol that
you are replacing or inserting above or below. In particular, keep in mind the description of the `replace_symbol_body` tool. If you want to add some new code at the end of the file, you should
In particular, keep in mind the description of the `replace_symbol_body` tool. If you want to add some new code at the end of the file, you should
use the `insert_after_symbol` tool with the last top-level symbol in the file. If you want to add an import, often a good strategy is to use
`insert_before_symbol` with the first top-level symbol in the file.
You can unterstand relationships between symbols by using the `find_referencing_symbols` tool. If not explicitly requested otherwise by a user,
You can understand relationships between symbols by using the `find_referencing_symbols` tool. If not explicitly requested otherwise by a user,
you make sure that when you edit a symbol, it is either done in a backward-compatible way, or you find and adjust the references as needed.
The `find_referencing_symbols` tool will give you code snippets around the references, as well as symbolic information.
You will generally be able to use the info from the snippets and the regex-based approach to adjust the references as well.
@@ -52,7 +51,7 @@ prompt: |
1. If the snippet to be replaced is likely to be unique within the file, you perform the replacement by directly using the escaped version of the
original.
2. If the snippet is probably not unique, and you want to replace all occurrences, you use the `allow_multiple_occurrences` flag.
3. If the snippet is not unique, and you want to replace a specific occurence, you make use of the code surrounding the snippet
3. If the snippet is not unique, and you want to replace a specific occurrence, you make use of the code surrounding the snippet
to extend the regex with content before/after such that the regex will have exactly one match.
4. You generally assume that a snippet is unique, knowing that the tool will return an error on multiple matches. You only read more file content
(for crafvarting a more specific regex) if such a failure unexpectedly occurs.
@@ -102,7 +101,7 @@ prompt: |
Generally, I remind you that you rely on the regex tool with providing you the correct feedback, no need for more verification!
IMPORTANT: REMEMBER TO USE WILDCARDS WEHEN APPROPRIATE! I WILL BE VERY UNHAPPY IF YOU WRITE LONG REGEXES WITHOUT USING WILDCARDS INSTEAD!
IMPORTANT: REMEMBER TO USE WILDCARDS WHEN APPROPRIATE! I WILL BE VERY UNHAPPY IF YOU WRITE LONG REGEXES WITHOUT USING WILDCARDS INSTEAD!
excluded_tools:
- replace_lines
- insert_at_line
src/serena/resources/config/prompt_templates/system_prompt.yml
+1 -1
View File
@@ -39,7 +39,7 @@ prompts:
use `find_symbol` with the name path `Foo/__init__` and `include_body=True`. If you don't know yet which methods in `Foo` you need to read or edit,
you can use `find_symbol` with the name path `Foo`, `include_body=False` and `depth=1` to get all (top-level) methods of `Foo` before proceeding
to read the desired methods with `include_body=True`
You can unterstand relationships between symbols by using the `find_referencing_symbols` tool.
You can understand relationships between symbols by using the `find_referencing_symbols` tool.
You generally have access to memories and it may be useful for you to read them, but also only if they help you
to answer the question or complete the task. You can infer which memories are relevant to the current task by reading
src/serena/resources/project.template.yml
+3 -1
View File
@@ -1,4 +1,6 @@
# language of the project (csharp, python, rust, java, typescript, javascript, go, cpp, or ruby)
# language of the project (csharp, python, rust, java, typescript, go, cpp, or ruby)
# * For C, use cpp
# * For JavaScript, use typescript
# Special requirements:
# * csharp: Requires the presence of a .sln file in the project folder.
language: python
src/serena/symbol.py
+98 -68
View File
@@ -1,7 +1,7 @@
import json
import logging
import os
from collections.abc import Iterator, Sequence
from collections.abc import Iterable, Iterator, Reversible, Sequence
from contextlib import contextmanager
from dataclasses import asdict, dataclass, field
from difflib import SequenceMatcher
@@ -9,9 +9,9 @@ from typing import TYPE_CHECKING, Any, Literal, NamedTuple, Self, Union
from sensai.util.string import ToStringMixin
from multilspy import SyncLanguageServer
from multilspy.language_server import ReferenceInSymbol as LSPReferenceInSymbol
from multilspy.multilspy_types import Position, SymbolKind, UnifiedSymbolInformation
from solidlsp import SolidLanguageServer
from solidlsp.ls import ReferenceInSymbol as LSPReferenceInSymbol
from solidlsp.ls_types import Position, SymbolKind, UnifiedSymbolInformation
if TYPE_CHECKING:
from .agent import SerenaAgent
@@ -256,6 +256,13 @@ class Symbol(ToStringMixin):
def symbol_kind(self) -> SymbolKind:
return self.symbol_root["kind"]
def is_neighbouring_definition_separated_by_empty_line(self) -> bool:
"""
:return: whether a symbol definition of this symbol's kind is usually separated from the
previous/next definition by at least one empty line.
"""
return self.symbol_kind in (SymbolKind.Function, SymbolKind.Method, SymbolKind.Class, SymbolKind.Interface, SymbolKind.Struct)
@property
def relative_path(self) -> str | None:
location = self.symbol_root.get("location")
@@ -488,15 +495,22 @@ class ReferenceInSymbol(ToStringMixin):
class SymbolManager:
def __init__(self, lang_server: SyncLanguageServer, agent: Union["SerenaAgent", None] = None) -> None:
def __init__(self, lang_server: SolidLanguageServer, agent: Union["SerenaAgent", None] = None) -> None:
"""
:param lang_server: the language server to use for symbol retrieval as well as editing operations.
:param agent: the agent to use (only needed for marking files as modified). You can pass None if you don't
need an agent to be avare of file modifications performed by the symbol manager.
need an agent to be aware of file modifications performed by the symbol manager.
"""
self.lang_server = lang_server
self._lang_server = lang_server
self.agent = agent
def set_language_server(self, lang_server: SolidLanguageServer) -> None:
"""
Set the language server to use for symbol retrieval and editing operations.
This is useful if you want to change the language server after initializing the SymbolManager.
"""
self._lang_server = lang_server
def find_by_name(
self,
name_path: str,
@@ -512,7 +526,7 @@ class SymbolManager:
to symbols within a specific file or directory.
"""
symbols: list[Symbol] = []
symbol_roots = self.lang_server.request_full_symbol_tree(within_relative_path=within_relative_path, include_body=include_body)
symbol_roots = self._lang_server.request_full_symbol_tree(within_relative_path=within_relative_path, include_body=include_body)
for root in symbol_roots:
symbols.extend(
Symbol(root).find(
@@ -522,14 +536,14 @@ class SymbolManager:
return symbols
def get_document_symbols(self, relative_path: str) -> list[Symbol]:
symbol_dicts, roots = self.lang_server.request_document_symbols(relative_path, include_body=False)
symbol_dicts, roots = self._lang_server.request_document_symbols(relative_path, include_body=False)
symbols = [Symbol(s) for s in symbol_dicts]
return symbols
def find_by_location(self, location: SymbolLocation) -> Symbol | None:
if location.relative_path is None:
return None
symbol_dicts, roots = self.lang_server.request_document_symbols(location.relative_path, include_body=False)
symbol_dicts, roots = self._lang_server.request_document_symbols(location.relative_path, include_body=False)
for symbol_dict in symbol_dicts:
symbol = Symbol(symbol_dict)
if symbol.location == location:
@@ -598,7 +612,7 @@ class SymbolManager:
assert symbol_location.relative_path is not None
assert symbol_location.line is not None
assert symbol_location.column is not None
references = self.lang_server.request_referencing_symbols(
references = self._lang_server.request_referencing_symbols(
relative_file_path=symbol_location.relative_path,
line=symbol_location.line,
column=symbol_location.column,
@@ -618,9 +632,9 @@ class SymbolManager:
@contextmanager
def _edited_file(self, relative_path: str) -> Iterator[None]:
with self.lang_server.open_file(relative_path) as file_buffer:
with self._lang_server.open_file(relative_path) as file_buffer:
yield
root_path = self.lang_server.language_server.repository_root_path
root_path = self._lang_server.language_server.repository_root_path
abs_path = os.path.join(root_path, relative_path)
with open(abs_path, "w", encoding="utf-8") as f:
f.write(file_buffer.contents)
@@ -641,7 +655,7 @@ class SymbolManager:
def _get_code_file_content(self, relative_path: str) -> str:
"""Get the content of a file using the language server."""
return self.lang_server.language_server.retrieve_full_file_content(relative_path)
return self._lang_server.language_server.retrieve_full_file_content(relative_path)
def replace_body(self, name_path: str, relative_file_path: str, body: str, *, use_same_indentation: bool = True) -> None:
"""
@@ -682,26 +696,36 @@ class SymbolManager:
if start_pos is None or end_pos is None:
raise ValueError(f"Symbol at {location} does not have a defined body range.")
start_line, start_col = start_pos["line"], start_pos["character"]
if use_same_indentation:
indent = " " * start_col
body_lines = body.splitlines()
body = body_lines[0] + "\n" + "\n".join(indent + line for line in body_lines[1:])
# make sure body always ends with at least one newline
if not body.endswith("\n"):
body += "\n"
self.lang_server.delete_text_between_positions(location.relative_path, start_pos, end_pos)
self.lang_server.insert_text_at_position(location.relative_path, start_line, start_col, body)
# make sure the replacement adds no additional newlines (before or after) - all newlines
# and whitespace before/after should remain the same, so we strip it entirely
body = body.strip()
def insert_after_symbol(
self,
name_path: str,
relative_file_path: str,
body: str,
*,
use_same_indentation: bool = True,
at_new_line: bool = True,
) -> None:
self._lang_server.delete_text_between_positions(location.relative_path, start_pos, end_pos)
self._lang_server.insert_text_at_position(location.relative_path, start_line, start_col, body)
@staticmethod
def _count_leading_newlines(text: Iterable) -> int:
cnt = 0
for c in text:
if c == "\n":
cnt += 1
elif c == "\r":
continue
else:
break
return cnt
@classmethod
def _count_trailing_newlines(cls, text: Reversible) -> int:
return cls._count_leading_newlines(reversed(text))
def insert_after_symbol(self, name_path: str, relative_file_path: str, body: str, *, use_same_indentation: bool = True) -> None:
"""
Inserts content after the symbol with the given name in the given file.
"""
@@ -715,13 +739,9 @@ class SymbolManager:
f"Found symbols at locations: \n" + json.dumps([s.location.to_dict() for s in symbol_candidates], indent=2)
)
symbol = symbol_candidates[-1]
return self.insert_after_symbol_at_location(
symbol.location, body, at_new_line=at_new_line, use_same_indentation=use_same_indentation
)
return self.insert_after_symbol_at_location(symbol.location, body, use_same_indentation=use_same_indentation)
def insert_after_symbol_at_location(
self, location: SymbolLocation, body: str, *, at_new_line: bool = True, use_same_indentation: bool = True
) -> None:
def insert_after_symbol_at_location(self, location: SymbolLocation, body: str, *, use_same_indentation: bool = True) -> None:
"""
Appends content after the given symbol
@@ -743,12 +763,23 @@ class SymbolManager:
if pos is None:
raise ValueError(f"Symbol at {location} does not have a defined end position.")
line, col = pos["line"], pos["character"]
if at_new_line:
line += 1
col = 0
if not body.startswith("\n"):
body = "\n" + body
# start at the beginning of the next line
col = 0
line = pos["line"] + 1
# make sure a suitable number of leading empty lines is used (at least 0/1 depending on the symbol type,
# otherwise as many as the caller wanted to insert)
original_leading_newlines = self._count_leading_newlines(body)
body = body.lstrip("\r\n")
min_empty_lines = 0
if symbol.is_neighbouring_definition_separated_by_empty_line():
min_empty_lines = 1
num_leading_empty_lines = max(min_empty_lines, original_leading_newlines)
if num_leading_empty_lines:
body = ("\n" * num_leading_empty_lines) + body
# make sure the one line break succeeding the original symbol, which we repurposed as prefix via
# `line += 1`, is replaced
body = body.rstrip("\r\n") + "\n"
if use_same_indentation:
symbol_start_pos = symbol.body_start_position
assert symbol_start_pos is not None, f"Symbol at {location=} does not have a defined start position."
@@ -771,20 +802,11 @@ class SymbolManager:
# > test test
# > second line
# > dataclass_instance.status = "active" # Reassign dataclass field
col = 0
with self._edited_symbol_location(location):
self.lang_server.insert_text_at_position(location.relative_path, line=line, column=col, text_to_be_inserted=body)
self._lang_server.insert_text_at_position(location.relative_path, line=line, column=col, text_to_be_inserted=body)
def insert_before_symbol(
self,
name_path: str,
relative_file_path: str,
body: str,
*,
at_new_line: bool = True,
use_same_indentation: bool = True,
) -> None:
def insert_before_symbol(self, name_path: str, relative_file_path: str, body: str, *, use_same_indentation: bool = True) -> None:
"""
Inserts content before the symbol with the given name in the given file.
"""
@@ -798,11 +820,9 @@ class SymbolManager:
f"Found symbols at locations: \n" + json.dumps([s.location.to_dict() for s in symbol_candidates], indent=2)
)
symbol = symbol_candidates[0]
self.insert_before_symbol_at_location(symbol.location, body, at_new_line=at_new_line, use_same_indentation=use_same_indentation)
self.insert_before_symbol_at_location(symbol.location, body, use_same_indentation=use_same_indentation)
def insert_before_symbol_at_location(
self, location: SymbolLocation, body: str, *, at_new_line: bool = True, use_same_indentation: bool = True
) -> None:
def insert_before_symbol_at_location(self, location: SymbolLocation, body: str, *, use_same_indentation: bool = True) -> None:
"""
Inserts content before the given symbol
@@ -813,21 +833,31 @@ class SymbolManager:
symbol_start_pos = symbol.body_start_position
if symbol_start_pos is None:
raise ValueError(f"Symbol at {location} does not have a defined start position.")
line = symbol_start_pos["line"]
col = symbol_start_pos["character"]
if use_same_indentation:
indent = " " * (col)
indent = " " * (symbol_start_pos["character"])
body = "\n".join(indent + line for line in body.splitlines())
# similar problems as in insert_after_symbol_at_location, see comment there
if at_new_line:
col = 0
line -= 1
if not body.endswith("\n"):
body += "\n"
# insert position is the start of line where the symbol is defined
line = symbol_start_pos["line"]
col = 0
original_trailing_empty_lines = self._count_trailing_newlines(body) - 1
# ensure eol is present at end
body = body.rstrip() + "\n"
# add suitable number of trailing empty lines after the body (at least 0/1 depending on the symbol type,
# otherwise as many as the caller wanted to insert)
min_trailing_empty_lines = 0
if symbol.is_neighbouring_definition_separated_by_empty_line():
min_trailing_empty_lines = 1
num_trailing_newlines = max(min_trailing_empty_lines, original_trailing_empty_lines)
body += "\n" * num_trailing_newlines
assert location.relative_path is not None
self.lang_server.insert_text_at_position(location.relative_path, line=line, column=col, text_to_be_inserted=body)
self._lang_server.insert_text_at_position(location.relative_path, line=line, column=col, text_to_be_inserted=body)
def insert_at_line(self, relative_path: str, line: int, content: str) -> None:
"""
@@ -837,7 +867,7 @@ class SymbolManager:
:param content: the content to insert
"""
with self._edited_file(relative_path):
self.lang_server.insert_text_at_position(relative_path, line, 0, content)
self._lang_server.insert_text_at_position(relative_path, line, 0, content)
def delete_lines(self, relative_path: str, start_line: int, end_line: int) -> None:
"""
@@ -852,7 +882,7 @@ class SymbolManager:
with self._edited_file(relative_path):
start_pos = Position(line=start_line, character=start_col)
end_pos = Position(line=end_line_for_delete, character=end_col)
self.lang_server.delete_text_between_positions(relative_path, start_pos, end_pos)
self._lang_server.delete_text_between_positions(relative_path, start_pos, end_pos)
def delete_symbol_at_location(self, location: SymbolLocation) -> None:
"""
@@ -862,7 +892,7 @@ class SymbolManager:
assert location.relative_path is not None
assert symbol.body_start_position is not None
assert symbol.body_end_position is not None
self.lang_server.delete_text_between_positions(location.relative_path, symbol.body_start_position, symbol.body_end_position)
self._lang_server.delete_text_between_positions(location.relative_path, symbol.body_start_position, symbol.body_end_position)
def delete_symbol(self, name_path: str, relative_file_path: str) -> None:
"""
src/serena/text_utils.py
+72 -19
View File
@@ -1,3 +1,4 @@
import fnmatch
import logging
import re
from collections.abc import Callable
@@ -6,8 +7,6 @@ from enum import StrEnum
from typing import Any, Self
from joblib import Parallel, delayed
from pathspec import PathSpec
from pathspec.patterns.gitwildmatch import GitWildMatchPattern
log = logging.getLogger(__name__)
@@ -153,19 +152,28 @@ def search_text(
# Convert pattern to a compiled regex if it's a string
if is_glob and isinstance(pattern, str):
# Convert glob pattern to regex
# Escape all regex special characters except * and ?
regex_special_chars = r"\^$.|+()[{"
escaped_pattern = ""
for char in pattern:
if char in regex_special_chars:
escaped_pattern += "\\" + char
elif char == "*":
escaped_pattern += ".*"
elif char == "?":
escaped_pattern += "."
else:
escaped_pattern += char
# Convert glob pattern with optional backslash escaping to regex
def glob_to_regex(glob_pat: str) -> str:
regex_parts: list[str] = []
i = 0
while i < len(glob_pat):
ch = glob_pat[i]
if ch == "*":
regex_parts.append(".*")
elif ch == "?":
regex_parts.append(".")
elif ch == "\\":
i += 1
if i < len(glob_pat):
regex_parts.append(re.escape(glob_pat[i]))
else:
regex_parts.append("\\")
else:
regex_parts.append(re.escape(ch))
i += 1
return "".join(regex_parts)
escaped_pattern = glob_to_regex(pattern)
# For glob patterns, don't anchor with ^ and $ to allow partial line matches
compiled_pattern = re.compile(escaped_pattern)
elif isinstance(pattern, str):
@@ -249,6 +257,52 @@ def default_file_reader(file_path: str) -> str:
return f.read()
def glob_match(pattern: str, path: str) -> bool:
"""
Match a file path against a glob pattern.
Supports standard glob patterns:
- * matches any number of characters except /
- ** matches any number of directories (zero or more)
- ? matches a single character except /
- [seq] matches any character in seq
:param pattern: Glob pattern (e.g., 'src/**/*.py', '**agent.py')
:param path: File path to match against
:return: True if path matches pattern
"""
pattern = pattern.replace("\\", "/") # Normalize backslashes to forward slashes
path = path.replace("\\", "/") # Normalize path backslashes to forward slashes
# Handle ** patterns that should match zero or more directories
if "**" in pattern:
# Method 1: Standard fnmatch (matches one or more directories)
regex1 = fnmatch.translate(pattern)
if re.match(regex1, path):
return True
# Method 2: Handle zero-directory case by removing /** entirely
# Convert "src/**/test.py" to "src/test.py"
if "/**/" in pattern:
zero_dir_pattern = pattern.replace("/**/", "/")
regex2 = fnmatch.translate(zero_dir_pattern)
if re.match(regex2, path):
return True
# Method 3: Handle leading ** case by removing **/
# Convert "**/test.py" to "test.py"
if pattern.startswith("**/"):
zero_dir_pattern = pattern[3:] # Remove "**/"
regex3 = fnmatch.translate(zero_dir_pattern)
if re.match(regex3, path):
return True
return False
else:
# Simple pattern without **, use fnmatch directly
return fnmatch.fnmatch(path, pattern)
def search_files(
file_paths: list[str],
pattern: re.Pattern | str,
@@ -272,15 +326,14 @@ def search_files(
:return: List of MatchedConsecutiveLines objects
"""
# Pre-filter paths (done sequentially to avoid overhead)
include_spec = PathSpec.from_lines(GitWildMatchPattern, [paths_include_glob]) if paths_include_glob else None
exclude_spec = PathSpec.from_lines(GitWildMatchPattern, [paths_exclude_glob]) if paths_exclude_glob else None
# Use proper glob matching instead of gitignore patterns
filtered_paths = []
for path in file_paths:
if include_spec and not include_spec.match_file(path):
if paths_include_glob and not glob_match(paths_include_glob, path):
log.debug(f"Skipping {path}: does not match include pattern {paths_include_glob}")
continue
if exclude_spec and exclude_spec.match_file(path):
if paths_exclude_glob and glob_match(paths_exclude_glob, path):
log.debug(f"Skipping {path}: matches exclude pattern {paths_exclude_glob}")
continue
filtered_paths.append(path)
src/serena/util/file_system.py
+30 -21
View File
@@ -1,3 +1,4 @@
import glob
import os
from collections.abc import Callable
from dataclasses import dataclass, field
@@ -78,17 +79,14 @@ def find_all_non_ignored_files(repo_root: str) -> list[str]:
@dataclass
class GitignoreSpec:
"""
Represents a single gitignore file and its parsed patterns.
:param file_path: Path to the gitignore file
:param patterns: List of adjusted patterns from the gitignore file
:param pathspec: Compiled PathSpec object for pattern matching
"""
file_path: str
"""Path to the gitignore file."""
patterns: list[str] = field(default_factory=list)
"""List of patterns from the gitignore file.
The patterns are adjusted based on the gitignore file location.
"""
pathspec: PathSpec = field(init=False)
"""Compiled PathSpec object for pattern matching."""
def __post_init__(self) -> None:
"""Initialize the PathSpec from patterns."""
@@ -137,18 +135,7 @@ class GitignoreParser:
:return: List of absolute paths to .gitignore files
"""
gitignore_files = []
for root, dirs, files in os.walk(self.repo_root):
# Skip .git directory
if ".git" in dirs:
dirs.remove(".git")
if ".gitignore" in files:
gitignore_path = os.path.join(root, ".gitignore")
gitignore_files.append(gitignore_path)
return gitignore_files
return glob.glob(self.repo_root + "/.gitignore") + glob.glob(self.repo_root + "/**/.gitignore")
def _create_ignore_spec(self, gitignore_file_path: str) -> GitignoreSpec:
"""
@@ -228,7 +215,13 @@ class GitignoreParser:
# Add the directory prefix but also allow matching in subdirectories
adjusted_pattern = os.path.join(rel_dir, "**", line)
else:
adjusted_pattern = line
if is_anchored:
# Anchored patterns in root should only match at root level
# Add leading slash back to indicate root-only matching
adjusted_pattern = "/" + line
else:
# Non-anchored patterns can match anywhere
adjusted_pattern = line
# Re-add negation if needed
if is_negation:
@@ -254,9 +247,14 @@ class GitignoreParser:
else:
rel_path = path
abs_path = os.path.join(self.repo_root, rel_path)
# Normalize path separators
rel_path = rel_path.replace(os.sep, "/")
if os.path.exists(abs_path) and os.path.isdir(abs_path) and not rel_path.endswith("/"):
rel_path = rel_path + "/"
# Check against each ignore spec
for spec in self.ignore_specs:
if spec.matches(rel_path):
@@ -276,3 +274,14 @@ class GitignoreParser:
"""Reload all gitignore files from the repository."""
self.ignore_specs.clear()
self._load_gitignore_files()
def match_path(path: str, path_spec: PathSpec) -> bool:
path = os.path.abspath(path)
normalized_path = str(path).replace(os.path.sep, "/")
# pathspec can't handle the matching of directories if they don't end with a slash!
# see https://github.com/cpburnz/python-pathspec/issues/89
if os.path.isdir(normalized_path) and not normalized_path.endswith("/"):
normalized_path = normalized_path + "/"
return path_spec.match_file(normalized_path)
src/serena/util/git.py
+20
View File
@@ -0,0 +1,20 @@
import logging
from sensai.util.git import GitStatus
from .shell import subprocess_check_output
log = logging.getLogger(__name__)
def get_git_status() -> GitStatus | None:
try:
commit_hash = subprocess_check_output(["git", "rev-parse", "HEAD"])
unstaged = bool(subprocess_check_output(["git", "diff", "--name-only"]))
staged = bool(subprocess_check_output(["git", "diff", "--staged", "--name-only"]))
untracked = bool(subprocess_check_output(["git", "ls-files", "--others", "--exclude-standard"]))
return GitStatus(
commit=commit_hash, has_unstaged_changes=unstaged, has_staged_uncommitted_changes=staged, has_untracked_files=untracked
)
except:
return None
src/serena/util/inspection.py
+1 -1
View File
@@ -3,8 +3,8 @@ import os
from collections.abc import Generator
from typing import TypeVar
from multilspy.multilspy_config import Language
from serena.util.file_system import find_all_non_ignored_files
from solidlsp.ls_config import Language
T = TypeVar("T")
src/serena/util/shell.py
+19 -1
View File
@@ -24,11 +24,14 @@ def execute_shell_command(command: str, cwd: str | None = None, capture_stderr:
if cwd is None:
cwd = os.getcwd()
is_windows = platform.system() == "Windows"
process = subprocess.Popen(
command,
shell=platform.system() != "Windows",
shell=not is_windows,
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE if capture_stderr else None,
creationflags=subprocess.CREATE_NO_WINDOW if is_windows else 0, # type: ignore
text=True,
encoding="utf-8",
errors="replace",
@@ -37,3 +40,18 @@ def execute_shell_command(command: str, cwd: str | None = None, capture_stderr:
stdout, stderr = process.communicate()
return ShellCommandResult(stdout=stdout, stderr=stderr, return_code=process.returncode, cwd=cwd)
def subprocess_check_output(args: list[str], encoding: str = "utf-8", strip: bool = True, timeout: float | None = None) -> str:
kwargs = {
"stdin": subprocess.DEVNULL,
"stderr": subprocess.PIPE,
"timeout": timeout,
"env": os.environ.copy(),
}
if platform.system() == "Windows":
kwargs["creationflags"] = subprocess.CREATE_NO_WINDOW # type: ignore
output = subprocess.check_output(args, **kwargs).decode(encoding) # type: ignore
if strip:
output = output.strip()
return output
src/solidlsp/__init__.py
+2
View File
@@ -0,0 +1,2 @@
# ruff: noqa
from .ls import SolidLanguageServer
src/multilspy/language_servers/clangd_language_server/clangd_language_server.py → src/solidlsp/language_servers/clangd_language_server/clangd_language_server.py
+49 -57
View File
@@ -2,32 +2,29 @@
Provides C/C++ specific instantiation of the LanguageServer class. Contains various configurations and settings specific to C/C++.
"""
import asyncio
import json
import logging
import os
import stat
import pathlib
from contextlib import asynccontextmanager
from typing import AsyncIterator
import stat
import threading
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import InitializeParams
from multilspy.multilspy_config import MultilspyConfig
from multilspy.multilspy_utils import FileUtils
from multilspy.multilspy_utils import PlatformUtils
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.ls_utils import FileUtils, PlatformUtils
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
class ClangdLanguageServer(LanguageServer):
class ClangdLanguageServer(SolidLanguageServer):
"""
Provides C/C++ specific instantiation of the LanguageServer class. Contains various configurations and settings specific to C/C++.
As the project gets bigger in size, building index will take time. Try running clangd multiple times to ensure index is built properly.
Also make sure compile_commands.json is created at root of the source directory. Check clangd test case for example.
"""
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
def __init__(self, config: LanguageServerConfig, logger: LanguageServerLogger, repository_root_path: str):
"""
Creates a ClangdLanguageServer instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead.
"""
@@ -39,15 +36,18 @@ class ClangdLanguageServer(LanguageServer):
ProcessLaunchInfo(cmd=clangd_executable_path, cwd=repository_root_path),
"cpp",
)
self.server_ready = asyncio.Event()
self.server_ready = threading.Event()
self.service_ready_event = threading.Event()
self.initialize_searcher_command_available = threading.Event()
self.resolve_main_method_available = threading.Event()
def setup_runtime_dependencies(self, logger: MultilspyLogger, config: MultilspyConfig) -> str:
def setup_runtime_dependencies(self, logger: LanguageServerLogger, config: LanguageServerConfig) -> str:
"""
Setup runtime dependencies for ClangdLanguageServer.
"""
platform_id = PlatformUtils.get_platform_id()
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json")) as f:
d = json.load(f)
del d["_description"]
@@ -55,12 +55,12 @@ class ClangdLanguageServer(LanguageServer):
"linux-x64",
"win-x64",
"osx-arm64",
], "Unsupported platform: " + platform_id.value
], (
"Unsupported platform: " + platform_id.value
)
runtime_dependencies = d["runtimeDependencies"]
runtime_dependencies = [
dependency for dependency in runtime_dependencies if dependency["platformId"] == platform_id.value
]
runtime_dependencies = [dependency for dependency in runtime_dependencies if dependency["platformId"] == platform_id.value]
assert len(runtime_dependencies) == 1
# Select dependency matching the current platform
dependency = next((dep for dep in runtime_dependencies if dep["platformId"] == platform_id.value), None)
@@ -74,9 +74,7 @@ class ClangdLanguageServer(LanguageServer):
logger.log(f"Clangd executable not found at {clangd_executable_path}. Downloading from {clangd_url}", logging.INFO)
os.makedirs(clangd_ls_dir, exist_ok=True)
if dependency["archiveType"] == "zip":
FileUtils.download_and_extract_archive(
logger, clangd_url, clangd_ls_dir, dependency["archiveType"]
)
FileUtils.download_and_extract_archive(logger, clangd_url, clangd_ls_dir, dependency["archiveType"])
else:
raise RuntimeError(f"Unsupported archive type: {dependency['archiveType']}")
if not os.path.exists(clangd_executable_path):
@@ -92,7 +90,7 @@ class ClangdLanguageServer(LanguageServer):
"""
Returns the initialize params for the clangd Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json")) as f:
d = json.load(f)
del d["_description"]
@@ -112,8 +110,7 @@ class ClangdLanguageServer(LanguageServer):
return d
@asynccontextmanager
async def start_server(self) -> AsyncIterator["ClangdLanguageServer"]:
def _start_server(self):
"""
Starts the Clangd Language Server, waits for the server to be ready and yields the LanguageServer instance.
@@ -126,7 +123,8 @@ class ClangdLanguageServer(LanguageServer):
# Shutdown the LanguageServer on exit from scope
# LanguageServer has been shutdown
"""
async def register_capability_handler(params):
def register_capability_handler(params):
assert "registrations" in params
for registration in params["registrations"]:
if registration["method"] == "workspace/executeCommand":
@@ -134,24 +132,24 @@ class ClangdLanguageServer(LanguageServer):
self.resolve_main_method_available.set()
return
async def lang_status_handler(params):
def lang_status_handler(params):
# TODO: Should we wait for
# server -> client: {'jsonrpc': '2.0', 'method': 'language/status', 'params': {'type': 'ProjectStatus', 'message': 'OK'}}
# Before proceeding?
if params["type"] == "ServiceReady" and params["message"] == "ServiceReady":
self.service_ready_event.set()
async def execute_client_command_handler(params):
def execute_client_command_handler(params):
return []
async def do_nothing(params):
def do_nothing(params):
return
async def check_experimental_status(params):
def check_experimental_status(params):
if params["quiescent"] == True:
self.server_ready.set()
async def window_log_message(msg):
def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
self.server.on_request("client/registerCapability", register_capability_handler)
@@ -163,31 +161,25 @@ class ClangdLanguageServer(LanguageServer):
self.server.on_notification("language/actionableNotification", do_nothing)
self.server.on_notification("experimental/serverStatus", check_experimental_status)
async with super().start_server():
self.logger.log("Starting Clangd server process", logging.INFO)
await self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log("Starting Clangd server process", logging.INFO)
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = await self.server.send.initialize(initialize_params)
assert init_response["capabilities"]["textDocumentSync"]["change"] == 2
assert "completionProvider" in init_response["capabilities"]
assert init_response["capabilities"]["completionProvider"] == {
"triggerCharacters": ['.', '<', '>', ':', '"', '/', '*'],
"resolveProvider": False,
}
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = self.server.send.initialize(initialize_params)
assert init_response["capabilities"]["textDocumentSync"]["change"] == 2
assert "completionProvider" in init_response["capabilities"]
assert init_response["capabilities"]["completionProvider"] == {
"triggerCharacters": [".", "<", ">", ":", '"', "/", "*"],
"resolveProvider": False,
}
self.server.notify.initialized({})
self.server.notify.initialized({})
self.completions_available.set()
# set ready flag
self.server_ready.set()
await self.server_ready.wait()
yield self
await self.server.shutdown()
await self.server.stop()
self.completions_available.set()
# set ready flag
self.server_ready.set()
self.server_ready.wait()
src/multilspy/language_servers/clangd_language_server/initialize_params.json → src/solidlsp/language_servers/clangd_language_server/initialize_params.json
View File
src/multilspy/language_servers/clangd_language_server/runtime_dependencies.json → src/solidlsp/language_servers/clangd_language_server/runtime_dependencies.json
View File
src/multilspy/language_servers/dart_language_server/dart_language_server.py → src/solidlsp/language_servers/dart_language_server/dart_language_server.py
+33 -60
View File
@@ -1,17 +1,16 @@
from contextlib import asynccontextmanager
import json
import logging
import os
import pathlib
import shutil
import stat
from typing import AsyncIterator
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
import json
from multilspy.multilspy_utils import FileUtils, PlatformUtils
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.ls_utils import FileUtils, PlatformUtils
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
class DartLanguageServer(LanguageServer):
class DartLanguageServer(SolidLanguageServer):
"""
Provides Dart specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Dart.
"""
@@ -20,7 +19,6 @@ class DartLanguageServer(LanguageServer):
"""
Creates a DartServer instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead.
"""
executable_path = self.setup_runtime_dependencies(logger)
super().__init__(
config,
@@ -30,17 +28,15 @@ class DartLanguageServer(LanguageServer):
"dart",
)
def setup_runtime_dependencies(self, logger: "MultilspyLogger") -> str:
def setup_runtime_dependencies(self, logger: "LanguageServerLogger") -> str:
platform_id = PlatformUtils.get_platform_id()
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json")) as f:
d = json.load(f)
del d["_description"]
runtime_dependencies = d["runtimeDependencies"]
runtime_dependencies = [
dependency for dependency in runtime_dependencies if dependency["platformId"] == platform_id.value
]
runtime_dependencies = [dependency for dependency in runtime_dependencies if dependency["platformId"] == platform_id.value]
assert len(runtime_dependencies) == 1
dependency = runtime_dependencies[0]
@@ -50,24 +46,18 @@ class DartLanguageServer(LanguageServer):
if not os.path.exists(dart_ls_dir):
os.makedirs(dart_ls_dir)
FileUtils.download_and_extract_archive(
logger, dependency["url"], dart_ls_dir, dependency["archiveType"]
)
FileUtils.download_and_extract_archive(logger, dependency["url"], dart_ls_dir, dependency["archiveType"])
assert os.path.exists(dart_executable_path)
os.chmod(dart_executable_path, stat.S_IEXEC)
return f"{dart_executable_path} language-server --client-id multilspy.dart --client-version 1.2"
def _get_initialize_params(self, repository_absolute_path: str):
"""
Returns the initialize params for the Dart Language Server.
"""
with open(
os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r"
) as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json")) as f:
d = json.load(f)
del d["_description"]
@@ -80,67 +70,50 @@ class DartLanguageServer(LanguageServer):
d["rootUri"] = pathlib.Path(repository_absolute_path).as_uri()
assert d["workspaceFolders"][0]["uri"] == "$uri"
d["workspaceFolders"][0]["uri"] = pathlib.Path(
repository_absolute_path
).as_uri()
d["workspaceFolders"][0]["uri"] = pathlib.Path(repository_absolute_path).as_uri()
assert d["workspaceFolders"][0]["name"] == "$name"
d["workspaceFolders"][0]["name"] = os.path.basename(repository_absolute_path)
return d
@asynccontextmanager
async def start_server(self) -> AsyncIterator["DartLanguageServer"]:
def _start_server(self):
"""
Start the language server and yield when the server is ready.
"""
async def execute_client_command_handler(params):
def execute_client_command_handler(params):
return []
async def do_nothing(params):
def do_nothing(params):
return
async def check_experimental_status(params):
def check_experimental_status(params):
pass
async def window_log_message(msg):
def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
self.server.on_request("client/registerCapability", do_nothing)
self.server.on_notification("language/status", do_nothing)
self.server.on_notification("window/logMessage", window_log_message)
self.server.on_request(
"workspace/executeClientCommand", execute_client_command_handler
)
self.server.on_request("workspace/executeClientCommand", execute_client_command_handler)
self.server.on_notification("$/progress", do_nothing)
self.server.on_notification("textDocument/publishDiagnostics", do_nothing)
self.server.on_notification("language/actionableNotification", do_nothing)
self.server.on_notification(
"experimental/serverStatus", check_experimental_status
self.server.on_notification("experimental/serverStatus", check_experimental_status)
self.logger.log("Starting dart-language-server server process", logging.INFO)
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request to dart-language-server",
logging.DEBUG,
)
init_response = self.server.send_request("initialize", initialize_params)
self.logger.log(
f"Received initialize response from dart-language-server: {init_response}",
logging.INFO,
)
async with super().start_server():
self.logger.log(
"Starting dart-language-server server process", logging.INFO
)
await self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request to dart-language-server",
logging.DEBUG,
)
init_response = await self.server.send_request(
"initialize", initialize_params
)
self.logger.log(
f"Received initialize response from dart-language-server: {init_response}",
logging.INFO,
)
self.server.notify.initialized({})
yield self
await self.server.shutdown()
await self.server.stop()
self.server.notify.initialized({})
src/multilspy/language_servers/dart_language_server/initialize_params.json → src/solidlsp/language_servers/dart_language_server/initialize_params.json
View File
src/multilspy/language_servers/dart_language_server/runtime_dependencies.json → src/solidlsp/language_servers/dart_language_server/runtime_dependencies.json
View File
src/multilspy/language_servers/eclipse_jdtls/eclipse_jdtls.py → src/solidlsp/language_servers/eclipse_jdtls/eclipse_jdtls.py
+67 -104
View File
@@ -2,7 +2,6 @@
Provides Java specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Java.
"""
import asyncio
import dataclasses
import json
import logging
@@ -10,21 +9,19 @@ import os
import pathlib
import shutil
import stat
import threading
import uuid
from contextlib import asynccontextmanager
from typing import AsyncIterator
from pathlib import PurePath
from overrides import override
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import InitializeParams
from multilspy.multilspy_config import MultilspyConfig
from multilspy.multilspy_settings import MultilspySettings
from multilspy.multilspy_utils import FileUtils
from multilspy.multilspy_utils import PlatformUtils
from pathlib import PurePath
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.ls_utils import FileUtils, PlatformUtils
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
from solidlsp.settings import SolidLSPSettings
@dataclasses.dataclass
@@ -43,24 +40,23 @@ class RuntimeDependencyPaths:
intellisense_members_path: str
class EclipseJDTLS(LanguageServer):
class EclipseJDTLS(SolidLanguageServer):
"""
The EclipseJDTLS class provides a Java specific implementation of the LanguageServer class
"""
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
def __init__(self, config: LanguageServerConfig, logger: LanguageServerLogger, repository_root_path: str):
"""
Creates a new EclipseJDTLS instance initializing the language server settings appropriately.
This class is not meant to be instantiated directly. Use LanguageServer.create() instead.
"""
runtime_dependency_paths = self.setupRuntimeDependencies(logger, config)
self.runtime_dependency_paths = runtime_dependency_paths
# ws_dir is the workspace directory for the EclipseJDTLS server
ws_dir = str(
PurePath(
MultilspySettings.get_language_server_directory(),
SolidLSPSettings.get_language_server_directory(),
"EclipseJDTLS",
"workspaces",
uuid.uuid4().hex,
@@ -68,9 +64,7 @@ class EclipseJDTLS(LanguageServer):
)
# shared_cache_location is the global cache used by Eclipse JDTLS across all workspaces
shared_cache_location = str(
PurePath(MultilspySettings.get_global_cache_directory(), "lsp", "EclipseJDTLS", "sharedIndex")
)
shared_cache_location = str(PurePath(SolidLSPSettings.get_global_cache_directory(), "lsp", "EclipseJDTLS", "sharedIndex"))
jre_path = self.runtime_dependency_paths.jre_path
lombok_jar_path = self.runtime_dependency_paths.lombok_jar_path
@@ -135,12 +129,12 @@ class EclipseJDTLS(LanguageServer):
]
)
self.service_ready_event = asyncio.Event()
self.intellicode_enable_command_available = asyncio.Event()
self.initialize_searcher_command_available = asyncio.Event()
self.service_ready_event = threading.Event()
self.intellicode_enable_command_available = threading.Event()
self.initialize_searcher_command_available = threading.Event()
super().__init__(config, logger, repository_root_path, ProcessLaunchInfo(cmd, proc_env, proc_cwd), "java")
@override
def is_ignored_dirname(self, dirname: str) -> bool:
# Ignore common Java build directories from different build tools:
@@ -150,22 +144,22 @@ class EclipseJDTLS(LanguageServer):
# - IntelliJ IDEA: out, .idea
# - General: classes, dist, lib
return super().is_ignored_dirname(dirname) or dirname in [
"target", # Maven
"build", # Gradle
"bin", # Eclipse
"out", # IntelliJ IDEA
"classes", # General
"dist", # General
"lib" # General
"target", # Maven
"build", # Gradle
"bin", # Eclipse
"out", # IntelliJ IDEA
"classes", # General
"dist", # General
"lib", # General
]
def setupRuntimeDependencies(self, logger: MultilspyLogger, config: MultilspyConfig) -> RuntimeDependencyPaths:
def setupRuntimeDependencies(self, logger: LanguageServerLogger, config: LanguageServerConfig) -> RuntimeDependencyPaths:
"""
Setup runtime dependencies for EclipseJDTLS.
"""
platformId = PlatformUtils.get_platform_id()
with open(str(PurePath(os.path.dirname(__file__), "runtime_dependencies.json")), "r", encoding="utf-8") as f:
with open(str(PurePath(os.path.dirname(__file__), "runtime_dependencies.json")), encoding="utf-8") as f:
runtimeDependencies = json.load(f)
del runtimeDependencies["_description"]
@@ -194,9 +188,7 @@ class EclipseJDTLS(LanguageServer):
assert os.path.exists(gradle_path)
dependency = runtimeDependencies["vscode-java"][platformId.value]
vscode_java_path = str(
PurePath(os.path.abspath(os.path.dirname(__file__)), "static", dependency["relative_extraction_path"])
)
vscode_java_path = str(PurePath(os.path.abspath(os.path.dirname(__file__)), "static", dependency["relative_extraction_path"]))
os.makedirs(vscode_java_path, exist_ok=True)
jre_home_path = str(PurePath(vscode_java_path, dependency["jre_home_path"]))
jre_path = str(PurePath(vscode_java_path, dependency["jre_path"]))
@@ -213,9 +205,7 @@ class EclipseJDTLS(LanguageServer):
os.path.exists(jdtls_readonly_config_path),
]
):
FileUtils.download_and_extract_archive(
logger, dependency["url"], vscode_java_path, dependency["archiveType"]
)
FileUtils.download_and_extract_archive(logger, dependency["url"], vscode_java_path, dependency["archiveType"])
os.chmod(jre_path, stat.S_IEXEC)
@@ -240,9 +230,7 @@ class EclipseJDTLS(LanguageServer):
os.path.exists(intellisense_members_path),
]
):
FileUtils.download_and_extract_archive(
logger, dependency["url"], intellicode_directory_path, dependency["archiveType"]
)
FileUtils.download_and_extract_archive(logger, dependency["url"], intellicode_directory_path, dependency["archiveType"])
assert os.path.exists(intellicode_directory_path)
assert os.path.exists(intellicode_jar_path)
@@ -264,7 +252,7 @@ class EclipseJDTLS(LanguageServer):
Returns the initialize parameters for the EclipseJDTLS server.
"""
# Look into https://github.com/eclipse/eclipse.jdt.ls/blob/master/org.eclipse.jdt.ls.core/src/org/eclipse/jdt/ls/core/internal/preferences/Preferences.java to understand all the options available
with open(str(PurePath(os.path.dirname(__file__), "initialize_params.json")), "r", encoding="utf-8") as f:
with open(str(PurePath(os.path.dirname(__file__), "initialize_params.json")), encoding="utf-8") as f:
d: InitializeParams = json.load(f)
del d["_description"]
@@ -300,47 +288,30 @@ class EclipseJDTLS(LanguageServer):
d["initializationOptions"]["bundles"] = bundles
assert d["initializationOptions"]["settings"]["java"]["configuration"]["runtimes"] == [
{"name": "JavaSE-17", "path": "static/vscode-java/extension/jre/17.0.8.1-linux-x86_64", "default": True}
{"name": "JavaSE-21", "path": "static/vscode-java/extension/jre/21.0.7-linux-x86_64", "default": True}
]
d["initializationOptions"]["settings"]["java"]["configuration"]["runtimes"] = [
{"name": "JavaSE-17", "path": self.runtime_dependency_paths.jre_home_path, "default": True}
{"name": "JavaSE-21", "path": self.runtime_dependency_paths.jre_home_path, "default": True}
]
for runtime in d["initializationOptions"]["settings"]["java"]["configuration"]["runtimes"]:
assert "name" in runtime
assert "path" in runtime
assert os.path.exists(
runtime["path"]
), f"Runtime required for eclipse_jdtls at path {runtime['path']} does not exist"
assert os.path.exists(runtime["path"]), f"Runtime required for eclipse_jdtls at path {runtime['path']} does not exist"
assert d["initializationOptions"]["settings"]["java"]["import"]["gradle"]["home"] == "abs(static/gradle-7.3.3)"
d["initializationOptions"]["settings"]["java"]["import"]["gradle"][
"home"
] = self.runtime_dependency_paths.gradle_path
d["initializationOptions"]["settings"]["java"]["import"]["gradle"]["home"] = self.runtime_dependency_paths.gradle_path
d["initializationOptions"]["settings"]["java"]["import"]["gradle"]["java"][
"home"
] = self.runtime_dependency_paths.jre_path
d["initializationOptions"]["settings"]["java"]["import"]["gradle"]["java"]["home"] = self.runtime_dependency_paths.jre_path
return d
@asynccontextmanager
async def start_server(self) -> AsyncIterator["EclipseJDTLS"]:
def _start_server(self):
"""
Starts the Eclipse JDTLS Language Server, waits for the server to be ready and yields the LanguageServer instance.
Usage:
```
async with lsp.start_server():
# LanguageServer has been initialized and ready to serve requests
await lsp.request_definition(...)
await lsp.request_references(...)
# Shutdown the LanguageServer on exit from scope
# LanguageServer has been shutdown
```
Starts the Eclipse JDTLS Language Server
"""
async def register_capability_handler(params):
def register_capability_handler(params):
assert "registrations" in params
for registration in params["registrations"]:
if registration["method"] == "textDocument/completion":
@@ -358,22 +329,22 @@ class EclipseJDTLS(LanguageServer):
self.intellicode_enable_command_available.set()
return
async def lang_status_handler(params):
def lang_status_handler(params):
# TODO: Should we wait for
# server -> client: {'jsonrpc': '2.0', 'method': 'language/status', 'params': {'type': 'ProjectStatus', 'message': 'OK'}}
# Before proceeding?
if params["type"] == "ServiceReady" and params["message"] == "ServiceReady":
self.service_ready_event.set()
async def execute_client_command_handler(params):
def execute_client_command_handler(params):
assert params["command"] == "_java.reloadBundles.command"
assert params["arguments"] == []
return []
async def window_log_message(msg):
def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
async def do_nothing(params):
def do_nothing(params):
return
self.server.on_request("client/registerCapability", register_capability_handler)
@@ -384,42 +355,34 @@ class EclipseJDTLS(LanguageServer):
self.server.on_notification("textDocument/publishDiagnostics", do_nothing)
self.server.on_notification("language/actionableNotification", do_nothing)
async with super().start_server():
self.logger.log("Starting EclipseJDTLS server process", logging.INFO)
await self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log("Starting EclipseJDTLS server process", logging.INFO)
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = await self.server.send.initialize(initialize_params)
assert init_response["capabilities"]["textDocumentSync"]["change"] == 2
assert "completionProvider" not in init_response["capabilities"]
assert "executeCommandProvider" not in init_response["capabilities"]
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = self.server.send.initialize(initialize_params)
assert init_response["capabilities"]["textDocumentSync"]["change"] == 2
assert "completionProvider" not in init_response["capabilities"]
assert "executeCommandProvider" not in init_response["capabilities"]
self.server.notify.initialized({})
self.server.notify.initialized({})
self.server.notify.workspace_did_change_configuration(
{"settings": initialize_params["initializationOptions"]["settings"]}
)
self.server.notify.workspace_did_change_configuration({"settings": initialize_params["initializationOptions"]["settings"]})
await self.intellicode_enable_command_available.wait()
self.intellicode_enable_command_available.wait()
java_intellisense_members_path = self.runtime_dependency_paths.intellisense_members_path
assert os.path.exists(java_intellisense_members_path)
intellicode_enable_result = await self.server.send.execute_command(
{
"command": "java.intellicode.enable",
"arguments": [True, java_intellisense_members_path],
}
)
assert intellicode_enable_result
java_intellisense_members_path = self.runtime_dependency_paths.intellisense_members_path
assert os.path.exists(java_intellisense_members_path)
intellicode_enable_result = self.server.send.execute_command(
{
"command": "java.intellicode.enable",
"arguments": [True, java_intellisense_members_path],
}
)
assert intellicode_enable_result
# TODO: Add comments about why we wait here, and how this can be optimized
await self.service_ready_event.wait()
yield self
await self.server.shutdown()
await self.server.stop()
# TODO: Add comments about why we wait here, and how this can be optimized
self.service_ready_event.wait()
src/multilspy/language_servers/eclipse_jdtls/initialize_params.json → src/solidlsp/language_servers/eclipse_jdtls/initialize_params.json
+3 -3
View File
@@ -510,8 +510,8 @@
"workspaceCacheLimit": 90,
"runtimes": [
{
"name": "JavaSE-17",
"path": "static/vscode-java/extension/jre/17.0.8.1-linux-x86_64",
"name": "JavaSE-21",
"path": "static/vscode-java/extension/jre/21.0.7-linux-x86_64",
"default": true
}
]
@@ -535,7 +535,7 @@
"version": null,
"home": "abs(static/gradle-7.3.3)",
"java": {
"home": "abs(static/launch_jres/17.0.6-linux-x86_64)"
"home": "abs(static/launch_jres/21.0.7-linux-x86_64)"
},
"offline": {
"enabled": false
src/multilspy/language_servers/eclipse_jdtls/runtime_dependencies.json → src/solidlsp/language_servers/eclipse_jdtls/runtime_dependencies.json
+22 -22
View File
@@ -9,53 +9,53 @@
},
"vscode-java": {
"darwin-arm64": {
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@darwin-arm64-1.23.0.vsix",
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.42.0/java-darwin-arm64-1.42.0-561.vsix",
"archiveType": "zip",
"relative_extraction_path": "vscode-java"
},
"osx-arm64": {
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@darwin-x64-1.23.0.vsix",
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.42.0/java-darwin-arm64-1.42.0-561.vsix",
"archiveType": "zip",
"relative_extraction_path": "vscode-java",
"jre_home_path": "extension/jre/17.0.8.1-macosx-x86_64",
"jre_path": "extension/jre/17.0.8.1-macosx-x86_64/bin/java",
"lombok_jar_path": "extension/lombok/lombok-1.18.30.jar",
"jdtls_launcher_jar_path": "extension/server/plugins/org.eclipse.equinox.launcher_1.6.500.v20230717-2134.jar",
"jre_home_path": "extension/jre/21.0.7-macosx-aarch64",
"jre_path": "extension/jre/21.0.7-macosx-aarch64/bin/java",
"lombok_jar_path": "extension/lombok/lombok-1.18.36.jar",
"jdtls_launcher_jar_path": "extension/server/plugins/org.eclipse.equinox.launcher_1.7.0.v20250424-1814.jar",
"jdtls_readonly_config_path": "extension/server/config_mac_arm"
},
"osx-x64": {
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@darwin-x64-1.23.0.vsix",
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.42.0/java-darwin-x64-1.42.0-561.vsix",
"archiveType": "zip",
"relative_extraction_path": "vscode-java",
"jre_home_path": "extension/jre/17.0.8.1-macosx-x86_64",
"jre_path": "extension/jre/17.0.8.1-macosx-x86_64/bin/java",
"lombok_jar_path": "extension/lombok/lombok-1.18.30.jar",
"jdtls_launcher_jar_path": "extension/server/plugins/org.eclipse.equinox.launcher_1.6.500.v20230717-2134.jar",
"jre_home_path": "extension/jre/21.0.7-macosx-x86_64",
"jre_path": "extension/jre/21.0.7-macosx-x86_64/bin/java",
"lombok_jar_path": "extension/lombok/lombok-1.18.36.jar",
"jdtls_launcher_jar_path": "extension/server/plugins/org.eclipse.equinox.launcher_1.7.0.v20250424-1814.jar",
"jdtls_readonly_config_path": "extension/server/config_mac"
},
"linux-arm64": {
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@linux-arm64-1.23.0.vsix",
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.42.0/java-linux-arm64-1.42.0-561.vsix",
"archiveType": "zip",
"relative_extraction_path": "vscode-java"
},
"linux-x64": {
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@linux-x64-1.23.0.vsix",
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.42.0/java-linux-x64-1.42.0-561.vsix",
"archiveType": "zip",
"relative_extraction_path": "vscode-java",
"jre_home_path": "extension/jre/17.0.8.1-linux-x86_64",
"jre_path": "extension/jre/17.0.8.1-linux-x86_64/bin/java",
"lombok_jar_path": "extension/lombok/lombok-1.18.30.jar",
"jdtls_launcher_jar_path": "extension/server/plugins/org.eclipse.equinox.launcher_1.6.500.v20230717-2134.jar",
"jre_home_path": "extension/jre/21.0.7-linux-x86_64",
"jre_path": "extension/jre/21.0.7-linux-x86_64/bin/java",
"lombok_jar_path": "extension/lombok/lombok-1.18.36.jar",
"jdtls_launcher_jar_path": "extension/server/plugins/org.eclipse.equinox.launcher_1.7.0.v20250424-1814.jar",
"jdtls_readonly_config_path": "extension/server/config_linux"
},
"win-x64": {
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@win32-x64-1.23.0.vsix",
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.42.0/java-win32-x64-1.42.0-561.vsix",
"archiveType": "zip",
"relative_extraction_path": "vscode-java",
"jre_home_path": "extension/jre/17.0.8.1-win32-x86_64",
"jre_path": "extension/jre/17.0.8.1-win32-x86_64/bin/java.exe",
"lombok_jar_path": "extension/lombok/lombok-1.18.30.jar",
"jdtls_launcher_jar_path": "extension/server/plugins/org.eclipse.equinox.launcher_1.6.500.v20230717-2134.jar",
"jre_home_path": "extension/jre/21.0.7-win32-x86_64",
"jre_path": "extension/jre/21.0.7-win32-x86_64/bin/java.exe",
"lombok_jar_path": "extension/lombok/lombok-1.18.36.jar",
"jdtls_launcher_jar_path": "extension/server/plugins/org.eclipse.equinox.launcher_1.7.0.v20250424-1814.jar",
"jdtls_readonly_config_path": "extension/server/config_win"
}
},
src/multilspy/language_servers/gopls/gopls.py → src/solidlsp/language_servers/gopls/gopls.py
+41 -50
View File
@@ -1,29 +1,24 @@
import asyncio
import json
import logging
import os
import pathlib
import subprocess
from contextlib import asynccontextmanager
from typing import AsyncIterator, List
import threading
from overrides import override
from multilspy import multilspy_types
from multilspy.lsp_protocol_handler import lsp_types
from multilspy.multilspy_exceptions import MultilspyException
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import Error, ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import InitializeParams
from multilspy.multilspy_config import MultilspyConfig
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
class Gopls(LanguageServer):
class Gopls(SolidLanguageServer):
"""
Provides Go specific instantiation of the LanguageServer class using gopls.
"""
@override
def is_ignored_dirname(self, dirname: str) -> bool:
# For Go projects, we should ignore:
@@ -36,7 +31,7 @@ class Gopls(LanguageServer):
def _get_go_version():
"""Get the installed Go version or None if not found."""
try:
result = subprocess.run(['go', 'version'], capture_output=True, text=True)
result = subprocess.run(["go", "version"], capture_output=True, text=True, check=False)
if result.returncode == 0:
return result.stdout.strip()
except FileNotFoundError:
@@ -47,7 +42,7 @@ class Gopls(LanguageServer):
def _get_gopls_version():
"""Get the installed gopls version or None if not found."""
try:
result = subprocess.run(['gopls', 'version'], capture_output=True, text=True)
result = subprocess.run(["gopls", "version"], capture_output=True, text=True, check=False)
if result.returncode == 0:
return result.stdout.strip()
except FileNotFoundError:
@@ -62,8 +57,10 @@ class Gopls(LanguageServer):
"""
go_version = cls._get_go_version()
if not go_version:
raise RuntimeError("Go is not installed. Please install Go from https://golang.org/doc/install and make sure it is added to your PATH.")
raise RuntimeError(
"Go is not installed. Please install Go from https://golang.org/doc/install and make sure it is added to your PATH."
)
gopls_version = cls._get_gopls_version()
if not gopls_version:
raise RuntimeError(
@@ -71,12 +68,12 @@ class Gopls(LanguageServer):
"Please install gopls as described in https://pkg.go.dev/golang.org/x/tools/gopls#section-readme\n\n"
"After installation, make sure it is added to your PATH (it might be installed in a different location than Go)."
)
return True
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
def __init__(self, config: LanguageServerConfig, logger: LanguageServerLogger, repository_root_path: str):
self.setup_runtime_dependency()
super().__init__(
config,
logger,
@@ -84,14 +81,14 @@ class Gopls(LanguageServer):
ProcessLaunchInfo(cmd="gopls", cwd=repository_root_path),
"go",
)
self.server_ready = asyncio.Event()
self.server_ready = threading.Event()
self.request_id = 0
def _get_initialize_params(self, repository_absolute_path: str) -> InitializeParams:
"""
Returns the initialize params for the TypeScript Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r", encoding="utf-8") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -111,16 +108,16 @@ class Gopls(LanguageServer):
return d
@asynccontextmanager
async def start_server(self) -> AsyncIterator["Gopls"]:
def _start_server(self):
"""Start gopls server process"""
async def register_capability_handler(params):
def register_capability_handler(params):
return
async def window_log_message(msg):
def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
async def do_nothing(params):
def do_nothing(params):
return
self.server.on_request("client/registerCapability", register_capability_handler)
@@ -128,30 +125,24 @@ class Gopls(LanguageServer):
self.server.on_notification("$/progress", do_nothing)
self.server.on_notification("textDocument/publishDiagnostics", do_nothing)
async with super().start_server():
self.logger.log("Starting gopls server process", logging.INFO)
await self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log("Starting gopls server process", logging.INFO)
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = await self.server.send.initialize(initialize_params)
# Verify server capabilities
assert "textDocumentSync" in init_response["capabilities"]
assert "completionProvider" in init_response["capabilities"]
assert "definitionProvider" in init_response["capabilities"]
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = self.server.send.initialize(initialize_params)
self.server.notify.initialized({})
self.completions_available.set()
# Verify server capabilities
assert "textDocumentSync" in init_response["capabilities"]
assert "completionProvider" in init_response["capabilities"]
assert "definitionProvider" in init_response["capabilities"]
# gopls server is typically ready immediately after initialization
self.server_ready.set()
await self.server_ready.wait()
self.server.notify.initialized({})
self.completions_available.set()
yield self
await self.server.shutdown()
await self.server.stop()
# gopls server is typically ready immediately after initialization
self.server_ready.set()
self.server_ready.wait()
src/multilspy/language_servers/gopls/initialize_params.json → src/solidlsp/language_servers/gopls/initialize_params.json
View File
src/multilspy/language_servers/intelephense/initialize_params.json → src/solidlsp/language_servers/intelephense/initialize_params.json
View File
src/multilspy/language_servers/intelephense/intelephense.py → src/solidlsp/language_servers/intelephense/intelephense.py
+56 -71
View File
@@ -2,40 +2,38 @@
Provides PHP specific instantiation of the LanguageServer class using Intelephense.
"""
import asyncio
import json
import shutil
import logging
import os
import subprocess
import pathlib
from contextlib import asynccontextmanager
import shutil
import subprocess
from time import sleep
from typing import AsyncIterator
from overrides import override
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import DefinitionParams, InitializeParams
from multilspy.multilspy_config import MultilspyConfig
from multilspy.multilspy_utils import PlatformUtils, PlatformId
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.ls_utils import PlatformId, PlatformUtils
from solidlsp.lsp_protocol_handler.lsp_types import DefinitionParams, InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
class Intelephense(LanguageServer):
class Intelephense(SolidLanguageServer):
"""
Provides PHP specific instantiation of the LanguageServer class using Intelephense.
"""
@override
def is_ignored_dirname(self, dirname: str) -> bool:
# For PHP projects, we should ignore:
# - vendor: third-party dependencies managed by Composer
# - node_modules: if the project has JavaScript components
# - cache: commonly used for caching
return super().is_ignored_dirname(dirname) or dirname in ["node_modules", "vendor", "cache"]
return super().is_ignored_dirname(dirname) or dirname in ["node_modules", "vendor", "cache"]
def setup_runtime_dependencies(self, logger: MultilspyLogger, config: MultilspyConfig) -> str:
def setup_runtime_dependencies(self, logger: LanguageServerLogger, config: LanguageServerConfig) -> str:
"""
Setup runtime dependencies for Intelephense.
"""
@@ -52,17 +50,17 @@ class Intelephense(LanguageServer):
]
assert platform_id in valid_platforms, f"Platform {platform_id} is not supported for multilspy PHP at the moment"
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r", encoding="utf-8") as f:
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
runtime_dependencies = d.get("runtimeDependencies", [])
intelephense_ls_dir = os.path.join(os.path.dirname(__file__), "static", "php-lsp")
# Verify both node and npm are installed
is_node_installed = shutil.which('node') is not None
is_node_installed = shutil.which("node") is not None
assert is_node_installed, "node is not installed or isn't in PATH. Please install NodeJS and try again."
is_npm_installed = shutil.which('npm') is not None
is_npm_installed = shutil.which("npm") is not None
assert is_npm_installed, "npm is not installed or isn't in PATH. Please install npm and try again."
# Install intelephense if not already installed
@@ -77,11 +75,12 @@ class Intelephense(LanguageServer):
check=True,
cwd=intelephense_ls_dir,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL
stderr=subprocess.DEVNULL,
)
else:
# On Unix-like systems, run as non-root user
import pwd
user = pwd.getpwuid(os.getuid()).pw_name
subprocess.run(
dependency["command"],
@@ -90,33 +89,26 @@ class Intelephense(LanguageServer):
user=user,
cwd=intelephense_ls_dir,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL
stderr=subprocess.DEVNULL,
)
intelephense_executable_path = os.path.join(intelephense_ls_dir, "node_modules", ".bin", "intelephense")
assert os.path.exists(intelephense_executable_path), "intelephense executable not found. Please install intelephense and try again."
return f"{intelephense_executable_path} --stdio"
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
def __init__(self, config: LanguageServerConfig, logger: LanguageServerLogger, repository_root_path: str):
# Setup runtime dependencies before initializing
intelephense_cmd = self.setup_runtime_dependencies(logger, config)
super().__init__(
config,
logger,
repository_root_path,
ProcessLaunchInfo(cmd=intelephense_cmd, cwd=repository_root_path),
"php"
)
self.server_ready = asyncio.Event()
super().__init__(config, logger, repository_root_path, ProcessLaunchInfo(cmd=intelephense_cmd, cwd=repository_root_path), "php")
self.request_id = 0
def _get_initialize_params(self, repository_absolute_path: str) -> InitializeParams:
"""
Returns the initialize params for the TypeScript Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r", encoding="utf-8") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -136,16 +128,16 @@ class Intelephense(LanguageServer):
return d
@asynccontextmanager
async def start_server(self) -> AsyncIterator["Intelephense"]:
def _start_server(self):
"""Start Intelephense server process"""
async def register_capability_handler(params):
def register_capability_handler(params):
return
async def window_log_message(msg):
def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
async def do_nothing(params):
def do_nothing(params):
return
self.server.on_request("client/registerCapability", register_capability_handler)
@@ -153,51 +145,44 @@ class Intelephense(LanguageServer):
self.server.on_notification("$/progress", do_nothing)
self.server.on_notification("textDocument/publishDiagnostics", do_nothing)
async with super().start_server():
self.logger.log("Starting Intelephense server process", logging.INFO)
await self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log("Starting Intelephense server process", logging.INFO)
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = await self.server.send.initialize(initialize_params)
self.logger.log(
"After sent initialize params",
logging.INFO,
)
# Verify server capabilities
assert "textDocumentSync" in init_response["capabilities"]
assert "completionProvider" in init_response["capabilities"]
assert "definitionProvider" in init_response["capabilities"]
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = self.server.send.initialize(initialize_params)
self.logger.log(
"After sent initialize params",
logging.INFO,
)
self.server.notify.initialized({})
self.completions_available.set()
# Verify server capabilities
assert "textDocumentSync" in init_response["capabilities"]
assert "completionProvider" in init_response["capabilities"]
assert "definitionProvider" in init_response["capabilities"]
# Intelephense server is typically ready immediately after initialization
self.server_ready.set()
await self.server_ready.wait()
self.server.notify.initialized({})
self.completions_available.set()
yield self
# Intelephense server is typically ready immediately after initialization
# TODO: This is probably incorrect; the server does send an initialized notification, which we could wait for!
await self.server.shutdown()
await self.server.stop()
@override
# For some reason, the LS may need longer to process this, so we just retry
async def _send_references_request(self, relative_file_path: str, line: int, column: int):
def _send_references_request(self, relative_file_path: str, line: int, column: int):
# TODO: The LS doesn't return references contained in other files if it doesn't sleep. This is
# despite the LS having processed requests already. I don't know what causes this, but sleeping
# one second helps. It may be that sleeping only once is enough but that's hard to reliably test.
# May be related to the time it takes to read the files or something like that.
# The sleeping doesn't seem to be needed on all systems
sleep(1)
return await super()._send_references_request(relative_file_path, line, column)
return super()._send_references_request(relative_file_path, line, column)
@override
async def _send_definition_request(self, definition_params: DefinitionParams):
def _send_definition_request(self, definition_params: DefinitionParams):
# TODO: same as above, also only a problem if the definition is in another file
sleep(1)
return await super()._send_definition_request(definition_params)
return super()._send_definition_request(definition_params)
src/multilspy/language_servers/intelephense/runtime_dependencies.json → src/solidlsp/language_servers/intelephense/runtime_dependencies.json
View File
src/multilspy/language_servers/jedi_language_server/initialize_params.json → src/solidlsp/language_servers/jedi_language_server/initialize_params.json
View File
src/multilspy/language_servers/jedi_language_server/jedi_server.py → src/solidlsp/language_servers/jedi_language_server/jedi_server.py
+30 -49
View File
@@ -6,24 +6,22 @@ import json
import logging
import os
import pathlib
from contextlib import asynccontextmanager
from typing import AsyncIterator, Tuple
from overrides import override
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import InitializeParams
from multilspy.multilspy_config import MultilspyConfig
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
class JediServer(LanguageServer):
class JediServer(SolidLanguageServer):
"""
Provides Python specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Python.
"""
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
def __init__(self, config: LanguageServerConfig, logger: LanguageServerLogger, repository_root_path: str):
"""
Creates a JediServer instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead.
"""
@@ -34,7 +32,7 @@ class JediServer(LanguageServer):
ProcessLaunchInfo(cmd="jedi-language-server", cwd=repository_root_path),
"python",
)
@override
def is_ignored_dirname(self, dirname: str) -> bool:
return super().is_ignored_dirname(dirname) or dirname in ["venv", "__pycache__"]
@@ -43,7 +41,7 @@ class JediServer(LanguageServer):
"""
Returns the initialize params for the Jedi Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r", encoding="utf-8") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -63,33 +61,22 @@ class JediServer(LanguageServer):
return d
@asynccontextmanager
async def start_server(self) -> AsyncIterator["JediServer"]:
def _start_server(self):
"""
Starts the JEDI Language Server, waits for the server to be ready and yields the LanguageServer instance.
Usage:
```
async with lsp.start_server():
# LanguageServer has been initialized and ready to serve requests
await lsp.request_definition(...)
await lsp.request_references(...)
# Shutdown the LanguageServer on exit from scope
# LanguageServer has been shutdown
```
Starts the JEDI Language Server
"""
async def execute_client_command_handler(params):
def execute_client_command_handler(params):
return []
async def do_nothing(params):
def do_nothing(params):
return
async def check_experimental_status(params):
def check_experimental_status(params):
if params["quiescent"] == True:
self.completions_available.set()
async def window_log_message(msg):
def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
self.server.on_request("client/registerCapability", do_nothing)
@@ -101,26 +88,20 @@ class JediServer(LanguageServer):
self.server.on_notification("language/actionableNotification", do_nothing)
self.server.on_notification("experimental/serverStatus", check_experimental_status)
async with super().start_server():
self.logger.log("Starting jedi-language-server server process", logging.INFO)
await self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log("Starting jedi-language-server server process", logging.INFO)
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = await self.server.send.initialize(initialize_params)
assert init_response["capabilities"]["textDocumentSync"]["change"] == 2
assert "completionProvider" in init_response["capabilities"]
assert init_response["capabilities"]["completionProvider"] == {
"triggerCharacters": [".", "'", '"'],
"resolveProvider": True,
}
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = self.server.send.initialize(initialize_params)
assert init_response["capabilities"]["textDocumentSync"]["change"] == 2
assert "completionProvider" in init_response["capabilities"]
assert init_response["capabilities"]["completionProvider"] == {
"triggerCharacters": [".", "'", '"'],
"resolveProvider": True,
}
self.server.notify.initialized({})
yield self
await self.server.shutdown()
await self.server.stop()
self.server.notify.initialized({})
src/multilspy/language_servers/kotlin_language_server/initialize_params.json → src/solidlsp/language_servers/kotlin_language_server/initialize_params.json
View File
src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py → src/solidlsp/language_servers/kotlin_language_server/kotlin_language_server.py
+64 -89
View File
@@ -2,23 +2,19 @@
Provides Kotlin specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Kotlin.
"""
import asyncio
import dataclasses
import json
import logging
import os
import stat
import pathlib
from contextlib import asynccontextmanager
from typing import AsyncIterator
import stat
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import InitializeParams
from multilspy.multilspy_config import MultilspyConfig
from multilspy.multilspy_utils import FileUtils
from multilspy.multilspy_utils import PlatformUtils
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.ls_utils import FileUtils, PlatformUtils
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
@dataclasses.dataclass
@@ -26,29 +22,30 @@ class KotlinRuntimeDependencyPaths:
"""
Stores the paths to the runtime dependencies of Kotlin Language Server
"""
java_path: str
java_home_path: str
kotlin_executable_path: str
class KotlinLanguageServer(LanguageServer):
class KotlinLanguageServer(SolidLanguageServer):
"""
Provides Kotlin specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Kotlin.
"""
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
def __init__(self, config: LanguageServerConfig, logger: LanguageServerLogger, repository_root_path: str):
"""
Creates a Kotlin Language Server instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead.
"""
runtime_dependency_paths = self.setup_runtime_dependencies(logger, config)
self.runtime_dependency_paths = runtime_dependency_paths
# Create command to execute the Kotlin Language Server script
cmd = f'"{self.runtime_dependency_paths.kotlin_executable_path}"'
# Set environment variables including JAVA_HOME
proc_env = {"JAVA_HOME": self.runtime_dependency_paths.java_home_path}
super().__init__(
config,
logger,
@@ -57,84 +54,82 @@ class KotlinLanguageServer(LanguageServer):
"kotlin",
)
def setup_runtime_dependencies(self, logger: MultilspyLogger, config: MultilspyConfig) -> KotlinRuntimeDependencyPaths:
def setup_runtime_dependencies(self, logger: LanguageServerLogger, config: LanguageServerConfig) -> KotlinRuntimeDependencyPaths:
"""
Setup runtime dependencies for Kotlin Language Server.
"""
platform_id = PlatformUtils.get_platform_id()
# Verify platform support
assert platform_id.value.startswith("win-") or platform_id.value.startswith("linux-") or platform_id.value.startswith("osx-"), "Only Windows, Linux and macOS platforms are supported for Kotlin in multilspy at the moment"
assert (
platform_id.value.startswith("win-") or platform_id.value.startswith("linux-") or platform_id.value.startswith("osx-")
), "Only Windows, Linux and macOS platforms are supported for Kotlin in multilspy at the moment"
# Load dependency information
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r", encoding="utf-8") as f:
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
kotlin_dependency = d["runtimeDependency"]
java_dependency = d["java"][platform_id.value]
# Setup paths for dependencies
static_dir = os.path.join(os.path.dirname(__file__), "static")
os.makedirs(static_dir, exist_ok=True)
# Setup Java paths
java_dir = os.path.join(static_dir, "java")
os.makedirs(java_dir, exist_ok=True)
java_home_path = os.path.join(java_dir, java_dependency["java_home_path"])
java_path = os.path.join(java_dir, java_dependency["java_path"])
# Download and extract Java if not exists
if not os.path.exists(java_path):
logger.log(f"Downloading Java for {platform_id.value}...", logging.INFO)
FileUtils.download_and_extract_archive(
logger, java_dependency["url"], java_dir, java_dependency["archiveType"]
)
FileUtils.download_and_extract_archive(logger, java_dependency["url"], java_dir, java_dependency["archiveType"])
# Make Java executable
if not platform_id.value.startswith("win-"):
os.chmod(java_path, 0o755)
assert os.path.exists(java_path), f"Java executable not found at {java_path}"
# Setup Kotlin Language Server paths
kotlin_ls_dir = os.path.join(static_dir, "server")
# Get platform-specific executable script path
if platform_id.value.startswith("win-"):
kotlin_script = os.path.join(kotlin_ls_dir, "bin", "kotlin-language-server.bat")
else:
kotlin_script = os.path.join(kotlin_ls_dir, "bin", "kotlin-language-server")
# Download and extract Kotlin Language Server if script doesn't exist
if not os.path.exists(kotlin_script):
logger.log("Downloading Kotlin Language Server...", logging.INFO)
FileUtils.download_and_extract_archive(
logger, kotlin_dependency["url"], static_dir, kotlin_dependency["archiveType"]
)
FileUtils.download_and_extract_archive(logger, kotlin_dependency["url"], static_dir, kotlin_dependency["archiveType"])
# Make script executable on Unix platforms
if os.path.exists(kotlin_script) and not platform_id.value.startswith("win-"):
os.chmod(kotlin_script, stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR | stat.S_IRGRP | stat.S_IXGRP | stat.S_IROTH | stat.S_IXOTH)
os.chmod(
kotlin_script, stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR | stat.S_IRGRP | stat.S_IXGRP | stat.S_IROTH | stat.S_IXOTH
)
# Use script file
if os.path.exists(kotlin_script):
kotlin_executable_path = kotlin_script
logger.log(f"Using Kotlin Language Server script at {kotlin_script}", logging.INFO)
else:
raise FileNotFoundError(f"Kotlin Language Server script not found at {kotlin_script}")
return KotlinRuntimeDependencyPaths(
java_path=java_path,
java_home_path=java_home_path,
kotlin_executable_path=kotlin_executable_path
java_path=java_path, java_home_path=java_home_path, kotlin_executable_path=kotlin_executable_path
)
def _get_initialize_params(self, repository_absolute_path: str) -> InitializeParams:
"""
Returns the initialize params for the Kotlin Language Server.
"""
with open(str(pathlib.PurePath(os.path.dirname(__file__), "initialize_params.json")), "r", encoding="utf-8") as f:
with open(str(pathlib.PurePath(os.path.dirname(__file__), "initialize_params.json")), encoding="utf-8") as f:
d: InitializeParams = json.load(f)
del d["_description"]
@@ -155,8 +150,8 @@ class KotlinLanguageServer(LanguageServer):
d["initializationOptions"]["workspaceFolders"] = [pathlib.Path(repository_absolute_path).as_uri()]
assert (
d["workspaceFolders"]
== '[\n {\n "uri": pathlib.Path(repository_absolute_path).as_uri(),\n "name": os.path.basename(repository_absolute_path),\n }\n ]'
d["workspaceFolders"]
== '[\n {\n "uri": pathlib.Path(repository_absolute_path).as_uri(),\n "name": os.path.basename(repository_absolute_path),\n }\n ]'
)
d["workspaceFolders"] = [
{
@@ -167,28 +162,18 @@ class KotlinLanguageServer(LanguageServer):
return d
@asynccontextmanager
async def start_server(self) -> AsyncIterator["KotlinLanguageServer"]:
def _start_server(self):
"""
Starts the Kotlin Language Server
"""
Starts the Kotlin Language Server, waits for the server to be ready and yields the LanguageServer instance.
Usage:
```
async with lsp.start_server():
# LanguageServer has been initialized and ready to serve requests
await lsp.request_definition(...)
await lsp.request_references(...)
# Shutdown the LanguageServer on exit from scope
# LanguageServer has been shutdown
```
"""
async def execute_client_command_handler(params):
def execute_client_command_handler(params):
return []
async def do_nothing(params):
def do_nothing(params):
return
async def window_log_message(msg):
def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
self.server.on_request("client/registerCapability", do_nothing)
@@ -199,36 +184,26 @@ class KotlinLanguageServer(LanguageServer):
self.server.on_notification("textDocument/publishDiagnostics", do_nothing)
self.server.on_notification("language/actionableNotification", do_nothing)
async with super().start_server():
self.logger.log("Starting Kotlin server process", logging.INFO)
await self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log("Starting Kotlin server process", logging.INFO)
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = await self.server.send.initialize(initialize_params)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = self.server.send.initialize(initialize_params)
capabilities = init_response["capabilities"]
assert "textDocumentSync" in capabilities, "Server must support textDocumentSync"
assert "hoverProvider" in capabilities, "Server must support hover"
assert "completionProvider" in capabilities, "Server must support code completion"
assert "signatureHelpProvider" in capabilities, "Server must support signature help"
assert "definitionProvider" in capabilities, "Server must support go to definition"
assert "referencesProvider" in capabilities, "Server must support find references"
assert "documentSymbolProvider" in capabilities, "Server must support document symbols"
assert "workspaceSymbolProvider" in capabilities, "Server must support workspace symbols"
assert "semanticTokensProvider" in capabilities, "Server must support semantic tokens"
self.server.notify.initialized({})
self.completions_available.set()
capabilities = init_response["capabilities"]
assert "textDocumentSync" in capabilities, "Server must support textDocumentSync"
assert "hoverProvider" in capabilities, "Server must support hover"
assert "completionProvider" in capabilities, "Server must support code completion"
assert "signatureHelpProvider" in capabilities, "Server must support signature help"
assert "definitionProvider" in capabilities, "Server must support go to definition"
assert "referencesProvider" in capabilities, "Server must support find references"
assert "documentSymbolProvider" in capabilities, "Server must support document symbols"
assert "workspaceSymbolProvider" in capabilities, "Server must support workspace symbols"
assert "semanticTokensProvider" in capabilities, "Server must support semantic tokens"
yield self
try:
await self.server.shutdown()
except Exception as e:
self.logger.log(f"Error during Kotlin server shutdown: {str(e)}", logging.WARNING)
finally:
await self.server.stop()
self.server.notify.initialized({})
self.completions_available.set()
src/multilspy/language_servers/kotlin_language_server/runtime_dependencies.json → src/solidlsp/language_servers/kotlin_language_server/runtime_dependencies.json
+15 -15
View File
@@ -8,34 +8,34 @@
},
"java": {
"win-x64": {
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@win32-x64-1.23.0.vsix",
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.42.0/java-win32-x64-1.42.0-561.vsix",
"archiveType": "zip",
"java_home_path": "extension/jre/17.0.8.1-win32-x86_64",
"java_path": "extension/jre/17.0.8.1-win32-x86_64/bin/java.exe"
"java_home_path": "extension/jre/21.0.7-win32-x86_64",
"java_path": "extension/jre/21.0.7-win32-x86_64/bin/java.exe"
},
"linux-x64": {
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@linux-x64-1.23.0.vsix",
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.42.0/java-linux-x64-1.42.0-561.vsix",
"archiveType": "zip",
"java_home_path": "extension/jre/17.0.8.1-linux-x86_64",
"java_path": "extension/jre/17.0.8.1-linux-x86_64/bin/java"
"java_home_path": "extension/jre/21.0.7-linux-x86_64",
"java_path": "extension/jre/21.0.7-linux-x86_64/bin/java"
},
"linux-arm64": {
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@linux-arm64-1.23.0.vsix",
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.42.0/java-linux-arm64-1.42.0-561.vsix",
"archiveType": "zip",
"java_home_path": "extension/jre/17.0.8.1-linux-aarch64",
"java_path": "extension/jre/17.0.8.1-linux-aarch64/bin/java"
"java_home_path": "extension/jre/21.0.7-linux-aarch64",
"java_path": "extension/jre/21.0.7-linux-aarch64/bin/java"
},
"osx-x64": {
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@darwin-x64-1.23.0.vsix",
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.42.0/java-darwin-x64-1.42.0-561.vsix",
"archiveType": "zip",
"java_home_path": "extension/jre/17.0.8.1-macosx-x86_64",
"java_path": "extension/jre/17.0.8.1-macosx-x86_64/bin/java"
"java_home_path": "extension/jre/21.0.7-macosx-x86_64",
"java_path": "extension/jre/21.0.7-macosx-x86_64/bin/java"
},
"osx-arm64": {
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.23.0/java@darwin-arm64-1.23.0.vsix",
"url": "https://github.com/redhat-developer/vscode-java/releases/download/v1.42.0/java-darwin-arm64-1.42.0-561.vsix",
"archiveType": "zip",
"java_home_path": "extension/jre/17.0.8.1-macosx-aarch64",
"java_path": "extension/jre/17.0.8.1-macosx-aarch64/bin/java"
"java_home_path": "extension/jre/21.0.7-macosx-aarch64",
"java_path": "extension/jre/21.0.7-macosx-aarch64/bin/java"
}
}
}
src/multilspy/language_servers/omnisharp/initialize_params.json → src/solidlsp/language_servers/omnisharp/initialize_params.json
View File
src/multilspy/language_servers/omnisharp/omnisharp.py → src/solidlsp/language_servers/omnisharp/omnisharp.py
+54 -88
View File
@@ -2,24 +2,23 @@
Provides C# specific instantiation of the LanguageServer class. Contains various configurations and settings specific to C#.
"""
import asyncio
import json
import logging
import os
import pathlib
import stat
from contextlib import asynccontextmanager
from typing import AsyncIterator, Iterable
import threading
from collections.abc import Iterable
from overrides import override
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import InitializeParams
from multilspy.multilspy_config import MultilspyConfig
from multilspy.multilspy_exceptions import MultilspyException
from multilspy.multilspy_utils import FileUtils, PlatformUtils, PlatformId, DotnetVersion
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_exceptions import LanguageServerException
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.ls_utils import DotnetVersion, FileUtils, PlatformId, PlatformUtils
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
def breadth_first_file_scan(root) -> Iterable[str]:
@@ -29,7 +28,7 @@ def breadth_first_file_scan(root) -> Iterable[str]:
"""
dirs = [root]
# while we has dirs to scan
while len(dirs):
while dirs:
next_dirs = []
for parent in dirs:
# scan each dir
@@ -55,12 +54,12 @@ def find_least_depth_sln_file(root_dir) -> str | None:
return None
class OmniSharp(LanguageServer):
class OmniSharp(SolidLanguageServer):
"""
Provides C# specific instantiation of the LanguageServer class. Contains various configurations and settings specific to C#.
"""
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
def __init__(self, config: LanguageServerConfig, logger: LanguageServerLogger, repository_root_path: str):
"""
Creates an OmniSharp instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead.
"""
@@ -69,7 +68,7 @@ class OmniSharp(LanguageServer):
slnfilename = find_least_depth_sln_file(repository_root_path)
if slnfilename is None:
logger.log("No *.sln file found in repository", logging.ERROR)
raise MultilspyException("No SLN file found in repository")
raise LanguageServerException("No SLN file found in repository")
cmd = " ".join(
[
@@ -103,13 +102,12 @@ class OmniSharp(LanguageServer):
"formattingOptions:indentationSize=4",
]
)
super().__init__(
config, logger, repository_root_path, ProcessLaunchInfo(cmd=cmd, cwd=repository_root_path), "csharp"
)
super().__init__(config, logger, repository_root_path, ProcessLaunchInfo(cmd=cmd, cwd=repository_root_path), "csharp")
self.server_ready = threading.Event()
self.definition_available = threading.Event()
self.references_available = threading.Event()
self.definition_available = asyncio.Event()
self.references_available = asyncio.Event()
@override
def is_ignored_dirname(self, dirname: str) -> bool:
return super().is_ignored_dirname(dirname) or dirname in ["bin", "obj"]
@@ -118,7 +116,7 @@ class OmniSharp(LanguageServer):
"""
Returns the initialize params for the Omnisharp Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r", encoding="utf-8") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -138,14 +136,14 @@ class OmniSharp(LanguageServer):
return d
def setupRuntimeDependencies(self, logger: MultilspyLogger, config: MultilspyConfig) -> tuple[str, str]:
def setupRuntimeDependencies(self, logger: LanguageServerLogger, config: LanguageServerConfig) -> tuple[str, str]:
"""
Setup runtime dependencies for OmniSharp.
"""
platform_id = PlatformUtils.get_platform_id()
dotnet_version = PlatformUtils.get_dotnet_version()
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r", encoding="utf-8") as f:
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -156,7 +154,7 @@ class OmniSharp(LanguageServer):
assert dotnet_version in [
DotnetVersion.V6,
DotnetVersion.V7,
DotnetVersion.V8
DotnetVersion.V8,
], "Only dotnet version 6 and 7 are supported in multilspy at the moment"
# TODO: Do away with this assumption
@@ -165,13 +163,11 @@ class OmniSharp(LanguageServer):
dotnet_version = DotnetVersion.V6
runtime_dependencies = d["runtimeDependencies"]
runtime_dependencies = [
dependency for dependency in runtime_dependencies if dependency["platformId"] == platform_id.value
]
runtime_dependencies = [dependency for dependency in runtime_dependencies if dependency["platformId"] == platform_id.value]
runtime_dependencies = [
dependency
for dependency in runtime_dependencies
if not ("dotnet_version" in dependency) or dependency["dotnet_version"] == dotnet_version.value
if "dotnet_version" not in dependency or dependency["dotnet_version"] == dotnet_version.value
]
assert len(runtime_dependencies) == 2
runtime_dependencies = {
@@ -185,9 +181,7 @@ class OmniSharp(LanguageServer):
omnisharp_ls_dir = os.path.join(os.path.dirname(__file__), "static", "OmniSharp")
if not os.path.exists(omnisharp_ls_dir):
os.makedirs(omnisharp_ls_dir)
FileUtils.download_and_extract_archive(
logger, runtime_dependencies["OmniSharp"]["url"], omnisharp_ls_dir, "zip"
)
FileUtils.download_and_extract_archive(logger, runtime_dependencies["OmniSharp"]["url"], omnisharp_ls_dir, "zip")
omnisharp_executable_path = os.path.join(omnisharp_ls_dir, runtime_dependencies["OmniSharp"]["binaryName"])
assert os.path.exists(omnisharp_executable_path)
os.chmod(omnisharp_executable_path, stat.S_IEXEC)
@@ -195,32 +189,18 @@ class OmniSharp(LanguageServer):
razor_omnisharp_ls_dir = os.path.join(os.path.dirname(__file__), "static", "RazorOmnisharp")
if not os.path.exists(razor_omnisharp_ls_dir):
os.makedirs(razor_omnisharp_ls_dir)
FileUtils.download_and_extract_archive(
logger, runtime_dependencies["RazorOmnisharp"]["url"], razor_omnisharp_ls_dir, "zip"
)
razor_omnisharp_dll_path = os.path.join(
razor_omnisharp_ls_dir, runtime_dependencies["RazorOmnisharp"]["dll_path"]
)
FileUtils.download_and_extract_archive(logger, runtime_dependencies["RazorOmnisharp"]["url"], razor_omnisharp_ls_dir, "zip")
razor_omnisharp_dll_path = os.path.join(razor_omnisharp_ls_dir, runtime_dependencies["RazorOmnisharp"]["dll_path"])
assert os.path.exists(razor_omnisharp_dll_path)
return omnisharp_executable_path, razor_omnisharp_dll_path
@asynccontextmanager
async def start_server(self) -> AsyncIterator["OmniSharp"]:
def _start_server(self):
"""
Starts the Omnisharp Language Server, waits for the server to be ready and yields the LanguageServer instance.
Usage:
```
async with lsp.start_server():
# LanguageServer has been initialized and ready to serve requests
await lsp.request_definition(...)
await lsp.request_references(...)
# Shutdown the LanguageServer on exit from scope
# LanguageServer has been shutdown
Starts the Omnisharp Language Server
"""
async def register_capability_handler(params):
def register_capability_handler(params):
assert "registrations" in params
for registration in params["registrations"]:
if registration["method"] == "textDocument/definition":
@@ -230,7 +210,7 @@ class OmniSharp(LanguageServer):
if registration["method"] == "textDocument/completion":
self.completions_available.set()
async def lang_status_handler(params):
def lang_status_handler(params):
# TODO: Should we wait for
# server -> client: {'jsonrpc': '2.0', 'method': 'language/status', 'params': {'type': 'ProjectStatus', 'message': 'OK'}}
# Before proceeding?
@@ -238,20 +218,20 @@ class OmniSharp(LanguageServer):
# self.service_ready_event.set()
pass
async def execute_client_command_handler(params):
def execute_client_command_handler(params):
return []
async def do_nothing(params):
def do_nothing(params):
return
async def check_experimental_status(params):
def check_experimental_status(params):
if params["quiescent"] == True:
self.server_ready.set()
async def window_log_message(msg):
def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
async def workspace_configuration_handler(params):
def workspace_configuration_handler(params):
# TODO: We do not know the appropriate way to handle this request. Should ideally contact the OmniSharp dev team
return [
{
@@ -373,37 +353,23 @@ class OmniSharp(LanguageServer):
self.server.on_notification("experimental/serverStatus", check_experimental_status)
self.server.on_request("workspace/configuration", workspace_configuration_handler)
async with super().start_server():
self.logger.log("Starting OmniSharp server process", logging.INFO)
await self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log("Starting OmniSharp server process", logging.INFO)
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = await self.server.send.initialize(initialize_params)
self.server.notify.initialized({})
with open(os.path.join(os.path.dirname(__file__), "workspace_did_change_configuration.json"), "r", encoding="utf-8") as f:
self.server.notify.workspace_did_change_configuration({
"settings": json.load(f)
})
assert "capabilities" in init_response
if (
"definitionProvider" in init_response["capabilities"]
and init_response["capabilities"]["definitionProvider"]
):
self.definition_available.set()
if (
"referencesProvider" in init_response["capabilities"]
and init_response["capabilities"]["referencesProvider"]
):
self.references_available.set()
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = self.server.send.initialize(initialize_params)
self.server.notify.initialized({})
with open(os.path.join(os.path.dirname(__file__), "workspace_did_change_configuration.json"), encoding="utf-8") as f:
self.server.notify.workspace_did_change_configuration({"settings": json.load(f)})
assert "capabilities" in init_response
if "definitionProvider" in init_response["capabilities"] and init_response["capabilities"]["definitionProvider"]:
self.definition_available.set()
if "referencesProvider" in init_response["capabilities"] and init_response["capabilities"]["referencesProvider"]:
self.references_available.set()
await self.definition_available.wait()
await self.references_available.wait()
yield self
await self.server.shutdown()
await self.server.stop()
self.definition_available.wait()
self.references_available.wait()
src/multilspy/language_servers/omnisharp/runtime_dependencies.json → src/solidlsp/language_servers/omnisharp/runtime_dependencies.json
View File
src/multilspy/language_servers/omnisharp/workspace_did_change_configuration.json → src/solidlsp/language_servers/omnisharp/workspace_did_change_configuration.json
View File
src/multilspy/language_servers/pyright_language_server/pyright_server.py → src/solidlsp/language_servers/pyright_language_server/pyright_server.py
+75 -39
View File
@@ -2,28 +2,28 @@
Provides Python specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Python.
"""
import json
import logging
import os
import pathlib
from contextlib import asynccontextmanager
from typing import AsyncIterator, Tuple
import re
import threading
from overrides import override
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import InitializeParams
from multilspy.multilspy_config import MultilspyConfig
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
class PyrightServer(LanguageServer):
class PyrightServer(SolidLanguageServer):
"""
Provides Python specific instantiation of the LanguageServer class using Pyright.
Contains various configurations and settings specific to Python.
"""
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
def __init__(self, config: LanguageServerConfig, logger: LanguageServerLogger, repository_root_path: str):
"""
Creates a PyrightServer instance. This class is not meant to be instantiated directly.
Use LanguageServer.create() instead.
@@ -37,7 +37,11 @@ class PyrightServer(LanguageServer):
ProcessLaunchInfo(cmd="python -m pyright.langserver --stdio", cwd=repository_root_path),
"python",
)
# Event to signal when initial workspace analysis is complete
self.analysis_complete = threading.Event()
self.found_source_files = False
@override
def is_ignored_dirname(self, dirname: str) -> bool:
return super().is_ignored_dirname(dirname) or dirname in ["venv", "__pycache__"]
@@ -47,7 +51,7 @@ class PyrightServer(LanguageServer):
Returns the initialize params for the Pyright Language Server.
"""
# Create basic initialization parameters
initialize_params: InitializeParams = { # type: ignore
initialize_params: InitializeParams = { # type: ignore
"processId": os.getpid(),
"rootPath": repository_absolute_path,
"rootUri": pathlib.Path(repository_absolute_path).as_uri(),
@@ -142,35 +146,57 @@ class PyrightServer(LanguageServer):
return initialize_params
@asynccontextmanager
async def start_server(self) -> AsyncIterator["PyrightServer"]:
def _start_server(self):
"""
Starts the Pyright Language Server, waits for the server to be ready and yields the LanguageServer instance.
Starts the Pyright Language Server and waits for initial workspace analysis to complete.
This prevents zombie processes by ensuring Pyright has finished its initial background
tasks before we consider the server ready.
Usage:
```
async with lsp.start_server():
# LanguageServer has been initialized and ready to serve requests
# LanguageServer has been initialized and workspace analysis is complete
await lsp.request_definition(...)
await lsp.request_references(...)
# Shutdown the LanguageServer on exit from scope
# LanguageServer has been shutdown
# LanguageServer has been shutdown cleanly
```
"""
async def execute_client_command_handler(params):
def execute_client_command_handler(params):
return []
async def do_nothing(params):
def do_nothing(params):
return
async def check_experimental_status(params):
if params["quiescent"] == True:
def window_log_message(msg):
"""
Monitor Pyright's log messages to detect when initial analysis is complete.
Pyright logs "Found X source files" when it finishes scanning the workspace.
"""
message_text = msg.get("message", "")
self.logger.log(f"LSP: window/logMessage: {message_text}", logging.INFO)
# Look for "Found X source files" which indicates workspace scanning is complete
# Unfortunately, pyright is unreliable and there seems to be no better way
if re.search(r"Found \d+ source files?", message_text):
self.logger.log("Pyright workspace scanning complete", logging.INFO)
self.found_source_files = True
self.analysis_complete.set()
self.completions_available.set()
async def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
def check_experimental_status(params):
"""
Also listen for experimental/serverStatus as a backup signal
"""
if params.get("quiescent") == True:
self.logger.log("Received experimental/serverStatus with quiescent=true", logging.INFO)
if not self.found_source_files:
self.analysis_complete.set()
self.completions_available.set()
# Set up notification handlers
self.server.on_request("client/registerCapability", do_nothing)
self.server.on_notification("language/status", do_nothing)
self.server.on_notification("window/logMessage", window_log_message)
@@ -180,24 +206,34 @@ class PyrightServer(LanguageServer):
self.server.on_notification("language/actionableNotification", do_nothing)
self.server.on_notification("experimental/serverStatus", check_experimental_status)
async with super().start_server():
self.logger.log("Starting pyright-langserver server process", logging.INFO)
await self.server.start()
self.logger.log("Starting pyright-langserver server process", logging.INFO)
self.server.start()
# Send proper initialization parameters
initialize_params = self._get_initialize_params(self.repository_root_path)
# Send proper initialization parameters
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request from LSP client to pyright server and awaiting response",
logging.INFO,
)
init_response = await self.server.send.initialize(initialize_params)
self.logger.log(f"Received initialize response from pyright server: {init_response}", logging.INFO)
self.logger.log(
"Sending initialize request from LSP client to pyright server and awaiting response",
logging.INFO,
)
init_response = self.server.send.initialize(initialize_params)
self.logger.log(f"Received initialize response from pyright server: {init_response}", logging.INFO)
# Verify that the server supports our required features
self.server.notify.initialized({})
# Verify that the server supports our required features
assert "textDocumentSync" in init_response["capabilities"]
assert "completionProvider" in init_response["capabilities"]
assert "definitionProvider" in init_response["capabilities"]
yield self
# Complete the initialization handshake
self.server.notify.initialized({})
await self.server.shutdown()
await self.server.stop()
# Wait for Pyright to complete its initial workspace analysis
# This prevents zombie processes by ensuring background tasks finish
self.logger.log("Waiting for Pyright to complete initial workspace analysis...", logging.INFO)
if self.analysis_complete.wait(timeout=5.0):
self.logger.log("Pyright initial analysis complete, server ready", logging.INFO)
else:
self.logger.log("Timeout waiting for Pyright analysis completion, proceeding anyway", logging.WARNING)
# Fallback: assume analysis is complete after timeout
self.analysis_complete.set()
self.completions_available.set()
src/multilspy/language_servers/rust_analyzer/initialize_params.json → src/solidlsp/language_servers/rust_analyzer/initialize_params.json
View File
src/multilspy/language_servers/rust_analyzer/runtime_dependencies.json → src/solidlsp/language_servers/rust_analyzer/runtime_dependencies.json
View File
src/multilspy/language_servers/rust_analyzer/rust_analyzer.py → src/solidlsp/language_servers/rust_analyzer/rust_analyzer.py
+47 -69
View File
@@ -2,32 +2,29 @@
Provides Rust specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Rust.
"""
import asyncio
import json
import logging
import os
import stat
import pathlib
from contextlib import asynccontextmanager
from typing import AsyncIterator
import stat
import threading
from overrides import override
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import InitializeParams
from multilspy.multilspy_config import MultilspyConfig
from multilspy.multilspy_utils import FileUtils
from multilspy.multilspy_utils import PlatformUtils
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.ls_utils import FileUtils, PlatformUtils
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
class RustAnalyzer(LanguageServer):
class RustAnalyzer(SolidLanguageServer):
"""
Provides Rust specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Rust.
"""
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
def __init__(self, config: LanguageServerConfig, logger: LanguageServerLogger, repository_root_path: str):
"""
Creates a RustAnalyzer instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead.
"""
@@ -39,19 +36,22 @@ class RustAnalyzer(LanguageServer):
ProcessLaunchInfo(cmd=rustanalyzer_executable_path, cwd=repository_root_path),
"rust",
)
self.server_ready = asyncio.Event()
self.server_ready = threading.Event()
self.service_ready_event = threading.Event()
self.initialize_searcher_command_available = threading.Event()
self.resolve_main_method_available = threading.Event()
@override
def is_ignored_dirname(self, dirname: str) -> bool:
return super().is_ignored_dirname(dirname) or dirname in ["target"]
def setup_runtime_dependencies(self, logger: MultilspyLogger, config: MultilspyConfig) -> str:
def setup_runtime_dependencies(self, logger: LanguageServerLogger, config: LanguageServerConfig) -> str:
"""
Setup runtime dependencies for rust_analyzer.
"""
platform_id = PlatformUtils.get_platform_id()
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r", encoding="utf-8") as f:
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -61,9 +61,7 @@ class RustAnalyzer(LanguageServer):
# ], "Only linux-x64 and win-x64 platform is supported for in multilspy at the moment"
runtime_dependencies = d["runtimeDependencies"]
runtime_dependencies = [
dependency for dependency in runtime_dependencies if dependency["platformId"] == platform_id.value
]
runtime_dependencies = [dependency for dependency in runtime_dependencies if dependency["platformId"] == platform_id.value]
assert len(runtime_dependencies) == 1
dependency = runtime_dependencies[0]
@@ -72,13 +70,9 @@ class RustAnalyzer(LanguageServer):
if not os.path.exists(rustanalyzer_ls_dir):
os.makedirs(rustanalyzer_ls_dir)
if dependency["archiveType"] == "gz":
FileUtils.download_and_extract_archive(
logger, dependency["url"], rustanalyzer_executable_path, dependency["archiveType"]
)
FileUtils.download_and_extract_archive(logger, dependency["url"], rustanalyzer_executable_path, dependency["archiveType"])
else:
FileUtils.download_and_extract_archive(
logger, dependency["url"], rustanalyzer_ls_dir, dependency["archiveType"]
)
FileUtils.download_and_extract_archive(logger, dependency["url"], rustanalyzer_ls_dir, dependency["archiveType"])
assert os.path.exists(rustanalyzer_executable_path)
os.chmod(rustanalyzer_executable_path, stat.S_IEXEC)
@@ -88,7 +82,7 @@ class RustAnalyzer(LanguageServer):
"""
Returns the initialize params for the Rust Analyzer Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r", encoding="utf-8") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -108,22 +102,12 @@ class RustAnalyzer(LanguageServer):
return d
@asynccontextmanager
async def start_server(self) -> AsyncIterator["RustAnalyzer"]:
def _start_server(self):
"""
Starts the Rust Analyzer Language Server, waits for the server to be ready and yields the LanguageServer instance.
Usage:
```
async with lsp.start_server():
# LanguageServer has been initialized and ready to serve requests
await lsp.request_definition(...)
await lsp.request_references(...)
# Shutdown the LanguageServer on exit from scope
# LanguageServer has been shutdown
Starts the Rust Analyzer Language Server
"""
async def register_capability_handler(params):
def register_capability_handler(params):
assert "registrations" in params
for registration in params["registrations"]:
if registration["method"] == "workspace/executeCommand":
@@ -131,24 +115,24 @@ class RustAnalyzer(LanguageServer):
self.resolve_main_method_available.set()
return
async def lang_status_handler(params):
def lang_status_handler(params):
# TODO: Should we wait for
# server -> client: {'jsonrpc': '2.0', 'method': 'language/status', 'params': {'type': 'ProjectStatus', 'message': 'OK'}}
# Before proceeding?
if params["type"] == "ServiceReady" and params["message"] == "ServiceReady":
self.service_ready_event.set()
async def execute_client_command_handler(params):
def execute_client_command_handler(params):
return []
async def do_nothing(params):
def do_nothing(params):
return
async def check_experimental_status(params):
def check_experimental_status(params):
if params["quiescent"] == True:
self.server_ready.set()
async def window_log_message(msg):
def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
self.server.on_request("client/registerCapability", register_capability_handler)
@@ -160,29 +144,23 @@ class RustAnalyzer(LanguageServer):
self.server.on_notification("language/actionableNotification", do_nothing)
self.server.on_notification("experimental/serverStatus", check_experimental_status)
async with super().start_server():
self.logger.log("Starting RustAnalyzer server process", logging.INFO)
await self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log("Starting RustAnalyzer server process", logging.INFO)
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = await self.server.send.initialize(initialize_params)
assert init_response["capabilities"]["textDocumentSync"]["change"] == 2
assert "completionProvider" in init_response["capabilities"]
assert init_response["capabilities"]["completionProvider"] == {
"resolveProvider": True,
"triggerCharacters": [":", ".", "'", "("],
"completionItem": {"labelDetailsSupport": True},
}
self.server.notify.initialized({})
self.completions_available.set()
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = self.server.send.initialize(initialize_params)
assert init_response["capabilities"]["textDocumentSync"]["change"] == 2
assert "completionProvider" in init_response["capabilities"]
assert init_response["capabilities"]["completionProvider"] == {
"resolveProvider": True,
"triggerCharacters": [":", ".", "'", "("],
"completionItem": {"labelDetailsSupport": True},
}
self.server.notify.initialized({})
self.completions_available.set()
await self.server_ready.wait()
yield self
await self.server.shutdown()
await self.server.stop()
self.server_ready.wait()
src/multilspy/language_servers/solargraph/initialize_params.json → src/solidlsp/language_servers/solargraph/initialize_params.json
View File
src/multilspy/language_servers/solargraph/runtime_dependencies.json → src/solidlsp/language_servers/solargraph/runtime_dependencies.json
View File
src/multilspy/language_servers/solargraph/solargraph.py → src/solidlsp/language_servers/solargraph/solargraph.py
+55 -70
View File
@@ -3,32 +3,29 @@ Provides Ruby specific instantiation of the LanguageServer class using Solargrap
Contains various configurations and settings specific to Ruby.
"""
import asyncio
import json
import logging
import os
import pathlib
import stat
import subprocess
import pathlib
from contextlib import asynccontextmanager
from typing import AsyncIterator, override
import threading
from typing import override
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import InitializeParams
from multilspy.multilspy_config import MultilspyConfig
from multilspy.multilspy_utils import FileUtils
from multilspy.multilspy_utils import PlatformUtils, PlatformId
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
class Solargraph(LanguageServer):
class Solargraph(SolidLanguageServer):
"""
Provides Ruby specific instantiation of the LanguageServer class using Solargraph.
Contains various configurations and settings specific to Ruby.
"""
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
def __init__(self, config: LanguageServerConfig, logger: LanguageServerLogger, repository_root_path: str):
"""
Creates a Solargraph instance. This class is not meant to be instantiated directly.
Use LanguageServer.create() instead.
@@ -41,18 +38,20 @@ class Solargraph(LanguageServer):
ProcessLaunchInfo(cmd=f"{solargraph_executable_path} stdio", cwd=repository_root_path),
"ruby",
)
self.server_ready = asyncio.Event()
self.server_ready = threading.Event()
self.service_ready_event = threading.Event()
self.initialize_searcher_command_available = threading.Event()
self.resolve_main_method_available = threading.Event()
@override
def is_ignored_dirname(self, dirname: str) -> bool:
return super().is_ignored_dirname(dirname) or dirname in ["vendor"]
def setup_runtime_dependencies(self, logger: MultilspyLogger, config: MultilspyConfig, repository_root_path: str) -> str:
def setup_runtime_dependencies(self, logger: LanguageServerLogger, config: LanguageServerConfig, repository_root_path: str) -> str:
"""
Setup runtime dependencies for Solargraph.
"""
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r", encoding="utf-8") as f:
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -64,38 +63,40 @@ class Solargraph(LanguageServer):
ruby_version = result.stdout.strip()
logger.log(f"Ruby version: {ruby_version}", logging.INFO)
except subprocess.CalledProcessError as e:
raise RuntimeError(f"Error checking for Ruby installation: {e.stderr}")
except FileNotFoundError:
raise RuntimeError("Ruby is not installed. Please install Ruby before continuing.")
raise RuntimeError(f"Error checking for Ruby installation: {e.stderr}") from e
except FileNotFoundError as e:
raise RuntimeError("Ruby is not installed. Please install Ruby before continuing.") from e
# Check if solargraph is installed
try:
result = subprocess.run(["gem", "list", "^solargraph$", "-i"], check=False, capture_output=True, text=True, cwd=repository_root_path)
result = subprocess.run(
["gem", "list", "^solargraph$", "-i"], check=False, capture_output=True, text=True, cwd=repository_root_path
)
if result.stdout.strip() == "false":
logger.log("Installing Solargraph...", logging.INFO)
subprocess.run(dependency["installCommand"].split(), check=True, capture_output=True, cwd=repository_root_path)
# Get the gem executable path directly
result = subprocess.run(["gem", "which", "solargraph"], check=True, capture_output=True, text=True, cwd=repository_root_path)
gem_path = result.stdout.strip()
bin_dir = os.path.join(os.path.dirname(os.path.dirname(gem_path)), "bin")
executable_path = os.path.join(bin_dir, "solargraph")
if not os.path.exists(executable_path):
raise RuntimeError(f"Solargraph executable not found at {executable_path}")
# Ensure the executable has the right permissions
os.chmod(executable_path, os.stat(executable_path).st_mode | stat.S_IEXEC)
return executable_path
except subprocess.CalledProcessError as e:
raise RuntimeError(f"Failed to check or install Solargraph. {e.stderr}")
raise RuntimeError(f"Failed to check or install Solargraph. {e.stderr}") from e
def _get_initialize_params(self, repository_absolute_path: str) -> InitializeParams:
"""
Returns the initialize params for the Solargraph Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r", encoding="utf-8") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -115,22 +116,12 @@ class Solargraph(LanguageServer):
return d
@asynccontextmanager
async def start_server(self) -> AsyncIterator["Solargraph"]:
def _start_server(self):
"""
Starts the Solargraph Language Server for Ruby, waits for the server to be ready and yields the LanguageServer instance.
Usage:
```
async with lsp.start_server():
# LanguageServer has been initialized and ready to serve requests
await lsp.request_definition(...)
await lsp.request_references(...)
# Shutdown the LanguageServer on exit from scope
# LanguageServer has been shutdown
Starts the Solargraph Language Server for Ruby
"""
async def register_capability_handler(params):
def register_capability_handler(params):
assert "registrations" in params
for registration in params["registrations"]:
if registration["method"] == "workspace/executeCommand":
@@ -138,20 +129,20 @@ class Solargraph(LanguageServer):
self.resolve_main_method_available.set()
return
async def lang_status_handler(params):
def lang_status_handler(params):
# TODO: Should we wait for
# server -> client: {'jsonrpc': '2.0', 'method': 'language/status', 'params': {'type': 'ProjectStatus', 'message': 'OK'}}
# Before proceeding?
if params["type"] == "ServiceReady" and params["message"] == "ServiceReady":
self.service_ready_event.set()
async def execute_client_command_handler(params):
def execute_client_command_handler(params):
return []
async def do_nothing(params):
def do_nothing(params):
return
async def window_log_message(msg):
def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
self.server.on_request("client/registerCapability", register_capability_handler)
@@ -162,31 +153,25 @@ class Solargraph(LanguageServer):
self.server.on_notification("textDocument/publishDiagnostics", do_nothing)
self.server.on_notification("language/actionableNotification", do_nothing)
async with super().start_server():
self.logger.log("Starting solargraph server process", logging.INFO)
await self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log("Starting solargraph server process", logging.INFO)
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
self.logger.log(f"Sending init params: {json.dumps(initialize_params, indent=4)}", logging.INFO)
init_response = await self.server.send.initialize(initialize_params)
self.logger.log(f"Received init response: {init_response}", logging.INFO)
assert init_response["capabilities"]["textDocumentSync"] == 2
assert "completionProvider" in init_response["capabilities"]
assert init_response["capabilities"]["completionProvider"] == {
"resolveProvider": True,
"triggerCharacters": [".", ":", "@"],
}
self.server.notify.initialized({})
self.completions_available.set()
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
self.logger.log(f"Sending init params: {json.dumps(initialize_params, indent=4)}", logging.INFO)
init_response = self.server.send.initialize(initialize_params)
self.logger.log(f"Received init response: {init_response}", logging.INFO)
assert init_response["capabilities"]["textDocumentSync"] == 2
assert "completionProvider" in init_response["capabilities"]
assert init_response["capabilities"]["completionProvider"] == {
"resolveProvider": True,
"triggerCharacters": [".", ":", "@"],
}
self.server.notify.initialized({})
self.completions_available.set()
self.server_ready.set()
await self.server_ready.wait()
yield self
await self.server.shutdown()
await self.server.stop()
self.server_ready.set()
self.server_ready.wait()
src/multilspy/language_servers/typescript_language_server/initialize_params.json → src/solidlsp/language_servers/typescript_language_server/initialize_params.json
View File
src/multilspy/language_servers/typescript_language_server/runtime_dependencies.json → src/solidlsp/language_servers/typescript_language_server/runtime_dependencies.json
View File
src/multilspy/language_servers/typescript_language_server/typescript_language_server.py → src/solidlsp/language_servers/typescript_language_server/typescript_language_server.py
+60 -55
View File
@@ -2,35 +2,33 @@
Provides TypeScript specific instantiation of the LanguageServer class. Contains various configurations and settings specific to TypeScript.
"""
import asyncio
import json
import logging
import os
import pathlib
import shutil
import subprocess
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
import threading
from time import sleep
from overrides import override
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.lsp_types import InitializeParams
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.multilspy_config import MultilspyConfig
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.multilspy_utils import PlatformId, PlatformUtils
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.ls_utils import PlatformId, PlatformUtils
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
# Platform-specific imports
if os.name != 'nt': # Unix-like systems
if os.name != "nt": # Unix-like systems
import pwd
else:
# Dummy pwd module for Windows
class pwd:
@staticmethod
def getpwuid(uid):
return type('obj', (), {'pw_name': os.environ.get('USERNAME', 'unknown')})()
return type("obj", (), {"pw_name": os.environ.get("USERNAME", "unknown")})()
# Conditionally import pwd module (Unix-only)
@@ -38,12 +36,12 @@ if not PlatformUtils.get_platform_id().value.startswith("win"):
import pwd
class TypeScriptLanguageServer(LanguageServer):
class TypeScriptLanguageServer(SolidLanguageServer):
"""
Provides TypeScript specific instantiation of the LanguageServer class. Contains various configurations and settings specific to TypeScript.
"""
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
def __init__(self, config: LanguageServerConfig, logger: LanguageServerLogger, repository_root_path: str):
"""
Creates a TypeScriptLanguageServer instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead.
"""
@@ -55,7 +53,8 @@ class TypeScriptLanguageServer(LanguageServer):
ProcessLaunchInfo(cmd=ts_lsp_executable_path, cwd=repository_root_path),
"typescript",
)
self.server_ready = asyncio.Event()
self.server_ready = threading.Event()
self.initialize_searcher_command_available = threading.Event()
@override
def is_ignored_dirname(self, dirname: str) -> bool:
@@ -66,7 +65,7 @@ class TypeScriptLanguageServer(LanguageServer):
"coverage",
]
def setup_runtime_dependencies(self, logger: MultilspyLogger, config: MultilspyConfig) -> str:
def setup_runtime_dependencies(self, logger: LanguageServerLogger, config: LanguageServerConfig) -> str:
"""
Setup runtime dependencies for TypeScript Language Server.
"""
@@ -92,9 +91,9 @@ class TypeScriptLanguageServer(LanguageServer):
tsserver_executable_path = os.path.join(tsserver_ls_dir, "typescript-language-server")
# Verify both node and npm are installed
is_node_installed = shutil.which('node') is not None
is_node_installed = shutil.which("node") is not None
assert is_node_installed, "node is not installed or isn't in PATH. Please install NodeJS and try again."
is_npm_installed = shutil.which('npm') is not None
is_npm_installed = shutil.which("npm") is not None
assert is_npm_installed, "npm is not installed or isn't in PATH. Please install npm and try again."
# Install typescript and typescript-language-server if not already installed
@@ -109,7 +108,7 @@ class TypeScriptLanguageServer(LanguageServer):
check=True,
cwd=tsserver_ls_dir,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL
stderr=subprocess.DEVNULL,
)
else:
# On Unix-like systems, run as non-root user
@@ -121,12 +120,14 @@ class TypeScriptLanguageServer(LanguageServer):
user=user,
cwd=tsserver_ls_dir,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL
stderr=subprocess.DEVNULL,
)
tsserver_executable_path = os.path.join(tsserver_ls_dir, "node_modules", ".bin", "typescript-language-server")
assert os.path.exists(tsserver_executable_path), "typescript-language-server executable not found. Please install typescript-language-server and try again."
assert os.path.exists(
tsserver_executable_path
), "typescript-language-server executable not found. Please install typescript-language-server and try again."
return f"{tsserver_executable_path} --stdio"
def _get_initialize_params(self, repository_absolute_path: str) -> InitializeParams:
@@ -153,8 +154,7 @@ class TypeScriptLanguageServer(LanguageServer):
return d
@asynccontextmanager
async def start_server(self) -> AsyncIterator["TypeScriptLanguageServer"]:
def _start_server(self):
"""
Starts the TypeScript Language Server, waits for the server to be ready and yields the LanguageServer instance.
@@ -168,7 +168,7 @@ class TypeScriptLanguageServer(LanguageServer):
# LanguageServer has been shutdown
"""
async def register_capability_handler(params):
def register_capability_handler(params):
assert "registrations" in params
for registration in params["registrations"]:
if registration["method"] == "workspace/executeCommand":
@@ -178,59 +178,64 @@ class TypeScriptLanguageServer(LanguageServer):
# self.resolve_main_method_available.set()
return
async def execute_client_command_handler(params):
def execute_client_command_handler(params):
return []
async def do_nothing(params):
def do_nothing(params):
return
async def window_log_message(msg):
def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
def check_experimental_status(params):
"""
Also listen for experimental/serverStatus as a backup signal
"""
if params.get("quiescent") == True:
self.server_ready.set()
self.completions_available.set()
self.server.on_request("client/registerCapability", register_capability_handler)
self.server.on_notification("window/logMessage", window_log_message)
self.server.on_request("workspace/executeClientCommand", execute_client_command_handler)
self.server.on_notification("$/progress", do_nothing)
self.server.on_notification("textDocument/publishDiagnostics", do_nothing)
self.server.on_notification("experimental/serverStatus", check_experimental_status)
async with super().start_server():
self.logger.log("Starting TypeScript server process", logging.INFO)
await self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log("Starting TypeScript server process", logging.INFO)
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = await self.server.send.initialize(initialize_params)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = self.server.send.initialize(initialize_params)
# TypeScript-specific capability checks
assert init_response["capabilities"]["textDocumentSync"] == 2
assert "completionProvider" in init_response["capabilities"]
assert init_response["capabilities"]["completionProvider"] == {
"triggerCharacters": ['.', '"', "'", '/', '@', '<'],
"resolveProvider": True
}
# TypeScript-specific capability checks
assert init_response["capabilities"]["textDocumentSync"] == 2
assert "completionProvider" in init_response["capabilities"]
assert init_response["capabilities"]["completionProvider"] == {
"triggerCharacters": [".", '"', "'", "/", "@", "<"],
"resolveProvider": True,
}
self.server.notify.initialized({})
self.completions_available.set()
# TypeScript server is typically ready immediately after initialization
self.server.notify.initialized({})
if self.server_ready.wait(timeout=1.0):
self.logger.log("TypeScript server is ready", logging.INFO)
else:
self.logger.log("Timeout waiting for TypeScript server to become ready, proceeding anyway", logging.INFO)
# Fallback: assume server is ready after timeout
self.server_ready.set()
await self.server_ready.wait()
yield self
await self.server.shutdown()
await self.server.stop()
self.completions_available.set()
@override
# For some reason, the LS may need longer to process this, so we just retry
async def _send_references_request(self, relative_file_path: str, line: int, column: int):
def _send_references_request(self, relative_file_path: str, line: int, column: int):
# TODO: The LS doesn't return references contained in other files if it doesn't sleep. This is
# despite the LS having processed requests already. I don't know what causes this, but sleeping
# one second helps. It may be that sleeping only once is enough but that's hard to reliably test.
# It may be that even this 1sec is not enough in larger TS projects, at some point we should find what
# causes this and solve it.
sleep(1)
return await super()._send_references_request(relative_file_path, line, column)
return super()._send_references_request(relative_file_path, line, column)
src/multilspy/language_server.py → src/solidlsp/ls.py
+439 -928
View File
File diff suppressed because it is too large Load Diff
src/multilspy/multilspy_config.py → src/solidlsp/ls_config.py
+14 -14
View File
@@ -1,10 +1,10 @@
"""
Configuration parameters for Multilspy.
Configuration objects for language servers
"""
import fnmatch
from enum import Enum
from typing import List
from dataclasses import dataclass, field
from enum import Enum
class FilenameMatcher:
@@ -32,7 +32,6 @@ class Language(str, Enum):
JAVA = "java"
KOTLIN = "kotlin"
TYPESCRIPT = "typescript"
JAVASCRIPT = "javascript"
GO = "go"
RUBY = "ruby"
DART = "dart"
@@ -50,9 +49,13 @@ class Language(str, Enum):
case self.JAVA:
return FilenameMatcher("*.java")
case self.TYPESCRIPT:
return FilenameMatcher("*.ts", "*.js", "*.jsx", "*.tsx")
case self.JAVASCRIPT:
return FilenameMatcher("*.js", "*.jsx")
# see https://github.com/oraios/serena/issues/204
path_patterns = []
for prefix in ["c", "m", ""]:
for postfix in ["x", ""]:
for base_pattern in ["ts", "js"]:
path_patterns.append(f"*.{prefix}{base_pattern}{postfix}")
return FilenameMatcher(*path_patterns)
case self.CSHARP:
return FilenameMatcher("*.cs")
case self.RUST:
@@ -76,17 +79,16 @@ class Language(str, Enum):
@dataclass
class MultilspyConfig:
class LanguageServerConfig:
"""
Configuration parameters
"""
code_language: Language
trace_lsp_communication: bool = False
start_independent_lsp_process: bool = True
ignored_paths: list[str] = field(default_factory=list)
"""Paths, dirs or glob-like patterns. The matching will follow the same logic as for .gitignore entries"""
gitignore_file_content: str | None = None
"""Optional content of the gitignore file. If passed, will be used in addition to the explicitly passed ignored_paths for deciding which paths to ignore."""
@classmethod
def from_dict(cls, env: dict):
@@ -94,7 +96,5 @@ class MultilspyConfig:
Create a MultilspyConfig instance from a dictionary
"""
import inspect
return cls(**{
k: v for k, v in env.items()
if k in inspect.signature(cls).parameters
})
return cls(**{k: v for k, v in env.items() if k in inspect.signature(cls).parameters})
src/multilspy/multilspy_exceptions.py → src/solidlsp/ls_exceptions.py
+3 -2
View File
@@ -2,7 +2,8 @@
This module contains the exceptions raised by the Multilspy framework.
"""
class MultilspyException(Exception):
class LanguageServerException(Exception):
"""
Exceptions raised by the Multilspy framework.
"""
@@ -11,4 +12,4 @@ class MultilspyException(Exception):
"""
Initializes the exception with the given message.
"""
super().__init__(message)
super().__init__(message)
src/multilspy/lsp_protocol_handler/server.py → src/solidlsp/ls_handler.py
+204 -262
View File
@@ -1,149 +1,63 @@
"""
This file provides the implementation of the JSON-RPC client, that launches and
communicates with the language server.
The initial implementation of this file was obtained from
https://github.com/predragnikolic/OLSP under the MIT License with the following terms:
MIT License
Copyright (c) 2023 Предраг Николић
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""
import asyncio
import dataclasses
import json
import logging
import os
import subprocess
import threading
import time
from collections.abc import Callable
from dataclasses import dataclass
from queue import Queue
from typing import Any
import psutil
from typing import Any, Callable, Dict, List, Optional, Union
from .lsp_requests import LspNotification, LspRequest
from .lsp_types import ErrorCodes
from ..multilspy_exceptions import MultilspyException
from solidlsp.ls_exceptions import LanguageServerException
from solidlsp.ls_request import LanguageServerRequest
from solidlsp.lsp_protocol_handler.lsp_requests import LspNotification
from solidlsp.lsp_protocol_handler.lsp_types import ErrorCodes
from solidlsp.lsp_protocol_handler.server import (
ENCODING,
Error,
MessageType,
PayloadLike,
ProcessLaunchInfo,
StringDict,
content_length,
create_message,
make_error_response,
make_notification,
make_request,
make_response,
)
StringDict = Dict[str, Any]
PayloadLike = Union[List[StringDict], StringDict, None]
CONTENT_LENGTH = "Content-Length: "
ENCODING = "utf-8"
log = logging.getLogger(__name__)
@dataclasses.dataclass
class ProcessLaunchInfo:
"""
This class is used to store the information required to launch a process.
"""
# The command to launch the process
cmd: str
# The environment variables to set for the process
env: Dict[str, str] = dataclasses.field(default_factory=dict)
# The working directory for the process
cwd: str = os.getcwd()
class Error(Exception):
def __init__(self, code: ErrorCodes, message: str) -> None:
super().__init__(message)
self.code = code
def to_lsp(self) -> StringDict:
return {"code": self.code, "message": super().__str__()}
@classmethod
def from_lsp(cls, d: StringDict) -> "Error":
return Error(d["code"], d["message"])
def __str__(self) -> str:
return f"{super().__str__()} ({self.code})"
def make_response(request_id: Any, params: PayloadLike) -> StringDict:
return {"jsonrpc": "2.0", "id": request_id, "result": params}
def make_error_response(request_id: Any, err: Error) -> StringDict:
return {"jsonrpc": "2.0", "id": request_id, "error": err.to_lsp()}
def make_notification(method: str, params: PayloadLike) -> StringDict:
return {"jsonrpc": "2.0", "method": method, "params": params}
def make_request(method: str, request_id: Any, params: PayloadLike) -> StringDict:
return {"jsonrpc": "2.0", "method": method, "id": request_id, "params": params}
class StopLoopException(Exception):
pass
def create_message(payload: PayloadLike):
body = json.dumps(payload, check_circular=False, ensure_ascii=False, separators=(",", ":")).encode(ENCODING)
return (
f"Content-Length: {len(body)}\r\n".encode(ENCODING),
"Content-Type: application/vscode-jsonrpc; charset=utf-8\r\n\r\n".encode(ENCODING),
body,
)
class MessageType:
error = 1
warning = 2
info = 3
log = 4
class Request:
@dataclass
class Result:
payload: PayloadLike | None = None
error: Error | None = None
def is_error(self) -> bool:
return self.error is not None
def __init__(self) -> None:
self.cv = asyncio.Condition()
self.result: Optional[PayloadLike] = None
self.error: Optional[Error] = None
self._result_queue = Queue()
async def on_result(self, params: PayloadLike) -> None:
self.result = params
async with self.cv:
self.cv.notify()
def on_result(self, params: PayloadLike) -> None:
self._result_queue.put(Request.Result(payload=params))
async def on_error(self, err: Error) -> None:
self.error = err
async with self.cv:
self.cv.notify()
def on_error(self, err: Error) -> None:
self._result_queue.put(Request.Result(error=err))
def get_result(self, timeout: float | None = None) -> Result:
return self._result_queue.get(timeout=timeout)
def content_length(line: bytes) -> Optional[int]:
if line.startswith(b"Content-Length: "):
_, value = line.split(b"Content-Length: ")
value = value.strip()
try:
return int(value)
except ValueError:
raise ValueError("Invalid Content-Length header: {}".format(value))
return None
class LanguageServerHandler:
class SolidLanguageServerHandler:
"""
This class provides the implementation of Python client for the Language Server Protocol.
A class that launches the language server and communicates with it
@@ -179,21 +93,17 @@ class LanguageServerHandler:
language server process in an independent process group. Default is `True`. Setting it to
`False` means that the language server process will be in the same process group as the
the current process, and any SIGINT and SIGTERM signals will be sent to both processes.
"""
def __init__(
self,
process_launch_info: ProcessLaunchInfo,
logger: Optional[Callable[[str, str, StringDict | str], None]] = None,
logger: Callable[[str, str, StringDict | str], None] | None = None,
start_independent_lsp_process=True,
request_timeout: float | None = None,
) -> None:
"""
Params:
cmd: A string that represents the command to launch the language server process.
logger: An optional function that takes two strings (source and destination) and
a payload dictionary, and logs the communication between the client and the server.
"""
self.send = LspRequest(self.send_request)
self.send = LanguageServerRequest(self.send_request)
self.notify = LspNotification(self.send_notification)
self.process_launch_info = process_launch_info
@@ -201,7 +111,7 @@ class LanguageServerHandler:
self._received_shutdown = False
self.request_id = 1
self._response_handlers: Dict[Any, Request] = {}
self._response_handlers: dict[Any, Request] = {}
self.on_request_handlers = {}
self.on_notification_handlers = {}
self.logger = logger
@@ -209,8 +119,27 @@ class LanguageServerHandler:
self.task_counter = 0
self.loop = None
self.start_independent_lsp_process = start_independent_lsp_process
self._request_timeout = request_timeout
async def start(self) -> None:
# Add thread locks for shared resources to prevent race conditions
self._stdin_lock = threading.Lock()
self._request_id_lock = threading.Lock()
self._response_handlers_lock = threading.Lock()
self._tasks_lock = threading.Lock()
def set_request_timeout(self, timeout: float | None) -> None:
"""
:param timeout: the timeout, in seconds, for all requests sent to the language server.
"""
self._request_timeout = timeout
def is_running(self) -> bool:
"""
Checks if the language server process is currently running.
"""
return self.process is not None and self.process.returncode is None
def start(self) -> None:
"""
Starts the language server process and creates a task to continuously read from its stdout to handle communications
from the server to the client
@@ -219,81 +148,62 @@ class LanguageServerHandler:
child_proc_env.update(self.process_launch_info.env)
log.info("Starting language server process via command: %s", self.process_launch_info.cmd)
self.process = await asyncio.create_subprocess_shell(
self.process = subprocess.Popen(
self.process_launch_info.cmd,
stdout=asyncio.subprocess.PIPE,
stdin=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
stdout=subprocess.PIPE,
stdin=subprocess.PIPE,
stderr=subprocess.PIPE,
env=child_proc_env,
cwd=self.process_launch_info.cwd,
start_new_session=self.start_independent_lsp_process,
shell=True,
)
# Check if process terminated immediately
if self.process.returncode is not None:
log.error("Language server has already terminated/could not be started")
# Process has already terminated
stderr_data = await self.process.stderr.read()
error_message = stderr_data.decode('utf-8', errors='replace')
stderr_data = self.process.stderr.read()
error_message = stderr_data.decode("utf-8", errors="replace")
raise RuntimeError(f"Process terminated immediately with code {self.process.returncode}. Error: {error_message}")
self.loop = asyncio.get_event_loop()
self.tasks[self.task_counter] = self.loop.create_task(self.run_forever())
self.task_counter += 1
self.tasks[self.task_counter] = self.loop.create_task(self.run_forever_stderr())
self.task_counter += 1
# start threads to read stdout and stderr of the process
threading.Thread(
target=self.run_forever,
name="LSP-stdout-reader",
daemon=True,
).start()
threading.Thread(
target=self.run_forever_stderr,
name="LSP-stderr-reader",
daemon=True,
).start()
async def stop(self) -> None:
def stop(self) -> None:
"""
Sends the terminate signal to the language server process and waits for it to exit, with a timeout, killing it if necessary
"""
# First cancel all tasks
await self._cancel_pending_tasks()
process = self.process
self.process = None
if not process:
return
# Clean up the process
await self._cleanup_process(process)
if process:
self._cleanup_process(process)
async def _cancel_pending_tasks(self):
"""Cancel all pending tasks and wait for them to complete or timeout."""
pending_tasks = []
for task in self.tasks.values():
if not task.done():
task.cancel()
pending_tasks.append(task)
if pending_tasks:
try:
await asyncio.wait_for(asyncio.gather(*pending_tasks, return_exceptions=True), timeout=5.0)
except (asyncio.TimeoutError, Exception):
pass
self.tasks = {}
async def _cleanup_process(self, process):
def _cleanup_process(self, process):
"""Clean up a process: close stdin, terminate/kill process, close stdout/stderr."""
# Close stdin first to prevent deadlocks
# See: https://bugs.python.org/issue35539
self._safely_close_pipe(process.stdin)
# Terminate/kill the process if it's still running
if process.returncode is None:
await self._terminate_or_kill_process(process)
self._terminate_or_kill_process(process)
# Close stdout and stderr pipes after process has exited
# This is essential to prevent "I/O operation on closed pipe" errors and
# "Event loop is closed" errors during garbage collection
# See: https://bugs.python.org/issue41320 and https://github.com/python/cpython/issues/88050
self._safely_close_pipe(process.stdout)
self._safely_close_pipe(process.stderr)
# Small delay to ensure OS has released file handles
await asyncio.sleep(0.5)
def _safely_close_pipe(self, pipe):
"""Safely close a pipe, ignoring any exceptions."""
@@ -303,14 +213,16 @@ class LanguageServerHandler:
except Exception:
pass
async def _terminate_or_kill_process(self, process):
def _terminate_or_kill_process(self, process):
"""Try to terminate the process gracefully, then forcefully if necessary."""
# First try to terminate the process tree gracefully
self._signal_process_tree(process, terminate=True)
# TODO
"""
# Wait for the process to exit (with timeout)
try:
await asyncio.wait_for(process.wait(), timeout=10)
asyncio.wait_for(process.wait(), timeout=10)
except (asyncio.TimeoutError, Exception):
# If termination failed, forcefully kill the process tree
self._signal_process_tree(process, terminate=False)
@@ -319,18 +231,19 @@ class LanguageServerHandler:
await asyncio.wait_for(process.wait(), timeout=2)
except Exception:
pass
"""
def _signal_process_tree(self, process, terminate=True):
"""Send signal (terminate or kill) to the process and all its children."""
signal_method = "terminate" if terminate else "kill"
# Try to get the parent process
parent = None
try:
parent = psutil.Process(process.pid)
except (psutil.NoSuchProcess, psutil.AccessDenied, Exception):
pass
# If we have the parent process and it's running, signal the entire tree
if parent and parent.is_running():
# Signal children first
@@ -339,7 +252,7 @@ class LanguageServerHandler:
getattr(child, signal_method)()
except (psutil.NoSuchProcess, psutil.AccessDenied, Exception):
pass
# Then signal the parent
try:
getattr(parent, signal_method)()
@@ -352,19 +265,25 @@ class LanguageServerHandler:
except Exception:
pass
async def shutdown(self) -> None:
def shutdown(self) -> None:
"""
Perform the shutdown sequence for the client, including sending the shutdown request to the server and notifying it of exit
"""
await self.send.shutdown()
self._log("Sending shutdown request to server")
self.send.shutdown()
self._log("Received shutdown response from server")
self._received_shutdown = True
self._log("Sending exit notification to server")
self.notify.exit()
self._log("Sent exit notification to server")
# TODO
"""
if self.process and self.process.stdout:
self.process.stdout.set_exception(StopLoopException())
# This yields the control to the event loop to allow the exception to be handled
# in the run_forever and run_forever_stderr methods
await asyncio.sleep(0)
"""
def _log(self, message: str | StringDict) -> None:
"""
@@ -373,14 +292,33 @@ class LanguageServerHandler:
if self.logger is not None:
self.logger("client", "logger", message)
async def run_forever(self) -> bool:
@staticmethod
def _read_bytes_from_process(process, stream, num_bytes):
"""Read exactly num_bytes from process stdout"""
if process.poll() is not None:
# Process has terminated, check if we can still read
pass
data = b""
while len(data) < num_bytes:
chunk = stream.read(num_bytes - len(data))
if not chunk:
if process.poll() is not None:
raise EOFError(f"Process terminated. Expected {num_bytes} bytes, got {len(data)}")
# Process still running but no data available yet
time.sleep(0.01) # Small delay
continue
data += chunk
return data
def run_forever(self) -> bool:
"""
Continuously read from the language server process stdout and handle the messages
invoking the registered response and notification handlers
"""
try:
while self.process and self.process.stdout and not self.process.stdout.at_eof():
line = await self.process.stdout.readline()
while self.process and self.process.stdout and self.process.stdout.readable():
line = self.process.stdout.readline()
if not line:
continue
try:
@@ -390,44 +328,43 @@ class LanguageServerHandler:
if num_bytes is None:
continue
while line and line.strip():
line = await self.process.stdout.readline()
line = self.process.stdout.readline()
if not line:
continue
body = await self.process.stdout.readexactly(num_bytes)
body = self._read_bytes_from_process(self.process, self.process.stdout, num_bytes)
self.tasks[self.task_counter] = asyncio.get_event_loop().create_task(self._handle_body(body))
self.task_counter += 1
except (BrokenPipeError, ConnectionResetError, StopLoopException):
self._handle_body(body)
except (BrokenPipeError, ConnectionResetError):
pass
return self._received_shutdown
async def run_forever_stderr(self) -> None:
def run_forever_stderr(self) -> None:
"""
Continuously read from the language server process stderr and log the messages
"""
try:
while self.process and self.process.stderr and not self.process.stderr.at_eof():
line = await self.process.stderr.readline()
while self.process and self.process.stderr and self.process.stderr.readable():
line = self.process.stderr.readline()
if not line:
continue
self._log("LSP stderr: " + line.decode(ENCODING, errors='replace'))
except (BrokenPipeError, ConnectionResetError, StopLoopException):
self._log("LSP stderr: " + line.decode(ENCODING, errors="replace"))
except (BrokenPipeError, ConnectionResetError):
pass
async def _handle_body(self, body: bytes) -> None:
def _handle_body(self, body: bytes) -> None:
"""
Parse the body text received from the language server process and invoke the appropriate handler
"""
try:
await self._receive_payload(json.loads(body))
except IOError as ex:
self._receive_payload(json.loads(body))
except OSError as ex:
self._log(f"malformed {ENCODING}: {ex}")
except UnicodeDecodeError as ex:
self._log(f"malformed {ENCODING}: {ex}")
except json.JSONDecodeError as ex:
self._log(f"malformed JSON: {ex}")
async def _receive_payload(self, payload: StringDict) -> None:
def _receive_payload(self, payload: StringDict) -> None:
"""
Determine if the payload received from server is for a request, response, or notification and invoke the appropriate handler
"""
@@ -436,70 +373,64 @@ class LanguageServerHandler:
try:
if "method" in payload:
if "id" in payload:
await self._request_handler(payload)
self._request_handler(payload)
else:
await self._notification_handler(payload)
self._notification_handler(payload)
elif "id" in payload:
await self._response_handler(payload)
self._response_handler(payload)
else:
self._log(f"Unknown payload type: {payload}")
except Exception as err:
self._log(f"Error handling server payload: {err}")
def send_notification(self, method: str, params: Optional[dict] = None) -> None:
def send_notification(self, method: str, params: dict | None = None) -> None:
"""
Send notification pertaining to the given method to the server with the given parameters
"""
self._send_payload_sync(make_notification(method, params))
self._send_payload(make_notification(method, params))
def send_response(self, request_id: Any, params: PayloadLike) -> None:
"""
Send response to the given request id to the server with the given parameters
"""
self.tasks[self.task_counter] = asyncio.get_event_loop().create_task(
self._send_payload(make_response(request_id, params))
)
self.task_counter += 1
self._send_payload(make_response(request_id, params))
def send_error_response(self, request_id: Any, err: Error) -> None:
"""
Send error response to the given request id to the server with the given error
"""
self.tasks[self.task_counter] = asyncio.get_event_loop().create_task(
self._send_payload(make_error_response(request_id, err))
)
self.task_counter += 1
# Use lock to prevent race conditions on tasks and task_counter
self._send_payload(make_error_response(request_id, err))
async def send_request(self, method: str, params: Optional[dict] = None) -> PayloadLike:
def send_request(self, method: str, params: dict | None = None) -> PayloadLike:
"""
Send request to the server, register the request id, and wait for the response
"""
request = Request()
request_id = self.request_id
self.request_id += 1
self._response_handlers[request_id] = request
async with request.cv:
await self._send_payload(make_request(method, request_id, params))
self._log(f"Waiting for asyncio condition for request {method} with params:\n{params}")
await request.cv.wait()
self._log(f"Finished waiting, processing result")
if isinstance(request.error, Error):
raise MultilspyException(f"Could not process request {method} with params:\n{params}.\n Language server error: {request.error}") from request.error
self._log(f"Returning non-error result, which is:\n{request.result}")
return request.result
def _send_payload_sync(self, payload: StringDict) -> None:
"""
Send the payload to the server by writing to its stdin synchronously
"""
if not self.process or not self.process.stdin:
return
msg = create_message(payload)
if self.logger:
self.logger("client", "server", payload)
self.process.stdin.writelines(msg)
# Use lock to prevent race conditions on request_id and _response_handlers
with self._request_id_lock:
request_id = self.request_id
self.request_id += 1
async def _send_payload(self, payload: StringDict) -> None:
with self._response_handlers_lock:
self._response_handlers[request_id] = request
self._send_payload(make_request(method, request_id, params))
self._log(f"Waiting for response to request {method} with params:\n{params}")
result = request.get_result(timeout=self._request_timeout)
self._log("Processing result")
if result.is_error():
raise LanguageServerException(
f"Could not process request {method} with params:\n{params}.\n Language server error: {result.error}"
) from result.error
self._log(f"Returning non-error result, which is:\n{result.payload}")
return result.payload
def _send_payload(self, payload: StringDict) -> None:
"""
Send the payload to the server by writing to its stdin asynchronously.
"""
@@ -507,8 +438,17 @@ class LanguageServerHandler:
return
self._log(payload)
msg = create_message(payload)
self.process.stdin.writelines(msg)
await self.process.stdin.drain()
# Use lock to prevent concurrent writes to stdin that cause buffer corruption
with self._stdin_lock:
try:
self.process.stdin.writelines(msg)
self.process.stdin.flush()
except (BrokenPipeError, ConnectionResetError, OSError) as e:
# Log the error but don't raise to prevent cascading failures
if self.logger:
self.logger("client", "logger", f"Failed to write to stdin: {e}")
return
def on_request(self, method: str, cb) -> None:
"""
@@ -522,19 +462,21 @@ class LanguageServerHandler:
"""
self.on_notification_handlers[method] = cb
async def _response_handler(self, response: StringDict) -> None:
def _response_handler(self, response: StringDict) -> None:
"""
Handle the response received from the server for a request, using the id to determine the request
"""
request = self._response_handlers.pop(response["id"])
if "result" in response and "error" not in response:
await request.on_result(response["result"])
elif "result" not in response and "error" in response:
await request.on_error(Error.from_lsp(response["error"]))
else:
await request.on_error(Error(ErrorCodes.InvalidRequest, ""))
with self._response_handlers_lock:
request = self._response_handlers.pop(response["id"])
async def _request_handler(self, response: StringDict) -> None:
if "result" in response and "error" not in response:
request.on_result(response["result"])
elif "result" not in response and "error" in response:
request.on_error(Error.from_lsp(response["error"]))
else:
request.on_error(Error(ErrorCodes.InvalidRequest, ""))
def _request_handler(self, response: StringDict) -> None:
"""
Handle the request received from the server: call the appropriate callback function and return the result
"""
@@ -547,18 +489,18 @@ class LanguageServerHandler:
request_id,
Error(
ErrorCodes.MethodNotFound,
"method '{}' not handled on client.".format(method),
f"method '{method}' not handled on client.",
),
)
return
try:
self.send_response(request_id, await handler(params))
self.send_response(request_id, handler(params))
except Error as ex:
self.send_error_response(request_id, ex)
except Exception as ex:
self.send_error_response(request_id, Error(ErrorCodes.InternalError, str(ex)))
async def _notification_handler(self, response: StringDict) -> None:
def _notification_handler(self, response: StringDict) -> None:
"""
Handle the notification received from the server: call the appropriate callback function
"""
@@ -569,7 +511,7 @@ class LanguageServerHandler:
self._log(f"unhandled {method}")
return
try:
await handler(params)
handler(params)
except asyncio.CancelledError:
return
except Exception as ex:
src/multilspy/multilspy_logger.py → src/solidlsp/ls_logger.py
+7 -5
View File
@@ -1,9 +1,11 @@
"""
Multilspy logger module.
"""
import inspect
import logging
from datetime import datetime
from pydantic import BaseModel
@@ -20,7 +22,7 @@ class LogLine(BaseModel):
message: str
class MultilspyLogger:
class LanguageServerLogger:
"""
Logger class
"""
@@ -30,11 +32,10 @@ class MultilspyLogger:
self.logger.setLevel(log_level)
self.json_format = json_format
def log(self, debug_message: str, level: int, sanitized_error_message: str = "") -> None:
def log(self, debug_message: str, level: int, sanitized_error_message: str = "", stacklevel: int = 2) -> None:
"""
Log the debug and santized messages using the logger
Log the debug and sanitized messages using the logger
"""
debug_message = debug_message.replace("'", '"').replace("\n", " ")
sanitized_error_message = sanitized_error_message.replace("'", '"').replace("\n", " ")
@@ -59,6 +60,7 @@ class MultilspyLogger:
self.logger.log(
level=level,
msg=debug_log_line.json(),
stacklevel=stacklevel,
)
else:
self.logger.log(level, debug_message)
self.logger.log(level, debug_message, stacklevel=stacklevel)
src/solidlsp/ls_request.py
+377
View File
@@ -0,0 +1,377 @@
from typing import Union
from solidlsp.lsp_protocol_handler import lsp_types
class LanguageServerRequest:
def __init__(self, send_request):
self.send_request = send_request
def implementation(self, params: lsp_types.ImplementationParams) -> Union["lsp_types.Definition", list["lsp_types.LocationLink"], None]:
"""A request to resolve the implementation locations of a symbol at a given text
document position. The request's parameter is of type [TextDocumentPositionParams]
(#TextDocumentPositionParams) the response is of type {@link Definition} or a
Thenable that resolves to such.
"""
return self.send_request("textDocument/implementation", params)
def type_definition(
self, params: lsp_types.TypeDefinitionParams
) -> Union["lsp_types.Definition", list["lsp_types.LocationLink"], None]:
"""A request to resolve the type definition locations of a symbol at a given text
document position. The request's parameter is of type [TextDocumentPositionParams]
(#TextDocumentPositionParams) the response is of type {@link Definition} or a
Thenable that resolves to such.
"""
return self.send_request("textDocument/typeDefinition", params)
def document_color(self, params: lsp_types.DocumentColorParams) -> list["lsp_types.ColorInformation"]:
"""A request to list all color symbols found in a given text document. The request's
parameter is of type {@link DocumentColorParams} the
response is of type {@link ColorInformation ColorInformation[]} or a Thenable
that resolves to such.
"""
return self.send_request("textDocument/documentColor", params)
def color_presentation(self, params: lsp_types.ColorPresentationParams) -> list["lsp_types.ColorPresentation"]:
"""A request to list all presentation for a color. The request's
parameter is of type {@link ColorPresentationParams} the
response is of type {@link ColorInformation ColorInformation[]} or a Thenable
that resolves to such.
"""
return self.send_request("textDocument/colorPresentation", params)
def folding_range(self, params: lsp_types.FoldingRangeParams) -> list["lsp_types.FoldingRange"] | None:
"""A request to provide folding ranges in a document. The request's
parameter is of type {@link FoldingRangeParams}, the
response is of type {@link FoldingRangeList} or a Thenable
that resolves to such.
"""
return self.send_request("textDocument/foldingRange", params)
def declaration(self, params: lsp_types.DeclarationParams) -> Union["lsp_types.Declaration", list["lsp_types.LocationLink"], None]:
"""A request to resolve the type definition locations of a symbol at a given text
document position. The request's parameter is of type [TextDocumentPositionParams]
(#TextDocumentPositionParams) the response is of type {@link Declaration}
or a typed array of {@link DeclarationLink} or a Thenable that resolves
to such.
"""
return self.send_request("textDocument/declaration", params)
def selection_range(self, params: lsp_types.SelectionRangeParams) -> list["lsp_types.SelectionRange"] | None:
"""A request to provide selection ranges in a document. The request's
parameter is of type {@link SelectionRangeParams}, the
response is of type {@link SelectionRange SelectionRange[]} or a Thenable
that resolves to such.
"""
return self.send_request("textDocument/selectionRange", params)
def prepare_call_hierarchy(self, params: lsp_types.CallHierarchyPrepareParams) -> list["lsp_types.CallHierarchyItem"] | None:
"""A request to result a `CallHierarchyItem` in a document at a given position.
Can be used as an input to an incoming or outgoing call hierarchy.
@since 3.16.0
"""
return self.send_request("textDocument/prepareCallHierarchy", params)
def incoming_calls(self, params: lsp_types.CallHierarchyIncomingCallsParams) -> list["lsp_types.CallHierarchyIncomingCall"] | None:
"""A request to resolve the incoming calls for a given `CallHierarchyItem`.
@since 3.16.0
"""
return self.send_request("callHierarchy/incomingCalls", params)
def outgoing_calls(self, params: lsp_types.CallHierarchyOutgoingCallsParams) -> list["lsp_types.CallHierarchyOutgoingCall"] | None:
"""A request to resolve the outgoing calls for a given `CallHierarchyItem`.
@since 3.16.0
"""
return self.send_request("callHierarchy/outgoingCalls", params)
def semantic_tokens_full(self, params: lsp_types.SemanticTokensParams) -> Union["lsp_types.SemanticTokens", None]:
"""@since 3.16.0"""
return self.send_request("textDocument/semanticTokens/full", params)
def semantic_tokens_delta(
self, params: lsp_types.SemanticTokensDeltaParams
) -> Union["lsp_types.SemanticTokens", "lsp_types.SemanticTokensDelta", None]:
"""@since 3.16.0"""
return self.send_request("textDocument/semanticTokens/full/delta", params)
def semantic_tokens_range(self, params: lsp_types.SemanticTokensRangeParams) -> Union["lsp_types.SemanticTokens", None]:
"""@since 3.16.0"""
return self.send_request("textDocument/semanticTokens/range", params)
def linked_editing_range(self, params: lsp_types.LinkedEditingRangeParams) -> Union["lsp_types.LinkedEditingRanges", None]:
"""A request to provide ranges that can be edited together.
@since 3.16.0
"""
return self.send_request("textDocument/linkedEditingRange", params)
def will_create_files(self, params: lsp_types.CreateFilesParams) -> Union["lsp_types.WorkspaceEdit", None]:
"""The will create files request is sent from the client to the server before files are actually
created as long as the creation is triggered from within the client.
@since 3.16.0
"""
return self.send_request("workspace/willCreateFiles", params)
def will_rename_files(self, params: lsp_types.RenameFilesParams) -> Union["lsp_types.WorkspaceEdit", None]:
"""The will rename files request is sent from the client to the server before files are actually
renamed as long as the rename is triggered from within the client.
@since 3.16.0
"""
return self.send_request("workspace/willRenameFiles", params)
def will_delete_files(self, params: lsp_types.DeleteFilesParams) -> Union["lsp_types.WorkspaceEdit", None]:
"""The did delete files notification is sent from the client to the server when
files were deleted from within the client.
@since 3.16.0
"""
return self.send_request("workspace/willDeleteFiles", params)
def moniker(self, params: lsp_types.MonikerParams) -> list["lsp_types.Moniker"] | None:
"""A request to get the moniker of a symbol at a given text document position.
The request parameter is of type {@link TextDocumentPositionParams}.
The response is of type {@link Moniker Moniker[]} or `null`.
"""
return self.send_request("textDocument/moniker", params)
def prepare_type_hierarchy(self, params: lsp_types.TypeHierarchyPrepareParams) -> list["lsp_types.TypeHierarchyItem"] | None:
"""A request to result a `TypeHierarchyItem` in a document at a given position.
Can be used as an input to a subtypes or supertypes type hierarchy.
@since 3.17.0
"""
return self.send_request("textDocument/prepareTypeHierarchy", params)
def type_hierarchy_supertypes(self, params: lsp_types.TypeHierarchySupertypesParams) -> list["lsp_types.TypeHierarchyItem"] | None:
"""A request to resolve the supertypes for a given `TypeHierarchyItem`.
@since 3.17.0
"""
return self.send_request("typeHierarchy/supertypes", params)
def type_hierarchy_subtypes(self, params: lsp_types.TypeHierarchySubtypesParams) -> list["lsp_types.TypeHierarchyItem"] | None:
"""A request to resolve the subtypes for a given `TypeHierarchyItem`.
@since 3.17.0
"""
return self.send_request("typeHierarchy/subtypes", params)
def inline_value(self, params: lsp_types.InlineValueParams) -> list["lsp_types.InlineValue"] | None:
"""A request to provide inline values in a document. The request's parameter is of
type {@link InlineValueParams}, the response is of type
{@link InlineValue InlineValue[]} or a Thenable that resolves to such.
@since 3.17.0
"""
return self.send_request("textDocument/inlineValue", params)
def inlay_hint(self, params: lsp_types.InlayHintParams) -> list["lsp_types.InlayHint"] | None:
"""A request to provide inlay hints in a document. The request's parameter is of
type {@link InlayHintsParams}, the response is of type
{@link InlayHint InlayHint[]} or a Thenable that resolves to such.
@since 3.17.0
"""
return self.send_request("textDocument/inlayHint", params)
def resolve_inlay_hint(self, params: lsp_types.InlayHint) -> "lsp_types.InlayHint":
"""A request to resolve additional properties for an inlay hint.
The request's parameter is of type {@link InlayHint}, the response is
of type {@link InlayHint} or a Thenable that resolves to such.
@since 3.17.0
"""
return self.send_request("inlayHint/resolve", params)
def text_document_diagnostic(self, params: lsp_types.DocumentDiagnosticParams) -> "lsp_types.DocumentDiagnosticReport":
"""The document diagnostic request definition.
@since 3.17.0
"""
return self.send_request("textDocument/diagnostic", params)
def workspace_diagnostic(self, params: lsp_types.WorkspaceDiagnosticParams) -> "lsp_types.WorkspaceDiagnosticReport":
"""The workspace diagnostic request definition.
@since 3.17.0
"""
return self.send_request("workspace/diagnostic", params)
def initialize(self, params: lsp_types.InitializeParams) -> "lsp_types.InitializeResult":
"""The initialize request is sent from the client to the server.
It is sent once as the request after starting up the server.
The requests parameter is of type {@link InitializeParams}
the response if of type {@link InitializeResult} of a Thenable that
resolves to such.
"""
return self.send_request("initialize", params)
def shutdown(self) -> None:
"""A shutdown request is sent from the client to the server.
It is sent once when the client decides to shutdown the
server. The only notification that is sent after a shutdown request
is the exit event.
"""
return self.send_request("shutdown")
def will_save_wait_until(self, params: lsp_types.WillSaveTextDocumentParams) -> list["lsp_types.TextEdit"] | None:
"""A document will save request is sent from the client to the server before
the document is actually saved. The request can return an array of TextEdits
which will be applied to the text document before it is saved. Please note that
clients might drop results if computing the text edits took too long or if a
server constantly fails on this request. This is done to keep the save fast and
reliable.
"""
return self.send_request("textDocument/willSaveWaitUntil", params)
def completion(self, params: lsp_types.CompletionParams) -> Union[list["lsp_types.CompletionItem"], "lsp_types.CompletionList", None]:
"""Request to request completion at a given text document position. The request's
parameter is of type {@link TextDocumentPosition} the response
is of type {@link CompletionItem CompletionItem[]} or {@link CompletionList}
or a Thenable that resolves to such.
The request can delay the computation of the {@link CompletionItem.detail `detail`}
and {@link CompletionItem.documentation `documentation`} properties to the `completionItem/resolve`
request. However, properties that are needed for the initial sorting and filtering, like `sortText`,
`filterText`, `insertText`, and `textEdit`, must not be changed during resolve.
"""
return self.send_request("textDocument/completion", params)
def resolve_completion_item(self, params: lsp_types.CompletionItem) -> "lsp_types.CompletionItem":
"""Request to resolve additional information for a given completion item.The request's
parameter is of type {@link CompletionItem} the response
is of type {@link CompletionItem} or a Thenable that resolves to such.
"""
return self.send_request("completionItem/resolve", params)
def hover(self, params: lsp_types.HoverParams) -> Union["lsp_types.Hover", None]:
"""Request to request hover information at a given text document position. The request's
parameter is of type {@link TextDocumentPosition} the response is of
type {@link Hover} or a Thenable that resolves to such.
"""
return self.send_request("textDocument/hover", params)
def signature_help(self, params: lsp_types.SignatureHelpParams) -> Union["lsp_types.SignatureHelp", None]:
return self.send_request("textDocument/signatureHelp", params)
def definition(self, params: lsp_types.DefinitionParams) -> Union["lsp_types.Definition", list["lsp_types.LocationLink"], None]:
"""A request to resolve the definition location of a symbol at a given text
document position. The request's parameter is of type [TextDocumentPosition]
(#TextDocumentPosition) the response is of either type {@link Definition}
or a typed array of {@link DefinitionLink} or a Thenable that resolves
to such.
"""
return self.send_request("textDocument/definition", params)
def references(self, params: lsp_types.ReferenceParams) -> list["lsp_types.Location"] | None:
"""A request to resolve project-wide references for the symbol denoted
by the given text document position. The request's parameter is of
type {@link ReferenceParams} the response is of type
{@link Location Location[]} or a Thenable that resolves to such.
"""
return self.send_request("textDocument/references", params)
def document_highlight(self, params: lsp_types.DocumentHighlightParams) -> list["lsp_types.DocumentHighlight"] | None:
"""Request to resolve a {@link DocumentHighlight} for a given
text document position. The request's parameter is of type [TextDocumentPosition]
(#TextDocumentPosition) the request response is of type [DocumentHighlight[]]
(#DocumentHighlight) or a Thenable that resolves to such.
"""
return self.send_request("textDocument/documentHighlight", params)
def document_symbol(
self, params: lsp_types.DocumentSymbolParams
) -> list["lsp_types.SymbolInformation"] | list["lsp_types.DocumentSymbol"] | None:
"""A request to list all symbols found in a given text document. The request's
parameter is of type {@link TextDocumentIdentifier} the
response is of type {@link SymbolInformation SymbolInformation[]} or a Thenable
that resolves to such.
"""
return self.send_request("textDocument/documentSymbol", params)
def code_action(self, params: lsp_types.CodeActionParams) -> list[Union["lsp_types.Command", "lsp_types.CodeAction"]] | None:
"""A request to provide commands for the given text document and range."""
return self.send_request("textDocument/codeAction", params)
def resolve_code_action(self, params: lsp_types.CodeAction) -> "lsp_types.CodeAction":
"""Request to resolve additional information for a given code action.The request's
parameter is of type {@link CodeAction} the response
is of type {@link CodeAction} or a Thenable that resolves to such.
"""
return self.send_request("codeAction/resolve", params)
def workspace_symbol(
self, params: lsp_types.WorkspaceSymbolParams
) -> list["lsp_types.SymbolInformation"] | list["lsp_types.WorkspaceSymbol"] | None:
"""A request to list project-wide symbols matching the query string given
by the {@link WorkspaceSymbolParams}. The response is
of type {@link SymbolInformation SymbolInformation[]} or a Thenable that
resolves to such.
@since 3.17.0 - support for WorkspaceSymbol in the returned data. Clients
need to advertise support for WorkspaceSymbols via the client capability
`workspace.symbol.resolveSupport`.
"""
return self.send_request("workspace/symbol", params)
def resolve_workspace_symbol(self, params: lsp_types.WorkspaceSymbol) -> "lsp_types.WorkspaceSymbol":
"""A request to resolve the range inside the workspace
symbol's location.
@since 3.17.0
"""
return self.send_request("workspaceSymbol/resolve", params)
def code_lens(self, params: lsp_types.CodeLensParams) -> list["lsp_types.CodeLens"] | None:
"""A request to provide code lens for the given text document."""
return self.send_request("textDocument/codeLens", params)
def resolve_code_lens(self, params: lsp_types.CodeLens) -> "lsp_types.CodeLens":
"""A request to resolve a command for a given code lens."""
return self.send_request("codeLens/resolve", params)
def document_link(self, params: lsp_types.DocumentLinkParams) -> list["lsp_types.DocumentLink"] | None:
"""A request to provide document links"""
return self.send_request("textDocument/documentLink", params)
def resolve_document_link(self, params: lsp_types.DocumentLink) -> "lsp_types.DocumentLink":
"""Request to resolve additional information for a given document link. The request's
parameter is of type {@link DocumentLink} the response
is of type {@link DocumentLink} or a Thenable that resolves to such.
"""
return self.send_request("documentLink/resolve", params)
def formatting(self, params: lsp_types.DocumentFormattingParams) -> list["lsp_types.TextEdit"] | None:
"""A request to to format a whole document."""
return self.send_request("textDocument/formatting", params)
def range_formatting(self, params: lsp_types.DocumentRangeFormattingParams) -> list["lsp_types.TextEdit"] | None:
"""A request to to format a range in a document."""
return self.send_request("textDocument/rangeFormatting", params)
def on_type_formatting(self, params: lsp_types.DocumentOnTypeFormattingParams) -> list["lsp_types.TextEdit"] | None:
"""A request to format a document on type."""
return self.send_request("textDocument/onTypeFormatting", params)
def rename(self, params: lsp_types.RenameParams) -> Union["lsp_types.WorkspaceEdit", None]:
"""A request to rename a symbol."""
return self.send_request("textDocument/rename", params)
def prepare_rename(self, params: lsp_types.PrepareRenameParams) -> Union["lsp_types.PrepareRenameResult", None]:
"""A request to test and perform the setup necessary for a rename.
@since 3.16 - support for default behavior
"""
return self.send_request("textDocument/prepareRename", params)
def execute_command(self, params: lsp_types.ExecuteCommandParams) -> Union["lsp_types.LSPAny", None]:
"""A request send from the client to the server to execute a command. The request might return
a workspace edit which the client will apply to the workspace.
"""
return self.send_request("workspace/executeCommand", params)
src/multilspy/multilspy_types.py → src/solidlsp/ls_types.py
+44 -24
View File
@@ -4,16 +4,19 @@ Defines wrapper objects around the types returned by LSP to ensure decoupling be
from __future__ import annotations
from enum import IntEnum, Enum
from typing_extensions import NotRequired, TypedDict, List, Dict, Union
from enum import Enum, IntEnum
from typing import NotRequired, Union
from typing_extensions import TypedDict
URI = str
DocumentUri = str
Uint = int
RegExp = str
class Position(TypedDict):
"""Position in a text document expressed as zero-based line and character
r"""Position in a text document expressed as zero-based line and character
offset. Prior to 3.17 the offsets were always based on a UTF-16 string
representation. So a string of the form `a𐐀b` the character offset of the
character `a` is 0, the character offset of `𐐀` is 1 and the character
@@ -39,7 +42,8 @@ class Position(TypedDict):
Positions are line end character agnostic. So you can not specify a position
that denotes `\r|\n` or `\n|` where `|` represents the character offset.
@since 3.17.0 - support for negotiated position encoding."""
@since 3.17.0 - support for negotiated position encoding.
"""
line: Uint
""" Line position in a document (zero-based).
@@ -67,7 +71,8 @@ class Range(TypedDict):
start: { line: 5, character: 23 }
end : { line 6, character : 0 }
}
```"""
```
"""
start: Position
""" The range's start position. """
@@ -77,12 +82,14 @@ class Range(TypedDict):
class Location(TypedDict):
"""Represents a location inside a resource, such as a line
inside a text file."""
inside a text file.
"""
uri: DocumentUri
range: Range
absolutePath: str
relativePath: Union[str, None]
relativePath: str | None
class CompletionItemKind(IntEnum):
"""The kind of a completion entry."""
@@ -113,9 +120,11 @@ class CompletionItemKind(IntEnum):
Operator = 24
TypeParameter = 25
class CompletionItem(TypedDict):
"""A completion item represents a text snippet that is
proposed to complete text that is being typed."""
proposed to complete text that is being typed.
"""
completionText: str
""" The completionText of this completion item.
@@ -131,6 +140,7 @@ class CompletionItem(TypedDict):
""" A human-readable string with additional information
about this item, like type or symbol information. """
class SymbolKind(IntEnum):
"""A symbol kind."""
@@ -161,17 +171,21 @@ class SymbolKind(IntEnum):
Operator = 25
TypeParameter = 26
class SymbolTag(IntEnum):
"""Symbol tags are extra annotations that tweak the rendering of a symbol.
@since 3.16"""
@since 3.16
"""
Deprecated = 1
""" Render a symbol as obsolete, usually using a strike-out. """
class UnifiedSymbolInformation(TypedDict):
"""Represents information about programming constructs like variables, classes,
interfaces etc."""
interfaces etc.
"""
deprecated: NotRequired[bool]
""" Indicates if this symbol is deprecated.
@@ -191,7 +205,7 @@ class UnifiedSymbolInformation(TypedDict):
""" The name of this symbol. """
kind: SymbolKind
""" The kind of this symbol. """
tags: NotRequired[List[SymbolTag]]
tags: NotRequired[list[SymbolTag]]
""" Tags for this symbol.
@since 3.16.0 """
@@ -207,7 +221,7 @@ class UnifiedSymbolInformation(TypedDict):
detail: NotRequired[str]
""" More detail for this symbol, e.g the signature of a function. """
range: NotRequired[Range]
""" The range enclosing this symbol not including leading/trailing whitespace but everything else
like comments. This information is typically used to determine if the clients cursor is
@@ -215,37 +229,40 @@ class UnifiedSymbolInformation(TypedDict):
selectionRange: NotRequired[Range]
""" The range that should be selected and revealed when this symbol is being picked, e.g the name of a function.
Must be contained by the `range`. """
body: NotRequired[str]
""" The body of the symbol. """
children: List[UnifiedSymbolInformation]
children: list[UnifiedSymbolInformation]
""" The children of the symbol.
Added to be compatible with `lsp_types.DocumentSymbol`,
since it is sometimes useful to have the children of the symbol as a user-facing feature."""
parent: NotRequired[UnifiedSymbolInformation | None]
"""The parent of the symbol, if there is any. Added with Serena, not part of the LSP.
All symbols except the root packages will have a parent.
"""
class MarkupKind(Enum):
"""Describes the content type that a client supports in various
result literals like `Hover`, `ParameterInfo` or `CompletionItem`.
Please note that `MarkupKinds` must not start with a `$`. This kinds
are reserved for internal usage."""
are reserved for internal usage.
"""
PlainText = "plaintext"
""" Plain text is supported as a content format """
Markdown = "markdown"
""" Markdown is supported as a content format """
class __MarkedString_Type_1(TypedDict):
language: str
value: str
MarkedString = Union[str, "__MarkedString_Type_1"]
""" MarkedString can be used to render human readable text. It is either a markdown string
or a code-block that provides a language and a code snippet. The language identifier
@@ -260,8 +277,9 @@ ${value}
Note that markdown strings will be sanitized - that means html will be escaped.
@deprecated use MarkupContent instead. """
class MarkupContent(TypedDict):
"""A `MarkupContent` literal represents a string value which content is interpreted base on its
r"""A `MarkupContent` literal represents a string value which content is interpreted base on its
kind flag. Currently the protocol supports `plaintext` and `markdown` as markup kinds.
If the kind is `markdown` then the value can contain fenced code blocks like in GitHub issues.
@@ -282,18 +300,20 @@ class MarkupContent(TypedDict):
```
*Please Note* that clients might sanitize the return markdown. A client could decide to
remove HTML from the markdown to avoid script execution."""
remove HTML from the markdown to avoid script execution.
"""
kind: "MarkupKind"
kind: MarkupKind
""" The type of the Markup """
value: str
""" The content itself """
class Hover(TypedDict):
"""The result of a hover request."""
contents: Union["MarkupContent", "MarkedString", List["MarkedString"]]
contents: MarkupContent | MarkedString | list[MarkedString]
""" The hover's content """
range: NotRequired["Range"]
range: NotRequired[Range]
""" An optional range inside the text document that is used to
visualize the hover, e.g. by changing the background color. """
visualize the hover, e.g. by changing the background color. """
src/multilspy/multilspy_utils.py → src/solidlsp/ls_utils.py
+102 -73
View File
@@ -5,27 +5,31 @@ This file contains various utility functions like I/O operations, handling paths
import gzip
import logging
import os
from typing import Tuple, Union
import requests
import shutil
import uuid
import platform
import shutil
import subprocess
import uuid
from enum import Enum
from pathlib import Path, PurePath
from multilspy.multilspy_exceptions import MultilspyException
from pathlib import PurePath, Path
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.multilspy_types import UnifiedSymbolInformation
import requests
from solidlsp.ls_exceptions import LanguageServerException
from solidlsp.ls_logger import LanguageServerLogger
from solidlsp.ls_types import UnifiedSymbolInformation
class InvalidTextLocationError(Exception):
pass
class TextUtils:
"""
Utilities for text operations.
"""
@staticmethod
def get_line_col_from_index(text: str, index: int) -> Tuple[int, int]:
def get_line_col_from_index(text: str, index: int) -> tuple[int, int]:
"""
Returns the zero-indexed line and column number of the given index in the given text
"""
@@ -33,7 +37,7 @@ class TextUtils:
c = 0
idx = 0
while idx < index:
if text[idx] == '\n':
if text[idx] == "\n":
l += 1
c = 0
else:
@@ -41,7 +45,7 @@ class TextUtils:
idx += 1
return l, c
@staticmethod
def get_index_from_line_col(text: str, line: int, col: int) -> int:
"""
@@ -49,46 +53,57 @@ class TextUtils:
"""
idx = 0
while line > 0:
assert idx < len(text), (idx, len(text), text)
if idx >= len(text):
raise InvalidTextLocationError
if text[idx] == "\n":
line -= 1
idx += 1
idx += col
return idx
@staticmethod
def _get_updated_position_from_line_and_column_and_edit(l: int, c: int, text_to_be_inserted: str) -> Tuple[int, int]:
def _get_updated_position_from_line_and_column_and_edit(l: int, c: int, text_to_be_inserted: str) -> tuple[int, int]:
"""
Utility function to get the position of the cursor after inserting text at a given line and column.
"""
num_newlines_in_gen_text = text_to_be_inserted.count('\n')
num_newlines_in_gen_text = text_to_be_inserted.count("\n")
if num_newlines_in_gen_text > 0:
l += num_newlines_in_gen_text
c = len(text_to_be_inserted.split('\n')[-1])
c = len(text_to_be_inserted.split("\n")[-1])
else:
c += len(text_to_be_inserted)
return (l, c)
@staticmethod
def delete_text_between_positions(text: str, start_line: int, start_col: int, end_line: int, end_col: int) -> Tuple[str, str]:
def delete_text_between_positions(text: str, start_line: int, start_col: int, end_line: int, end_col: int) -> tuple[str, str]:
"""
Deletes the text between the given start and end positions.
Returns the modified text and the deleted text.
"""
del_start_idx = TextUtils.get_index_from_line_col(text, start_line, start_col)
del_end_idx = TextUtils.get_index_from_line_col(text, end_line, end_col)
deleted_text = text[del_start_idx:del_end_idx]
new_text = text[:del_start_idx] + text[del_end_idx:]
return new_text, deleted_text
@staticmethod
def insert_text_at_position(text: str, line: int, col: int, text_to_be_inserted: str) -> Tuple[str, int, int]:
def insert_text_at_position(text: str, line: int, col: int, text_to_be_inserted: str) -> tuple[str, int, int]:
"""
Inserts the given text at the given line and column.
Returns the modified text and the new line and column.
"""
change_index = TextUtils.get_index_from_line_col(text, line, col)
try:
change_index = TextUtils.get_index_from_line_col(text, line, col)
except InvalidTextLocationError:
num_lines_in_text = text.count("\n") + 1
max_line = num_lines_in_text - 1
if line == max_line + 1 and col == 0: # trying to insert at new line after full text
# insert at end, adding missing newline
change_index = len(text)
text_to_be_inserted = "\n" + text_to_be_inserted
else:
raise
new_text = text[:change_index] + text_to_be_inserted + text[change_index:]
new_l, new_c = TextUtils._get_updated_position_from_line_and_column_and_edit(line, col, text_to_be_inserted)
return new_text, new_l, new_c
@@ -98,6 +113,7 @@ class PathUtils:
"""
Utilities for platform-agnostic path operations.
"""
@staticmethod
def uri_to_path(uri: str) -> str:
"""
@@ -106,14 +122,15 @@ class PathUtils:
This method was obtained from https://stackoverflow.com/a/61922504
"""
try:
from urllib.parse import urlparse, unquote
from urllib.parse import unquote, urlparse
from urllib.request import url2pathname
except ImportError:
# backwards compatability
from urlparse import urlparse
# backwards compatibility
from urllib import unquote, url2pathname
from urlparse import urlparse
parsed = urlparse(uri)
host = "{0}{0}{mnt}{0}".format(os.path.sep, mnt=parsed.netloc)
host = f"{os.path.sep}{os.path.sep}{parsed.netloc}{os.path.sep}"
return os.path.normpath(os.path.join(host, url2pathname(unquote(parsed.path))))
@staticmethod
@@ -126,10 +143,10 @@ class PathUtils:
@staticmethod
def is_glob_pattern(pattern: str) -> bool:
"""Check if a pattern contains glob-specific characters."""
return any(c in pattern for c in '*?[]!')
return any(c in pattern for c in "*?[]!")
@staticmethod
def get_relative_path(path: str, base_path: str) -> Union[str, None]:
def get_relative_path(path: str, base_path: str) -> str | None:
"""
Gets relative path if it's possible (paths should be on the same drive),
returns `None` otherwise.
@@ -145,22 +162,22 @@ class FileUtils:
"""
@staticmethod
def read_file(logger: MultilspyLogger, file_path: str) -> str:
def read_file(logger: LanguageServerLogger, file_path: str) -> str:
"""
Reads the file at the given path and returns the contents as a string.
"""
if not os.path.exists(file_path):
logger.log(f"File read '{file_path}' failed: File does not exist.", logging.ERROR)
raise MultilspyException(f"File read '{file_path}' failed: File does not exist.")
raise LanguageServerException(f"File read '{file_path}' failed: File does not exist.")
try:
with open(file_path, "r", encoding="utf-8") as inp_file:
with open(file_path, encoding="utf-8") as inp_file:
return inp_file.read()
except Exception as exc:
logger.log(f"File read '{file_path}' failed to read with encoding 'utf-8': {exc}", logging.ERROR)
raise MultilspyException("File read failed.") from None
raise LanguageServerException("File read failed.") from None
@staticmethod
def download_file(logger: MultilspyLogger, url: str, target_path: str) -> None:
def download_file(logger: LanguageServerLogger, url: str, target_path: str) -> None:
"""
Downloads the file from the given URL to the given {target_path}
"""
@@ -168,15 +185,15 @@ class FileUtils:
response = requests.get(url, stream=True, timeout=60)
if response.status_code != 200:
logger.log(f"Error downloading file '{url}': {response.status_code} {response.text}", logging.ERROR)
raise MultilspyException("Error downoading file.")
raise LanguageServerException("Error downloading file.")
with open(target_path, "wb") as f:
shutil.copyfileobj(response.raw, f)
except Exception as exc:
logger.log(f"Error downloading file '{url}': {exc}", logging.ERROR)
raise MultilspyException("Error downoading file.") from None
raise LanguageServerException("Error downloading file.") from None
@staticmethod
def download_and_extract_archive(logger: MultilspyLogger, url: str, target_path: str, archive_type: str) -> None:
def download_and_extract_archive(logger: LanguageServerLogger, url: str, target_path: str, archive_type: str) -> None:
"""
Downloads the archive from the given URL having format {archive_type} and extracts it to the given {target_path}
"""
@@ -201,10 +218,10 @@ class FileUtils:
shutil.copyfileobj(f_in, f_out)
else:
logger.log(f"Unknown archive type '{archive_type}' for extraction", logging.ERROR)
raise MultilspyException(f"Unknown archive type '{archive_type}'")
raise LanguageServerException(f"Unknown archive type '{archive_type}'")
except Exception as exc:
logger.log(f"Error extracting archive '{tmp_file_name}' obtained from '{url}': {exc}", logging.ERROR)
raise MultilspyException("Error extracting archive.") from exc
raise LanguageServerException("Error extracting archive.") from exc
finally:
for tmp_file_name in tmp_files:
if os.path.exists(tmp_file_name):
@@ -215,6 +232,7 @@ class PlatformId(str, Enum):
"""
multilspy supported platforms
"""
WIN_x86 = "win-x86"
WIN_x64 = "win-x64"
WIN_arm64 = "win-arm64"
@@ -232,6 +250,7 @@ class DotnetVersion(str, Enum):
"""
multilspy supported dotnet versions
"""
V4 = "4"
V6 = "6"
V7 = "7"
@@ -260,11 +279,11 @@ class PlatformUtils:
platform_id = system_map[system] + "-" + machine_map[machine]
if system == "Linux" and bitness == "64bit":
libc = platform.libc_ver()[0]
if libc != 'glibc':
if libc != "glibc":
platform_id += "-" + libc
return PlatformId(platform_id)
else:
raise MultilspyException(f"Unknown platform: {system=}, {machine=}, {bitness=}")
raise LanguageServerException(f"Unknown platform: {system=}, {machine=}, {bitness=}")
@staticmethod
def _determine_windows_machine_type():
@@ -274,13 +293,13 @@ class PlatformUtils:
class SYSTEM_INFO(ctypes.Structure):
class _U(ctypes.Union):
class _S(ctypes.Structure):
_fields_ = [("wProcessorArchitecture", wintypes.WORD),
("wReserved", wintypes.WORD)]
_fields_ = [("dwOemId", wintypes.DWORD),
("s", _S)]
_fields_ = [("wProcessorArchitecture", wintypes.WORD), ("wReserved", wintypes.WORD)]
_fields_ = [("dwOemId", wintypes.DWORD), ("s", _S)]
_anonymous_ = ("s",)
_fields_ = [("u", _U),
_fields_ = [
("u", _U),
("dwPageSize", wintypes.DWORD),
("lpMinimumApplicationAddress", wintypes.LPVOID),
("lpMaximumApplicationAddress", wintypes.LPVOID),
@@ -289,22 +308,22 @@ class PlatformUtils:
("dwProcessorType", wintypes.DWORD),
("dwAllocationGranularity", wintypes.DWORD),
("wProcessorLevel", wintypes.WORD),
("wProcessorRevision", wintypes.WORD)]
("wProcessorRevision", wintypes.WORD),
]
_anonymous_ = ("u",)
sys_info = SYSTEM_INFO()
ctypes.windll.kernel32.GetNativeSystemInfo(ctypes.byref(sys_info))
arch_map = {
9: 'AMD64',
5: 'ARM',
12: 'arm64',
6: 'Intel Itanium-based',
0: 'i386',
9: "AMD64",
5: "ARM",
12: "arm64",
6: "Intel Itanium-based",
0: "i386",
}
return arch_map.get(sys_info.wProcessorArchitecture, f'Unknown ({sys_info.wProcessorArchitecture})')
return arch_map.get(sys_info.wProcessorArchitecture, f"Unknown ({sys_info.wProcessorArchitecture})")
@staticmethod
def get_dotnet_version() -> DotnetVersion:
@@ -313,29 +332,39 @@ class PlatformUtils:
"""
try:
result = subprocess.run(["dotnet", "--list-runtimes"], capture_output=True, check=True)
version = ''
for line in result.stdout.decode('utf-8').split('\n'):
if line.startswith('Microsoft.NETCore.App'):
version = line.split(' ')[1]
break
if version == '':
raise MultilspyException("dotnet not found on the system")
if version.startswith("8"):
return DotnetVersion.V8
elif version.startswith("7"):
return DotnetVersion.V7
elif version.startswith("6"):
return DotnetVersion.V6
elif version.startswith("4"):
return DotnetVersion.V4
else:
raise MultilspyException("Unknown dotnet version: " + version)
available_version_cmd_output = []
for line in result.stdout.decode("utf-8").split("\n"):
if line.startswith("Microsoft.NETCore.App"):
version_cmd_output = line.split(" ")[1]
available_version_cmd_output.append(version_cmd_output)
if not available_version_cmd_output:
raise LanguageServerException("dotnet not found on the system")
# Check for supported versions in order of preference (latest first)
for version_cmd_output in available_version_cmd_output:
if version_cmd_output.startswith("8"):
return DotnetVersion.V8
for version_cmd_output in available_version_cmd_output:
if version_cmd_output.startswith("7"):
return DotnetVersion.V7
for version_cmd_output in available_version_cmd_output:
if version_cmd_output.startswith("6"):
return DotnetVersion.V6
for version_cmd_output in available_version_cmd_output:
if version_cmd_output.startswith("4"):
return DotnetVersion.V4
# If no supported version found, raise exception with all available versions
raise LanguageServerException(
f"No supported dotnet version found. Available versions: {', '.join(available_version_cmd_output)}. Supported versions: 4, 6, 7, 8"
)
except (FileNotFoundError, subprocess.CalledProcessError):
try:
result = subprocess.run(["mono", "--version"], capture_output=True, check=True)
return DotnetVersion.VMONO
except (FileNotFoundError, subprocess.CalledProcessError):
raise MultilspyException("dotnet or mono not found on the system")
raise LanguageServerException("dotnet or mono not found on the system")
class SymbolUtils:
src/multilspy/lsp_protocol_handler/lsp_constants.py → src/solidlsp/lsp_protocol_handler/lsp_constants.py
+2 -1
View File
@@ -2,6 +2,7 @@
This module contains constants used in the LSP protocol.
"""
class LSPConstants:
"""
This class contains constants used in the LSP protocol.
@@ -31,7 +32,7 @@ class LSPConstants:
# key used to represent the language a document is in - "java", "csharp", etc.
LANGUAGE_ID = "languageId"
# key used to represent the version of a document (a shared value betwen the client and server)
# key used to represent the version of a document (a shared value between the client and server)
VERSION = "version"
# key used to represent the text of a document being sent from the client to the server on open
src/multilspy/lsp_protocol_handler/lsp_requests.py → src/solidlsp/lsp_protocol_handler/lsp_requests.py
+174 -228
View File
@@ -29,8 +29,10 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""
from typing import List, Union
from multilspy.lsp_protocol_handler import lsp_types
from typing import Union
from solidlsp.lsp_protocol_handler import lsp_types
class LspRequest:
def __init__(self, send_request):
@@ -38,96 +40,94 @@ class LspRequest:
async def implementation(
self, params: lsp_types.ImplementationParams
) -> Union["lsp_types.Definition", List["lsp_types.LocationLink"], None]:
) -> Union["lsp_types.Definition", list["lsp_types.LocationLink"], None]:
"""A request to resolve the implementation locations of a symbol at a given text
document position. The request's parameter is of type [TextDocumentPositionParams]
(#TextDocumentPositionParams) the response is of type {@link Definition} or a
Thenable that resolves to such."""
Thenable that resolves to such.
"""
return await self.send_request("textDocument/implementation", params)
async def type_definition(
self, params: lsp_types.TypeDefinitionParams
) -> Union["lsp_types.Definition", List["lsp_types.LocationLink"], None]:
) -> Union["lsp_types.Definition", list["lsp_types.LocationLink"], None]:
"""A request to resolve the type definition locations of a symbol at a given text
document position. The request's parameter is of type [TextDocumentPositionParams]
(#TextDocumentPositionParams) the response is of type {@link Definition} or a
Thenable that resolves to such."""
Thenable that resolves to such.
"""
return await self.send_request("textDocument/typeDefinition", params)
async def document_color(
self, params: lsp_types.DocumentColorParams
) -> List["lsp_types.ColorInformation"]:
async def document_color(self, params: lsp_types.DocumentColorParams) -> list["lsp_types.ColorInformation"]:
"""A request to list all color symbols found in a given text document. The request's
parameter is of type {@link DocumentColorParams} the
response is of type {@link ColorInformation ColorInformation[]} or a Thenable
that resolves to such."""
that resolves to such.
"""
return await self.send_request("textDocument/documentColor", params)
async def color_presentation(
self, params: lsp_types.ColorPresentationParams
) -> List["lsp_types.ColorPresentation"]:
async def color_presentation(self, params: lsp_types.ColorPresentationParams) -> list["lsp_types.ColorPresentation"]:
"""A request to list all presentation for a color. The request's
parameter is of type {@link ColorPresentationParams} the
response is of type {@link ColorInformation ColorInformation[]} or a Thenable
that resolves to such."""
that resolves to such.
"""
return await self.send_request("textDocument/colorPresentation", params)
async def folding_range(
self, params: lsp_types.FoldingRangeParams
) -> Union[List["lsp_types.FoldingRange"], None]:
async def folding_range(self, params: lsp_types.FoldingRangeParams) -> list["lsp_types.FoldingRange"] | None:
"""A request to provide folding ranges in a document. The request's
parameter is of type {@link FoldingRangeParams}, the
response is of type {@link FoldingRangeList} or a Thenable
that resolves to such."""
that resolves to such.
"""
return await self.send_request("textDocument/foldingRange", params)
async def declaration(
self, params: lsp_types.DeclarationParams
) -> Union["lsp_types.Declaration", List["lsp_types.LocationLink"], None]:
) -> Union["lsp_types.Declaration", list["lsp_types.LocationLink"], None]:
"""A request to resolve the type definition locations of a symbol at a given text
document position. The request's parameter is of type [TextDocumentPositionParams]
(#TextDocumentPositionParams) the response is of type {@link Declaration}
or a typed array of {@link DeclarationLink} or a Thenable that resolves
to such."""
to such.
"""
return await self.send_request("textDocument/declaration", params)
async def selection_range(
self, params: lsp_types.SelectionRangeParams
) -> Union[List["lsp_types.SelectionRange"], None]:
async def selection_range(self, params: lsp_types.SelectionRangeParams) -> list["lsp_types.SelectionRange"] | None:
"""A request to provide selection ranges in a document. The request's
parameter is of type {@link SelectionRangeParams}, the
response is of type {@link SelectionRange SelectionRange[]} or a Thenable
that resolves to such."""
that resolves to such.
"""
return await self.send_request("textDocument/selectionRange", params)
async def prepare_call_hierarchy(
self, params: lsp_types.CallHierarchyPrepareParams
) -> Union[List["lsp_types.CallHierarchyItem"], None]:
async def prepare_call_hierarchy(self, params: lsp_types.CallHierarchyPrepareParams) -> list["lsp_types.CallHierarchyItem"] | None:
"""A request to result a `CallHierarchyItem` in a document at a given position.
Can be used as an input to an incoming or outgoing call hierarchy.
@since 3.16.0"""
@since 3.16.0
"""
return await self.send_request("textDocument/prepareCallHierarchy", params)
async def incoming_calls(
self, params: lsp_types.CallHierarchyIncomingCallsParams
) -> Union[List["lsp_types.CallHierarchyIncomingCall"], None]:
) -> list["lsp_types.CallHierarchyIncomingCall"] | None:
"""A request to resolve the incoming calls for a given `CallHierarchyItem`.
@since 3.16.0"""
@since 3.16.0
"""
return await self.send_request("callHierarchy/incomingCalls", params)
async def outgoing_calls(
self, params: lsp_types.CallHierarchyOutgoingCallsParams
) -> Union[List["lsp_types.CallHierarchyOutgoingCall"], None]:
) -> list["lsp_types.CallHierarchyOutgoingCall"] | None:
"""A request to resolve the outgoing calls for a given `CallHierarchyItem`.
@since 3.16.0"""
@since 3.16.0
"""
return await self.send_request("callHierarchy/outgoingCalls", params)
async def semantic_tokens_full(
self, params: lsp_types.SemanticTokensParams
) -> Union["lsp_types.SemanticTokens", None]:
async def semantic_tokens_full(self, params: lsp_types.SemanticTokensParams) -> Union["lsp_types.SemanticTokens", None]:
"""@since 3.16.0"""
return await self.send_request("textDocument/semanticTokens/full", params)
@@ -137,157 +137,143 @@ class LspRequest:
"""@since 3.16.0"""
return await self.send_request("textDocument/semanticTokens/full/delta", params)
async def semantic_tokens_range(
self, params: lsp_types.SemanticTokensRangeParams
) -> Union["lsp_types.SemanticTokens", None]:
async def semantic_tokens_range(self, params: lsp_types.SemanticTokensRangeParams) -> Union["lsp_types.SemanticTokens", None]:
"""@since 3.16.0"""
return await self.send_request("textDocument/semanticTokens/range", params)
async def linked_editing_range(
self, params: lsp_types.LinkedEditingRangeParams
) -> Union["lsp_types.LinkedEditingRanges", None]:
async def linked_editing_range(self, params: lsp_types.LinkedEditingRangeParams) -> Union["lsp_types.LinkedEditingRanges", None]:
"""A request to provide ranges that can be edited together.
@since 3.16.0"""
@since 3.16.0
"""
return await self.send_request("textDocument/linkedEditingRange", params)
async def will_create_files(
self, params: lsp_types.CreateFilesParams
) -> Union["lsp_types.WorkspaceEdit", None]:
async def will_create_files(self, params: lsp_types.CreateFilesParams) -> Union["lsp_types.WorkspaceEdit", None]:
"""The will create files request is sent from the client to the server before files are actually
created as long as the creation is triggered from within the client.
@since 3.16.0"""
@since 3.16.0
"""
return await self.send_request("workspace/willCreateFiles", params)
async def will_rename_files(
self, params: lsp_types.RenameFilesParams
) -> Union["lsp_types.WorkspaceEdit", None]:
async def will_rename_files(self, params: lsp_types.RenameFilesParams) -> Union["lsp_types.WorkspaceEdit", None]:
"""The will rename files request is sent from the client to the server before files are actually
renamed as long as the rename is triggered from within the client.
@since 3.16.0"""
@since 3.16.0
"""
return await self.send_request("workspace/willRenameFiles", params)
async def will_delete_files(
self, params: lsp_types.DeleteFilesParams
) -> Union["lsp_types.WorkspaceEdit", None]:
async def will_delete_files(self, params: lsp_types.DeleteFilesParams) -> Union["lsp_types.WorkspaceEdit", None]:
"""The did delete files notification is sent from the client to the server when
files were deleted from within the client.
@since 3.16.0"""
@since 3.16.0
"""
return await self.send_request("workspace/willDeleteFiles", params)
async def moniker(
self, params: lsp_types.MonikerParams
) -> Union[List["lsp_types.Moniker"], None]:
async def moniker(self, params: lsp_types.MonikerParams) -> list["lsp_types.Moniker"] | None:
"""A request to get the moniker of a symbol at a given text document position.
The request parameter is of type {@link TextDocumentPositionParams}.
The response is of type {@link Moniker Moniker[]} or `null`."""
The response is of type {@link Moniker Moniker[]} or `null`.
"""
return await self.send_request("textDocument/moniker", params)
async def prepare_type_hierarchy(
self, params: lsp_types.TypeHierarchyPrepareParams
) -> Union[List["lsp_types.TypeHierarchyItem"], None]:
async def prepare_type_hierarchy(self, params: lsp_types.TypeHierarchyPrepareParams) -> list["lsp_types.TypeHierarchyItem"] | None:
"""A request to result a `TypeHierarchyItem` in a document at a given position.
Can be used as an input to a subtypes or supertypes type hierarchy.
@since 3.17.0"""
@since 3.17.0
"""
return await self.send_request("textDocument/prepareTypeHierarchy", params)
async def type_hierarchy_supertypes(
self, params: lsp_types.TypeHierarchySupertypesParams
) -> Union[List["lsp_types.TypeHierarchyItem"], None]:
) -> list["lsp_types.TypeHierarchyItem"] | None:
"""A request to resolve the supertypes for a given `TypeHierarchyItem`.
@since 3.17.0"""
@since 3.17.0
"""
return await self.send_request("typeHierarchy/supertypes", params)
async def type_hierarchy_subtypes(
self, params: lsp_types.TypeHierarchySubtypesParams
) -> Union[List["lsp_types.TypeHierarchyItem"], None]:
async def type_hierarchy_subtypes(self, params: lsp_types.TypeHierarchySubtypesParams) -> list["lsp_types.TypeHierarchyItem"] | None:
"""A request to resolve the subtypes for a given `TypeHierarchyItem`.
@since 3.17.0"""
@since 3.17.0
"""
return await self.send_request("typeHierarchy/subtypes", params)
async def inline_value(
self, params: lsp_types.InlineValueParams
) -> Union[List["lsp_types.InlineValue"], None]:
async def inline_value(self, params: lsp_types.InlineValueParams) -> list["lsp_types.InlineValue"] | None:
"""A request to provide inline values in a document. The request's parameter is of
type {@link InlineValueParams}, the response is of type
{@link InlineValue InlineValue[]} or a Thenable that resolves to such.
@since 3.17.0"""
@since 3.17.0
"""
return await self.send_request("textDocument/inlineValue", params)
async def inlay_hint(
self, params: lsp_types.InlayHintParams
) -> Union[List["lsp_types.InlayHint"], None]:
async def inlay_hint(self, params: lsp_types.InlayHintParams) -> list["lsp_types.InlayHint"] | None:
"""A request to provide inlay hints in a document. The request's parameter is of
type {@link InlayHintsParams}, the response is of type
{@link InlayHint InlayHint[]} or a Thenable that resolves to such.
@since 3.17.0"""
@since 3.17.0
"""
return await self.send_request("textDocument/inlayHint", params)
async def resolve_inlay_hint(
self, params: lsp_types.InlayHint
) -> "lsp_types.InlayHint":
async def resolve_inlay_hint(self, params: lsp_types.InlayHint) -> "lsp_types.InlayHint":
"""A request to resolve additional properties for an inlay hint.
The request's parameter is of type {@link InlayHint}, the response is
of type {@link InlayHint} or a Thenable that resolves to such.
@since 3.17.0"""
@since 3.17.0
"""
return await self.send_request("inlayHint/resolve", params)
async def text_document_diagnostic(
self, params: lsp_types.DocumentDiagnosticParams
) -> "lsp_types.DocumentDiagnosticReport":
async def text_document_diagnostic(self, params: lsp_types.DocumentDiagnosticParams) -> "lsp_types.DocumentDiagnosticReport":
"""The document diagnostic request definition.
@since 3.17.0"""
@since 3.17.0
"""
return await self.send_request("textDocument/diagnostic", params)
async def workspace_diagnostic(
self, params: lsp_types.WorkspaceDiagnosticParams
) -> "lsp_types.WorkspaceDiagnosticReport":
async def workspace_diagnostic(self, params: lsp_types.WorkspaceDiagnosticParams) -> "lsp_types.WorkspaceDiagnosticReport":
"""The workspace diagnostic request definition.
@since 3.17.0"""
@since 3.17.0
"""
return await self.send_request("workspace/diagnostic", params)
async def initialize(
self, params: lsp_types.InitializeParams
) -> "lsp_types.InitializeResult":
async def initialize(self, params: lsp_types.InitializeParams) -> "lsp_types.InitializeResult":
"""The initialize request is sent from the client to the server.
It is sent once as the request after starting up the server.
The requests parameter is of type {@link InitializeParams}
the response if of type {@link InitializeResult} of a Thenable that
resolves to such."""
resolves to such.
"""
return await self.send_request("initialize", params)
async def shutdown(self) -> None:
"""A shutdown request is sent from the client to the server.
It is sent once when the client decides to shutdown the
server. The only notification that is sent after a shutdown request
is the exit event."""
is the exit event.
"""
return await self.send_request("shutdown")
async def will_save_wait_until(
self, params: lsp_types.WillSaveTextDocumentParams
) -> Union[List["lsp_types.TextEdit"], None]:
async def will_save_wait_until(self, params: lsp_types.WillSaveTextDocumentParams) -> list["lsp_types.TextEdit"] | None:
"""A document will save request is sent from the client to the server before
the document is actually saved. The request can return an array of TextEdits
which will be applied to the text document before it is saved. Please note that
clients might drop results if computing the text edits took too long or if a
server constantly fails on this request. This is done to keep the save fast and
reliable."""
reliable.
"""
return await self.send_request("textDocument/willSaveWaitUntil", params)
async def completion(
self, params: lsp_types.CompletionParams
) -> Union[List["lsp_types.CompletionItem"], "lsp_types.CompletionList", None]:
) -> Union[list["lsp_types.CompletionItem"], "lsp_types.CompletionList", None]:
"""Request to request completion at a given text document position. The request's
parameter is of type {@link TextDocumentPosition} the response
is of type {@link CompletionItem CompletionItem[]} or {@link CompletionList}
@@ -300,85 +286,72 @@ class LspRequest:
"""
return await self.send_request("textDocument/completion", params)
async def resolve_completion_item(
self, params: lsp_types.CompletionItem
) -> "lsp_types.CompletionItem":
async def resolve_completion_item(self, params: lsp_types.CompletionItem) -> "lsp_types.CompletionItem":
"""Request to resolve additional information for a given completion item.The request's
parameter is of type {@link CompletionItem} the response
is of type {@link CompletionItem} or a Thenable that resolves to such."""
is of type {@link CompletionItem} or a Thenable that resolves to such.
"""
return await self.send_request("completionItem/resolve", params)
async def hover(
self, params: lsp_types.HoverParams
) -> Union["lsp_types.Hover", None]:
async def hover(self, params: lsp_types.HoverParams) -> Union["lsp_types.Hover", None]:
"""Request to request hover information at a given text document position. The request's
parameter is of type {@link TextDocumentPosition} the response is of
type {@link Hover} or a Thenable that resolves to such."""
type {@link Hover} or a Thenable that resolves to such.
"""
return await self.send_request("textDocument/hover", params)
async def signature_help(
self, params: lsp_types.SignatureHelpParams
) -> Union["lsp_types.SignatureHelp", None]:
async def signature_help(self, params: lsp_types.SignatureHelpParams) -> Union["lsp_types.SignatureHelp", None]:
return await self.send_request("textDocument/signatureHelp", params)
async def definition(
self, params: lsp_types.DefinitionParams
) -> Union["lsp_types.Definition", List["lsp_types.LocationLink"], None]:
async def definition(self, params: lsp_types.DefinitionParams) -> Union["lsp_types.Definition", list["lsp_types.LocationLink"], None]:
"""A request to resolve the definition location of a symbol at a given text
document position. The request's parameter is of type [TextDocumentPosition]
(#TextDocumentPosition) the response is of either type {@link Definition}
or a typed array of {@link DefinitionLink} or a Thenable that resolves
to such."""
to such.
"""
return await self.send_request("textDocument/definition", params)
async def references(
self, params: lsp_types.ReferenceParams
) -> Union[List["lsp_types.Location"], None]:
async def references(self, params: lsp_types.ReferenceParams) -> list["lsp_types.Location"] | None:
"""A request to resolve project-wide references for the symbol denoted
by the given text document position. The request's parameter is of
type {@link ReferenceParams} the response is of type
{@link Location Location[]} or a Thenable that resolves to such."""
{@link Location Location[]} or a Thenable that resolves to such.
"""
return await self.send_request("textDocument/references", params)
async def document_highlight(
self, params: lsp_types.DocumentHighlightParams
) -> Union[List["lsp_types.DocumentHighlight"], None]:
async def document_highlight(self, params: lsp_types.DocumentHighlightParams) -> list["lsp_types.DocumentHighlight"] | None:
"""Request to resolve a {@link DocumentHighlight} for a given
text document position. The request's parameter is of type [TextDocumentPosition]
(#TextDocumentPosition) the request response is of type [DocumentHighlight[]]
(#DocumentHighlight) or a Thenable that resolves to such."""
(#DocumentHighlight) or a Thenable that resolves to such.
"""
return await self.send_request("textDocument/documentHighlight", params)
async def document_symbol(
self, params: lsp_types.DocumentSymbolParams
) -> Union[
List["lsp_types.SymbolInformation"], List["lsp_types.DocumentSymbol"], None
]:
) -> list["lsp_types.SymbolInformation"] | list["lsp_types.DocumentSymbol"] | None:
"""A request to list all symbols found in a given text document. The request's
parameter is of type {@link TextDocumentIdentifier} the
response is of type {@link SymbolInformation SymbolInformation[]} or a Thenable
that resolves to such."""
that resolves to such.
"""
return await self.send_request("textDocument/documentSymbol", params)
async def code_action(
self, params: lsp_types.CodeActionParams
) -> Union[List[Union["lsp_types.Command", "lsp_types.CodeAction"]], None]:
async def code_action(self, params: lsp_types.CodeActionParams) -> list[Union["lsp_types.Command", "lsp_types.CodeAction"]] | None:
"""A request to provide commands for the given text document and range."""
return await self.send_request("textDocument/codeAction", params)
async def resolve_code_action(
self, params: lsp_types.CodeAction
) -> "lsp_types.CodeAction":
async def resolve_code_action(self, params: lsp_types.CodeAction) -> "lsp_types.CodeAction":
"""Request to resolve additional information for a given code action.The request's
parameter is of type {@link CodeAction} the response
is of type {@link CodeAction} or a Thenable that resolves to such."""
is of type {@link CodeAction} or a Thenable that resolves to such.
"""
return await self.send_request("codeAction/resolve", params)
async def workspace_symbol(
self, params: lsp_types.WorkspaceSymbolParams
) -> Union[
List["lsp_types.SymbolInformation"], List["lsp_types.WorkspaceSymbol"], None
]:
) -> list["lsp_types.SymbolInformation"] | list["lsp_types.WorkspaceSymbol"] | None:
"""A request to list project-wide symbols matching the query string given
by the {@link WorkspaceSymbolParams}. The response is
of type {@link SymbolInformation SymbolInformation[]} or a Thenable that
@@ -390,78 +363,60 @@ class LspRequest:
"""
return await self.send_request("workspace/symbol", params)
async def resolve_workspace_symbol(
self, params: lsp_types.WorkspaceSymbol
) -> "lsp_types.WorkspaceSymbol":
async def resolve_workspace_symbol(self, params: lsp_types.WorkspaceSymbol) -> "lsp_types.WorkspaceSymbol":
"""A request to resolve the range inside the workspace
symbol's location.
@since 3.17.0"""
@since 3.17.0
"""
return await self.send_request("workspaceSymbol/resolve", params)
async def code_lens(
self, params: lsp_types.CodeLensParams
) -> Union[List["lsp_types.CodeLens"], None]:
async def code_lens(self, params: lsp_types.CodeLensParams) -> list["lsp_types.CodeLens"] | None:
"""A request to provide code lens for the given text document."""
return await self.send_request("textDocument/codeLens", params)
async def resolve_code_lens(
self, params: lsp_types.CodeLens
) -> "lsp_types.CodeLens":
async def resolve_code_lens(self, params: lsp_types.CodeLens) -> "lsp_types.CodeLens":
"""A request to resolve a command for a given code lens."""
return await self.send_request("codeLens/resolve", params)
async def document_link(
self, params: lsp_types.DocumentLinkParams
) -> Union[List["lsp_types.DocumentLink"], None]:
async def document_link(self, params: lsp_types.DocumentLinkParams) -> list["lsp_types.DocumentLink"] | None:
"""A request to provide document links"""
return await self.send_request("textDocument/documentLink", params)
async def resolve_document_link(
self, params: lsp_types.DocumentLink
) -> "lsp_types.DocumentLink":
async def resolve_document_link(self, params: lsp_types.DocumentLink) -> "lsp_types.DocumentLink":
"""Request to resolve additional information for a given document link. The request's
parameter is of type {@link DocumentLink} the response
is of type {@link DocumentLink} or a Thenable that resolves to such."""
is of type {@link DocumentLink} or a Thenable that resolves to such.
"""
return await self.send_request("documentLink/resolve", params)
async def formatting(
self, params: lsp_types.DocumentFormattingParams
) -> Union[List["lsp_types.TextEdit"], None]:
async def formatting(self, params: lsp_types.DocumentFormattingParams) -> list["lsp_types.TextEdit"] | None:
"""A request to to format a whole document."""
return await self.send_request("textDocument/formatting", params)
async def range_formatting(
self, params: lsp_types.DocumentRangeFormattingParams
) -> Union[List["lsp_types.TextEdit"], None]:
async def range_formatting(self, params: lsp_types.DocumentRangeFormattingParams) -> list["lsp_types.TextEdit"] | None:
"""A request to to format a range in a document."""
return await self.send_request("textDocument/rangeFormatting", params)
async def on_type_formatting(
self, params: lsp_types.DocumentOnTypeFormattingParams
) -> Union[List["lsp_types.TextEdit"], None]:
async def on_type_formatting(self, params: lsp_types.DocumentOnTypeFormattingParams) -> list["lsp_types.TextEdit"] | None:
"""A request to format a document on type."""
return await self.send_request("textDocument/onTypeFormatting", params)
async def rename(
self, params: lsp_types.RenameParams
) -> Union["lsp_types.WorkspaceEdit", None]:
async def rename(self, params: lsp_types.RenameParams) -> Union["lsp_types.WorkspaceEdit", None]:
"""A request to rename a symbol."""
return await self.send_request("textDocument/rename", params)
async def prepare_rename(
self, params: lsp_types.PrepareRenameParams
) -> Union["lsp_types.PrepareRenameResult", None]:
async def prepare_rename(self, params: lsp_types.PrepareRenameParams) -> Union["lsp_types.PrepareRenameResult", None]:
"""A request to test and perform the setup necessary for a rename.
@since 3.16 - support for default behavior"""
@since 3.16 - support for default behavior
"""
return await self.send_request("textDocument/prepareRename", params)
async def execute_command(
self, params: lsp_types.ExecuteCommandParams
) -> Union["lsp_types.LSPAny", None]:
async def execute_command(self, params: lsp_types.ExecuteCommandParams) -> Union["lsp_types.LSPAny", None]:
"""A request send from the client to the server to execute a command. The request might return
a workspace edit which the client will apply to the workspace."""
a workspace edit which the client will apply to the workspace.
"""
return await self.send_request("workspace/executeCommand", params)
@@ -469,92 +424,87 @@ class LspNotification:
def __init__(self, send_notification):
self.send_notification = send_notification
def did_change_workspace_folders(
self, params: lsp_types.DidChangeWorkspaceFoldersParams
) -> None:
def did_change_workspace_folders(self, params: lsp_types.DidChangeWorkspaceFoldersParams) -> None:
"""The `workspace/didChangeWorkspaceFolders` notification is sent from the client to the server when the workspace
folder configuration changes."""
folder configuration changes.
"""
return self.send_notification("workspace/didChangeWorkspaceFolders", params)
def cancel_work_done_progress(
self, params: lsp_types.WorkDoneProgressCancelParams
) -> None:
def cancel_work_done_progress(self, params: lsp_types.WorkDoneProgressCancelParams) -> None:
"""The `window/workDoneProgress/cancel` notification is sent from the client to the server to cancel a progress
initiated on the server side."""
initiated on the server side.
"""
return self.send_notification("window/workDoneProgress/cancel", params)
def did_create_files(self, params: lsp_types.CreateFilesParams) -> None:
"""The did create files notification is sent from the client to the server when
files were created from within the client.
@since 3.16.0"""
@since 3.16.0
"""
return self.send_notification("workspace/didCreateFiles", params)
def did_rename_files(self, params: lsp_types.RenameFilesParams) -> None:
"""The did rename files notification is sent from the client to the server when
files were renamed from within the client.
@since 3.16.0"""
@since 3.16.0
"""
return self.send_notification("workspace/didRenameFiles", params)
def did_delete_files(self, params: lsp_types.DeleteFilesParams) -> None:
"""The will delete files request is sent from the client to the server before files are actually
deleted as long as the deletion is triggered from within the client.
@since 3.16.0"""
@since 3.16.0
"""
return self.send_notification("workspace/didDeleteFiles", params)
def did_open_notebook_document(
self, params: lsp_types.DidOpenNotebookDocumentParams
) -> None:
def did_open_notebook_document(self, params: lsp_types.DidOpenNotebookDocumentParams) -> None:
"""A notification sent when a notebook opens.
@since 3.17.0"""
@since 3.17.0
"""
return self.send_notification("notebookDocument/didOpen", params)
def did_change_notebook_document(
self, params: lsp_types.DidChangeNotebookDocumentParams
) -> None:
def did_change_notebook_document(self, params: lsp_types.DidChangeNotebookDocumentParams) -> None:
return self.send_notification("notebookDocument/didChange", params)
def did_save_notebook_document(
self, params: lsp_types.DidSaveNotebookDocumentParams
) -> None:
def did_save_notebook_document(self, params: lsp_types.DidSaveNotebookDocumentParams) -> None:
"""A notification sent when a notebook document is saved.
@since 3.17.0"""
@since 3.17.0
"""
return self.send_notification("notebookDocument/didSave", params)
def did_close_notebook_document(
self, params: lsp_types.DidCloseNotebookDocumentParams
) -> None:
def did_close_notebook_document(self, params: lsp_types.DidCloseNotebookDocumentParams) -> None:
"""A notification sent when a notebook closes.
@since 3.17.0"""
@since 3.17.0
"""
return self.send_notification("notebookDocument/didClose", params)
def initialized(self, params: lsp_types.InitializedParams) -> None:
"""The initialized notification is sent from the client to the
server after the client is fully initialized and the server
is allowed to send requests from the server to the client."""
is allowed to send requests from the server to the client.
"""
return self.send_notification("initialized", params)
def exit(self) -> None:
"""The exit event is sent from the client to the server to
ask the server to exit its process."""
ask the server to exit its process.
"""
return self.send_notification("exit")
def workspace_did_change_configuration(
self, params: lsp_types.DidChangeConfigurationParams
) -> None:
def workspace_did_change_configuration(self, params: lsp_types.DidChangeConfigurationParams) -> None:
"""The configuration change notification is sent from the client to the server
when the client's configuration has changed. The notification contains
the changed configuration as defined by the language client."""
the changed configuration as defined by the language client.
"""
return self.send_notification("workspace/didChangeConfiguration", params)
def did_open_text_document(
self, params: lsp_types.DidOpenTextDocumentParams
) -> None:
def did_open_text_document(self, params: lsp_types.DidOpenTextDocumentParams) -> None:
"""The document open notification is sent from the client to the server to signal
newly opened text documents. The document's truth is now managed by the client
and the server must not try to read the document's truth using the document's
@@ -562,47 +512,43 @@ class LspNotification:
mean that its content is presented in an editor. An open notification must not
be sent more than once without a corresponding close notification send before.
This means open and close notification must be balanced and the max open count
is one."""
is one.
"""
return self.send_notification("textDocument/didOpen", params)
def did_change_text_document(
self, params: lsp_types.DidChangeTextDocumentParams
) -> None:
def did_change_text_document(self, params: lsp_types.DidChangeTextDocumentParams) -> None:
"""The document change notification is sent from the client to the server to signal
changes to a text document."""
changes to a text document.
"""
return self.send_notification("textDocument/didChange", params)
def did_close_text_document(
self, params: lsp_types.DidCloseTextDocumentParams
) -> None:
def did_close_text_document(self, params: lsp_types.DidCloseTextDocumentParams) -> None:
"""The document close notification is sent from the client to the server when
the document got closed in the client. The document's truth now exists where
the document's uri points to (e.g. if the document's uri is a file uri the
truth now exists on disk). As with the open notification the close notification
is about managing the document's content. Receiving a close notification
doesn't mean that the document was open in an editor before. A close
notification requires a previous open notification to be sent."""
notification requires a previous open notification to be sent.
"""
return self.send_notification("textDocument/didClose", params)
def did_save_text_document(
self, params: lsp_types.DidSaveTextDocumentParams
) -> None:
def did_save_text_document(self, params: lsp_types.DidSaveTextDocumentParams) -> None:
"""The document save notification is sent from the client to the server when
the document got saved in the client."""
the document got saved in the client.
"""
return self.send_notification("textDocument/didSave", params)
def will_save_text_document(
self, params: lsp_types.WillSaveTextDocumentParams
) -> None:
def will_save_text_document(self, params: lsp_types.WillSaveTextDocumentParams) -> None:
"""A document will save notification is sent from the client to the server before
the document is actually saved."""
the document is actually saved.
"""
return self.send_notification("textDocument/willSave", params)
def did_change_watched_files(
self, params: lsp_types.DidChangeWatchedFilesParams
) -> None:
def did_change_watched_files(self, params: lsp_types.DidChangeWatchedFilesParams) -> None:
"""The watched files notification is sent from the client to the server when
the client detects changes to file watched by the language client."""
the client detects changes to file watched by the language client.
"""
return self.send_notification("workspace/didChangeWatchedFiles", params)
def set_trace(self, params: lsp_types.SetTraceParams) -> None:
src/multilspy/lsp_protocol_handler/lsp_types.py → src/solidlsp/lsp_protocol_handler/lsp_types.py
+458 -409
View File
File diff suppressed because it is too large Load Diff
src/solidlsp/lsp_protocol_handler/server.py
+122
View File
@@ -0,0 +1,122 @@
"""
This file provides the implementation of the JSON-RPC client, that launches and
communicates with the language server.
The initial implementation of this file was obtained from
https://github.com/predragnikolic/OLSP under the MIT License with the following terms:
MIT License
Copyright (c) 2023 Предраг Николић
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""
import dataclasses
import json
import logging
import os
from typing import Any, Union
from .lsp_types import ErrorCodes
StringDict = dict[str, Any]
PayloadLike = Union[list[StringDict], StringDict, None]
CONTENT_LENGTH = "Content-Length: "
ENCODING = "utf-8"
log = logging.getLogger(__name__)
@dataclasses.dataclass
class ProcessLaunchInfo:
"""
This class is used to store the information required to launch a process.
"""
# The command to launch the process
cmd: str
# The environment variables to set for the process
env: dict[str, str] = dataclasses.field(default_factory=dict)
# The working directory for the process
cwd: str = os.getcwd()
class Error(Exception):
def __init__(self, code: ErrorCodes, message: str) -> None:
super().__init__(message)
self.code = code
def to_lsp(self) -> StringDict:
return {"code": self.code, "message": super().__str__()}
@classmethod
def from_lsp(cls, d: StringDict) -> "Error":
return Error(d["code"], d["message"])
def __str__(self) -> str:
return f"{super().__str__()} ({self.code})"
def make_response(request_id: Any, params: PayloadLike) -> StringDict:
return {"jsonrpc": "2.0", "id": request_id, "result": params}
def make_error_response(request_id: Any, err: Error) -> StringDict:
return {"jsonrpc": "2.0", "id": request_id, "error": err.to_lsp()}
def make_notification(method: str, params: PayloadLike) -> StringDict:
return {"jsonrpc": "2.0", "method": method, "params": params}
def make_request(method: str, request_id: Any, params: PayloadLike) -> StringDict:
return {"jsonrpc": "2.0", "method": method, "id": request_id, "params": params}
class StopLoopException(Exception):
pass
def create_message(payload: PayloadLike):
body = json.dumps(payload, check_circular=False, ensure_ascii=False, separators=(",", ":")).encode(ENCODING)
return (
f"Content-Length: {len(body)}\r\n".encode(ENCODING),
"Content-Type: application/vscode-jsonrpc; charset=utf-8\r\n\r\n".encode(ENCODING),
body,
)
class MessageType:
error = 1
warning = 2
info = 3
log = 4
def content_length(line: bytes) -> int | None:
if line.startswith(b"Content-Length: "):
_, value = line.split(b"Content-Length: ")
value = value.strip()
try:
return int(value)
except ValueError:
raise ValueError(f"Invalid Content-Length header: {value}")
return None
src/multilspy/multilspy_settings.py → src/solidlsp/settings.py
+3 -5
View File
@@ -1,14 +1,12 @@
"""
Defines the settings for multilspy.
Defines settings for Solid-LSP
"""
import os
import pathlib
class MultilspySettings:
"""
Provides the various settings for multilspy.
"""
class SolidLSPSettings:
@staticmethod
def get_language_server_directory() -> str:
"""Returns the directory for language servers"""
test/conftest.py
+25 -8
View File
@@ -1,10 +1,15 @@
import logging
from pathlib import Path
import pytest
from sensai.util.logging import configure
from multilspy.language_server import SyncLanguageServer
from multilspy.multilspy_config import Language, MultilspyConfig
from multilspy.multilspy_logger import MultilspyLogger
from serena.util.file_system import GitignoreParser
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls_config import Language, LanguageServerConfig
from solidlsp.ls_logger import LanguageServerLogger
configure(level=logging.DEBUG)
@pytest.fixture(scope="session")
@@ -22,13 +27,25 @@ def get_repo_path(language: Language) -> Path:
return Path(__file__).parent / "resources" / "repos" / language / "test_repo"
def create_ls(language: Language, repo_path: str):
config = MultilspyConfig(code_language=language)
logger = MultilspyLogger()
return SyncLanguageServer.create(config, logger, repo_path)
def create_ls(
language: Language,
repo_path: str | None = None,
ignored_paths: list[str] | None = None,
trace_lsp_communication: bool = False,
log_level: int = logging.INFO,
) -> SolidLanguageServer:
ignored_paths = ignored_paths or []
if repo_path is None:
repo_path = str(get_repo_path(language))
gitignore_parser = GitignoreParser(str(repo_path))
for spec in gitignore_parser.get_ignore_specs():
ignored_paths.extend(spec.patterns)
config = LanguageServerConfig(code_language=language, ignored_paths=ignored_paths, trace_lsp_communication=trace_lsp_communication)
logger = LanguageServerLogger(log_level=log_level)
return SolidLanguageServer.create(config, logger, repo_path)
def create_default_ls(language: Language) -> SyncLanguageServer:
def create_default_ls(language: Language) -> SolidLanguageServer:
repo_path = str(get_repo_path(language))
return create_ls(language, repo_path)
test/resources/repos/java/test_repo/pom.xml
+17 -2
View File
@@ -8,7 +8,22 @@
<packaging>jar</packaging>
<name>Java Test Repo</name>
<properties>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
<maven.compiler.plugin.version>3.13.0</maven.compiler.plugin.version>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.plugin.version}</version>
<configuration>
<source>21</source>
<target>21</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
test/resources/repos/python/test_repo/ignore_this_dir_with_postfix/ignored_module.py
+141
View File
@@ -0,0 +1,141 @@
"""
Example demonstrating user management with the test_repo module.
This example showcases:
- Creating and managing users
- Using various object types and relationships
- Type annotations and complex Python patterns
"""
import logging
from dataclasses import dataclass
from typing import Any
from test_repo.models import User, create_user_object
from test_repo.services import UserService
# Set up logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
@dataclass
class UserStats:
"""Statistics about user activity."""
user_id: str
login_count: int = 0
last_active_days: int = 0
engagement_score: float = 0.0
def is_active(self) -> bool:
"""Check if the user is considered active."""
return self.last_active_days < 30
class UserManager:
"""Example class demonstrating complex user management."""
def __init__(self, service: UserService):
self.service = service
self.active_users: dict[str, User] = {}
self.user_stats: dict[str, UserStats] = {}
def register_user(self, name: str, email: str, roles: list[str] | None = None) -> User:
"""Register a new user."""
logger.info(f"Registering new user: {name} ({email})")
user = self.service.create_user(name=name, email=email, roles=roles)
self.active_users[user.id] = user
self.user_stats[user.id] = UserStats(user_id=user.id)
return user
def get_user(self, user_id: str) -> User | None:
"""Get a user by ID."""
if user_id in self.active_users:
return self.active_users[user_id]
# Try to fetch from service
user = self.service.get_user(user_id)
if user:
self.active_users[user.id] = user
return user
def update_user_stats(self, user_id: str, login_count: int, days_since_active: int) -> None:
"""Update statistics for a user."""
if user_id not in self.user_stats:
self.user_stats[user_id] = UserStats(user_id=user_id)
stats = self.user_stats[user_id]
stats.login_count = login_count
stats.last_active_days = days_since_active
# Calculate engagement score based on activity
engagement = (100 - min(days_since_active, 100)) * 0.8
engagement += min(login_count, 20) * 0.2
stats.engagement_score = engagement
def get_active_users(self) -> list[User]:
"""Get all active users."""
active_user_ids = [user_id for user_id, stats in self.user_stats.items() if stats.is_active()]
return [self.active_users[user_id] for user_id in active_user_ids if user_id in self.active_users]
def get_user_by_email(self, email: str) -> User | None:
"""Find a user by their email address."""
for user in self.active_users.values():
if user.email == email:
return user
return None
# Example function demonstrating type annotations
def process_user_data(users: list[User], include_inactive: bool = False, transform_func: callable | None = None) -> dict[str, Any]:
"""Process user data with optional transformations."""
result: dict[str, Any] = {"users": [], "total": 0, "admin_count": 0}
for user in users:
if transform_func:
user_data = transform_func(user.to_dict())
else:
user_data = user.to_dict()
result["users"].append(user_data)
result["total"] += 1
if "admin" in user.roles:
result["admin_count"] += 1
return result
def main():
"""Main function demonstrating the usage of UserManager."""
# Initialize service and manager
service = UserService()
manager = UserManager(service)
# Register some users
admin = manager.register_user("Admin User", "admin@example.com", ["admin"])
user1 = manager.register_user("Regular User", "user@example.com", ["user"])
user2 = manager.register_user("Another User", "another@example.com", ["user"])
# Update some stats
manager.update_user_stats(admin.id, 100, 5)
manager.update_user_stats(user1.id, 50, 10)
manager.update_user_stats(user2.id, 10, 45) # Inactive user
# Get active users
active_users = manager.get_active_users()
logger.info(f"Active users: {len(active_users)}")
# Process user data
user_data = process_user_data(active_users, transform_func=lambda u: {**u, "full_name": u.get("name", "")})
logger.info(f"Processed {user_data['total']} users, {user_data['admin_count']} admins")
# Example of calling create_user directly
external_user = create_user_object(id="ext123", name="External User", email="external@example.org", roles=["external"])
logger.info(f"Created external user: {external_user.name}")
if __name__ == "__main__":
main()
test/serena/__snapshots__/test_symbol_editing.ambr
+925 -713
View File
File diff suppressed because it is too large Load Diff
test/serena/test_make_tool_process_isolation.py
+162
View File
@@ -0,0 +1,162 @@
"""Tests for make_tool consistency between regular tools and ProcessIsolatedTool."""
import pytest
from mcp.server.fastmcp.tools.base import Tool as MCPTool
from serena.agent import SerenaAgent, ToolRegistry
from serena.mcp import SerenaMCPFactory
from serena.process_isolated_agent import ProcessIsolatedSerenaAgent, ProcessIsolatedTool
from test.serena.test_serena_agent import SerenaConfigForTests
make_tool = SerenaMCPFactory.make_mcp_tool
@pytest.fixture
def in_memory_config():
"""Create an in-memory configuration for tests."""
return SerenaConfigForTests()
@pytest.fixture
def regular_agent(in_memory_config):
"""Create a regular SerenaAgent for comparison."""
return SerenaAgent(serena_config=in_memory_config)
@pytest.fixture
def process_isolated_agent(in_memory_config):
"""Create a ProcessIsolatedSerenaAgent for comparison."""
agent = ProcessIsolatedSerenaAgent(serena_config=in_memory_config)
agent.start()
yield agent
agent.stop()
class TestMakeToolProcessIsolation:
"""Test that make_tool produces identical metadata for regular and process-isolated tools."""
@pytest.mark.parametrize("tool_name", ToolRegistry.get_tool_names())
def test_make_tool_metadata_consistency(
self, tool_name: str, regular_agent: SerenaAgent, process_isolated_agent: ProcessIsolatedSerenaAgent
):
"""Test that make_tool produces identical metadata for regular and process-isolated tools."""
# Get regular tool instance
tool_class = ToolRegistry.get_tool_class_by_name(tool_name)
regular_tool = regular_agent.get_tool(tool_class)
# Get ProcessIsolatedTool instance
isolated_tool = ProcessIsolatedTool(process_isolated_agent, tool_name)
# Create MCP tools from both
regular_mcp_tool = make_tool(regular_tool)
isolated_mcp_tool = make_tool(isolated_tool)
# Verify both are MCPTool instances
assert isinstance(regular_mcp_tool, MCPTool)
assert isinstance(isolated_mcp_tool, MCPTool)
# Test name consistency
assert regular_mcp_tool.name == isolated_mcp_tool.name
assert regular_mcp_tool.name == tool_name
# Test description consistency
assert regular_mcp_tool.description == isolated_mcp_tool.description, (
f"Tool {tool_name}: descriptions differ\n"
f"Regular: {regular_mcp_tool.description}\n"
f"Isolated: {isolated_mcp_tool.description}"
)
# Test parameters schema consistency
assert regular_mcp_tool.parameters == isolated_mcp_tool.parameters, (
f"Tool {tool_name}: parameter schemas differ\n"
f"Regular: {regular_mcp_tool.parameters}\n"
f"Isolated: {isolated_mcp_tool.parameters}"
)
# Test function metadata consistency (compare schemas, not class objects)
regular_schema = regular_mcp_tool.fn_metadata.arg_model.model_json_schema()
isolated_schema = isolated_mcp_tool.fn_metadata.arg_model.model_json_schema()
assert (
regular_schema == isolated_schema
), f"Tool {tool_name}: function metadata schemas differ\nRegular: {regular_schema}\nIsolated: {isolated_schema}"
# Test async flag consistency
assert regular_mcp_tool.is_async == isolated_mcp_tool.is_async
# Test context kwarg consistency
assert regular_mcp_tool.context_kwarg == isolated_mcp_tool.context_kwarg
@pytest.mark.parametrize("tool_name", ToolRegistry.get_tool_names()[:5]) # Test first 5 tools for faster execution
def test_tool_protocol_methods_consistency(
self, tool_name: str, regular_agent: SerenaAgent, process_isolated_agent: ProcessIsolatedSerenaAgent
):
"""Test that Tool methods return identical results for regular and process-isolated tools."""
# Get regular tool instance
tool_class = ToolRegistry.get_tool_class_by_name(tool_name)
regular_tool = regular_agent.get_tool(tool_class)
# Get ProcessIsolatedTool instance
isolated_tool = ProcessIsolatedTool(process_isolated_agent, tool_name)
# Test get_name()
assert regular_tool.get_name_from_cls() == isolated_tool.get_name()
assert regular_tool.get_name_from_cls() == tool_name
# Test get_apply_docstring()
regular_docstring = regular_tool.get_apply_docstring()
isolated_docstring = isolated_tool.get_apply_docstring()
assert (
regular_docstring == isolated_docstring
), f"Tool {tool_name}: docstrings differ\nRegular: {regular_docstring}\nIsolated: {isolated_docstring}"
# Test get_apply_fn_metadata()
regular_metadata = regular_tool.get_apply_fn_metadata()
isolated_metadata = isolated_tool.get_apply_fn_metadata()
# Compare metadata properties (compare schemas, not class objects)
regular_schema = regular_metadata.arg_model.model_json_schema()
isolated_schema = isolated_metadata.arg_model.model_json_schema()
assert (
regular_schema == isolated_schema
), f"Tool {tool_name}: metadata schemas differ\nRegular: {regular_schema}\nIsolated: {isolated_schema}"
def test_process_isolated_tool_uses_tool_registry(self, process_isolated_agent: ProcessIsolatedSerenaAgent):
"""Test that ProcessIsolatedTool correctly uses ToolRegistry for metadata."""
tool_name = ToolRegistry.get_tool_names()[0] # Use first available tool
isolated_tool = ProcessIsolatedTool(process_isolated_agent, tool_name)
# Verify that the tool uses ToolRegistry
assert isolated_tool._tool_class == ToolRegistry.get_tool_class_by_name(tool_name)
# Verify that metadata comes from the tool class
expected_docstring = isolated_tool._tool_class.get_apply_docstring_from_cls()
expected_metadata = isolated_tool._tool_class.get_apply_fn_metadata_from_cls()
assert isolated_tool.get_apply_docstring() == expected_docstring
# Compare schemas, not class objects
isolated_schema = isolated_tool.get_apply_fn_metadata().arg_model.model_json_schema()
expected_schema = expected_metadata.arg_model.model_json_schema()
assert isolated_schema == expected_schema
def test_tool_registry_completeness(self, regular_agent: SerenaAgent, process_isolated_agent: ProcessIsolatedSerenaAgent):
"""Test that all tools are available in both agents and the registry."""
# Get tool names from both agents
regular_active_tools = set(regular_agent.get_active_tool_names())
regular_all_tools = set(tool.get_name_from_cls() for tool in regular_agent.get_exposed_tool_instances())
isolated_tool_names = set(process_isolated_agent.get_exposed_tool_names())
# The process isolated agent should have all tools (exposed, not just active)
assert regular_all_tools == isolated_tool_names, (
f"Tool sets differ:\n"
f"Regular exposed only: {regular_all_tools - isolated_tool_names}\n"
f"Isolated only: {isolated_tool_names - regular_all_tools}"
)
# Active tools should be a subset of all tools
assert regular_active_tools.issubset(
regular_all_tools
), f"Some active tools not in exposed tools: {regular_active_tools - regular_all_tools}"
# All tools should be in the registry
registry_tool_names = set(ToolRegistry.get_tool_names())
assert regular_all_tools.issubset(registry_tool_names), f"Some tools not in registry: {regular_all_tools - registry_tool_names}"
test/serena/test_mcp.py
+5 -3
View File
@@ -4,7 +4,9 @@ import pytest
from mcp.server.fastmcp.tools.base import Tool as MCPTool
from serena.agent import Tool, ToolRegistry
from serena.mcp import make_tool
from serena.mcp import SerenaMCPFactory
make_tool = SerenaMCPFactory.make_mcp_tool
class BaseMockTool(Tool):
@@ -224,7 +226,7 @@ def test_make_tool_missing_apply() -> None:
""",
"",
),
("", ""),
("Description without params.", "Description without params."),
],
)
def test_make_tool_descriptions(docstring, expected_description) -> None:
@@ -287,7 +289,7 @@ def test_make_tool_all_tools(tool_class) -> None:
# Basic validation
assert isinstance(mcp_tool, MCPTool)
assert mcp_tool.name == tool_class.get_name()
assert mcp_tool.name == tool_class.get_name_from_cls()
# The description should be a string (either from docstring or default)
assert isinstance(mcp_tool.description, str)
test/serena/test_serena_agent.py
+91 -21
View File
@@ -5,9 +5,10 @@ from dataclasses import dataclass
import pytest
from multilspy.multilspy_config import Language
from serena.agent import FindReferencingSymbolsTool, FindSymbolTool, SerenaAgent, SerenaConfigBase
from test.conftest import LanguageParamRequest, get_repo_path
from serena.agent import FindReferencingSymbolsTool, FindSymbolTool, Project, ProjectConfig, SerenaAgent, SerenaConfigBase
from serena.process_isolated_agent import ProcessIsolatedSerenaAgent
from solidlsp.ls_config import Language
from test.conftest import get_repo_path
@dataclass
@@ -19,20 +20,71 @@ class SerenaConfigForTests(SerenaConfigBase):
gui_log_window_enabled: bool = False
web_dashboard: bool = False
def __post_init__(self):
# Initialize with empty projects list if not already set
if not hasattr(self, "projects") or self.projects is None:
self.projects = []
@pytest.fixture
def serena_config():
return SerenaConfigForTests()
"""Create an in-memory configuration for tests with test repositories pre-registered."""
# Create test projects for all supported languages
test_projects = []
for language in [Language.PYTHON, Language.GO, Language.JAVA, Language.RUST, Language.TYPESCRIPT, Language.PHP]:
repo_path = get_repo_path(language)
if repo_path.exists():
project_name = f"test_repo_{language}"
project = Project(
project_root=str(repo_path),
project_config=ProjectConfig(
project_name=project_name,
language=language,
ignored_paths=[],
excluded_tools=set(),
read_only=False,
ignore_all_files_in_gitignore=True,
initial_prompt="",
encoding="utf-8",
),
)
test_projects.append(project)
config = SerenaConfigForTests()
config.projects = test_projects
return config
@pytest.fixture
def serena_agent(request: LanguageParamRequest, serena_config) -> SerenaAgent:
def serena_agent(request: pytest.FixtureRequest, serena_config):
language = Language(request.param)
repo = get_repo_path(language)
return SerenaAgent(project=str(repo), serena_config=serena_config)
project_name = f"test_repo_{language}"
# Check if this test should use process isolation by looking at the test parameters
isolated_process = False
if hasattr(request, "node") and hasattr(request.node, "callspec"):
# Get the isolated_process parameter value from the test
params = request.node.callspec.params
isolated_process = params.get("isolated_process", False)
if isolated_process:
agent = ProcessIsolatedSerenaAgent(project=project_name, serena_config=serena_config)
agent.start()
# Add cleanup to stop the process
def cleanup():
agent.stop()
request.addfinalizer(cleanup)
return agent
else:
return SerenaAgent(project=project_name, serena_config=serena_config)
class TestSerenaAgent:
@pytest.mark.parametrize(
"isolated_process", [pytest.param(False, id="direct"), pytest.param(True, id="isolated", marks=pytest.mark.isolated_process)]
)
@pytest.mark.parametrize(
"serena_agent,symbol_name,expected_kind,expected_file",
[
@@ -45,16 +97,20 @@ class TestSerenaAgent:
],
indirect=["serena_agent"],
)
def test_find_symbol(self, serena_agent: SerenaAgent, symbol_name: str, expected_kind: str, expected_file: str):
def test_find_symbol(self, serena_agent, symbol_name: str, expected_kind: str, expected_file: str, isolated_process: bool):
agent = serena_agent
find_symbol_tool = agent.get_tool(FindSymbolTool)
result = find_symbol_tool.apply(symbol_name)
result = find_symbol_tool.apply_ex(name_path=symbol_name)
symbols = json.loads(result)
assert any(
symbol_name in s["name_path"] and expected_kind.lower() in s["kind"].lower() and expected_file in s["relative_path"]
for s in symbols
), f"Expected to find {symbol_name} ({expected_kind}) in {expected_file} for {agent.get_active_project().language.name}"
), f"Expected to find {symbol_name} ({expected_kind}) in {expected_file}"
@pytest.mark.parametrize(
"isolated_process", [pytest.param(False, id="direct"), pytest.param(True, id="isolated", marks=pytest.mark.isolated_process)]
)
@pytest.mark.parametrize(
"serena_agent,symbol_name,def_file,ref_file",
[
@@ -79,23 +135,30 @@ class TestSerenaAgent:
],
indirect=["serena_agent"],
)
def test_find_symbol_references(self, serena_agent: SerenaAgent, symbol_name: str, def_file: str, ref_file: str) -> None:
def test_find_symbol_references(self, serena_agent, symbol_name: str, def_file: str, ref_file: str, isolated_process: bool) -> None:
agent = serena_agent
# Find the symbol location first
find_symbol_tool = agent.get_tool(FindSymbolTool)
result = find_symbol_tool.apply(symbol_name, relative_path=def_file)
result = find_symbol_tool.apply_ex(name_path=symbol_name, relative_path=def_file)
time.sleep(1)
symbols = json.loads(result)
# Find the definition
def_symbol = symbols[0]
# Now find references
find_refs_tool = agent.get_tool(FindReferencingSymbolsTool)
result = find_refs_tool.apply(name_path=def_symbol["name_path"], relative_path=def_symbol["relative_path"])
result = find_refs_tool.apply_ex(name_path=def_symbol["name_path"], relative_path=def_symbol["relative_path"])
refs = json.loads(result)
assert any(
ref["relative_path"] == ref_file for ref in refs
), f"Expected to find reference to {symbol_name} in {ref_file} for {agent._active_project.language.name}. refs={refs}"
), f"Expected to find reference to {symbol_name} in {ref_file}. refs={refs}"
@pytest.mark.parametrize(
"isolated_process", [pytest.param(False, id="direct"), pytest.param(True, id="isolated", marks=pytest.mark.isolated_process)]
)
@pytest.mark.parametrize(
"serena_agent,name_path,substring_matching,expected_symbol_name,expected_kind,expected_file",
[
@@ -164,16 +227,18 @@ class TestSerenaAgent:
)
def test_find_symbol_name_path(
self,
serena_agent: SerenaAgent,
serena_agent,
name_path: str,
substring_matching: bool,
expected_symbol_name: str,
expected_kind: str,
expected_file: str,
isolated_process: bool,
):
agent = serena_agent
find_symbol_tool = agent.get_tool(FindSymbolTool)
result = find_symbol_tool.apply(
result = find_symbol_tool.apply_ex(
name_path=name_path,
depth=0,
relative_path=None,
@@ -182,6 +247,7 @@ class TestSerenaAgent:
exclude_kinds=None,
substring_matching=substring_matching,
)
symbols = json.loads(result)
assert any(
expected_symbol_name == s["name_path"].split("/")[-1]
@@ -190,6 +256,9 @@ class TestSerenaAgent:
for s in symbols
), f"Expected to find {name_path} ({expected_kind}) in {expected_file} for {agent._active_project.language.name}. Symbols: {symbols}"
@pytest.mark.parametrize(
"isolated_process", [pytest.param(False, id="direct"), pytest.param(True, id="isolated", marks=pytest.mark.isolated_process)]
)
@pytest.mark.parametrize(
"serena_agent,name_path",
[
@@ -210,17 +279,18 @@ class TestSerenaAgent:
)
def test_find_symbol_name_path_no_match(
self,
serena_agent: SerenaAgent,
serena_agent,
name_path: str,
isolated_process: bool,
):
agent = serena_agent
find_symbol_tool = agent.get_tool(FindSymbolTool)
result = find_symbol_tool.apply(
result = find_symbol_tool.apply_ex(
name_path=name_path,
depth=0,
substring_matching=True,
)
symbols = json.loads(result)
assert (
not symbols
), f"Expected to find no symbols for {name_path} for {agent._active_project.language.name}. Symbols found: {symbols}"
assert not symbols, f"Expected to find no symbols for {name_path}. Symbols found: {symbols}"

Some files were not shown because too many files have changed in this diff Show More