Should Pygls Provide More Diagnostics Support Utilties?

Jun 15, 2025 by ADMIN 56 views

Should pygls Provide More Diagnostics Support Utilities? A Deep Dive

In the realm of Language Server Protocol (LSP) implementations, the robustness and efficiency of diagnostic handling are paramount. During my work on a language server, I encountered a situation where quick fixes were unavailable in the KDE editor kate. This issue stemmed from kate not providing diagnostics in the context parameter of the code action request. My server relied on this field, along with the diagnostic.data field, rather than recomputing diagnostics at the time of the code action request. This experience highlighted a potential gap in the diagnostic support utilities offered by pygls, prompting the question: Should pygls provide more built-in tools for managing diagnostics?

The Initial Problem: Missing Quick Fixes

The core issue arose because my diagnostic code inherently needed to determine valid values to generate diagnostics. Adhering to the principle of "Don't Repeat Yourself" (DRY), I stored valid solutions within the data attribute of the diagnostic, as these had already been computed. This approach, while efficient in avoiding redundant calculations, exposed a vulnerability when the client (in this case, kate) did not provide the necessary diagnostic context in the code action request.

The Role of `diagnostic.data`

The diagnostic.data field is a powerful mechanism for carrying additional information alongside a diagnostic message. This can be particularly useful for storing pre-computed solutions or relevant context, allowing for more efficient code action generation. However, the reliance on data becomes problematic when the client does not preserve this field or when the server needs to recompute information due to the client's capabilities.

The Behavior of `kate`

Ideally, kate should have included the relevant diagnostics in the code action request. However, even if it had, the data field would not have been preserved due to the client's capabilities. This would have still necessitated recomputation of the solutions, defeating the purpose of caching them in the first place. This experience underscored the importance of designing language servers that are resilient to varying client capabilities and behaviors.

The LSP Specification and Diagnostic Context

The Language Server Protocol specification states that "The primary parameter to compute code actions is the provided range." This implies that servers should not solely rely on the context.diagnostics field for corrections. This guidance emphasizes the need for language servers to be able to independently determine diagnostics based on the given range, rather than depending on the client to provide them. This is a critical consideration for ensuring consistent behavior across different LSP clients.

Decoupling from Client-Provided Diagnostics

By decoupling the code action generation from client-provided diagnostics, language servers can achieve greater robustness and portability. This approach ensures that quick fixes and other code actions are available even when the client does not provide comprehensive diagnostic information. This decoupling also aligns with the LSP's intention of making servers more independent and self-sufficient.

The Challenge of Recomputation

Recomputing diagnostics for code action requests can be computationally expensive, particularly for complex languages or large codebases. This can lead to noticeable delays in providing quick fixes, which can negatively impact the user experience. Therefore, efficient caching and retrieval mechanisms are crucial for maintaining responsiveness.

The Solution: Caching Diagnostics Internally

To address this issue, I implemented a caching mechanism within my language server. This involved creating a mapping of positions to an iterable of diagnostics, enabling the server to efficiently find diagnostics overlapping a given range. This solution allowed the language server to generate code actions even when the client lacked dataSupport or did not provide diagnostics in the code action request. This highlights the importance of server-side diagnostic management for robust LSP implementations.

The Benefits of Caching

The benefits of caching diagnostics are manifold. First and foremost, it avoids redundant computations, leading to improved performance and responsiveness. Second, it enables the server to function correctly even when the client's diagnostic support is limited. Third, it provides a consistent and reliable source of diagnostic information, regardless of the client's behavior. This caching strategy is a cornerstone of building resilient and efficient language servers.

The Importance of Efficient Retrieval

The effectiveness of a diagnostic cache hinges on the efficiency of its retrieval mechanism. The ability to quickly identify diagnostics within a given range is crucial for minimizing latency and ensuring a smooth user experience. This necessitates the use of appropriate data structures and algorithms for indexing and searching diagnostics.

A Potential Building Block for `pygls`

Considering the challenges and solutions outlined above, I believe that pygls could benefit from providing a building block for diagnostic management. This could take the form of an importable module that language servers can optionally use to cache and retrieve diagnostics efficiently. Such a utility would empower language server developers to create more robust and client-agnostic LSP implementations. This is particularly crucial in the diverse ecosystem of LSP clients, each with varying levels of support for advanced features.

The Core Idea: `DiagnosticRangeHelper`

To illustrate this concept, I've provided a code snippet of a DiagnosticRangeHelper class that I use in my language server. This class efficiently stores and retrieves diagnostics based on their range, allowing for quick identification of diagnostics overlapping a given position. The code leverages binary search techniques to optimize the retrieval process, making it suitable for large diagnostic sets.

Code Example: `DiagnosticRangeHelper`

The DiagnosticRangeHelper class, along with its helper functions, provides a foundation for efficient diagnostic management. Let's examine the code:

class DiagnosticRangeHelper:
__slots__ = (&quot;diagnostics&quot;, &quot;by_start_index&quot;, &quot;by_end_index&quot;)

def __init__(self, diagnostics: Sequence[types.Diagnostic]) -&gt; None:
    self.diagnostics = diagnostics
    self.by_start_index = sorted(
        (
            (_pos_as_tuple(diagnostics[i].range.start), i)
            for i in range(len(diagnostics))
        ),
    )
    self.by_end_index = sorted(
        (
            (_pos_as_tuple(diagnostics[i].range.end), i)
            for i in range(len(diagnostics))
        ),
    )

def diagnostics_in_range(self, text_range: types.Range) -&gt; List[types.Diagnostic]:
    start_pos = _pos_as_tuple(text_range.start)
    end_pos = _pos_as_tuple(text_range.end)

    try:
        lower_index_limit = _find_gt(
            self.by_end_index,
            start_pos,
            key=lambda t: t[0],
        )[1]
    except NoSuchElementError:
        lower_index_limit = len(self.diagnostics)

    try:
        upper_index_limit = _find_lt(
            self.by_start_index,
            end_pos,
            key=lambda t: t[0],
        )[1]

        upper_index_limit += 1
    except NoSuchElementError:
        upper_index_limit = 0

    return self.diagnostics[lower_index_limit:upper_index_limit]

class NoSuchElementError(ValueError):
pass
def _find_lt(a: Sequence[Any], x: Any, *, key: Any = None):
"""Find rightmost value less than x"""
i = bisect_left(a, x, key=key)
if i:
return a[i - 1]
raise NoSuchElementError
def _find_gt(a: Sequence[Any], x: Any, *, key: Any = None):
"""Find leftmost value greater than x"""
i = bisect_right(a, x, key=key)
if i != len(a):
return a[i]
raise NoSuchElementError
def _pos_as_tuple(pos: types.Position) -> Tuple[int, int]:
return pos.line, pos.character

This code efficiently retrieves diagnostics within a specified range by leveraging sorted indices and binary search. The _find_lt and _find_gt functions, adapted from the Python bisect module, are crucial for the binary search implementation. This approach ensures that the retrieval process remains efficient even with a large number of diagnostics. This example showcases a practical solution for managing diagnostics in an LSP server.

Usage Pattern

The typical usage pattern involves creating a new instance of DiagnosticRangeHelper with a list of diagnostics just before publishing them. This promotes an immutable approach, but the class could be easily adapted for mutable scenarios if that better suits the pygls API. During a code action request, the range is passed to the diagnostics_in_range method, which returns a list of overlapping diagnostics. This pattern provides a clear and concise way to manage diagnostics in an LSP server.

Client vs. Server Encoded Ranges

In my implementation, I consistently used client-encoded ranges. However, the code would function equally well with server-encoded ranges, as long as consistency is maintained. This flexibility allows developers to choose the encoding that best fits their codebase. This highlights the adaptability of the DiagnosticRangeHelper to different range encoding schemes.

The Benefits of an Opt-In Building Block

Providing this functionality as an opt-in building block would allow language server developers to choose whether or not to use it. This aligns with the pygls philosophy of providing tools and libraries that can be used as needed, without imposing unnecessary dependencies. This approach gives developers the flexibility to tailor their diagnostic management strategies to the specific needs of their language servers.

Addressing Client Limitations

The key advantage of this approach is that it provides reliable diagnostic support even for clients that do not provide diagnostics in the code action request or lack dataSupport for diagnostics. This ensures a consistent and robust experience across a wider range of LSP clients. This is a critical consideration for creating language servers that are truly client-agnostic.

Enhancing Code Action Reliability

By providing a mechanism for efficient diagnostic retrieval, pygls can help language server developers create more reliable and responsive code actions. This can significantly improve the user experience, particularly in scenarios where quick fixes and other code actions are heavily relied upon. This enhancement to code action reliability is a key benefit of improved diagnostic support.

Conclusion: A Proposal for Enhanced Diagnostic Support

In conclusion, the experience of developing a language server and encountering limitations with client-provided diagnostics has highlighted the need for robust server-side diagnostic management. The DiagnosticRangeHelper class provides a concrete example of how this can be achieved efficiently. I believe that incorporating such a utility into pygls as an opt-in building block would be a valuable addition, empowering language server developers to create more resilient and client-agnostic LSP implementations. This would ultimately lead to a better experience for users of language servers, regardless of the client they are using.

I welcome feedback and discussion on this proposal. If there is interest, I am happy to contribute further to this effort. If not, that is perfectly acceptable, and this issue can be closed. The goal is to explore ways to enhance the pygls ecosystem and make it even more powerful for language server development.

Improved Reliability: Ensures code actions are available even when clients lack diagnostic support.
Enhanced Efficiency: Caches diagnostics to avoid redundant computations.
Client-Agnostic Design: Works consistently across different LSP clients.
Simplified Development: Provides a building block for diagnostic management.

By addressing the challenges of client limitations and the computational cost of recomputing diagnostics, this proposal aims to elevate the quality and robustness of language servers built with pygls. This proactive approach to diagnostic support is essential for creating a thriving LSP ecosystem.