Creating the SyntaxDocumentation Command - Part 2: OCaml LSP

In the first part of this series, Creating the SyntaxDocumentation Command - Part 1: Merlin, we explored how to create a new command in Merlin, particularly the SyntaxDocumentation command. In this continuation, we will be looking at the amazing OCaml LSP project and how we have integrated our SyntaxDocumentation command into it. OCaml LSP is a broad and complex project, so we will be limiting the scope of this article just to what's relevant for the SyntaxDocumentation command.

Language Server Protocol

The Language Server Protocol (LSP) defines the protocol used between an editor or IDE and a language server that provides language features like auto complete, go to definition, find all references, etc. In turn, the protocol defines the format of the messages sent using JSON-RPC between the development tool and the language server. With LSP, a single language server can be used with multiple development tools, such as:

• Integrated Development Environments (IDEs): Visual Studio Code, Atom, or IntelliJ IDEA
• Code editors: Sublime Text, Vim, or Emacs
• Text editors with code-related features
• Command-line tools for code management, building, or testing

How LSP Works

Here's a typical interaction between a development tool and a language server:

1. Document Opened: When the user opens a document, this notifies the language server that a document is open (textDocument/didOpen).
2. Editing: When the user edits the document, this notifies the server about the changes (textDocument/didChange). The server analyses the changes and notifies the tool of any detected errors and warnings (textDocument/publishDiagnostics).
3. Go to Definition: The user executes "Go to Definition" on a symbol. The tool sends a textDocument/definition request to the server, which responds with the location of the symbol's definition.
4. Document Closed: The user closes the document. A textDocument/didClose notification is sent to the server.

OCaml LSP

ocaml-lsp is an implementation of the Language Server Protocol for OCaml in OCaml. It provides language features like code completion, go to definition, find references, type information on hover, and more, to editors and IDEs that support the Language Server Protocol. OCaml LSP is built on top of Merlin, which provides the actual analysis and type information.

Currently, OCaml LSP supports several LSP requests such as textDocument/completion, textDocument/hover, textDocument/codelens, etc. For the purposes of this article, we will limit the scope to textDocument\hover requests because this is where our command is implemented. You can find out more about supported OCaml LSP requests at Features | OCaml LSP.

Hover Requests

When a user hovers over a symbol or some syntax, their development tool sends a textDocument/hover request to the language server. To better understand this process, let us consider some sample code:

let get_children (position:Lexing.position) (root:node) =
  ...some code...

When the user hovers over the function name, get_children, the hover request (taken from the server logs) is as follows:

[Trace - 4:07:21 AM] Sending request 'textDocument/hover - (13)'.
Params: {
    "textDocument": {
        "uri": "file:///home/../../merlin/src/kernel/mbrowse.ml"
    },
    "position": {
        "line": 279,
        "character": 10
    }
}

This request includes the following information:

• The URI of the document where the user is hovering
• The position (line and character) within the document where the hover event occurred

The language server then responds with information corresponding to what its hover query should do. This could be type information, documentation information, etc.

[Trace - 4:07:21 AM] Received response 'textDocument/hover - (13)' in 2ms.
Result: {
    "contents": {
        "kind": "markdown",
        "value": "```ocaml\nLexing.position -> ('a * node) list -> node\n```"
    },
    "range": {
        "end": {
            "character": 16,
            "line": 279
        },
        "start": {
            "character": 4,
            "line": 279
        }
    }
}

The response received indicates that at this position the type signature is Lexing.position -> ('a * node) list -> node, and it's formatted with Markdown, since it was done in VSCode. For development tools that don't support Markdown, this response will simply be plaintext. The range is used by the editor to highlight the relevant line(s) for the user.

SyntaxDocumentation Implementation

With OCaml LSP, type information displayed from a hover request is taken from Merlin using the type_enclosing command, and the information returned is passed onto the hover functionality to be displayed as a response. With this, we can attach the result from querying Merlin about the SyntaxDocumentation command and add the results to the type_enclosing response.

type type_enclosing =
{
    loc : Loc.t;
    typ : string;
    doc : string option;
    syntax_doc : Query_protocol.syntax_doc_result option
}

To query Merlin for something, we use Query_protocol and Query_command. You can read more about what these do from Part 1 of this article series.

 let syntax_doc pipeline pos =
    let res =
      let command = Query_protocol.Syntax_document pos in
      Query_commands.dispatch pipeline command
    in
    match res with
    | `Found s -> Some s
    | `No_documentation -> None

Making SyntaxDocumentation Configurable

Sometimes, too much information can be problematic, which is the case with the hover functionality. Most times, users just want a specific kind of information, and presenting a lot of unrelated information can have a negative effect on their productivity. For this reason, SyntaxDocumentation is made to be configurable, so users can toggle it on or off. This is made possible by passing configuration settings to the server.

syntaxDocumentation: { enable : boolean }

For a piece of code such as:

type color = Red | Blue

When SyntaxDoc is turned off, we receive the following response:

{
      "contents": { "kind": "plaintext", "value": "type color = Red | Blue" },
      "range": {
        "end": { "character": 21, "line": 1 },
        "start": { "character": 0, "line": 1 }
      }
    }

When SyntaxDoc is turned on, we receive the following response:

{
      "contents": {
        "kind": "plaintext",
        "value": "type color = Red | Blue. `syntax` Variant Type: Represent's data that may take on multiple different forms..See [Manual](https://v2.ocaml.org/releases/4.14/htmlman/typedecl.html#ss:typedefs)"
      },
      "range": {
        "end": { "character": 21, "line": 1 },
        "start": { "character": 0, "line": 1 }
      }
    }

Conclusion

In this article, we looked at the LSP protocol and a few examples of how it is implemented in OCaml. With OCaml LSP, the SyntaxDocumentation command becomes a very handy tool, empowering developers to get documentation information by just hovering over the syntax. If you wish to support the OCaml LSP project, you are welcome to submit issues and code constibutions to the repository at Issues | OCaml LSP. In the next and final part of this series, we will look at the VSCode Platform Extension for OCaml and how we can add a visual checkbox to the UI for toggling on/off SyntaxDocumentation.