> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/microsoft/playwright-mcp/llms.txt
> Use this file to discover all available pages before exploring further.

# Coordinate-Based Interactions

> Low-level mouse operations using screen coordinates

<Warning>
  Coordinate-based tools require the `vision` capability. Start the server with `--caps=vision` to enable these tools.
</Warning>

Coordinate-based tools provide low-level mouse control using x/y coordinates. These tools are useful for interacting with elements that don't have proper accessibility attributes or for pixel-perfect interactions like drawing applications.

## Enabling Vision Capability

To use coordinate-based tools, start the MCP server with the vision capability:

```bash theme={null}
npx @playwright/mcp-server --caps=vision
```

You can combine it with other capabilities:

```bash theme={null}
npx @playwright/mcp-server --caps=vision,pdf,testing
```

## Mouse Click Operations

### browser\_mouse\_click\_xy

Click left mouse button at a given position.

<ParamField path="x" type="number" required>
  X coordinate on the page
</ParamField>

<ParamField path="y" type="number" required>
  Y coordinate on the page
</ParamField>

**Read-only**: No

```javascript theme={null}
// Example: Click at position (100, 200)
{
  "tool": "browser_mouse_click_xy",
  "arguments": {
    "x": 100,
    "y": 200
  }
}
```

### browser\_mouse\_down

Press mouse button down at current position.

<ParamField path="button" type="string">
  Button to press, defaults to "left". Options: "left", "right", "middle"
</ParamField>

**Read-only**: No

```javascript theme={null}
// Example: Press right mouse button
{
  "tool": "browser_mouse_down",
  "arguments": {
    "button": "right"
  }
}
```

### browser\_mouse\_up

Release mouse button at current position.

<ParamField path="button" type="string">
  Button to release, defaults to "left". Options: "left", "right", "middle"
</ParamField>

**Read-only**: No

```javascript theme={null}
// Example: Release left mouse button
{
  "tool": "browser_mouse_up",
  "arguments": {
    "button": "left"
  }
}
```

## Mouse Movement

### browser\_mouse\_move\_xy

Move mouse to a given position.

<ParamField path="x" type="number" required>
  X coordinate to move to
</ParamField>

<ParamField path="y" type="number" required>
  Y coordinate to move to
</ParamField>

**Read-only**: No

```javascript theme={null}
// Example: Move mouse to position (300, 400)
{
  "tool": "browser_mouse_move_xy",
  "arguments": {
    "x": 300,
    "y": 400
  }
}
```

### browser\_mouse\_drag\_xy

Drag left mouse button from one position to another.

<ParamField path="startX" type="number" required>
  Start X coordinate
</ParamField>

<ParamField path="startY" type="number" required>
  Start Y coordinate
</ParamField>

<ParamField path="endX" type="number" required>
  End X coordinate
</ParamField>

<ParamField path="endY" type="number" required>
  End Y coordinate
</ParamField>

**Read-only**: No

```javascript theme={null}
// Example: Drag from (100, 100) to (300, 300)
{
  "tool": "browser_mouse_drag_xy",
  "arguments": {
    "startX": 100,
    "startY": 100,
    "endX": 300,
    "endY": 300
  }
}
```

## Scrolling

### browser\_mouse\_wheel

Scroll using the mouse wheel.

<ParamField path="deltaX" type="number" required>
  Horizontal scroll amount (positive = right, negative = left)
</ParamField>

<ParamField path="deltaY" type="number" required>
  Vertical scroll amount (positive = down, negative = up)
</ParamField>

**Read-only**: No

```javascript theme={null}
// Example: Scroll down by 100 pixels
{
  "tool": "browser_mouse_wheel",
  "arguments": {
    "deltaX": 0,
    "deltaY": 100
  }
}
```

## Use Cases

### Drawing Applications

Coordinate-based tools are ideal for interacting with canvas-based drawing applications:

```javascript theme={null}
// Draw a line on a canvas
{ "tool": "browser_mouse_move_xy", "arguments": { "x": 50, "y": 50 } }
{ "tool": "browser_mouse_down", "arguments": { "button": "left" } }
{ "tool": "browser_mouse_move_xy", "arguments": { "x": 200, "y": 200 } }
{ "tool": "browser_mouse_up", "arguments": { "button": "left" } }
```

### Interactive Maps

Interact with map interfaces that use coordinate-based interactions:

```javascript theme={null}
// Pan the map
{
  "tool": "browser_mouse_drag_xy",
  "arguments": {
    "startX": 400,
    "startY": 300,
    "endX": 200,
    "endY": 100
  }
}
```

### Games and Simulations

Control game interfaces or simulations that require precise coordinate input:

```javascript theme={null}
// Click a specific location in a game
{
  "tool": "browser_mouse_click_xy",
  "arguments": {
    "x": 640,
    "y": 360
  }
}
```

### Elements Without Accessibility Attributes

When elements lack proper accessibility attributes, coordinate-based clicks can serve as a fallback:

```javascript theme={null}
// Click an unlabeled button at known coordinates
{
  "tool": "browser_mouse_click_xy",
  "arguments": {
    "x": 150,
    "y": 75
  }
}
```

<Tip>
  Prefer semantic element-based tools (like `browser_click`) when possible. Use coordinate-based tools only when necessary, as they are more fragile and can break with layout changes.
</Tip>

## Best Practices

<AccordionGroup>
  <Accordion title="Use as Last Resort">
    Coordinate-based interactions are fragile and can break with layout changes. Always prefer semantic element selection using `browser_click`, `browser_type`, and other element-based tools when available.
  </Accordion>

  <Accordion title="Take Screenshots First">
    Use `browser_take_screenshot` to verify element positions before using coordinate-based tools. This helps ensure you're clicking the right location.
  </Accordion>

  <Accordion title="Consider Viewport Size">
    Coordinates are relative to the viewport. Ensure the browser window size is consistent with your expected coordinates using `browser_resize`.
  </Accordion>

  <Accordion title="Account for Scrolling">
    Coordinates are relative to the current viewport. If the page is scrolled, the coordinates will be different. Use `browser_mouse_wheel` to scroll if needed.
  </Accordion>
</AccordionGroup>

## Combining with Other Tools

Coordinate-based tools work well with other Playwright MCP tools:

```javascript theme={null}
// 1. Resize browser for consistent coordinates
{ "tool": "browser_resize", "arguments": { "width": 1280, "height": 720 } }

// 2. Take a screenshot to verify positions
{ "tool": "browser_take_screenshot", "arguments": { "filename": "before-click.png" } }

// 3. Perform coordinate-based click
{ "tool": "browser_mouse_click_xy", "arguments": { "x": 640, "y": 360 } }

// 4. Take another screenshot to verify result
{ "tool": "browser_take_screenshot", "arguments": { "filename": "after-click.png" } }
```

## Related Tools

<CardGroup cols={2}>
  <Card title="Core Automation" icon="browser" href="/tools/core-automation">
    Prefer semantic element-based interactions
  </Card>

  <Card title="Browser Resize" icon="arrows-alt" href="/tools/core-automation#browser_resize">
    Set consistent viewport for coordinate accuracy
  </Card>
</CardGroup>