Computer API

The following functions are designed for language models to use in Open Interpreter, currently only supported in OS Mode.

​

Display - View

Takes a screenshot of the primary display.

interpreter.computer.display.view()

​

Display - Center

Gets the x, y value of the center of the screen.

x, y = interpreter.computer.display.center()

​

Keyboard - Hotkey

Performs a hotkey on the computer

interpreter.computer.keboard.hotkey(" ", "command")

​

Keyboard - Write

Writes the text into the currently focused window.

interpreter.computer.keyboard.write("hello")

​

Mouse - Click

Clicks on the specified coordinates, or an icon, or text. If text is specified, OCR will be run on the screenshot to find the text coordinates and click on it.

# Click on coordinates
interpreter.computer.mouse.click(x=100, y=100)

# Click on text on the screen
interpreter.computer.mouse.click("Onscreen Text")

# Click on a gear icon
interpreter.computer.mouse.click(icon="gear icon")

​

Mouse - Move

Moves to the specified coordinates, or an icon, or text. If text is specified, OCR will be run on the screenshot to find the text coordinates and move to it.

# Click on coordinates
interpreter.computer.mouse.move(x=100, y=100)

# Click on text on the screen
interpreter.computer.mouse.move("Onscreen Text")

# Click on a gear icon
interpreter.computer.mouse.move(icon="gear icon")

​

Mouse - Scroll

Scrolls the mouse a specified number of pixels.

# Scroll Down
interpreter.computer.mouse.scroll(-10)

# Scroll Up
interpreter.computer.mouse.scroll(10)

​

Clipboard - View

Returns the contents of the clipboard.

interpreter.computer.clipboard.view()

​

OS - Get Selected Text

Get the selected text on the screen.

interpreter.computer.os.get_selected_text()

​

Mail - Get

Retrieves the last number emails from the inbox, optionally filtering for only unread emails. (Mac only)

interpreter.computer.mail.get(number=10, unread=True)

​

Mail - Send

Sends an email with the given parameters using the default mail app. (Mac only)

interpreter.computer.mail.send("john@email.com", "Subject", "Body", ["path/to/attachment.pdf", "path/to/attachment2.pdf"])

​

Mail - Unread Count

Retrieves the count of unread emails in the inbox. (Mac only)

interpreter.computer.mail.unread_count()

​

SMS - Send

Send a text message using the default SMS app. (Mac only)

interpreter.computer.sms.send("2068675309", "Hello from Open Interpreter!")

​

Contacts - Get Phone Number

Returns the phone number of a contact name. (Mac only)

interpreter.computer.contacts.get_phone_number("John Doe")

​

Contacts - Get Email Address

Returns the email of a contact name. (Mac only)

interpreter.computer.contacts.get_phone_number("John Doe")

​

Calendar - Get Events

Fetches calendar events for the given date or date range from all calendars. (Mac only)

interpreter.computer.calendar.get_events(start_date=datetime, end_date=datetime)

​

Calendar - Create Event

Creates a new calendar event. Uses first calendar if none is specified (Mac only)

interpreter.computer.calendar.create_event(title="Title", start_date=datetime, end_date=datetime, location="Location", notes="Notes", calendar="Work")

​

Calendar - Delete Event

Delete a specific calendar event. (Mac only)

interpreter.computer.calendar.delete_event(event_title="Title", start_date=datetime, calendar="Work")