OpenAI is taking ChatGPT to the next level by allowing it to communicate with some of your Mac’s desktop apps!
This exciting update means that developers can now integrate ChatGPT directly with coding apps like VS Code, Xcode, TextEdit, Terminal, and iTerm2. No more tedious copy-pasting—ChatGPT can now read the code you’re working on and use it as context for your prompts.
While it’s not quite an AI agent yet, this feature, dubbed "Work with Apps," is a significant step towards creating more interactive and agentic systems. The challenge? Teaching AI to understand not just prompts but the entire computer screen. OpenAI is starting with coding apps, given the popularity of AI coding assistants, and plans to expand to text-based apps for writing tasks.
Want to hear more? Join Mal on the Property AI Report Podcast each week!
Access from your preferred podcast provider by clicking here.
Currently, ChatGPT can’t write code directly into your apps, unlike tools like GitHub Copilot. However, it can suggest code, which you’ll need to paste back into your environment. The feature is available for Plus and Teams users, with plans to roll out to Enterprise and Edu soon.
To make this magic happen, OpenAI is leveraging macOS’s accessibility API, which has been around for ages and is pretty reliable for reading text. However, it doesn’t handle visual elements, so there’s room for growth.
As OpenAI continues to refine this feature, it’s a thrilling time for developers and tech enthusiasts. With whispers of a general-purpose AI agent named “Operator” on the horizon, the future of AI integration looks bright and full of possibilities!
Want to hear more? Join Mal on the Property AI Report Podcast each week!
Access from your preferred podcast provider by clicking here.
Made with TRUST_AI - see the Charter: https://www.modelprop.co.uk/trust-ai
Comments