Show HN：我的元眼镜API更新，“嘿Meta，给ChatGPT发条信息”

Show HN：我的元眼镜API更新，“嘿Meta，给ChatGPT发条信息”
Show HN: Update to my meta glasses API "Hey Meta send a message to ChatGPT"

原始链接: https://github.com/dcrebbin/meta-vision-api

这款浏览器扩展程序连接了Meta Rayban智能眼镜（或Messenger应用）和OpenAI、Perplexity、Claude等AI服务，实现自定义的AI交互。用户可以通过语音指令（例如“嘿Meta，给ChatGPT发条信息”）或文本输入，将照片或信息发送到指定的Messenger群聊中。扩展程序会监控该群聊，并将请求转发给选择的AI提供商进行处理。AI的回复随后会发送回用户，并且可以选择使用OpenAI的文本转语音功能生成音频剪辑，并通过Messenger发送。它还支持视频通话监控，将截图发送给AI进行分析并记录输出结果。设置过程包括创建一个伪装成AI联系人的Messenger群聊，在Meta View应用中重新同步Messenger以识别新的联系人，并在扩展程序中配置API密钥。这使得用户能够利用AI进行图像分析、获取准确答案等等，所有操作都可以通过免提语音指令完成。

这个Hacker News帖子讨论了“嘿Meta，给ChatGPT发个消息”，这是一个项目的更新，允许Meta智能眼镜与ChatGPT等AI模型交互。此更新用Chrome/Firefox的浏览器扩展程序取代了“粗糙”的实现，该扩展程序监控一个单独的Facebook Messenger帐户。这个扩展程序允许用户通过指定的Messenger群聊向AI模型发送消息和照片，有效地绕过了Meta内置的Llama AI。该扩展程序还具有视频监控功能，使用户能够将视频通话中的快照发送给AI提供商。开发者表达了对Meta封闭生态系统的沮丧，并暗示了开源智能眼镜或现有Meta设备越狱解决方案的可能性。评论者赞赏该项目能够利用更好的AI模型与眼镜互动，并指出了Meta自身AI的局限性。讨论包括诸如位置追踪和更容易访问语音助手等未来功能。

原文

This is a browser extension to add custom AI bots to messenger that can be used with the Meta Rayban Smart Glasses or the standalone messenger app.

Hey Meta send a photo to my food log: Video Demo
Hey Meta send a message to ChatGPT
Video Monitoring: Send screenshots of your video calls to your chosen provider (ChatGPT, Claude etc) and then log the output to the log viewer

a) Meta Rayban Smart Glasses (or the standalone messenger app)

b) OpenAI/Perplexity/Claude etc Api Key

c) Alternative Facebook/Messenger account

bun install
bun run dev:chrome (or brave, firefox)
This should build and run the extension and automatically open it
Add any api keys you want to use in the extension: the API settings tab has more information on how to get them
Sigin into your alt Facebook account and head to facebook.com/messages/t and start monitoring the conversation

Hey Meta Send a Photo/Message to _____

Before we setup our extension we're going to trick the Meta Glasses into allowing us to send a message to (nearly) any name e.g: "Hey Meta send a message to ChatGPT".

Create a messenger group chat with 2 other facebook accounts (the minimum amount allowed to create a group chat)

Remove the account you're not going to use
Change the name of the chat
Update the group chat photo (for a legit feel)
Set a nickname for your alt bot account
Go to the Meta view app within the communications section
Go to Messenger and disconnect then reconnect your messenger account

I believe this step resyncs all your latest chats and friends which then allows that Meta Glasses to become aware of your newly created group chat to allow for voice commands!

On your alt account head to messenger.com or facebook.com/messages then open your newly created group chat
Start monitoring the chat
With each new message/image request it will send it to your chosen provider (ChatGPT, Claude etc) and then respond to you with the output
If enabled: It will then generate an audio clip of that output using OpenAI and send it back to you

All chats can be done via voice commands "Hey Meta send a message to ___" or by simply messaging the group chat.

OpenAI query with Minimax text to speech

Using Perplexity to answer a question accurately

Using GPT 4.1 to describe an image

by Devon Crebbin

Please reach out if there are any issues or feature requests :)

Hopefully the Meta Reality Labs team will provide an SDK in the future so these types of integrations can be ✨productionised✨

Credits:

Anime Sky from Vecteezy is used in the icon