Artificial Ignorance

Artificial Ignorance

Share this post

Artificial Ignorance
Artificial Ignorance
I made an Alexa
Copy link
Facebook
Email
Notes
More

I made an Alexa

Meet Hugh.

Charlie Guo's avatar
Charlie Guo
Mar 08, 2023
∙ Paid
7

Share this post

Artificial Ignorance
Artificial Ignorance
I made an Alexa
Copy link
Facebook
Email
Notes
More
3
1
Share

I recently came across this tweet1:

Twitter avatar for @DrJimFan
Jim Fan @DrJimFan
Here’s the recipe to make Siri/Alexa 10x better: 1. Whisper to convert speech to text. Best open-source speech model out there. 2. ChatGPT to generate smart home API calls and/or text response. 3. VALL-E to synthesize speech. It can mimic anyone’s voice sample! Quick figure 1/3
Image
5:08 PM ∙ Jan 9, 2023
5,571Likes1,105Retweets

And I realized that with OpenAI and ElevenLabs, you could do this - make your own smart assistant - just by connecting APIs. No fancy machine learning necessary.

So I built it. Meet Hugh.

Share

The plan

To build Hugh, I first sketched out my plan at a high level.

  1. Write the server code:

    • Transcribe audio and return the resulting text.

    • Generate an AI response given a text prompt.

    • Convert an AI response into speech and save it as an mp3 file.

    • Send an mp3 file to the user to be played.

  2. Write the browser code:

    • Record audio and send it off for transcription.

    • Submit a text box to generate a spoken AI response.

    • Play an audio file.

  3. Add some polish (nice design, images, etc).

I also wanted to try using a few different AI tools:

  • ChatGPT to write the code itself (as much as it could)

  • Whisper to transcribe audio

  • ChatGPT (API) to respond to questions

  • ElevenLabs (and the Hugh voice I created) to generate speech

  • Midjourney to create an avatar

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 Charlie Guo
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More