Monthly Retrospective - October 2024

By Max Sherman ยท Nov 5, 2024

Stats

  • Not sure how many hours worked
  • Streamed 0 hours on Twitch (0% M/M)
  • Stripe payouts $2,429.04 (+32% M/M)
  • 590 Twitter followers (+5% M/M)
  • 171 Twitch followers (0% M/M)

FB Ads A/A test

We ran an A/A test with our FB ad to test if doubling ad spend doubles conversions. The test ran for one week, and the results were 25% worse on link clicks and cost per conversion (sign ups not purchases).

This is a pretty large difference but is potentially explained by the "control" A group having been running for months, so maybe more optimized by the FB system.

My takeaway is that when we run an A/B test, we need to beat the control on these metrics by 25% or more. Otherwise it may just be noise.

SEO

An SEO course from an entrepreneur I follow on Twitter released this last week. I want to take a look at it this month and see if I can scope out some work to invest in SEO, as this is a high ROI channel for us.

Improving performance

We shipped a huge upgrade to our diarization (speaker segmentation) ML model. There's a lot more work to do, but I think we can realize enough perf improvement to show the fast preview meeting minutes almost instantly. This may involve some tweaks to the open source whisper model as well.

I think we can get a lot of wins by using ffmpeg in process to hold a buffer of the decoded (as WAV) file. Then only serving ML models with this buffer. There's a lot of room for improvement.

Shipped new features

  • Sidebar and transcript redesign
  • Upgraded diarization pipeline

Goals for November

  • Run one FB ads A/B test
  • Scope and ship an SEO project
  • Improve server performance a lot
  • Ship 2 major features