How we built a video in email experience

Many start every new year with new resolutions and that sometimes helps some with building new habits. I like to think the word BUILD in this context should not involve starting from the roots but rather building upon something existing. This lets me approach new beginnings with reflections that will direct future builds.

Last year, we shipped a feature that enables creators using our platform to send videos with their emails that can be consumed in-line with text by their subscribers. This is a feature that relies on the support of videos in the email clients used by those subscribers and it came with its challenges.

Before getting into what it took to deliver these emails to subscribers, let’s dive a little into how this feature was built.

For our URL upload process, we take a Vimeo, YouTube, or Wistia URL, extract its thumbnail if that’s all the creator needs, and switch the state in the editor from idle to ready. That looks like this:

So far, we have recreated the most simple case of using video in emails without providing a way to watch those videos within email. In order to move further, we had to consider all the states we will encounter.

States

At the start of the project, we thought about the states this way:

At the time, it felt easier to think about the state management in parallel so all the possible states were flat for development simplicity. We learned very quickly, that what might appear more convenient in a development experience may be a compromise on user experience.

With a little time, we saw that since we are mostly able to instantly generate a thumbnail, we can skip on the loading state. But this would mean that we only do this for creators that will only like to send a thumbnail link for their videos. Not a GIF nor a playable in-line video. That means rethinking the states.

Instead of keeping all states in parallel, we divided them into two which I’ll call:

• State of format
• State of readiness

Then we nested the state of readiness within each state of format to meet the creator’s needs based on their desired format. Resulting in a compound state.

Upload Latency

If a creator decides they want an in-email video experience for their subscribers whose email clients affords such comfort, they have to wait for their videos to upload, and doing so through a URL means an additional time of download from the current platform where it exists.

In a world of 4K and 8K videos, media file sizes are over the roof so a little delay in these processes is inevitable.

We temporarily download the video to our file server using youtube-dl, store necessary data from it to our database, and upload to mux. Mux helps simplify this process of managing videos with their detailed API and dashboard.

Because of how quick the process of writing emails is compared to creating video content like creators typically would on YouTube, the video upload process may be perceived to be even more prolonged.

Post production

Our brilliant designers had the idea that while we offer video and gif format options, they do not need to be mutually exclusive. If an email client does not support playing videos, they can fallback to a GIF instead of a thumbnail. Here’s what that experience looks like:

Because GIFs are limited in length, we offer creators a way to select the frames of the video they want in their GIF preview:

Delivery

Unlike the world of web browsers where it is a lot easier to narrow down support for commonly used Firefox, Chrome, Safari, the world of email clients is massive and more people have outdated email clients than they do outdated web browsers.

This poses a challenge with this feature as many email client versions will never get to benefit from it. At the time of this writing AppleMail and Spark are 2 of the clients that still handle videos best.

This leads to the question “What does lack of support look like?” It varies per client. Most clients completely yank out the video element, some will show the fallback nested inside the <video> element, and one email client just keeps the video and throws a security error.

For the clients that let us keep a fallback inside the <video> element, this is the traditional way videos were used when they had less support on web browsers. It looks like this:

<video controls>
  <source src="myvideo.mp4" type="video/mp4">
  <img src="fallback.jpg" alt="video of fallback">
</video>

When an email client yanks the entire video element, then this does not work because our fallback is gone with the video element since it is contained within it.

We had to write a fallback to sit adjacently to the video and stack the video above the the fallback visually. This gives us:

<div style="display: grid">
<img src="fallback.jpg" alt="video of fallback" style="grid-area: 1/1/3/2">
<video controls style="grid-area: 1/1/3/2">
  <source src="myvideo.mp4" type="video/mp4">
</video>
</div>

which should look like this based on the markup.

The grid styles applied lets us really end up in this:

On email providers like Gmail that yank the video element, it does not matter that there is no grid support, we simply get this:

<div style="display: grid">
<img src="fallback.jpg" alt="video of fallback" style="grid-area: 1/1/3/2">
</div>

and it gives the desired result of an image fallback.

Final thoughts

As a creator marketing company with a strong focus on emails, we may not always have all the controls over a feature with the limitations of email clients. However, we will always put our best foot forward and build till the brink of our limits.

If you are yet to try out the video feature at ConvertKit, what are you waiting for?