How we used AMP Emails to get feedback on AMP Emails!

AMP is considerably a nascent technology. When you are dealing with such an adolescent piece of tech, it is not the easiest thing to build a top-of-the-line production-ready feature for your product that ticks all your requirements.

Storytime!

In one of the quarters, we were trying to adopt the OKR way of developing features. Now features were just not features; they were initiatives. The process became more than just coding or solving a problem. It became about genuinely solving real customer problems to ensure they have a delightful user experience.

With this in mind, we built a dynamic reminder using AMP Emails that allowed the user to submit their expense report right from their emails.

During this process, we learned feedback from users and having other metrics like regular stats on the feature usage were essential steps to make the whole initiative successful. This required us to log and monitor feedback from users constantly.

The roadblocks we faced

The feature was outside our product, actually inside Gmail.

This led us to the questions:

How would we know what the user might have seen on the screen? And, how would we know the actions he performed as all of this was happening inside Gmail?

Now the problem to solve was to build an infra around:

  1. How to log and monitor actions that are happening inside the email
  2. How to get feedback from users considering there are so many limitations in AMP Emails

First, the monitoring part!

As mentioned earlier, AMP Emails being a nascent technology don’t have the required logging and monitoring support.

Our dynamic reminder is like a mini web app in itself. This resulted in us recording a lot of steps that were needed. We also realised, logging each of the events in all the screens was necessary to understand user behavior.

This is how our dynamic reminder looks!

What did we do?

Our dynamic reminder is smart, and so are we 😎 . We handled most of the things in our backend notification service to keep up with all the good practices:

  • Exposed our APIs using signed URLs
  • Handled most errors in the backend
  • Queued infra to handle the load on emails
  • Had a common wrapper for all the fallback emails
  • Made all triggers agnostic of the AMP events
  • Most of the inter-service communication happened within the cluster, i.e., through Internal Calls. This made the API responses faster. (You don't want the loader to spin forever!)

Fun fact: While building all of this we used telepresence to create a two-way network proxy for our stag environment's Kubernetes pod. This helped build and test real-life scenarios to emulate the whole email sending cycle end to end.

The Monitoring & Logging Part!

We logged everything into Mixpanel from our backend service and later created a report out of it. This helped us know at what step the user would drop. This enabled us to accurately predict what the user could've seen on his email before dropping off.

Since we log other events from the web app too, we could combine events to map a conclusion to know when a particular user came from the email to the web app or if he saw something on the email and then took some action.

Problem 1 solved!
This is how the logging and monitoring Infra worked in parallel with the Dynamic Reminder

The feedback?

How do we take feedback?

Well, we used AMP to get feedback on AMP!

We built a small infra for triggering an AMP email whenever our dynamic reminders were triggered using a few Google Sheets and webhooks. The responses were directly sent on a slack channel.

Sample code snippet on how we raised the triggers.
This is how our feedback AMP Email looked!

How was all this hard work helpful?

  • We got regulars usage stats which helped us with our OKRs.

- Over conversion rate was 27%

- On average, it took a user just 10-13 seconds to select and submit his expenses belonging to different categories from the moment he opened the email

- 72% of the total feedback was a 5/5 star rating!

  • Users could directly send their feedback right away. And we could act upon it quickly.
  • We could observe user behavior and take the necessary steps to improve the feature.

Overall, we got some great feedback; here are a few!

All in all, this was fun!

Shahbaz Zaidi

I am a Member Of Technical Staff. While not coding or racing past everything, I like spending time creating memes or pulling pranks on people.

More of our stories from

Engineering
The curse of being a Senior Engineer, how to deal with timelines, frustrations, etc

Being a good developer is 50% skill and 50% emotional support; here's my secret to balancing both at the right amount!

Read more...
How did I build cropping of receipts in the mobile app?

Follow Yash's journey of what it takes to reduce manual work for our customers when receipts come in all shapes and sizes!

Read more...
How did we increase Data Extraction accuracy by a whopping ~50%?

Wanna know the secret of data extraction, the complex machine learning models we use, the experiments we did? Read on...

Read more...
The not so secret sauce of my work

From chaos to clarity, follow Chethan's not so secret sauce to excelling at work!

Read more...
From Zero to Hero: The Policy Tests Journey!

The story of policy tests at Fyle

Read more...
How Fyle changed my life from a naive intern to a confident Engineering Lead

A blogpost that documents Shwetabh's journey at Fyle.

Read more...
Vikas Prasad @ Fyle

This document is a user guide to Vikas at work.

Read more...
Gokul K's README

This document is a user guide to Gokul at work.

Read more...
How we used AMP Emails to get feedback on AMP Emails!

This blog discusses the how's & why's behind us building a feedback & monitoring system for AMP Emails using AMP Emails

Read more...

All Topics