Working on Fyle-ops

Summary of my Internship, Learnings and Growth after joining Fyle

Introduction

Greetings, I'm Harshal Patil, and my journey with Fyle as a front-end intern commenced in July 2023.

While conventional expectations for a front-end intern primarily revolve around front-end tasks, my role diverged as I started working on tasks, focusing on APIs, AWS, and automation within the fyle-ops realm. Let's delve into the repository central to our endeavors - fyle-ops marking my initial project upon joining.

Fyle-Ops Overview

Fyle-ops is a repository containing a collection of lambda functions used to establish automation that aids in Service Requests (SR) and Bug Reports (BR).

These lambda functions were uploaded to AWS using the workflow in GitHub actions. From assigning open SRs and BRs every morning to broadcasting reminder messages on Slack, these automated processes have significantly alleviated the workload for both the Customer Success (CS) and engineering teams.

Buoyed by its success, we have extended this automation framework to encompass Feature Requests (FR) and Product Clarifications (PC), further enhancing operational efficiency and team productivity.

Initiative Overview

Throughout the project, I undertook various tasks within the repository, collaborating closely with Abhishek as the sole contributor. Together, we identified issues and devised strategies to address them effectively. Some of the key tasks encompassed:

• Introduction of ESLint and establishment of pre-commit hooks in the repository using Husky and GitHub actions. Further insights on this initiative are detailed in this Medium article I wrote a few months back.

• Development of a GitHub actions workflow facilitating the seamless deployment of lambda functions from the repository to AWS.

• Implementation of automation utilizing APIs for platforms like Notion, ClickUp, and Slack, aimed at aggregating and storing statistical data about Service Requests (SRs) and Bug Reports (BRs) in Notion databases.

Challenges faced

Designing the workflow to update all Lambda functions in AWS from the repository emerged as the most daunting task. Despite its apparent simplicity, it posed unforeseen challenges, particularly regarding shared helper files and linking Lambda functions with repository files.

To surmount one of these hurdles to identify changes, we turned to the tj-actions library to identify modified files and determine whether to update all files or specific ones. Additionally, we opted to refactor Lambda function names to eliminate hardcoded pairs, fostering a more dynamic approach.

- name: Get all modified Lambdas
   id: changed-files
   uses: tj-actions/changed-files@v40
   with:
      separator: ","
      files: |
         lambdas/**

- name: Check for updates in services/helpers
   id: changed-files-specific
   uses: tj-actions/changed-files@v37
   with:
   files: |
      services/
      helpers/

This allowed us to obtain a list of changes and decide on updating strategies.

Another obstacle surfaced when linking Lambda functions with repository files using pairs, Consequently, we opted to recreate all Lambda functions, renaming them in AWS to match the camel-case version of their respective file names:

strategy:
      matrix:
        lambda:
          - name: 'acceptedBrsReminder'
            file: 'accepted-brs-reminder.js'

We automated this process in GitHub jobs using the following code:

function_name=$(echo "$file_name" | awk -F"-" '{gsub(/\.js$/,""); result=$1; for (i=2; i<=NF; i++) result = result toupper(substr($i,1,1)) substr($i,2); print result}')

But the challenges didn't end there.

As the number of Lambda functions in the repository grew to 25, the efficiency of our deployment process came under scrutiny. With each function requiring a separate job, we faced increased billing in GitHub actions. To mitigate this, I opted for a simpler solution - a single job with a loop to upload all functions.

This seemingly straightforward solution proved to be the most arduous. Uploading a Lambda function to AWS entails zipping it along with the Node modules and helper functions. While previous methods abstracted this process, the loop-based approach necessitated updating the zip procedure. All said and done, it took me three months to complete this endeavor.

- name: Loop through Lambda functions and deploy
        run: |
          lambda_files= getAllLambdaFiles

          changed_files_list=getChangedFiles

          for file_name in "${lambda_files[@]}"; do
            function_name=getFunctionName($file_name)

            if # Check if the file needs to be updated
              (
                # zip and deploy to AWS
              )
              rm -rf zip_package
            else
              echo "No changes detected in $function_name, skipping deployment."
            fi
          done

We successfully reduced the billing from 25 to around 3 minutes by implementing a more efficient deployment process.

Learning and Growth Journey:

Working on this repository provided me with significant learning experiences, including APIs from Notion, ClickUp, to GitHub actions. It taught me the importance of proper task delegation in a team setting.

The most frequent PR commits I had were related to the naming conventions I used and the names I used for the functions 🥹. Many of the commits in my PR were only because I used vague or unintentional names, but thanks to Abhishek for keeping up with my issues, at least I was able to push commits that had not zero but fewer changes than before. It made me understand what a professional way of writing code is and to make it maintainable and readable for others.

Also, I started working with AWS, so I had a lot of tasks where I had to work with AWS services which were new to me, but the knowledge I gained from this initiative increased my understanding of the cloud service, and I will be looking forward to implementing and learning more stuff related to the cloud.

I enhanced my flexibility, code reading, and writing skills as a developer. Working in a team provided insights that tutorials and docs often miss, emphasizing the importance of mentorship.

So overall, when I look back at my previous projects, I can notice that my current coding practices and thinking process have improved.

Noteworthy Achievements:

Throughout my tenure, several notable accomplishments underscored my contributions to the Fyle-ops repository and beyond:

Cost Optimization in GitHub Actions: Implementing strategic optimizations within GitHub actions resulted in a significant reduction in resource consumption and associated costs for the Fyle-ops repository. This initiative not only streamlined workflows but also demonstrated my proficiency in resource management and optimization strategies.

Streamlined Monthly Stats Retrieval: Aiming to streamline the retrieval of monthly statistics for Bug Reports (BRs) and Service Requests (SRs), I developed a robust function leveraging APIs from Notion, ClickUp, and Slack. This function facilitated the extraction of comprehensive statistics, empowering stakeholders to make data-driven decisions efficiently.

Service Level Agreement (SLA) Implementation: Tasked with implementing Service Level Agreements (SLAs) within ClickUp, I embarked on finding innovative solutions to ensure adherence to predefined rules regarding task durations. Despite initial challenges, including limitations within the ClickUp premium tier, I devised a workaround utilizing ClickUp's API. Although the initial approach was deemed impermissible 😅, the experience provided invaluable insights into Notion databases and API integrations, culminating in the successful completion of SLA-related tasks.

In summary, my internship experience in Fyle and working with Fyle-Ops was realy amazing and helped me learn a lot as a developer and also as a team member.