GORUCK: 360 data analytics platform for a multi-million dollar sports eCommerce brand

GORUCK is a popular eCommerce sports brand and rucking enthusiast community. Founded by ex Special Forces, GORUCK started as a small project to build a rucksack and show people how to use it to survive in harshest environments. Now with a huge fan following it generates revenue of over $50 Million a year.

• GORUCK wanted to upgrade the backend of their ecommerce website from Bigcommerce to Shopify plus
• GORUCK wanted to create a highly personalized digital shopping experience and efficient customer journey
• Understand the advertising ROI and leverage the right-mix of marketing channels

Building a robust architecture

As a first step we defined a re-architecture roadmap and cloud migration plan on AWS. We designed a database architecture to enable a cloud-based data warehouse using AWS RDS. Microservices architecture was used to trigger independent syncing jobs for various system integrations.

Better analytics and ETL solution

In the old system, users could not perform dynamic queries or advanced filters over reports and data to generate more powerful analytics. The data was outdated and static. In the newer Shopify Plus version, we implemented dynamic report generation. This obviously gives users much more control and power over advanced analytics. To make the apps perform better, cloud and database side caching with pre- calculations processing helps reduce runtime overheads.

Implementing DevOps and QA practices

Building a CI/CD pipeline using Jenkins that would trigger Glue jobs or System integration Microservices.

Business intelligence module

We developed an analytics app suite capable of creating powerful visualizations, dashboards, reports, and alerts. They can also get insights into different trends over their data and create alerts or scheduled reports/infographics to be sent via email, Slack, SMS, etc over a duration of time or when critical triggers or trends are spotted.

Alexa app

We also developed an Alexa app that allows users to perform complex queries over different data points using voice only. So that when users want a quick answer on some data they can directly use Alexa to get what they need.

Role of AWS Lambda

• The Lambda functions are singletons and each one of them is independent and serves a unique request. Mainly lambdas are triggered by Scheduled CRON events (for periodically calling as scheduled events) and AWS S3 bucket insertion. We have developed lambdas for ETL processing to store the bulky data in a structural way. The data visualization is done using AWS QuickSight dashboard
• We have implemented AWS S3 as a database for the application. We are getting data into S3 bucket from multiple sources which triggers lambda functions and causes database connectors (functions) to import incoming data into the RDS database.

Custom ETL process

• As an AWS Partner, we used the well architected framework to design a scalable custom ETL architecture using AWS Glue, AWS RDS and Apache Airflow.
• We store the data in RDS clusters and process data with lightning-fast speed for insights generation.
• This architecture allowed whole data sets to be processed within milliseconds and according to the analytical functions being performed, rather than by volume.

Business intelligence process

• We used the MERN stack with component-driven development to develop an analytics app suite.
• Microservices architecture on the backend and component-driven architecture on the front end along with client-side caching made the app suite snappy and responsive.

Monitoring and caching using AWS Services:

• AWS CloudTrail: Though we don’t retain logs, CloudTrail is used periodically
  for tracing activities and stack failure.
• Amazon Cloudwatch: The application relies on Cloudwatch logs for error
  monitoring that were produced by Lambda and other services
• Amazon CloudFront acts as a CDN and helps in frontend caching, while Amazon GuardDuty helps in security and monitoring.