Sweet Analytics: 360 marketing analytics platform for ecommerce leaders

Sweet Analytics is a platform for marketing and customer analytics catering to eCommerce retailers. Its services encompass an all-in-one marketing data automation tool, empowering businesses to boost sales and gain deeper insights into their customers.

Our client, Sweet Analytics, is a marketing and customer analytics platform specifically designed for the eCommerce industry. However, they were facing various challenges related to scalability, availability, integration, and security because their cloud infrastructure was hosted on the digital ocean cloud platform.

With the previous cloud service platform, the client encountered limitations with regard to Linux installations, as the platform only supported predefined options. This restricted our flexibility in choosing custom Linux setups that would best suit our requirements.

As the platform expanded and demanded additional resources, it encountered performance problems and lacked extensive global coverage. Moreover, integrating with other services and ensuring data security posed challenges, ultimately impeding its ability to efficiently serve its customers.

We successfully addressed the challenges faced by Sweet Analytics by leveraging Amazon Elastic Kubernetes Service (EKS) alongside various other AWS and third-party services.

To begin with, Amazon EKS played a pivotal role in delivering a scalable and dependable solution for Sweet Analytics. It allowed us to effortlessly manage and coordinate containerized applications, facilitating a user-friendly platform for deployment, scaling, and management. The built-in features of Amazon EKS, such as high availability, scalability, and security, proved crucial for accommodating the growing demands of Sweet Analytics. Additionally, the seamless integration capabilities of Amazon EKS with other AWS services like Amazon RDS, AWS Lambda, and Amazon EKS contributed to the development of a fully integrated and secure platform.

Our team established a resilient and scalable infrastructure by implementing Karpenter for auto-scaling and Nginx ingress for load balancing. This ensured that the platform could handle substantial workloads and sudden spikes in traffic without encountering performance issues.

To ensure data security, we utilized AWS KMS for encryption and AWS IAM for access control, offering robust security features to safeguard customer data.

Furthermore, we implemented monitoring and alerting mechanisms using Grafana, Prometheus, and Loki, enabling proactive detection and resolution of issues.

Our team also constructed a CI/CD pipeline for each microservice within the application, enabling faster deployment cycles.

We assisted the client in creating the following modules:

1. Admin module: Provides access to offline reports and additional analytics.
2. E-commerce module: Includes the Sweet benchmarking index, offering market insights at the sector, sub-sector, and market levels.
3. Core module: Offers insights into discount analysis, shipping analysis, top-performing products, SKU analysis, sales analytics by dimension, and key metrics analysis.

Through these comprehensive solutions and modules, Sweet Analytics was equipped to overcome its challenges and enhance its platform’s performance and capabilities.

• By implementing Amazon EKS and deploying Sweet Analytics’ application across multiple availability zones (AZs) worldwide, we achieved a significant reduction of 30% in system latency. This improvement directly enhanced the overall user experience for Sweet Analytics’ customers, providing them with faster response times and smoother interactions with the platform.
• We introduced spot instances for the development environment as a cost optimization measure. 
• By utilizing spot instances, we achieved a significant cost reduction of over 50% specifically for the development environment. 

Amazon ECR

We leveraged Amazon ECR (Elastic Container Registry) to store our Docker images and seamlessly deploy them within our Kubernetes cluster. This allowed us to efficiently manage our containerized applications and streamline the deployment process.

Amazon EKS 

Furthermore, we utilized Amazon EKS (Elastic Kubernetes Service), a managed Kubernetes service, to deploy our microservices along with essential monitoring tools such as Grafana, Prometheus, and Loki. By leveraging the capabilities of Amazon EKS, we ensured a highly scalable and resilient infrastructure for our application, facilitating seamless management and monitoring of our services.

Grafana, Prometheus and Loki

We employed Amazon ECR and Amazon EKS not only for deploying applications but also for monitoring infrastructure and services, as well as viewing application logs. These tools enabled us to monitor several crucial data points pertaining to the infrastructure and applications, including:

• Number of running containers
• CPU percentage
• Cluster-level RAM usage
• Currently active services
• Node-level monitoring
• Network traffic
• Disk I/Os

By leveraging these monitoring tools, we were able to proactively identify and address issues before they could impact end-users. This proactive approach ensured a high level of performance and availability for the client’s microservices-based architecture, allowing for a seamless user experience.

Karpenter is an open-source Kubernetes add-on that automates instance scaling, enabling faster response to workload demands, optimizing resource allocation in our case study.

We used it to launch appropriate EC2 instances based on workload requirements in our Kubernetes cluster.

New relic

We utilized it for application lifecycle management (ALM).

AWS Lambda

We utilized AWS Lambda for image processing and analytics in event-driven workloads. This service allowed us to run custom code without the need to provision or manage servers, making it ideal for our processing needs.

PostgreSQL RDS

We utilized PostgreSQL RDS to store our analytical workload, which included data from ETL processing and 27 third-party integrations with services such as Shopify, Google, Facebook, Snapchat, TikTok, Amazon, Google Ads, and Hubspot. In addition, we used Apache Airflow as an application database. 

MongoDB Atlas

To manage our application data, we relied on MongoDB Atlas as our transactional database. It provided a scalable and flexible solution for storing and retrieving our application data.

Amazon S3

For storing application reports and as a data lake for our application data, we utilized Amazon S3. This service offered durable object storage with high scalability, making it suitable for our reporting and data storage needs.

AWS IAM

AWS IAM (Identity and Access Management) played a crucial role in our system by providing permission management at both the user and application levels. We leveraged OIDC (OpenID Connect) for cluster-level access sharing, ensuring secure and controlled access to our resources.

AWS KMS

To encrypt sensitive data stored in PostgreSQL RDS and S3, we employed AWS Key Management Service (KMS). It provided us with a secure and managed solution for key management and encryption.

Amazon CloudFront

To enhance our front-end content strategy and improve content delivery speed, we utilized Amazon CloudFront. This content delivery network (CDN) allowed us to cache and distribute content stored in S3, resulting in faster access and improved user experience.

AWS DMS (Database Migration Service)

We utilized AWS DMS to migrate our PostgreSQL database from DigitalOcean to Amazon RDS. AWS DMS simplifies and automates the process of migrating databases to AWS. It allows for seamless migration of data, schema, and ongoing replication from the source database to the target database. In our case, we used it specifically to migrate our PostgreSQL database from Digital Ocean to an Amazon RDS instance running PostgreSQL.

Self-hosted OpenVPN

We implemented a self-hosted OpenVPN inside an EKS cluster to establish secure connections between our local environment and the services running within our Amazon EKS (Elastic Kubernetes Service) cluster. This setup provided a secure and controlled way to access and interact with the internal services for testing and development purposes.

Amazon CloudWatch

We utilized Amazon CloudWatch for monitoring and managing our AWS resources, particularly for RDS (Amazon Relational Database Service). CloudWatch allows us to collect and track metrics, logs, and events related to our RDS instances. We leveraged CloudWatch to monitor the performance and health of our RDS databases, set up alarms for specific metrics, and gain insights into the database’s behavior. Additionally, we used CloudWatch Logs to centralize and analyze logs generated by our RDS instances, helping us troubleshoot issues and monitor database activity effectively.