Pixis: Elevating Data Collaboration with Pixis.ai in a Privacy-Focused World

Pixis.ai is a codeless AI infrastructure company that helps brands scale all aspects of their marketing and augment their decision-making in a world of infinitely complex consumer behavior.

• Scalability and Data Security Concerns – As Pixis.ai continued to grow, the limitations of their on-premise infrastructure became increasingly clear. Not only did this infrastructure lack the scalability to accommodate a burgeoning customer base, but it also posed significant data security risks. With an ever-increasing volume of consumer data to analyze and share, the existing system was ill-equipped to handle secure data collaboration requirements.
• Cost Inefficiencies and Data Management – The on-premise infrastructure was not only costly to maintain but also inefficient in handling secure data collaborations. These inefficiencies made it difficult for Pixis.ai to implement effective cost tracking and management for their privacy-enhanced data collaborations. Their lack of a centralized, cloud-based system led to operational bottlenecks and security vulnerabilities, making it challenging to collaborate securely and cost-effectively with external data providers.
• Lack of DevOps Practices for Secure Data Collaboration – Pixis.ai’s on-premise infrastructure inhibited the rapid provisioning of resources and posed limitations on implementing DevOps practices essential for privacy-enhanced data collaboration. The lack of containerization and continuous deployment options prevented Pixis from swiftly reacting to changing data security norms and collaboration requirements. 
• Transitioning to Privacy-First Infrastructure – The client had multiple infrastructure challenges, including high operational costs, a lack of standardized DevOps practices, and issues related to secure data collaboration. Their existing setup made it difficult to migrate services while ensuring that enhanced data security and consumer protection measures were in place. The overarching goal was to transition to an AWS-based infrastructure designed to not only be cost-effective and scalable but also prioritize data privacy and secure collaboration.

• Secure Data Collaboration Architecture – Simform collaborated with Pixis to design an architecture on AWS that would facilitate secure data collaboration, making it easy to manage and collaborate with data providers. This was achieved through the use of AWS EKS for containerization, allowing for isolated environments where data can be securely processed and shared. AWS EKS was selected for its built-in security features, including role-based access control and encryption at rest and in transit.
• Data Privacy and Cost Management – The team at Simform used AWS S3 with specific tagging policies to not only store static content, machine learning models, and datasets but also to enforce data governance and privacy policies. This cloud-based storage option was combined with cost-monitoring tools, aligning with Pixis’ need for privacy-enhanced data collaboration while keeping costs in check.
• Seamless Transition to a Secure Cloud Environment – To ensure a smooth transition to this more secure and collaborative environment, Simform employed AWS’s Database Migration Service and Server Migration Service. These services simplified the migration process while ensuring that data remained secure throughout, thereby enhancing Pixis’ ability to collaborate securely with external data providers.
• Elastic and Scalable Data Processing – AWS services such as EC2, Auto Scaling, and Elastic Load Balancing were integrated into the architecture. These services provided the required scalability and elasticity to manage large, fluctuating datasets, essential for real-time analytics and secure data sharing. Simform also suggested using que-based scaling solutions for background tasks and Redis for caching, facilitating real-time data processing securely and efficiently.
• Advanced Monitoring for Data Security – To ensure ongoing data security and facilitate secure collaboration, monitoring tools such as Grafana, Prometheus, and AWS CloudWatch were integrated. These tools helped Pixis proactively identify any security or performance issues before they could impact the collaborative data environment.

• Infrastructure Cost Savings – By migrating to AWS EKS and incorporating Simform’s cost-optimization strategies, Pixis reduced its infrastructure costs by 25%. This has not only achieved financial efficiency but also allows for better allocation of resources to secure data collaboration features.
• Uptime Improvement – System uptime has significantly improved from 95% to 99.996% after moving to AWS EKS. This near-perfect uptime ensures that secure data collaboration services are highly available, minimizing any disruption in secure data sharing and processing.
• Metrics for Success in Secure Development – Reduced Time-to-Market for Secure Features: After adopting AWS infrastructure, Pixis saw a 50% reduction in the time required to release new secure features, accelerating the rate at which privacy-centered enhancements can be rolled out.
• Increase in System Capacity for Data Handling – Post-migration, the system’s capacity increased by 75%, enabling a 3x increase in user load without any performance issues. This is crucial for managing large-scale secure data collaborations.

• Amazon RDS – We leveraged Amazon RDS as our primary storage solution for various types of data, including campaign data, ad accounts, cross-platform engagement scores, ML datasets, tenant data, and more. By using Amazon RDS, we were able to easily manage and scale our relational databases while ensuring high availability and durability. 
• MQ – In our solution, we utilized MQ with RabbitMQ as a message broker to enable communication and coordination between our microservices. We configured it to handle long-running background tasks, such as processing recommendation data, image and video rendering, and other computationally intensive tasks, allowing for smoother and more efficient workflow management. 
• AWS Trusted Advisor – In our solution, we are using Trusted Advisor to identify overprovisioned resources and improve our security posture. By regularly running Trusted Advisor checks, we are able to stay on top of potential issues and ensure that our infrastructure is optimized for performance, security, and cost.
• AWS CloudTrail – AWS CloudTrail enables auditing, security monitoring, and operational troubleshooting by tracking user activity and API usage. The AWS CloudTrail logs continuously, monitors, and retains account activity related to actions across our AWS infrastructure.
• AWS ECR – In our solution, we utilized AWS ECR to securely manage and scan the Docker images of our microservices. This helped us maintain the reliability and security of our system while ensuring the high performance of our services. 
• NAT gateway – In our solution, we used a NAT gateway to allow our resources in private subnets to securely access the internet and other AWS services without exposing them directly to the public internet. 
• AWS Lambda – We used AWS Lambda to trigger and run machine learning pipelines and alerts.
• Redis – We leveraged Redis to store user sessions, which allowed us to easily retrieve session data and provide a better user experience. Additionally, Redis helped us reduce data access latency by caching frequently accessed data in memory, which reduced the need for costly database queries. 
• Amazon EKS – We leveraged Amazon EKS to easily manage and scale our microservices and background jobs on a containerized infrastructure. EKS provided us with a managed Kubernetes environment, allowing us to focus on application development and deployment without worrying about the underlying infrastructure. We used EKS to easily deploy, manage, and scale our containerized applications and to automate container deployments and updates. 
• Amazon CloudWatch – We used AWS Cloudwatch to generate alarms and for application log generation and as a monitoring solution to monitor the resource utilization metrics.
• Amazon S3 buckets – We leveraged Amazon S3 buckets as a highly scalable and secure storage solution for storing various types of data in our system, including configuration files and customer data files. With S3’s ability to store and retrieve any amount of data from anywhere on the web, we were able to easily manage, secure, and retrieve files whenever required.
• AWS SecurityHub – We utilized Security Hub to get a comprehensive view of our security state in AWS and to ensure our environment adhered to security industry standards and best practices.
• AWS NLB – We utilized AWS NLB as our load balancer to distribute incoming traffic across multiple targets in different availability zones. This helped us achieve higher availability and fault tolerance for our application.
• AWS Cloudwatch alarm – In our solution, we set up CloudWatch alarms to monitor various metrics such as CPU utilization, memory usage, and network traffic for our AWS resources. Whenever a metric crossed a threshold, an alarm was triggered, and sent notifications to our team via email or SMS. 
• Amazon CloudWatch – In our solution, we used CloudWatch for logging and monitoring various AWS native services such as Amazon RDS. We were able to set up customized metrics, dashboards, and alarms to monitor the health of our infrastructure and quickly respond to any issues.
• AWS secrets manager – In our solution, we used AWS Secrets Manager to store and manage the secret data of our microservices, including database credentials, API keys, and other sensitive information. This helped us keep our secrets secure and easily manage and retrieve them when needed.
• Fluent bit – We deployed Fluent Bit as our log processor and forwarder to collect, process, and forward logs from our microservices to AWS CloudWatch. This helped us gain real-time visibility into our application and infrastructure logs and make informed decisions based on the insights provided by the log data.
• Grafana – We used them for infrastructure and service monitoring. Using these tools, we help the client monitor various data points related to the infrastructure and applications, such as:
• Number of containers running
• ML pipeline status
• CPU percentage
• RAM usage at a cluster level
• Services running currently
• Node-level monitoring
• Network traffic
• Disk I/Os
• Using these monitoring tools, our solution can proactively identify and resolve issues before they impact end-users, ensuring a high level of performance and availability for the client’s microservices-based architecture.
• AWS DMS – Simform leveraged AWS Database Migration Service (DMS) to efficiently migrate Pixis’ databases from their on-premise environment to AWS. 
• AWS SMS – Simform employed AWS Server Migration Service (SMS) to streamline the migration of Pixis’ applications and servers to the AWS infrastructure.