Utilizing AWS to build a scalable and cost-effective application at AppHarvest

Background

AppHarvest is a sustainable food company in Appalachia that is developing and operating some of the world’s largest high-tech indoor farms with robotics and artificial intelligence with the goal of building a reliable, climate-resilient food system. AppHarvest’s farms are designed to grow produce using sunshine, rainwater, and up to 90% less water than open-field growing, all while producing yields up to 30 times that of traditional agriculture and preventing pollution from agricultural runoff. AppHarvest currently operates a four-farm network that consists of 165 acres under glass, including: 

  • Its 60-acre flagship tomato farm in Morehead, Kentucky

  • A 15-acre salad greens farm in Berea, Kentucky 

  • A 30-acre strawberries and cucumbers farm in Somerset, Kentucky

  • A 60-acre tomato farm in Richmond, Kentucky

Challenge

AppHarvest's harvesting robots generate a tremendous amount of data in the course of regular operations. All of this data is uploaded, transformed, and distributed to AppHarvest's employees and management. When Keyrus initially engaged with AppHarvest, the analytics process combined bash scripts, Python scripts, manual steps, and manipulation with Excel; the final Excel workbooks were shared via email with technical and executive leadership.

Much of the code was kept in a source control system; the engineer tasked with creating analytics for a production run would be required to pull the latest revision of the code and follow the instructions manually while looking for potential problems in each step, which took 2-3 hours of engineering time, typically over the course of more than a day.

Approach

Keyrus built an entirely new stack for data visualization, with Redshift and QuickSight at its core. The entire process for transforming data was transformed into microservices running on Amazon ECS based on the original scripts. The code remained in the same git repository, but Keyrus worked with AppHarvest to create a full CI/CD workflow for modifying the microservices, comprised of AWS CodePipeline, AWS CodeBuild, Amazon Elastic Container Registry, and AWS CodeDeploy. In this manner, the running analytics pipeline can be modified by simply merging changes into the main branch in git. In addition, the actual execution of the analytics can be automated, allowing the engineering and analytics team to focus on actual development rather than execution.

Key Results

  1. Conversion of data analytics processes to microservices reduced the time required to process a production run by more than 60%.

  2. Changes to the analytics code can be deployed in minutes with a simple code merge.

  3. Processing all data automatically enables processing in smaller batches, eventually enabling real-time analytics.

Benefits

After working with Keyrus, the engineers at AppHarvest no longer have to manage updating or babysitting their analytics processes and can focus on better analytics to optimize crop growth, improve yields, and reduce costs.