Continuously Train + Deploy TensorFlow Models in Prod (Kafka + Kubernetes + GPU)
Note : This will be a LiveStream from our sister meetup in the Bay Area to here in Vancouver: https://www.meetup.com/Advanced-Spark-and-TensorFlow-Meetup/events/247141015/. Questions will be taken from the audience _live_here in Vancouver and relayed back!
Agenda
6:00pm: Doors Open
6:30pm: Talks
Talk 0: Meetup Updates and Announcements by Chris Fregly, Founder & Engineer @ PipelineAI
Talk 1: Continuous Training and Deploying of High Performance, Serverless TensorFlow Models in Production with Jupyter, TensorFlow, Scikit-Learn, Kafka, Kubernetes, Istio, OpenFaaS, Prometheus, Grafana, Slack, and GPUs
Abstract
Using the latest advancements in real-time AI from the open source PipelineAI project, I will demonstrate how to continuously train and deploy GPU-based TensorFlow models using live streaming data on a hybrid-cloud Kubernetes cluster.
Streaming data is generated in real-time from the audience using a Slack to crowd-source the data labeling. This newly-labeled data automatically generates new model variants.
I will use OpenFaaS and Istio with Kubernetes to quickly - and safely - deploy the new model variants to live production traffic.
Similar to canary deployments of classic microservices, the new model variants are deployed safely to production in a controlled manner. Initially, they are exposed to only a small amount of traffic.
Using reinforcement learning, multi-armed bandits, and metrics from Prometheus/Grafana, live traffic is automatically routed to the winning models based on a given reward function such as MAXIMIZE(number of signups) or MINIMIZE(cost per prediction).
All demos run on a hybrid-cloud, open source, GPU-based, Kubernetes cluster optimized for the machine learning and artificial intelligence use cases that we commonly see at PipelineAI.
Bio
Chris Fregly is Founder and Applied AI Engineer at PipelineAI, a Real-Time Machine Learning and Artificial Intelligence Startup based in San Francisco. He is also an Apache Spark Contributor, a Netflix Open Source Committer, founder of the Global Advanced Spark and TensorFlow Meetup, author of the O'Reilly Training and Video Series titled, "High Performance TensorFlow in Production with Kubernetes and GPUs."
Previously, Chris was a Distributed Systems Engineer at Netflix, a Data Solutions Engineer at Databricks, and a Founding Member and Principal Engineer at the IBM Spark Technology Center in San Francisco.
Talk 2: Alpha Go Zero/AlphaZero with TensorFlow, Probabilistic Methods, and Neural Network Techniques by Brett Koonce (https://www.linkedin.com/in/brettkoonce), CTO @ Quarkworks (https://quarkworks.net/)
* https://deepmind.com/blog/alphago-zero-learning-scratch/
In this talk, I will discuss how Google built Alpha Go Zero/AlphaZero to master the game of Go by combining probabilistic methods, neural network techniques and Tensorflow.