Spark机器学习

Preface
Chapter 1: Getting Up and Running with Spark
 Installing and setting up Spark locally
 Spark clusters
 The Spark programming model
 SparkContext and SparkConf
 The Spark shell
 Resilient Distributed Datasets
 Creating RDDs
 Spark operations
 Caching RDDs
 Broadcast variables and accumulators
 The first step to a Spark program in Scala
 The first step to a Spark program in Java
 The first step to a Spark program in Python
 Getting Spark running on Amazon EC2
 Launching an EC2 Spark cluster
 Summary
Chapter 2: Designing a Machine Learning System
 Introducing MovieStream
 Business use cases for a machine learning system
 Personalization
 Targeted marketing and customer segmentation
 Predictive modeling and analytics
 Types of machine learning models
 The components of a data-driven machine learning system
 Data ingestion and storage
 Data cleansing and transformation
 Model training and testing loop
 Model deployment and integration
 Model monitoring and feedback
 Batch versus real time
 An architecture for a machine learning system
 Practical exercise
 Summary
Chapter 3: Obtaining, Processing, and Preparing Data
with Spark
 Accessing publicly available datasets
 The MovieLens lOOk dataset
 Exploring and visualizing your data
 Exploring the user dataset
 Exploring the movie dataset
 Exploring the rating dataset
 Processing and transforming your data
 Filling in bad or missing data
 Extracting useful features from your data
 Numerical features
 Categorical features
 Derived features
 Transforming timestamps into categorical features
 Text features
 Simple text feature extraction
 Normalizing features
 Using MLlib for feature normalization
 Using packages for feature extraction
 Summary
Chapter 4: Building a Recommendation Engine with Spark
 Types of recommendation models
 Content-based filtering
 Collaborative filtering
 Matrix factorization
 Extracting the right features from your data
 Extracting features from the MovieLens 100k dataset
 Training the recommendation model
 Training a model on the MovieLens 100k dataset
 Training a model using implicit feedback data
 Using the recommendation model
 User recommendations
 Generating movie recommendations from the MovieLens 100k dataset
 Item recommendations
 Generating similar movies for the MovieLens 100k dataset
 Evaluating the performance of recommendation models
 Mean Squared Error
 Mean average precision at K
 Using MLlib's built-in evaluation functions
 RMSE and MSE
 MAP
 Summary
Chapter 5: Building a Classification Model with Spark
 Types of classification models
 Linear models
 Logistic regression
 Linear support vector machines
 The na'fve Bayes model
 Decision trees
 Extracting the right features from your data
 Extracting features from the Kaggle/StumbleUpon
 evergreen classification dataset
 Training classification models
 Training a classification model on the Kaggle/StumbleUpon
 evergreen classification dataset
 Using classification models
 Generating predictions for the Kaggle/StumbleUpon
 evergreen classification dataset
 Evaluating the performance of classification models
 Accuracy and prediction error
 Precision and recall
 ROC curve and AUC
 Improving model performance and tuning parameters
 Feature standardization
 Additional features
 Using the correct form of data
 Tuning model parameters
 Linear models
 Decision trees
 The na'fve Bayes model
 Cross-validation
 Summary
Chapter 6: Buildin a~ssion Model with Spark
 Types of regression models
 Least squares regression
 Decision trees for regression
 Extracting the right features from your data
 Extracting features from the bike sharing dataset
 Creating feature vectors for the linear model
 Creating feature vectors for the decision tree
 Training and using regression models
 Training a regression model on the bike sharing dataset
 Evaluating the performance of regression models
 Mean Squared Error and Root Mean Squared Error
 Mean Absolute Error
 Root Mean Squared Log Error
 The R-squared coefficient
 Computing performance metrics on the bike sharing dataset
 Linear model
 Decision tree
 Improving model performance and tuning parameters
 Transforming the target variable
 Impact of training on log-transformed targets
 Tuning model parameters
 Creating training and testing sets to evaluate parameters
 The impact of parameter settings for linear models
 The impact of parameter settings for the decision tree
 Summary
Chapter 7: Building a Clustering Model with Spark
 Types of clustering models
 K-means clustering
 Initialization methods
 Variants
 Mixture models
 Hierarchical clustering
 Extracting the right features from your data
 Extracting features from the MovieLens dataset
 Extracting movie genre labels
 Training the recommendation model
 Normalization
 Training a clustering model
 Training a clustering model on the MovieLens dataset
 Making predictions using a clustering model
 Interpreting cluster predictions on the MovieLens dataset
 Interpreting the movie clusters
 Evaluating the performance of clustering models
 Internal evaluation metrics
 External evaluation metrics
 Computing performance metrics on the MovieLens dataset
 Tuning parameters for clustering models
 Selecting K through cross-validation
 Summary
Chapter 8: Dimensionality Reduction with Spark
 Types of dimensionality reduction
 Principal Components Analysis
 Singular Value Decomposition
 Relationship with matrix factorization
 Clustering as dimensionality reduction
 Extracting the right features from your data
 Extracting features from the LFW dataset
 Exploring the face data
 Visualizing the face data
 Extracting facial images as vectors
 Normalization
 Training a dimensionality reduction model
 Running PCA on the LFW dataset
 Visualizing the Eigenfaces
 Interpreting the Eigenfaces
 Using a dimensionality reduction model
 Projecting data using PCA on the LFW dataset
 The relationship between PCA and SVD
 Evaluating dimensionality reduction models
 Evaluating k for SVD on the LFW dataset
 Summary
Chapter 9: Advanced Text Processing with Spark
 What's so special about text data?
 Extracting the right features from your data
 Term weighting schemes
 Feature hashing
 Extracting the TF-IDF features from the 20 Newsgroups dataset
 Exploring the 20 Newsgroups data
 Applying basic tokenization
 Improving our tokenization
 Removing stop words
 Excluding terms based on frequency
 A note about stemming
 Training a TF-IDF model
 Analyzing the TF-IDF weightings
 Using a TF-IDF model
 Document similarity with the 20 Newsgroups dataset and
 TF-IDF features
 Training a text classifier on the 20 Newsgroups dataset
 using TF-IDF
 Evaluating the impact of text processing
 Comparing raw features with processed TF-IDF features on the
 20 Newsgroups dataset
 Word2Vec models
 Word2Vec on the 20 Newsgroups dataset
 Summary
Chapter 10: Real-time Machine Learning withSpark Streaming
 Online learning
 Stream processing
 An introduction to Spark Streaming
 Input sources
 Transformations
 Actions
 Window operators
 Caching and fault tolerance with Spark Streaming
 Creating a Spark Streaming application
 The producer application
 Creating a basic streaming application
 Streaming analytics
 Stateful streaming
 Online learning with Spark Streaming
 Streaming regression
 A simple streaming regression program
 Creating a streaming data producer
 Creating a streaming regression model
 Streaming K-means
 Online model evaluation
 Comparing model performance with Spark Streaming
 Summary
Index