Model engineering

cross-industry standard process for data mining

to track and evaluate model performance against dynamic data, and ensure the model remains relevant Real world data are dynamic and changing over time. It also helps to identify model performance decay and give an early notification to the data science team in order to rebuild the model.

• lower quartile • maximum • median • minimum • upper quartile for numeric data

It is an iterative, adaptive and incremental process that allows the data science team looping back to previous steps at any stage of the project.

baseline reference point for demonstrating to stakeholders whether the additional modeling process adds value, and whether the newly built model is worth putting into deployment

Human Level Performance accuracy / error rate when the task is handled by human workforce

Tensorflow Extended

Tensorflow transform w/ data preprocessing components that can be embedded into a TFX project

TensorFlow model analysis library for evaluating TensorFlow models

batch scoring: model prediction and inference executed for large numbers of data points, triggered by a schedule or a specific event real-time scoring: predictions served almost instantly after requests arrive from clients

data version control

information of model name, version ID, model registry location, model input, and output directories

It ensures that changes to the models are tracked properly, and previously generated models are reproducible. It helps to increase the efficiency of team collaboration and allows continuous model experimentations and improvements.

Equivalence Class Transformation — the original expansion (Zaki et al., 1997); the form used across most of the primary literature, and a clean letter-fit (E·CLA·T). backronym: Equivalence Class Clustering and bottom-up Lattice Traversal What it is: a depth-first algorithm that finds frequent itemsets from a vertical layout (each item → its TID-set / tidset), computing support by intersecting TID-sets instead of rescanning the database on every pass the way Apriori does.

frequent pattern proposed by Han, Pei & Yin (2000) In the first pass, the algorithm counts item frequencies and stores them in a header table. In the second pass, it builds a compressed FP-tree by inserting transactions in descending frequency order. Frequent itemsets are then mined by recursively constructing conditional FP-trees — without candidate generation.

State–Action–Reward–State–Action on-policy, model-free, temporal-difference algorithm for learning Q The name is read directly off the quintuple (st, at, rt+1, st+1, at+1) consumed by its update rule: Q(st, at) ← Q(st, at) + α [ rt+1 + γ Q(st+1, at+1) − Q(st, at) ]

Overfitting results in high variance models, with low bias in training data and high bias in test data; underfitting results in low variance models, with high bias in both training and test data.

Leave One Out Cross Validation

Advantage: generates more reliable model evaluation because it performs multiple iterations of training and testing on different portions of sample data. Disadvantage: time-consuming.

stratified K-fold cross validation

K

Interpretable models help the debugging and reasoning process of model predictions. more effective communication w/ stakeholders w/ different reqs More importantly, building interpretable models helps to build trust with end users and decision makers by providing transparency and visibility of the model’s internal process.

class activation map

local interpretable model-agnostic explanations