PyTorch教程-4.3. 基本分类模型

您可能已经注意到，在回归的情况下，从头开始的实现和使用框架功能的简洁实现非常相似。分类也是如此。由于本书中的许多模型都处理分类，因此值得添加专门支持此设置的功能。本节为分类模型提供了一个基类，以简化以后的代码。

import torch
from d2l import torch as d2l

from mxnet import autograd, gluon, np, npx
from d2l import mxnet as d2l

npx.set_np()

from functools import partial
import jax
import optax
from jax import numpy as jnp
from d2l import jax as d2l

No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)

import tensorflow as tf
from d2l import tensorflow as d2l

4.3.1. 类Classifier_

我们在下面定义Classifier类。在中，validation_step我们报告了验证批次的损失值和分类准确度。我们为每个批次绘制一个更新num_val_batches 。这有利于在整个验证数据上生成平均损失和准确性。如果最后一批包含的示例较少，则这些平均数并不完全正确，但我们忽略了这一微小差异以保持代码简单。

class Classifier(d2l.Module): #@save
  """The base class of classification models."""
  def validation_step(self, batch):
    Y_hat = self(*batch[:-1])
    self.plot('loss', self.loss(Y_hat, batch[-1]), train=False)
    self.plot('acc', self.accuracy(Y_hat, batch[-1]), train=False)

We define the Classifier class below. In the validation_step we report both the loss value and the classification accuracy on a validation batch. We draw an update for every num_val_batches batches. This has the benefit of generating the averaged loss and accuracy on the whole validation data. These average numbers are not exactly correct if the last batch contains fewer examples, but we ignore this minor difference to keep the code simple.

class Classifier(d2l.Module): #@save
  """The base class of classification models."""
  def validation_step(self, batch):
    Y_hat = self(*batch[:-1])
    self.plot('loss', self.loss(Y_hat, batch[-1]), train=False)
    self.plot('acc', self.accuracy(Y_hat, batch[-1]), train=False)

We define the Classifier class below. In the validation_step we report both the loss value and the classification accuracy on a validation batch. We draw an update for every num_val_batches batches. This has the benefit of generating the averaged loss and accuracy on the whole validation data. These average numbers are not exactly correct if the last batch contains fewer examples, but we ignore this minor difference to keep the code simple.

We also redefine the training_step method for JAX since all models that will subclass Classifier later will have a loss that returns auxiliary data. This auxiliary data can be used for models with batch normalization (to be explained in Section 8.5), while in all other cases we will make the loss also return a placeholder (empty dictionary) to represent the auxiliary data.

class Classifier(d2l.Module): #@save
  """The base class of classification models."""
  def training_step(self, params, batch, state):
    # Here value is a tuple since models with BatchNorm layers require
    # the loss to return auxiliary data
    value, grads = jax.value_and_grad(
      self.loss, has_aux=True)(params, batch[:-1], batch[-1], state)
    l, _ = value
    self.plot("loss", l, train=True)
    return value, grads

  def validation_step(self, params, batch, state):
    # Discard the second returned value. It is used for training models
    # with BatchNorm layers since loss also returns auxiliary data
    l, _ = self.loss(params, batch[:-1], batch[-1], state)
    self.plot('loss', l, train=False)
    self.plot('acc', self.accuracy(params, batch[:-1], batch[-1], state),
         train=False)

We define the Classifier class below. In the validation_step we report both the loss value and the classification accuracy on a validation batch. We draw an update for every num_val_batches batches. This has the benefit of generating the averaged loss and accuracy on the whole validation data. These average numbers are not exactly correct if the last batch contains fewer examples, but we ignore this minor difference to keep the code simple.

class Classifier(d2l.Module): #@save
  """The base class of classification models."""
  def validation_step(self, batch):
    Y_hat = self(*batch[:-1])
    self.plot('loss', self.loss(Y_hat, batch[-1]), train=False)
    self.plot('acc', self.accuracy(Y_hat, batch[-1]), train=False)

默认情况下，我们使用随机梯度下降优化器，在小批量上运行，就像我们在线性回归的上下文中所做的那样。

@d2l.add_to_class(d2l.Module) #@save
def configure_optimizers(self):
  return torch.optim.SGD(self.parameters(), lr=self.lr)

@d2l.add_to_class(d2l.Module) #@save
def configure_optimizers(self):
  params = self.parameters()
  if isinstance(params, list):
    return d2l.SGD(params, self.lr)
  return gluon.Trainer(params, 'sgd', {'learning_rate': self.lr})

@d2l.add_to_class(d2l.Module) #@save
def configure_optimizers(self):
  return optax.sgd(self.lr)

@d2l.add_to_class(d2l.Module) #@save
def configure_optimizers(self):
  return tf.keras.optimizers.SGD(self.lr)

4.3.2. 准确性

给定预测概率分布y_hat，每当我们必须输出硬预测时，我们通常会选择预测概率最高的类别。事实上，许多应用程序需要我们做出选择。例如，Gmail 必须将电子邮件分类为“主要”、“社交”、“更新”、“论坛”或“垃圾邮件”。它可能会在内部估计概率，但最终它必须在类别中选择一个。

当预测与标签 class 一致时y，它们是正确的。分类准确度是所有正确预测的分数。尽管直接优化精度可能很困难（不可微分），但它通常是我们最关心的性能指标。它通常是基准测试中的相关数量。因此，我们几乎总是在训练分类器时报告它。

准确度计算如下。首先，如果y_hat是一个矩阵，我们假设第二个维度存储每个类别的预测分数。我们使用argmax每行中最大条目的索引来获取预测类。然后我们将预测的类别与真实的元素进行比较y。由于相等运算符== 对数据类型敏感，因此我们转换 的y_hat数据类型以匹配 的数据类型y。结果是一个包含条目 0（假）和 1（真）的张量。求和得出正确预测的数量。

@d2l.add_to_class(Classifier) #@save
def accuracy(self, Y_hat, Y, averaged=True):
  """Compute the number of correct predictions."""
  Y_hat = Y_hat.reshape((-1, Y_hat.shape[-1]))
  preds = Y_hat.argmax(axis=1).type(Y.dtype)
  compare = (preds == Y.reshape(-1)).type(torch.float32)
  return compare.mean() if averaged else compare

@d2l.add_to_class(Classifier) #@save
def accuracy(self, Y_hat, Y, averaged=True):
  """Compute the number of correct predictions."""
  Y_hat = Y_hat.reshape((-1, Y_hat.shape[-1]))
  preds = Y_hat.argmax(axis=1).astype(Y.dtype)
  compare = (preds == Y.reshape(-1)).astype(np.float32)
  return compare.mean() if averaged else compare

@d2l.add_to_class(d2l.Module) #@save
def get_scratch_params(self):
  params = []
  for attr in dir(self):
    a = getattr(self, attr)
    if isinstance(a, np.ndarray):
      params.append(a)
    if isinstance(a, d2l.Module):
      params.extend(a.get_scratch_params())
  return params

@d2l.add_to_class(d2l.Module) #@save
def parameters(self):
  params = self.collect_params()
  return params if isinstance(params, gluon.parameter.ParameterDict) and len(
    params.keys()) else self.get_scratch_params()

@d2l.add_to_class(Classifier) #@save
@partial(jax.jit, static_argnums=(0, 5))
def accuracy(self, params, X, Y, state, averaged=True):
  """Compute the number of correct predictions."""
  Y_hat = state.apply_fn({'params': params,
              'batch_stats': state.batch_stats}, # BatchNorm Only
              *X)
  Y_hat = Y_hat.reshape((-1, Y_hat.shape[-1]))
  preds = Y_hat.argmax(axis=1).astype(Y.dtype)
  compare = (preds == Y.reshape(-1)).astype(jnp.float32)
  return compare.mean() if averaged else compare

@d2l.add_to_class(Classifier) #@save
def accuracy(self, Y_hat, Y, averaged=True):
  """Compute the number of correct predictions."""
  Y_hat = tf.reshape(Y_hat, (-1, Y_hat.shape[-1]))
  preds = tf.cast(tf.argmax(Y_hat, axis=1), Y.dtype)
  compare = tf.cast(preds == tf.reshape(Y, -1), tf.float32)
  return tf.reduce_mean(compare) if averaged else compare

4.3.3. 概括

分类是一个足够普遍的问题，它保证了它自己的便利功能。分类中最重要的是 分类器的准确性。请注意，虽然我们通常主要关心准确性，但出于统计和计算原因，我们训练分类器以优化各种其他目标。然而，无论在训练过程中哪个损失函数被最小化，有一个方便的方法来根据经验评估我们的分类器的准确性是有用的。

4.3.4. 练习

表示为Lv验证损失，让Lvq是通过本节中的损失函数平均计算的快速而肮脏的估计。最后，表示为lvb最后一个小批量的损失。表达Lv按照Lvq, lvb，以及样本和小批量大小。

表明快速而肮脏的估计Lvq是公正的。也就是说，表明E[Lv]=E[Lvq]. 为什么你还想使用Lv反而？

给定多类分类损失，表示为l(y,y′) 估计的惩罚y′当我们看到y并给出一个概率p(y∣x), 制定最佳选择规则y′. 提示：表达预期损失，使用 l和p(y∣x).