Data Callbacks

Open In Colab

Callbacks which work with a learner’s data

  1. /usr/local/lib/python3.8/dist-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  /pytorch/c10/cuda/CUDAFunctions.cpp:100.)
  2.   return torch._C._cuda_getDeviceCount() > 0

class CollectDataCallback[source]

CollectDataCallback(after_create=None, before_fit=None, before_epoch=None, before_train=None, before_batch=None, after_pred=None, after_loss=None, before_backward=None, before_step=None, after_cancel_step=None, after_step=None, after_cancel_batch=None, after_batch=None, after_cancel_train=None, after_train=None, before_validate=None, after_cancel_validate=None, after_validate=None, after_cancel_epoch=None, after_epoch=None, after_cancel_fit=None, after_fit=None) :: Callback

Collect all batches, along with pred and loss, into self.data. Mainly for testing

class CudaCallback[source]

CudaCallback(device=None) :: Callback

Move data to CUDA device

You don’t normally need to use this Callback, because fastai’s DataLoader will handle passing data to a device for you. However, if you already have a plain PyTorch DataLoader and can’t change it for some reason, you can use this transform.

  1. learn = synth_learner(cbs=CudaCallback)
  2. learn.model
  3. learn.fit(1)
  4. test_eq(next(learn.model.parameters()).device.type, 'cuda')
  1. [0, 6.35821008682251, 4.982691287994385, '00:00']

class WeightedDL[source]

WeightedDL(dataset=None, bs=None, wgts=None, shuffle=False, num_workers=None, verbose=False, do_setup=True, pin_memory=False, timeout=0, batch_size=None, drop_last=False, indexed=None, n=None, device=None, persistent_workers=False, wif=None, before_iter=None, after_item=None, before_batch=None, after_batch=None, after_iter=None, create_batches=None, create_item=None, create_batch=None, retain=None, get_idxs=None, sample=None, shuffle_fn=None, do_batch=None) :: TfmdDL

Transformed DataLoader

Datasets.weighted_dataloaders[source]

Datasets.weighted_dataloaders(wgts, bs=64, shuffle_train=None, shuffle=True, val_shuffle=False, n=None, path='.', dl_type=None, dl_kwargs=None, device=None, drop_last=None, val_bs=None)

  1. n = 160
  2. dsets = Datasets(torch.arange(n).float())
  3. dls = dsets.weighted_dataloaders(wgts=range(n), bs=16)
  4. learn = synth_learner(data=dls, cbs=CollectDataCallback)
  1. learn.fit(1)
  2. t = concat(*learn.collect_data.data.itemgot(0,0))
  3. plt.hist(t.numpy());
  1. [0, nan, None, '00:01']

Data - 图2

class PartialDL[source]

PartialDL(dataset=None, bs=None, partial_n=None, shuffle=False, num_workers=None, verbose=False, do_setup=True, pin_memory=False, timeout=0, batch_size=None, drop_last=False, indexed=None, n=None, device=None, persistent_workers=False, wif=None, before_iter=None, after_item=None, before_batch=None, after_batch=None, after_iter=None, create_batches=None, create_item=None, create_batch=None, retain=None, get_idxs=None, sample=None, shuffle_fn=None, do_batch=None) :: TfmdDL

Select randomly partial quantity of data at each epoch

FilteredBase.partial_dataloaders[source]

FilteredBase.partial_dataloaders(partial_n, bs=64, shuffle_train=None, shuffle=True, val_shuffle=False, n=None, path='.', dl_type=None, dl_kwargs=None, device=None, drop_last=None, val_bs=None)

Create a partial dataloader PartialDL for the training set

  1. dls = dsets.partial_dataloaders(partial_n=32, bs=16)
  1. assert len(dls[0])==2
  2. for batch in dls[0]:
  3.     assert len(batch[0])==16

Company logo

©2021 fast.ai. All rights reserved.
Site last generated: Mar 31, 2021