module

Base class for all neural network modules.

commen usage

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
	def __init__(self):
		super().__init__()
		self.conv1 = nn.Conv2d(1,20,5)
		self.conv2 = nn.Conv2d(20,20,5)
		
	def forward(self,x):
		x = F.relu(self.conv1(x))
		return F.relu(self.conv2(x))

to()

example:

model = network()
device = torch.device('cuda')
model.to(device)
# or 
# model = model.to(device)

this will moves the model to the device, usually on GPU.

model.eval() & with torch.no_grad()

these have different goals:

• model.eval() will notify all your layers that you are in eval mode, that way, batchnorm or dropout will not work in the eval mode.
• torch.no_grad() impacts the autograd engine and deactivate it.
• the commen practice for evaluating/validation is using with torch.no_grad() in pair with model.eval() :

  model.eval()
  
  with torch.no_grad():
  	...
  	output = model(data)
  	...
  	
  #training step 
  model.train()

  and after that don’t forget to turn back to training mode using model.train()