Layers

The Layers module contains set of convolutional neural network units, which can be used to create custom neural network architectures.

Modules

Conv3x3

This is a simple conv 3x3 layer.

Parameters:

in_planes (int,required): The number of channels in the input feature map.
out_planes (int,requied): The number of channels in the output feature map.
stride (int, default = 1): The stride used in each convolution filter.
groups (int, default = 1): The number of groups to be considered while performing convolution.
dilation (int, default = 1): The field of view to be considered while performing convolution.
bias (int, default = False)

Usage:

from niftytorch.layers.layers import conv3x3
import torch
in_planes = 512
input = torch.rand(32,in_planes,32,32,32)
convolution = conv3x3(in_planes = in_planes,out_planes = 32,stride = 1,groups = 1,dilation = 1,bias = False)
output = convolution(input)

Conv1x1

This is a simple conv 1x1 layer.

Parameters:

in_planes (int,required): The number of channels in the input feature map.
out_planes (int,requied): The number of channels in the output feature map.
stride (int, default = 1): The stride used in each convolution filter.
groups (int, default = 1): The number of groups to be considered while performing convolution.
bias (int, default = False)

Usage:

from niftytorch.layers.layers import conv1x1
import torch
in_planes = 512
input = torch.rand(32,in_planes,32,32,32)
convolution = conv1x1(in_planes = in_planes,out_planes = 32,stride = 1,groups = 1,bias = False)
output = convolution(input)

Basic Block

BasicBlock is a series of convolution, batchnorm, activation layers with one jump skip connection.

The structure of this block is as follows:

input -> conv1 -> bn1 -> relu -> conv2 -> bn2 + input -> relu : output

Parameters:

inplanes (int,required): The number of channels in the input feature map.
planes (int,required): The number of channels in the output feature map.
stride (int,default = 1): The number of strides to be used in the conv1
downsample (int,default = None): The downsampler to be used to reduce the feature map size F.interpolate
groups (int,default = 1): The number of groups to be considered in convolution
dilation (int, default = 1): The field of view to be considered while performing convolution.
norm_layer (int,default = None): The normalization layer to be used ex: nn.BatchNorm()

Usage:

from niftytorch.layers.layers import basicblock
import torch
in_planes = 512
input = torch.rand(32,in_planes,32,32,32)
convolution_block = basicblock(in_planes = in_planes,planes = 256,out_planes = 32,stride = 1,groups = 1,bias = False)
output = convolution_block(input)

BottleNeck Block

BottleNeck is a series of convolution,batchnorm,activation layers with two layer jump skip connection.

The structure of this block is as follows:

input -> conv1 -> bn1 -> relu -> conv2 -> bn2 -> relu -> conv3 -> bn3 + input -> relu : output

Parameters:

inplanes (int,required): The number of channels in the input feature map.
planes (int,required): The number of channels in the output feature map.
stride (int,default = 1): The number of strides to be used in the conv1
downsample (int,default = None): The downsampler to be used to reduce the feature map size F.interpolate
groups (int,default = 1): The number of groups to be considered in convolution.
base_width (int,default = 1): The base width and expansion are used to calculate the number of filters for conv2 in bottleneck block.
dilation (int, default = 1): The field of view to be considered while performing convolution.
norm_layer (int,default = None): The normalization layer to be used ex: nn.BatchNorm()
expansion (int,default = 1):The base width and expansion are used to calculate the number of filters for conv2 in bottleneck block.

Usage:

from niftytorch.layers.layers import bottleneck
import torch
in_planes = 512
planes = 256
input = torch.rand(32,in_planes,32,32,32)
convolution_block = bottleneck(inplanes = in_planes, planes = planes, stride=1, downsample=None, groups=1,base_width=64, dilation=1, norm_layer=None,expansion = 1)
output = convolution_block(input)

ShuffleUnit

ShuffleUnit is the class definition for ShuffleUnit Module, which uses channel shuffling and combination to improve the network performance with fewer parameters.
The ShuffleUnit is commonly used in ShuffleNet 1.0 and ShuffleNet 2.0, but they can be used along with any network. For more information, please read the ShuffleNet papers [ShuffleNet 1.0, ShuffleNet 2.0].

Parameters:

in_channels (int,required): number of channels in the input tensor the shuffleunit.
out_channels (int,required): number of channels in the tensor generated from the shuffleunit.
groups (int,default = 3): define number of groups in which the filters are combined before shuffle operation.
grouped_conv (bool,default = True): defines whether grouped_convolution is to be used or not.
combine ('add' or 'concat',default = 'add'): defines the method in which we can combine the channels it has two options add or concat.
compresstion_ratio (int,default = 4): The number of channels to be required in the bottleneck channels.

Usage:

from niftytorch.layers.layers import shuffleunit
import torch
in_channels = 512
out_channels = 256
input = torch.rand(32,in_planes,32,32,32)
convolution_block = shuffleunit(in_channels = in_channels, out_channels = out_channels, groups=5, grouped_conv=True, combine = 'concat',compression_ratio = 4)
output = convolution_block(input)

Fire Module

Fire is the class definition for Fire Module, which uses a combination of 1x1 and 3x3 filters to improve the network performance with fewer parameters.
The Fire Module is commonly used in Squeezenet 1.0 and SqueezeNet 2.0 but they can used along with any network. For more information please read the SqueezeNet paper.

Parameters:

in_channels (int,required): number of channels in the input tensor the shuffleunit.
out_channels (int,required): number of channels in the tensor generated from the shuffleunit.
groups (int,default = 3): define number of groups in which the filters are combined before shuffle operation.
grouped_conv (bool,default = True): defines whether grouped_convolution is to be used or not.
combine ('add' or 'concat',default = 'add'): defines the method in which we can combine the channels it has two options add or concat.
compresstion_ratio (int,default = 4): The number of channels to be required in the bottleneck channels.

Usage:

from niftytorch.layers.layers import fire
import torch
in_planes = 20
input = torch.rand(32,20,32,32,32)
convolution_block = fire(inplanes = in_planes, squeeze_planes = 3, expand1x1_planes = 12, expand3x3_planes = 12)
output = convolution_block(input)

BinActive

The BinActive class is for calling binary activation method, which gives the sign of the input tensor as an activation.
This is used in Binary Networks and XNOR Network.

Usage:

import torch
from niftytorch.layers.layers import binactive
input = torch.ones(32,512,32,32,32)
activation = binactive()
output = activation(input)

BinConv3d

Parameters:

input_channels (int,required): The number of channels in the input tensor.
output_channels (int,required): The number of channels in the output tensor.
kernel_size (int,default = 3): Kernel size for the convolution filter.
stride (int,default = 1): The number of strides in the convolutional filters.
padding (int,default = 0): The padding which is to be done on the ends.
groups (int,default = 1): Number of groups in the convolutional filters.
dropout (int,default = 0): The dropout probability.
Linear (bool,default = False): If True, instead of convolution.

Usage:

import torch
from niftytorch.layers.layers import binconv3d
in_channels = 512
input = torch.ones(32,in_channels,32,32,32)
activation = binconv3d(input_channels = in_channels,output_channels = 256,kernel_size = 3,stride = 2,padding = 1,groups = 1,dropout = 0.5,Linear = False)
output = activation(input)