Apache MXNet’s module API is like a FeedForward model and it is easier to compose similar to Torch module. It consists of following classes −
BaseModule([logger])
It represents the base class of a module. A module can be thought of as computation component or computation machine. The job of a module is to execute forward and backward passes. It also updates parameters in a model.
Methods
Following table shows the methods consisted in BaseModule class−
| Methods | Definition |
|---|---|
| backward([out_grads]) | As name implies this method implements the backward computation. |
| bind(data_shapes[, label_shapes, …]) | It binds the symbols to construct executors and it is necessary before one can perform computation with the module. |
| fit(train_data[, eval_data, eval_metric, …]) | This method trains the module parameters. |
| forward(data_batch[, is_train]) | As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes. |
| forward_backward(data_batch) | It is a convenient function, as name implies, that calls both forward and backward. |
| get_input_grads([merge_multi_context]) | This method will gets the gradients to the inputs which is computed in the previous backward computation. |
| get_outputs([merge_multi_context]) | As name implies, this method will gets outputs of the previous forward computation. |
| get_params() | It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device. |
| get_states([merge_multi_context]) | This method will get states from all devices |
| init_optimizer([kvstore, optimizer, …]) | This method installs and initialize the optimizers. It also initializes kvstore for distribute training. |
| init_params([initializer, arg_params, …]) | As name implies, this method will initialize the parameters and auxiliary states. |
| install_monitor(mon) | This method will install monitor on all executors. |
| iter_predict(eval_data[, num_batch, reset, …]) | This method will iterate over predictions. |
| load_params(fname) | It will, as name specifies, load model parameters from file. |
| predict(eval_data[, num_batch, …]) | It will run the prediction and collects the outputs as well. |
| prepare(data_batch[, sparse_row_id_fn]) | The operator prepares the module for processing a given data batch. |
| save_params(fname) | As name specifies, this function will save the model parameters to file. |
| score(eval_data, eval_metric[, num_batch, …]) | It runs the prediction on eval_data and also evaluates the performance according to the given eval_metric. |
| set_params(arg_params, aux_params[, …]) | This method will assign the parameter and aux state values. |
| set_states([states, value]) | This method, as name implies, sets value for states. |
| update() | This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch. |
| update_metric(eval_metric, labels[, pre_sliced]) | This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation. |
| backward([out_grads]) | As name implies this method implements the backward computation. |
| bind(data_shapes[, label_shapes, …]) | It set up the buckets and binds the executor for the default bucket key. This method represents the binding for a BucketingModule. |
| forward(data_batch[, is_train]) | As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes. |
| get_input_grads([merge_multi_context]) | This method will get the gradients to the inputs which is computed in the previous backward computation. |
| get_outputs([merge_multi_context]) | As name implies, this method will get outputs from the previous forward computation. |
| get_params() | It gets the current parameters especially those which are potentially copies of the actual parameters used to do computation on the device. |
| get_states([merge_multi_context]) | This method will get states from all devices. |
| init_optimizer([kvstore, optimizer, …]) | This method installs and initialize the optimizers. It also initializes kvstore for distribute training. |
| init_params([initializer, arg_params, …]) | As name implies, this method will initialize the parameters and auxiliary states. |
| install_monitor(mon) | This method will install monitor on all executors. |
| load(prefix, epoch[, sym_gen, …]) | This method will create a model from the previously saved checkpoint. |
| load_dict([sym_dict, sym_gen, …]) | This method will create a model from a dictionary (dict) mapping bucket_key to symbols. It also shares arg_params and aux_params. |
| prepare(data_batch[, sparse_row_id_fn]) | The operator prepares the module for processing a given data batch. |
| save_checkpoint(prefix, epoch[, remove_amp_cast]) | This method, as name implies, saves the current progress to the checkpoint for all buckets in BucketingModule. It is recommended to use mx.callback.module_checkpoint as epoch_end_callback to save during training. |
| set_params(arg_params, aux_params[,…]) | As name specifies, this function will assign parameters and aux state values. |
| set_states([states, value]) | This method, as name implies, sets value for states. |
| switch_bucket(bucket_key, data_shapes[, …]) | It will switche to a different bucket. |
| update() | This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch. |
| update_metric(eval_metric, labels[, pre_sliced]) | This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation. |
Attributes
Following table shows the attributes consisted in the methods of BaseModule class −
| Attributes | Definition |
|---|---|
| data_names | It consists of the list of names for data required by this module. |
| data_shapes | It consists of the list of (name, shape) pairs specifying the data inputs to this module. |
| label_shapes | It shows the list of (name, shape) pairs specifying the label inputs to this module. |
| output_names | It consists of the list of names for the outputs of this module. |
| output_shapes | It consists of the list of (name, shape) pairs specifying the outputs of this module. |
| symbol | As name specified, this attribute gets the symbol associated with this module. |
data_shapes: You can refer the link available at for details.
output_shapes: More
output_shapes: More information is available at
BucketingModule(sym_gen[…])
It represents the Bucketingmodule class of a Module which helps to deal efficiently with varying length inputs.
Methods
Following table shows the methods consisted in BucketingModule class −
Attributes
Following table shows the attributes consisted in the methods of BaseModule class −
| Attributes | Definition |
|---|---|
| data_names | It consists of the list of names for data required by this module. |
| data_shapes | It consists of the list of (name, shape) pairs specifying the data inputs to this module. |
| label_shapes | It shows the list of (name, shape) pairs specifying the label inputs to this module. |
| output_names | It consists of the list of names for the outputs of this module. |
| output_shapes | It consists of the list of (name, shape) pairs specifying the outputs of this module. |
| Symbol | As name specified, this attribute gets the symbol associated with this module. |
data_shapes − You can refer the link at for more information.
output_shapes− You can refer the link at for more information.
Module(symbol[,data_names, label_names,…])
It represents a basic module that wrap a symbol.
Methods
Following table shows the methods consisted in Module class −
| Methods | Definition |
|---|---|
| backward([out_grads]) | As name implies this method implements the backward computation. |
| bind(data_shapes[, label_shapes, …]) | It binds the symbols to construct executors and it is necessary before one can perform computation with the module. |
| borrow_optimizer(shared_module) | As name implies, this method will borrow the optimizer from a shared module. |
| forward(data_batch[, is_train]) | As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes. |
| get_input_grads([merge_multi_context]) | This method will gets the gradients to the inputs which is computed in the previous backward computation. |
| get_outputs([merge_multi_context]) | As name implies, this method will gets outputs of the previous forward computation. |
| get_params() | It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device. |
| get_states([merge_multi_context]) | This method will get states from all devices |
| init_optimizer([kvstore, optimizer, …]) | This method installs and initialize the optimizers. It also initializes kvstore for distribute training. |
| init_params([initializer, arg_params, …]) | As name implies, this method will initialize the parameters and auxiliary states. |
| install_monitor(mon) | This method will install monitor on all executors. |
| load(prefix, epoch[, sym_gen, …]) | This method will create a model from the previously saved checkpoint. |
| load_optimizer_states(fname) | This method will load an optimizer i.e. the updater state from a file. |
| prepare(data_batch[, sparse_row_id_fn]) | The operator prepares the module for processing a given data batch. |
| reshape(data_shapes[, label_shapes]) | This method, as name implies, reshape the module for new input shapes. |
| save_checkpoint(prefix, epoch[, …]) | It saves the current progress to checkpoint. |
| save_optimizer_states(fname) | This method saves the optimizer or the updater state to a file. |
| set_params(arg_params, aux_params[,…]) | As name specifies, this function will assign parameters and aux state values. |
| set_states([states, value]) | This method, as name implies, sets value for states. |
| update() | This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch. |
| update_metric(eval_metric, labels[, pre_sliced]) | This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation. |
Attributes
Following table shows the attributes consisted in the methods of Module class −
| Attributes | Definition |
|---|---|
| data_names | It consists of the list of names for data required by this module. |
| data_shapes | It consists of the list of (name, shape) pairs specifying the data inputs to this module. |
| label_shapes | It shows the list of (name, shape) pairs specifying the label inputs to this module. |
| output_names | It consists of the list of names for the outputs of this module. |
| output_shapes | It consists of the list of (name, shape) pairs specifying the outputs of this module. |
| label_names | It consists of the list of names for labels required by this module. |
data_shapes: Visit the link for further details.
output_shapes: The link given herewith will offer other important information.
PythonLossModule([name,data_names,…])
The base of this class is mxnet.module.python_module.PythonModule. PythonLossModule class is a convenient module class which implements all or many of the module APIs as empty functions.
Methods
Following table shows the methods consisted in PythonLossModule class:
| Methods | Definition |
|---|---|
| backward([out_grads]) | As name implies this method implements the backward computation. |
| forward(data_batch[, is_train]) | As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes. |
| get_input_grads([merge_multi_context]) | This method will gets the gradients to the inputs which is computed in the previous backward computation. |
| get_outputs([merge_multi_context]) | As name implies, this method will gets outputs of the previous forward computation. |
| install_monitor(mon) | This method will install monitor on all executors. |
PythonModule([data_names,label_names…])
The base of this class is mxnet.module.base_module.BaseModule. PythonModule class also is a convenient module class which implements all or many of the module APIs as empty functions.
Methods
Following table shows the methods consisted in PythonModule class −
| Methods | Definition |
|---|---|
| bind(data_shapes[, label_shapes, …]) | It binds the symbols to construct executors and it is necessary before one can perform computation with the module. |
| get_params() | It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device. |
| init_optimizer([kvstore, optimizer, …]) | This method installs and initialize the optimizers. It also initializes kvstore for distribute training. |
| init_params([initializer, arg_params, …]) | As name implies, this method will initialize the parameters and auxiliary states. |
| update() | This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch. |
| update_metric(eval_metric, labels[, pre_sliced]) | This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation. |
Attributes
Following table shows the attributes consisted in the methods of PythonModule class −
| Attributes | Definition |
|---|---|
| data_names | It consists of the list of names for data required by this module. |
| data_shapes | It consists of the list of (name, shape) pairs specifying the data inputs to this module. |
| label_shapes | It shows the list of (name, shape) pairs specifying the label inputs to this module. |
| output_names | It consists of the list of names for the outputs of this module. |
| output_shapes | It consists of the list of (name, shape) pairs specifying the outputs of this module. |
data_shapes − Follow the link for details.
output_shapes − For more details, visit the link available at
SequentialModule([logger])
The base of this class is mxnet.module.base_module.BaseModule. SequentialModule class also is a container module that can chain more than two (multiple) modules together.
Methods
Following table shows the methods consisted in SequentialModule class
| Methods | Definition |
|---|---|
| add(module, **kwargs) | This is most important function of this class. It adds a module to the chain. |
| backward([out_grads]) | As name implies this method implements the backward computation. |
| bind(data_shapes[, label_shapes, …]) | It binds the symbols to construct executors and it is necessary before one can perform computation with the module. |
| forward(data_batch[, is_train]) | As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes. |
| get_input_grads([merge_multi_context]) | This method will gets the gradients to the inputs which is computed in the previous backward computation. |
| get_outputs([merge_multi_context]) | As name implies, this method will gets outputs of the previous forward computation. |
| get_params() | It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device. |
| init_optimizer([kvstore, optimizer, …]) | This method installs and initialize the optimizers. It also initializes kvstore for distribute training. |
| init_params([initializer, arg_params, …]) | As name implies, this method will initialize the parameters and auxiliary states. |
| install_monitor(mon) | This method will install monitor on all executors. |
| update() | This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch. |
| update_metric(eval_metric, labels[, pre_sliced]) | This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation. |
Attributes
Following table shows the attributes consisted in the methods of BaseModule class −
| Attributes | Definition |
|---|---|
| data_names | It consists of the list of names for data required by this module. |
| data_shapes | It consists of the list of (name, shape) pairs specifying the data inputs to this module. |
| label_shapes | It shows the list of (name, shape) pairs specifying the label inputs to this module. |
| output_names | It consists of the list of names for the outputs of this module. |
| output_shapes | It consists of the list of (name, shape) pairs specifying the outputs of this module. | output_shapes | It consists of the list of (name, shape) pairs specifying the outputs of this module. |
data_shapes − The link given herewith will help you in understanding the attribute in much detail.
output_shapes − Follow the link available at for details.
Implementation Examples
In the example below, we are going create a mxnet module.
import mxnet as mx
input_data = mx.symbol.Variable(''input_data'')
f_connected1 = mx.symbol.FullyConnected(data, name=''f_connected1'', num_hidden=128)
activation_1 = mx.symbol.Activation(f_connected1, name=''relu1'', act_type="relu")
f_connected2 = mx.symbol.FullyConnected(activation_1, name = ''f_connected2'', num_hidden = 64)
activation_2 = mx.symbol.Activation(f_connected2, name=''relu2'',
act_type="relu")
f_connected3 = mx.symbol.FullyConnected(activation_2, name=''fc3'', num_hidden=10)
out = mx.symbol.SoftmaxOutput(f_connected3, name = ''softmax'')
mod = mx.mod.Module(out)
print(out)
Output
The output is mentioned below −
<Symbol softmax>
Example
print(mod)
Output
The output is shown below −
<mxnet.module.module.Module object at 0x00000123A9892F28>
In this example below, we will be implementing forward computation
import mxnet as mx
from collections import namedtuple
Batch = namedtuple(''Batch'', [''data''])
data = mx.sym.Variable(''data'')
out = data * 2
mod = mx.mod.Module(symbol=out, label_names=None)
mod.bind(data_shapes=[(''data'', (1, 10))])
mod.init_params()
data1 = [mx.nd.ones((1, 10))]
mod.forward(Batch(data1))
print (mod.get_outputs()[0].asnumpy())
Output
When you execute the above code, you should see the following output −
[[2. 2. 2. 2. 2. 2. 2. 2. 2. 2.]]
Example
data2 = [mx.nd.ones((3, 5))] mod.forward(Batch(data2)) print (mod.get_outputs()[0].asnumpy())
Output
Given below is the output of the code −
[[2. 2. 2. 2. 2.] [2. 2. 2. 2. 2.] [2. 2. 2. 2. 2.]]