Model Editing¶
Commands for editing DNN file contents.
extract¶
Extract weights and model architecture information from a DNN file into JSON and numpy files.
Usage
usage: softneuro extract [--help] DNN OUTDIR
Arguments
| Argument | Description |
|---|---|
| DNN | DNN file to be extracted. |
| OUTDIR | Output directory where the data will be extracted to. |
Flags
| Flag | Description |
|---|---|
| -h, --help | Shows the command help. |
Example
Running this command will create, in the vgg16_extract folder, model.json, weights.json and a directory containing numpy format files with weights data
in a weights directory.
※There's no terminal output
$ softneuro extract vgg16.dnn vgg16_extract
$ ls vgg16_extract
model.json weights weights.json
archive¶
Pack the files in the same format as what the extract command outputs into a DNN model file.
Usage
usage: softneuro archive [--help] SRCDIR DNN
Arguments
| Argument | Description |
|---|---|
| SRCDIR | Directory with contents in the same format as the extract command output. |
| DNN | DNN file to be outputed. |
Flags
| Flag | Description |
|---|---|
| -h, --help | Shows the command help. |
Example
This command will generate a vgg16_archive.dnn file.
※There's no terminal output
$ softneuro archive vgg16_extract vgg16_archive.dnn
$ ls
vgg16_extract vgg16_archive.dnn
refine¶
Refine a DNN file for faster inference times.
Generally, the processing used for optimizations is the same as that for converting a model to DNN format. When optimization has not been applied during model conversion, it can be performed later using refine.
Usage
usage: softneuro refine [--batch-norm-to-madd] [--fuse-dropout] [--fuse-transpose] [--fuse-madd]
[--fuse-relu] [--fuse-padding] [--swap-act-pool]
[--strip-constant] [--strip-immutable] [--strip-layer] [--strip-net]
[--norm-weight] [--dedup-weight] [--strip-weight][--help]
INPUT OUTPUT
Arguments
| Argument | Description |
|---|---|
| INPUT | DNN file to be refined. |
| OUTPUT | Refined DNN file output. |
Flags
If a flag is set only that specific optimization will be done. For instance,
if --batch-norm-to-madd is set, other optimizations like relu fuse won't be done.
| Flags | Description |
|---|---|
| --batch-norm-to-madd | Transforms batch normalization into madd layers. |
| --fuse-dropout | Fuses dropout into the previous layer. |
| --fuse-transpose | Fuses transpose into the previous layer.。 |
| --fuse-madd | Fuses multiply-add into the previous layer. |
| --fuse-relu | Fuses relu into the previous layer. |
| --fuse-padding | Fuses zero padding into the previous layer. |
| --swap-act-pool | Transforms activation-pooling into pooling activation. |
| --strip-constant | Removes layers which are constant。 |
| --strip-immutable | Removes layers which are immutable。 |
| --strip-layer | Removes unnecessary layers. |
| --strip-net | Removes sub-nets without layers. |
| --strip-weight | Removes weights not referenced by any layer. |
| --dedup-weight | Deduplicates weights. |
| --norm-weight | Normalizes subnormal, nan, inf weight values. |
| -h, --help | Shows the command help. |
Example
Refine a dnn file by deduplicating weights.
$ softneuro refine --dedup-weight Pose_small.dnn Pose_small_refined.dnn
------------------------------------------------------------------
Deduplicate Weights
------------------------------------------------------------------
'weight_7_0' is deduplicated to 'weight_4_0' (SNR: inf).
'weight_10_0' is deduplicated to 'weight_3_0' (SNR: inf).
'weight_13_0' is deduplicated to 'weight_2_0' (SNR: inf).
'weight_14_0' is deduplicated to 'weight_2_0' (SNR: inf).
'weight_15_0' is deduplicated to 'weight_2_0' (SNR: inf).
5 weights are deduplicated.
compress¶
Reduces the DNN file weights data size.
Usage
usage: softneuro compress [--dtype [[LTYPE][.WNAME]=]DTYPE[@LAYER_INDICES]]...
[--pass PASSWORD] [--help]
INPUT OUTPUT
Arguments
| Argument | Description |
|---|---|
| INPUT | DNN file to be compressed. |
| OUTPUT | Compressed DNN file output. |
Flags
| Flag | Description |
|---|---|
| --dtype [[LTYPE][.WNAME]=]DTYPE[@LAYER_INDICES] | Select what data type will be used to save the weights. The available DTYPE options are float16 or qint8. If this flag isn't set, all main network layer weights will be saved as qint8. If LTYPE is set, only layers of that type will have their weights converted. If WNAME is set, only the weight with that name will be converted. If LAYER_INDICES is set, only the layers at the given indices will have their weights converted. For more information on layer indices, use the softneuro help layer_indices command. |
| --pass PASSWORD | Set an encryption password. |
| -h, --help | Shows the command help. |
Example
This command creates a vgg16_qint8.dnn file with qint8 weights.
※There's no terminal output
$ softneuro compress vgg16.dnn vgg16_qint8.dnn
$ ls -lh
-rw-rw-r-- 1 user user 528M 1月 1 10:00 vgg16.dnn
-rw-rw-r-- 1 user user 133M 1月 1 10:00 vgg16_qint8.dnn
cipher¶
Encrypts a DNN file.
Usage
usage: softneuro cipher [--pass PASSWORD] [--new-pass PASSWORD] [--secret] [--help] INPUT OUTPUT
Arguments
| Argument | Description |
|---|---|
| INPUT | DNN file to be encrypted. |
| OUTPUT | Output encrypted DNN file. |
Flags
| Flag | Description |
|---|---|
| -p PASSWORD, --pass PASSWORD | Password for input dnn. |
| --new-pass PASSWORD | New password for OUTPUT. If not given, the password does not change from INPUT. If empty ('') is given, |
| decrypt INPUT and write it to OUTPUT without password encryption. | |
| --secret | Encrypts to a secret dnn. |
| -h, --help | Shows the command help. |
Example
This command creates a vgg16_enc.dnn encrypted DNN file.
※There's no terminal output
$ softneuro cipher --new-pass 4u9pnza vgg16.dnn vgg16_enc.dnn
$ ls
vgg16.dnn vgg16_enc.dnn
$ softneuro run vgg16_enc.dnn
mor_dnn.c:921:Error: Invalid password for dnn.
softneuro:Error: Load error.
$ softneuro run vgg16_enc.dnn -p 4u9pnza
---------------------------------
Statistics
---------------------------------
FUNCTION AVE(us) MIN(us) MAX(us) #RUN
Dnn_load() 36,055 36,055 36,055 1
Dnn_compile() 727,957 727,957 727,957 1
Dnn_forward() 1,080,363 1,080,363 1,080,363 1
Used memory: 1,140,400,128 Bytes
calibrate¶
Calibrate a DNN file for static quantization. Calibrate operation fixes the quantization range by calculating max/min outputs of each layer, using prepared inputs.
Usage
usage: softneuro calibrate [--keep_img_ar PADDINGCOLOR]
[--img_resize_mode RESIZEMODE][--pass PASSWORD]
[--reset] [-trunc RATIO] [--help]
IDNN ODNN [INPUT]...
Arguments
| Argument | Description |
|---|---|
| IDNN | DNN file to be calibrated. |
| ODNN | Calibrated DNN file output. |
| INPUT | Required input files for calibration. Image files, numpy files, or a directory containing those files, can be specified. |
Flags
| Flag | Description |
|---|---|
| --keep_img_ar PADDINGCOLOR | Keeps aspect ratio when resizing input image. The aspect ratio is not kept by default. Margin space is filled with the color specified by PADDINGCOLOR. PADDINGCOLOR can be specified by RGB value, for example, '0, 0, 0'. |
| --img_resize_mode RESIZEMODE | Specifies the resizing mode. 'bilinear' or 'nearest' can be specified. Default is 'bilinear'. |
| -p, --pass PASSWORD | Encryption password. |
| --reset | Resets the calibration information (max/min). |
| --trunc RATIO | Removes the top and bottom data as outliers, specified by 'RATIO' ratio. When RATIO is 0.02, top/bottom 2% data are removed, the min/max values are calculated by the remaining 96% data. Default RATIO is 0. |
| -h, --help | Shows the command help. |
Example
This command creates a vgg16_calib.dnn calibrated DNN file.
※There's no terminal output
$ softneuro calibrate vgg16.dnn vgg16_calib.dnn input0.jpg input1.jpg input2.jpg
$ softneuro tune vgg16_calib.dnn vgg16_calib_tuned.dnn
decompose¶
Decompose a net of dnn file into two subnets.
Usage
usage: softneuro decompose [--pass PASSWORD] [--split SPLIT] [--compress COMPRESS] [--help] INPUT OUTPUT
Arguments
| Argument | Description |
|---|---|
| INPUT | Input dnn file. |
| OUTPUT | Output dnn file |
Flags
| Flag | Description |
|---|---|
| -p PASSWORD, --pass PASSWORD | Password for input file. |
| -s SPLIT, --split SPLIT | Set layer to be split from |
| -c COMPRESS, --compress COMPRESS | Set net to compress。 |
| -h, --help | Show the command help。 |
Example
Split main net of ssd.dnn from conv_pw_1 layer.
$ softneuro decompose -s conv_pw_1 ssd.dnn ssd_dcp.dnn
[0] from input_1 to conv_pw_1
[1] from conv_dw_2 to sink_0
In the case of decomposing the main network of vgg16.dnn, and then tuning it for an external engine (Qualcomm HTA), the commands are as follows:
$ softneuro decompose -c main vgg16.dnn vgg16_decomposed.dnn
$ softneuro tune vgg16_decomposed.dnn vgg16_decomposed_tuned.dnn --routine cpu/faster/hta