The Coding Notebook
Memorable coding moments of a software engineer
Converting to TensorFlow Lite models
Notes on converting object detection model (ssd), and image segmentation model (Deeplab) to tflite



# Note about Versions
As of today (May 7, 2020) tflite conversion scripts are not supported in TF v2.x, I am using TF 1.15.0 installed via `pip` (`pip install tensorflow==1.15.0`).

Also, TensorFlow API is changing rapidly, this is why all the links I provide for github repos will be to a specific commit and not to `master`, this is the commit I tested the procedure against.

# TensorFlow Model Garden
We will be using scripts from the TF Model Garden, so start by cloning this repo:
```sh
git clone https://github.com/tensorflow/models.git
```
I tested against commit `719439`:
```sh
cd models
git checkout 71943914beaa3a0a74c073657193f7e31a3b1b0e
```

# SSD Object Detection
In this example we will convert the model `ssd_mobilenet_v3_small_coco` provided by [Tensorflow detection model zoo](https://github.com/tensorflow/models/blob/71943914beaa3a0a74c073657193f7e31a3b1b0e/research/object_detection/g3doc/detection_model_zoo.md).

We are going to use scripts from the [Tensorflow Object Detection API](https://github.com/tensorflow/models/tree/71943914beaa3a0a74c073657193f7e31a3b1b0e/research/object_detection) so first make sure to [install](https://github.com/tensorflow/models/blob/71943914beaa3a0a74c073657193f7e31a3b1b0e/research/object_detection/g3doc/installation.md) that first.
**Note** to set `PYTHONPATH` correctly (as explained in the installation doc).

## Freeze Graph
Before we can convert the raw training output to tflite we first have to "freeze" the model, there is a specific freeze script for SSD, it needs as input the model checkpoint and the training config file, then to freeze the model:
```sh
python <TF_MODEL_GARDEN>/research/object_detection/export_tflite_ssd_graph.py \
--pipeline_config_path ssd_mobilenet_v3_small_coco/pipeline.config \
--trained_checkpoint_prefix ssd_mobilenet_v3_small_coco/model.ckpt \
--output_directory ssd_mobilenet_v3_small_coco/ssd_export \
--add_postprocessing_op=true
```

Note the last parameter `--add_postprocessing_op` is optional, it will add post processing to the graph (decode box centers / Non-Max suppression etc), you can skip that if you want todo it yourself.

In case of adding the post processing, the output has 4 arrays: `boxes, classes, scores, num_detections`. More info on how to use it in the post [Cross Platform Object Detection](https://www.thecodingnotebook.com/2020/04/cross-platform-object-detection-with.html) post.

## Convert to tflite
Now we can convert the frozen `tflite_graph.pb` to tflite.
When installing TensorFlow from via pip it also install the `tflite_convert` utility, we will use it for the conversion.

Before we begin it is important you know the names of your input/output arrays (nodes), and the shape of your input array. For the input node, if it is the first node in the graph, this python snippet can help:
```py
import tensorflow as tf
f=tf.gfile.GFile('tflite_graph.pb', 'rb')
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
print(graph_def.node[0])
```

For the output array, if you used `add_postprocessing_op=true` when running `export_tflite_ssd_graph`, the output arrays are: `TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3`, otherwise it really depends on your network...

Another really good option is to use [netron](https://github.com/lutzroeder/Netron) to inspect the model graph.

Full doc on `tflite_convert` is [here](https://www.tensorflow.org/lite/convert/cmdline_reference).

For our case, converting to tflite is as follow:
```sh
tflite_convert \
--output_file ssd_mobilenet_v3_small_coco/ssd_export/model-float.tflite \
--graph_def_file ssd_mobilenet_v3_small_coco/ssd_export/tflite_graph.pb \
--input_arrays normalized_input_image_tensor \
--input_shapes 1,320,320,3 \
--output_arrays TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3 \
--inference_type FLOAT \
--allow_custom_ops
```

We can also quantize the output model:
```sh
tflite_convert \
--output_file ssd_mobilenet_v3_small_coco/ssd_export/model-quant.tflite \
--graph_def_file ssd_mobilenet_v3_small_coco/ssd_export/tflite_graph.pb \
--input_arrays normalized_input_image_tensor \
--input_shapes 1,320,320,3 \
--output_arrays TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3 \
--inference_type QUANTIZED_UINT8 \
--mean_values=128 \
--std_dev_values=128 \
--default_ranges_min=0 \
--default_ranges_max=255 \
--allow_custom_ops
```
Note that `default_ranges_min` and `default_ranges_max` are not necessary if the trainging config had `graph_rewriter quantization`, otherwise these 2 might be required and specify the min/max value possible.

Note II: For this specific model, the quantized version gives VERY bad results, I'm not sure why

# Deeplab (semantic image segmentation)
Download the pre-trained model from [DeepLab model zoo](https://github.com/tensorflow/models/blob/71943914beaa3a0a74c073657193f7e31a3b1b0e/research/deeplab/g3doc/model_zoo.md), this method was tested on the model based on MobileNetV2 pascal dataset (`mobilenetv2_dm05_coco_voc_trainval`).

Deeplab graph contains some pre/post processing, we will take only the part without these processing, which is from node `sub_7` to node `ResizeBilinear_2`:
```sh
tflite_convert \
--output_file=./deeplabv3_mnv2_dm05_pascal_trainval/deeplabv3_21.tflite \
--graph_def_file=./deeplabv3_mnv2_dm05_pascal_trainval/frozen_inference_graph.pb \
--input_arrays=sub_7 \
--output_arrays=ResizeBilinear_2 \
--input_shapes=1,513,513,3 \
--inference_type=FLOAT
```

The output shape is `[1,513,513,21]`, which is, for each pixel, the probability it belongs to one of the 21 classes:
```
"background", "aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow", "diningtable", "dog", "horse", "motorbike", "person", "pottedplant", "sheep", "sofa", "train", "tv".
```

# The End