Bounding Box Annotator - Identifying objects in images through boxes¶
The current tutorial will illustrate how to use Ipyannotator to annotate images using bounding boxes.
The task of identifying what an image represents is called image classification. Often an image contains multiple objects which can be identified individually.
Ipyannotator allows users to explore an entire set of images and given labels; manually create datasets by associating labels to bounding boxes drawn on top of the images; improve existing annotations.
This tutorial is divided in the following steps:
Select dataset¶
For this tutorial an artificial minimal dataset is generated by Ipyannotator with 50 images in 2 classes to be labeled (circle and rectangle).
dataset = DS.ARTIFICIAL_DETECTION
Setup annotator¶
This section will set up the paths and the input/output pair needed to classify the images.
The following cell imports the project file and directory where the images were generated. For this tutorial we simplify the process using the get_settings function instead of hardcoding the paths.
settings_ = get_settings(dataset)
settings_.project_file, settings_.image_dir
/home/runner/work/ipyannotator/ipyannotator/ipyannotator/datasets/generators.py:333: FutureWarning: `multichannel` is a deprecated argument name for `random_shapes`. It will be removed in version 1.0. Please use `channel_axis` instead.
image, shapes = random_shapes(size, n_objects_max, multichannel=True,
(Path('data/artificial_detection/annotations.json'), 'images')
Ipyannotator uses pairs of input/output data to set up the annotation.
The Bounding Box annotator uses InputImage and OutputImageBoxas the pair to set up the annotator.
The InputImage function provides information about the directory that contains the images to be labeled, and the images itself. The OutputImageBox function provides information about the directory that contains the classes that can be associated with the bounding boxes drawn on the images.
input_ = InputImage(image_dir=settings_.image_dir,
image_width=settings_.im_width,
image_height=settings_.im_height)
output_ = OutputImageBbox(classes=['Circle', 'Rectangle'])
input_.dir
'images'
The final part in setting up the Ipyannotator is the configuration of the Annotator factory with the pair of input/output data.
The factory allows three types of annotator tools: explore, create, improve. The next sections will guide you through every step.
anni = Annotator(input_, output_, settings_)
Explore¶
The explore option allows users to navigate across the images in the dataset using next/previous buttons. In case the dataset was already labeled, the labeling results can also be displayed. This function is used for data visualization only, improvement and addition of labels is done in the next steps.
explorer = anni.explore()
explorer
Sometimes the classes are not defined yet or incomplete. To explore the input images without worring about any classes you can use the NoOutput option on the annotator factory which is done in the following:
unlabel_factory = Annotator(input_, NoOutput(), settings_)
unlabel_factory.explore()
Create¶
The create option allows users to manually create their annotated datasets.
The bbox annotator allows users to create multiple bounding boxes on the image and associate labels to the bboxes. Additionally features are the following:
The lamp button can be used to highlight the annotation
The coordinate inputs can be changed to improve the annotated bounding box
The trash button can delete the annotation
The next cell removes already created annotation files to create a new dataset.
if dataset == DS.ARTIFICIAL_DETECTION:
dirpath = 'data/artificial_detection/create_results'
if os.path.exists(dirpath) and os.path.isdir(dirpath):
shutil.rmtree(dirpath)
The next cell initializes the create option.
For this tutorial, a function was defined that imitates human work. You can choose between performing the annotation manually yourself or letting the function do the work for you. Use the mouse to draw on the canvas.
creator = anni.create()
creator
The next cell imitates human work by randomly annotating all images automatically. If you want to manually annotate then skip the next step.
HELPER = Tutorial(dataset, settings_.project_path)
HELPER.add_random_bboxes(creator)
Improve¶
The improve feature allows users to refine the annotated dataset.
As before, for the purpose of the tutorial, a function can be used to performe the annotation and you don’t have to annotate manually. If you want to annotate manually then make sure selecting all missed or incorrect bounding boxes.
If you chose to annotate manually don’t forget to click the SAVE button when finished with each class.
if dataset == DS.ARTIFICIAL_DETECTION:
dirpath = 'data/artificial_detection/create_results/missed'
if os.path.exists(dirpath) and os.path.isdir(dirpath):
shutil.rmtree(dirpath)
improver = anni.improve()
improver
The next cell imitates the human work. If you chose to annotate manually make sure to skip this cell.
HELPER.fix_incorrect_bboxes(improver, creator)
Now we obtain a list of all images with incorrect bboxes:
[k for k, v in improver.to_dict().items() if v == {'answer': True}]
['data/artificial_detection/captured/img_13.jpg',
'data/artificial_detection/captured/img_15.jpg',
'data/artificial_detection/captured/img_18.jpg',
'data/artificial_detection/captured/img_20.jpg',
'data/artificial_detection/captured/img_24.jpg',
'data/artificial_detection/captured/img_25.jpg',
'data/artificial_detection/captured/img_28.jpg',
'data/artificial_detection/captured/img_29.jpg',
'data/artificial_detection/captured/img_35.jpg',
'data/artificial_detection/captured/img_5.jpg']