Most annotation tools for computer vision resemble basic image editors, and generally follow the same patterns of design and user interaction. The annotator is handed a toolbox of drawing tools, the annotator picks a tool for drawing a shape, draws a shape, classifies it, adds additional metadata, and repeats.
While this pattern is simple and generic, it fails to address many of the key challenges in reducing cognitive load, time on task, and maximizing quality. Unlike free-form editing, annotation tasks can be complex, repetitive, and driven by precise guidelines. The best tools are those that provide a structured experience for the annotator, based on the annotation guidelines, while being generalized enough to be used off-the-shelf for a wide variety of use cases.
For example, Alegion’s image annotation tools have the annotator choose the entity, or class, to annotate, first, and then, based on the guidelines, the annotator is given the correct tools for labeling. By simply turning the flow around, the experience for the annotator is guided and simple.
There are also families of use cases that purpose-built annotation tooling can vastly simplify. Entity Grouping is a new Alegion capability for computer vision tasks that extends this philosophy.
Annotation of skeletal or facial landmarks are good examples. In these cases, annotators need to mark a set of predefined landmarks using keypoints, provide some classifications, and relate the key points to a parent object, like a person. The parent object may also have a localization using a bounding box or polygon.
Annotating several persons in a scene can involve as many as one hundred key points. Making sure that all landmarks have been identified and associated with the correct person can be tedious, time consuming, and error prone. The time to perform quality assurance review also needs to be taken into consideration.
Using Entity Groups, this type of task is vastly simplified. When enabled, it organizes all of the items to be annotated for a particular type of object into a ‘shopping list’ for the annotator.
Instead of having to remember or refer to guidelines, the annotator can simply work down the list as each object is annotated and classified. In addition to reducing cognitive load, mouse travel is reduced significantly. As each part of an object is annotated, the annotator is automatically transitioned to draw the next part in a logical order. Quality review is also simpler. Each object presents the reviewer with s simple checklist of what has and has not been annotated.
In a recent project that required an ordered set of facial landmarks for measurement of driver attentiveness, the use of entity groups delivered an immediate 25% increase in efficiency across the entire workflow - not just the annotation stage. Missing key point errors were also significantly reduced. Annotators also loved the elimination of the most tedious aspects of the task - allowing them to focus more on accuracy.
To see this capability in action, take a look at the video in this post. In this example, a person is localized with a bounding box, the body pose is classified, and a set of named skeletal keypoints are annotated. Because everything is organized using an entity group, the annotator can simply work down the list, and all of the 20+ annotations are automatically related to the correct person. The shopping list interface makes it very clear which points have been completed and which needed to be skipped because they are not visible. Time on task is significantly reduced and quality is higher.
Alongside tools like SmartPoly and best in class video annotation, Entity Groups shows Alegion's commitment to a labeling platform that delivers the fastest path to quality whether you use the Alegion Global Workforce or your own internal labeling teams.