University of Washington,  Microsoft Research
of the system learning how a model is built from scratch
We demonstrate a realtime system which infers and tracks the assembly
process of a snap-together block model using a Kinect sensor.
The inference enables us to build a virtual replica of the model at
every step. Tracking enables us to provide context specific visual
feedback on a screen by augmenting the rendered virtual model aligned
with the physical model. The system allows users to author a new model
and uses the inferred assembly process to guide its recreation by
others. We propose a novel way of assembly guidance where the next
block to be added is rendered in blinking mode with the tracked virtual
model on screen. The system is also able to detect any mistakes made
and helps correct them by providing appropriate feedback. We focus on
assemblies of Duplo blocks.
We discuss the shortcomings of existing methods of guidance —
static figures or recorded videos — and demonstrate how our
method avoids those shortcomings. We also report on a user study to
compare our system with standard figure-based guidance methods found in
user manuals. The results of the user study suggest that our method is
able to aid users’ structural perception of the model better,
leads to fewer assembly errors, and reduces model construction time.
A., Fox D., Curless B., Cohen M. DuploTrack:
A Realtime System for Authoring and Guiding Duplo Block Assembly.In
Proceedings of 25th ACM Symposium on User Interface Software and