Origami and Augmented Reality (continued 2)
Earlier we looked at these issues of using Augmented Reality to create origami:
This article focuses on the technical details of using barcodes to generate hints in augmented reality.
Suppose we have a virtual origami model, a real piece of paper and a webcam. On the sheet image received from the camera, the program should draw a hint for the current step of folding the model, for example, an arrow from one corner of the sheet to another.
To create a model, you need someone who initially folds origami using a computer, you can read more here: all components .
The problem is to correctly determine where our objects are on the sheet. The sheet can be arbitrarily oriented and have an arbitrary shape at some step. It is practically impossible to “see” all the details of a folded single-color sheet with a conventional web-camera. Using a color printer to mark areas of the sheet is not available to everyone.
Barcodes are placed on certain sections of the sheet. The program finds them, easily identifies them, determines by them the orientation of the sheet and the necessary offset to the objects included in the tooltip. How will we arrange barcodes on a sheet? The following factors must be considered:
To complete the first paragraph, it is enough to use such useful information obtained from the model as area triangulation. Since the folds can pass only along the borders of the triangles, it is enough to store the barcodes inside them, and it is desirable with some margin from the edge. We can always enter a circle into a triangle, easily find its center and radius, and place the barcode inside it.
The dimensions of the triangles included in the partitioning of the area impose a limit on the maximum size of the barcode. Of course, it should not occupy half a sheet, but at the same time should be visible to the camera. To fulfill the second requirement, we also need information on the visibility of triangles at each step. Fortunately, it is also contained in the model.
For the last paragraph, a frequency dictionary of the appearance of triangles in the visibility zone is also necessary. It is logical to place barcodes in the most common triangles and reuse, then their number will decrease sharply.
At each step, we give each triangle an estimate that meets our requirements, and choose the one that received the best. Each criterion has an adjustable degree of influence (weight), so that the system can be optimally configured.
It is also possible to select print on one or both sides of the sheet, it depends on whether we can see the tooltip on either side, or only on a certain side. In barcodes, their unique identifiers are encoded, and they are used as a key to search for information about a tooltip (type of action, offsets to objects, and so on). Despite all the precautions taken, the barcode can still be corrupted to some extent, therefore, Reed-Solomon coding is used, which allows restoring the original data in case of partial damage.
About labels and orientation in space (in our opinion, the most entertaining detail) we will tell in the next article.
This article focuses on the technical details of using barcodes to generate hints in augmented reality.
Task
Suppose we have a virtual origami model, a real piece of paper and a webcam. On the sheet image received from the camera, the program should draw a hint for the current step of folding the model, for example, an arrow from one corner of the sheet to another.
To create a model, you need someone who initially folds origami using a computer, you can read more here: all components .
Limitations
The problem is to correctly determine where our objects are on the sheet. The sheet can be arbitrarily oriented and have an arbitrary shape at some step. It is practically impossible to “see” all the details of a folded single-color sheet with a conventional web-camera. Using a color printer to mark areas of the sheet is not available to everyone.
Appearance of the solution
Barcodes are placed on certain sections of the sheet. The program finds them, easily identifies them, determines by them the orientation of the sheet and the necessary offset to the objects included in the tooltip. How will we arrange barcodes on a sheet? The following factors must be considered:
- the barcode should not lie on the fold lines, as it may jam and be damaged;
- the barcode should be visible at all steps of folding the model, on which it is used to display a hint;
- the barcode should be located at a sufficiently large distance from the tooltip objects, since the user will most likely grab them with his hands to complete the fold and close it;
- at the same time, the barcode should not be too far so that the errors in determining the orientation do not greatly affect the points obtained after the shift;
- the barcode should not be too large or too small;
- the fewer they are, the better, the paint is worth the money.
To complete the first paragraph, it is enough to use such useful information obtained from the model as area triangulation. Since the folds can pass only along the borders of the triangles, it is enough to store the barcodes inside them, and it is desirable with some margin from the edge. We can always enter a circle into a triangle, easily find its center and radius, and place the barcode inside it.
The dimensions of the triangles included in the partitioning of the area impose a limit on the maximum size of the barcode. Of course, it should not occupy half a sheet, but at the same time should be visible to the camera. To fulfill the second requirement, we also need information on the visibility of triangles at each step. Fortunately, it is also contained in the model.
For the last paragraph, a frequency dictionary of the appearance of triangles in the visibility zone is also necessary. It is logical to place barcodes in the most common triangles and reuse, then their number will decrease sharply.
At each step, we give each triangle an estimate that meets our requirements, and choose the one that received the best. Each criterion has an adjustable degree of influence (weight), so that the system can be optimally configured.
It is also possible to select print on one or both sides of the sheet, it depends on whether we can see the tooltip on either side, or only on a certain side. In barcodes, their unique identifiers are encoded, and they are used as a key to search for information about a tooltip (type of action, offsets to objects, and so on). Despite all the precautions taken, the barcode can still be corrupted to some extent, therefore, Reed-Solomon coding is used, which allows restoring the original data in case of partial damage.
About labels and orientation in space (in our opinion, the most entertaining detail) we will tell in the next article.