Handwriting Recognition of mathematical Formula

“A way of learning and acquiring mathematical formula knowledge easier”

As a Research Assistant Work at Document and Pattern Recognition Lab@RIT

To search resources in the Internet, not only junior students, but also math-major students and mathematicians, have to **input formulas** via plain symbols, or using more professional tools such as LaTex, or incomprehensible GUIs.

Users can input formula in the screen via hand-writting. The search engine will show the results about it.

Taking a photo or an image, the system will capture and recognize all formulas in the image.

Saving the formula you searched, you can use it next time by dragging it out

- Lack the design system
- Multiple stroked symbol
- Ability to fix unwanted recognition
- …

- Bounding box to show reco area
- Always show candidates
- Keep strokes until user pick one

Creating a mathematical formula searching engine to find an easy way for different types of people in learning math.

Our product is supporting user write formula by the electrical pen and take pictures to recognize it. Both of the function and UI design should be intuitive and explicit. Fixing the unwanted result should be easy.

Besides under the guidance of Prof. Zanibbi, there’re four more engineers deliberate over the UX questions before I design for it. I appreciate all of yours’ help in here!

The candidate will show simultaneously as you write on the canvas

The similar formulas as you input will be in the below, if the candidate showing is not what you need, find it from below!

Re-editing query (formula or text) would fold the SERP and revoke the canvas back. (* SERP is faked with Google.com screenshot)

Re-editing query (formula or text) would fold the SERP and revoke the canvas back.

Here are some of my design sketches which is start from mobile first. The initial idea of is to achieve cross-platform functionality. But I suggested the tablet is more practical as the product showing and using. So I restart to build this wireframe as well for our team.

During the design iterations and getting feedback from our team, I found out that there is a lot to consider when designing for the handwriting recognition and image-recognize platform.

First of all, I need to consider different formula has different weight and height. I’m going to try different types of mathematicals formula as I searched, to fix them in the box container. And then I also need to figure out potential problems when user use it as much as possible, such as using the function of self-association, to correct mistake formula that user drawed.

To find a method to make UI elements looks well-organized, I leant a lot from **Google Material Design** and experienced how important that design system works in UI.