Matthew and Jack interned with Novetta’s Machine Learning Center of Excellence during the summer of 2019. This blog series discusses ADSynth, an app that creates a digital architecture diagram from a photo of a whiteboard sketch.
This summer, we interned with Novetta’s Machine Learning Center of Excellence (ML COE). As computer science students with a passion for machine learning and problem solving, we were excited to have the opportunity to apply our classroom knowledge in an industry setting. The experience was rewarding, though transferring our theoretical understanding of machine learning to practice presented a steep learning curve (pun intended).
Our primary project was “Architecture Diagram Synthesis”, or ADSynth. This is the first in a series of blog posts that discuss how we created ADSynth, the approaches we took, and our successful end result.
Background
Imagine you’re an engineer thinking about how to structure the backend of the solution you’re designing with AWS. The planning phase usually begins with whiteboarding potential approaches to your specific problem. You then have your architecture hand-drawn on a whiteboard with all of the AWS components and connecting arrows, but what next? If you have to present a proposal to your project manager, he or she doesn’t want to see your whiteboard scribbles, but rather a nicely formatted digital diagram. You then spend the next hour in Visio or PowerPoint importing the images of the AWS components you used in your diagram, formatting them, and making sure you’re connecting them correctly with your whiteboard drawing as a reference. What if you could save this time and go right from the whiteboard to an editable, digital diagram that can be placed in your proposal or slides?
ADSynth provides an all-in-one solution to speed up the process of creating an architecture diagram.
Figure 1: The Architect’s Dream
End-to-End Application
Our requirements began with ingesting a whiteboard photograph and passing it through a series of subroutines to generate the final output diagram. After developing our initial prototype, we quickly realized that it was not a user-friendly approach. It would be much better to treat the process in between taking the photo and getting the synthesized diagram as a black box, eliminating the complications of remembering what order to call the scripts in and what parameters to invoke them with.
The answer was to create a user-friendly app. With this solution, the inner workings of the synthesis process are abstracted away from the user. After creating the mockup on a whiteboard, all that is necessary is to launch the ADSynth app and take a picture. That’s it! In less time than it takes to Instagram a picture of your dinner, the resulting diagram pops up in the app and is easily saved to your camera roll.
Read the series!
Part 2: Object Detection Model and Training Data
