Back

Voice Pluck

Warehouse Management System and iOS app with a voice interface for stacking products.

Status

Finished

My Role(s)

Team Leader
Full Stack Developer

Technologies

Swift
Spring Boot
Keycloak
Terraform
GitLab

About

Voice Pluck is the project me and three students created for our Bachelor Thesis at NTNU. Solwr specializes in logistics, and provides a voice recognition system to warehouses, which helps them load orders more efficiently. The existing system worked, however, not optimally. It only used a voice-interface, and some tasks in a warehouse are inherently less efficiently solved without a touch-interface. Additionally, Solwr used an expensive third-party software to handle the voice recognition. Therefore, Solwr tasked us with creating a mobile application which handled the voice-recognition without the need of a paid library, and with the addition of a touch-interface. We also created a REST API which acts as a Warehouse Management System. The WMS supports adding, removing, and updating products, orders, and users. Last but not least, we researched a way to implement a smart watch application with voice recognition.

Challenges

From project description to technical features, the project served the team with several challenges. The initial project description was somewhat vague. So the first challenge of the project was to understand the user requirements and needs. To solve this, the whole team worked a day in a warehouse, to understand the workflow of the end-users. Speech recognition brought a series of challenges into our project. The warehouse being a loud environment, parsing the input, user privacy, internet connectivity, the list goes on. We solved this using Apple's SFSpeechRecognition library. We also quickly understood that the end-user had to be able to use the voice-interface and touch-interface seamlessly. There are several steps in the process of preparing an order for transportation. For the user to be able to switch between interfaces seamlessly, we needed a way to keep track of the current step in the plucking-process. We solved this by using a PluckService, that keeps track of the current step, and exposes a doAction method. This method was executed whenever a keyword were recognized, or the user pressed specific buttons in the app. Creating the REST API was no easy task either. There are complex relations between entities in a warehouse environment. Rigorous testing made it easier to implement all of these entities correctly. I encourage you to read more about the challenges of our project in our report.

Results

Our project had a wide scope, and many possibilites. The finished product consisted of a iOS app featuring authentication, speech recognition, text-to-speech, a WMS REST API, and a implementation plan for a WatchOS app with voice-recognition. Most importantly, the team learned a lot from this project. Ranging from iOS development with Swift, Spring Boot REST API development, authentication with Keycloak, testing, and non-technical skills such as agile development and researching user requirements. As the team leader, I also learned a lot about managing Jira, and addressing timeline issues with the team.