Week 1

April 6 - April 7

Full Project Description

          Today, keyboards and mouses are commonplace with every computer. Yet, while mouses can be used to type, keyboards cannot be used to navigate. Mouses also can inefficient when having to navigate through multiple UIs to select numerous buttons and also be harder to use with medical issues such as Essential Tremors or Parkinson's disease, which can cause tremors that make it difficult to move a mouse with precision. KeyFlare will solve this problem by allowing you to navigate with your keyboard. Through the use of an AI-powered image classification system, the user can simply type "f" and be presented with a list of options of places they can click. This list will updated in real-time and organized through an algorithm that will perform OCR and icon classification to process screenshots, filter out irrelevant ones, and provide an enhanced experience through data collection.

          My project aims to create an AI-powered image classification system that enables more efficient and accessible user interface navigation through keyboard inputs, specifically targeting users with motor impairments, such as Essential Tremors or Parkinson's disease. By offering an intuitive, keyboard-based navigation system, this project has the potential to create a new market for AI UI controllers and improve the user experience for a wide range of individuals. My project has the faculty mentor, my AP Calculus BC teacher Mr. Kraemer, and an outside mentor, Architect and Senior Software Developer Mr. Honnudike.

          The most popular alternative called Vimium C is available for only chrome through the use of identifying javascript UIs. Vimium C has a steep learning curve (the numerous keyboard shortcuts to memorize and a visually unappealing and difficult UI exemplifies how the software was neither built by nor made for normal users), limited compatibility (it only works well with Firefox and Chrome), and a stagnant community. The project will hope to make a more competitive product compared to Vimium C.

Project Background

AI Writer: A keywords approach to AI explainability for word-based AIs

Recent developments in word-based AI models, such as chatGPT, could revolutionize various fields, including education, writing, and accessibility to diverse content mediums, so advances in these fields can offer real value to the world. To gain insights into the decision-making processes of these models, I propose a novel approach that identifies the most influential words in shaping an AI's output. My approach involves using the Natural Language Toolkit (nltk) for text preprocessing, including parsing the input text into word-only tokens based on the Wikipedia WordNet corpus (which contains 234,936 words). I then used GPT-2 to gather training data by removing individual words from the input and determining their impact on the model's output. I then performed statistical analyses on the frequency of change over the distance from the output and the accuracy of the model. To classify the influential words, I trained a keyword classification algorithm using the bag-of-words algorithm, multinomial Naive Bayes model, and nltk text preprocessing. With a brute-force approach similar to attention weights for collecting data, I performed a statistical analysis. Remarkably, I discovered that a prediction accuracy of 50% can be achieved using significantly fewer words compared to a prediction based on over 5.8 times the amount of text, suggesting the existence of certain influential keywords that can significantly improve the efficiency of a prediction. By developing this model as capable of learning from any text, future research can analyze and gain insight into a myriad of human works. Overall, my proposed approach has the potential to provide valuable efficiency improvements and insights into the decision-making processes of word-based AI models.  This research suggests a greater focus should be placed on Natural Language Processing’s preprocessing of text.