Tags

#swift

#ARKit

#Machine Learning

#Flask

Introduction

At HackUTD X, one of the largest hackathons in the United States, my team and I embarked on an ambitious project. Our goal? To revolutionize the real estate industry by integrating cutting-edge artificial intelligence and machine learning technologies. Here's the story of how we created RenoVision, an innovative tool designed to transform property assessments.

The Genesis of RenoVision

The real estate industry has long relied on traditional property evaluation methods - often time-consuming and limited in scope. Recognizing these constraints, we set out to develop RenoVision, a solution aimed at bringing efficiency, accuracy, and depth to property assessments. Our inspiration stemmed from the desire to enhance the value proposition and customer experience in real estate by leveraging the power of AI and machine learning.

Innovative Tech Stack

RenoVision is not just an app; it's a synergy of multiple advanced technologies. The frontend was developed using Swift and Apple's ARKit. This allows users to take detailed scans of properties with only a phone camera. The app captures extensive data from these scans, including physical attributes and environmental conditions. This data is then sent to be processed in the backend, powered by Flask. First, it feeds into our TensorFlow model, which will assess the various aspects of the property to determine its condition. The model will determine whether any repairs or renovations are in order, as well as how certain features of the property can affect its value. Despite its simplicity, with just 18,817 parameters, our model achieved a loss of 0.0445 and a mean absolute error of 0.1559 compared to the training data. Finally, we produce a detailed narrative for the user using an LLM powered by LangChain and OpenAI.

We used TensorFlow and Google Cloud's Vertex AI to train a sequential neural network model that evaluates property conditions based on attributes like type (residential or commercial), and positive and negative features.

Google Cloud Integration

We initially ran into many computing and logistical roadblocks when developing RenoVision for commercial use. Integrating multidimensional LiDAR data with variable environmental factors was complex. Furthermore, the runtime of the models was difficult for some devices to handle. We ultimately overcame this by utilizing Google Cloud to streamline all aspects of our app. For example, we stored all of the property data on Firestore in order to store and sync all of the property data. Furthermore, our TensorFlow model was integrated into Google's Vertex AI, which allowed us to store everything on Cloud "Buckets" for easy access. This way, all of the app's computing and storage is handled remotely and does not rely on the individual's device.

Conclusion

I'm proud to have developed a full-stack mobile application that utilizes a phone's hardware to its full potential, combined with our machine learning and AI models. This project deepened my understanding of ARKit, machine learning, AI, and LiDAR technologies. The journey at HackUTD X was challenging, enlightening, and above all, a testament to the power of collaborative innovation in technology.