About the Refrigerator
Our device consists of three main functions: food recognition, food rotten alerting, and recipe recommendation
We used AWS to store and deal with datas
We developed Android application to deploy monitoring and recommendation results
We used AWS to store and deal with datas
We developed Android application to deploy monitoring and recommendation results
System Block Diagram
We build a IOT system on top of Raspberry Pi, Amazon EC2, dynamoDB, and mobile phone. Below is our system block diagram:
We use raspberry Pi and camera to take the picture of the fruit, store the information in dynamoDB and S3, use EC2 server to do both the recommendation and alter, and host a server for Android Application.
|
We use a weight sensor to detect whether there’s fruit and then take picture if the fruit is put on our test table. We send the data we collected to database, do fruit classification based on the picture and send back the results to the database. We also retrieve the data periodically from database and check them, and then send alert to users and tell them what fruit are going to expire. Our model can do recommendation based on the data of fruits that users have eaten. All of these information are presented to end users using Android Application. We send the specific data to users when they request.
|
Hardware Support
The hardware consists of a raspberry pie and a camera. The raspberry pie is connected to a weight sensor and the camera. While we are running our server, at the time when the weight sensor detected something is placed on it, the camera starts to take one picture once. After the pictures have been taken, they are uploaded into AWS S3. Then these pictures can be loaded to EC2 and do image identification.
Achieving three main functions
Food RecognitionWe trained a Convolution Neural Networks (CNNs) to recognize the photo. Instead of training a new model, we use existing trained CNNs to extract features and finetune the model with Kaggle’s fruit360 Dataset. Our model can now recognize 81 different kinds of fruits.
|
Food Rotten Alerting Tons of fruit got rotten every day and they are thrown into the trash. Our application stores all the information about the items in the refrigerator, when the system consider if there is some food that will no longer nourishing in a few days, it would send a reminder to the user that they should eat them to prevent waste.
|
Recipe RecommendationWe do recommendation based on the fruit that the uses have taken. We store the weight of the fruit when the user put a fruit on weight scale. We also record the time that users eat the fruit if they decrease the amount of fruit on Android application. Then we calculate all of vitamin that users have taken based on the historical data. We calculate vitamin A, B1, B2, B6 and C.
|