Machine-Learning

Exercise and Project Responses

View project on GitHub

Project 1

Social Distance Detector

For this week’s project, I implemented a social distance detector with two different videos of people walking in large crowds - one taken from practically a bird’s eye view and one taken from the horizon. I have attached these videos below, along with their respective output videos. In the repository, the bird’s eye view is saved as “PeopleWalkingFromAbove.mp4” and the horizon view is saved as “CrowdOfPeople.mp4”. Their respective outputs are saved as “PeopleWalkingFromAboveOutput.avi” and “CrowdOfPeopleOutput.mp4”. “CrowdOfPeopleOutput.mp4” is slightly shorter (with a duration of 10 seconds) than the video below (with a duration of 16 seconds) because of the large file size.

1. Was your social distance detector effective at detecting potential violations?

  • I think my social distance detector was quite effective at detecting potential violations, especially if the video was taken from a higher angle. The first set of videos below shows a crowd of people mulling through a lobby. It was filmed from an angle high above the crowd, which allowed for a clear view of each person and thus a pretty accurate measure of distance between each of them. Consequently, the detector did a great job at highlighting violations. By contrast, the second set of videos below was filmed within a crowd. As a result, it may be difficult for the detector to get an unobstructed view of the entire crowd unless it is filmed from high above the location. For example, there were some instances in the video footage where people overlapped others around them as they passed through the videoframe. Similarly, there were a few instances in the video where someone was walking directly in front of, or directly behind, a separate group of people. This would likely make it challenging for the social distance detector to accurately determine how far apart this person is from the group. Nonetheless, it performed well in effectively detecting potential violations at this lower angle. Even if these potential concerns had caused the social distance detector to falsely detect a few violations, it would be better to err on the side of caution given the potential severity of the consequences if someone were to contract an illness, especially during the COVID-19 pandemic. When it comes to protecting public health, it could be acceptable for a social distance detector to be more sensitive and falsely identify a few potential social-distancing violations, as this would allow the detector to warn people that they are starting to get too close before it happens.

2. Do you think this approach would be effective for estimating new infections in real time? How would you implement such an approach in response to the COVID-19 pandemic we are currently experiencing?

  • I think this approach could be very effective for estimating new infections in real time, but I think it would be more effective in preventing the spread of infections. This could be implemented in high-traffic areas, like parks or shopping centers, so that a speaker system could announce when it sees violations and advise people to move farther apart. The only downside of using a social distance detector to provide warnings is that the system could be overwhelmed by the numerous violations it sees when families walk through the videoframe and constantly have to repeat its announcements, even if the people are quarantining together or just barely violated social distancing for a split second as they passed each other on the sidewalk. To prevent this, you could adjust the announcements so that they only occur when a certain number of people are violating social distancing rules, and when fewer than that particular number are violating social distancing rules for a prolonged period of time. For example, rather than announce whenever it sees two people quickly pass each other on the sidewalk, the detector could advise people to move farther apart when it sees four people approach from different directions (since they arrived separately, which would indicate they are not quarantining together). Or, it could advise people to move apart when it sees more than two people standing/walking together for more than 30 seconds. This social distance detector would be an extremely beneficial component of the response to the COVID-19 pandemic we are currently experiencing because the virus is most easily spread through air droplets among people within 6 feet of each other: The Importance of Social Distancing.

3. What limitations or improvements might you include in order to improve your proposed design?

  • A limitation to my proposed design would be the previously mentioned issue with the detector being unable to keep up with announcing every single violation. My suggested way to improve upon this limitation (please see my response to Question #2 above) was to essentially make the detector less zealous in alerting people to social distancing violations. However, this “improvement” is also problematic, because people can still catch an illness like COVID-19 within seconds of standing/walking within 6 feet of someone who just coughed, for instance. Perhaps this could be improved upon by providing everyone with their own personal social distancing detector that alerts them when they are getting too close to someone. This solves the problem of a single detector being overwhelmed with violation alerts and would likely be better at capturing people’s attention.

    Another improvement to my proposed design would be to utilize a camera with a bird’s eye view of the location in which the detector is placed. This allows for a more accurate calculation of distances between people than a camera that is placed within a crowd.

Bird's Eye View Bird's Eye View Output

Horizontal View Horizontal View Output

Face Mask Detector on Image

After completing the social distance detector, I also decided to implement a mask detector using the image below. The detector very confidently assessed that the man to right is not wearing a mask - in fact, it predicted with 100% confidence that there wasn’t a mask on his face. The detector was also very confident in its prediction that the man to the left is wearing a mask. However, it was approximately 2.67% less confident in this prediction than in its prediction for the man to the right. Perhaps this slight decrease in confidence is due to the fact that the man on the left’s mask is slightly crooked and is not fully covering his nose. Another direction to take with this face mask detector would be to adjust for different positions of the mask and to be able to detect masks on people outside of the foreground of the images provided.

Face Mask Detector on Video

I then decided to implement the face mask detector with a video stream. I provided the detector with a video of a man putting on a face mask as input and it was mostly accurate in its predictions - both when the man was and wasn’t wearing a mask. In fact, it was completely accurate when the man had not yet put on a face mask at the beginning of the video, and also when the man had finished putting on a face mask at the end of the video. The mask detector was approximately 98% confident in its prediction that the man wasn’t wearing a mask in the beginning, and around 96% confident in its prediction that he was wearing a mask at the end of the video. I have attached both the input and output videos below. They are saved in the repository as “wearing_mask.mp4” and “wearing_mask_output.avi”, respectively.

The only time I found that the face mask detector was inaccurate in its prediction was when the man was first putting his face mask on. At the 1 second mark of the output video, when the mask is stretched out to place the straps on his ears, the detector is approximately 75% confident that the man is not wearing a mask, even though his mouth and nose are covered. While the detector wasn’t very highly confident in this decision, indicating that it realized it may have been incorrect, it may have mistakenly predicted he was not wearing a mask because it was not conformed to his face shape. This error could also have occurred because the mask was not yet positioned very high up on his face, causing some of his nose and the area below his eyes to be uncovered.

Shortly after this initial error, while the mask was still being placed on his face, another error in the detector’s prediction occurred. Similar to the first error, the detector likely falsely predicted he was not wearing a mask because it was still stretched out and did not yet conform to the shape of his face. This time, however, the face mask detector was less sure of its prediction that he was not wearing a mask, at only about 56% confidence.

Putting on Face Mask Putting on Face Mask Output

4. Do you think implementing a face mask detector could add value to a social distancing detector?

  • Adding a face mask detector to the social distancing detector would be very valuable, because face masks and social distancing go hand-in-hand in preventing the spread of illness. Earlier, I had suggested that the social distancing detector be implemented to announce warnings when people are violating social distancing protocols. If a face mask detector were added to this, the warning system could be even more effective at reducing the spread of disease because it could be adjusted to be extra sensitive to social distancing violations by people not wearing masks. It could also provide warnings to those who are not wearing masks and advise them to put on a mask before entering a building. For example, the combined detector could be used in classrooms to remind students not wearing masks to put on a mask before entering the room, as well as to warn students of social distancing violations as they select a seat.