Carnegie Mellon University

Data Privacy Center

Data Privacy Course


Project Track 3: Ubiquitous Video




In Lab 5, you had the opportunity to demonstrate how a simple automated surveillance system might work using the cameras available in SOS CameraWatch. In this track, you will write programs that detect or extract specific kinds of information from images that may be captured on publicly available cameras. These extensions are related to services we discussed in lecture that are scheduled to be added to CameraWatch.

Assignment 1

The first assignment you must complete if you want to do a project in this track is to do an experiment to determine how many cameras in a a set of cameras are operational.

  1. Select 30 URLS of cameras from CameraWatch that provide still JPEG images. Do not use streaming video or zoom cameras for this experiment.

  2. Write a program that automatically loads the JPEG image from a camera URL and saves it to disk.

  3. Using your program, download a copy of the JPEG image from each camera URL and save it to disk. Some time later download a subsequent image from each camera URL. Compare the second image to the first to see if there has been any change in the captured video image. Select a time interval such that if the camera is working, the image should have been updated and would therefore the two images would not be the same. Use the difference in images as a way to verify whether the camera is working.

Using the steps above, conduct an experiment to determine how many of the 30 selected cameras are working. Repeat the experiment on a subsequent day (or optionally, over multiple days) and report which of the cameras are working each day tested. For each camera URL tested, visit the site and record how often your program was right or wrong in predicting the operation of the camera. For more improved results at estimating the "uptime" of these cameras, you may elect to select two different sets of 30 cameras on each day tested.

Write a 2-page report on your experiment. In the methods section, you can describe your program generally and then include a copy of your program as an appendix. Give a 5-minute presentation to the class on your experiment and findings.

Note. You must complete this this task within the first weeks of the course. You may complete assignment 1 and then later change your mind about which project you will in fact provide as your term project, provided your final decision occurs prior to the second project assignment and is approved by the instructor. See the course schedule.


Project 3-1: Track Volume of Entities Using Camera Images

For a particular camera in the SOS CameraWatch database that displays a sidewalk or highway, write a program that tracks the volume of pedestrians or vehicles that appear each hour of daylight (or alternatively at night). After operating for awhile, some patterns on daily and weekly volume should emerge. Your program should keep a log of the number of entities (pedestrians or vehicles) that appeared.

One way to accomplish this feat by tracking people is to use Henry Schneiderman's face detection program, as was used in Lab 6. Your program will most likely have to resize the images to make them sufficiently large enough to process faces. You will then have to count the number of faces that typically go undetected, so as to predict the total number of people from the number of detected faces.

A way to accomplish this feat by tracking vehicles during daylight is for you to write a routine similar in spirit to Henry Schneiderman's face detection program. In this case, it would be a vehicle detection program. First, you identify the parts of the image where vehicles are likely to appear. Then, you devise templates for the general shapes or features vehicles are likely to have. You can then use your own routine to track vehicles occuring in the images. In scoring the correctness of your program, you will have to count the number of vehicles that typically go undetected (or conversely any over counts due to false positives), so as to predict the total number of vehicles from the number of detected vehicles.

A simple way to track vehicles at night is to use the headlights in an otherwise dark scene. In this case, you write a program that tracks the number of headlights and rear lights appearing in the scene and use these values to predict the number of vehicles. In scoring the correctness of your program, you will have to count the number of vehicles that typically go undetected (or conversely any over counts due to false positives), so as to predict the total number of vehicles from the number of detected vehicles.

You may elect to do any of these three approaches. You do not have to do more than one approach for one camera. However, we do not have control over these cameras, so you may want to have a back-up URL for which your program works in case the original URL becomes inoperable.

Assignment 2. Report your algorithmic design for the approach you will take. If you elect to do the first approach, which uses Henry Schneiderman's face detection algorithm, then your algorithmic design will consist of the enlargement routine you will write. Your approach will become a web resource. As you progress in this project, you may further modify or even abandon your original design. That is allowed, but for assignment 2 you report your original algorithm.

Final report. Complete your program and gather findings. Based on your findings, explore the use of the program for surveillance purposes. Express benefits as well as privacy concerns. Propose a policy, technology or experiment that would help allow the benefits while addressing the concerns. Write a final report for the project and prepare and conduct an in-person poster presentation of your work.

Graduate credit: If you are taking this course for graduate credit, you must complete the analysis for at least 3 different cameras. Rather than writing a project report, you will write a conference-style paper on your work.


Project 3-2: Identify Vehicles in Video Images

For a particular camera in the SOS CameraWatch database that displays a highway or street scene of vehicles, write a program that logs the date and time and kind of vehicle appearing. This log can report operation for a time period in regular intervals during daylight (or alternatively at night). Below is an overview of steps for this project.

  1. Select a camera URL from CameraWatch. You may want to get more than one URL in case a camera becomes inoperable you will have a back-up. You may elect to use either a color or a grayscale image. You may elect either a still or video image.

  2. Design a simple algorithm for detecting vehicles. You may want to begin with simple shapes of vehicles (i.e., car, van, truck), or if you have a color camera, you may want to use color. Test your algorithm on some images captured from the camera.

    You may want to write a routine similar in spirit to Henry Schneiderman's face detection program. If so, you would identify the parts of the image where vehicles are likely to appear, and then devise templates for the general shapes or features vehicles are likely to have.

  3. Write a program based on your algorithm that logs vehicles appearing in an image. In scoring the correctness of your program, you will have to count the number of vehicles that typically go undetected (or conversely any over counts due to false positives).

Assignment 2. Report your algorithmic design for identifying a vehicle. This will become a web resource. As you progress in this project, you may further modify or even abandon your original design. That is allowed, but for assignment 2 you report your original algorithm.

Final report. Complete the tasks above. Run the the program for a stretch of time on two or more days. Confirm the correctness of the logs generated. Based on your findings, explore the use of the program for law-enforcement purposes. Express benefits as well as privacy concerns. Propose a policy, technology or experiment that would help allow the benefits while addressing the concerns. Write a final report for the project and prepare and conduct an in-person poster presentation of your work.

Graduate credit: If you are taking this course for graduate credit, you must complete the analysis for at least 3 different cameras. Rather than writing a project report, you will write a conference-style paper on your work.


Project 3-3: Weather Predictions from Video Images

For a particular camera in the SOS CameraWatch database that displays an aerial view. You will write a program that predicts aspects of the weather -- amount of cloudiness, or amount of precipitation (such as rain) -- found in images captured by the camera. Such a program could be useful at predicting "pinpoint" weather conditions. For example, the weather in Pittsburgh may be rain, but often not all locations throughout Pittsburgh are actually receiving rain! A pinpoint weather report can identify places within the city where there is rain or not. You will compare your program's predictions to the actual weather report and your inspection of the image. Below are the basic steps in this project.

  1. Select a camera URL. In fact, you should select two in case one becomes inoperable. In selecting a camera, you cannot select a location in which the weather does not change very much! You must also select a camera in an area where you can get weather reports (even though they may not always be correct!).

  2. Make a sample database. Grab some images from the camera (at roughly the same times of day) and append to each the weather reported at that time. By weather, we are interested in cloudiness and precipitation.

  3. Using your sample database, you may be able to use measures related to those computed in lab 6 to predict the weather condition. You do not have to use the measures reported in Lab 6. You may come up with your own measures if you prefer.

  4. Design an algorithm that given a captured image at the location and at the designated time of day will predict the weather at the location.

  5. Write a program based on your algorithm that predicts the weather. In scoring the correctness of your program, you will have to visually inspect the correctness of the program against the weather report and the actual image. Remember: the weather report may not be correct.

Assignment 2. Report your algorithmic design for predicting weather. This will become a web resource. As you progress in this project, you may further modify or even abandon your original design. That is allowed, but for assignment 2 you report your original algorithm.

Final report. Complete the tasks above. Run the the program for some days and record its correctness. Based on your findings, explore the use of the program for predicting pinpoint weather conditions. Express benefits as well as privacy concerns. Propose a policy, technology or experiment that would help allow the benefits while addressing the concerns. Write a final report for the project and prepare and conduct an in-person poster presentation of your work.

Graduate credit: If you are taking this course for graduate credit, you must complete the analysis for at least 3 different cameras. Rather than writing a project report, you will write a conference-style paper on your work.


Spring 2004 Privacy and Anonymity in Data
Professor: Latanya Sweeney, Ph.D. [latanya@dataprivacylab.org]