Carnegie Mellon University

Data Privacy Course

Objective

The objective of this lab is for you to explore benefits and privacy concerns related to video surveillance.

Considerations that should come to mind:

What is the near-term future of video surveillance?
What are the possible benefits and related privacy concerns?
What kind of privacy technology is needed?

Overview of activities

The in-class activities concern your designing a simple, automated surveillance system. The space you will survey will be public spaces available through publicly available, on-line cameras.

Part I. Automated Video Surveillance (30 minutes)

For the first activity in this lab, you will take a set of consecutive still images available from an on-line camera and detect changes in the images to see how such changes could be used for automated video surveillance. Below are the steps you must complete.

Find 2 camera URLs.
Remember the CameraWatch project examined earlier. This project consists of publicly-available, on-line cameras that are observing people and vehicles in public spaces. Visit the CameraWatch website and find cameras showing two kinds of scenes, as described below. (Or alternatively, search Google for webcams showing the kinds of scences described below.) The cameras must provide still images. For this activity, you cannot use cameras with zoom features or those showing streaming video. CameraWatch identifies whether the camera provides a still or video image. Also, make sure the camera is updating images. The date and time of the image typically appears on or near the image. You want a camera that provides still images that are being updated.
- Find a camera that provides still images of a relatively calm scene with little or no changes. Record the URL. This will be camera C, named after the calm scene it shows.
- Find a camera that provides still images of a scene having reasonable or constant changes. This will be camera A, named after the active image it displays. Record the URL.
Capture a consecutive sequence of images.
For each of the 2 camera URLs you found, you will need to capture a sequence of images. To do this, visit the URL. The image should appear as a JPEG (.jpg) image in the web page. You can right click on the image and save a copy of the image file to disk. Save a sequence of 6 consecutive images. These images should be the next image immediately available from the camera. It is very important to have the next available image from the camera, especially for scenes that have changes between images. For the calm camera, C, the images must be named CP1.jpg, ..., CP6.jpg. For the active camera, A, the images must be named AP1.jpg, ..., AP6.jpg.
Share the original images and the difference images.
Examine the original sequence of images for a camera. One way to do this is to FTP the images to a web-accessible location along with an HTML page that will display them for you.
1. FTP to dataprivacylab.org. Login as padstud with the password given in class.
2. Go to your assigned directory (the folder that bears your name!). Place your files within your assigned directory. Be sure to transfer files in "binary" mode. Below are the files you must FTP:
  - CP1.jpg, ..., CP6.jpg
  - AP1.jpg, ..., AP6.jpg
3. Vist URL to view your results, along with those of your classmates.

Questions

Lab Results

What are some interesting differences? Volunteers accepted to show some image results.
What can be detected and what are possible uses of this kind of technology?
What kind of false signals might result?

Part II. JPEG Notes

JPEG images consists of pixels displayed in a rectangular format having a given width and height. On left below is a JPEG image (blown up by a factor of 100 with borders addded) representing an image having 4 pixels across and 2 pixels down. To the right of the image is a listing of the pixel values, numbered left to right, top down. [This image was made using this Java code.]

0 1 2 3

4 5 6 7
For manipulation in the Java code presented on the links in this Lab, JPEG images are read from disk and then stored in a one-dimensional integer arrays for processing. In one version of the one-dimensional array, each pixel is a cell in the array. Pixels appearing in the rectangular view, left to right and top to bottom, are stored in the on-dimensional array. The table above shows how the pixels would be referenced in the rectangular view and the table below shows the cell positions those same pixels would be stored in the one-dimensional array.

0

1

2

3

4

5

6

7

Each pixel value is a 24-bit field which decomposes into the concatenation of 3 8-bit values (0 to 255). These three 8-bit values represent the amount of red, green and blue appearing in the pixel color. The first 8 bits are the amount of blue, the second 8-bits represent the amount of green, and the last 8 bits are the amount of red.

In the sample JPEG image above, the following colors are being displayed in the cells.

0 (blue=255, green=0, red=0)	1 (blue=0, green=255, red=0)	2 (blue=0, green=0, red=255)	3 (blue=255, green=255, red=255)
4 (blue=100, green=100, red=100)	5 (blue=200, green=100, red=200)	6 (blue=100, green=200, red=200)	7 (blue=0, green=0, red=0)

As shown above, 0,0,0 is black and 255,255,255 is white. In the other version of the one-dimensional array used in the Java code, each pixel uses 3 consecutive cells in the array, rather than one. In this version, the first cell assigned to th pixel stores the integer value for blue, the second for green, and the third for red.

Part III. Statistical Comparison of Images (15 minutes)

Compute the average image for your series.
For each camera, A and C, you have a sequence of images [CP1, ..., CP6] and [AP1, ..., AP6]. Using the Java code, generate corresponding matrix files, avgc.jpg and avga.jpg for [CP1, ..., CP6], and [AP1, ..., AP6], respectively, that is the pixel-wise average of those sets of images. Below is a copy of the header comment from the file.
```
 ** This application achieves the single operation of producing a 
 ** JPEG file whose pixel values are the color value average
 ** of the images appearing in the given directory.
 **  
 ** Usage: java JPGavg dirname avg.jpg
```
FTP these two images to your space on dataprivacylab.org. Examine how the average image compares with the original images.
Compute the pixel-wise standard deviation of your images.
Using the Java code, generate corresponding matrix files, stdc.jpg and stda.jpg for [CP1, ..., CP6], and for [AP1, ..., AP6], respectively, that is the pixel-wise standard deviation of those sets of images. Below is a copy of the header comment from the file.
```
 ** This application achieves the single operation of producing a 
 ** an JPEG file whose pixel values are the standard deviation 
 ** of the color values found in the images appearing in the 
 ** given directory using the avg.jpg.
 **  
 ** Usage: java JPGstd avg.jpg dirname std.jpg
```
FTP these two images to your space on dataprivacylab.org. Examine how the image of standard deviations compares with the original images.
Report comparative statistics to determine the similarity of images.
Using the Java code, generate corresponding matrix files, stdc.jpg and stda.jpg for [CP1, ..., CP6], and for [AP1, ..., AP6], respectively, that is the pixel-wise standard deviation of those sets of images. Below is a copy of the header comment from the file.
```
 ** This application displays the number and percentage of pixels
 ** for each JPEG file in dirname that is 1, 2, or 3 standard
 ** deviations from avg using std.
 **  
 ** Usage: java JPGstats avg.jpg std.jpg dirname
```
Examine the results and notice how similar they appear.

Part IV. Statistical Comparison of Difference Images (15 minutes)

Make a new directory to hold your "difference" images. These are the original images less the average. The result shows the extent to which the original image differs from the group average.

Using the Java code, generate corresponding matrix files for [CP1, ..., CP6], and [AP1, ..., AP6], respectively, that report the pixel-wise difference of those images from the average image, computed above. Below is a copy of the header comment from the file.
```
 ** This application reads the JPEG files in the indir and 
 ** writes a corresponding JPEG file in the outdir that has its 
 ** pixel values reduced by those in a.jpg. The absolute value of 
 ** the difference in the color values for each pixel is stored 
 ** in the outdir image. Files not ending in .jpg are ignored. 
 **  
 ** Usage: java JPGdiffAll a.jpg indir outdir
```
Re-compute the average and standard deviation images based on the difference images. In the work above, you used the regular images. This time you will only use the difference images.
Re-run the comparative statistics based on the difference images. Write down these results. You can cut-and-paste them to a text file for review later. How do they compare with your earlier findings?

Questions

Lab Results

What are some interesting differences? Volunteers accepted to show some image results.
What can be detected and what are possible uses of this kind of technology?
What kind of false signals might result?

Technical observations

We used simple difference matrices in this lab. What would be achieved if we used variance? How would it be computed? Give it a try and record and view your findings.

Spring 2006 Privacy Technology
Professor: Latanya Sweeney, Ph.D. [latanya@dataprivacylab.org]