Harvard University The Politics of Personal Data Gov 1430

Home | Schedule | Syllabus

Back to Lab

Lab 1 Supplement: Working With Images

Goal.

In this lab, you will experiment with detecting changes in jpeg images.

Getting Started

Divide yourselves into groups of 2 or 3 people. At least one person in your group will need a machine capable of executing java. You will not have to write any Java code, but you will have to execute it (or possibly compile it). Try to work with students with whom you have not yet worked. Select one student in the group to be a group leader. If there is a student in the group who has not yet been a group leader, then that students should be the group leader. Once decided, the group leader should send Professor Sweeney an email message to let her know the members of your group.

Part I. Automated Video Surveillance (30 minutes)

For the first activity in this lab, you will take a set of consecutive still images available from an on-line camera and detect changes in the images to see how such changes could be used for automated video surveillance. Below are the steps you should complete.

1. Find 2 camera URLs. Search Google for webcams showing the kinds of scences described below. Make sure the camera is updating images.

   Find a camera that provides still images of a relatively calm scene with little or no changes. Record the URL. This will be camera C, named after the calm scene it shows.

   Find a camera that provides still images of a scene having reasonable or constant changes. This will be camera A, named after the active image it displays. Record the URL.

2. Capture a consecutive sequence of images.

   For each of the 2 camera URLs you found, you will need to capture a sequence of images. To do this, visit the URL. The image should appear as a JPEG (.jpg) image in the web page. You can right click on the image and save a copy of the image file to disk or do a screen capture of the browser window. Save a sequence of 6 consecutive images. These images should be the next image immediately available from the camera. It is very important to have the next available image from the camera, especially for scenes that have changes between images. For the calm camera, C, the images must be named CP1.jpg, ..., CP6.jpg. For the active camera, A, the images must be named AP1.jpg, ..., AP6.jpg.

3. Email the original images to Professor Sweeney (latanya@mit.edu). She will put these online for the class to share.

Discussion Questions

• What are some interesting differences? Volunteers accepted to show some image results.
• What can be detected and what are possible uses of this kind of technology?
• What kind of false signals might result?

Part II. JPEG Notes

PEG images consists of pixels displayed in a rectangular format having a given width and height. On left below is a JPEG image (blown up by a factor of 100 with borders addded) representing an image having 4 pixels across and 2 pixels down. To the right of the image is a listing of the pixel values, numbered left to right, top down. (Java code)

For manipulation in the Java code presented on the links in this Lab, JPEG images are read from disk and then stored in a one-dimensional integer arrays for processing. In one version of the one-dimensional array, each pixel is a cell in the array. Pixels appearing in the rectangular view, left to right and top to bottom, are stored in the on-dimensional array. The table above shows how the pixels would be referenced in the rectangular view and the table below shows the cell positions those same pixels would be stored in the one-dimensional array.

Each pixel value is a 24-bit field which decomposes into the concatenation of 3 8-bit values (0 to 255). These three 8-bit values represent the amount of red, green and blue appearing in the pixel color. The first 8 bits are the amount of blue, the second 8-bits represent the amount of green, and the last 8 bits are the amount of red.

In the sample JPEG image above, the following colors are being displayed in the cells.

As shown above, 0,0,0 is black and 255,255,255 is white. In the other version of the one-dimensional array used in the Java code, each pixel uses 3 consecutive cells in the array, rather than one. In this version, the first cell assigned to th pixel stores the integer value for blue, the second for green, and the third for red.

Part III. Statistical Comparison of Images (15 minutes)

Compute the average image for your series. For each camera, A and C, you have a sequence of images [CP1, ..., CP6] and [AP1, ..., AP6]. Using the Java code, generate corresponding matrix files, avgc.jpg and avga.jpg for [CP1, ..., CP6], and [AP1, ..., AP6], respectively, that is the pixel-wise average of those sets of images. Below is a copy of the header comment from the file.

 ** This application achieves the single operation of producing a 
 ** JPEG file whose pixel values are the color value average
 ** of the images appearing in the given directory.
 **  
 ** Usage: java JPGavg dirname avg.jpg

Email these two images to Professor Sweeney for sharing with the class. Examine how the average image compares with the original images.

Compute the pixel-wise standard deviation of your images. Using the Java code, generate corresponding matrix files, stdc.jpg and stda.jpg for [CP1, ..., CP6], and for [AP1, ..., AP6], respectively, that is the pixel-wise standard deviation of those sets of images. Below is a copy of the header comment from the file.

 ** This application achieves the single operation of producing a 
 ** an JPEG file whose pixel values are the standard deviation 
 ** of the color values found in the images appearing in the 
 ** given directory using the avg.jpg.
 **  
 ** Usage: java JPGstd avg.jpg dirname std.jpg

Email these two images to Professor Sweeney for sharing with the group. Examine how the image of standard deviations compares with the original images.

Report comparative statistics to determine the similarity of images.
Using the Java code, generate corresponding matrix files, stdc.jpg and stda.jpg for [CP1, ..., CP6], and for [AP1, ..., AP6], respectively, that is the pixel-wise standard deviation of those sets of images. Below is a copy of the header comment from the file.

 ** This application displays the number and percentage of pixels
 ** for each JPEG file in dirname that is 1, 2, or 3 standard
 ** deviations from avg using std.
 **  
 ** Usage: java JPGstats avg.jpg std.jpg dirname

Examine the results and notice how similar they appear.

Part IV. Statistical Comparison of Difference Images (15 minutes)

In Part I, you compared adjacent images. In part II, you compared a set of images together (using an average image). Now you will compare the differences of the images from the average image.

Make a new directory to hold your "difference" images. These are the original images less the average. The result shows the extent to which the original image differs from the group average.

Using the Java code, generate corresponding matrix files for [CP1, ..., CP6], and [AP1, ..., AP6], respectively, that report the pixel-wise difference of those images from the average image, computed above. Below is a copy of the header comment from the file.

 ** This application reads the JPEG files in the indir and 
 ** writes a corresponding JPEG file in the outdir that has its 
 ** pixel values reduced by those in a.jpg. The absolute value of 
 ** the difference in the color values for each pixel is stored 
 ** in the outdir image. Files not ending in .jpg are ignored. 
 **  
 ** Usage: java JPGdiffAll a.jpg indir outdir

Re-compute the average and standard deviation images based on the difference images. In the work above, you used the regular images. This time you will only use the difference images.

Re-run the comparative statistics based on the difference images. Write down these results. You can cut-and-paste them to a text file for review later. How do they compare with your earlier findings?

Discussion Questions

Copyright © 2013-15 President and Fellows Harvard University | Data Privacy Lab