Project3 - Guanyou Li

Part 1. Defining Correspondences

We need to manually define pairs of corresponding points on the two images. It’s essential to maintain a consistent order of these points on both images so that each point in Image A maps to the same point in Image B, ensuring a coherent morphing process. Even if the shapes or structures of the images differ, consistent point labeling will make the transformation process smoother.

Once the corresponding points are defined and displayed, the next step is to perform a triangulation. The purpose of triangulation is to divide the images into a series of non-overlapping triangular regions, which helps maintain the local structure during the morphing process.

Delaunay triangulation is a commonly used method as it avoids producing overly skinny triangles, which might lead to distortions. The triangulation is applied to one set of points only, but it’s best to use the average shape (the midpoint of corresponding points from both images) to reduce triangle deformations during morphing.

Part 2. Computing the "Mid-way Face"

We need to calculate the "mid-way face" between the two images. This involves determining the average of each corresponding point’s position from Image 1 and Image 2. The resulting average shape is a set of points that serves as the target for morphing both images. This step ensures that both images are warped towards a common shape, facilitating the blending process.

Once the average shape is defined, the next step is to warp both Image 1 and Image 2 to this common shape. The core of this process involves applying an affine transformation for each triangle in the triangulated mesh.

For each triangle, a transformation matrix is computed that maps the original triangle in the source image to the corresponding triangle in the average shape. This transformation matrix allows for mapping points from the source image to the average shape. Importantly, the warping is done by computing an inverse mapping, which ensures that all destination points are covered correctly without any gaps.

Part 3. The Morph Sequence

The goal of this process is to morph two images by gradually transforming one into the other. This is done by creating intermediate shapes and colors based on point correspondences between the two images.

Two main parameters, warp_frac and dissolve_frac, control the process. warp_frac determines how much of the shape warping is applied, and dissolve_frac controls how much of the color blending happens. Both parameters start at 0 and gradually increase to 1.

Part 4. The "Mean face" of a population

The first step involves selecting a facial dataset with annotated keypoints (facial landmarks). The keypoints are used to describe the geometry of each face in the dataset. Based on these points, the "average face shape" of the population (or a specific sub-population, e.g., age, gender, ethnicity) is computed.

The average shape is calculated by taking the mean of all the keypoints from each face in the dataset. This average shape represents the common geometry of the entire population and serves as the target shape for warping each individual face in the dataset.

Part 5. Caricatures: Extrapolating from the mean

First, the point correspondences (keypoints) of your face and the average shape calculated in the previous step are used. The difference between the keypoints of your face and the average shape is computed.

This difference represents how much your face deviates from the average facial geometry. For example, if your nose is slightly longer than average, this difference will reflect that deviation.

Part 6. Bells and Whistles

The program reads two images: one is the example face, and the other is the average face of the desired demographic (e.g., "Chinese average woman" in this example). Each image has corresponding keypoints (landmarks) stored in a JSON file, defining important facial features like eyes, nose, and mouth.

Both images are warped to this average shape using an affine transformation. This involves calculating a transformation matrix for each triangular region (from the triangulation of the average shape) and mapping it to the corresponding region in the original images.