Good afternoon. Today we will talk about the image formation, types of images, and their analysis and processing. Formation images The formation of a digital image in the camera occurs as follows. Light reflected from the objects in the scene passes through the lens or lens system of

the camera lens and is focused on the sensor, which consists of photocells covered with light filters. During the image formation process, various distortions occur, such as Raditional distortion due to the geometry of the lens Claritium to reflection in the optical system Blurred errors of the image due to focusing or shutter speed errors

Darkened or overexposed parts of the image Most of these distortions can be compensated for using digital image processing techniques. To simplify the mathematical description of the image formation process, the so-called pinhole camera model is often used in which it is assumed that light rays pass through a small hole and hit the center, as feature 1.

In this case, the dependence of the coordinates of the projected point and the point in the world coordinate system is described by the perspective projection equation. Technician 1. Pinhole. As a pinhole, other names stand out, or a camera obstacle.

A lens is a camera with a very small lens opening, less than 1 mm. The operating principle of a pinhole camera is quite simple. Light rays reflected from the subject pass through an opening in the camera. They create an inverted image with a light-sensitive meaning, with a graphic film, with a graphic paper, or the digital matrix.

This we can see on a formula. Where f is the local lens, as p, x, eg and z coordinates, or is the coordinate of the point in the camera coordinate system. Where p, as x, eg and z, is the coordinate of the point in the world coordinate system.

Figure 1. Data from the sensor elements is read in two dimensional array-holder restaurants. The element of the restaurant is a pixel, and each pixel can contain one or more values depending on the type of image. It should be noted that computer vision is used not only for processing and analyzing images generated by color or black and white cameras,

but also by devices that allow you to see a scene in the Ephraim, millimeter and other ranges of the electromagnetical spectrum. Image types. The first definition is an analog image is a two-dimensional image as f, x and eg, characterized by infinite accuracy of representation in the spatial parameter to x and eg,

and infinite accuracy of representation of intensity values at each spatial point is x and eg. Definition stream, that's a digital image, is a two-dimensional array as i, r, uc of elements are pixels that represent one or more discrete values.

The next definition, that's an intensity function, is a mathematical representation of an image as a function f, x2, eg or y, depending on two spatial variables x and y. The x and y variables take real values that specify the position of a point in the image.

The f, x2, y values are usually also real and define the intensity of the image at a point. Next definition, that's a binary image, is a digital image, b, r, uc, which pixels take on the values 0 or 1.

The next definition, that's a halftone, is known as gray, monochrome, black and white. Image is a digital image, i, r, uc, in which each pixel corresponds to the intensity or brightness value. Next definition is a multispectral image is a digital image, m, r, uc, in which each pixel corresponds to a vector of values.

For color images, the dimension of the vector is 3. In the process of solving the computer vision problem, auxiliary halftone or binary images can be obtained from the original color image. Each class of images has its own processing methods. Let's consider methods for processing binary image.

Processing analysis of binary images. In a computer vision system, binary images are often used to highlight or mask certain areas of an image for later analysis. For example, highlighting areas of letters for their further recognition. In this case, non-zero pixels values highlight areas of interest.

Next feature is the binary images. The process of binarization in the conversation of a color or a grayscale image in the two colors black and white. The main parameter of this transformation is a threshold t, with the value of which the brightness of all it is then compared. After comparison, the pixel is assigned one of two possible values.

Zero is object boundary or one remaining area. The general binarization shape is presented in the features tree. And features tree is a general binarization shape. Whereas the main goal of binarization is to radically reduce the amount of information that has to be worked with.

Successful binarization greatly simplifies subsequent work with the image. There are various binarization methods which can be divided in two groups. First one is global, or threshold, and local. It's adapted. Global binarization methods work with the entire image at once. During the work, the binarization threshold t is found, with the help of which the division into black and white occurs.

And the value of the threshold t remains enchanted through the entire binarization process. Global thresholding. Stressful binarization method includes, first of all, binarization with a lower threshold, binarization with an

upper threshold, binarization with a double restriction, incomplete threshold processing, and a multilevel threshold transformation. One of the simplest methods of image transformation is binarization with a lower threshold in which only one threshold value is considered by formula. If in the above formula for an image point the first condition is met, then such a point is an object point.

But if the second condition is met, then the point will be a background point. In some cases, you can use a variant of the binarization method with a lower threshold, which results in a negative of the original image. This method is called binarization with an upper threshold and is represented by the next formula.

If it is necessary to select certain areas in which the pixel brightness values can vary within a certain range, then the double construct binarization method is used. This method is called binarization with an upper threshold and is represented by the next formula. If you need to obtain the easiest image for the first analysis, then it is worth using an equivalent thresholding

algorithm, during which the image is deprived of the background within all the details that were in the original image. The formula for the equivalent threshold binarization is presented below as formula 4. If you need to obtain an image that contains segments of different brightness, you can use the multilevel threshold transformation method.

But however, in this case, the image obtained during the conversion will no longer be binary. That's shown in formula 5. As an example of converting an original image using the threshold binarization method is shown in picture 4 and 5.

Using this method, a threshold T is calculated that minimizes the average segmentation error as the average error from deciding whether image pixels belong to an object or a background. The brightness values of image pixels can be considered as random

variables, and their histogram as an estimate of the probability density distribution. If the probability distribution densities are known, then the optimal, in the sense of minimal error, threshold for segmenting the image into two classes, as c0 and c1, objects in the background, and they can be determined.

The histogram is built according to the values p from i, that's n, where i divided to n. In this formula, n is the total number of pixels in the image with brightness level i. The threshold T is an integral value between 0 and the formula n, where it's maximum.

Using a histogram, all pixels are divided into a usable object and a background. Each type corresponds to related frequencies, away from 0 and v1, in the formulas 6 and 7. Next, the average levels for each type of image are calculated using formula 8 and 9. Next, we look for a threshold that reduces the variance with the pixel type, determined by the formula 10.

And the next step is to determine the inter-class variance using formula 11. Then the maximum value is calculated to assess the quality of dividing the image into two parts, which corresponds to the desired threshold. As for formula 12, the advantages of the odds method are ease of implementation,

adoption to various types of images by choosing the optimal threshold, and a fast execution time, filtering binary images, morphological filtering. During the execution of a binary image, noise pixels often appear that need to be filtered out.

Morphological filtering is often used to filter binary images. Primary mathematical morphological is used to assess the main properties of an image that are useful for its representation and description. For example, contours, skeletons, convex holes. Also of interest are morphological methods used at the stages of preliminary and final image processing.

For example, morphological filtering, sticking, or thinning. The input data for the apparatus of mathematical morphological are two images. A persist one and a special one, depending on the type of operation and the problem being solved.

Such a special image is usually called a primitive or structural element. As a rule, the structural element is much smaller than the image being perceived. A structural element can be considered a description of an area or some form. It is clear that the shape can be any. The main thing is that it can be represented as a binary image of a given size.

In the many image processing packages, the most common structural elements are special names. As figure 7, box from 6.6, figure 8, disc from 4, and figure 9, ring from 4. The result of morphological processing depends both on the size and configuration of the original image and on the structural primitive.

The size of the structural element is usually 3 to 3, 4 to 4, or 5 to 5 pixels. This is due to the main idea of morphological processing, during which characteristic details of the image are found. The desired detail is described by a primitive, and as a result of

morphological processing, such details can be emphasized or removed from the entire image. One of the main advantages of morphological processing is its simplicity. At both input and output of the processing procedure, you receive a binary image. Other matters as a rule first obtain a graceful image from the original image, which is then reduced to binary using a threshold function.

The main operations of mathematical pathology are growth, erosion, closure, and opening. These names reflect the essence of the operation. Extension increases the image area, and erosion makes it smaller.

The snapping operation allows you to close the internal opening of an area and eliminate holes along the edge of the area. The opening operation helps to get rid of small fragments from growing out of the area near its border. Let's look at specific example. Let us have the following binary image and structural element. As feature 10, binary image B.

And feature 11, structural element C. The structural element C is applied to all pixels of the binary image. Each time the origin of a structural element is aligned with a single binary pixel, a translation and subsequent logical addition is applied to the entire structural element. With the corresponding pixels of the binary image.

The results of a logical addition are written to the output binary image, which is initially initialized with zero values. As a feature 12, image extension B is a structural element C. And feature the erosion when conforming the erosion operation. The structural element also passes through the other pixel of the image. If at some position each single pixel of the structural element considered with a single pixel of the binary image,

then a logical addition of the central pixel of the structural element is performed with the corresponding pixel of the output image. As a result of applying the erosion operation, all objects smaller than the structural element are erased. Objects connected by the same lines become disconnected and the sizes of all objects are reduced.

The erosion operation is useful to form a moving small object and various noises. But this operation has a disadvantage, but all remaining objects are reduced in size. This effect can be avoided if at the erosion operation a bullet operation is used with the same structural element.

Unlocking eliminates all objects smaller than the structural element, but at the same time helps to avoid greatly reducing the size of object. Breaking is also ideal for moving lines with thickness is less than the diameter of a structural element. It is also important to remember that after the operation, the contents of the object become smoother.

Image softening by structural element C If we first apply segmentation operation to the image, we can get rid of small holes and cracks, but at the same time the outline of the object will increase. This increase can be avoided by erosion surgery performed immediately after extension with the same structural element.

Image freezing B by structural element C Conditional buildup One of the typical applications of binary morphology is the selection of components in a binary image with shape and size statistically given restrictions. In many similar problems, it is possible to construct a structural element that, when applied to a binary image,

removes components that do not satisfy exact constants and leaves a few single pieces corresponding to the components that satisfy constraints. But subsequent processing may require entire components, not just their fragments remaining after erosion. But to solve this problem, a conditional equipment operation was introduced.

The set obtained as a result of erosion is likely increased by the structural element S, and at each step, the result is reduced to the subsequent pixels that have unit values in the original image P. The operation of conditional freezing is explained in the figure 16. In the figure 16, binary image B has been eroded by V to extract components containing three pixel-high practical fragments.

The result in image C has two such components. But to highlight these components entirely, image C is conditionally augmented by element D related to the original image. Border selection Orthological operation can also be used to highlight the boundaries of a binary object.

This operation is very important because the boundary is complete and, at the same time, a very complete description of the object. It is easy to notice that the boundary points have at least one big brown pixel at their vicinity. But by applying the erosion operation in a structural element containing all the possible neighboring elements, we will remove all boundary points.

Then the boundary is obtained using the set difference operation between the original image and the image obtained as a result of erosion. Line filters Let the initial hop to image A be given and let us denote the limitations of its pixels as A X to Y.

The linear filter is divided by a real valid function f, defined on the raster. This function is called the filter canon, and the filtering itself is performed using this discrete convolution operation, or beta summation, as formula 13. The result is an image P. Typically, the filter canon is non-zero only in some neighborhood m of the point 0,0.

But outside of this neighborhood, f i to j is either exactly 0 or very close to the filter corner. The summation is performed over i to j, m, and the value of each pixel b, x to y, is determined by the pixels of image A that lie in the middle and center of the point x to y, where we will denote the set n x to y.

A filter corner defined on rectangle neighborhood n can be viewed as an m-by-n matrix, where the set lengths are odd numbers. When specifying the kernel by the matrix m, k, l, it should be centered as formula 14.

Condition and the border If the pixel x to y is in the vicinity of the edges of the image, in this case, a, x plus y, and e, g, plus j, it may correspond to a pixel a that lies outside of the image a. This problem can be resolved in several ways, as do not filter such pixels by cropping image b at the edges or painting them, for example with black.

Do not include the corresponding pixel in the summation, distributing its weight as f i to j evenly among other pixels in the neighborhood as n x to y. Additional determination picks the values outside the image boundaries using extrapolation.

Define pixel values beyond the boundaries of the image in the mirror reflection. Matrix image processing filters Convolution matrix Convolution matrix is a matrix of coefficient that is multiplied by the pixel value of the image to produce the desired result. PGA25 shows the application of a convolution matrix, and using convolution

matrix is a normalization coefficient so that the average intensity remains unchanged. In the image the matrix is 3 to 3, also the size could be larger. Medium filter Noise is a form of white or black dots in the image, it's pulse-type noise. Linear filters do not eliminate them completely, but only locally average their values.

This type of noise is removed using nonlinear filters, such as medium filters. Classic medium filtering uses the concept of a neighborhood and its center, but does not specify, baiting, or if it seems. It's feature 26. The medium filter is typically used to reduce noise or smooth an image. The neighborhood can have an arbitrary shape and size, and the center can be located arbitrarily related to the neighborhood.

When the center of the neighborhood is aligned with the analyzed pixel, the neighborhood becomes a window into which a number of pixels adjacent to the center fall. The brightness values of pixels that fall into the window are sorted in an ascending order.

The value of the average element in the row or median after sorting will be the result of medium filtering in this window. The window is then shifted and the procedure is repeated for all pixels in the original image. In practice, a window is sculpted in a rectangle and is shaped with odd numbers of elements, and the center is located at the geometrical center.

For example, let there be 9 pixels in a 3 to 3 window. After sorting their values as a result of medium filtering, the central pixel takes often the value 6, as feature 27, for the algorithm for the operation of the medium filter. But since a window can have an arbitrary shape, a rectangle mask is used to describe it, the elements of which contain the values 0 and 1.

Instead of 1 describing a window of the desired shape, during the filter process only pixels that correspond to non-zero elements of such a mask are involved in sorting. The weighted median filtering algorithm is feature 28, and the mask corresponding to the window of such a filter, in addition to 0 and 1, integers 2, 3, and others, are used.

They mean how many times repeat the brightness of the corresponding pixel before sorting. Typically, larger weights are placed closer to the center of the window to enhance the effect of the brightness of the central pixel on the result. The number of brightness values n involved in the sorting is equal to the sum of the mask weights.

The number of the median elements after sorting will be equal to n plus 1 divided by 2. The median filter has the following properties. It is non-separable, non-linear, only halved in images, does not introduce new brightness powers that are not present in the original image. And the last one relatively removes pulse-type noise. The median filter successfully improves scan all paper photographs with the white streaks and the folds.

Thank you for your attention. See you next time.