# Difference between revisions of "ParaView/Line Integral Convolution"

The line integral convolution(LIC) vector field visualization technique convolves noise with a vector field producing streaks along vector field tangents. Originally the technique was developed for use with 2D image based data but has since been extended to use on arbitrary surfaces and volumes. ParaView supports LIC on arbitrary surfaces via the Surface LIC plugin.

# REPORT

\section{Features for interactive data exploration} \begin{figure}[ht!]

\centering
\includegraphics[width=0.9\textwidth]{./images-data/dia-serial/color-ce-curves-khB-both-h.png}
% color-ce-curves-khB-both.png: 2175x2441 pixel, 72dpi, 76.72x86.10 cm, bb=0 0 2175 2441
\caption{Improving the transfer of scalar coloring via contrast enhancement stages. The top pannels show visualization of surface LIC of magentic field on a slice colored by magnetic field and the RGB color curves for the image. The bottom pannels show the same visualization but using the contrast enhencement stages. The contrast enhancement improves the effectiveness of the scalar coloring and LIC in the resulting image.}
\label{fig:cce-khb}


\end{figure} What characteristics in the output LIC make for an effective visualization across a wide variety of input data and rendering conditons? Some of the important characteristics in an effective surface LIC visualizaion are \begin{enumerate}

\item The LIC patterns accurately represent the characteristics of the underlying vector field. If desired the relative strength and features in strongly vearying fields should be maintained.
\item When scalar coloring is desired, LIC pattern must not be lost during the application of colors and similarly scalar colors once applied should not be dull or have greatly diminished intensity.
\item Over all a high dynamic range in the resulting image is desirable. This make it easy to identify feature in LIC and scalar colors. The lower the dyanmic range the less decernable detail there is in the result.
\item Lighting characteristics must be maintained, for example shadows, specular reflections, etc.
\item The algorithm should perform well on very large surface geometry


\end{enumerate}

\subsection{Scalar color shading} When thinking about how best to acheive these goals it's important to consider how scalar colored, lit geometry and and image LIC are combined to produce the final result. In our algorithm scalar coloring and lighting caclulations are rendered and stored in a texture. The image LIC is computed using projected vectors and also stored in a texture. There are two common approaches for combining these two textures into the final resut, a multiplciative or mapping approach\cite{image-lic} and an addative or blending approach\cite{surface lic}.

The mapping approach is described by the following equation: $$c_{ij} = ( L_{ij} + f ) * S_{ij} \label{eqn:color-map}$$ where the indices $i,j$ identify a specific fragment, $c$ is final RGB color, $L$ is LIC gray scale intensity, $S$ is the scalar RGB color, and $f$ is a biasing parameter, typically 0, that may be used for fine tuning. When $f=0$, the typical case, colors are transfered directly to the final image where the LIC is 1, and a linearly scaled transfer of scalar colors where LIC gray scale color is less than one down to 0, where the final color is black. The bias parameter $f$ may be set to small positiver or negative values between -1 and 1 to increase or decrease LIC values uniformly resulting in brighter or darker images. When $f\neq0$ final fragment colors, $c$, are clamped such that $0 \leq c \leq 1$.

With the mapping approach the distribution of intensity values in the LIC directly affect the accuracy and intensity with which scalar colors and lighting effects are transfered to the final result and the average brightness and contrast of the result. Note in the final result that individual RGB channel values will be less then or equal to the maximum grayscale value in the image LIC. Also, the greater the number of pixels close to 1 in the image LIC, the more acurately and intensly scalar coloring and lighting are transfered into the final image. However, this must be balanced with a sufficient number of highly contrasting pixels where the value is closer to 0 in order to accurately represent the LIC pattern. Put sucintly it's critical that the image LIC has high contrast and dynamic range with a good mix of light and dark values if it is to be effectively mapped onto scalar colors. However, the convolution process inheriently reduces both contrast and dynamic range in the image LIC. To correct this we've introduced contrast enhancment stages in three places in the pipeline. An example demonstrating the effectiveness of the contrast enhancement stages is shown in figure \ref{fig:cce-khb}. The top

The blending approach for combining scalar colors, lighting effects, and the image LIC is described by the following equation: $$c_{ij} = L_{ij} * I + S_{ij} * (1 - I) \label{eqn:color-blend}$$ where the indices $i,j$ identify a specific fragment, $c$ is final RGB color, $L$ is LIC gray scale value, $S$ is the scalar RGB color, and $I$ is a constant ranging from 0 to 1, with a default of 0.8. Decreasing $I$ to obtain brighter colors diminishes the intensity of the LIC, and vise versa. When colors are bright the LIC is difficult to see. Currently, the best results are obtained by sacrificing slightly on both fronts. The blending approach bennefits from an image LIC with high contrast and dyanmic range as this tends to make patterns in the image LIC more easily visible after blending. Note that despite the fact that it inheriently decreases visibility of features in scalar coloring and image LIC, the blending approach is especially useful with curved surfaces and pronounced lighting effects and also when scalar color map is very intense. \FloatBarrier

\subsection{Contrast enahancement} \label{sec:contrast} \begin{figure}[h!]

\centering


\begin{subfigure}{0.48\textwidth} \includegraphics[width=\textwidth]{./images-data/dia-serial/gray-ce1-curves.png} % gray-ce1-curves.png: 1187x1078 pixel, 200dpi, 15.07x13.69 cm, bb=0 0 427 388 \caption{} \label{fig:gray-ce-curves1} \end{subfigure} \hspace{0.15in} \begin{subfigure}{0.48\textwidth} \includegraphics[width=\textwidth]{./images-data/dia-serial/gray-ce2-curves.png} % gray-ce1-curves.png: 1187x1078 pixel, 200dpi, 15.07x13.69 cm, bb=0 0 427 388 \caption{} \label{fig:gray-ce-curves2} \end{subfigure} \begin{subfigure}{0.48\textwidth} \centering \includegraphics[width=\textwidth]{./images-data/dia-serial/color-ce-curves.png} % gray-ce1-curves.png: 1187x1078 pixel, 200dpi, 15.07x13.69 cm, bb=0 0 427 388 \caption{} \label{fig:gray-ce-curves2} \end{subfigure} \caption{} \end{figure}

The convolution process tends to decrease both contrast and dynamic range, often producing dark and dull results. Figure \ref{fig:motivation}.a illustrates this issue. The narrowing of dynamic range and diminishing of contrast are consequences of the convolution operation which convolves a noise texture with a vector field. These reductions are made worse by the use of Gaussian noise because by definition centrally concentrates its values about a mid gray tone.

In order to counteract this, optional contrast enhancement stages have been added. The new stages increase the dynamic range and contrast without modifying the patterns that emerge from the LIC process or modifying the scalar colors. VTK's surface LIC implementation is provided by two classes, the vtkSurfaceLICPainter and vtkLineIntegralConvolution2D. The vtkSurfaceLICPainter, projects vectors, sets up noise texture, and passes control to the vtkLineIntegralConvolution2D to generate a LIC, after which it shades scalar colors combined with the generated LIC onto the surface geometry. Contrast enhancement stages have been added to both classes.

% \subsection{vtkLineIntegralConvolution2D} The contrast enhancement stages are implemented by histogram stretching of the gray scale colors in the LIC'ed image as follows: $$c_{ij} = \frac{c_{ij} - m}{M - m}$$ where, the indices $i,j$ identify a specific fragment, $c$ is the fragment's gray scale color, $m$ is the gray scale color value to map to 0, $M$ is the gray scale color value to map to 1. When the contrast enhancement stage is applied on the input of the high-pass filter stage, $m$ and $M$ are always set to the minimum and maximum gray scale color of all fragments. In the final contrast enhancement stage $m$ and $M$ take on minimum and maximum gray scale colors by default but may be individually adjusted by the following set of equations: $$m = min(C) + F_{m} * ( max(C) - min(C) )$$ $$M = max(C) - F_{M} * ( max(C) - min(C) )$$ where, $C = \{c_{00},c_{01},...,c_{nm}\}$, are all of the gray scale fragments in the LIC image and $F_m$ and $F_M$ are adjustment factors that take on values between 0 and 1. When $F_m$ and $F_M$ are 0 minimum and maximum are gray scale values are used. This is the default. Adjusting $F_m$ and $F_M$ above zero controls the saturation of normalization. This is useful, for example, if the brightness of pixels near the border dominate because these are convolved less because we can't integrate outside of the dataset.

Occasionally, often depending on the contrast and dynamic range and graininess of the noise texture, jagged or pixelated patterns may emerge in the LIC. These can be reduced by enabling the optional anti-aliasing pass.

% \subsection{vtkSurfaceLICPainter} The color contrast enhancement stage is implemented with histogram stretching on the fragments lightness in the HSL color space. $$L_{ij} = \frac{L_{ij} - m}{M - m}$$ where, the indices $i,j$ identify a specific fragment, $L$ is the fragment's lightness in HSL space, $m$ is the lightness to map to 0, $M$ is the lightness to map to 1. $m$ and $M$ take on minimum and maximum lightness over all fragments by default but may be individually adjusted by the following set of equations: $$m = min(L) + F_{m} * ( max(L) - min(L) )$$ $$M = max(L) - F_{M} * ( max(L) - min(L) )$$ where, $L$ are fragment lightness values and $F_m$ and $F_M$ are the adjustment factors that take on values between 0 and 1. When $F_m$ and $F_M$ are 0 minimum and maximum are lightness values are used. This is the default. Adjusting $F_m$ and $F_M$ above zero provides fine-tuning control over the saturation.

\FloatBarrier

\subsection{Noise generator} \begin{figure}[ht]

\centering
\includegraphics[width=0.9\textwidth]{./images-data/dia-serial/ensight-blow-multiblock-frame.png}
% ensight-blow-multiblock-frame.png: 1414x1074 pixel, 72dpi, 49.88x37.88 cm, bb=0 0 1414 1074


\caption{Visualizations demonstrating affect of varying noise texture and scalar color shader. An Ensight Multi-Block dataset from a simulation of shell being crushed is used.

   {\bf (a)} Blending scalar color shader, solid color.
{\bf (b)} Mapping scalar color shader, solid color.
{\bf (c)} Mapping scalar color shader, a custom noise texture (Gaussian, grain size 1), and contrast enhancement.
{\bf (d)} Blending scalar color shader, color by scalars.
{\bf (e)} Mapping scalar color shader, color by scalars.
{\bf (f)} Mapping scalar color shader, color by scalars, a custom noise texture (Perlin, grain size 4), and contrast enhancement.}
\label{fig:noise}


\end{figure}

Important factors in determining the characteristcs of streaking patterns in the LIC and in overall contrast and dynamic range of the final image are the characteristics of the noise to be convolved with the vector field. By varying the properties of the noise input into the image LIC algorithm one gains much control over the output. Using the same noise texture produces markedly different results on different datasets and even on the same dataset at different screen resolutions. A single noise texture will not work well in all cases. Thus a run time tunable noise texture generator is a key component in our algoirthm.

Our noise texture generator defines the following 9 run time modifiable degrees of freedom, we have found that these 9 paramters are highly effective for controlling streaking pattern, dynamic range, and contrast, in in the resulting LIC. \begin{description} \item[Noise type] This parameter controls the underlying statistical distribution of values in the generated noise texture, or type of noise generated. The user may choose from Gaussian noise, uniformly distributed noise, or Perlin noise. By default Gaussian noise is used. \item[Texture size] This paramter controls the size of the square noise texture in each direction. Support for non-power of 2 textures is assumed. However in the case of Perlin noise the texture size is adjusted to the nearest power of 2. \item[Grain size] Select the number of pixels in each direction that each generated noise element fills in the resulting texture. For Perlin noise this sets the size of the largest scale, and must be a power of 2. \item[Min value] This parameter sets the lowest gray scale value realizable in the generated noise texture. This paramter can range between 0 and 1 and the default value is 0. \item[Max value] This parameter sets the highest gray scale value realizable in the generated noise texture. This paramter can range between 0 and 1 and the default value is 0.8. \item[Number of levels] Set the number of realizable gray scale values. This parameter can range from 2 to 1024 and the default value is 1024. \item[Impulse probability] This parameter controls how likely a given element is to be assigned a value. When set to 1 all elements are filled. When set to some number lower than one a fraction of the texture's elements are filled with generated noise. Elements that are not filled take on a background color value. The default impulse probablity is 1. \item[Background color] The gary scale value to use for un-touched pixels when the impulse probability paramter is set less than 1. The default background value is 0. \item[RNG seed] Modify the seed value of the random number generators. \end{description} Changing the noise texture gives one greater control over the look of the final image. For example, with impulse noise by varying the impulse probability along with the choice of background intensity control over number of light or dark pixels in the LIC image is attained. This greatly helps the transfer of patterns in scalar coloring and lighting to the final image. Varying the noise grain size along with impulse probabilty and impulse background back color give greater control over the width of the streaks generated by the image LIC stage. With the ability to selecting a noise distribution, its minimum, maximum, and number of noise levels, control over the contrast is gained. \FloatBarrier

\subsection{Integrator Normalization} \begin{figure}[htpb]

\centering
\includegraphics[width=\textwidth]{./images-data/vector-norm/vec-norm.png}
% vec-norm.png: 2100x1028 pixel, 72dpi, 74.07x36.26 cm, bb=0 0 2100 1028
\caption{vector normal}
\label{fig:vec-norm}


\end{figure}

Normalizing vectors during integration is a trick that can be used to simplify integrator configuration and give the resulting LIC a uniformly smooth look. By using normalized vector field values the convolution occurs over the same integrated arclength for all pixels in the image. This gives the result a smooth and uniform look and makes it possible to provide reasonable default values for step size and number of steps to the integrator independent of the input vector field. The resulting visualization acurately shows the tangent field however variations in the relative strength of the vector field is lost and can alter the relationships of flow features making weak features prominent and strong features less so. In the context of developing a general puprose tool for interactive data exploration it's important to provide both options and let the user select the option that best fits her needs. For example figure \ref{fig:vec-norm} shows the result of the algorithm applied to a simulation of magnetic reconnection in a hot plasma with and without integrator normalization. In this case normalization can give a false sense of the importance of a number of flow features.

% \subsection{Fragment Masking} % \label{sec:mask} % Currently the criteria for masking fragments is that all vector components must be identically zero. This doesn't work for many numerical simulations where stagnant flow is not identically zero due to numerical rounding. For example, stagnate flow might be where $|V|<1e^{-6}$. Also, the lack of control over the color characteristics of the masked fragments result in masked fragments looking drastically different than the LIC'ed fragments. This is illustrated in figure \ref{fig:motivation}.d where all fragments on the outer walls have been masked , the masked fragments are much brighter than the the LIC'ed fragments. % % New fragment masking implementation makes use of a user specified threshold value below which fragments are masked. The masking test may be applied either to the original vectors or the surface projected vectors. By applying the test to the original vectors the masked fragments will match scalar colors when coloring by $|V|$. % % Fragments where $|V|<t$ are masked. The fragment masking implementation provides control over the intensity and color of the masked fragments via the following equation: % $$% c_{ij} = M * I + S_{ij} * ( 1 - I ) % \label{eqn:color-blend} %$$ % where the indices $i,j$ identify a specific fragment, $c$ is final RGB color, $M$ is the RGB mask color, $S$ is the scalar RGB color, and $I$ is the mask color intensity. This allows one control over the masking process so that masked fragments may be: highlighted (by setting a unique mask color and mask intensity > 0), made invisible with and without passing the un-convolved noise texture (by setting mask intensity 0), or made to show the scalar color at a similar level of intensity as the LIC (mask intensity > 0).

% \subsection{Optimizations for interactivity}

\section{Parallelization}

# PV DOC

${\displaystyle \operatorname {erfc} (x)={\frac {2}{\sqrt {\pi }}}\int _{x}^{\infty }e^{-t^{2}}\,dt={\frac {e^{-x^{2}}}{x{\sqrt {\pi }}}}\sum _{n=0}^{\infty }(-1)^{n}{\frac {(2n)!}{n!(2x)^{2n}}}}$


${\displaystyle c_{ij}=L_{ij}*I+S_{ij}*(1-I)}$

# Surface LIC Plugin Parameters

## Integrator

Select Input Vectors
This is used to select the vector field.
Number Of Steps
Number of integration steps.
Step Size
Step size given in the original vector field's units.
Normalize Vectors
When on the vector field will be normalized during integration. Normalization doesn't change the tangent field and makes it so the same step size and number of steps parameters work well on a variety of data. Disabling normalization can help reveal relationships between individual flow features.
Enhanced LIC
Enabling enhanced LIC activates an algorithm sub-pipeline that applies some image processing techniques to improve the visibility of streaking patterns in the result. The enhanced LIC algorithm requires two LIC passes. In the first pass a traditional LIC is computed, in the second pass image processing is applied to the output of pass 1 which is then used in place of noise in the second LIC pass over 1/2 of the number of integration steps.

Color Mode
This selects the shader that is used to combine mapped scalar colors with the gray scale LIC.
Blend
LIC and mapped scalar colors are combined in inverse proportion to produce the final color, $$c_{ij} = L_{ij} * I + S_{ij} * (1 - I)$$
Multiply
LIC and mapped scalar colors are multiplied together to produce the final color, $$c_{ij} = ( L_{ij} + f ) * S_{ij}$$
LIC Intensity
This sets the intensity for LIC pattern when using Blend shader.
Map Mode Bias
An additive term that could be used to brighten or darken the final colors when using the Multiply shader.

### Contrast Enhancement

The contrast enhancement(CE) feature applies an image processing technique called histogram stretching at various points in the algorithm to increase contrast and dynamic range in the LIC streaks and final colored image. We've used CE in three places internally. A gray scale implementation occurs after each LIC pass and a color based implementation after scalar colors and LIC have been combined. The latter is referred to as color contrast enhancement(CCE) and is applied in HSL space on the L channel but in other respects is similar to the gray scale implementation.

We've implemented this as follows: $$c_{ij} = \frac{c_{ij} - m}{M - m}$$ where, the indices $i,j$ identify a specific fragment, $c$ is the fragment's gray scale intensity (or HSL lightness for CCE), $m$ is the gray scale color value to map to 0, $M$ is the gray scale color value to map to 1. When the contrast enhancement stage is applied on the input of the high-pass filter stage, $m$ and $M$ are always set to the minimum and maximum gray scale color of all fragments. In the final contrast enhancement stage $m$ and $M$ take on minimum and maximum gray scale colors by default but may be individually adjusted by the following set of equations: $$m = min(C) + F_{m} * ( max(C) - min(C) )$$ $$M = max(C) - F_{M} * ( max(C) - min(C) )$$ where, $C = \{c_{00},c_{01},...,c_{nm}\}$, are all of the gray scale(L channel) fragments in the image and $F_m$ and $F_M$ are adjustment factors that take on values between 0 and 1. When $F_m$ and $F_M$ are 0 minimum and maximum are input values are used. This is the default. Adjusting $F_m$ and $F_M$ above zero controls the saturation of normalization. This is useful, for example, if the brightness of pixels near the border dominate because these are convolved less because we can't integrate outside of the dataset.

Enhance Contrast
Low LIC Contrast Enhancement Factor
This can be used to adjust the minimum value in the output of the contrast enhancement. This applies only to the final CE stage in the LIC.
High LIC Contrast Enhancement Factor
This can be used to adjust the minimum value in the output of the contrast enhancement. This applies only to the final CE stage in the LIC.
Low Color Contrast Enhancement Factor
High Color Contrast Enhancement Factor

AntiAlias

## Noise texture generator

Generate Noise Texture
Noise Type
Noise Texture Size
Noise Grain Size
Min Noise Value
Max Noise Value
Number Of Noise Levels
Impulse Noise Probability
Impulse Noise Background Value
Noise Generator Seed

## Parallelization

Composite Strategy

Use LIC For LOD