Tessellation is the Vertex Processing stage in the OpenGL rendering pipeline where patches of vertex data are subdivided into smaller Primitives.
This process is governed by two shader stages and a fixed-function stage.
Two of the stages are programmable; between them is a fixed function stage.
Generally, the process of tessellation involves subdividing a patch of some type, then computing new vertex values (position, color, texture coordinates, etc.) for each of the vertices generated by this process. Each stage of the tessellation pipeline performs part of this process.