Expresii's novel Vexel Rendering makes pixels zoom like vector! (」゜ロ゜)」
Introduction - Digital Illustrator's Dilemma: Raster or Vector?
Trying to get the best of both worlds?
Mischief (released 2013 ; discontinued 2019) claimed to provide you “the richness of pixel-based brushes AND the scalability of vectors”. Mischief is a vector program, but the vector strokes cannot be edited, at least in all the released incarnations. Like other pure-vector programs, it's not easy to get painterly without having too many vector strokes, which may bog down the system, and the raster look was achieved mainly using their airbrush-like brush that gives a dithered falloff effect or even just stacking semi-transparent strokes.
Concepts (released 2012 ) is another vector drawing app, which also supports mapping pixel-stamps to vector paths, in the same vein as Expression's skeletal strokes. Strokes made in Concepts are editable (in the paid version). With their basic tools like the airbrush (similar to that of Mischief), it's possible to achieve what Mischief can do, namely smooth shades that give a traditional raster painting look. Further with pixel-stamp-mapped strokes, raster richness can also be achieved. However, such pixel stamp strokes still give interpolation blur when zoomed in.
Affinity Designer (released 2014) also allows both pixel and vectors in the same artwork. They provide a better linkage between the vector and the raster personas by allowing clipping of raster strokes onto crisp vector shapes for texture or grain. They also allow pixel-mapped vector strokes. Whenever bitmap is involved, strokes would still pixelate or blur when zoomed in.
HeavyPaint (released 2019) saves your artwork as stroke information and can re-generate the painting at a higher resolution, typically at 2x or 4x. Stroke paths being vector, this can be considered a hybrid raster-vector approach. In fact, we can regard all the vector-based apps to be a raster app that redraws the visible strokes when the user edits them or changes the view. Such apps need to be highly optimized to allow a large number of strokes and stay responsive.
To the best of our knowledge, none of the apps out there can really give raster richness and ultra-high-res output at the same time.
Previous Academic Work
These stem from the need to render old video games from the 80's and early 90's on modern hardware. One notable work on this is the SIGGRAPH 2011 paper by Kopf and Lischinski. Very nice results are obtained, but the global nature of the algorithm makes it hard to be implemented on the GPU. Two years later Silva et al. published an algorithm on GPU to give real-time performance:
Around 2004-2005, we saw several papers dealing with rendering discontinuity in sampled images. They include the Bixel (2004), the Silhouette Maps (2004), the Feature-based Textures (2004) and the Pinchmaps (2005).
Ray et al. 2005's Vector Texture Map was one early use of implicit function values to define discontinuity in glyphs to avoid evaluation of explicit curves on the GPU. Qin et al. 2006 later used Signed Distance Function (SDF) as the implicit function. These functions are evaluated on the fly and in a hierarchical manner for robustness. In 2007, Chris Green of VALVE showed us how they used SDF for rendering glyphs in a game environment. In such an application, they did not strive to render the glyphs exactly (like having sharp corners) so they can just use rasterized SDF values in a much simplified way, which runs well even on low-end graphics hardware. Note that all these work well only if you are rendering binary-masked glyphs.
Our solution: Vexel Rendering
We perform such magic using a shader program. A shader is a program that runs on the GPU and calculates the final image output. Your image data may be stored in an array, but not in the traditional 'pixel' sense. Each slot stores color or other attributes and it's then up to the shader on how to render the final image from such information. In a way, you can consider the traditional pixels as your final image discretized and most other paint programs display them as colored squares tiled to give the final image. Our shader, on the other hand, takes the stored information and generates the final image. Our vexel rendering works well for our paint simulation output, ordinary raster illustrations, and photos of watercolor artwork. For examples, the following video taking a sample illustration from irasutoya as input and we show that you can still add our simulated strokes to it. The first demo image in this article is a photo of real watercolor marks as input.
* We will try to extend our program to render other paint media like oil paint later.
Tailor-made for our need
Existing vector programs are great for creating graphics comprised of clean lines or shapes, and if you are after such graphics, by all means you should use those tools. On the other hand for digital painting, one major goal among many paint programs is to give a natural-media look. It seems we are stuck with the thought that vector primitive should be simple, clean curves and thus results from current raster-to-vector conversions still look bland. Here, we render the raster data as 'vexels' that allow huge magnification while keeping all textural details.
Currently Expresii only simulates water-based art media. We plan to add other media like pastel and oil in the near future. Stay tuned.
When developing new tech to avoid seeing fat pixels in digital painting, I wanted to give it a name that suggests it's a combination of vector and pixel, and I came up with 'vexel'. Later I found out 'vexel' is already coined by Seth Woolley since at least 2006 to refer to raster images that look like vector graphics. I tried to find another name, but couldn't find one giving the same level of meaningfulness. So, I decided to stick with 'vexel'. After all, our rendering does look like it's vexel art in Seth's definition.