Expresii's novel Vexel Rendering makes pixels zoom like vector! (」゜ロ゜)」
Left: Naïve blending of raw image pixels with paper texture. Right: Expresii's vexel rendering at 100x zoom.
Imported watercolor image displayed at 20x zoom with Expresii's vexel rendering
Introduction - Digital Illustrator's Dilemma: Raster or Vector?
If you as a digital painter ever needed to print your artwork large, you know how important it is for your artwork to be in high resolution. For over 40 years, digital paint programs have mostly been raster-based and they treat pixels as squares - if you zoom into your artwork, you see big, fat pixels. If you don't want to see fat pixels or interpolation blur, the current solution is to use vector-based programs instead.
 Most paint apps render artwork as square pixels when zoomed in |  A few paint apps use bilinear interpolation |
The following is a real-world example of having insufficient resolution. It is obvious that the artist is trying to mimic watercolor in this painting, which was printed as a mural. It looks fine from a far, but when you walk close, you will notice the blurry interpolation.
Trying to get the best of both worlds?
Over the years, there have been several attempts to solve the dilemma of raster vs vector. One of them is Creature House Expression (first released in 1996; discontinued 2003), in which they allow mapping bitmaps onto vector strokes. Because those bitmaps are resolution-limited, such strokes would be blurry when rendered large. When largely deformed, stretched bitmaps also give unnatural look.
Mischief (released 2013 ; discontinued 2019) claimed to provide you “the richness of pixel-based brushes AND the scalability of vectors”. Mischief is a vector program, but the vector strokes cannot be edited, at least in all the released incarnations. Like other pure-vector programs, it's not easy to get painterly without having too many vector strokes, which may bog down the system, and the raster look was achieved mainly using their airbrush-like brush that gives a dithered falloff effect or even just stacking semi-transparent strokes.
Concepts (released 2012 ) is another vector drawing app, which also supports mapping pixel-stamps to vector paths, in the same vein as Expression's skeletal strokes. Strokes made in Concepts are editable (in the paid version). With their basic tools like the airbrush (similar to that of Mischief), it's possible to achieve what Mischief can do, namely smooth shades that give a traditional raster painting look. Further with pixel-stamp-mapped strokes, raster richness can also be achieved. However, such pixel stamp strokes still give interpolation blur when zoomed in.
Mischief (released 2013 ; discontinued 2019) claimed to provide you “the richness of pixel-based brushes AND the scalability of vectors”. Mischief is a vector program, but the vector strokes cannot be edited, at least in all the released incarnations. Like other pure-vector programs, it's not easy to get painterly without having too many vector strokes, which may bog down the system, and the raster look was achieved mainly using their airbrush-like brush that gives a dithered falloff effect or even just stacking semi-transparent strokes.
Concepts (released 2012 ) is another vector drawing app, which also supports mapping pixel-stamps to vector paths, in the same vein as Expression's skeletal strokes. Strokes made in Concepts are editable (in the paid version). With their basic tools like the airbrush (similar to that of Mischief), it's possible to achieve what Mischief can do, namely smooth shades that give a traditional raster painting look. Further with pixel-stamp-mapped strokes, raster richness can also be achieved. However, such pixel stamp strokes still give interpolation blur when zoomed in.
Pixel-stamped 'Pencil', generated 'Airbrush' and pure-vector 'Pen' strokes in app Concepts at 16x zoom
Mingling vector & raster strokes in Adobe Fresco Adobe's latest paint program Fresco (released 2019) allows both pixels and vectors in the same artwork (code-named 'Gemini' for 'the combination of pixels and vectors in a single app'). However, putting the two types of strokes together would make the whole artwork look incohesive when you zoom in on the image, with vector strokes being very sharp and raster strokes pixelated. The effective resolution is thus limited by that of the raster strokes. Their previous app Adobe Eazel (released 2011; discontinued 2015?) generated thousands of semi-transparent polygons to mimic raster richness but the result doesn't look nice due to the smooth, clean curves being far from organic.
Affinity Designer (released 2014) also allows both pixel and vectors in the same artwork. They provide a better linkage between the vector and the raster personas by allowing clipping of raster strokes onto crisp vector shapes for texture or grain. They also allow pixel-mapped vector strokes. Whenever bitmap is involved, strokes would still pixelate or blur when zoomed in.
HeavyPaint (released 2019) saves your artwork as stroke information and can re-generate the painting at a higher resolution, typically at 2x or 4x. Stroke paths being vector, this can be considered a hybrid raster-vector approach. In fact, we can regard all the vector-based apps to be a raster app that redraws the visible strokes when the user edits them or changes the view. Such apps need to be highly optimized to allow a large number of strokes and stay responsive.
To the best of our knowledge, none of the apps out there can really give raster richness and ultra-high-res output at the same time.
Affinity Designer (released 2014) also allows both pixel and vectors in the same artwork. They provide a better linkage between the vector and the raster personas by allowing clipping of raster strokes onto crisp vector shapes for texture or grain. They also allow pixel-mapped vector strokes. Whenever bitmap is involved, strokes would still pixelate or blur when zoomed in.
HeavyPaint (released 2019) saves your artwork as stroke information and can re-generate the painting at a higher resolution, typically at 2x or 4x. Stroke paths being vector, this can be considered a hybrid raster-vector approach. In fact, we can regard all the vector-based apps to be a raster app that redraws the visible strokes when the user edits them or changes the view. Such apps need to be highly optimized to allow a large number of strokes and stay responsive.
To the best of our knowledge, none of the apps out there can really give raster richness and ultra-high-res output at the same time.
Previous Academic Work
Trying to improve the rendering quality of magnified bitmaps is surely not new. Here, let's compare our novel vexel rendering technique with other related methods that involve raster-to-vector conversion.
Pixel-Art-Specific
These stem from the need to render old video games from the 80's and early 90's on modern hardware. One notable work on this is the SIGGRAPH 2011 paper by Kopf and Lischinski. Very nice results are obtained, but the global nature of the algorithm makes it hard to be implemented on the GPU. Two years later Silva et al. published an algorithm on GPU to give real-time performance:
These stem from the need to render old video games from the 80's and early 90's on modern hardware. One notable work on this is the SIGGRAPH 2011 paper by Kopf and Lischinski. Very nice results are obtained, but the global nature of the algorithm makes it hard to be implemented on the GPU. Two years later Silva et al. published an algorithm on GPU to give real-time performance:
These methods are specifically designed for very low-res images using heuristics on connected pixels and thus not suitable for digital painting at common canvas sizes.
Embedding Explicit Edges
Around 2004-2005, we saw several papers dealing with rendering discontinuity in sampled images. They include the Bixel (2004), the Silhouette Maps (2004), the Feature-based Textures (2004) and the Pinchmaps (2005).
Around 2004-2005, we saw several papers dealing with rendering discontinuity in sampled images. They include the Bixel (2004), the Silhouette Maps (2004), the Feature-based Textures (2004) and the Pinchmaps (2005).
 The Silhouette Map [Sen 2004] encodes edge information into image data |  The Pinchmap [Tarini and Cignoni 2005] shifts texture coordinates so that the interpolation across edges are 'pinched' to give sharp edges |
The main insight here is that the usual image interpolations (bilinear or bicubic) are bad for discontinuity. All these methods attempt to preserve sharp edges by encoding boundary information in the images. Earlier methods are not very fast because edge info derivation and utilization are complex. The later Pinchmap improves the run-time performance by removing the need to divide the situation in cases and thus can be rendered on the GPU very efficiently. However, time-consuming pre-processing is still required to derive the 'pinching' configuration and thus is still not suitable for real-time painting applications.
Implicit-Function-Based
Ray et al. 2005's Vector Texture Map was one early use of implicit function values to define discontinuity in glyphs to avoid evaluation of explicit curves on the GPU. Qin et al. 2006 later used Signed Distance Function (SDF) as the implicit function. These functions are evaluated on the fly and in a hierarchical manner for robustness. In 2007, Chris Green of VALVE showed us how they used SDF for rendering glyphs in a game environment. In such an application, they did not strive to render the glyphs exactly (like having sharp corners) so they can just use rasterized SDF values in a much simplified way, which runs well even on low-end graphics hardware. Note that all these work well only if you are rendering binary-masked glyphs.
Ray et al. 2005's Vector Texture Map was one early use of implicit function values to define discontinuity in glyphs to avoid evaluation of explicit curves on the GPU. Qin et al. 2006 later used Signed Distance Function (SDF) as the implicit function. These functions are evaluated on the fly and in a hierarchical manner for robustness. In 2007, Chris Green of VALVE showed us how they used SDF for rendering glyphs in a game environment. In such an application, they did not strive to render the glyphs exactly (like having sharp corners) so they can just use rasterized SDF values in a much simplified way, which runs well even on low-end graphics hardware. Note that all these work well only if you are rendering binary-masked glyphs.
[Green 2007] uses SDF values stored as texture to rendering vector 'line art' images.
SIGGRAPH 2010 paper Vector Solid Textures uses SDF to generate solid texture, but the encoding requires complex pre-processing and is still limited by the non-overlapping-region requirement. To allow overlapping regions, more SDF are needed thus more storage and processing. To our knowledge, no one has found a way to use implicit function to represent full-color overlapping regions efficiently yet.
Our solution: Vexel Rendering
Is it actually possible to have raster richness and be able to scale it much larger? I think we found a solution, at least for the case of organic, natural-media digital painting. Specifically, our goal is to render our artwork as if done on real paper*.
We perform such magic using a shader program. A shader is a program that runs on the GPU and calculates the final image output. Your image data may be stored in an array, but not in the traditional 'pixel' sense. Each slot stores color or other attributes and it's then up to the shader on how to render the final image from such information. In a way, you can consider the traditional pixels as your final image discretized and most other paint programs display them as colored squares tiled to give the final image. Our shader, on the other hand, takes the stored information and generates the final image. Our vexel rendering works well for our paint simulation output, ordinary raster illustrations, and photos of watercolor artwork. For examples, the following video taking a sample illustration from irasutoya as input and we show that you can still add our simulated strokes to it. The first demo image in this article is a photo of real watercolor marks as input.
* We will try to extend our program to render other paint media like oil paint later.
We perform such magic using a shader program. A shader is a program that runs on the GPU and calculates the final image output. Your image data may be stored in an array, but not in the traditional 'pixel' sense. Each slot stores color or other attributes and it's then up to the shader on how to render the final image from such information. In a way, you can consider the traditional pixels as your final image discretized and most other paint programs display them as colored squares tiled to give the final image. Our shader, on the other hand, takes the stored information and generates the final image. Our vexel rendering works well for our paint simulation output, ordinary raster illustrations, and photos of watercolor artwork. For examples, the following video taking a sample illustration from irasutoya as input and we show that you can still add our simulated strokes to it. The first demo image in this article is a photo of real watercolor marks as input.
* We will try to extend our program to render other paint media like oil paint later.
Raster Richness
Our paint information is stored in an array allowing raster richness. In fact, our Expresii simulates watercolor/ink flow giving rich, organic outcome unmatched by any other app. On top of that, we render the artwork as if it was done on a real piece of paper, showing details up to paper-fiber level. Due to its raster nature, you can add infinite number of strokes to the page without bogging down the system like in conventional vector programs. Raster operations like blurring or smudging are possible. Such operations would be difficult in a pure-vector program.
Vector Scalability
Because the outcome is generated, our final output is flexible on its resolution. Our current maximum output resolution is limited by our raster paper texture, which becomes blurry if we go beyond 30x zoom. Given new tools like Material Maker, we can design substrate textures using shaders so it's possible that in future version of Expresii, we can replace the paper texture with one that is resolution independent.
Comparison between raw pixels and vexel rendering at 50x zoom
At moderate zoom like 20x, our rendered image looks like real non-volumetric paint marks on paper. By varying the shading from paper texture, we can emulate watercolor/ink soaked into the paper fibers, or crayon/pencil marks laid on the paper surface. At very high zoom like 100x, our rendering reveals curved shapes (see the first image of this article), which look like paper cutouts. Our vexel rendering can smoothly transit from the look of ordinary raster image to shaded vector shapes that integrate with the paper substrate very nicely.
Tailor-made for our need
Comparison of a raster illustration (Center), its hand-traced vector (Left) and Expresii rendering (Right) at 40x zoom
As shown in the above image, traditionally vectorized images are too clean, too sterile. Current auto-trace results tend to give you simple geometric shapes (like the left part of the above figure), which are not a good representation of textural details like those from watercolor marks. Traditional raster digital painting are good at capturing details but they usually cannot be as large as 16k x 16k pixels, unless your system is beefy enough and that your app does support such a large canvas (popular iPad app Procreate's max size is 8k x 8k or equivalent, as of Oct 2021). In comparison, our current Youji 2.0 rendering engine can output very nice textural details up to around 40x magnification, and shaded vector shapes at 500x. We also don't need any model training like in those AI-based approaches that give you a 2x to 8x magnification at non-interactive rate. Everything is local and instant.
For organic digital painting, we actually prefer our vexel rendering over the simple curves resultant from image tracers of existing software tools, which give a flat, planar look. You can tune your image tracer to output more polygons like in the above figure but it is still not easy to get as detailed as our vexel rendering.
A Paradigm shift
Existing vector programs are great for creating graphics comprised of clean lines or shapes, and if you are after such graphics, by all means you should use those tools. On the other hand for digital painting, one major goal among many paint programs is to give a natural-media look. It seems we are stuck with the thought that vector primitive should be simple, clean curves and thus results from current raster-to-vector conversions still look bland. Here, we render the raster data as 'vexels' that allow huge magnification while keeping all textural details.
Existing vector programs are great for creating graphics comprised of clean lines or shapes, and if you are after such graphics, by all means you should use those tools. On the other hand for digital painting, one major goal among many paint programs is to give a natural-media look. It seems we are stuck with the thought that vector primitive should be simple, clean curves and thus results from current raster-to-vector conversions still look bland. Here, we render the raster data as 'vexels' that allow huge magnification while keeping all textural details.
Conclusion
Expresii is the first digital painting app that can really give both raster richness and ultra-high-res output scaling, thanks to its novel GPU-based simulation and rendering algorithms. Many artists used to traditional media do not like digital counterparts because digital paintings get pixelated when zoomed in on. We believe Expresii has largely fixed this issue and hope more artists are willing to go digital. Our vexel rendering is not a general solution to the problem of image interpolation, since our method rely on the fact that for many natural media there is a substrate to give grain texture.
Currently Expresii only simulates water-based art media. We plan to add other media like pastel and oil in the near future. Stay tuned.
Currently Expresii only simulates water-based art media. We plan to add other media like pastel and oil in the near future. Stay tuned.
Update: The following video shows the use of 'vexel rendering' in actual painting and that we are able to export 32k images thanks to our ultra-zooming capability:
Why the term 'vexel'
When developing new tech to avoid seeing fat pixels in digital painting, I wanted to give it a name that suggests it's a combination of vector and pixel, and I came up with 'vexel'. Later I found out 'vexel' is already coined by Seth Woolley since at least 2006 to refer to raster images that look like vector graphics. I tried to find another name, but couldn't find one giving the same level of meaningfulness. So, I decided to stick with 'vexel'. After all, our rendering does look like it's vexel art in Seth's definition.
When developing new tech to avoid seeing fat pixels in digital painting, I wanted to give it a name that suggests it's a combination of vector and pixel, and I came up with 'vexel'. Later I found out 'vexel' is already coined by Seth Woolley since at least 2006 to refer to raster images that look like vector graphics. I tried to find another name, but couldn't find one giving the same level of meaningfulness. So, I decided to stick with 'vexel'. After all, our rendering does look like it's vexel art in Seth's definition.
RSS Feed
