Expresii 寫意
  • Home
  • Showcase
    • Moxi Paint Engine
    • Yibi Brush Engine
    • A New Way to Paint
    • youji Rendering Engine
  • Download
    • Get / Buy Expresii >
      • Update Log
    • Sample Artwork Files
    • Third-Party
  • Support
    • Documentation >
      • End User License Agreement
      • Privacy Policy
    • Video Tutorials
    • User Forum
    • FAQ
    • Feedback
    • Help Translate
  • About Us
    • Contact Us
    • Media Resources
    • Privacy
  • Blog
  • 主頁
  • 示範
  • 網誌
  • 下載
  • 關於我們
    • 聯絡我們

Vexel: Marrying Pixels with Vector for organic Digital Painting

30/10/2021

Comments

 
Expresii's novel Vexel Rendering makes pixels zoom like vector!    (」゜ロ゜)」
Picture
Left: Naïve blending of raw image pixels with paper texture. Right: Expresii's vexel rendering at 100x zoom.
Imported watercolor image displayed at 20x zoom with Expresii's vexel rendering

Introduction - Digital Illustrator's Dilemma: ​Raster or Vector?

If you as a digital painter ever needed to print your artwork large, you know how important it is for your artwork to be in high resolution.  For over 40 years, digital paint programs have mostly  been raster-based and they treat pixels as squares - if you zoom into your artwork, you see big, fat pixels.  If you don't want to see fat pixels or interpolation blur, the current solution is to use vector-based programs instead.
Picture
Most paint apps render artwork as square pixels when zoomed in
Picture
A few paint apps use bilinear interpolation
The following is a real-world example of having insufficient resolution. It is obvious that the artist is trying to mimic watercolor in this painting, which was printed as a mural.  It looks fine from a far, but when you walk close, you will notice the blurry interpolation.

Trying to get the best of both worlds?

 Over the years, there have been several attempts to solve the dilemma of raster vs vector.​ One of them is Creature House Expression (first released in 1996; discontinued 2003), in which they allow mapping bitmaps onto vector strokes. Because those bitmaps are resolution-limited, such strokes would be blurry when rendered large. When largely deformed, stretched bitmaps also give unnatural look. 

Mischief (released 2013 ; discontinued 2019) claimed to provide you “the richness of pixel-based brushes AND the scalability of vectors”. Mischief is a vector program, but the vector strokes cannot be edited, at least in all the released incarnations.  Like other pure-vector programs, it's not easy to get painterly without having too many vector strokes, which may bog down the system, and the raster look was achieved mainly using their airbrush-like brush that gives  a dithered falloff effect or even just stacking semi-transparent strokes.   

Concepts (released 2012 ) is another vector drawing app, which also supports mapping   pixel-stamps to vector paths, in the same vein as Expression's skeletal strokes.  Strokes made in Concepts are editable (in the paid version). With their  basic tools like the airbrush (similar to that of Mischief),  it's possible to achieve what Mischief can do, namely smooth shades that give a traditional raster painting look.   Further with   pixel-stamp-mapped strokes,    raster richness can also be achieved. However, such pixel stamp strokes still give interpolation blur  when zoomed in.
Picture
Pixel-stamped 'Pencil', generated 'Airbrush' and pure-vector 'Pen' strokes in app Concepts at 16x zoom
PictureMingling vector & raster strokes in Adobe Fresco
Adobe's latest paint program Fresco (released 2019) allows both pixels and vectors in the same artwork (code-named 'Gemini' for 'the combination of pixels and vectors in a single app'). However, putting the two types of strokes together would make the whole artwork look  incohesive when you zoom in on the image, with vector strokes being very sharp and raster strokes pixelated. The effective resolution is thus limited by that of the raster strokes. Their previous app Adobe Eazel (released 2011; discontinued 2015?) generated thousands of semi-transparent polygons  to mimic raster richness but the result doesn't look nice due to the smooth, clean curves being far from organic. 

Affinity Designer  (released 2014) also allows both pixel and vectors in the same artwork.   They provide a better linkage between  the  vector and the raster personas by allowing clipping of raster strokes  onto crisp vector shapes for texture  or grain. They also allow  pixel-mapped vector strokes. Whenever bitmap is involved, strokes would still pixelate or blur when zoomed in.

HeavyPaint  (released 2019) saves your artwork as stroke information and can re-generate the painting at  a higher resolution, typically at 2x or 4x.  Stroke paths being vector, this can be considered a hybrid raster-vector approach.   In fact, we can regard all the vector-based apps to be a raster app that redraws the visible strokes when the user edits them  or changes the view. Such apps need to be highly optimized  to allow a large number of strokes and stay responsive.

To the best of our knowledge,  none of the apps out there can really give raster richness   and   ultra-high-res output at the same time.

Previous Academic Work

Trying to improve the  rendering quality of magnified  bitmaps is surely not new. Here, let's compare our novel vexel rendering technique with other related methods that involve raster-to-vector conversion. 
Pixel-Art-Specific
These stem from the need to render old video games from the 80's and early 90's on modern hardware.  One notable work on this is the SIGGRAPH 2011 paper  by   Kopf and Lischinski. Very nice results are obtained, but the global nature of the algorithm makes it hard to be implemented on the GPU.  Two years later  Silva et al.  published an algorithm on GPU to give real-time performance:
​These methods are specifically designed for very low-res images using heuristics on connected pixels and thus not suitable for digital painting at common canvas sizes.
Embedding  Explicit Edges
Around 2004-2005, we saw several papers dealing with rendering discontinuity in sampled images. They include the Bixel    (2004),  the Silhouette Maps (2004), the Feature-based Textures  (2004) and the Pinchmaps  (2005).
Picture
The Silhouette Map [Sen 2004] encodes edge information into image data
Picture
The Pinchmap [Tarini and Cignoni 2005] shifts texture coordinates so that the interpolation across edges are 'pinched' to give sharp edges
​The main insight here is that the usual image interpolations (bilinear or bicubic) are bad for discontinuity. All these methods attempt to preserve sharp edges by encoding  boundary information in the images.  Earlier methods are not very fast because edge info derivation and utilization are complex. The later Pinchmap improves the run-time performance by removing the need to divide the situation in cases and thus can be rendered on the GPU very efficiently.  However, time-consuming pre-processing is still required to derive the 'pinching' configuration and thus is still not suitable for real-time painting applications.    
​Implicit-Function-Based
Ray et al. 2005's Vector Texture Map was one early use of implicit function values to define discontinuity in glyphs to avoid evaluation of explicit   curves on the GPU. ​Qin et al. 2006 later used Signed Distance Function (SDF) as the implicit function. These functions are evaluated on the fly and in a hierarchical manner for robustness. In 2007, Chris Green of VALVE showed us how they used SDF for rendering glyphs in a game environment. In such an application, they did not strive to render the glyphs exactly (like having sharp corners) so they can just use rasterized SDF values in a much simplified way, which runs well even on low-end graphics hardware.   Note that all these work well only if you are rendering binary-masked glyphs.
Picture
[Green 2007] uses SDF values stored as texture to rendering vector 'line art' images.
​​SIGGRAPH 2010 paper Vector Solid Textures   uses SDF to generate solid texture, but the encoding requires complex pre-processing and is still limited by the non-overlapping-region requirement.  To   allow   overlapping regions, more SDF are needed thus more storage and processing. To our knowledge, no one has found a way to use implicit function to represent full-color overlapping regions efficiently yet.

Our solution: Vexel Rendering

Is it actually possible to have raster richness and be able to scale it much larger?  I think we found a solution, at least for the case of organic, natural-media digital painting.  Specifically, our goal is to render our artwork as if done on real paper*.   
​
We perform such magic using a shader program.    A shader is a  program that runs on the GPU and   calculates the final image output. Your image data may be stored in an array, but not in the traditional 'pixel' sense. Each slot stores color  or other attributes and it's then up to the shader on how to render the final image  from such information.   In a way, you  can consider the traditional pixels as your final image discretized and most other paint programs display them  as colored squares tiled to give the final image.   Our shader, on the other hand, takes the stored information and generates the final image.   Our vexel rendering works well for our paint simulation output, ordinary   raster illustrations, and photos of watercolor artwork. For examples, the following video  taking a   sample illustration  from   irasutoya  as input and we show that you can still add our simulated strokes to it. The first demo image in this article is a photo of real watercolor marks as input.

* We will try to extend our program to render other paint media like oil paint later.

Raster Richness

Our paint information is stored in an array allowing raster richness. In fact, our Expresii simulates watercolor/ink flow giving rich, organic outcome unmatched by any other app. On top of that, we render the artwork  as if it was done on a real piece of paper, showing details up to paper-fiber level. Due to its raster nature, you can add infinite number of strokes to the page without bogging down the system like in conventional vector programs.   Raster operations like blurring or smudging are possible. Such operations would  be difficult in a pure-vector program.

Vector Scalability

Because the outcome is generated, our final output is flexible on its resolution. Our  current maximum output resolution  is limited by our raster paper texture, which becomes blurry if we go beyond 30x zoom. Given new tools like Material Maker, we can design substrate textures using shaders so it's possible that in future version of Expresii, we can replace the paper texture with one that is resolution independent.
Comparison between raw pixels and  vexel rendering at 50x zoom
​At moderate zoom like 20x, our rendered image looks like real non-volumetric paint marks on paper. By varying the shading from paper texture, we can emulate watercolor/ink soaked into the paper fibers, or crayon/pencil marks laid on the paper surface. At very high  zoom like 100x, our rendering reveals curved shapes (see the first image of this article), which look like   paper cutouts.  Our vexel rendering can smoothly transit from the look of ordinary raster image to shaded vector shapes  that integrate with the paper substrate very nicely.

Tailor-made for our need

Picture
Comparison of a raster illustration (Center), its hand-traced vector (Left) and Expresii rendering (Right) at 40x zoom
As shown in the above image, traditionally vectorized images are too clean, too sterile.  Current auto-trace results tend to  give you simple geometric shapes (like the left part of the above figure), which are not a good representation of textural details like  those from watercolor marks. Traditional raster digital painting are good at capturing details but they usually cannot be as large as 16k x 16k pixels, unless your system is beefy enough and that your app does support such a large canvas (popular iPad app Procreate's max size is 8k x 8k or equivalent, as of Oct 2021). In comparison, our current Youji 2.0  rendering engine can output very nice textural details up to around 40x magnification, and  shaded vector shapes  at 500x. We also don't need any model training like in those AI-based approaches that give you a 2x to 8x magnification at non-interactive rate. Everything is local and instant.
 For organic digital painting, we actually prefer our vexel rendering over the simple curves resultant from image tracers  of existing software tools, which  give a flat, planar look.   You can tune your image tracer to output more polygons like in the above figure but it is still not easy to get as detailed as our vexel rendering. 
A Paradigm shift
Existing vector programs are great for creating graphics comprised of clean lines or shapes, and if you are after such graphics, by all means you should use those tools. On the other hand for digital painting, one major goal among many paint programs is to give a   natural-media look.  It seems we are stuck with the thought that vector  primitive should be simple, clean curves and thus results from current raster-to-vector conversions still look bland.   Here, we render the raster data as 'vexels' that allow huge magnification while keeping all textural details.

Conclusion 

Expresii is the first digital painting   app that can really give both raster richness and ultra-high-res output scaling,  thanks to its novel GPU-based simulation and rendering algorithms.   Many artists used to traditional media do not like digital counterparts because digital paintings get pixelated when zoomed in on. We believe Expresii has largely fixed this issue and hope more artists are willing to go digital.  Our vexel rendering is not a general solution to the problem of image interpolation, since our method rely on the fact that for many natural media there is a substrate to give grain texture.

Currently Expresii only simulates water-based art media.  We plan to add other media like pastel and oil in the near future. Stay tuned.
Update: The following video shows  the use of 'vexel rendering' in actual painting and that we are able to export 32k images thanks to our ultra-zooming capability:

​Why the term 'vexel'
When developing new tech to avoid seeing fat pixels in digital painting, I wanted to give it a name that suggests it's a combination of vector and pixel,   and   I came up with 'vexel'.  Later I found out 'vexel'   is already coined by Seth Woolley​  since at least  2006 to refer to raster images that look like vector graphics.   I tried to find another name, but couldn't find one giving the same level of meaningfulness.  So,  I decided to stick with 'vexel'. After all, our rendering does look like it's vexel art in Seth's definition.
Comments
comments powered by Disqus

    Expresii 寫意

    Previous Posts
    以前的文章

    November 2022
    October 2021
    July 2021
    June 2021
    May 2021
    April 2021
    March 2021
    February 2021
    December 2020
    November 2020
    October 2020
    August 2020
    June 2020
    April 2020
    January 2020
    December 2019
    November 2019
    October 2019
    September 2019
    August 2019
    July 2019
    May 2019
    April 2019
    March 2019
    February 2019
    January 2019
    December 2018
    November 2018
    September 2018
    July 2018
    June 2018
    April 2018
    March 2018
    February 2018
    January 2018
    December 2017
    October 2017
    August 2017
    July 2017
    June 2017
    May 2017
    April 2017
    February 2017
    January 2017
    December 2016
    November 2016
    October 2016
    September 2016
    July 2016
    June 2016
    May 2016
    April 2016
    March 2016
    February 2016
    January 2016
    December 2015
    October 2015
    September 2015
    August 2015
    July 2015
    June 2015
    March 2015
    February 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014
    August 2014
    July 2014
    June 2014
    April 2014
    November 2013
    October 2013
    July 2013
    February 2013
    January 2013
    December 2012

    Categories 分類

    All
    3rd Party Tool
    Animated
    Artists
    Calligraphy
    Demo
    Hardware
    Mac
    New Feature
    Publicity
    Purchase
    Surface
    Tech

    RSS Feed

Picture
© Expresii.com 2023. All Rights Reserved.