Imported watercolor image displayed at 20x zoom with Expresii's vexel rendering

If you as a digital painter ever needed to print your artwork large, you know how important it is for your artwork to be in high resolution.  For over 40 years, digital paint programs have mostly  been raster-based and they treat pixels as squares - if you zoom into your artwork, you see big, fat pixels.  If you don't want to see fat pixels or interpolation blur, the current solution is to use vector-based programs instead.

The following is a real-world example of having insufficient resolution. It is obvious that the artist is trying to mimic watercolor in this painting, which was printed as a mural.  It looks fine from a far, but when you walk close, you will notice the blurry interpolation.

 Over the years, there have been several attempts to solve the dilemma of raster vs vector.​ One of them is Creature House Expression (first released in 1996; discontinued 2003), in which they allow mapping bitmaps onto vector strokes. Because those bitmaps are resolution-limited, such strokes would be blurry when rendered large. When largely deformed, stretched bitmaps also give unnatural look. 

Mischief (released 2013 ; discontinued 2019) claimed to provide you “the richness of pixel-based brushes AND the scalability of vectors”. Mischief is a vector program, but the vector strokes cannot be edited, at least in all the released incarnations.  Like other pure-vector programs, it's not easy to get painterly without having too many vector strokes, which may bog down the system, and the raster look was achieved mainly using their airbrush-like brush that gives  a dithered falloff effect or even just stacking semi-transparent strokes.   

Concepts (released 2012 ) is another vector drawing app, which also supports mapping   pixel-stamps to vector paths, in the same vein as Expression's skeletal strokes.  Strokes made in Concepts are editable (in the paid version). With their  basic tools like the airbrush (similar to that of Mischief),  it's possible to achieve what Mischief can do, namely smooth shades that give a traditional raster painting look.   Further with   pixel-stamp-mapped strokes,    raster richness can also be achieved. However, such pixel stamp strokes still give interpolation blur  when zoomed in.

Adobe's latest paint program Fresco (released 2019) allows both pixels and vectors in the same artwork (code-named 'Gemini' for 'the combination of pixels and vectors in a single app'). However, putting the two types of strokes together would make the whole artwork look  incohesive when you zoom in on the image, with vector strokes being very sharp and raster strokes pixelated. The effective resolution is thus limited by that of the raster strokes. Their previous app Adobe Eazel (released 2011; discontinued 2015?) generated thousands of semi-transparent polygons  to mimic raster richness but the result doesn't look nice due to the smooth, clean curves being far from organic. 

Affinity Designer  (released 2014) also allows both pixel and vectors in the same artwork.   They provide a better linkage between  the  vector and the raster personas by allowing clipping of raster strokes  onto crisp vector shapes for texture  or grain. They also allow  pixel-mapped vector strokes. Whenever bitmap is involved, strokes would still pixelate or blur when zoomed in.

HeavyPaint  (released 2019) saves your artwork as stroke information and can re-generate the painting at  a higher resolution, typically at 2x or 4x.  Stroke paths being vector, this can be considered a hybrid raster-vector approach.   In fact, we can regard all the vector-based apps to be a raster app that redraws the visible strokes when the user edits them  or changes the view. Such apps need to be highly optimized  to allow a large number of strokes and stay responsive.

To the best of our knowledge,  none of the apps out there can really give raster richness   and   ultra-high-res output at the same time.

Trying to improve the  rendering quality of magnified  bitmaps is surely not new. Here, let's compare our novel vexel rendering technique with other related methods that involve raster-to-vector conversion. 

Pixel-Art-Specific
These stem from the need to render old video games from the 80's and early 90's on modern hardware.  One notable work on this is the SIGGRAPH 2011 paper  by   Kopf and Lischinski. Very nice results are obtained, but the global nature of the algorithm makes it hard to be implemented on the GPU.  Two years later  Silva et al.  published an algorithm on GPU to give real-time performance:

​These methods are specifically designed for very low-res images using heuristics on connected pixels and thus not suitable for digital painting at common canvas sizes.

Embedding  Explicit Edges
Around 2004-2005, we saw several papers dealing with rendering discontinuity in sampled images. They include the Bixel    (2004),  the Silhouette Maps (2004), the Feature-based Textures  (2004) and the Pinchmaps  (2005).

​The main insight here is that the usual image interpolations (bilinear or bicubic) are bad for discontinuity. All these methods attempt to preserve sharp edges by encoding  boundary information in the images.  Earlier methods are not very fast because edge info derivation and utilization are complex. The later Pinchmap improves the run-time performance by removing the need to divide the situation in cases and thus can be rendered on the GPU very efficiently.  However, time-consuming pre-processing is still required to derive the 'pinching' configuration and thus is still not suitable for real-time painting applications.    

​Implicit-Function-Based
Ray et al. 2005's Vector Texture Map was one early use of implicit function values to define discontinuity in glyphs to avoid evaluation of explicit   curves on the GPU. ​Qin et al. 2006 later used Signed Distance Function (SDF) as the implicit function. These functions are evaluated on the fly and in a hierarchical manner for robustness. In 2007, Chris Green of VALVE showed us how they used SDF for rendering glyphs in a game environment. In such an application, they did not strive to render the glyphs exactly (like having sharp corners) so they can just use rasterized SDF values in a much simplified way, which runs well even on low-end graphics hardware.   Note that all these work well only if you are rendering binary-masked glyphs.

​​SIGGRAPH 2010 paper Vector Solid Textures   uses SDF to generate solid texture, but the encoding requires complex pre-processing and is still limited by the non-overlapping-region requirement.  To   allow   overlapping regions, more SDF are needed thus more storage and processing. To our knowledge, no one has found a way to use implicit function to represent full-color overlapping regions efficiently yet.

Is it actually possible to have raster richness and be able to scale it much larger?  I think we found a solution, at least for the case of organic, natural-media digital painting.  Specifically, our goal is to render our artwork as if done on real paper*.   
​
We perform such magic using a shader program.    A shader is a  program that runs on the GPU and   calculates the final image output. Your image data may be stored in an array, but not in the traditional 'pixel' sense. Each slot stores color  or other attributes and it's then up to the shader on how to render the final image  from such information.   In a way, you  can consider the traditional pixels as your final image discretized and most other paint programs display them  as colored squares tiled to give the final image.   Our shader, on the other hand, takes the stored information and generates the final image.   Our vexel rendering works well for our paint simulation output, ordinary   raster illustrations, and photos of watercolor artwork. For examples, the following video  taking a   sample illustration  from   irasutoya  as input and we show that you can still add our simulated strokes to it. The first demo image in this article is a photo of real watercolor marks as input.

* We will try to extend our program to render other paint media like oil paint later.

Our paint information is stored in an array allowing raster richness. In fact, our Expresii simulates watercolor/ink flow giving rich, organic outcome unmatched by any other app. On top of that, we render the artwork  as if it was done on a real piece of paper, showing details up to paper-fiber level. Due to its raster nature, you can add infinite number of strokes to the page without bogging down the system like in conventional vector programs.   Raster operations like blurring or smudging are possible. Such operations would  be difficult in a pure-vector program.

Because the outcome is generated, our final output is flexible on its resolution. Our  current maximum output resolution  is limited by our raster paper texture, which becomes blurry if we go beyond 30x zoom. Given new tools like Material Maker, we can design substrate textures using shaders so it's possible that in future version of Expresii, we can replace the paper texture with one that is resolution independent.

Comparison between raw pixels and  vexel rendering at 50x zoom

As shown in the above image, traditionally vectorized images are too clean, too sterile.  Current auto-trace results tend to  give you simple geometric shapes (like the left part of the above figure), which are not a good representation of textural details like  those from watercolor marks. Traditional raster digital painting are good at capturing details but they usually cannot be as large as 16k x 16k pixels, unless your system is beefy enough and that your app does support such a large canvas (popular iPad app Procreate's max size is 8k x 8k or equivalent, as of Oct 2021). In comparison, our current Youji 2.0  rendering engine can output very nice textural details up to around 40x magnification, and  shaded vector shapes  at 500x. We also don't need any model training like in those AI-based approaches that give you a 2x to 8x magnification at non-interactive rate. Everything is local and instant.

 For organic digital painting, we actually prefer our vexel rendering over the simple curves resultant from image tracers  of existing software tools, which  give a flat, planar look.   You can tune your image tracer to output more polygons like in the above figure but it is still not easy to get as detailed as our vexel rendering. 

Expresii is the first digital painting   app that can really give both raster richness and ultra-high-res output scaling,  thanks to its novel GPU-based simulation and rendering algorithms.   Many artists used to traditional media do not like digital counterparts because digital paintings get pixelated when zoomed in on. We believe Expresii has largely fixed this issue and hope more artists are willing to go digital.  Our vexel rendering is not a general solution to the problem of image interpolation, since our method rely on the fact that for many natural media there is a substrate to give grain texture.

Currently Expresii only simulates water-based art media.  We plan to add other media like pastel and oil in the near future. Stay tuned.

Update: The following video shows  the use of 'vexel rendering' in actual painting and that we are able to export 32k images thanks to our ultra-zooming capability:

​Why the term 'vexel'
When developing new tech to avoid seeing fat pixels in digital painting, I wanted to give it a name that suggests it's a combination of vector and pixel,   and   I came up with 'vexel'.  Later I found out 'vexel'   is already coined by Seth Woolley​  since at least  2006 to refer to raster images that look like vector graphics.   I tried to find another name, but couldn't find one giving the same level of meaningfulness.  So,  I decided to stick with 'vexel'. After all, our rendering does look like it's vexel art in Seth's definition.

The SonarPen was launched through Kickstarter in 2018.  It was primarily designed for iOS and Android smartphones and tablets as a cheap alternative to the expensive styli like the Apple Pencil. 

We actually have been in contact with  SonarPen's creator Elton Leung since 2018.   We were hoping they can add Windows support ever since. To date, there's still no Windows driver but thanks to the support from SpaceDesk, we can now use it for Windows apps too.

SpaceDesk is a software tool that allows a host machine to use other machines like tablets as its monitors.  Their ability to build a monitor wall is amazing. Recently they added support for relaying pen input too, so it's possible to use SpaceDesk as a wireless Pen Display. With SpaceDesk, we're now able to use SonarPen on Windows apps, including our Expresii that features beautiful organic digital watercolor. 

Previously, we tried   Easy Canvas Pro  (US$5 / year) and   SuperDisplay   (US$10 one time)   for Pen-Display application. Currently, we think SuperDisplay  gives the best performance among the three.  SuperDisplay  does support pen tilt, which is quite important for our app Expresii.   SpaceDesk doesn't support pen tilt nor  multi-touch gestures yet. SpaceDesk is currently free to use, while they also offer a Business license. Our current recommendation is still   SuperDisplay, which is the fastest, the most feature-complete and is inexpensive.   We look forward to SuperDisplay supporting the SonarPen.

2021.05.06 Update:  you can use USB tethering for faster connection on SpaceDesk.

What can still be added: ​It'd be great  if any of these programs can relay accelerometer/orientation sensor  readings and act as a G-Sensor  of Windows tablets so that we can directly control paper tilt in Expresii to direct the paint flow. Using other apps like  Sensor stream IMU+GPS  alongside is no good, as only one  app can be in the foreground and nowadays smartphones would have background apps dormant  after a few moments. We've tried putting the app in Battery Saving Exception or disable Battery  Optimization (on Android 9) but to no vain.

In the above video, we also show the new features of  Brush Tilt Control Pad    and   Auto Settle Timer:

You can now stack paint in the same layer by settling paint. We've also added a timer for auto-settling. You can try these new features in a beta version of Expresii, which you can enable by selecting beta option in the System>Advanced tab. Please let us know how you like them.

2020 went too quickly. No travel. In fact, our activities were mostly confined in our neighborhood.​  Nevertheless, the development of Expresii never stopped. Just check our Update Log for a (truncated) list of what have been done.  

Have been collaborating with a small team of animators this year on a short film Find Find , whose production started in March.  It serves as an experiment for how Expresii can be used for animation. We are developing a system that can automatically control the brush (and other attributes like color loading) in Expresii to produce a sequence of animation frames.  This lays a ground for modern ink-painting animation  (水墨動畫). Click the octopus below to watch an illustrative video.

The animation is set to premiere  in mid-2021.   

In mid-2020, it's been proven that Expresii can run on an Intel-based Mac via Parallels Desktop 15 or later, complete with Apple Pencil support (with pressure and tilt) via Sidecar, but the save / export function of Expresii was not working  due to limitation of Parallels' OpenGL implementation. Since version 2020.11.01, we updated Expresii to overcome this limitation so that it can now run  fully via Parallels. So, yeah, apart from running Expresii via Bootcamp, you can now run it directly from within MacOS via Parallels. 

However, it seems Apple is ditching Intel processors for future Mac's in flavor of their own ARM-based processors. For Expresii to run on ARM,  a crucial part from Microsoft is the support of  OpenGL 3.3+ for Windows on ARM. They have good progress on this, citing   in November   an   Insider version of the compatibility package  that one can test with.  Since one can already   run Windows on ARM on Apple's new M1-based machines, it's quite possible we can run Expresii on ARM-based Apple hardware. 

Those of you having an ARM-based machine can try Windows on ARM (if on Apple M1, here's an   installation guide) and get the   Insider preview     of the OpenGL 3.3 package to try Expresii there already. For those of you hoping to run it via   Parallels,  Parallels just released a   tech preview  in which you can run Windows on ARM as virtual machine  on an M1 Apple machines. So for those of you having a Parallels 16 license, you can already try it.   If you do try them, let me know how it goes!  

2021-04 Update:   Yes, Expresii  runs on Windows 10 ARM version via Parallels 16.5 on Apple Silicon M1 machine!

In March 2020, we got a call from Taiwanese company Ufro.  Their CEO would like us to support their new color-picking mechanism, in preparation for the then-upcoming Lenovo E-Color Pen. We're happy to help. In July this E-Color Pen started to hit the market, and we did some demos for it. Expresii became the only app that can show off this E-Color Pen with gorgeous  color blending effects. 

In May, we added support for on-the-fly adjustment of loading color hue, lightness & saturation via hotkeys in Expresii.   More pen tablet makers are adding dials to their hardware and we also showed you can use those dials for real-time  color adjustment  by assigning the dial to our hotkeys. 

We have been in contact with other hardware makers  too.  Would reveal more when collaborations come to fruition.

Since February, the file size of an Expresii artwork file and in-memory storage size are significantly reduced.   In October, the artwork export rendering implementation is replaced with one that is more amenable to having more layers.  In December, artwork  rendering is also optimized to up to 50% faster (or  80% for a bit lower quality). These path the way  to allowing more layers  or future enhancements. 

Sorry for the lack of a roadmap thus far.   Since the internal of Expresii is much more complex than ordinary paint programs, it takes more effort to design and implement stuffs.  Quite some users still don't have a good GPU and without  a good GPU, we have to be economical when using GPU resources. Now that hardware has improved and capable GPUs are more common in new machines, we'd like to update Expresii to allow better experience given more available GPU compute and memory   resources.

Suminagashi is a form of marbling.  In October, we   gave a preview   of our implementation in Expresii and we passed it to our trusted user Shuen Leung, and she produced the following artwork:

There is still some more work to be done before we can release it to the public. Let us know how you like it so far.

I know many of you find it hard to paint when the paint never really dries and needing to use a new layer for sharp overlapping shapes.   That would change soon. 

With those optimizations mentioned above, we're ready to add support for more layers.

We'll start adding Western watercolor specific features like drips, backruns or granulation.

Some of you were asking for image import.  With that above-mentioned dry-up change, import would be possible.

Our Nelson Chu has been thinking about how to do even better than what he did for oil paint simulation in  Fresh Paint (Project Gustav) . Hopefully, we can come up with a good solution in the coming year following Western watercolor.

We already have an initial implementation of the RYB color model. This makes the colors look more natural, but it actually narrows the range of possible colors you can get. The greys become kind of sepia. 

Existing customers will get these enhancements for free.  So, you don't need to wait if you want to purchase Expresii now.  ​ (^人^)

​If you like what we're doing, please support this endeavor by buying the app (or giving a tip by paying for a trial key) and tell your friends who might be interested.  Everything counts!     Thanks you!      m(_    _)m 

This is an adaptation of the fluid simulation in Expresii to do Suminagashi ('墨流し' in Japanese), a form of paper marbling.   Suminagashi is said   to have originated from China and today it's called 水拓畫 in Chinese. What we show here is a WIP (work-in-progress) and has not yet been released in the public version of Expresii. 

We show that it's possible to load an image  acting as obstacle for the flow, thus creating flow patterns that relate to whatever you put in that image. One useful application is  making animated title sequence.

Changing the palette used results in interesting organic patterns. ​And what about applying the sprayer with suminagashi?

Let us know how you like it.

Perhaps  the highlight of the Lenovo Yoga Duet 7i is its support for the brand new Lenovo E-Color Pen.  Among all the pressure-sensitive styli for iPads, Windows or Android devices, the   Lenovo E-Color Pen is the first that can capture colors from the real world.  Recently, the Apple Pencil was rumored to add such a feature given they filed a related patent application. However, It's not likely Apple would be granted such a patent, as suggested by Jeremy Shu, CEO of Ufro Inc, the company that provides the color capture technology in the   E-Color Pen . 

The Lenovo E-Color Pen is a rechargeable Wacom AES pen. Its USB-C charging port is hidden behind the color capture module, which unplugs from the top of the pen.  The pen is pressure sensitive and drives the cursor when hovering. It doesn't support  tilt sensing, but you can use one of the two side buttons to adjust brush tilt in Expresii. 

​The pen is not magnetic like the Surface Pen, so you can't just stick it to the side of the tablet. Previously Lenovo supplies  holder for their active pens that plugs into USB-A ports of the tablets, but now they switched to USB-C for the Yoga Duet 7i so you will need to find another solution if you want to keep it attached e.g. this pen sleeve  on Amazon, or the same thing on TMall of China for half the price. 

Currently our app Expresii is one of the select apps that natively support the E-Color Pen color capture function.  Its watercolor simulation is a good reason  for anyone to use it for showing the capture capability beautifully.  When you pick a yellow and a cyan from the real world, the virtual paint mixes together to give a nice green and Expresii's watercolor flow further mingles the paint with gorgeous flow patterns in very high resolution.  Expresii supports Ufro's   Mozbii pen since 2018  and we thank Wacom for linking us up leading to   hardware and software innovations working well together.

The following is a video showing general color picking  and the use of the pen for Eastern calligraphy on the Yoga Duet 7i. The machine used in the i5 model, but it's already fast enough for using Expresii. There's the i7 option that gives even better performance for you to choose from.

I came from the Surface Pro family, and I really prefer the keyboard of the Duet 7i being able to operate via bluetooth when detached. It's good that Lenovo is including both the keyboard and the E-Color pen, at least in the markets of Hong Kong and the Philippines, instead of needing customers to   fork out    another   USD 90 to 160 for a keyboard cover  and   USD 99 for the official Surface Pen.  There're cheaper aftermarket alternatives for both items (like these pen alternatives), but consumers need to shop around separately. The Yoga Duet 7i is called Yoga Duet 2020    in the Chinese market, and a normal Active pen instead of the E-Color Pen is included.

Expresii  receives colors from Color King, the proxy software that sends the captured colors to supported apps.  Our Expresii not only can directly receive colors from Color King, but it also offers you the option to auto-load single or multiple-colors into the virtual brush. This is a unique feature made possible by the native support for multi-color loading in Expresii.   As demonstrated in the above video, you could pick a few colors in sequence to make a gradient in the brush. You then can make strokes with such a gradient for even more interesting marks!

We  further demo coloring a piece of ink work using the E-Color Pen.   In this demo, we celebrate  brush stroke economy. We appreciate watercolor having a life of its own. From the thumbnail, you may not be able to appreciate the simulation quality so we urge you to watch the video:

The following video shows the performance of color capture and making strokes on the i5 version of the Yoga Duet 7i.  The color transfer can be immediate. In the above videos, most of the clips are played back at 2x speed up for editing purpose. The following video shows you the performance in real time.

​Our app uses GPU for rendering and flow simulation, so it's often the GPU being the bottleneck.  If you find it too slow on your machine, we can still enable Quick Stroke Mode (Fast Mode), and Expresii would be more responsive as you stroke. If you're on Win10, it's easy to download this Store Demo version of Expresii for you to test out the performance.

How accurate is the color capture? I'd say pretty accurate, as you can see from our demo videos. I tried picking the neon   orange from my sport shirt and the color captured was obviously off. But other than difficult situations like that, it's pretty accurate for the most part.

We appreciate   the startup   Ufro Inc   bringing low-cost color capture tech to the masses. Color sampling devices used to be rather expansive, and only specialized  people would buy them. FYI, you can also order Ufro's standalone color picker InstaPick for as low as USD 60.   We also appreciate   Wacom and   Lenovo for adopting the innovation.

We hope that the Lenovo E-Color Pen would be supported on future pen-enabled machines from Lenovo.  They just announced  the Yoga 9i, which uses Intel’s new and much improved Intel Xe integrated graphics of the 11th Gen CPU. We also wish Lenovo would opt for tilt sensing for their coming pen models. The future looks bright for creative professionals!

Expresii version 2019.08.02 gets a face-lift!   The Windows-7-styled frame of the Settings Window is now replaced with modern Windows 10 frame.   Buttons on the Caption Bar now gives clearer  clues on what options are there for the particular windows.  We also jump onto the bandwagon of  Windows 10 Fluent UI acrylic blur (shown in the Exit Prompt dialog):

However, we got rendering bug of only a portion of the window gets rendered (when DPI scaling is 200%) like this on Windows 10 ver 1803 Build  17134.885:

Updating to latest version 1903 Build   18362.267 fixed the rendering bug. So, be sure to  update your Windows to enjoy our new UI facelift. And In case you don't like Arcylic blur, you can disable it in Expresii's  System Setting Dialog:

The Acrylic Blur is applied on the exit prompt only at the moment and we plan to add the blur to where is appropriate.   Let us know how you like the new design!    ​( •́ .̫ •̀ )

When using 'System Enhanced' mode  override for high DPI, we notice shifted rendering of our UI. 

We believe it's a Windows bug because when we don't use high DPI override, the UI rendering is correct. So, before Microsoft fixes this, please don't use override for high DPI.

Now  in Expresii you can assign a color to a particular brush using the color tag like this:  

It's more beautiful and  easier to see which brush you're holding. It also works perfectly along with Wacom's Pen ID feature.  BTW, You may also noticed the latest version of Expresii  has Quick Access Panels for brushes and papers too (in addition for Colors).  

Color Rings are not new. In fact, we had  them back in the MoXi prototype  more than 10 years ago:  

Years  later Wacom’s  Intuos  4 (came out 2009)  also incorporated similar  color rings like these:

We did meet with Wacom's people in the USA (along with Adobe folks) and gave them a copy of MoXi in 2006.  Maybe they got inspiration from us  for the color rings?  I dunno. Anyway, please don't say we copied Wacom's design, because we have this way before they do. ;)

Since version 2016.06.19 we have added several new papers for you to play with:  

You can zoom in to see the amazing details:

Existing users  would need to do  a Factory Reset to the Paper Presets in order for the new default papers to appear, as illustrated in the figure.  

Enjoy the new papers and let us know what you think!

​

February went too quickly.  On the last day of Feb, we took a day off and visited some friends in Tai Po, Hong Kong.  We also took the chance to get some inspiration for  sample sketches.  

Now with "Whole View" window, you can always check the overall balance (thanks to a user from Japan for his feedback).  You can reposition and resize the  "Whole View" window to suit your need.

 Zoom in to do the details and use the "Whole View" window to check the overall look. 

If it's not for the recent nvidia driver  issue with Expresii, we'd have made  an announcement to call for more artists to try Expresii Lite preview.  We notified nvidia  about the issue but we never hear back from them.  Hopefully this could be resolved soon. If you're an nvidia GPU user and you can't  wait,  you can still enjoy Expresii Lite by using an nvidia driver version 345 or earlier.  If you're *not* using an nvidia GPU , great, just download Expresii Lite Preview  and try it (and give us some feedback)!

Expresii Lite is something waiting to be discovered.  Try it yourself to see why early adopters are saying that it's the most organic among all the tools they've ever used!

Finally, we wish you a  happy and prosperous  Year of the Goat  2015 !