Use Of Fluid Dynamics For Painting 流體力學的使用

26/7/2016

Curtis' painting took 7 hours to render in 1997.

I know many of you are artists and may not be too aware of the historical development in the use of Computational Fluid Dynamics (CFD) for watercolor-like digital painting. I think it's useful to give a summary here.

Cassidy Curtis et al. was the first to apply CFD in watercolor painting [SIGGRAPH 1997]. He used what is technically called Explicit method (which requires taking small time steps) in solving the fluid equations. He was able to render one painting at 640 x 480 pixels in 7 hours on a workstation-class machine. At that time, fluid simulation was believed to be too slow for interactive applications.

Two years later Jos Stam published a paper called Stable Fluids [SIGGRAPH 1999]. He proposed to use a 'Semi-Lagrangian' method to solve the fluid equations so that large time steps can be taken in the simulation iterations to allow user-interactivity. Because the implementation is simple and that he also released sample program for it, his method was widely spread. "Everybody can have his/her own happy meal and fluid sim", so to speak. At that time, usually simulation was done at a resolution of 256 x 256 pixels or lower.

Later in 2003, Mark Harris et al. adapted Stam's method to the GPU for better efficiency. Today, maybe 95% of interactive fluid sim's out there (e.g. this & this) are based on Stam's work.

Note that Stam in his 2003 paper wrote:

The idea of tracing back and interpolating which lies at the heart of the advect() routine apparently goes back to the work by Courant et al. [Courant52]. It has since then been rediscovered by many researchers in various fields and is generally classified under the heading of “Semi-Lagrangian” techniques. Parts of our work are also protected under U. S. patent # 6,266,071 B1 [Patent].

Above: A demo of Stam's 1999 fluid solver.
Below: One of the many implementations by others.

acknowledging that what makes his solver special was actually something that had been done before. While I'm not a lawyer, we are not sure if anyone implementing Stam's method for a profit would infringe their patent or not. That might be one reason why many Stam's-based fluid sims we see today are mostly 'experiment' or 'toy programs' instead of commercial ones.

In 2003, our Nelson Chu started working on simulating Eastern ink on paper. He found Stam's method not very accurate if large steps were taken to maintain

Nelson's 2005 MoXi demo

interactivity, so he looked for alternatives. He finally adapted a CFD method called Lattice Boltzmann and the results were presented in his MoXi SIGGRAPH 2005 paper. If you take a closer look, you will find that Nelson's method produces more realistic ink dispersion compared to that of a usual Stam fluid sim, the latter being more like smoke in space than ink on paper. After Nelson resigned from Microsoft in 2011, he continued to improve the simulation method and that's what went into Expresii you see today.

Wave-like behavior in Rebelle's watercolor sim

Another painting simulator featuring CFD is Peter Blaškovič's Fluid Painter, apparently started in 2009 (judging from the earliest comment there). Peter didn't state what method he used, but it looks very much like one based on Stam's without much change. His other paint program Rebelle that came out in 2015 doesn't seem to model watercolor faithfully since the spread is like waves that bounce back and forth off the boundary (see above) and the results lack minute flow details. They might actually be using shallow water equations to model the flow.

The Verve Painter by Taron came out in 2014. Taron also didn't state what fluid sim method he used, but if he based it on Stam's, at least he adapted the fluid density to volume and added shading, brush options, etc, making impressive results possible.

In around 2013, nvidia entered the tablet market with their own mobile chip & stylus. With that, they also developed the paint app Dabbler (2014-2015). They modeled the viscosity of oil and the flow of watercolor but it's not clear if actual CFD was used for watercolor since the effect is mostly just blurring away the colors, which can well be done with burring instead of CFD.

Oh, there's one more: Adobe's new Adobe Sketch for iPad that came out in 2015. Adobe licensed MoXi in 2006, but apparently they finally didn't really make use of our fluid code. However, now you see this watercolor sim in this 2015 iPad app, whose effect is similar to what we had back in our 2005 MoXi, maybe somehow simplified since the flow is mostly an expanding motion and seems no free flow (see the Expresii video on the right) is possible. Adobe usually publishes paper on their research work (like in their previous attempt on watercolor, which uses some improvised physics and polygons instead of CFD), but so far we haven't found any documentation describing the tech behind this.

Okay, that's all paint apps with CFD for watercolor-like effect that we are aware of.

p.s. There were also attempts to use CFD for thick, oil-like paint simulation (e.g. Bill Baxter's), but that's another long story so we restrict ourselves to watercolor-like painting here.

Update 2016/10/22: I think we should also show our Expresii here too, just for the record. The video on the right shows Expresii running on a mobile device, namely a Windows tablet powered by Intel processors.